Archive for the ‘ Mail ’ Category

A Few Words About Setting Up Postfix Multi Instance

Monday, April 12th, 2010

I work with email and Postfix. On every mailing machine I have Postfix setup on, I have at least 2 instances, sometimes more (in fact, sometimes its as many as 6 instances). I was recently setting up a new set of mailers and decided to give Postfix multi-instance seutp a try. It was excellent. There really isn’t too many complex setups that have a simple installation. And to that end, I give Postfix credit where credit is due. It usually takes a little more than just following the README.
(more…)

List of Feedback Loops

Friday, January 15th, 2010

This is the most comprehensive list of feedback loops that I have been able to find and put together. If there are others that you know of, please let me know and I’ll add them.

The other item that should be noted is that some ISPs don’t have a feedback loop and just respond to the abuse@ or postmaster@ email addresses for a domain. In addition to the feedback loop setup, ensure you have those addresses as valid addresses for your domain.

For more information on Feedback loops and how they help your deliverability, see the Wikipedia entry on Feedback Loops.
(more…)

Transferring Email From Gmail/Google Apps to Dovecot With Larch

Wednesday, December 9th, 2009

As regular readers of this blog know, I am in the process of trying to back up Google Apps accounts to Dovecot. Well I have finally found my solution. Not only does it work, but its in Ruby.

First thing that you’ll need to do is grab yourself a copy of Larch. I did this simply by typing and it installed everything nicely, but click the link to the repository on Github if it doesn’t work for you.
(more…)

Backing Up Gmail/Google Apps to a Dovecot Server

Thursday, December 3rd, 2009

I have been trying to find a way to copy everything from a Gmail account to a Dovecot mail server. The way I have ended up doing it so far is simply by using Apple Mail (if you regularly read this blog, you’d know that I use a Mac). The steps are as follows:

  1. Create 2 accounts in Apple Mail: Gmail and the Dovecot account
  2. Sync the Gmail account to your local computer
  3. Copy everything to the Dovecot server

This works, but I have to use a slow connection (my home connection) and I have a lot of accounts to do this for, so I would much prefer to script this. The problem is that I have been trying to get this to work with either imapsync or imapcopy. Neither seem to work properly.
(more…)

Redacted On A Feedback Loop

Thursday, August 27th, 2009

This post is a little more of a rant than I usually make, but I think its warranted. If you don’t know what a feedback loop is, read here.

I’m not sure who thinks its a good idea to replace all instances of an email addresses in a feedback loop with [redacted]@feedbackloopcompany.com, but it is of no help to anyone. An argument can be made for protecting the identity of the recipient, but that argument holds little weight because there is little the sender can do about it.

If a sender needs to go through the authorization process of a larger recipient domain (like AOL, Yahoo!, or Excite for example) where their IP reputation is checked and their history is checked, etc. then why should there still be restrictions placed on the information going between the two domains (you as the sender and them as the recipient domains). I am aware that the draft specification allow the operating domain for the feedback loop to keep the identity private of the user clicking the “Report SPAM” button, but that forces the sending domains to use tactics to circumvent this to keep their reputation up.

Therefore I believe that if a sending company has verified their feedback loop address, they should be able to see which recipient reported their email as “Junk”. Get rid of the redacted and leave the email address intact.

More Efficient SPAM Fighting with Amavisd-logwatch

Friday, May 22nd, 2009

This is the first in a multipart series on better SPAM fighting through log parsing. I have found that better Systems Administration can usually be achieved through proper log handling and analysis. In fact, I will use the data from one of the secondary mail servers in my personal mail setup in order to demonstrate this data analysis. I will do this by going through the report generated by amavisd-logwatch piece meal until complete.

I previously posted about a program that parses your amavisd-new SPAM log file called amavisd-logwatch. Now I am going to give you some tutorials of how to make efficient use of the results. I am assuming that you have access your SpamAssassin scoring config files. I am also assuming that you have access to the log parsing results. I have mine sent via email daily.

One item I would like to mention is that when making changes to SPAMAssassin, ensure that you make them in a separate file from the default configuration files. I use /etc/spamassassin/local_tests.cf. I strongly recommend this setup as this makes it easier to segment your configuration files by type when your rule sets and modifications start to get larger and larger.

Section: Bayes Probability
First things first, skip the majority of the summary sections and go right down to the section on Bayes probability:

Bayes Probability Information

Bayes Probability Information

You’ll notice that of the 14,627 times that the Bayesian filter was run on messages, that it came up with BAYES_99 11,825 of those times (or 80.85%) . You’ll also notice that all the subsequent BAYES_XX probability tests were extremely low (2nd and 3rd place being 5.4% and 4.5% respectively).

Conclusion: Assuming that you are relatively happy with your current level of SPAM filtering, that would mean that your Bayes filter is doing fairly well (in general). You may not need to tweak it. If you are feeling frisky though, to tweak the impact that the BAYES_99. To change this, open up your local_test.cf and add the line:

1
score BAYES_99 (1.25)

This increases your BAYES_99 score by 1.25 points from its base. It doesn’t have to be 1.25 points, start small to see what you are comfortable with and slowly work your way up. Be careful as too high a jump will cause false positives which makes for angry users.

Section: SPAM Score Frequency
The SPAM score frequency refers to how often a piece of email scores within a given range.

SPAM Score Frequency

SPAM Score Frequency

Conclusion: Taking note of the fact that nearly 60% of the emails scored a 30 or higher, and assuming again that you are comfortable with your SPAM filter, you can adjust the SPAM kill score threshold in amavisd-new accordingly. I trust my SPAM filter, but I have written many rules and made many tweaks to it. So I have set my SPAM kill threshold low enough (15.8 to be exact). As you can see, this is pretty close to the middle of the set of numbers (also known as the median). This eliminates the delivery of the vast majority of the obvious SPAM.

Stay tuned for the next part in the series where we will tweak the individual scores based on the results report.

Deploying Amavisd-logwatch

Friday, May 8th, 2009

I was looking for way to make my SPAM filtering more effective and came across this great tool from Mike Cappella called amavisd-logwatch.

On his web site, it says he doesn’t like waiting for package maintainers, so its just a tarball. Since my installs are Debian based, I created a deb for it. My .deb creating skills are not perfect, but it works. The deb was built on sid and is available here.

Download the Debian package and install it:

1
2
3
4
5
6
mail:~# dpkg -i amavis-logwatch_1.49.09-1.1_i386.deb
Selecting previously deselected package amavis-logwatch.
(Reading database ... 37342 files and directories currently installed.)
Unpacking amavis-logwatch (from amavis-logwatch_1.49.09-1.1_i386.deb) ...
Setting up amavis-logwatch (1.49.09-1.1) ...
Processing triggers for man-db ...

Leaving the defaults are safe in the config file. The one thing that does need to be changed is the additional cron script that I added to the installer. It will email the output of the script when cron.daily runs. If you do not want this to happen, then just delete the file /etc/cron.daily/amavis-logwatch. To have the script run, you have to edit it and change the defaults to reasonable defaults (like proper From, To, and CC email addresses). Also make sure to change the /var/log/mail.log file if that isn’t the location of your mail log.

1
2
3
4
5
6
7
$SUMMARY=`/usr/bin/amavis-logwatch --detail 5 -f /etc/amavis-logwatch.conf /var/log/mail.log`;
...
# Set the email header fun
$FROM = "\"Postmaster\" <postmaster \@example.com>";
$TO = "\"To\" <to \@example.com>";
$CC = "\"CC\" <cc \@example.com>";
</cc></to></postmaster>

Once you have made those changes, you will receive a nightly report with your amavisd-new log information.

What is a DRD

Wednesday, April 29th, 2009

I have Google’d around asked a lot of smart people and still can’t come up with a solid answer. More specifically, the question I have is:

Why, when sending emails to certain domains, do I get the error: 451 Could not load DRD for domain

Because I deal with fairly large mailing lists, I see odd errors a lot. Most of them I can disregard. In any case in which I see an error over 1,000 times, I make it a policy to deal with it. I have seen one close to 9 million.

It appears to be an error from a Symantec appliance, but people write about it coming from Exchange servers as well.

If anyone has any idea what this, please email me, leave a comment, or post a link to somewhere.

Setting Up DKIM and Postfix on CentOS 5.2

Monday, April 20th, 2009

I spent a while trying to set up DKIM with Postfix on CentOS 5.2. I read the HOWTOs on HOWToForge written by Andrew Colin Kissa (aka TopDog) who subsequently helped me towards getting this setup working.

My setup is that I have a mail spooler and multiple mail senders. This is to say that the emails are created on spooler.domain.com and sent via sender1.domain.com and sender2.domain.com. I will walk through how to setup DKIM on the sender machines so that all mail spooled from the spooler still gets signed.

First start out by installing DKIM. At the time the HOWTO was published, I downloaded the RPM from Topdog.

1
2
3
4
5
6
[root@sender1 dkim]# wget http://www.topdog-software.com/oss/dkim-milter/dkim-milter-2.8.2-0.$(uname -i).rpm
...
[root@sender1 dkim]# rpm -Uvh dkim-milter-2.8.2-0.x86_64.rpm
warning: dkim-milter-2.8.2-0.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 990dd808
Preparing...                ########################################### [100%]
   1:dkim-milter            ########################################### [100%]

Once you have installed DKIM you have to create the public and private keys. Do this using the dkim-genkey.sh shell script.

1
[root@sender1 dkim]# sh /usr/share/doc/dkim-milter-2.8.2/dkim-genkey.sh -r -d yourdomain.com

By running this script, 2 files will be generated; default.txt: the public key which gets published via DNS; default.private: private key used for signing the emails.

Move the private key to the dkim directory and secure it.

1
2
3
[root@sender1 dkim]# mv default.private /etc/mail/dkim/default.key.pem
[root@sender1 dkim]# chmod 600 /etc/mail/dkim/default.key.pem
[root@sender1 dkim]# chown dkim-milt.dkim-milt /etc/mail/dkim/default.key.pem

Now create the DNS entries. The p= section is the public key created using the dkim-genkey.sh script. Don’t forget to increment the SOA and reload DNS.

1
2
_ssp._domainkey.yourdomain.com      TXT t=y; o=-
default._domainkey.yourdomain.com   TXT v=DKIM1; g=*; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GWETBNiQKBgQC5KT1eN2lqCRQGDX+20I4liM2mktrtjWkV6mW9WX7q46cZAYgNrus53vgfl2z1Y/95mBv6Bx9WOS56OAVBQw62+ksXPT5cRUAUN9GkENPdOoPdpvrU1KdAMW5c3zmGOvEOa4jAlB4/wYTV5RkLq/1XLxXfTKNy58v+CKETLQS/eQIDAQAB

The reason for this peer_list file is so that the senders know that its ok for them to sign emails relayed via the spooler.

1
2
3
4
5
6
7
8
9
[root@sender1 dkim]# cat /etc/mail/dkim/peer_list
mail.yourdomain.com
spooler.yourdomain.com
sender2.yourdomain.com
1.2.4.7
1.2.4.5
localhost
localhost.localdomain
127.0.0.1

Onto the configuring of the system. It should look something like the following. I chose to have the port be a local port, but it could be done via a network connection as well. Ensure you change the SIGNING_DOMAIN variable and be sure to note the EXTRA_ARGS variable and where PEER_LIST is used.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
[root@sender1 dkim]# cat /etc/sysconfig/dkim-milter
# Default values

USER="dkim-milt"
PORT="local:/var/run/dkim-milter/dkim.sock"
SIGNING_DOMAIN="yourdomain.com"
SELECTOR_NAME="default"
KEYFILE="/etc/mail/dkim/default.key.pem"
SIGNER=yes
VERIFIER=yes
CANON=simple
SIGALG=rsa-sha1
REJECTION="bad=r,dns=t,int=t,no=a"

PEER_LIST="/etc/mail/dkim/peer_list"

EXTRA_ARGS="-h -l -D -i ${PEER_LIST} -I ${PEER_LIST}"

Let’s start up dkim. Since it is a daemon running separately from Postfix, running it and restarting it won’t affect mail (yet).

1
[root@sender1 dkim]# /etc/init.d/dkim-filter start

And it’s finally time for Postfix. You need to add 2 simple lines to your Postfix main.cf. These 2 lines should match the PORT variable in the dkim-milter.conf sysconfig file.

1
2
smtpd_milters = local:/var/run/dkim-milter/dkim.sock
non_smtpd_milters = local:/var/run/dkim-milter/dkim.sock

Now you’re asking yourself, how do I test this? The easy answer is to use a Gmail account. When you receive an email, click on the show details link on the right hand side of the screen (to the left of the time). If you have performed this sequence correctly, you will see a line that says:

1
signed-by   yourdomain.com

Another way to test the success of this is to view the source of the email. You should have some lines that look similar to this:

1
2
3
4
5
6
7
8
9
X-DKIM: Sendmail DKIM Filter v2.8.2 sender1.yourdomain.com 75866730012
DKIM-Signature: v=1; a=rsa-sha1; c=simple/simple; d=yourdomain.com;
    s=default; t=1239981026; bh=+NNkD6jOlYKtY2AIGNRToH2tkm0=;
    h=Date:List-Help:List-Subscribe:List-Unsubscribe:List-Owner:
     List-Post:From:Reply-To:To:Subject:MIME-Version:Content-Type:
     Message-Id;
    b=MrjXBShjNexWy62fC4Uu7xS3Hxav+cHtqIBzwMlcufadsffLtW9KmF5sO58+yHjyy
     I3SiX0TNyEbvXtSHvRKm9z630zDiN0dxVXGqhgEfdklaj4jlkfhR6GrsRgzW2YOW6/9
     sKFnz214AkhAPrFBD30hNmZfRfY75v5q94FnGDUo=

Congratulations, you have a working DKIM installation.