Posts Tagged ‘ SPAM ’

List of Feedback Loops

Friday, January 15th, 2010

This is the most comprehensive list of feedback loops that I have been able to find and put together. If there are others that you know of, please let me know and I’ll add them.

The other item that should be noted is that some ISPs don’t have a feedback loop and just respond to the abuse@ or postmaster@ email addresses for a domain. In addition to the feedback loop setup, ensure you have those addresses as valid addresses for your domain.

For more information on Feedback loops and how they help your deliverability, see the Wikipedia entry on Feedback Loops.
(more…)

More Efficient SPAM Fighting with Amavisd-logwatch

Friday, May 22nd, 2009

This is the first in a multipart series on better SPAM fighting through log parsing. I have found that better Systems Administration can usually be achieved through proper log handling and analysis. In fact, I will use the data from one of the secondary mail servers in my personal mail setup in order to demonstrate this data analysis. I will do this by going through the report generated by amavisd-logwatch piece meal until complete.

I previously posted about a program that parses your amavisd-new SPAM log file called amavisd-logwatch. Now I am going to give you some tutorials of how to make efficient use of the results. I am assuming that you have access your SpamAssassin scoring config files. I am also assuming that you have access to the log parsing results. I have mine sent via email daily.

One item I would like to mention is that when making changes to SPAMAssassin, ensure that you make them in a separate file from the default configuration files. I use /etc/spamassassin/local_tests.cf. I strongly recommend this setup as this makes it easier to segment your configuration files by type when your rule sets and modifications start to get larger and larger.

Section: Bayes Probability
First things first, skip the majority of the summary sections and go right down to the section on Bayes probability:

Bayes Probability Information

Bayes Probability Information

You’ll notice that of the 14,627 times that the Bayesian filter was run on messages, that it came up with BAYES_99 11,825 of those times (or 80.85%) . You’ll also notice that all the subsequent BAYES_XX probability tests were extremely low (2nd and 3rd place being 5.4% and 4.5% respectively).

Conclusion: Assuming that you are relatively happy with your current level of SPAM filtering, that would mean that your Bayes filter is doing fairly well (in general). You may not need to tweak it. If you are feeling frisky though, to tweak the impact that the BAYES_99. To change this, open up your local_test.cf and add the line:

1
score BAYES_99 (1.25)

This increases your BAYES_99 score by 1.25 points from its base. It doesn’t have to be 1.25 points, start small to see what you are comfortable with and slowly work your way up. Be careful as too high a jump will cause false positives which makes for angry users.

Section: SPAM Score Frequency
The SPAM score frequency refers to how often a piece of email scores within a given range.

SPAM Score Frequency

SPAM Score Frequency

Conclusion: Taking note of the fact that nearly 60% of the emails scored a 30 or higher, and assuming again that you are comfortable with your SPAM filter, you can adjust the SPAM kill score threshold in amavisd-new accordingly. I trust my SPAM filter, but I have written many rules and made many tweaks to it. So I have set my SPAM kill threshold low enough (15.8 to be exact). As you can see, this is pretty close to the middle of the set of numbers (also known as the median). This eliminates the delivery of the vast majority of the obvious SPAM.

Stay tuned for the next part in the series where we will tweak the individual scores based on the results report.

Deploying Amavisd-logwatch

Friday, May 8th, 2009

I was looking for way to make my SPAM filtering more effective and came across this great tool from Mike Cappella called amavisd-logwatch.

On his web site, it says he doesn’t like waiting for package maintainers, so its just a tarball. Since my installs are Debian based, I created a deb for it. My .deb creating skills are not perfect, but it works. The deb was built on sid and is available here.

Download the Debian package and install it:

1
2
3
4
5
6
mail:~# dpkg -i amavis-logwatch_1.49.09-1.1_i386.deb
Selecting previously deselected package amavis-logwatch.
(Reading database ... 37342 files and directories currently installed.)
Unpacking amavis-logwatch (from amavis-logwatch_1.49.09-1.1_i386.deb) ...
Setting up amavis-logwatch (1.49.09-1.1) ...
Processing triggers for man-db ...

Leaving the defaults are safe in the config file. The one thing that does need to be changed is the additional cron script that I added to the installer. It will email the output of the script when cron.daily runs. If you do not want this to happen, then just delete the file /etc/cron.daily/amavis-logwatch. To have the script run, you have to edit it and change the defaults to reasonable defaults (like proper From, To, and CC email addresses). Also make sure to change the /var/log/mail.log file if that isn’t the location of your mail log.

1
2
3
4
5
6
7
$SUMMARY=`/usr/bin/amavis-logwatch --detail 5 -f /etc/amavis-logwatch.conf /var/log/mail.log`;
...
# Set the email header fun
$FROM = "\"Postmaster\" <postmaster \@example.com>";
$TO = "\"To\" <to \@example.com>";
$CC = "\"CC\" <cc \@example.com>";
</cc></to></postmaster>

Once you have made those changes, you will receive a nightly report with your amavisd-new log information.

Setting Up DKIM and Postfix on CentOS 5.2

Monday, April 20th, 2009

I spent a while trying to set up DKIM with Postfix on CentOS 5.2. I read the HOWTOs on HOWToForge written by Andrew Colin Kissa (aka TopDog) who subsequently helped me towards getting this setup working.

My setup is that I have a mail spooler and multiple mail senders. This is to say that the emails are created on spooler.domain.com and sent via sender1.domain.com and sender2.domain.com. I will walk through how to setup DKIM on the sender machines so that all mail spooled from the spooler still gets signed.

First start out by installing DKIM. At the time the HOWTO was published, I downloaded the RPM from Topdog.

1
2
3
4
5
6
[root@sender1 dkim]# wget http://www.topdog-software.com/oss/dkim-milter/dkim-milter-2.8.2-0.$(uname -i).rpm
...
[root@sender1 dkim]# rpm -Uvh dkim-milter-2.8.2-0.x86_64.rpm
warning: dkim-milter-2.8.2-0.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 990dd808
Preparing...                ########################################### [100%]
   1:dkim-milter            ########################################### [100%]

Once you have installed DKIM you have to create the public and private keys. Do this using the dkim-genkey.sh shell script.

1
[root@sender1 dkim]# sh /usr/share/doc/dkim-milter-2.8.2/dkim-genkey.sh -r -d yourdomain.com

By running this script, 2 files will be generated; default.txt: the public key which gets published via DNS; default.private: private key used for signing the emails.

Move the private key to the dkim directory and secure it.

1
2
3
[root@sender1 dkim]# mv default.private /etc/mail/dkim/default.key.pem
[root@sender1 dkim]# chmod 600 /etc/mail/dkim/default.key.pem
[root@sender1 dkim]# chown dkim-milt.dkim-milt /etc/mail/dkim/default.key.pem

Now create the DNS entries. The p= section is the public key created using the dkim-genkey.sh script. Don’t forget to increment the SOA and reload DNS.

1
2
_ssp._domainkey.yourdomain.com      TXT t=y; o=-
default._domainkey.yourdomain.com   TXT v=DKIM1; g=*; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GWETBNiQKBgQC5KT1eN2lqCRQGDX+20I4liM2mktrtjWkV6mW9WX7q46cZAYgNrus53vgfl2z1Y/95mBv6Bx9WOS56OAVBQw62+ksXPT5cRUAUN9GkENPdOoPdpvrU1KdAMW5c3zmGOvEOa4jAlB4/wYTV5RkLq/1XLxXfTKNy58v+CKETLQS/eQIDAQAB

The reason for this peer_list file is so that the senders know that its ok for them to sign emails relayed via the spooler.

1
2
3
4
5
6
7
8
9
[root@sender1 dkim]# cat /etc/mail/dkim/peer_list
mail.yourdomain.com
spooler.yourdomain.com
sender2.yourdomain.com
1.2.4.7
1.2.4.5
localhost
localhost.localdomain
127.0.0.1

Onto the configuring of the system. It should look something like the following. I chose to have the port be a local port, but it could be done via a network connection as well. Ensure you change the SIGNING_DOMAIN variable and be sure to note the EXTRA_ARGS variable and where PEER_LIST is used.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
[root@sender1 dkim]# cat /etc/sysconfig/dkim-milter
# Default values

USER="dkim-milt"
PORT="local:/var/run/dkim-milter/dkim.sock"
SIGNING_DOMAIN="yourdomain.com"
SELECTOR_NAME="default"
KEYFILE="/etc/mail/dkim/default.key.pem"
SIGNER=yes
VERIFIER=yes
CANON=simple
SIGALG=rsa-sha1
REJECTION="bad=r,dns=t,int=t,no=a"

PEER_LIST="/etc/mail/dkim/peer_list"

EXTRA_ARGS="-h -l -D -i ${PEER_LIST} -I ${PEER_LIST}"

Let’s start up dkim. Since it is a daemon running separately from Postfix, running it and restarting it won’t affect mail (yet).

1
[root@sender1 dkim]# /etc/init.d/dkim-filter start

And it’s finally time for Postfix. You need to add 2 simple lines to your Postfix main.cf. These 2 lines should match the PORT variable in the dkim-milter.conf sysconfig file.

1
2
smtpd_milters = local:/var/run/dkim-milter/dkim.sock
non_smtpd_milters = local:/var/run/dkim-milter/dkim.sock

Now you’re asking yourself, how do I test this? The easy answer is to use a Gmail account. When you receive an email, click on the show details link on the right hand side of the screen (to the left of the time). If you have performed this sequence correctly, you will see a line that says:

1
signed-by   yourdomain.com

Another way to test the success of this is to view the source of the email. You should have some lines that look similar to this:

1
2
3
4
5
6
7
8
9
X-DKIM: Sendmail DKIM Filter v2.8.2 sender1.yourdomain.com 75866730012
DKIM-Signature: v=1; a=rsa-sha1; c=simple/simple; d=yourdomain.com;
    s=default; t=1239981026; bh=+NNkD6jOlYKtY2AIGNRToH2tkm0=;
    h=Date:List-Help:List-Subscribe:List-Unsubscribe:List-Owner:
     List-Post:From:Reply-To:To:Subject:MIME-Version:Content-Type:
     Message-Id;
    b=MrjXBShjNexWy62fC4Uu7xS3Hxav+cHtqIBzwMlcufadsffLtW9KmF5sO58+yHjyy
     I3SiX0TNyEbvXtSHvRKm9z630zDiN0dxVXGqhgEfdklaj4jlkfhR6GrsRgzW2YOW6/9
     sKFnz214AkhAPrFBD30hNmZfRfY75v5q94FnGDUo=

Congratulations, you have a working DKIM installation.