More Efficient SPAM Fighting with Amavisd-logwatch

This is the first in a multipart series on better SPAM fighting through log parsing. I have found that better Systems Administration can usually be achieved through proper log handling and analysis. In fact, I will use the data from one of the secondary mail servers in my personal mail setup in order to demonstrate this data analysis. I will do this by going through the report generated by amavisd-logwatch piece meal until complete.

I previously posted about a program that parses your amavisd-new SPAM log file called amavisd-logwatch. Now I am going to give you some tutorials of how to make efficient use of the results. I am assuming that you have access your SpamAssassin scoring config files. I am also assuming that you have access to the log parsing results. I have mine sent via email daily.

One item I would like to mention is that when making changes to SPAMAssassin, ensure that you make them in a separate file from the default configuration files. I use /etc/spamassassin/local_tests.cf. I strongly recommend this setup as this makes it easier to segment your configuration files by type when your rule sets and modifications start to get larger and larger.

Section: Bayes Probability
First things first, skip the majority of the summary sections and go right down to the section on Bayes probability:

Bayes Probability Information

Bayes Probability Information

You’ll notice that of the 14,627 times that the Bayesian filter was run on messages, that it came up with BAYES_99 11,825 of those times (or 80.85%) . You’ll also notice that all the subsequent BAYES_XX probability tests were extremely low (2nd and 3rd place being 5.4% and 4.5% respectively).

Conclusion: Assuming that you are relatively happy with your current level of SPAM filtering, that would mean that your Bayes filter is doing fairly well (in general). You may not need to tweak it. If you are feeling frisky though, to tweak the impact that the BAYES_99. To change this, open up your local_test.cf and add the line:

score BAYES_99 (1.25)

This increases your BAYES_99 score by 1.25 points from its base. It doesn’t have to be 1.25 points, start small to see what you are comfortable with and slowly work your way up. Be careful as too high a jump will cause false positives which makes for angry users.

Section: SPAM Score Frequency
The SPAM score frequency refers to how often a piece of email scores within a given range.

SPAM Score Frequency

SPAM Score Frequency

Conclusion: Taking note of the fact that nearly 60% of the emails scored a 30 or higher, and assuming again that you are comfortable with your SPAM filter, you can adjust the SPAM kill score threshold in amavisd-new accordingly. I trust my SPAM filter, but I have written many rules and made many tweaks to it. So I have set my SPAM kill threshold low enough (15.8 to be exact). As you can see, this is pretty close to the middle of the set of numbers (also known as the median). This eliminates the delivery of the vast majority of the obvious SPAM.

Stay tuned for the next part in the series where we will tweak the individual scores based on the results report.

Deploying Amavisd-logwatch

I was looking for way to make my SPAM filtering more effective and came across this great tool from Mike Cappella called amavisd-logwatch.

On his web site, it says he doesn’t like waiting for package maintainers, so its just a tarball. Since my installs are Debian based, I created a deb for it. My .deb creating skills are not perfect, but it works. The deb was built on sid and is available here.

Download the Debian package and install it:

mail:~# dpkg -i amavis-logwatch_1.49.09-1.1_i386.deb
Selecting previously deselected package amavis-logwatch.
(Reading database ... 37342 files and directories currently installed.)
Unpacking amavis-logwatch (from amavis-logwatch_1.49.09-1.1_i386.deb) ...
Setting up amavis-logwatch (1.49.09-1.1) ...
Processing triggers for man-db ...

Leaving the defaults are safe in the config file. The one thing that does need to be changed is the additional cron script that I added to the installer. It will email the output of the script when cron.daily runs. If you do not want this to happen, then just delete the file /etc/cron.daily/amavis-logwatch. To have the script run, you have to edit it and change the defaults to reasonable defaults (like proper From, To, and CC email addresses). Also make sure to change the /var/log/mail.log file if that isn’t the location of your mail log.

$SUMMARY=`/usr/bin/amavis-logwatch --detail 5 -f /etc/amavis-logwatch.conf /var/log/mail.log`;
...
# Set the email header fun
$FROM = "\"Postmaster\" <postmaster \@example.com>";
$TO = "\"To\" <to \@example.com>";
$CC = "\"CC\" <cc \@example.com>";
</cc></to></postmaster>

Once you have made those changes, you will receive a nightly report with your amavisd-new log information.

Posted in Mail, SPAM. Tags: , , . 4 Comments »

Joe Job and SPF

First off, get your mind out of the gutter. A joe job has absolutely nothing to do with what you’re thinking about. It’s email related and it can be a pain in the ass to deal with.

What is a Joe Job?
Joe Job is the term used to describe the act of forging bulk email to appear to the recipient as if the email were coming from the victim. Generally speaking, this term is used to describe an attack of this nature. This is to say that when a spambot or botnet sends a massive amount of email to a victim. The named was coined by an attack launched against http://www.joes.com/ in January of 1997. The perpetrator (SPAMMER) sent a flood of emails from spoofed addresses in a (successful) attempt to enrage the recipients to take action against the company.

Why do I care?
There are many reasons, but I will just cover a few until you get the picture. The main victim of a SPAM attack of this nature ends up having an INBOX full of junk. This junk can potentially include malware, virii, and any number of phishing or scam based attacks. Also, since there is so much email traversing the connection, the bandwidth gets sucked up and depending on the actual amount of SPAM coming in, could render the connection unusable until all the mail is filtered through. The problem comes in when there are thousands of messages, that could take days or even weeks. Since the originating address is spoofed, those who don’t know are going to get very upset with who they *believe* to be responsible for sending the email. The last item I am going to touch on is that the person whose email address was spoofed now has to deal with all the auto-responses and whatever else may automatically come their way. (I think you get the idea).

What I can do?
There is nothing that you can do to completely avoid it besides not using the internet or email. There are some steps that you can take. One of the first things is to take a look at SPF (Sender Policy Framework). To set this up in DNS, you need to do the following:

In your DNS zone file for server.com, you should add something like the following:

server.com.  IN TXT    "v=spf1 a mx -all"
  • v – The version of SPF to use
  • a mx – The DNS attributes permitted to send messages for server.com
  • -all – Reject everything else that does match a or mx

This can also get more in depth depending on the number of email accounts you have and from where. For instance, let’s say your mail server’s name is mail.server.com and you also have email accounts on gmail (gmail.com)and your work email (myjob.com). Your line would look something similar to the following:

server.com.   IN   TXT   "v=spf1 mx a:mail.server.com include:gmail.com include:myjob.com -all"

The a line is saying that mail.server.com is authorized to send mail via your mail server. The include statements are basically saying that everything considered legitimate by either gmail.com or myjob.com should also be considered legitimate by you.

There is a lot more information on configuring SPF. The documentation should be read thoroughly as improperly configured SPF can prevent legitimate email from flowing. For more information of SPF and configuring it, check out:

SPF is just one method that can be used to fight against being a victim of a Joe job. You should always be using some method of SPAM filtering in addition to SPF. Layered security needs to be the approach when locking down any type of server or service.