Stopping spam with sender address checks. (tummy.com, ltd. Journal Entry)
tummy.com: we do linux

Sunday December 04, 2005 at 17:19
Subject: Stopping spam with sender address checks.
Keywords: Spam, Technical
Posted by: Sean Reifschneider

Related entries:
   What we are doing for anti-spam. by Sean Reifschneider, Sunday December 03, 2006 at 14:33

I installed some capacity analysis software, which I'll probably write about shortly, on our mail server yesterday. Part of that was that I found that there were over 30 messages in the mail queue. All of these were bounces related to spam, which couldn't be delivered because of the sender address wasn't valid. So I decided to do a little survey to see how much of our e-mail comes from invalid senders...

I wrote a quick Python program that, given an e-mail address, would contact the domain's mail server(s) and then ask if it would accept e-mail for that address. I pulled sender addresses from our mail logs over the last week to feed the system and then fired it off to check the addresses.

I started it off running and went to bed. When I got up, it had only made it 15% of the way through the 26,093 addresses. So, I converted from sequential-processing to running 50 checks in parallel. While the checks used up 40% of the CPU time on a fairly speedy box, it was able to complete 97% of them within an hour. The remaining few percent took 2 more hours.

Here's how the results came out:

  • Total sender addresses checked: 26,093
  • Mail servers which accepted the sender address with a 2xx or 4xx response: 10,789
  • Sender address which had servers which accepted an SMTP connection: 22,562
  • Obvious addresses that were rejected with a 5xx response but I wanted to get e-mail from anyway: 16

So, clearly there could be a huge benefit for a mail server to check the sender's address against the published MXs, and reject messages which do not have a valid sender. On the other hand, if this is prevalent it would make it easy for a spammer to DDoS a domain's mail server. On the other other hand, if you put these checks after SPF and Greylisting, the attacked domain would just have to publish a (even passingly) valid SPF record to shut down the attack.

I'll probably be trying this on some test domains by writing a Postfix external policy filter, to test it out. It's worth trying at least. And it's not like nobody is currently doing this. Exim I know can be configured to do this.
(Post Reply)

Comment
Alan McNeil
Subject: monitoring the mail queue
I have to make sure the boss' love notes get thru within 5 minutes SO I monitor my mailq remotely with mrtg. On the mail server I have this script:
#!/bin/sh
# smtp-qstats: show the number of things left in the queue
#
if [ -x "/usr/bin/mailq" ]
then
  exec mailq -OMaxQueueRunSize=0 | tail -1 | cut -d ' ' -f 3
  exec mailq | grep -c "[[:space:]]\{6\}<"
fi

Which returns number of items in the mailq on line 1 and number of recipients on line 2. Next add a line in inetd.conf:


smtp-qstat      stream tcp nowait root /root/scripts/smtp-qstat

(I define smtp-qstat and some other management ports in /etc/services on all servers)

Then on the mrtg machine, I have this qstat.pl script:


#!/usr/bin/perl 
# this script relies on port 7777 of mailserver returning qlength
use strict;
use Socket;
my (@remotes, $port);
@remotes = ("in-server","out-server");
$port = "7777"; #smtp-qstats

# Straight out of the blue Camel - pp. 349
my ($iaddr, $paddr, $proto, $line);
my @num;
@num = (0,0);
my $index = 0;
my $mx;
foreach $mx (@remotes) {
        $iaddr = inet_aton($mx) or die "no host: $mx";
        $paddr = sockaddr_in($port, $iaddr);
        $proto = getprotobyname ('tcp');

        socket (SOCK, PF_INET, SOCK_STREAM, $proto) or die "socket: $!";
        connect (SOCK, $paddr) or die "connect: $!";

        # munge the data
        while () {
                if ($_ =~ /(\d+)$/) {
                $num[$index] = $1;
                }
        } 
close (SOCK) or die "close: $!";
$index++;
}
# print the data for suitable for mrtg
print "$num[0]\n$num[1]\n1\n$remotes[0]\n";

The script takes the qstat from the outbound and inbound mail servers and formats the number the way mrtg likes.

Then add this in your /etc/mrtg.conf

Title[qmail]: Mail
Options[qmail]: gauge,growright,noinfo
YLegend[qmail]: emails
LegendI[qmail]: inbound
LegendO[qmail]: outbound
Legend1[qmail]: inbound queue entries
Legend2[qmail]: outbound queue entries
Legend3[qmail]: max inbound queue 5m
Legend4[qmail]: max outbound queue 5m
Target[qmail]: `/root/qstats.pl`
SetEnv[qmail]: MRTG_INT_IP="" MRTG_INT_DESCR="qmail"
Directory[qmail]: qmail
MaxBytes[qmail]: 50000
XSize[qmail]:340
YSize[qmail]:120
PageTop[qmail]: <H1>Mail Queues</H1>
 <TABLE>
   <TR^gt;<TD>System:</TD>     <TD>Queues</TD></TR>
   <TR><TD>Description:</TD><TD>Mail Queue size</TD></TR>
 </TABLE>

Presto - instant mailq monitoring! One day I saw the queue heading for 2000, I tracked it down to one of the tech who had forwarded all his mail to yahoo, where he was forwarding it back.

Comment
Author: Sean Reifschneider
Subject: Better to be more active...
Graphs are nice for historic and trending information, and they're pretty to look at, but if you want to react quickly and consistently to mail queue problems there's nothing like alerting. From a simple script that checks the queue and send an e-mail whenever it exceeds a certain threshold, to alerting only when it's been over the threshold for a particular length of time or based on 95th percentiles... Nagios is a pretty good tool for doing more complicated alerting, we use it for some pretty complicated monitoring and alerting.

So, instead of it only getting fixed when you happen to be around a browser and checking the graphs, you can get a message on your cell phone any time of the day or night, if you so desire. It allows you to be much more proactive to your boss' love-letter log-jam.

Sean

Comment
Alan McNeil
Subject: Monitoring is my friend
You're right, monitoring is my friend. Nagios in particular. I access the same mailq script remotely from the nagios monitoring machine. Most of the mrtg stuff is eye candy that gives the tech support people some idea of whats going on without calling network ops.

Long ago I learn to rate my todo list items with "effort" and "looks" factors. Do one high-looks item for the boss, one hard one to make real progress.

Comment
Nataraj
Subject: Postfix address verification
Hi Sean,

Have you looked at the postfix address verification support? It's described in http://www.postfix.org/ADDRESS_VERIFICATION_README.html

Comment
Author: Sean Reifschneider
Subject: Yes, we use address verification.
We do use address verification, which is part of why our mail queue was so empty. If we didn't, it would probably be thousands of entries. Many spammers and viruses are using legitimate sender addresses, which pass validation just fine.

Sean