tom's blog » Mailserver

exim graphs

tom — Wed, 25 Feb 2009 21:21:27 +0000

Because I did not found a tool to create graphs (the way I want it)
from exim logs, I wrote a little bash script that suits my needs.

Example graph:

Download: exim_rrd-0.9.5.1 (updated 2009-03-07 [bugfix])

I’m not a programmer, so the script can be improved – no question : )
If you have any hints, please tell me.

Usage: run as cronjob, i.e.
# mail statistic */5 * * * * /usr/local/bin/exim_rrd > /dev/null 2> /dev/null

dependencies:
rrdtool [ http://oss.oetiker.ch/rrdtool/ ] ^ thanks to Tobias Oetiker
(e)grep
- – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – -
tested with:
rrdtool 1.2.23, rrdtool 1.2.27
openSUSE 10.3 (i586), exim 4.69 main log
openSUSE 11.0 (i586), exim 4.69 main log

Should I use DNSBL?

tom — Tue, 04 Mar 2008 23:18:25 +0000

RBLs – yes / no.. a big discussion always..
Right now, I can recommend zen.spamhaus.org & ix.dnsbl.manitu.net (<- especially for german MXs).
I’ve never seen a false positive on these lists ..

Nevertheless, use the lists for scoring (i.e. with spamassassin), not for instant blocking!

Nowadays, prosecuting “backscatter”, sender callouts, etc. seems to be a new trend – and it could be useful in future..
I’ve tested the only free list I know – backscatterer.org.
Don’t you use that one for immediate blocking!
Scoring can be ok, but even there – watch your logs!

Some “hits” (from a test run) to show what I found:
…
2008-03-04 17:17:42 H=lizzard.sbs.de [194.138.37.39] – possible backscatter
2008-03-04 17:21:59 H=mail.space.net [195.30.0.8] – possible backscatter
2008-03-04 17:25:33 H=relay4.ptmail.sapo.pt [212.55.154.24] – possible backscatter
2008-03-04 17:32:46 H=relay23.arbeitsagentur.de [212.204.77.151] – possible backscatter
2008-03-04 17:33:38 H=mout1.mail.vrmd.de [81.28.224.19] – possible backscatter
2008-03-04 17:48:33 H=dgate1.fujitsu-siemens.com [217.115.66.35] – possible backscatter
2008-03-04 17:50:05 H=mailout05.sul.t-online.de [194.25.134.82] – possible backscatter
2008-03-04 17:51:27 H=relay0-0.brigade.com [209.249.158.73] – possible backscatter
2008-03-04 18:04:42 H=mailout07.sul.t-online.de [194.25.134.83] – possible backscatter
2008-03-04 18:11:21 H=bay0-omc2-s24.bay0.hotmail.com [65.54.246.160] – possible backscatter
2008-03-04 18:13:37 H=mail.space.net [195.30.0.8] – possible backscatter
2008-03-04 18:13:42 H=smtp1.versatel.nl [62.58.50.88] – possible backscatter
2008-03-04 18:15:29 H=mailout09.sul.t-online.de [194.25.134.84] – possible backscatter
2008-03-04 18:16:33 H=ip17.be3a.com (be3a.com) [213.92.9.17] – possible backscatter
2008-03-04 18:18:12 H=gamwsm02.mwga.mailwatch.com [216.157.255.16] – possible backscatter
2008-03-04 18:20:15 H=aps67.muc.ec-messenger.com [195.140.186.67] – possible backscatter
2008-03-04 18:22:56 H=mout1.mail.vrmd.de [81.28.224.19] – possible backscatter
2008-03-04 18:25:46 H=mail.gmx.net [213.165.64.20] – possible backscatter
2008-03-04 18:27:56 H=mail004.thyssenkrupp.com [149.211.153.66] – possible backscatter
2008-03-04 18:30:43 H=mailout04.sul.t-online.de [194.25.134.18] – possible backscatter
2008-03-04 18:33:06 H=mailout03.sul.t-online.de [194.25.134.81] – possible backscatter
2008-03-04 18:39:33 H=mail.gmx.net [213.165.64.20] – possible backscatter
2008-03-04 18:45:20 H=mail.schule.bayern.de [194.95.207.92] – possible backscatter
2008-03-04 18:48:56 H=skibayf20.kirche-bayern.de [141.78.101.100] – possible backscatter
…

A lot of the BIG players (german companies in this example mainly) are found on the list ..
So don’t get yourself in trouble with users that complain all day long and think about what you’re blocking ..

Any suggestion/comment ist highly appreciated.

postfix / (mass) remove mails from queue

tom — Fri, 18 Jan 2008 19:33:57 +0000

A little hack to remove mails for a specific recipient from the postfix mail queue:

mailq | tail +2 | awk 'BEGIN { RS = "" } \ / user@example\.tld$/ { print $1 } \ ' | tr -d '*!' | postsuper -d -

exim / implementing greylisting without db

tom — Wed, 28 Mar 2007 16:03:36 +0000

I had greylisting running for a while with a little perl script, that only worked with ip-addresses. As spam is growing an growing, I wanted to use the whole triplet (ip/sender/recipient) for the filter.

As I did not want to use a database backend, the decision led to greylistd – an easy configurable daemon. I needed the packages for (open)SuSE, that can be found in openSuSE’s software repository.

After installing it and checking the basics at /etc/greylistd/config, you have to add a little code to your acls in exim.conf, i.e.:

defer message = greylisted $sender_host_address. please try again later condition = ${readsocket{/var/run/greylistd/socket}\ {--grey $sender_host_address $sender_address $local_part@$domain} {5s}{}{false}}

That was all I had to do to get it working. It’s advisable to add some hosts, that are known for problems with greylisting, not to be checked. Therefor you have to extend your acl:

defer message = greylisted $sender_host_address. please try again later !hosts = : ${if exists {/etc/greylistd/whitelist-hosts}\ {/etc/greylistd/whitelist-hosts}{}} : \ ${if exists {/var/lib/greylistd/whitelist-hosts}\ {/var/lib/greylistd/whitelist-hosts}{}} condition = ${readsocket{/var/run/greylistd/socket}\ {--grey $sender_host_address $sender_address $local_part@$domain} {5s}{}{false}}

Many thanks for the documentations from Arne Schirmacher and Ben Charlton.

exim / remove (all) frozen messages from queue

tom — Thu, 22 Mar 2007 18:03:35 +0000

Just a quick & dirty hack to delete frozen messages for a single / some recipient(s)…

#!/bin/bash # # what are we searching for? # (part of the email-address) SEARCH="anyrecipient.tld" # # exim-bin EXIM=`which exim` # # execute (frozen messages only).. $EXIM -Mrm $(mailq | grep $SEARCH -B1 | grep frozen |cut -c 11-27)

Thanks to Mark -> this can be done a lot of easier if you want to kill all frozen messages:

exiqgrep -z -i | xargs exim -Mrm

If you want to do this only for some domains / email-addresses, use the first example.

PAM-Authentication & MySQL (v5) / b1gMail

tom — Thu, 15 Mar 2007 15:46:23 +0000

Enviroment:

SuSE Linux 10.1
- pam-0.99.3.0-29.3, pam-modules-10.1-7
- postfix-2.2.9-10
- mysql-5.0.18-20.8, mysql-client-5.0.18-16

I was trying to implement SASL-authentication via PAM (which is using a mysql-backend) in a postfix installation.
Output from maillog:

postfix/smtpd: warning: SASL authentication failure: Password verification failed postfix/smtpd: warning: [x.x.x.x]: SASL PLAIN authentication failed

Checking the syslog:

saslauthd: pam_b1gmail: cannot connect to mysql database (Access denied for user 'xx'@'localhost' (using password: YES)) saslauthd: DEBUG: auth_pam: pam_authenticate failed: Permission denied saslauthd: do_auth : auth failure: [user=xx] [service=smtp] [realm=domain.tld] [mech=pam] [reason=PAM auth error]

Reason:
The PAM module is using a deprecated way to connect to the MySQL-DB.

Solution:
Set the MySQL-Password to OLD_PASSWORD:

SET PASSWORD FOR user'@'localhost' = OLD_PASSWORD('password');

Pimp my spamassassin / FuzzyOCR

tom — Mon, 26 Feb 2007 21:26:35 +0000

Today I’ve implemented a new tool in our anti-spam system:
~~FuzzyOCR~~ (Dec 13, 2007: URL contains ads only now)

It’s an OCR software used as a plugin for SpamAssassin.
OCR means “optical character recognition” and describes the procedure to recognize characters and words from images. It’s quite useful when you try to catch so-called “Image Spam”, which uses normal text where the real message is hidden in images (inline gifs, etc.)

The results are quite good and I’m confident : )

Additionally to the packages described on the homepage of FuzzyOCR you’ll need another piece of software (at least with openSuSE 10.0): giflib-progs-4.1.3-7.i586.rpm

Here you can see an example, that I’ve just recieved and that was recognized as spam correctly:
0.7 EXTRA_MPART_TYPE: Header has extraneous Content-type…
1.1 HTML_20_30 BODY: Message is 20% to 30% HTML
0.0 HTML_MESSAGE BODY: HTML included in message
0.0 BAYES_50 BODY: Bayesian spam probability is 40 to 60%
0.8 SARE_GIF_ATTACH FULL: Email has a inline gif
0.7 MY_CID_AND_STYLE SARE: cid and style
8.0 FUZZY_OCR BODY: Mail contains an image with common spam text inside
Words found:
“viagra” in 1 lines
“cialis” in 1 lines
“xanax” in 1 lines
“valium” in 1 lines
“pharmacy” in 1 lines
(5 word occurrences found)

—
From: Antonia [mailto:ademakerzcxl@xxxx.xxx]
Sent: Wednesday, February 26, 2007 9:04 PM
To: xxxx@xxxx.xxx
Subject: *****SPAM***** How’s It Going

exim / filtering spam

tom — Thu, 01 Feb 2007 21:48:24 +0000

A graph to show you how many UCE-Mails are filtered per week.
This is a statistic from one of my company’s mailservers.
From about 3000 mails per hour -> more than 90% of all traffic is spam, UCE or virus-infected mail!

Explanation:
wrong-syntax: Somebody tried to fake a mail-programm
too-many-conn.: Anybody tried to send a mass-mail
faked-sender: Tried to spoof sender
virus: recognized virus (which was not blocked by other filters)
spam: recognized spam (which was not blocked by other filters)
known-dialup: known dialup notwork; normally no mail from here
dynamic ip: dialup network or forgot to set proper dns-reverse entry
no-reverse-dns: still allowed, but reverse dns should be set
greylisted: mail is rejected first time; after that it should be ok
rbl: dns blacklists
clean mail: passed through and should be ok

Pimp my exim / DynaStop

tom — Sun, 17 Dec 2006 12:43:13 +0000

I found a (new) software that finally helps to ban spam that is not blocked by various dns blacklists, greylisting, etc …

Have a look at:
http://tanaya.net/DynaStop/
and the forum:
http://www.exim-users.org/forums/forumdisplay.php?f=36

2007/05/08: Update of my personal DynaStop Whitelist

Here you’ll find a great piece of software, that filters incoming mail on the basis of whether a dynamic IP address is used. This reduces system load and resources in processing unwanted mail because all legitimate mail from mainly all ISPs will be sent from a proper mail exchange server (respectively the dns name). Over 350 million IP addresses were used in testing DynaStop for integrity and stability in identifying dynamic addresses and cross checked false positives.

Although the software is still in beta status, I use it in my company to kick out the rest of unwanted connections that are not recognized by RBLs or those that “survive” greylisting.

Result:

System load is massive reduced because of less work for spamassassin and antivirus-software
Connections that come back multiple times and overwit greylisting are banned without scanning the message
Combined with a daily-report (to control false positives) of blocked networks it’s really a powerful software