Sendmail introduced a
new API beginning with version 8.10 -
libmilter. The milter module for Python
provides a python interface to libmilter that exploits all its features.
Sendmail 8.12 officially releases libmilter. Version 8.12 seems to be more robust, and includes new privilege separation features to enhance security. I recommend upgrading.
Release 0.6.1 adds a full milter based dspam application.
To use header triage, you must have DSPAM installed, and select a dictionary that is well moderated by someone who gets lots of spam. That dictionary can be used to block spam that is obvious from the headers (e.g. X-Mailer and Subject) before it ties up any more resources. I have yet to see any false positives from this approach (check the milter log), but if there are, the sender will get a REJECT with the message "Your message looks spammy."
To Bcc a message, call self.add_recipient(rcpt) in envfrom after
determining whether you want to copy (e.g. whether the sender is local). For
example,
def envfrom(...
...
if len(t) == 2:
self.rejectvirus = t[1] in reject_virus_from
if t[0] in wiretap_users.get(t[1],()):
self.add_recipient(wiretap_dest)
if t[1] == 'mydomain.com':
self.add_recipient('<copy-%s>' % t[0])
...
To make this a generic feature requires thinking about how the configuration would look. Feel free to make specific suggestions about config file entries. Be sure to handle both Bcc and file copies, and designating what mail should be copied. How should "outgoing" be defined? Implementing it is easy once the configuration is designed.
At the lowest level, the 'milter' module provides a thin wrapper around the sendmail libmilter API. This API lets you register callbacks for a number of events in the process of sendmail receiving a message via SMTP. These events include the initial connection from a MTA, the envelope sender and recipients, the top level mail headers, and the message body. There are options to mangle all of these components of the message as it passes through the milter.
At the next level, the 'Milter' module (note the case difference) provides a Python friendly object oriented wrapper for the low level API. To use the Milter module, an application registers a 'factory' to create an object for each connection from a MTA to sendmail. These connection objects must provide methods corresponding to the libmilter callback events.
Each event method returns a code to tell sendmail whether to proceed with processing the message. This is a big advantage of milters over other mail filtering systems. Unwanted mail can be stopped in its tracks at the earliest possible point.
The Milter.Milter class provides default implementations for event methods that do nothing, and also provides wrappers for the libmilter methods to mutate the message.
Finally, the bms.py application is both a sample of how to use the Milter module, and the beginnings of a general purpose SPAM filtering, wiretapping, and Win32 virus protection milter.
The latest version is 0.6.7. See the Change Log.
milter-0.6.7.tar.gz Explicit local socket bug, SRS forgery detection, thread resource starvation detection. SRS support requires pysrs.
milter-0.6.7-3.i386.rpm Binary RPM for Redhat 7.x, now requires
sendmail-8.12 and
python2.3.
milter-0.6.7-3.src.rpm Source RPM for Redhat 7.x.
Release 0.6.7-3 patches:
Stable milter-0.6.6.tar.gz Plug another memory leak, SPF support, hello blacklist. SPF support requires pydns. NOTE - the spf.py module included is modified from the official 1.6 version at wayforward.net. I neglected to add the CVS log. The changes are expanded result codes and tolerating common method misspellings in SPF records. I have notified the author, but haven't heard back. At some point, the RPM will include the official pyspf tarball and apply patches.
milter-0.6.6-2.i386.rpm Binary RPM for Redhat 7.x, now requires
sendmail-8.12 and
python2.3. Release 2 fixes sysv init script bug for python2.3.
milter-0.6.6-2.src.rpm Source RPM for Redhat 7.x
milter-0.6.5.tar.gz Plug memory leak, progress reporting, trusted relay. Redhat RPM now requires sendmail-8.12.
milter-0.6.5-2.i386.rpm Binary RPM for Redhat 7.x
milter-0.6.5-2.src.rpm Source RPM for Redhat 7.x
milter-0.6.4.tar.gz Numerous Dspam fixes. Requires pydspam-1.1.5 and dspam-2.6.5.2 for Dspam features. The dspam-python RPM has been replaced by pydspam.
milter-0.6.4-1.i386.rpm Binary RPM for Redhat 7.x
milter-0.6.3.1.tar.gz New dspam SCREENER feature with pydspam-1.1.4. Don't save a defang copy of false positives. Fixed an oops from last fix, rejecting false positives. BUG: sendmail-8.11 doesn't invoke milter when sending mail via sendmail from command line (8.12 works). Therefore, the supplied falsepositive script for milter based dspam doesn't work with stock RedHat 7.x. I am writing a HOWTO for configuring milter based dspam that will address this (and a fix in the next version).
milter-0.6.3-1.i386.rpm Binary RPM for Redhat 7.x
milter-0.6.2.tar.gz work around email.Message.get_filename bug, dspam_exempt list, REJECT messages with missing MIME boundaries (which are almost always spam), DISCARD messages which any dspam user flags as spam, start.sh was calling python instead of python2 on Linux.
milter-0.6.2-1.src.rpm Source RPM for Redhat 7.x (and likely higher versions)
milter-0.6.1.tar.gz dspam milter application, python-2.2.3 support.
You must have dspam and dspam-python loaded for the dspam feature to work. Brief instructions for configuring are in the default config file. This is working at a customer, but I'm sure a few more iterations will be required to make setup as smooth as possible.
NOTE: Outlook destroys dspam tags when forwarding mail (while converting HTML to text). Perhaps some config option will turn this abominable "feature" off. Working around this by making dspam tags visble on HTML mail is ugly. My suggestion is to not use Outlook, for this and many other reasons - especially security. Any other suggestions for those married to Microsoft are welcome. The DSPAM LDA works around this by making the tags visible in HTML attachments. This is ugly, and occasionally corrupts attachments.
We have to supply workarounds for bugs in the email module (reported to sourceforge). The workarounds reference some internal variables which change with python versions.
milter-0.6.1-1.i386.rpm Binary RPM for Redhat 7.x
milter-0.6.1-1.src.rpm Source RPM for Redhat 7.x (and likely higher versions)
milter-0.6.0.tar.gz simple dspam pre-filtering, use email module, requires python >= 2.2.2.
milter-0.6.0-1.i386.rpm Binary RPM for Redhat 7.x
milter-0.6.0-1.src.rpm Source RPM for Redhat 7.x (and likely higher versions)
milter-0.5.5.tar.gz IPV6 support, passing None to set_XXX_callback, set_reply, chg_header, detect internal connections. Note, this release did not work on AIX4.1.5, probably due to IPV6 support breaking something. The milter.so module from 0.5.4 can be installed to use this release with AIX.
milter-0.5.4.tar.gz wiretap, smart alias features, quarantine support.
The name of the production "sample" milter "bms.py" now stands for "Basic Milter System" until someone suggests a better name. The test coverage is rather sparse at present. Please email with proposals for what to name the milter application.
self.readahead = "" to mimepart.seek.
Python-2.2 multifile reads one less newline per section than
2.1. I'm not not sure which is correct. After adding some calls to
rstrip() in testmime.py, all milter modules pass unit testing
with python-2.2. Python-2.2 patches have been released since 0.5.3.
milter-0.5.2.tar.gz Fix and unittest another HTML parsing bug.
milter-0.5.1.tar.gz Handle encoded rfc822 attachments.
milter-0.5.0.tar.gz Use a config file so users don't have to
keep syncing with bms.py.
milter-0.4.5.tar.gz Work with sgmlop. Reduce local hacks to config variables.
Python milter is under GPL. The authors can probably be convinced to change this to LGPL.
The Python milter package includes a sample milter that replaces dangerous
attachments with a warning message, discards mail addressed to
MAILER-DAEMON, and demonstrates several SPAM abatement strategies.
The MimeMessage class to do this used to be based on the
mimetools and multifile standard python packages.
As of milter version 0.6.0, it is based on the email standard
python packages, which were derived from the
mimelib project.
The MimeMessage class patches several bugs in the email package,
and provides some backward compatibility.
The "defang" function of the sample milter was inspired by MIMEDefang, a Perl milter with flexible attachment processing options. The latest version of MIMEDefang uses an apache style process pool to avoid reloading the Perl interpreter for each message. This makes it fast enough for production and does not use Perl threading.
mailchecker is a Python project to provide flexible attachment processing for mail. I will be looking at plugging mailchecker into a milter.
TMDA is a Python project to require confirmation the first time someone tries to send to your mailbox. This would be a nice feature to have in a milter.
There is also a Milter community website where milter software and gory details of the API are discussed.
For example, the HTML parsing feature to remove scripts from HTML attachments is rather CPU intensive in pure python. Using the C replacement for sgmllib greatly speeds things up.
| Operating System | Compiler | Python | Sendmail | milter |
|---|---|---|---|---|
| Mandrake 8.0 | gcc-3.0.1 | 2.1.1 | 8.12.0 | 0.3.3 |
| Mandrake 8.0 | gcc-2.96 | 2.0 | 8.11.2 | 0.3.6 |
| RedHat 6.2 | egcs-1.1.2 | 2.2.2 | 8.11.6 | 0.5.4 |
| RedHat 7.1 | gcc-2.96 | ? | 8.12.1 | 0.3.5 |
| RedHat 7.2 | gcc-2.96 | 2.1.1 | 8.11.6 | 0.4.1 |
| RedHat 7.2 | gcc-2.96 | 2.2.1 | 8.11.6 | 0.4.5 |
| RedHat 7.2 | gcc-2.96 | 2.2.2 | 8.11.6 | 0.5.5 |
| RedHat 7.2 | gcc-2.96 | 2.3.3 | 8.12.10 | 0.6.6 |
| RedHat 7.3 | gcc-2.96 | 2.2.2 | 8.11.6 | 0.5.5 |
| RedHat 7.3 | gcc-2.96 | 2.3.3 | 8.12.10 | 0.6.6 |
| RedHat 8.0 | gcc-3.2 | 2.2.1 | 8.12.6 | 0.5.2 |
| Debian Linux | gcc-2.95.2 | 2.1.1 | 8.12.0 | 0.3.7 |
| Debian Linux | gcc-3.2.2 | 2.2.2 | 8.12.7 | 0.5.4 |
| AIX-4.1.5 | gcc-2.95.2 | 2.1.1 | 8.11.5 | 0.3.3 |
| AIX-4.1.5 | gcc-2.95.2 | 2.1.1 | 8.12.1 | 0.3.4 |
| AIX-4.1.5 | gcc-2.95.2 | 2.1.3 | 8.12.3 | 0.4.2 |
| AIX-4.1.5 | gcc-2.95.2 | 2.2.2 | 8.12.6 | 0.5.4 |
| Slackware 7.1 | ? | ? | 8.12.1 | 0.3.8 |
| Slackware 9.0 | gcc-3.2.2 | 2.2.3 | 8.12.9 | 0.5.4 |
| OpenBSD | ? | 2.1.1 | 8.11.6 | 0.3.9 |
| SuSE 7.3 | gcc-2.95.3 | 2.1.1 | 8.12.2 | 0.3.9 |
| FreeBSD | gcc-2.95.3 | 2.2.1 | 8.12.3 | 0.4.0 |
| FreeBSD | gcc-2.95.3 | 2.2.2 | ? | 0.5.5 |
| FreeBSD 4.4 | gcc-2.95.3 | ? | 8.12.10 | 0.6.6 |
Sendmail-8.12 renames libsmutil.a to libsm.a. Unfortunately, libsm.a is an important AIX system shared library. Therefore, I rename libsm.a back to libsmutil.a for AIX. This presents a problem for setup.py.
/usr/include/libmilter/mfapi.h.
If you do modify the SRPM, I suggest renaming libsmutil.a
to libsm.a - just like sendmail-8.12 will. If you manually install
mfapi.h or don't rename libsmutil.a, you'll
need to force libs = ["milter", "smutil"] in setup.py.
If you have installed python2, and want
python-milter to use python2, add python=python2 to setup.cfg
and build with python2 setup.py bdist_rpm.
If y'all trust me, you can pick up source and binary sendmail RPMs for RH6.2 from my linux downloads directory. The lastest RPMs were built by taking a RH7.2 SRPMS and removing some RPM features from the spec file that RH6.2 doesn't support, then recompiling on RH6.2. You can check this by installing the RH7.2 SRPM, then diffing my sendmail.spec with theirs. Then run "rpm -bb sendmail-rhmilter.spec" when you are satisfied.
If you have installed python2, and want
python-milter to use python2, add python=python2 to setup.cfg
and build with python2 setup.py bdist_rpm.
You'll need to install the sendmail-devel package to compile milter.