Copyright 2006-2008
Patrick R. McDonald <marlowe@antagonism.org> (GPG key)This document last modified 2008-07-14 14:42:24 -0700
Maildrop is the mail filter/mail delivery agent used by the Courier Mail Server. maildrop reads a mail message from standard input and based on your filtering rules, performs the appropriate action on the message. The maildrop package allows you to perform server side mail filtering. This allows you to write your filters once rather than having to write them for each additional client. I use maildrop to sort my email, report my spam and forward email from my wife to all my accounts. Needless to say it has made my life very easy.
Maildrop provides a logging functionality. This functionality allows you to log what actions maildrop is taking with your messages and where you want it to write. There are a couple of problems with the maildrop logging functionality. First, there is no means to limit the size of the log file. This means the logfile could grow to fill the partition and/or filesystem. Second, it does not possess the ability to rotate files by criteria. Programs such as logrotate while capable of handling these problem, do not posses an automatic restart capability and are not OS agnostic. Fortunately, DJB's multilog statisfies both of these requirements.
My previous method of install involved setting the maildrop file to /dev/stdout and called multilog from the appropriate .qmail file. While this works quite successfully, it is not without issues. Multilog expects to have exclusive control over the files in the directory named by the dir argument. A conflict can arise if the qmail-send process schedules two separate deliveries to the same address at the same time. This would cause two instances of multilog to attempt to access the directory with unknown results.
While some people use the setlock command to prevent this problem. However my friend, John Simpson, provided another solution. This solution involves the creation of a named pipe. The logfile of the mailfilter is set to write to the named pipe. Since the kernel automatically serializes writes to a pipe, you no longer have to worry about the race condition described above.
The installation instructions for maildrop can be found here in the INSTALL file in the tarball.
The installation instructions for multilog and parent package daemontools can be found here.
To use maildrop with multilog, maildrop must be run in delivery mode. The steps below show how to configure this for a qmail mail server. Other mail servers can be used with minimal changes to the instructions. I place the logs in "/var/log/maildrop", however any directory will do.
# mkdir -m 755 /var/log/maildrop
in the appropriate mailfilter file
logfile "/dev/stdout"
in the appropriate .qmail-default or .qmail file
| preline -f /usr/local/bin/maildrop .mailfilter | multilog t s1048576 n10 /var/log/maildrop/main
The above example will rotate the maildrop log file located in "/var/log/maildrop/main" every 1048576 bytes (1MB) and multilog will only keep the 10 latest copies of the logfile. The "n10" argument is redundant as the default is ten. The most recent logfile will be titled "current". Instructions for installing the multilog man page can be found here.
The new way involves creating a logger service using a named pipe and having maildrop log to the named pipe. For the purposes of this example, the named pipe will be "/tmp/log-maildrop".
First verify the daemontools "svscan" process is running. Choose a location where you want the physical service directories. I usually use "/var/service", however any directory may be used as long as it is not "/service".
Create the service directories with the following commands.
# mkdir -m 1755 /var/service/maildrop-logger
# mkdir -m 755 /var/serivce/maildrop-logger/log
# cd /var/service/maildrop-logger/log
# wget -c
http://www.antagonism.org/scripts/log-run
# mv log-run run
# chmod 755 run
# cd ..
# wget -c
http://www.antagonism.org/scripts/log-maildrop
# wget -c
http://www.antagonism.org/scripts/pipe-watcher
# wget -c
http://www.antagonism.org/scripts/maildrop-logger-run
# mv maildrop-logger-run run
# chmod 755 pipe-watcher log-maildrop run
The "pipe-watcher" script is available in its orginal form here. John Simpson's discussion of the variables inside the script is quoted in full below.
"The variables at the top of the "pipe-watcher" script control how it works. You should check these variables to make sure it works the way you need it to.
$pfile contains the full pathname of the
named pipe you will be watching. If you place the named pipe under the
/tmp directory (as I have done), then any process will be
able to write data to the pipe.
Note that at the beginning of the code, there is a
mkfifo command which actually creates the named pipe. The
-m u=rw,go=w argument controls the permissions of
the named pipe itself. This argument allows processes running as other
userids to write to the pipe. If this is not what you want, you should
adjust this parameter to reflect your wishes.
$cmd contains the full pathname of the
program you wish to run when data is received on the pipe. The
pipe-watcher program was written to be fairly generic- it
can be used for any number of other things which are not related to
qmail.
$cmd_needs_data tells whether or not the data
received from the pipe should be given to the program you wish to run.
For this application, the answer is no. If you set this variable to 1,
the data received from the pipe will be present on the "stdin" channel
of the program it runs.
$min_delay sets a length of time (in seconds)
which must pass between the program running. This is used to prevent a
flood of requests causing the program to run over and over
again."
I commented out the section which calls the $cmd and
$cmd_needs_data variables as the purpose of this
program is to merely log the data passed to it, not as in the case of
qmail-updater to run any external commands.
Below are the appropriate configuration changes for the ".mailfilter" and ".qmail" files.
in the appropriate mailfilter file
logfile "/tmp/log-maildrop"
in the appropriate .qmail-default or .qmail file
| preline -f /usr/local/bin/maildrop .mailfilter
(The below section is taken almost verbatim from the following page created by John Simpson. I felt that his description on what happens when you activate a service was the most clear and easy to understand, so why change a thing?)
Once the directories are set up, you need to make them start running. This is done by creating a symbolic link from /service/(whatever) to the physical directory where the service lives. The "svscan" program checks /service every five seconds, and when it sees a new directory (or symbolic link) there, it starts a "supervise" process for that directory. In addition, if the directory has the sticky bit set and a child directory called "log", it starts a "supervise" process for the "log" child directory and sets up a pipe between the two processes (so that the main process's logs end up being sent to the log process).
The "supervise" program works by running the "run" script inside of whatever directory it's watching. If that child process (either the "run" script itself, or whatever process it runs using "exec") stops, it starts it back up by running the "run" script again.
The following command will create the symbolic link needed to start the maildrop-logger service.
# ln -s /var/service/maildrop-logger /service
After running this command, wait ten seconds (to give it time to start) and then run the "svstat" command to see what's running:
# svstat /service/maildrop-logger /service/maildrop-logger/log
/service/maildrop-logger: up (pid 2508) 7 seconds
/service/maildrop-logger/log: up (pid 2510) 7 seconds
As long as the new services show "up" with a timer of more than one second, the services are running correctly. If the timer on a service is 0 or 1 second, then wait about five seconds and run the same command - it should now be higher than 1 second. If it's still 0 or 1, then the service is having a problem and you need to fix it. This page provides some steps to troubleshoot daemontools service installations.
Testing is easy... in one window (or on one console) you should watch the log file, and in another window/console, simply "touch" the named pipe (or send data to it.) You should make sure that the service reacts correctly to "touches" and data from both root and non-root users.
| In one window, watch the log file scroll by. | In another window (as a non-root user), trigger the service. |
|
# tail -F /service/maildrop-logger/log/main/current |
$ echo testing > /tmp/log-maildrop |
|
||||||||||||||
|
||||||||||||||
|
||||||||||||||
|
2008-05-05 Thanks to David Wadson for noting I transposed log-maildrop in the "Testing Service" and "Configuring Mailfilter" sections.
2007-03-12 Thanks to John Simpson for noting the error regarding which directory multilog desires exclusive control over. It is the one specified by the dir argument not the one preceding the "t" command.
2007-01-29 Thanks again to Bill Olson to noting I failed to include the activating service section.
2007-01-18 Thanks to Bill Olson for reminding me to add "cd /var/service/maildrop-logger/log" to the installation instructions.
2007-01-03 Added named pipe section based on idea provided by John Simpson.
Copyright 2006-2008
Patrick R. McDonald <marlowe@antagonism.org> (GPG key)