Wietse has got a new patch level release out for our beloved Postfix mailserver. Mostly bugfixes, some of them don’t bother us like problems with Redhat of FreeBSD but the others might. So I’ve compiled and checked it and I’m running it on my test servers and production servers without a problem. Just follow the install documentation and you’ll be alright.

Some of the bugfixes are:

• Message headers longer than 65535 broke the Milter protocol. To
  make matters worse the cleanup server could then dereference a
  null pointer. When Milter support is enabled, the length of each
  message header is now limited to 60000.
• Several fixes to improve worst-case behavior of the (new) queue
  manager with multi-recipient mail. The queue manager now reads
  new recipients earlier from the queue file, instead of becoming
  starved while waiting for the slowest in-memory recipients to
  complete; and it now reads recipients in smaller chunks to avoid
  spending too much time not talking to delivery agents.
• With remote SMTP server tarpit delays larger than the Postfix
  SMTP client’s smtp_rset_timeout (default: 20s), the client would
  get out of sync with the server while reusing a connection. The
  symptoms were “recipient rejected .. in reply to DATA”.

You can read all about how to setup maildrop in the documentation. If you have problems in using it please post your errors and problems in the forum.

Next up is to have maildrop create folders that not already exist in your inbox when needed and to have users have their own filtering rules. Hope you like it as it is currently…

I found out I needed it when I wanted to do some fancy stuff in my DSPAM setup. I’m almost there in releasing the documentation, still looking for more volunteers to beta test for me. The DSPAM filter is running as a content filter in Postfix and I’ve got a shell script running daily that empties my Junk folder and trains DSPAM to learn to be a better filter. More about this subject later…

It looks like the software update replaces our postfix binary with the original one. This is not listed in the knowledge base article so I didn’t know beforehand. I can’t confirm this problem is caused by the update, I need to research this or get more confirmations about it.

The problem is likely to only affect Intel macs, I haven’t installed everything on my Intel mac yet so I hadn’t noticed the problem yet but I didn’t have any problem on my G4 mac minis.

To solve the problem you need to reinstall postfix. If you still have the source tree intact on your harddrive you can do it quickly with just running a 'sudo make install'. If it’s no longer there then you need to recompile the source again from scratch.

Your existing configuration files won’t be touched so no worries there. Sorry I had not noticed this problem earlier. I’ll install my Intel mac mini asap to better understand what’s happening here and prevent any further problems with software updates.

I first did Postfix and Courier-auth as they would cause the least problems when it would go wrong. My backup mailserver would still receuve all mail and forward it to me when the mailserver is back online. It all went without a hitch, just did a restart of the programs and all was running as expected.

Apache posed an issue, how to compile and install, followed by compiling PHP without disturbing all the people looking at my sites. In the end it was simple. Compile and install Apache2, don’t restart Apache so the old version keeps running but the new software is available on disk. Then compile and install PHP using the new Apache installation files and when that is done restart Apache and all would be fine. Well that’s what I thought…

There was a small error in the modules that I included in the httpd.conf. Some of them, all concerning authentication, where rewritten and given different names. I was still trying to load the old ones as I didn’t update my httpd.conf to reflect these changes. So Apache2 wouldn’t start. I just commented out the old modules and added the new ones in my httpd.conf and presto, Apache 2.2.3 and PHP 5.1.6 are now running on my server, together with Postfix 2.2.3.

They’ve been running for almost a day now and I haven’t gotten any complaints yet.

If you are still running “ISP in a box” version 1 and want to upgrade but are still a bit afraid on how to do this let me know and I’ll help out.

Update:
Just found out there was an error, got a call on the phone from some of my friends that they couldn’t email anymore. When checking the logfiles I foud out I forgot a step after compiling and installing Courier auth:

sudo chmod o+x /usr/local/var/spool/authdaemon

In the logfile there was the error:

SASL authentication failure: cannot connect to Courier
      authdaemond: Permission denied

Which means I didn’t follow my own instructions !

But in the ongoing endeavor to stop spammers in sending emails to us the Postfix developers set stricter rules on their server communications. One of them being the stricter rules on sending multiple commands to the mailserver without listening for an answer from the mailserver after each command, which is typical for spam software. However this was also what PostfixAdmin was doing, sending all these commands to send an email to the newly created mail address without listening properly to the answers Postfix was sending.

The bug is fixed in PostfixAdmin CVS and is, as far as I can tell, scheduled to be released in the 2.1.1 release. But we encounter the bug now, I get remarks from people who are installing PostfixAdmin on top of Postfix 2.2.0 or later.

To help myself and them I copied some of the code that would solve the problem from CVS and put it into the current 2.1.0 source of PostfixAdmin. It solves the problem of Postfix rejecting emails being send from PostfixAdmin with the error:

Data command rejected: Improper use of SMTP command pipelining.

The error is solved by changing the ‘smtp_mail’ function to listen for answers send by the Postfix server in the ‘functions.in.php’ source file.

To help you out you can download the edited ‘functions.inc.php’ from here. To see what I’ve changed look at the forum entry describing the error.

I didn’t create the fix, I just copied some of the code that was available in CVS and back ported it to the current stable 2.1.0 version.

The important issues that got fixed in this release:

• File corruption while executing a Milter “header insert” action
  with headers-only mail (found with dk-filter). Delivery agents
  would go into an infinite loop because some queue file update
  had been done in the wrong order. As a precaution, delivery
  agents now detect such loops, and the queue manager now saves
  such mail to the “corrupt” directory.
• Segmentation fault in the SMTP client while saving a cached
  connection with unsent data. Postfix indexed some table with -1,
  because some I/O cleanup had been done in the wrong order. The
  same problem should exist in Postfix 2.2.
• Postfix no longer announces its name in delivery status notifications.
  All other details of the default bounce text remain unchanged.
  The reason for this change is that too many people believe that
  Wietse provides a free helpdesk service that solves all their
  email problems.

postfix/master[1120]: fatal: open lock file pid/master.pid
unable to set exclusive lock: Resource temporarily unavailable

So I went looking for the file and found it in /var/spool/postfix/pid/ and it looked fine. I stopped Postfix and the file disappeared and reappeared when I started it. Why ?

Next up, a quest for Google. The answer was strangely enough quite simple. There was another instance of Postfix running as one would expect. It was started by lauchd when new mail arrived. Somehow a leftover of trying to use the build-in Postfix application before we tried to install my version.

To make sure that the same problem is also generating errors on your machine, execute the following command:

sudo launchctl list

You would see a list like:

com.apple.dashboard.advisory.fetch
com.apple.KernelEventAgent
com.apple.mDNSResponder
com.apple.nibindd
com.apple.periodic-daily
com.apple.periodic-monthly
com.apple.periodic-weekly
com.apple.portmap
com.apple.syslogd

If in this list you see an item called: org.postfix.master then the other running instance of Postfix is indeed you problem and you can try the following solution.

Edit the file /System/Library/LaunchDaemons/org.postfix.master.plist with your favourite editor (as root or using sudo) and change the following block:

<dict>
        <key>Label</key>
        <string>org.postfix.master</string>
</dict>

into:

<dict>
        <key>Disabled</key>
        <key>Label</key>
        <string>org.postfix.master</string>
</dict>

As you can see you can just disable it by adding the line<key>Disabled</key>.

Please restart your Mac to restart the launchctl system. If it’s started check if your changes worked by sending yourself mail and check the logfile or run the command sudo launchctl list again to see if the item still appears.

Note: I’ve also seen systems where the master.pid file wasn’t removed by a previous instance of Postfix (because of a possible crash). Just deleting the file if no known instance of Postfix is running and try to run your instance again might also work on your systems. I’ve encountered both possibilities.

It was to do with the line
reject_rbl_client opm.blitzed.org
in the configuration of /etc/postfix/main.cf

By trying to find out why I was getting rejected, I got worried I was being tagged as a spamserver, I found out that opm.blitzed.org has shutdown it’s service. They shutdown in May 2006, I wonder why I haven’t noticed it earlier. The reasons where technical and you can read more on it here.

My advice is, if you still have the above line in your main.cf please remove it until the service is back up again. I’m very sorry if this has caused any inconvenience.

But in the week I was off-line Postfix got updated to a new release with all new functionality as well. I haven’t tested it or got into the details yet but I wanted to get the word out on this. I will spend time this week to find what impact these new changes mean to our setup as there are some major changes made to the software:

• DSN (delivery status notification) support as described in RFC
  3461 .. RFC 3464. This gives email senders control over notification
  of successful, delayed, and failed delivery. DSN involves extra
  parameters to the SMTP “MAIL FROM” and “RCPT TO” commands, as well
  as extra Postfix sendmail command line options for mail submission.
• Major updates to the TLS (SMTP encryption and authentication)
  support. Postfix 2.3 introduces a configuration user interface
  that is based on the concept of TLS security levels (none, may,
  encrypt, verify, secure) and that can more effectively deal with
  DNS spoofing. The old configuration user interface, with multiple
  boolean parameters to enable or enforce TLS, is still supported but
  will be removed after a few releases.
• Milter (mail filter) application support, compatible with Sendmail
  version 8.13.6 and earlier. This allows you to run a large number
  of plug-ins to reject unwanted mail, and to sign mail with for
  example domain keys. All Milter functions are implemented except
  the one that replaces the message body (this will be added later).
• Enhanced status codes (RFC 3463). For example, status code 5.1.1
  means “recipient unknown”. Mail clients can translate these status
  codes into text in the user’s own language, and greatly improve the
  user experience. Enhanced status codes can be specified in Postfix
  access tables, in header/body_checks content filter rules, in “rbl”
  reply templates, and so on.
• Configurable bounce messages with support for non-ASCII character
  sets.
• Plug-in support for SASL authentication in the Postfix SMTP server
  and client. With this, Postfix can support multiple SASL implementations
  without conflicting source code patches. Postfix 2.3 has Dovecot
  SASL support built into the SMTP server. As before, support for
  Cyrus SASL is available as add-on feature for the Postfix SMTP
  server and client. (we have this working already in the current setup)
• Support for sender-dependent ISP accounts, in the form of
  sender-dependent relayhost lookup and sender-dependent SASL
  username/password lookup.
• The Postfix SMTP client now implements both the SMTP and LMTP
  protocols. This means that a lot of features have become available
  for LMTP mail delivery, including the shared TCP connection cache.
• After TLS handshake failure, the SMTP client will now reconnect
  to the same server to try plaintext delivery (if TLS policy permits).
  Earlier Postfix versions would skip the server and defer delivery
  if no alternate MX host was available.
• All delay logging now has sub-second resolution. Besides the total
  delay, Postfix logs separate delays for different stages of delivery
  (time in queue, time in queue manager, time to set up connection,
  and time to deliver). This gives better insight into the nature of
  performance bottle necks.
• Smarter utilisation of cached SMTP connections. When one destination
  has multiple inbound SMTP servers, the Postfix SMTP client will now
  send less mail via the slower ones, and more mail via the faster ones.
• Support for empty MX records. Older Postfix versions treat this
  as a malformed response and defer mail delivery.

Most interesting new features, I think, will be the DSN functionality and Milter (the mail filter). I will spend some time the coming week to see how we can use them in our setup and if there are more benefits to this new version.

Again, I haven’t implemented this new version yet due to lack of time but I will let you know when and how you can safely upgrade your setup to this new 2.3 version.