Postfix before-queue Milter support


Introduction

Postfix implements support for the Sendmail version 8 Milter (mail filter) protocol. This protocol is used by applications that run outside the MTA to inspect SMTP events (CONNECT, DISCONNECT), SMTP commands (HELO, MAIL FROM, etc.) as well as mail content (headers and body). All this happens before mail is queued.

The reason for adding Milter support to Postfix is that there exists a large collection of applications, not only to block unwanted mail, but also to verify authenticity (examples: OpenDKIM, DomainKeys Identified Mail (DKIM), SenderID+SPF and DomainKeys) or to digitally sign mail (examples: OpenDKIM, DomainKeys Identified Mail (DKIM), DomainKeys). Having yet another Postfix-specific version of all that software is a poor use of human and system resources.

The Milter protocol has evolved over time, and different Postfix versions implement different feature sets. See the workarounds and limitations sections at the end of this document for differences between Postfix and Sendmail implementations.

This document provides information on the following topics:

How Milter applications plug into Postfix

The Postfix Milter implementation uses two different lists of mail filters: one list of filters for SMTP mail only, and one list of filters for non-SMTP mail. The two lists have different capabilities, which is unfortunate. Avoiding this would require major restructuring of Postfix.

For those who are familiar with the Postfix architecture, the figure below shows how Milter applications plug into Postfix. Names followed by a number are Postfix commands or server programs, while unnumbered names inside shaded areas represent Postfix queues. To avoid clutter, the path for local submission is simplified (the OVERVIEW document has a more complete description of the Postfix architecture).

SMTP-only
filters
non-SMTP
filters
^
|
|
v
^
|
|
|
|
|
|
v
Network -> smtpd(8)
\
Network -> qmqpd(8) -> cleanup(8) -> incoming
/
pickup(8)
:
Local -> sendmail(1)

Building Milter applications

Milter applications have been written in C, JAVA and Perl, but this document deals with C applications only. For these, you need an object library that implements the Sendmail 8 Milter protocol. Postfix currently does not provide such a library, but Sendmail does.

Running Milter applications

To run a Milter application, see the documentation of the filter for options. A typical command looks like this:

# /some/where/dkim-filter -u userid -p inet:portnumber@localhost ...other options...

Please specify a userid value that isn't used for other applications (not "postfix", not "www", etc.).

Configuring Postfix

Like Sendmail, Postfix has a lot of configuration options that control how it talks to Milter applications. Besides global options that apply to all Milter applications, Postfix 2.12 and later support per-Milter timeouts, per-Milter error handling, etc.

Information in this section:

SMTP-Only Milter applications

The SMTP-only Milter applications handle mail that arrives via the Postfix smtpd(8) server. They are typically used to filter unwanted mail, and to sign mail from authorized SMTP clients. Mail that arrives via the Postfix smtpd(8) server is not filtered by the non-SMTP filters that are described in the next section.

NOTE: Do not use the header_checks(5) IGNORE action to remove Postfix's own Received: message header. This causes problems with mail signing filters. Instead, keep Postfix's own Received: message header and use the header_checks(5) REPLACE action to sanitize information.

You specify SMTP-only Milter applications (there can be more than one) with the smtpd_milters parameter. Each Milter application is identified by the name of its listening socket; other Milter configuration options will be discussed in later sections. Milter applications are applied in the order as specified, and the first Milter application that rejects a command will override the responses from other Milter applications.

/etc/postfix/main.cf:
    # Milters for mail that arrives via the smtpd(8) server.
    # See below for socket address syntax.
    smtpd_milters = inet:localhost:portnumber ...other filters...

The general syntax for listening sockets is as follows:

unix:pathname

Connect to the local UNIX-domain server that is bound to the specified pathname. If the smtpd(8) or cleanup(8) process runs chrooted, an absolute pathname is interpreted relative to the Postfix queue directory.

inet:host:port

Connect to the specified TCP port on the specified local or remote host. The host and port can be specified in numeric or symbolic form.

NOTE: Postfix syntax differs from Milter syntax which has the form inet:port@host.

Non-SMTP Milter applications

The non-SMTP Milter applications handle mail that arrives via the Postfix sendmail(1) command-line or via the Postfix qmqpd(8) server. They are typically used to digitally sign mail. Although non-SMTP filters can be used to filter unwanted mail, there are limitations as discussed later in this section. Mail that arrives via the Postfix smtpd(8) server is not filtered by the non-SMTP filters.

NOTE: Do not use the header_checks(5) IGNORE action to remove Postfix's own Received: message header. This causes problems with mail signing filters. Instead, keep Postfix's own Received: message header and use the header_checks(5) REPLACE action to sanitize information.

You specify non-SMTP Milter applications with the non_smtpd_milters parameter. This parameter uses the same syntax as the smtpd_milters parameter in the previous section. As with the SMTP-only filters, you can specify more than one Milter application; they are applied in the order as specified, and the first Milter application that rejects a command will override the responses from the other applications.

/etc/postfix/main.cf:
    # Milters for non-SMTP mail.
    # See below for socket address syntax.
    non_smtpd_milters = inet:localhost:portnumber ...other filters...

There's one small complication when using Milter applications for non-SMTP mail: there is no SMTP session. To keep Milter applications happy, the Postfix cleanup(8) server actually has to simulate the SMTP client CONNECT and DISCONNECT events, and the SMTP client EHLO, MAIL FROM, RCPT TO and DATA commands.

This generally works as expected, with only one exception: non-SMTP filters must not REJECT or TEMPFAIL simulated RCPT TO commands. When a non_smtpd_milters application REJECTs or TEMPFAILs a recipient, Postfix will report a configuration error, and mail will stay in the queue.

Signing internally-generated bounce messages

  • Postfix normally does not apply content filters to mail that is generated internally such as bounces or Postmaster notifications. Filtering internally-generated bounces would result in loss of mail when a filter rejects a message, as the resulting double-bounce message would almost certainly also be blocked.

    To sign Postfix's own bounce messages, enable filtering of internally-generated bounces (line 2 below), and don't reject any internally-generated bounces with non_smtpd_milters, header_checks or body_checks (lines 3-5 below).

    1 /etc/postfix/main.cf:
    2     internal_mail_filter_classes = bounce
    3     non_smtpd_milters = don't reject internally-generated bounces
    4     header_checks = don't reject internally-generated bounces
    5     body_checks = don't reject internally-generated bounces
    

    Milter error handling

    The milter_default_action parameter specifies how Postfix handles Milter application errors. The default action is to respond with a temporary error status, so that the client will try again later. Specify "accept" if you want to receive mail as if the filter does not exist, and "reject" to reject mail with a permanent status. The "quarantine" action is like "accept" but freezes the message in the "hold" queue, and is available with Postfix 2.6 or later.

    /etc/postfix/main.cf:
        # What to do in case of errors? Specify accept, reject, tempfail,
        # or quarantine (Postfix 2.6 or later).
        milter_default_action = tempfail
    

    See "Different settings for different Milter applications" for advanced configuration options.

    Milter protocol version

    As Postfix is not built with the Sendmail libmilter library, you may need to configure the Milter protocol version that Postfix should use. The default version is 6 (before Postfix 2.6 the default version is 2).

    /etc/postfix/main.cf:
        # Postfix ≥ 2.6
        milter_protocol = 6
        # 2.3 ≤ Postfix ≤ 2.5
        milter_protocol = 2
    

    If the Postfix milter_protocol setting specifies a too low version, the libmilter library will log an error message like this:

    application name: st_optionneg[xxxxx]: 0xyy does not fulfill action requirements 0xzz
    

    The remedy is to increase the Postfix milter_protocol version number. See, however, the limitations section below for features that aren't supported by Postfix.

    With Postfix 2.7 and earlier, if the Postfix milter_protocol setting specifies a too high version, the libmilter library simply hangs up without logging a warning, and you see a Postfix warning message like one of the following:

    warning: milter inet:host:port: can't read packet header: Unknown error : 0
    warning: milter inet:host:port: can't read packet header: Success
    warning: milter inet:host:port: can't read SMFIC_DATA reply packet header: No such file or directory
    

    The remedy is to lower the Postfix milter_protocol version number. Postfix 2.8 and later will automatically turn off protocol features that the application's libmilter library does not expect.

    See "Different settings for different Milter applications" for advanced configuration options.

    Milter protocol timeouts

    Postfix uses different time limits at different Milter protocol stages. The table shows the timeout settings and the corresponding protocol stages (EOH = end of headers; EOM = end of message).

    Postfix parameter Time limit Milter protocol stage
    milter_connect_timeout 30s CONNECT
    milter_command_timeout 30s HELO, MAIL, RCPT, DATA, UNKNOWN
    milter_content_timeout 300s HEADER, EOH, BODY, EOM

    Beware: 30s may be too short for Milter applications that do lots of DNS lookups. However, if you increase the above timeouts too much, remote SMTP clients may hang up and mail may be delivered multiple times. This is an inherent problem with before-queue filtering.

    See "Different settings for different Milter applications" for advanced configuration options.

    Different settings for different Milter applications

    The previous sections list a number of Postfix main.cf parameters that control time limits and other settings for all Postfix Milter clients. This is sufficient for simple configurations. With more complex configurations it becomes desirable to have different settings for different Milter clients. This is supported with Postfix 2.12 and later.

    The following example shows a "non-critical" Milter client with a short connect timeout, and with "accept" as default action when the service is unvailable.

    1 /etc/postfix/main.cf:
    2     smtpd_milters = { inet:host:port,
    3         connect_timeout=10s, default_action=accept }
    

    Instead of a server endpoint, we now have a list enclosed in {}.

    Inside the list, syntax is similar to what we already know from main.cf: items separated by space or comma. There is one difference: you must enclose a setting in parentheses, as in "{ name = value }", if you want to have space or comma within a value or around "=".

    Sendmail macro emulation

    Postfix emulates a limited number of Sendmail macros, as shown in the table. Some macro values depend on whether a recipient is rejected (rejected recipients are available on request by the Milter application). Different macros are available at different Milter protocol stages (EOH = end-of-header, EOM = end-of-message); their availability is not always the same as in Sendmail. See the workarounds section below for solutions.

    Sendmail macro Milter protocol stage Description
    i DATA, EOH, EOM Queue ID, also Postfix queue file name
    j Always Value of myhostname
    _ Always The validated client name and address
    {auth_authen} MAIL, DATA, EOH, EOM SASL login name
    {auth_author} MAIL, DATA, EOH, EOM SASL sender
    {auth_type} MAIL, DATA, EOH, EOM SASL login method
    {client_addr} Always Client IP address
    {client_connections} CONNECT Connection concurrency for this client (zero if the client is excluded from all smtpd_client_* limits).
    {client_name} Always Client hostname
    When address → name lookup or name → address verification fails: "unknown"
    {client_port} Always (Postfix ≥2.5) Client TCP port
    {client_ptr} CONNECT, HELO, MAIL, DATA Client name from address → name lookup
    When address → name lookup fails: "unknown"
    {cert_issuer} HELO, MAIL, DATA, EOH, EOM TLS client certificate issuer
    {cert_subject} HELO, MAIL, DATA, EOH, EOM TLS client certificate subject
    {cipher_bits} HELO, MAIL, DATA, EOH, EOM TLS session key size
    {cipher} HELO, MAIL, DATA, EOH, EOM TLS cipher
    {daemon_name} Always value of milter_macro_daemon_name
    {mail_addr} MAIL Sender address
    {mail_host} MAIL (Postfix ≥ 2.6, only with smtpd_milters) Sender next-hop destination
    {mail_mailer} MAIL (Postfix ≥ 2.6, only with smtpd_milters) Sender mail delivery transport
    {rcpt_addr} RCPT Recipient address
    With rejected recipient: descriptive text
    {rcpt_host} RCPT (Postfix ≥ 2.6, only with smtpd_milters) Recipient next-hop destination
    With rejected recipient: enhanced status code
    {rcpt_mailer} RCPT (Postfix ≥ 2.6, only with smtpd_milters) Recipient mail delivery transport
    With rejected recipient: "error"
    {tls_version} HELO, MAIL, DATA, EOH, EOM TLS protocol version
    v Always value of milter_macro_v

    Postfix sends specific sets of macros at different Milter protocol stages. The sets are configured with the parameters as described in the table (EOH = end of headers; EOM = end of message). The protocol version is a number that Postfix sends at the beginning of the Milter protocol handshake.

    As of Sendmail 8.14.0, Milter applications can specify what macros they want to receive at different Milter protocol stages. An application-specified list takes precedence over a Postfix-specified list.

    Postfix parameter Milter protocol version Milter protocol stage
    milter_connect_macros 2 or higher CONNECT
    milter_helo_macros 2 or higher HELO/EHLO
    milter_mail_macros 2 or higher MAIL FROM
    milter_rcpt_macros 2 or higher RCPT TO
    milter_data_macros 4 or higher DATA
    milter_end_of_header_macros 6 or higher EOH
    milter_end_of_data_macros 2 or higher EOM
    milter_unknown_command_macros 3 or higher unknown command

    Workarounds

    Limitations

    This section lists limitations of the Postfix Milter implementation. Some limitations will be removed as the implementation is extended over time. Of course the usual limitations of before-queue filtering will always apply. See the CONTENT_INSPECTION_README document for a discussion.