Postfix rate limiting – Politeness goes a long way

When sending large amounts of bulk emails, it may be necessary to enforce rate limiting to prevent the ISP’s from blocking your servers.

Reputation

In the world of high volume senders, a key metric for detecting spammers is the IP addresses’s reputation. Building a good reputation for your servers takes time and hence patience but it pays off with both delivery rate and performance, allowing you to send more email from a single resource.
Until you have good reputation, and to gain a better one to start with, it is recommended that you enforce rate limiting on your MTA. In this post I will focus on Postfix rate limiting, which I’m guessing is why you are here anyway.

The dreaded 421 error

Most email service provers allows you to send them emails at a certain unpublished rate. Some are politer than others, such as lets say, Yahoo, but when sending to a list of more than about 1000 recipients, it more than possible that you will face a temporary block. To prevent it from happening you should fine tune your postfix policy to create some form of delay between the emails that are sent to the same domain.
If you already encountered this error, it may look like:

Our system has detected an unusual rate of 421-4.7.0 unsolicited mail originating from your IP address. To protect our 421-4.7.0 users from spam, mail sent from your IP address has been temporarily 421-4.7.0 blocked. Please visit http://www.google.com/mail/help/bulk_mail.html 421 4.7.0 to review our Bulk Email Senders Guidelines.

Postfix rate limiting configuration

My first example will be of a global configuration and assumes that your server is handling only outgoing emails.
We need to edit the main.cf file and add the following lines:

smtp_destination_concurrency_limit = 2
smtp_destination_rate_delay = 1s
smtp_extra_recipient_limit = 10

Lets go over each line:

  • default_destination_concurrency_limit = 2
    This means that postfix will up to two concurrent connections per receiving domains.
    The default value is 20.
  • default_destination_rate_delay = 1s
    Postfix will add a delay between each message to the same receiving domain. It overrides the previous rule and in this example, it will send one email after another with a delay of 1 second. If you want to disable this rule, either delete it or set to 0.
  • default_extra_recipient_limit = 10
    Limit the number of recipients of each message. If a message had 20 recipients on the same domain, postfix will break it out to two different email messages instead of one.

Restart your postfix service and you are done.

If you are sending thousands of emails, and need to do so fairly quickly you will learn that this configuration will delay your rate considerably. Once you gain a better reputation, you will be able to move to the next step.

Different policies for different domains

Now, as you fine tune your policy, it may be apparent that some providers allow you to send emails at a higher rate, and some may need more delay between each message.
Lets create separate policies so we can optimise our server’s performance.

The first step will be to edit the master.cf file and add SMTP transports. Each transport will represent a policy for a group of receiving domain:

polite unix - - n - - smtp
turtle unix - - n - - smtp

Next, we need to map a domain to it’s transport name:
Edit the /etc/postfix/transport file and add the lines

gmail.com polite:
yahoo.com turtle:
hotmail.com polite:

Now we can add the policy to the master.cfmain.cf file:

transport_maps = hash:/etc/postfix/transport

polite_destination_concurrency_limit = 2
polite_destination_rate_delay = 0
polite_destination_recipient_limit = 5

turtle_destination_concurrency_limit = 1
turtle_destination_rate_delay = 3s
turtle_destination_recipient_limit = 2

Note that these values are just an example and you will have to tweak it yourself over time.

To activate these changes, we need to reload the postfix configuration:
postmap /etc/postfix/transport
service postfix restart

Read the ISP’s guidelines for bulk senders to better understand how to improve your reputation:

If you have any questions you can find me on IRC – #steamio on Freenode

Be Sociable, Share!

    , , , , , , , , , , , , , , , , , , , , , ,

    • Mela

      Great Article, A tip make sure you are Indenting your master.cf 2 lines or it will fail to load.

      Also need to do postmap transport

      Also how would I know if its working? Some kind of status on the logs?

      • YuvalO

        Thanks for the note, I updated the post.

        One way to know if its working, is to grep the mail log file for a domain and compare the timestamps.

    • sacksmack

      nice share Yuval

    • Marco

      Hi, you may want to check the settings after reading this conversation where the author of Postfix participates:

      http://postfix.1071664.n5.nabble.com/Destination-rate-delay-not-working-td50420i20.html

      • YuvalO

        Hi Marco,

        Thanks for the link. This is very useful information straight from the source.
        I don’t see anything that contradicts the configuration examples here, but I will add this information to the post.

        Yuval

    • Fausto Paiva

      Thanks. Useful information when you need setup a mail server inside your company and send too many emails simultaneously.

    • Arthur Pemberton

      Thanks for the this information… I should have guessed that this
      feature was baked into Postfix and needn’t be implemented in the
      application.

    Join my Online Postfix Class
    Join my Online Postfix Class
    I am working on a new Postfix training class that will be launching soon. Let me know if you have a topic that you want me to include in the class.

    Powered by WordPress. Designed by Woo Themes