When sending large amounts of bulk emails, it may be necessary to enforce rate limiting to prevent the ISP’s from blocking your servers.
In the world of high volume senders, a key metric for detecting spammers is the IP addresses’s reputation. Building a good reputation for your servers takes time and hence patience but it pays off with both delivery rate and performance, allowing you to send more email from a single resource.
Until you have good reputation, and to gain a better one to start with, it is recommended that you enforce rate limiting on your MTA. In this post I will focus on Postfix rate limiting, which I’m guessing is why you are here anyway.
The dreaded 421 error
Most email service provers allows you to send them emails at a certain unpublished rate. Some are politer than others, such as lets say, Yahoo, but when sending to a list of more than about 1000 recipients, it more than possible that you will face a temporary block. To prevent it from happening you should fine tune your postfix policy to create some form of delay between the emails that are sent to the same domain.
If you already encountered this error, it may look like:
Our system has detected an unusual rate of 421-4.7.0 unsolicited mail originating from your IP address. To protect our 421-4.7.0 users from spam, mail sent from your IP address has been temporarily 421-4.7.0 blocked. Please visit http://www.google.com/mail/help/bulk_mail.html 421 4.7.0 to review our Bulk Email Senders Guidelines.
Postfix rate limiting configuration
My first example will be of a global configuration and assumes that your server is handling only outgoing emails.
We need to edit the main.cf file and add the following lines:
smtp_destination_concurrency_limit = 2
smtp_destination_rate_delay = 1s
smtp_extra_recipient_limit = 10
Lets go over each line:
default_destination_concurrency_limit = 2
This means that postfix will up to two concurrent connections per receiving domains.
The default value is 20.
default_destination_rate_delay = 1s
Postfix will add a delay between each message to the same receiving domain. It overrides the previous rule and in this example, it will send one email after another with a delay of 1 second. If you want to disable this rule, either delete it or set to 0.
default_extra_recipient_limit = 10
Limit the number of recipients of each message. If a message had 20 recipients on the same domain, postfix will break it out to two different email messages instead of one.
Restart your postfix service and you are done.
If you are sending thousands of emails, and need to do so fairly quickly you will learn that this configuration will delay your rate considerably. Once you gain a better reputation, you will be able to move to the next step.
Different policies for different domains
Now, as you fine tune your policy, it may be apparent that some providers allow you to send emails at a higher rate, and some may need more delay between each message.
Lets create separate policies so we can optimise our server’s performance.
The first step will be to edit the master.cf file and add SMTP transports. Each transport will represent a policy for a group of receiving domain:
polite unix - - n - - smtp
turtle unix - - n - - smtp
Next, we need to map a domain to it’s transport name:
Edit the /etc/postfix/transport file and add the lines
Now we can add the policy to the
transport_maps = hash:/etc/postfix/transport
polite_destination_concurrency_limit = 2
polite_destination_rate_delay = 0
polite_destination_recipient_limit = 5
turtle_destination_concurrency_limit = 1
turtle_destination_rate_delay = 3s
turtle_destination_recipient_limit = 2
Note that these values are just an example and you will have to tweak it yourself over time.
To activate these changes, we need to reload the postfix configuration:
service postfix restart
Read the ISP’s guidelines for bulk senders to better understand how to improve your reputation:
- Gmail – Best practices for bulk mail senders
- Yahoo – Bulk Senders Guidelines
- AOL – Bulk Sender Best Practices
- Hotmail – Troubleshooting
- A good reference book that I use – Postfix: The Definitive Guide
If you have any questions you can find me on IRC – #steamio on Freenode