Mail server HOWTO – Postfix and Dovecot with MySQL and TLS/SSL, Postgrey and DSPAM

This guide describes how to setup a mail server where Postfix is the SMTP service, and Dovecot provides IMAP and POP services. The users may be virtual and stored in a MySQL table, or real system users, or both. The primary focus for this guide however, is a fast and secure mail server using virtual users.

All this may be installed in either Debian 4.0 Etch or Ubuntu Feisty Fawn, since both systems are quite similar. Note however that there may be some minor issues if you use the default version of Dovecot, but I will try to note them down for you when they arise.

If you are a Ubuntu user, note that I will not use "sudo" in front of every command. Instead, I will launch a root shell using the command "sudo -s". Note that ticks like ' and " are converted by WordPress, and may cause problems if copied directly into your configuration.

MySQL and Postfix admin

Installing software in Ubuntu and Debian is very easy, and to get the MySQL server and client installed, just execute the following.

# apt-get install mysql-server mysql-client

Step one is to set a password for the administrative user, which can be done with the following statement. See the MySQL documentation for more information.

# mysqladmin -u root password "newpwd"

Of couse, replace newpwd with your intended password. While you are at it, add a user mail with password mail, with access to database mail. You can of course change the password to something else if you wish, and you may give more restrictive permissions if you know what you are doing. Postfix and Dovecot will only SELECT from the database, while postfix admin will SELECT, UPDATE and INSERT into the database. You may even create two separate accounts for this. This guide however will use the same account.

# mysql -uroot -p
mysql> CREATE DATABASE mail;
mysql> GRANT ALL PRIVILEGES ON mail.* TO 'mail'@'localhost' IDENTIFIED BY 'mail';
mysql> quit</code>

Now we need to download Postfix admin, since its distribution contains the required SQL schema which we want to use.

Once you have downloaded the Postfix admin distribution, you should have a tarball, and unpack it like this.

# tar xvfz postfixadmin-2.1.0.tgz

Open postfixadmin-2.1.0/DATABASE_MYSQL.TXT with your favorite editor such as vim, nano or gedit and comment out or remove all lines under section "Postfix / MySQL", since we have created our own use for the mail server. The section is currently lines 26 to 39.

When this is done, just load the file into mysql using the following command.

# mysql -umail -p mail < postfixadmin-2.1.0/DATABASE_MYSQL.TXT

If you get an error saying "Access denied for user 'mail'@'localhost' to database 'mysql'", you didn't comment out the lines like I told you. If you get no message after entering your password, everything went fine.

Postfix

Since we are using a Debian based system, installing Postfix is a walk in the park.

# apt-get install postfix postfix-mysql

If you get questions about Postfix during the installation, just select "No configuration". You have this guide, which will configure it for you instead!

The Postfix configuration directory is located in /etc/postfix, so head over there and open the file main.cf. Don't worry if it does not exist. We will make it!

Start with entering the following basic information.

smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
biff = no
append_dot_mydomain = no
#delay_warning_time = 4h
myhostname = mail.mycompany.com
myorigin = mycompany.com
mydestination = localhost
relayhost =
mynetworks = 127.0.0.0/8
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all

This is actually all you need to get a fully functional mail server with standard unix users. Remember however to change myhostname and myorigin to your own domain settings.

Virtual users

Now for adding support for virtual users in MySQL, also append the following lines to the main.cf configuration file.

# Virtual mailbox settings
virtual_mailbox_domains = proxy:mysql:$config_directory/mysql_virtual_domains_maps.cf
virtual_mailbox_base = /var/vmail
virtual_mailbox_maps = proxy:mysql:$config_directory/mysql_virtual_mailbox_maps.cf
virtual_alias_maps = proxy:mysql:$config_directory/mysql_virtual_alias_maps.cf
virtual_minimum_uid = 150
virtual_uid_maps = static:150
virtual_gid_maps = static:8
virtual_transport = dovecot
dovecot_destination_recipient_limit = 1
# SASL Authentication
smtpd_sasl_auth_enable = yes
smtpd_sasl_exceptions_networks = $mynetworks
smtpd_sasl_security_options = noanonymous
broken_sasl_auth_clients = yes
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth

There are several gotchas here. This configuration will host the virtual user mailboxes in /var/vmail. Should these mailboxes be hosted somewhere else, be sure to change that as appropriate.

The virtual_minimum_uid and virtual_uid_maps point to user id 150 in my case, which is a user I created specifically for handling virtual mail. It uses the standard "mail" group with the default gid 8 (in Debian and Ubuntu). Create the user and directories like this:

# useradd -r -u 150 -g mail -d /var/vmail -s /sbin/nologin -c "Virtual mailbox" vmail
# mkdir /var/vmail
# chmod 770 /var/vmail/
# chown vmail:mail /var/vmail/

Select which clients to permit

We also need to specify some rules, which will enable authenticated users to send mail, but not anyone. Being an open relay is absolutely forbidden and will get you blacklisted!

smtpd_recipient_restrictions =
  permit_mynetworks
  permit_sasl_authenticated
  reject_unauth_destination
  permit

For antispam measures and enabling TLS, see further down in this document.

Postfix MySQL configuration

Postfix needs to know where and how it can lookup all mailbox related information, so it needs to be provided with all MySQL files defined in the main.cf configuration. Note that the last line contains a comment line with the full query. Recent versions of Postfix may use that instead of the other statements, and in that case, just comment all lines out, and uncomment the last one.

/etc/postfix/mysql_virtual_alias_maps.cf

user = mail
password = mail
hosts = localhost
dbname = mail
table = alias
select_field = goto
where_field = address
additional_conditions = and active = '1'
#query = SELECT goto FROM alias WHERE address='%s' AND active = '1'

/etc/postfix/mysql_virtual_domains_maps.cf

user = mail
password = mail
hosts = localhost
dbname = mail
table = domain
select_field = domain
where_field = domain
additional_conditions = and backupmx = '0' and active = '1'
#query = SELECT domain FROM domain WHERE domain='%s' AND backupmx = '0' AND active = '1'

/etc/postfix/mysql_virtual_mailbox_limit_maps.cf

user = mail
password = mail
hosts = localhost
dbname = mail
table = mailbox
select_field = quota
where_field = username
additional_conditions = and active = '1'
#query = SELECT quota FROM mailbox WHERE username='%s' AND active = '1'

/etc/postfix/mysql_virtual_mailbox_maps.cf

user = mail
password = mail
hosts = localhost
dbname = mail
table = mailbox
select_field = CONCAT(domain,'/',maildir)
where_field = username
additional_conditions = and active = '1'
#query = SELECT CONCAT(domain,'/',maildir) FROM mailbox WHERE username='%s' AND active = '1'

Dovecot delivery for Postfix

Postfix will hand the mail over to Dovecot for local delivery, and to set this up, you need to open the file /etc/postfix/master.cf and add the following like at the bottom.

dovecot unix - n n - - pipe flags=DRhu user=vmail:mail argv=/usr/lib/dovecot/deliver -d $(recipient)

Dovecot IMAP and POP

The first step is to get the packages installed, and it's simply a matter of using apt-get like before.

apt-get install dovecot-imapd dovecot-pop3d

You can of course omit the dovecot-pop3d package, if you are not planning on using POP, and why should you, when IMAP provides such wonderful extensions such as IDLE?

Open up the Dovecot configuration file located in /etc/dovecot/dovecot.conf and make it look the following. There is probably already lots of default configuration done, so you will probably only need to uncomment certain sections and change minor things.

## Dovecot configuration file
#
base_dir = /var/run/dovecot/
#
# imap imaps pop3 pop3s (use imaps and pop3s if configured for SSL)
protocols = imap pop3
#
# Uncomment the ssl_listen statements and comment out listen if using SSL
protocol imap {
  listen = *:143
  # ssl_listen = *:993
}
protocol pop3 {
  listen = *:110
  # ssl_listen = *:995
}
#
log_timestamp = "%Y-%m-%d %H:%M:%S "
syslog_facility = mail
#
# Uncomment these if using SSL
#ssl_cert_file = /etc/ssl/mycompany/mailserver/mail-cert.pem
#ssl_key_file = /etc/ssl/mycompany/mailserver/mail-key.pem
#ssl_ca_file = /etc/ssl/mycompany/ca/mycompany.pem
#ssl_verify_client_cert = yes
#ssl_parameters_regenerate = 168
#verbose_ssl = no
#
# Where the mailboxes are located
mail_location = maildir:/var/vmail/%d/%u
#
mail_extra_groups = mail
mail_debug = no
first_valid_uid = 150
last_valid_uid = 150
maildir_copy_with_hardlinks = yes
#
protocol imap {
  login_executable = /usr/lib/dovecot/imap-login
  mail_executable = /usr/lib/dovecot/imap
  imap_max_line_length = 65536
}
protocol pop3 {
  login_executable = /usr/lib/dovecot/pop3-login
  mail_executable = /usr/lib/dovecot/pop3
  pop3_uidl_format = %08Xu%08Xv
}
protocol lda {
  postmaster_address = [email protected]
  sendmail_path = /usr/lib/sendmail
  auth_socket_path = /var/run/dovecot/auth-master
}
#
auth_verbose = no
auth_debug = no
auth_debug_passwords = no
#
auth default {
  mechanisms = plain
  passdb sql {
    args = /etc/dovecot/dovecot-sql.conf
  }
  userdb sql {
    args = /etc/dovecot/dovecot-sql.conf
  }
  user = nobody
  socket listen {
    master {
      path = /var/run/dovecot/auth-master
      mode = 0660
      user = vmail
      group = mail
    }
    client {
      path = /var/spool/postfix/private/auth
      mode = 0660
      user = postfix
      group = postfix
    }
  }
  #
  # If you want client certificates, use these lines
  # ssl_require_client_cert = yes
  # ssl_username_from_cert = yes
}

Yes, the indentation was lost, but don't fear. Just incorporate my changes to the existing file. The Dovecot configuration is almost finished now, but we need to add definitions on how it can find our password and user database stored in MySQL.

Open the file /etc/dovecot/dovecot-sql.conf and make sure that the following is present.

driver = mysql
connect = host=localhost dbname=mail user=mail password=mail
# The new name for MD5 is MD5-CRYPT so you might need to change this depending on version
default_pass_scheme = MD5
# Get the mailbox
user_query = SELECT '/var/vmail/%d/%n' as home, 'maildir:/var/vmail/%d/%n' as mail, 150 AS uid, 8 AS gid, concat('dirsize:storage=', quota) AS quota FROM mailbox WHERE username = '%u' AND active = '1'
# Get the password
password_query = SELECT username as user, password, '/var/vmail/%d/%n' as userdb_home, 'maildir:/var/vmail/%d/%n' as userdb_mail, 150 as userdb_uid, 8 as userdb_gid FROM mailbox WHERE username = '%u' AND active = '1'
# If using client certificates for authentication, comment the above and uncomment the following
#password_query = SELECT null AS password, '%u' AS user

Set the permissions for Dovecot.

# chmod 600 /etc/dovecot/*.conf
# chown vmail /etc/dovecot/*.conf

The Dovecot configuration is now finished, but all SSL-specific parameters have been disabled for now.

Postfix admin

Since Postfix admin requires a running web server, Apache and PHP needs to be installed first. As always, apt-get comes to the rescure, and we need to restart Apache after the installation of php5-mysql.

# apt-get install apache2 libapache2-mod-php5 php5-mysql
# invoke-rc.d apache2 restart

To make things easy, just move the postfixadmin directory downloaded earlier to /var/www/ and copy the config file to its real name.

# mv postfixadmin-2.1.0 /var/www/postfixadmin
# cd /var/www/postfixadmin/
# cp config.inc.php.sample config.inc.php

There are some directives you will need to change, and they are quite self-explanatory, but here are the highlights.

$CONF['postfix_admin_url'] = '/var/www/postfixadmin/';
$CONF['postfix_admin_path'] = 'http://localhost/postfixadmin';
$CONF['database_type'] = 'mysqli';
$CONF['database_user'] = 'mail';
$CONF['database_password'] = 'mail';
$CONF['database_name'] = 'mail';

Do some tests to see if everything works

The server should now be fully up and running, so go to http://localhost/postfixadmin/admin/ and create a new domain and user. If you then look at the log file /var/log/syslog, you should see something like this.

postfix/smtpd[1819]: connect from localhost[127.0.0.1]
postfix/smtpd[1819]: 1A0DF66886: client=localhost[127.0.0.1]
postfix/cleanup[1824]: 1A0DF66886: message-id=<[email protected]>
postfix/qmgr[1067]: 1A0DF66886: from= , size=408, nrcpt=1 (queue active)
postfix/smtpd[1819]: disconnect from localhost[127.0.0.1]
deliver([email protected]): msgid=<[email protected]>: saved mail to INBOX
postfix/pipe[1825]: 1A0DF66886: to=, relay=dovecot, delay=0.09, delays=0.06/0.01/0/0.02, dsn=2.0.0, status=sent (delivered via dovecot service)
postfix/qmgr[1067]: 1A0DF66886: removed

If you received an error, extract the error message and go from there.

TLS and SSL for Postfix and Dovecot

Before you push this server out into production, you will most definitely want to enable encryption. You can have different levels of security, and here are the most common scenarios.

The simplest form is having a simple self-signed certificate on the server. This will generate a warning message when the clients first connect, but they should be able to save it for further use. It is not really secure, since anyone can execute a man-in-the-middle attack if you don't save the certificate.

The next level is using a server certificate signed by a Certificate Authority (CA), either a commercial one, or perhaps the company internal CA. This way, the server certificate will be trusted, and if you now receive a warning, there is potentially something bad going on.

Last but definitely not least is using client certificates for logging in to the server, and using a server certificate to authenticate the server to the clients. This is quite secure, but it is not supported in all mail clients. Thunderbird among others do have support for it.

Self-signed server certificate

First create the directories, create the private key, and lastly create the certificate.

# mkdir -p /etc/ssl/mycompany/mailserver/
# cd /etc/ssl/mycompany/mailserver/
# openssl genrsa 1024 > mail-key.pem
# chmod 400 mail-key.pem
# openssl req -new -x509 -nodes -sha1 -days 365 -key mail-key.pem > mail-cert.pem</code>

Note that "Common Name (eg, YOUR name)" MUST match the name of the server, which in this case is mail.mycompany.com

Dovecot SSL configuration

When you have the certificate ready, it needs to be enabled in both Dovecot and Postfix. You will need to uncomment the following directives in dovecot.conf:

protocols
ssl_listen
ssl_cert_file
ssl_key_file
ssl_parameters_regenerate = 168
verbose_ssl = no

Dovecot should now be accepting SSL-connections on port 993. Check the log-file for an entry like this when you login using IMAP.

dovecot: imap-login: Login: user=, method=PLAIN, rip=127.0.1.1, lip=127.0.1.1, TLS

The keyword is TLS to the right, which shows that you are using a TLS encrypted session.

Postfix TLS configuration

You will need to add a few lines to your main.cf configuration file to enable TLS.

smtpd_tls_cert_file = /etc/ssl/mycompany/mailserver/mail-cert.pem
smtpd_tls_key_file = /etc/ssl/mycompany/mailserver/mail-key.pem
smtpd_tls_session_cache_database = btree:/var/spool/postfix/smtpd_tls_session_cache
smtpd_tls_security_level = may
smtpd_tls_received_header = no
smtpd_tls_loglevel = 0
tls_random_source = dev:/dev/urandom

Now you should be able to authenticate with Postfix and send mail anywhere.

CA-signed certificate

Using a real CA-signed certificate is no different from using a self-signed one. It's just another step in the key-pair creation. If your company has its own CA, then they should issue a certificate for the mail server. A Google search for be your own ca will give you enough answers to create one yourself, if you have the need.

CA-signed client and server certificates

If you want to use CA-signed client certificates, you will need to take further steps, both in Postfix and in Dovecot to make this work. If you want the user names to be taken from the certificate itself, you currently must set the common name to the user name, for example [email protected], which has been used in this document.

In Postfix, you can either use a directory of CA certificates, or a composite file with all the certificates concatenated together.

smtpd_tls_CAfile = /etc/ssl/mycompany/ca/all.pem
#smtpd_tls_CApath = /etc/ssl/mycompany/ca/

In Dovecot, you must have the CRL together with the certificate for the authentication to work. The directives themselves are the following.

ssl_ca_file = /etc/ssl/mycompany/ca/all.pem
ssl_verify_client_cert = yes
ssl_require_client_cert = yes
ssl_username_from_cert = yes

You will also need to change the password_query to the commented one in /etc/dovecot/dovecot-sql.conf. If you are running Dovecot release candidate 28 or older, the server will not send out the list of accepted CA names, which could make clients with multiple client certificates unable to connect. Please upgrade or install this patch.

If you have several CAs and CRLs, it could be difficult to concatenate them each time, so I have created a small script which will do that for you. Just stick it in your /etc/ssl/mycompany/ca/ directory and run it. It will create an all.pemwith all certificates and all CRLs.

make.sh:
#!/bin/bash
rm all.pem 2> /dev/null
cat *.pem *.crl > all.pem

Like I said before, there are some settings in Postfix that need to be changed as well, so open up main.cf and note the following.

smtp_tls_CAfile = /etc/ssl/mycompany/ca/all.pem
smtpd_tls_ask_ccert = yes
smtpd_tls_req_ccert = no
smtpd_recipient_restrictions =
  permit_mynetworks
  permit_tls_all_clientcerts
  reject_unauth_destination
  permit

Now you should have an enterprise ready email server with client certificates.

Anti-spam measures

Everyone hates spam, so you will need to take some measures to protect your users from it.

Postgrey

Greylisting is a fairly effective countermeasure against spam, so we of course want to enable it for Postfix. First of all, let's get Postgrey installed and as always, that is very easy in Debian and Ubuntu.

You can use other greylisting daemons which use MySQL for example if you like, but then you are on your own.

# apt-get install postgrey

Postgrey will be injected before Postfix handles over the delivery to Dovecot, so we need to add it in the very end of smtpd_recipient_restrictions in main.cf, just before the final "permit" rule.

check_policy_service inet:127.0.0.1:60000

Open up /etc/default/postgrey in your favourit editor and change the options line to the following. Yes, there should be two dashes, - -, but Wordpress makes them to one long.

POSTGREY_OPTS="--inet=127.0.0.1:60000 --delay=55"

Then restart postgrey and incoming mail will be delayed 55 seconds, and you will hopefully get a lot less spam!

invoke-rc.d postgrey restart

Postfix RBL and other rules

There are other rules you can add to Postfix which will reduce spam. Incorporate the following into your main.cfconfiguration.

smtpd_recipient_restrictions =
  permit_mynetworks
  permit_tls_all_clientcerts
  #permit_sasl_authenticated
  reject_non_fqdn_hostname
  reject_non_fqdn_sender
  reject_non_fqdn_recipient
  reject_unauth_destination
  reject_unauth_pipelining
  reject_invalid_hostname
  #reject_unknown_sender_domain
  #reject_unknown_hostname
  reject_rbl_client zen.spamhaus.org
  reject_rbl_client bl.spamcop.net
  reject_rbl_client cbl.abuseat.org
  reject_rbl_client dnsbl.njabl.org
  reject_rbl_client dnsbl.sorbs.net
  reject_rhsbl_sender dsn.rfc-ignorant.org
  check_policy_service inet:127.0.0.1:60000
  permit
#
smtpd_data_restrictions =
  reject_unauth_pipelining,
  reject_multi_recipient_bounce,
  permit

Note that I have commented out reject_unknown_hostname, because there are some legit mail servers out there that send the wrong host name. That rule does however catch lots of spam, so it's a call you will have to make.

You now have a reasonable protection against spam. There are other server-side filtering services which use the bayesian algorithm to catch most spam.

DSPAM

Sorry, to be added later.

Final remarks

This guide will aid you in setting up a large-scale mail server, but it is always important to understand that you should take care and explore all options before diving in and setting this up in a real environment. There are probably lots of things that can be improved in this setup, but I will leave that up to you.

This setup will scale rather well, as you can cleanly separate the different component onto separate machines to reduce the load.

To conclude, always pay close attention to the logs. I always tail syslog when doing these kinds of things and you should too.

# tail -f /var/log/syslog

The above command will give you a real-time log when things happen, so keep one terminal running with just this.

References and further reading