Mail Server: Ubuntu, Postfix, Dovecot, MySQL, SpamAssassin, and SquirrelMail

This post details how to set up your own mail server using Postfix and Dovecot with virtual users and domains on Ubuntu 12.04. Eventually we’ll add a webmail client (we’ll use SquirrelMail) and an anti-spam solution (SpamAssassin).

This assumes you have MySQL running and understand how to configure your local firewall (probably with ufw on Ubuntu). This also assumes you have your domains forwarding to your server and that your /etc/hostname file is set to mydomain.com. Some understanding of SQL and MySQL administration is useful also.

Step 0: Install everything.

$ apt-get install postfix postfix-mysql dovecot-core dovecot-imapd dovecot-lmtpd dovecot-mysql

Include dovecot-pop3d if you want POP email; I usually don’t bother.

Step 1: Create a database and tables to store your mail server configuration information.

$ mysql -uroot -p 
mysql> CREATE DATABASE mailserver DEFAULT CHARACTER SET = 'utf8'; GRANT SELECT ON mailserver.* TO '(mail_username)'@'127.0.0.1' IDENTIFIED BY '(password)'; FLUSH PRIVILEGES;
mysql> USE mailserver;

Now, create tables for your virtual domains and users, and virtual aliases if you want forwarding. (For some further discussion of this, see the comments below.)

mysql> CREATE TABLE `virtual_domains` (
      `id` int(11) NOT NULL auto_increment,
      `name` varchar(50) NOT NULL,
      PRIMARY KEY (`id`)
    );
mysql> CREATE TABLE `virtual_users` (
      `id` int(11) NOT NULL auto_increment,
      `domain_id` int(11) NOT NULL,
      `password` varchar(106) NOT NULL,
      `email` varchar(100) NOT NULL,
      PRIMARY KEY (`id`),
      UNIQUE KEY `email` (`email`),
      FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE
    );
mysql> CREATE TABLE `virtual_aliases` (
      `id` int(11) NOT NULL auto_increment,
      `domain_id` int(11) NOT NULL,
      `source` varchar(100) NOT NULL,
      `destination` varchar(100) NOT NULL,
      PRIMARY KEY (`id`),
      FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE
    );

Step 2: Add some email addresses and domains.

mysql> INSERT INTO `mailserver`.`virtual_domains`
      (`id` ,`name`)
    VALUES
      ('1', 'mydomain.com'),
      ('2', 'subdomain.mydomain.com');
mysql> INSERT INTO `mailserver`.`virtual_users`
      (`id`, `domain_id`, `password` , `email`)
    VALUES
      ('1', '1', ENCRYPT('firstpassword', CONCAT('$6$', SUBSTRING(SHA(RAND()), -16))), 'email1@mydomain.com'),
      ('2', '1', ENCRYPT('secondpassword', CONCAT('$6$', SUBSTRING(SHA(RAND()), -16))), 'email2@mydomain.com');
mysql> INSERT INTO `mailserver`.`virtual_aliases`
      (`id`, `domain_id`, `source`, `destination`)
    VALUES
      ('1', '1', 'alias@domain1.com', 'email1@mydomain.com');

Step 3: Set up Postfix to handle incoming email.

Edit /etc/postfix/main.cf (my comments are included with the ## symbol):

# See /usr/share/postfix/main.cf.dist for a commented, more complete version

# Debian specific:  Specifying a file name will cause the first
# line of that file to be used as the name.  The Debian default
# is /etc/mailname.
#myorigin = /etc/mailname

smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
biff = no

# appending .domain is the MUA's job.
append_dot_mydomain = no

# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h

readme_directory = no

# TLS parameters

## We're commenting out the default system self-signed cert and using the free SSL cert included with Dovecot. We're also requiring TLS encryption for users to connect. If you bought an SSL certificate, substitute it for the Dovecot entries in smtpd_tls_cert_file and smtpd_tls_auth_only.

#smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
#smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
#smtpd_use_tls=yes
#smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
#smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

smtpd_tls_cert_file=/etc/ssl/certs/dovecot.pem
smtpd_tls_key_file=/etc/ssl/private/dovecot.pem
smtpd_use_tls=yes
smtpd_tls_auth_only = yes

## Allow authenticated users to send email, and use Dovecot to authenticate them. Tells Postfix to use Dovecot for authentication. 
smtpd_sasl_type = dovecot
## Path to the Postfix auth socket, relative to /var/spool/postfix/. 
smtpd_sasl_path = private/auth
## Tells Postfix to let people send email if they've authenticated to the server. Otherwise they can only send if they're logged in (SSH).
smtpd_sasl_auth_enable = yes
## Tells Postfix who can send email: SASL-authenticated users connecting from a network specified in 'mynetworks' below. Also prevents anybody from sending mail to users who aren't on the server.
smtpd_recipient_restrictions =
        permit_sasl_authenticated,
        permit_mynetworks,
        reject_unauth_destination

# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
# information on enabling SSL in the smtp client.

myhostname = hostname.mydomain.com
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname

## Comment out the mydestination value and put in 'localhost'. This allows you to use the virtual domains in MySQL for mail delivery. If there's overlap between the virtual domains and mydestination you'll see warnings in the mail log.
#mydestination = mydomain.com, hostname.mydomain.com, localhost.mydomain.com, localhost
mydestination = localhost

relayhost =

## Mynetworks is important. If it isn't set to localhost, anybody may be able to use your server to send spam (i.e., an open relay). In other words, there are two situations where a client doesn't have to authenticate to send email through your server: 1) They send to a recipient who has an account on the server, 2) They send from a client whose IP is listed in mynetworks in /etc/postfix/main.cf. 
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128

mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all

## Tells Postfix to use Dovecot's LMTP instead of its own LDA to save emails to the local mailboxes.
virtual_transport = lmtp:unix:private/dovecot-lmtp

## Tells Postfix you're using MySQL to store virtual domains, and gives the paths to the database connections. 
virtual_mailbox_domains = mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf
virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf
virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-alias-maps.cf

Now, create the three files we specified in the last part of main.cf.

/etc/postfix/mysql-virtual-mailbox-domains.cf:

user = mail_username
password = mail_password
hosts = 127.0.0.1
dbname = mailserver
query = SELECT 1 FROM virtual_domains WHERE name='%s'

/etc/postfix/mysql-virtual-mailbox-maps.cf:

user = mail_username
password = mail_password
hosts = 127.0.0.1
dbname = mailserver
query = SELECT 1 FROM virtual_users WHERE email='%s'

/etc/postfix/mysql-virtual-alias-maps.cf:

user = mail_username
password = mail_password
hosts = 127.0.0.1
dbname = mailserver
query = SELECT destination FROM virtual_aliases WHERE source='%s'

Restart postfix and test your connections:

$ service postfix restart
$ postmap -q mydomain.com mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf
$ postmap -q myemail@mydomain.com mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf
$ postmap -q myalias@mydomain.com mysql:/etc/postfix/mysql-virtual-alias-maps.cf

If the first two postmap commands return anything but 1, and the third one returns anything but the destination email address, there’s a problem.

Note: You should ensure that port TCP/25 is open on your firewall so your server can receive email from the wider internet. You might also want to allow SMTP on port TCP/587 for users to connect securely with email clients. If so, modify your /etc/postfix/master.cf file by uncommenting these lines:

submission inet n       -       -       -       -       smtpd
  -o syslog_name=postfix/submission
  -o smtpd_tls_security_level=encrypt
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_client_restrictions=permit_sasl_authenticated,reject

Step 4: Set up the Dovecot mail delivery agent.

Edit /etc/dovecot/dovecot.conf:

## Add pop3 here if you want that too.
# Enable installed protocols
!include_try /usr/share/dovecot/protocols.d/*.protocol
protocols = imap lmtp

Also verify that dovecot.conf is including all the other configuration files:

!include conf.d/*.conf

Edit /etc/dovecot/conf.d/10-mail.conf:

## Tells Dovecot where to find mail in maildir format, in this case in /var/mail/vhosts/yourdomain.com/email@yourdomain.com.
mail_location = maildir:/var/mail/vhosts/%d/%n

## Tells Dovcot to write to the /var/mail folder.
mail_privileged_group = mail

Verify permissions, and create the virtual mail directory and user/group:

$ ls -ld /var/mail
drwxrwsr-x 2 root mail 4096 Apr  12 22:14 /var/mail

$ mkdir -p /var/mail/vhosts/example.com
$ groupadd -g 5000 vmail
$ useradd -g vmail -u 5000 vmail -d /var/mail
$ chown -R vmail:vmail /var/mail

Edit /etc/dovecot/conf.d/10-auth.conf:

## Uncomment this line
disable_plaintext_auth = yes
## Modify this line
auth_mechanisms = plain login
## Comment out this line:
#!include auth-system.conf.ext
## Uncomment the MySQL auth line
#!include auth-system.conf.ext
!include auth-sql.conf.ext
#!include auth-ldap.conf.ext
#!include auth-passwdfile.conf.ext
#!include auth-checkpassword.conf.ext
#!include auth-vpopmail.conf.ext
#!include auth-static.conf.ext

Create /etc/dovecot/conf.d/auth-sql.conf.ext to hold your authentication info:

passdb {
  driver = sql
  args = /etc/dovecot/dovecot-sql.conf.ext
}
userdb {
  driver = static
  args = uid=vmail gid=vmail home=/var/mail/vhosts/%d/%n
}

Edit /etc/dovecot/dovecot-sql.conf.ext:

## Uncomment and specify the driver
driver = mysql
## Uncomment and set connection information
connect = host=127.0.0.1 dbname=mailserver user=mail_username password=mail_password
## Uncomment and set the encryption format for passwords
default_pass_scheme = SHA512-CRYPT
## Uncomment and set password query
password_query = SELECT email as user, password FROM virtual_users WHERE email='%u';

Set permissions on the /etc/dovecot directory so the vmail user can use it:

chown -R vmail:dovecot /etc/dovecot
chmod -R o-rwx /etc/dovecot

Edit /etc/dovecot/conf.d/10-master.conf:

## Disable unencrypted IMAP by setting the port to 0
service imap-login {
  inet_listener imap {
    port = 0
  }
...
}
## (Leave imaps-login alone)

## Create LMTP socket for Dovecot where we told Postfix to find it
service lmtp {
 unix_listener /var/spool/postfix/private/dovecot-lmtp {
   mode = 0600
   user = postfix
   group = postfix
  }
  # Create inet listener only if you can't use the above UNIX socket
  #inet_listener lmtp {
    # Avoid making LMTP visible for the entire internet
    #address =
    #port =
  #}
}

## Create authorization socket where we told Postfix to find it 
service auth {
  # auth_socket_path points to this userdb socket by default. It's typically
  # used by dovecot-lda, doveadm, possibly imap process, etc. Its default
  # permissions make it readable only by root, but you may need to relax these
  # permissions. Users that have access to this socket are able to get a list
  # of all usernames and get results of everyone's userdb lookups.
  unix_listener /var/spool/postfix/private/auth {
    mode = 0666
    user = postfix
    group = postfix
  }

  unix_listener auth-userdb {
    mode = 0600
    user = vmail
    #group =
  }

  # Postfix smtp-auth
  #unix_listener /var/spool/postfix/private/auth {
  #  mode = 0666
  #}

  # Auth process is run as this user.
  user = dovecot
}

## Set the auth-worker user to vmail
service auth-worker {
  # Auth worker process is run as root by default, so that it can access
  # /etc/shadow. If this isn't necessary, the user should be changed to
  # $default_internal_user.
  user = vmail
}

Edit /etc/dovecot/conf.d/10-ssl.conf:

## If you have your own SSL cert and key, specify them here. We're using the free ones that come with Dovecot.
ssl_cert = </etc/ssl/certs/dovecot.pem
ssl_key = </etc/ssl/private/dovecot.pem
## Force clients to use SSL
ssl = required

Restart Dovecot:

$ service dovecot restart

Note: Make sure that port TCP/993 is open on your server’s firewall for IMAP. If you added POP, open port TCP/995.

Now you should be good to go. Set up a test account in your email client to ensure everything works:

Username: myemail@mydomain.com
Password: Whatever you added to the MySQL table in step 2 for this email address
IMAP server: yourdomain.com
SMTP server: yourdomain.com
Ports: 993 for secure IMAP, 25 or 587 for SMTP

Try sending mail and see what happens. If it didn’t work, check the log in /var/log/mail.log. You can also turn on verbose logging for Dovecot by adding a few lines to /etc/dovecot.conf:

## Verbose logging
auth_debug_passwords=yes
mail_debug=yes
auth_verbose=yes
verbose_ssl=yes
auth_verbose_passwords=plain

Once things are working, you should probably comment these out again so they don’t clog up your mail log.

Finally, to add users/domains/forwards to your mail server, just add them to the appropriate tables in MySQL like in step 2. That’s all you need to do since authentication and incoming mail query the database to determine validity. For new accounts, the appropriate folders are created in /var/mail/ when each account first receives a message.

In our next installment, we’ll set up an anti-spam system.

10 thoughts on “Mail Server: Ubuntu, Postfix, Dovecot, MySQL, SpamAssassin, and SquirrelMail

    • No problem! Looks like you have a good writeup there as well. I really need to set up Squid here at some point also.

  1. Thanks for all the work you’ve put into this. Unfortunately, I’ve tried it twice, with a apt-get purge in between, and continue to have a SASL issue. The first time through, I tried to follow all the online suggestions, only to bastardize my conf files, thus the purge. Do you have any suggestions? The log entries are;

    warning: SASL: Connect to private/auth failed: No such file or directory
    fatal: no SASL authentication mechanisms
    warning: process /usr/lib/postfix/smtpd pid 12233 exit status 1
    warning: /usr/lib/postfix/smtpd: bad command startup — throttling

    whenever I try to connect from OS X mail client.

    Thanks in advance.

    • No problem. The private/auth connection it’s referring to is a Unix socket at /var/spool/postfix/private/auth so I’d make sure that exists and has the right owner/group and permissions (postfix:postfix, 0666). There should be other sockets in that directory (retry, relay, maildrop, etc.) all with the same permissions. The parent directory /var/spool/postfix should also have various directories and files in it, some owned by root, so if any of these locations appear empty or missing I’d do a clean reinstall of Postfix.

      I’d also make sure your /etc/dovecot/conf.d/10-master.conf file is correct, since it designates the private/auth socket for authentication.

  2. Great tutorial!

    Instead on manually manipulating tables I use a command line tools custom made for this type of installation.

    Google the following to find it:
    # beas.sh – Bash Email Admin Script
    # Version 0.2 (Lenny edition)
    # (c) 2009 Emmanuel Revah – manu-at-manurevah.com

    • That certainly works, although oftentimes I just use a visual tool (MySQL Workbench, SQL Explorer for Eclipse, etc.) over SSH for adding users. You can save your queries in there so you don’t have to copy+paste them into the console every time.

  3. Excellent tutorial, worked almost perfectly :)

    Upon testing dovecot I was getting login errors, turning on the logging showed:

    Jan 5 13:01:00 mail dovecot: auth-worker: Error: md5_verify(yourEmail): Not a valid MD5-CRYPT or PLAIN-MD5 password
    Jan 5 13:01:00 mail dovecot: auth-worker: Warning: Invalid OTP data in passdb
    Jan 5 13:01:00 mail dovecot: auth-worker: Warning: Invalid OTP data in passdb
    Jan 5 13:01:00 mail dovecot: auth-worker: Debug: sql(yourEmail,127.0.0.1): SHA512-CRYPT(yourPassword) != ‘aLongHashString’

    The MySQL statement at the top of the tutorial creates a ‘password’ varchar column not quite long enough resulting in a truncated hash…. failed login everytime.

    When creating your MySQL tables a little modification is needed
    CREATE TABLE `virtual_users` […] `password` varchar(100) NOT NULL, […]

    CREATE TABLE `virtual_users` […] `password` varchar(128) NOT NULL, […]

    (If already created just run : ALTER TABLE virtual_users MODIFY password varchar(128) NOT NULL;)

    After altering the tables (and re-inserting the previously truncated password to restore it) was then able to login to dovecot.

    • Good catch, it actually needs to be varchar(106) to work on Linux systems. I use this all the time in my setups but my documentation here had a typo. You might also prefer to use char(106), since the password will always be fixed at 106 characters if you use the SQL statement listed above to create new users.

      The way you arrive at 106 characters is: 86-character encrypted password + 4 separator characters + 16-character salt.

      As for why the password is always 86 characters, the full ENCRYPT(‘firstpassword’, CONCAT(‘$6$’, SUBSTRING(SHA(RAND()), -16))) statement tells MySQL to:

      a) Generate a random floating point value between 0 and 1.0
      b) Calculate an SHA1 checksum for the random value, expressed as a string of 40 hexadecimal digits, e.g., ‘a9993e364706816aba3e25717850c26c9cd0d89d’
      c) Select a substring of (b), starting with the character in the 25th position and running to position 40 (16 total characters)
      d) Concatenate ‘$6$’ and your substring from (c), so you end up with a value like ‘$6$7850c26c9cd0d89d’
      e) Encrypt ‘firstpassword’ into a binary string using your concatenated randomized value from (d) as the salt. Per the MySQL documentation, ENCRYPT() relies on the crypt() Unix system call, so results will vary by platform, but on current Linux systems this gives you a string like ‘$6$7850c26c9cd0d89d$encrypted-password’. On Linux systems, the ‘$6$’ appended to the salt also tells crypt() to use SHA-512 encryption, giving you an 86-character encrypted string (see the crypt man page at http://man7.org/linux/man-pages/man3/crypt.3.html). So now you have ‘$6$’ + 16-character salt + ‘$’ + 86-character encrypted password = 106 characters.
      f) Return the final result to the INSERT statement.

  4. Hi, Thanks for the write up, very detailed.

    I’m getting the following error in my /var/log/mail.err log, and I’m pretty sure I’ve gone through everything:

    dovecot: auth: Fatal: master: service(auth): child 19059 returned error 89 (Fatal failure)
    dovecot: master: Error: service(auth): command startup failed, throttling for 2 secs
    dovecot: auth: Error: Can’t open configuration file uid=vmail gid=vmail home=/var/mail/vhosts/%d/%n: No such file or directory

    Other messages for “Can’t open configuration file” seem to point to an actual file, however I’ve only changed the ‘/etc/dovecot/conf.d/auth-sql.conf.ext’ file to contain this value, in the args section:

    passdb {
    driver = sql

    # Path for SQL configuration file, see example-config/dovecot-sql.conf.ext
    args = /etc/dovecot/dovecot-sql.conf.ext
    }

    # “prefetch” user database means that the passdb already provided the
    # needed information and there’s no need to do a separate userdb lookup.
    #
    #userdb {
    # driver = prefetch
    #}

    userdb {
    driver = sql
    args = uid=vmail gid=vmail home=/var/mail/vhosts/%d/%n
    }

    Do you have any ideas why this could be happening?
    Cheers,
    Justin

Leave a Reply

Your email address will not be published. Required fields are marked *