This post details how to set up your own mail server using Postfix and Dovecot with virtual users and domains on Ubuntu 12.04. Eventually we’ll add a webmail client (we’ll use SquirrelMail) and an anti-spam solution (SpamAssassin).
This assumes you have MySQL running and understand how to configure your local firewall (probably with ufw on Ubuntu). This also assumes you have your domains forwarding to your server and that your /etc/hostname file is set to mydomain.com. Some understanding of SQL and MySQL administration is useful also.
Step 0: Install everything.
$ apt-get install postfix postfix-mysql dovecot-core dovecot-imapd dovecot-lmtpd dovecot-mysql
Include dovecot-pop3d if you want POP email; I usually don’t bother.
Step 1: Create a database and tables to store your mail server configuration information.
$ mysql -uroot -p mysql> CREATE DATABASE mailserver DEFAULT CHARACTER SET = 'utf8'; GRANT SELECT ON mailserver.* TO '(mail_username)'@'127.0.0.1' IDENTIFIED BY '(password)'; FLUSH PRIVILEGES; mysql> USE mailserver;
Now, create tables for your virtual domains and users, and virtual aliases if you want forwarding. (For some further discussion of this, see the comments below.)
mysql> CREATE TABLE `virtual_domains` (
`id` int(11) NOT NULL auto_increment,
`name` varchar(50) NOT NULL,
PRIMARY KEY (`id`)
);
mysql> CREATE TABLE `virtual_users` (
`id` int(11) NOT NULL auto_increment,
`domain_id` int(11) NOT NULL,
`password` varchar(106) NOT NULL,
`email` varchar(100) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `email` (`email`),
FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE
);
mysql> CREATE TABLE `virtual_aliases` (
`id` int(11) NOT NULL auto_increment,
`domain_id` int(11) NOT NULL,
`source` varchar(100) NOT NULL,
`destination` varchar(100) NOT NULL,
PRIMARY KEY (`id`),
FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE
);
Step 2: Add some email addresses and domains.
mysql> INSERT INTO `mailserver`.`virtual_domains`
(`id` ,`name`)
VALUES
('1', 'mydomain.com'),
('2', 'subdomain.mydomain.com');
mysql> INSERT INTO `mailserver`.`virtual_users`
(`id`, `domain_id`, `password` , `email`)
VALUES
('1', '1', ENCRYPT('firstpassword', CONCAT('$6$', SUBSTRING(SHA(RAND()), -16))), 'email1@mydomain.com'),
('2', '1', ENCRYPT('secondpassword', CONCAT('$6$', SUBSTRING(SHA(RAND()), -16))), 'email2@mydomain.com');
mysql> INSERT INTO `mailserver`.`virtual_aliases`
(`id`, `domain_id`, `source`, `destination`)
VALUES
('1', '1', 'alias@domain1.com', 'email1@mydomain.com');
Step 3: Set up Postfix to handle incoming email.
Edit /etc/postfix/main.cf (my comments are included with the ## symbol):
# See /usr/share/postfix/main.cf.dist for a commented, more complete version
# Debian specific: Specifying a file name will cause the first
# line of that file to be used as the name. The Debian default
# is /etc/mailname.
#myorigin = /etc/mailname
smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
biff = no
# appending .domain is the MUA's job.
append_dot_mydomain = no
# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h
readme_directory = no
# TLS parameters
## We're commenting out the default system self-signed cert and using the free SSL cert included with Dovecot. We're also requiring TLS encryption for users to connect. If you bought an SSL certificate, substitute it for the Dovecot entries in smtpd_tls_cert_file and smtpd_tls_auth_only.
#smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
#smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
#smtpd_use_tls=yes
#smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
#smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtpd_tls_cert_file=/etc/ssl/certs/dovecot.pem
smtpd_tls_key_file=/etc/ssl/private/dovecot.pem
smtpd_use_tls=yes
smtpd_tls_auth_only = yes
## Allow authenticated users to send email, and use Dovecot to authenticate them. Tells Postfix to use Dovecot for authentication.
smtpd_sasl_type = dovecot
## Path to the Postfix auth socket, relative to /var/spool/postfix/.
smtpd_sasl_path = private/auth
## Tells Postfix to let people send email if they've authenticated to the server. Otherwise they can only send if they're logged in (SSH).
smtpd_sasl_auth_enable = yes
## Tells Postfix who can send email: SASL-authenticated users connecting from a network specified in 'mynetworks' below. Also prevents anybody from sending mail to users who aren't on the server.
smtpd_recipient_restrictions =
permit_sasl_authenticated,
permit_mynetworks,
reject_unauth_destination
# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
# information on enabling SSL in the smtp client.
myhostname = hostname.mydomain.com
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
## Comment out the mydestination value and put in 'localhost'. This allows you to use the virtual domains in MySQL for mail delivery. If there's overlap between the virtual domains and mydestination you'll see warnings in the mail log.
#mydestination = mydomain.com, hostname.mydomain.com, localhost.mydomain.com, localhost
mydestination = localhost
relayhost =
## Mynetworks is important. If it isn't set to localhost, anybody may be able to use your server to send spam (i.e., an open relay). In other words, there are two situations where a client doesn't have to authenticate to send email through your server: 1) They send to a recipient who has an account on the server, 2) They send from a client whose IP is listed in mynetworks in /etc/postfix/main.cf.
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
## Tells Postfix to use Dovecot's LMTP instead of its own LDA to save emails to the local mailboxes.
virtual_transport = lmtp:unix:private/dovecot-lmtp
## Tells Postfix you're using MySQL to store virtual domains, and gives the paths to the database connections.
virtual_mailbox_domains = mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf
virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf
virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-alias-maps.cf
Now, create the three files we specified in the last part of main.cf.
/etc/postfix/mysql-virtual-mailbox-domains.cf:
user = mail_username password = mail_password hosts = 127.0.0.1 dbname = mailserver query = SELECT 1 FROM virtual_domains WHERE name='%s'
/etc/postfix/mysql-virtual-mailbox-maps.cf:
user = mail_username password = mail_password hosts = 127.0.0.1 dbname = mailserver query = SELECT 1 FROM virtual_users WHERE email='%s'
/etc/postfix/mysql-virtual-alias-maps.cf:
user = mail_username password = mail_password hosts = 127.0.0.1 dbname = mailserver query = SELECT destination FROM virtual_aliases WHERE source='%s'
Restart postfix and test your connections:
$ service postfix restart $ postmap -q mydomain.com mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf $ postmap -q myemail@mydomain.com mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf $ postmap -q myalias@mydomain.com mysql:/etc/postfix/mysql-virtual-alias-maps.cf
If the first two postmap commands return anything but 1, and the third one returns anything but the destination email address, there’s a problem.
Note: You should ensure that port TCP/25 is open on your firewall so your server can receive email from the wider internet. You might also want to allow SMTP on port TCP/587 for users to connect securely with email clients. If so, modify your /etc/postfix/master.cf file by uncommenting these lines:
submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,reject
Step 4: Set up the Dovecot mail delivery agent.
Edit /etc/dovecot/dovecot.conf:
## Add pop3 here if you want that too. # Enable installed protocols !include_try /usr/share/dovecot/protocols.d/*.protocol protocols = imap lmtp
Also verify that dovecot.conf is including all the other configuration files:
!include conf.d/*.conf
Edit /etc/dovecot/conf.d/10-mail.conf:
## Tells Dovecot where to find mail in maildir format, in this case in /var/mail/vhosts/yourdomain.com/email@yourdomain.com. mail_location = maildir:/var/mail/vhosts/%d/%n ## Tells Dovcot to write to the /var/mail folder. mail_privileged_group = mail
Verify permissions, and create the virtual mail directory and user/group:
$ ls -ld /var/mail drwxrwsr-x 2 root mail 4096 Apr 12 22:14 /var/mail $ mkdir -p /var/mail/vhosts/example.com $ groupadd -g 5000 vmail $ useradd -g vmail -u 5000 vmail -d /var/mail $ chown -R vmail:vmail /var/mail
Edit /etc/dovecot/conf.d/10-auth.conf:
## Uncomment this line disable_plaintext_auth = yes ## Modify this line auth_mechanisms = plain login ## Comment out this line: #!include auth-system.conf.ext ## Uncomment the MySQL auth line #!include auth-system.conf.ext !include auth-sql.conf.ext #!include auth-ldap.conf.ext #!include auth-passwdfile.conf.ext #!include auth-checkpassword.conf.ext #!include auth-vpopmail.conf.ext #!include auth-static.conf.ext
Create /etc/dovecot/conf.d/auth-sql.conf.ext to hold your authentication info:
passdb {
driver = sql
args = /etc/dovecot/dovecot-sql.conf.ext
}
userdb {
driver = static
args = uid=vmail gid=vmail home=/var/mail/vhosts/%d/%n
}
Edit /etc/dovecot/dovecot-sql.conf.ext:
## Uncomment and specify the driver driver = mysql ## Uncomment and set connection information connect = host=127.0.0.1 dbname=mailserver user=mail_username password=mail_password ## Uncomment and set the encryption format for passwords default_pass_scheme = SHA512-CRYPT ## Uncomment and set password query password_query = SELECT email as user, password FROM virtual_users WHERE email='%u';
Set permissions on the /etc/dovecot directory so the vmail user can use it:
chown -R vmail:dovecot /etc/dovecot chmod -R o-rwx /etc/dovecot
Edit /etc/dovecot/conf.d/10-master.conf:
## Disable unencrypted IMAP by setting the port to 0
service imap-login {
inet_listener imap {
port = 0
}
...
}
## (Leave imaps-login alone)
## Create LMTP socket for Dovecot where we told Postfix to find it
service lmtp {
unix_listener /var/spool/postfix/private/dovecot-lmtp {
mode = 0600
user = postfix
group = postfix
}
# Create inet listener only if you can't use the above UNIX socket
#inet_listener lmtp {
# Avoid making LMTP visible for the entire internet
#address =
#port =
#}
}
## Create authorization socket where we told Postfix to find it
service auth {
# auth_socket_path points to this userdb socket by default. It's typically
# used by dovecot-lda, doveadm, possibly imap process, etc. Its default
# permissions make it readable only by root, but you may need to relax these
# permissions. Users that have access to this socket are able to get a list
# of all usernames and get results of everyone's userdb lookups.
unix_listener /var/spool/postfix/private/auth {
mode = 0666
user = postfix
group = postfix
}
unix_listener auth-userdb {
mode = 0600
user = vmail
#group =
}
# Postfix smtp-auth
#unix_listener /var/spool/postfix/private/auth {
# mode = 0666
#}
# Auth process is run as this user.
user = dovecot
}
## Set the auth-worker user to vmail
service auth-worker {
# Auth worker process is run as root by default, so that it can access
# /etc/shadow. If this isn't necessary, the user should be changed to
# $default_internal_user.
user = vmail
}
Edit /etc/dovecot/conf.d/10-ssl.conf:
## If you have your own SSL cert and key, specify them here. We're using the free ones that come with Dovecot. ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem ## Force clients to use SSL ssl = required
Restart Dovecot:
$ service dovecot restart
Note: Make sure that port TCP/993 is open on your server’s firewall for IMAP. If you added POP, open port TCP/995.
Now you should be good to go. Set up a test account in your email client to ensure everything works:
Username: myemail@mydomain.com
Password: Whatever you added to the MySQL table in step 2 for this email address
IMAP server: yourdomain.com
SMTP server: yourdomain.com
Ports: 993 for secure IMAP, 25 or 587 for SMTP
Try sending mail and see what happens. If it didn’t work, check the log in /var/log/mail.log. You can also turn on verbose logging for Dovecot by adding a few lines to /etc/dovecot.conf:
## Verbose logging auth_debug_passwords=yes mail_debug=yes auth_verbose=yes verbose_ssl=yes auth_verbose_passwords=plain
Once things are working, you should probably comment these out again so they don’t clog up your mail log.
Finally, to add users/domains/forwards to your mail server, just add them to the appropriate tables in MySQL like in step 2. That’s all you need to do since authentication and incoming mail query the database to determine validity. For new accounts, the appropriate folders are created in /var/mail/ when each account first receives a message.
In our next installment, we’ll set up an anti-spam system.
Hi,
Thank you for your nice page, very useful!
Recently I setup a homemade e-mail server and wrote a full detailed tutorial that you can find in
http://cosmolinux.no-ip.org/raconetlinux2/mail.html
using Debian Squeeze, Postfix, Dovecot, SASL, Spamassassin and Squirrel (and a Google account for SMTP relay).
I wish it is helpful to someone.
No problem! Looks like you have a good writeup there as well. I really need to set up Squid here at some point also.
Thanks for all the work you’ve put into this. Unfortunately, I’ve tried it twice, with a apt-get purge in between, and continue to have a SASL issue. The first time through, I tried to follow all the online suggestions, only to bastardize my conf files, thus the purge. Do you have any suggestions? The log entries are;
warning: SASL: Connect to private/auth failed: No such file or directory
fatal: no SASL authentication mechanisms
warning: process /usr/lib/postfix/smtpd pid 12233 exit status 1
warning: /usr/lib/postfix/smtpd: bad command startup — throttling
whenever I try to connect from OS X mail client.
Thanks in advance.
No problem. The private/auth connection it’s referring to is a Unix socket at /var/spool/postfix/private/auth so I’d make sure that exists and has the right owner/group and permissions (postfix:postfix, 0666). There should be other sockets in that directory (retry, relay, maildrop, etc.) all with the same permissions. The parent directory /var/spool/postfix should also have various directories and files in it, some owned by root, so if any of these locations appear empty or missing I’d do a clean reinstall of Postfix.
I’d also make sure your /etc/dovecot/conf.d/10-master.conf file is correct, since it designates the private/auth socket for authentication.
Great tutorial!
Instead on manually manipulating tables I use a command line tools custom made for this type of installation.
Google the following to find it:
# beas.sh – Bash Email Admin Script
# Version 0.2 (Lenny edition)
# (c) 2009 Emmanuel Revah – manu-at-manurevah.com
That certainly works, although oftentimes I just use a visual tool (MySQL Workbench, SQL Explorer for Eclipse, etc.) over SSH for adding users. You can save your queries in there so you don’t have to copy+paste them into the console every time.
Excellent tutorial, worked almost perfectly
Upon testing dovecot I was getting login errors, turning on the logging showed:
Jan 5 13:01:00 mail dovecot: auth-worker: Error: md5_verify(yourEmail): Not a valid MD5-CRYPT or PLAIN-MD5 password
Jan 5 13:01:00 mail dovecot: auth-worker: Warning: Invalid OTP data in passdb
Jan 5 13:01:00 mail dovecot: auth-worker: Warning: Invalid OTP data in passdb
Jan 5 13:01:00 mail dovecot: auth-worker: Debug: sql(yourEmail,127.0.0.1): SHA512-CRYPT(yourPassword) != ‘aLongHashString’
The MySQL statement at the top of the tutorial creates a ‘password’ varchar column not quite long enough resulting in a truncated hash…. failed login everytime.
When creating your MySQL tables a little modification is needed
CREATE TABLE `virtual_users` […] `password` varchar(100) NOT NULL, […]
CREATE TABLE `virtual_users` […] `password` varchar(128) NOT NULL, […]
(If already created just run : ALTER TABLE virtual_users MODIFY password varchar(128) NOT NULL;)
After altering the tables (and re-inserting the previously truncated password to restore it) was then able to login to dovecot.
Good catch, it actually needs to be varchar(106) to work on Linux systems. I use this all the time in my setups but my documentation here had a typo. You might also prefer to use char(106), since the password will always be fixed at 106 characters if you use the SQL statement listed above to create new users.
The way you arrive at 106 characters is: 86-character encrypted password + 4 separator characters + 16-character salt.
As for why the password is always 86 characters, the full ENCRYPT(‘firstpassword’, CONCAT(‘$6$’, SUBSTRING(SHA(RAND()), -16))) statement tells MySQL to:
a) Generate a random floating point value between 0 and 1.0
b) Calculate an SHA1 checksum for the random value, expressed as a string of 40 hexadecimal digits, e.g., ‘a9993e364706816aba3e25717850c26c9cd0d89d’
c) Select a substring of (b), starting with the character in the 25th position and running to position 40 (16 total characters)
d) Concatenate ‘$6$’ and your substring from (c), so you end up with a value like ‘$6$7850c26c9cd0d89d’
e) Encrypt ‘firstpassword’ into a binary string using your concatenated randomized value from (d) as the salt. Per the MySQL documentation, ENCRYPT() relies on the crypt() Unix system call, so results will vary by platform, but on current Linux systems this gives you a string like ‘$6$7850c26c9cd0d89d$encrypted-password’. On Linux systems, the ‘$6$’ appended to the salt also tells crypt() to use SHA-512 encryption, giving you an 86-character encrypted string (see the crypt man page at http://man7.org/linux/man-pages/man3/crypt.3.html). So now you have ‘$6$’ + 16-character salt + ‘$’ + 86-character encrypted password = 106 characters.
f) Return the final result to the INSERT statement.
Hi, Thanks for the write up, very detailed.
I’m getting the following error in my /var/log/mail.err log, and I’m pretty sure I’ve gone through everything:
dovecot: auth: Fatal: master: service(auth): child 19059 returned error 89 (Fatal failure)
dovecot: master: Error: service(auth): command startup failed, throttling for 2 secs
dovecot: auth: Error: Can’t open configuration file uid=vmail gid=vmail home=/var/mail/vhosts/%d/%n: No such file or directory
Other messages for “Can’t open configuration file” seem to point to an actual file, however I’ve only changed the ‘/etc/dovecot/conf.d/auth-sql.conf.ext’ file to contain this value, in the args section:
passdb {
driver = sql
# Path for SQL configuration file, see example-config/dovecot-sql.conf.ext
args = /etc/dovecot/dovecot-sql.conf.ext
}
# “prefetch” user database means that the passdb already provided the
# needed information and there’s no need to do a separate userdb lookup.
#
#userdb {
# driver = prefetch
#}
userdb {
driver = sql
args = uid=vmail gid=vmail home=/var/mail/vhosts/%d/%n
}
Do you have any ideas why this could be happening?
Cheers,
Justin
Your userdb {} block should specify driver = static instead of driver = sql. If you want to look up users in a database rather than by home directory, you’d need to specify a user_query, like on http://wiki2.dovecot.org/Authentication/MultipleDatabases#Example_with_overriding_mail_location .