A mail server to send email

In the first part of the tutorial “A mail server to receive email”, we install the software and set up the server to receive emails. In this part, we setup the server to send emails.

1. Relaying outgoing emails through Postfix

We use postfix to relay our email.

Relaying

Please note that there is a difference on how users send emails versus how servers send emails. For comparison:

Incoming email

When someone on the internet sends an email to john@example.org, some other mail server will deliver the email using SMTP to your mail server. Postfix will determine that it’s responsible for email addresses in the example.org domain and accept the email. John can then get the email from your server.

Outgoing email (without authentication)

John is on the internet somewhere and wants to send an email to lisa@example.com. Your mail server is not responsible for the “example.com” domain so it receives John’s email and would have to forward (relay) it to the actual mail server that is responsible for …@example.com email addresses. This may seem like a harmless scenario but your mail server must deny that.

Why? Because anyone can claim to be John and make your mail server forward mail. If an attacker (like a spammer) would send millions of spam emails in John’s name through your server then other organisations will accuse you as the operator of the mail server of spamming. Your mail server would be what people call an open relay. That is not what you want because your mail server would get blacklisted and you will not be able to send out mail to most other servers. So, without any proof that the sender is indeed John, your server must reject the email.

Outgoing email (with authentication)

So how does John prove his identity? He needs to use authenticated SMTP. This is similar to the previous case but John’s email program will also send his username and password. We are making sure that his authentication happens over an encrypted connection so John’s password is safe.

Postfix setting “mynetworks”

In addition to using SMTP authentication, you can tell Postfix to always allow relaying for certain IP addresses. The mynetworks setting contains the list of IP networks or IP addresses that you trust. Usually, you define your own local network here. The reason John had to authenticate in the above example is because his IP address is not part of mynetworks.

Make Postfix use Dovecot for authentication

Enabling SMTP authentication in Postfix is surprisingly easy. You already configured Dovecot so it knows all about your users from the SQL database. So, let’s just make Postfix use that by telling it to ask the Dovecot server to verify the username and password. Postfix just needs some extra configuration. Run these commands on the shell:

sudo postconf smtpd_sasl_type=dovecot
sudo postconf smtpd_sasl_path=private/auth
sudo postconf smtpd_sasl_auth_enable=yes

This enables SMTP authentication and tells Postfix that it can talk to Dovecot through a socket file located at /var/spool/postfix/private/auth. Do you remember that Postfix runs in a sandboxed chroot directory? That’s at /var/spool/postfix. It cannot access any files outside of that directory. But fortunately, in a previous section you edited the /etc/dovecot/conf.d/10-master.conf file and made Dovecot place a socket file into /var/spool/postfix/private/auth to allow communication from Postfix.

Enable encryption

The following settings enable encryption, set the key and certificate paths for Postfix. Just run these commands:

sudo postconf smtpd_tls_security_level=may
sudo postconf smtpd_tls_auth_only=yes
sudo postconf smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.example.org/fullchain.pem
sudo postconf smtpd_tls_key_file=/etc/letsencrypt/live/webmail.example.org/privkey.pem
sudo postconf smtp_tls_security_level=may

smtp or smtpd?

Look closely. Some settings start with “smtp_” and others with “smtpd_”. That is not a typo. “smtp_” refers to the SMTP client. That is the component that sends out emails from Postfix to other servers.

Whereas “smtpd_” means the SMTP server. That in turn is the component that receives emails from other systems – either from a remote mail server or one of your users.

The above settings allow encrypted incoming (smtpd_) and outgoing (smtp_) connections. But they do not enforce it. If a remote mail server does not have encryption enabled, we will still accept their emails. You may be tempted to enforce encryption but that would reject email communication with servers who have been configured without encryption.

However, the smtpd_tls_auth_only=yes setting makes sure that the user’s authentication information (email address and password) is always encrypted between the user and your mail server.

How SMTP authentication works

Are you curious how SMTP authentication looks on a low level? You probably are not. But let’s do it anyway. Just once, so that you get the idea.

In previous versions of this guide, we used “telnet” to connect to TCP port 25 and speak SMTP. But now we enforce encryption and can’t do SMTP authentication unencrypted. Let’s try it anyway:

Install telnet if necessary and then connect to localhost through SMTP:

sudo apt install telnet

telnet localhost smtp

The server will let you in:

Trying ::1…
Connected to localhost.
Escape character is '^\]'.
220 webmail.example.org ESMTP Postfix (Debian/GNU)

Say hello:

ehlo example.com

Postfix will present a list of features that are available for you:

250-webmail.example.org
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250-DSN
250-SMTPUTF8
250 CHUNKING

Let me briefly explain what these lines mean: However, one important line is missing here that would allow us to send our username and password:

250-AUTH PLAIN LOGIN

We told Postfix to only allow authentication when the connection is encrypted. So, we are not offered authentication over this plain text connection. That’s what we want.

Are you still connected? Okay, good. So, we need an encrypted connection using TLS. Using the STARTTLS feature we can switch over from unencrypted to encrypted without having to reconnect. Enter…

STARTTLS

And the server replies:

220 2.0.0 Ready to start TLS

However now it’s getting weird because you would have to speak TLS encryption which is not a language that humans speak. So, let’s quit this using the “QUIT” command and do that differently:

QUIT

Before going to the next section, we need to know the username and password in the base64-encoded format. Type the following command in bash:

printf '\0john@example.org\0summersun' | base64

You will get:

AGpvaG5ANC4yMTMuMTgyLjEzNy5uaXAuaW8Ac3VtbWVyc3Vu

Now we can use OpenSSL to help us with the decryption. Run:

openssl s_client -connect localhost:25 -starttls smtp

You will see a lot of output. OpenSSL has connected to TCP port 25 and issued a STARTTLS command to switch to an encrypted connection. So, whatever you type now will get encrypted. Enter:

EHLO umd.eastus.cloudapp.azure.com

And Postfix will send a list of capabilities that will look like this:

250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-AUTH PLAIN LOGIN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN

And now that we are using an encrypted connection Postfix offers us to authenticate. So let us send the authentication string with a Base64-encoded password (username john@example.org, password summersun):

AUTH PLAIN AGpvaG5ANC4yMTMuMTgyLjEzNy5uaXAuaW8Ac3VtbWVyc3Vu

The server should accept that authentication:

235 2.7.0 Authentication successful

Excellent. You are logged in through SMTP. You could now send an email to be forwarded to another mail server. I just wanted to show that authentication works if you use an encrypted connection.

Disconnect from Postfix:

QUIT

Authentication works. Well done.

Base64-encoded passwords

Wait a minute. What was that crazy cryptic string? There was no username and password in it. Was it encrypted?

No, that was no encryption. It was merely an encoded version of the username and password. Why that? Well usually in SMTP you can only transfer ASCII characters. But the password may contain special characters which are not covered by ASCII. So, in the PLAIN method that information is Base64 encoded.

What is actually converted to Base64…

NULL-BYTE + USERNAME + NULL-BYTE + PASSWORD

So, for John’s case you can easily create the Base64 string using:

printf '\0john@example.org\0summersun' | base64

As a result, you will get the string you used for “AUTH PLAIN”.

The submission port

Although I have been talking about SMTP on port 25 to relay mails, it is actually not the proper way. End users should not use port 25 but rather the submission service on TCP port 587 (as described in RFC 4409). The idea is to use port 25 for transporting email (MTA = mail transport agent) from server to server and port 587 for submitting (MSA = mail submission agent) email from a user to a mail server.

For comparison:
TCP portService NameEncryptionAuthenticationMeant forYour ISP
25STMPOptionalOptionalServer-to-Servermay block this port
587SubmissionMandatoryMandatoryUser-to-Servershould allow this port

I hope that makes the distinction a bit clearer. So, let’s enable the submission service. All Postfix services are declared in the /etc/postfix/master.cf file. Please edit the file and look for the submission section that is commented out by default. Turn that section into the following. Basically, I removed the ‘#’ character on all lines of this section and removed the lines with the mua_* variables. It does not matter even if you switch the value of recipient and relay restrictions.

        submission inet n       -       y       -       -       smtpd
          -o syslog_name=postfix/submission
          -o smtpd_tls_security_level=encrypt
          -o smtpd_sasl_auth_enable=yes
          -o smtpd_tls_auth_only=yes
          -o smtpd_reject_unlisted_recipient=no
          -o smtpd_client_restrictions=
          -o smtpd_helo_restrictions=
          -o smtpd_sender_restrictions=
          -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
          -o smtpd_recipient_restrictions=
          -o milter_macro_daemon_name=ORIGINATING
Make sure to start the first line in the first column and indent the following lines.

This service uses the “smtpd” daemon (see the last word of the first line) which is the piece of software that responds if you open an SMTP connection on TCP port 25. But it gets a few extra options set…

Restart the Postfix server:

sudo systemctl restart postfix

Your users can now use the submission port to send email. They just use the port 587 in their mail clients instead of port 25.

Feel free to send a test mail using the submission port and encryption. You will need to install the libnet-ssleay-perl package first, so that SWAKS can speak TLS:

        sudo apt install -y libnet-ssleay-perl
        swaks --server localhost --to john@example.org \
            --port 587 -tls \
            --auth-user john@example.org \
            --auth-password summersun
Note that the auth-password this way should not contain special characters such as $!, so I have changed it back to summersun.

What is Port 465?

This TCP port belongs to the “submission over TLS” service. It is used for the submission service but expects an encrypted connection from the first byte. This port is hardly ever used so you don’t have to care about it. The submission service you just configured is also encrypted but uses the STARTTLS mechanism to switch to a TLS connection after the welcome message.

You can now go ahead and use PHP mailer to send message to yourself. The settings to send email to others could be as follows:

    
        $mail->isSMTP();                                            //Send using SMTP
        $mail->Host       = 'tls://172.191.204.33.nip.io';          //Set the SMTP server to send through
        $mail->SMTPAuth   = true;                                   //Enable SMTP authentication
        $mail->Username   = 'john@172.191.204.33.nip.io';           //SMTP username
        $mail->Password   = 'summersun';                            //SMTP password
        $mail->SMTPSecure = PHPMailer::ENCRYPTION_SMTPS;            //Enable implicit TLS encryption
        $mail->Port       = 587;                                    //TCP port to connect to; 

        //Recipients
        $mail->setFrom('john@172.191.204.33.nip.io', 'UMD MSGIS');
        $mail->addAddress('john@172.191.204.33.nip.io', 'Xin Tao'); //Add a recipient

Use an email management software, such as Thunderbird or Outlook, and login using your account, such as john@example.org along with your password. Once you login, you should be able to see the mail in the mailbox.

2. Managing users, aliases and domains

Maybe you already know what you have to do to create mail domains and mail users. After all I tried to explain the database schema in the section that dealt with preparing the database. But if that wasn’t clear enough let me explain what you need to do to manage your mail accounts.

Using SQL queries

The following sections explain the changes and SQL queries you can use for common management tasks:

Create a new mail domain

INSERT INTO virtual_domains (name) VALUES ("example.org");

Delete a mail domain

DELETE FROM virtual_domains where name='example.org';

Create a mail user

INSERT INTO virtual_users (domain_id, email, password) VALUES ((SELECT id FROM virtual_domains WHERE name='example.org'), 'j@example.org',' '{BLF-CRYPT}$2y$05$sKw0P0FqUQnh.tzSbuA7aO7j4SRf9lZFoh9a70V4scxLI2Vb91p1q');

Change the password of a user

UPDATE virtual_users SET password='{BLF-CRYPT}$2y$05$.We…' WHERE email='email@address';

Delete a mail user

Find the row in the virtual_users table by looking for the right “email” field and delete it. The mailbox will stay on disk at /var/vmail/… and you need to delete it manually

Create a mail forwarding

You can forward emails from one (source) email to other addresses (destinations) – even outside of your mail server. Find out the “id” of the right domain (the part after the “@” of the source email address) from the virtual_domains table. Create a new row in the virtual_aliases table for each destination (if you have multiple destination addresses). Set the “source” field to the complete source email address. And set the “destination” field to the respective complete destination email address.

Delete a mail forwarding

Find all rows in the virtual_aliases table by looking for the right “source” email address. Remove all rows that you lead to “destination” addresses you don’t want to forward email to.