Mail Server

Now that I had recently bought my own domain, I wanted to try and make my own mail server. The thought of being able to put my own domain inside of a registration prompt on a website was very exciting and empowering to me. To begin, I researched the various mail servers available, and narrowed them down to 2:

• Mailcow
• Postfix & Dovecot

Of the two, Mailcow is supposed to be much easier. It can easily be installed with docker. I went ahead and started the installation. I went to Mailcow’s official website to check the instructions.

First, I had to run these commands:

I also ran the command to generate a config file:

Now, I have to start Mailcow with:

And on the last command to start Mailcow, I was met with errors.

Docker was trying to install for amd64, which is the x86 processor architecture. This is the most common architecture at the time of writing this article. However, my server uses arm64, or the ARM architecture. This is because it’s more power efficient, so I get more resources for the same price from my server host.

Unfortunately, after doing some research, it turns out that Mailcow’s support for ARM is still in the alpha stages. I had no choice but to use Postfix and Dovecot.

Before setting up Postfix, I set up a DNS record on my domain for mail.jofroop.com.

First, I followed this basic tutorial from the Ubuntu website. My server runs Ubuntu so I thought this would work well.

I won’t bore you with the process of me following the tutorial, I simply entered the commands in the order that they were displayed in on the website. But, to summarise, I’ve set it up so that when Postfix receives mail, it is stored in the Maildir/ directory. To check my mail, I use the mail command from the mailutils package.

When I finished the process, I first tried to send an email to myself from my outlook account. When I did, nothing happened. Outlook didn’t give me an error, and nothing appeared in my mail directory on my Linux server. I realised I had forgotten to unblock ports on my server. I used ufw (uncomplicated firewall) to unblock port 25 on my Linux server, and got this bounce message in my Outlook account.

The mail delivery was unsuccessful, with the error 550 Unroutable address. As you can see, I’ve sent the address to root@jofroop.com. However, in my Postfix config, I’ve set the mail server to run on mail.jofroop.com, so I tried sending it to mail.jofroop.com instead. When I tried this, I received the mail. In my Maildir directory.

The message contained the test data that I sent.

Next, I tried the inverse. I tested sending a mail to my Outlook account, using the mailutils command line tool.

This is not the actual mail that I sent. That was lost a while ago.

Unfortunately, after waiting for 10 mins, the E-Mail did not appear in my Outlook inbox. After doing some research, I found out that large mail providers like Google and Microsoft generally block small, unknown addresses like mine, since they are usually spam. To counteract this, you’re supposed to set up DKIM, SPF and DMARC records, but I wasn’t ready to do that yet. Instead, I went into my Outlook account and whitelisted my domain as a temporary fix.

After testing once more, I was able to see my mail.

This is the actual mail that I sent and received.

As you can see, the mail is from root@jofroop.com. The first mail that I managed to receive was actually from root@mail.jofroop.com. It seemed like my configuration was a bit broken, it was only sending from and receiving mail to mail.jofroop.com. It’s fine, but not what I wanted. I wanted to send and receive mail from Jofroop.com.

Sending mail from Jofroop.com, or my “apex domain" is actually quite easy, you simply set a “from" address on mailutils. However, this is when things started getting very irritating and confusing.

My knowledge of domains and mail servers was very limited. The process of setting my mail server to use my apex domain was a long and tedious process that took more than 3 days. I ended up uninstalling and re-installing Postfix and Dovecot about 5 times. I had issues where even though I used apt purge postfix to try and get rid of configuration files, when I re-installed Postfix, it didn’t give me the full installation configuration prompt.

I followed many tutorials online, most of which contained confusing and conflicting information. It seemed like no one really knew what they were talking about, and most tutorials were not very detailed. Overall, it was very painful, and I felt like I was at my wit’s end. However, I finally managed to work it out, and In the end, I had a Postfix server that could send and receive mail from my apex domain.

At this point, I was sending all my mail using the mailutils or s-nail commands. I wanted to “log into my Postfix server" with a client like Mozilla Thunderbird or Microsoft Outlook. I tried logging into my root account, root@jofroop.com.

As you can see, it asks for a password. I had no idea what this password was meant to be. I’d never even set up an account for Postfix. I had no idea what I was talking about. I tried my root password, and got an error.

I did some research after this to fill in the gaps in my knowledge. It turns out I had misunderstood how the mail stack works. First of all, Postfix isn’t a mail server. It’s a mail relay client, or an MDA (Mail Delivery Agent). Postfix is responsible for sending and receiving email using SMTP, but it is not responsible for logins from another client like Thunderbird.

I hadn’t used Dovecot until now. It turns out that Dovecot was responsible for creating an IMAP or POP3 server, which is what Thunderbird logs into. For another 2 or 3 days, I tried my best to configure Dovecot. Once again, I ended up re-installing it another 2 or 3 times. I found out that Dovecot actually doesn’t allow you to log in as root. Dovecot uses Linux system users, so I created a new account called Jofroop on my Linux machine, as my ultimate goal was to create the mail account jofroop@jofroop.com.

At long last, I was able to log in. I entered my credentials for what seemed like the 1 billionth time, and got a different prompt from Thunderbird.

Thunderbird was able to find my login. I clicked the done button, and was greeted with this:

A re-enaction of what I saw.

Unfortunately, even though I managed to log in, I couldn’t see my E-Mails in my inbox. I tried manually downloading them using the button in the top left corner, but nothing happened. I was, however, able to send mail. This, at least, made it much easier to send mails than using the command line.

I tested and made sure it ended up in my Outlook inbox.

After some debugging, I found that this was because the Dovecot mailbox and Postfix mailbox were not in the same location. I edited /etc/postfix/postfix.conf and set the mailbox location. And then, I was able to see my mail.

Now I had a “fully functioning" mail client. I put fully functioning in quotes, because the delivery of my mail was horrible. I used mail-tester.com to verify this.

Initially my mail score was actually even lower, around 0.1 or 0.2 though I don’t have a screenshot of that. The reason my mail score was so low was for the reasons that I mentioned earlier; I don’t have a DKIM, SPF or DMARC record.

I used this tutorial video to set up my DKIM and DMARC records. I set it up, word for word, as the man in the tutorial had, but I got an error: /etc/opendkim/TrustedHosts: dkimf_db_open(): No such file or directory.

This took me a while to resolve. I thought that maybe opendkim didn’t have the permissions to access the folder. That wasn’t the issue. It turns out that the following lines:

Had to be changed to:

The tutorial was literally just wrong. Anyway, after setting this up, I tried to send an email, but it didn’t go through. Thunderbird gave me a vague error that the server wasn’t ready. When I checked the mail log, opendkim said it wasn’t able to access /etc/opendkim/TrustedHosts. Once again, I tried giving opendkim permissions, and trying various things online.

I tried logging out of my Thunderbird client, and logging back in, but it wouldn’t let me. It couldn’t authenticate the username and password. I had broken my Dovecot installation. This was incredibly infuriating, I had a working mail client just seconds ago, and now it had been broken since I followed a wrong tutorial. After some searching online, I came across the holy grail of Postfix and Dovecot articles that would lead me to success:

https://www.linuxbabe.com/mail-server/secure-email-server-ubuntu-postfix-dovecot

This article was incredibly comprehensive, and covered everything I need to know. I couldn’t believe that I hadn’t found this before. It’s just proof that the first results on Google aren’t to be trusted, they’re SEO and nothing more.

I decided to start all over again, with this new article. Once again, I purged Postfix and Dovecot, and followed this tutorial. After following the tutorial, and tinkering a little bit since my configuration files at this point were horribly mutilated, I once again got my Dovecot working. I was able to log in. Thank you, Xiao Guoan. I love you. However, I wasn’t able to send emails, since this Dovecot configuration needed DKIM, and It wasn’t set up properly.

This website also had other tutorials, like how to set up Postfix – though it was a bit late for that – and one for setting up DKIM. I now had faith in linux babe, and decided to follow the tutorial:

https://www.linuxbabe.com/mail-server/setting-up-dkim-and-spf

I purged opendkim, and followed the words of Xiao Guoan. I changed a few things in my DNS, and allowed it to propagate across the internet before testing. And when I did, I was pleasantly surprised:

Not only was I able to send mail, my deliverability was very high. NOW I had a fully functioning mail client, thanks to Xiao Guoan, and linux babe.

I tested my shiny new mail client. I logged into my Dovecot server with Outlook. It worked. I added another user called test-500 and tried sending mails with that. It worked. I tried logging in with my mobile Gmail app. It worked. I could send to gmail accounts, and I could send to outlook accounts without going into spam. It was amazing.

All in all, setting up this mail server was a frustrating, but rewarding experience. It was mostly frustrating due to the lack of good tutorials, and conflicting information. Or maybe the tutorials are good, and I’m just bad at following them. I don’t know. Either way, when I’m registering for a website, I can now enter jofroop@jofroop.com. And I think that’s awesome.

I'd like to say that this article isn't 100% accurate to my experiences, I wrote this after I had finished the mail server, and I can't exactly remember everything that I did in the correct order. It's more just a general idea. My actual experience was much, much more painful, long, and detailed.

A massive thank you to Xiao Guoan and his website, linux babe. I would probably still be stuck if it wasn’t for that article.