Install Self-Hosted Email Server

CompactHost · March 26, 2026

Running your own email server feels like crossing into serious infrastructure territory—and honestly, it is. But I've done it, and the control is worth the complexity. No corporate filtering, no data handed over to third parties, and full encryption end-to-end. I'll walk you through a production-ready setup using Postfix, Dovecot, and a few hardening layers that actually work.

Why Self-Host Email?

Before we dive into the technical weeds, let me be honest about what you're signing up for. Self-hosting email means you own the entire delivery pipeline: sending, receiving, storage, and security. You'll need a VPS (or a very stable home connection with static IP), proper DNS records, and monitoring. The upside is real: you control your data, you're not subject to someone else's spam policies, and you can run it exactly how you want.

I use a RackNerd KVM VPS for this—their performance is solid and the cost is unbeatable for what you get. A basic 2GB plan runs about $20/year and handles mail for a small domain without breaking a sweat.

Prerequisites

You'll need:

• A VPS with a static IP (or home server with port forwarding and DDNS)
• A domain name you own
• Root access to your server
• Ports 25, 587, 993, and 110 open (or 143 for IMAP unencrypted—don't do that)
• SPF, DKIM, and DMARC records ready to configure
• About 2GB of RAM minimum (4GB recommended)

Install Postfix and Dovecot

Postfix handles outbound and inbound SMTP; Dovecot provides IMAP (and optional POP3) for clients to retrieve mail. On Ubuntu 22.04+, this is straightforward:

sudo apt update && sudo apt upgrade -y
sudo apt install postfix dovecot-core dovecot-imapd dovecot-pop3d -y
sudo apt install spamassassin spamc opendkim opendkim-tools -y
sudo apt install certbot python3-certbot-nginx -y

During the Postfix installer, select "Internet Site" and enter your domain when prompted. Don't stress if you get it wrong—we'll fix it in the config files.

Configure Postfix

The main config lives at /etc/postfix/main.cf. Here's my tested configuration:

sudo cp /etc/postfix/main.cf /etc/postfix/main.cf.bak

cat > /tmp/postfix_additions.cf << 'EOF'
myhostname = mail.yourdomain.com
mydomain = yourdomain.com
myorigin = $mydomain
inet_interfaces = all
inet_protocols = ipv4

mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain

virtual_mailbox_domains = yourdomain.com
virtual_mailbox_base = /var/mail/vmail
virtual_mailbox_maps = hash:/etc/postfix/vmailbox
virtual_uid_maps = static:5000
virtual_gid_maps = static:5000

smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes

smtpd_tls_cert_file=/etc/letsencrypt/live/mail.yourdomain.com/fullchain.pem
smtpd_tls_key_file=/etc/letsencrypt/live/mail.yourdomain.com/privkey.pem
smtp_tls_cert_file=/etc/letsencrypt/live/mail.yourdomain.com/fullchain.pem
smtp_tls_key_file=/etc/letsencrypt/live/mail.yourdomain.com/privkey.pem

smtpd_use_tls = yes
smtp_tls_security_level = may
smtpd_tls_security_level = may
smtpd_sasl_security_options = noanonymous

smtpd_relay_restrictions = permit_mynetworks, permit_sasl_authenticated, defer_unauth_destination
relayhost = 

milter_default_action = accept
milter_protocol = 6
smtpd_milters = inet:localhost:12345
non_smtpd_milters = inet:localhost:12345

smtpd_helo_required = yes
smtpd_helo_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_invalid_helo_hostname
smtpd_sender_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unknown_sender_domain
smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination, reject_unverified_recipient

header_checks = regexp:/etc/postfix/header_checks
body_checks = regexp:/etc/postfix/body_checks
EOF

sudo tee -a /etc/postfix/main.cf < /tmp/postfix_additions.cf

Replace yourdomain.com and mail.yourdomain.com with your actual domain and mail hostname.

Next, create the virtual mailbox file:

sudo mkdir -p /var/mail/vmail
sudo useradd -r -u 5000 vmail
sudo chown -R vmail:vmail /var/mail/vmail
sudo chmod 700 /var/mail/vmail

echo "[email protected] yourdomain.com/user1/" | sudo tee /etc/postfix/vmailbox
echo "[email protected] yourdomain.com/user2/" | sudo tee -a /etc/postfix/vmailbox

sudo postmap /etc/postfix/vmailbox
sudo postfix reload

Configure Dovecot

Edit /etc/dovecot/dovecot.conf and make sure these settings are in place:

sudo tee /etc/dovecot/conf.d/99-custom.conf > /dev/null << 'EOF'
mail_location = maildir:/var/mail/vmail/%d/%n
user = vmail
group = vmail

protocols = imap pop3
listen = *, ::

service auth {
  unix_listener /var/spool/postfix/private/auth {
    mode = 0660
    user = postfix
    group = postfix
  }
}

ssl = required
ssl_cert = 

sudo certbot certonly --standalone -d mail.yourdomain.com
sudo systemctl restart postfix dovecot

sudo mkdir -p /etc/opendkim/keys/yourdomain.com
cd /etc/opendkim/keys/yourdomain.com

sudo opendkim-genkey -b 2048 -d yourdomain.com -D /etc/opendkim/keys/yourdomain.com -s default -v

sudo chown -R opendkim:opendkim /etc/opendkim/keys
sudo chmod 600 /etc/opendkim/keys/yourdomain.com/default.private

cat /etc/opendkim/keys/yourdomain.com/default.txt

sudo tee /etc/opendkim.conf > /dev/null << 'EOF'
Domain yourdomain.com
KeyFile /etc/opendkim/keys/yourdomain.com/default.private
Selector default
Socket inet:12345@localhost
LogWhy yes
SyslogSuccess yes
EOF

sudo systemctl restart opendkim

sudo tee /etc/dovecot/users > /dev/null << 'EOF'
[email protected]:{PLAIN}password123:5000:5000::/var/mail/vmail/yourdomain.com/user1::
[email protected]:{PLAIN}password456:5000:5000::/var/mail/vmail/yourdomain.com/user2::
EOF

sudo chmod 600 /etc/dovecot/users
sudo mkdir -p /var/mail/vmail/yourdomain.com/user1
sudo mkdir -p /var/mail/vmail/yourdomain.com/user2
sudo chown -R vmail:vmail /var/mail/vmail

sudo systemctl restart dovecot

telnet localhost 25
telnet localhost 587

sudo tail -f /var/log/mail.log

sudo apt install fail2ban -y
sudo systemctl enable fail2ban