Network Security Best Practices for Self-Hosting on a VPS

CompactHost · March 26, 2026

When I first spun up a VPS to host my own services, I made every rookie mistake in the book. I left SSH on port 22 with password authentication, opened all my ports without thinking, and deployed services with default credentials. Within 48 hours, my logs were flooded with brute-force attempts. That's when I learned: a VPS sitting on the public internet is attacked the moment it boots. This guide covers the real-world hardening techniques I've implemented across multiple providers—from RackNerd to Hetzner—that actually work.

Start with SSH Hardening: The First Line of Defense

SSH is almost always the first attack vector. Bots scan for port 22, and if you're using password authentication or weak credentials, you're compromised. I disable password auth entirely and rely on key pairs. Here's my baseline SSH configuration:

sudo nano /etc/ssh/sshd_config

These are the non-negotiable changes I make to every VPS:

# /etc/ssh/sshd_config
# Disable password authentication entirely
PasswordAuthentication no
PubkeyAuthentication yes

# Disable root login (use sudo instead)
PermitRootLogin no

# Change the default port away from 22
Port 2247

# Only allow specific users
AllowUsers myusername

# Disable X11 forwarding (not needed for self-hosting)
X11Forwarding no

# Limit authentication attempts
MaxAuthTries 3
MaxSessions 2

# Use only strong key exchange algorithms
KexAlgorithms curve25519-sha256,[email protected]
Ciphers [email protected],[email protected]
MACs [email protected],[email protected]

After editing, test the configuration before restarting:

sudo sshd -t
sudo systemctl restart ssh

Generate a strong keypair locally (not on the server) and copy your public key to the VPS:

# On your local machine
ssh-keygen -t ed25519 -C "vps-key-$(date +%Y%m%d)" -f ~/.ssh/id_ed25519_vps -N "your-passphrase"

# Copy the public key to your VPS
ssh-copy-id -i ~/.ssh/id_ed25519_vps.pub -p 2247 myusername@your-vps-ip

I prefer Ed25519 over RSA for its smaller key size and better security properties. Once that's working, remove the ability to authenticate with passwords on the server entirely.

Firewall: UFW Is Your Best Friend

Ubuntu's Uncomplicated Firewall (UFW) is perfect for self-hosters because it abstracts iptables complexity without hiding what's happening. I enable it and then explicitly allow only what I need:

# Enable UFW (careful: allow SSH first!)
sudo ufw allow 2247/tcp
sudo ufw enable

# Check the status
sudo ufw status

# Allow specific services
sudo ufw allow 80/tcp    # HTTP
sudo ufw allow 443/tcp   # HTTPS
sudo ufw allow 8080/tcp  # Common app port
sudo ufw allow from 10.0.0.0/8 to any port 3306  # MySQL only from internal network

# Deny everything else (default policy)
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw default deny routed

The key principle: deny by default, allow explicitly. This means if I forget to open a port for a new service, it simply won't be accessible—which is better than accidentally leaving something exposed.

If you're using a VPS provider like RackNerd with their KVM offerings, combine UFW with the provider's built-in firewall rules. Defense in depth is critical when your server is reachable from anywhere on the internet.

Fail2Ban: Automated Defense Against Brute Force

Even with SSH hardened, attackers will probe your services. Fail2Ban watches logs and automatically blocks IPs that show malicious patterns. When combined with strong SSH config, it's nearly bulletproof:

sudo apt update && sudo apt install fail2ban

# Copy the default config to create a local override
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

# Edit the local config
sudo nano /etc/fail2ban/jail.local

These are the settings I change:

[DEFAULT]
# Ban time in seconds (1 hour)
bantime = 3600
# Find time window (10 minutes)
findtime = 600
# Max retries before ban
maxretry = 3

# Enable these jails
[sshd]
enabled = true
port = 2247
maxretry = 2

[recidive]
enabled = true
action = %(action_mwl)s
bantime = 604800  # 1 week for repeat offenders
findtime = 86400  # Over 1 day
maxretry = 1

Restart fail2ban and check that it's monitoring your SSH logs:

sudo systemctl restart fail2ban
sudo fail2ban-client status sshd
sudo tail -f /var/log/fail2ban.log

After the first few days, you'll see dozens of IPs being banned automatically. It's oddly satisfying.

Network Segmentation and Service Isolation

I run multiple self-hosted services on each VPS: Nextcloud, Vaultwarden, Jellyfin, sometimes an Ollama instance. Ideally, these shouldn't be able to talk to each other directly. I use Docker networks to enforce this:

# Create isolated networks for different services
docker network create nextcloud-net --driver bridge
docker network create vaultwarden-net --driver bridge
docker network create monitoring-net --driver bridge

# When running containers, explicitly attach them
docker run -d --network nextcloud-net --name nextcloud nextcloud:latest

# Services on different networks can't communicate
# Even if one is compromised, lateral movement is blocked

For non-Docker services, use iptables or UFW rules to restrict inter-service communication. For example, if your database server should only accept connections from your application server:

sudo ufw allow from 127.0.0.1 to any port 3306
sudo ufw allow from APPLICATION_SERVER_IP to any port 3306
sudo ufw deny 3306/tcp

Regular Updates and Automated Patching

Unpatched systems are compromised systems. I set up automatic security updates on every VPS:

sudo apt install unattended-upgrades

# Configure it
sudo nano /etc/apt/apt.conf.d/50unattended-upgrades

Enable these lines:

Unattended-Upgrade::Allowed-Origins {
    "${distro_id}:${distro_codename}-security";
    "${distro_id}:${distro_codename}-updates";
};
Unattended-Upgrade::AutoFixInterruptedDpkg "true";
Unattended-Upgrade::MinimalSteps "true";
Unattended-Upgrade::Mail "root";
Unattended-Upgrade::MailReport "on-change";

This automatically applies security patches and notifies you of what was updated. I've never had a security incident on systems with this enabled.

Monitoring and Logging

You can't defend what you don't see. I use a centralized approach: send logs from all VPS instances to a single monitoring location using rsyslog or Ship them to a service like Uptime Kuma for alerting:

sudo apt install rsyslog auditd

# Enable auditd to track system changes
sudo systemctl enable auditd
sudo systemctl start auditd

# Review interesting logs
sudo tail -f /var/log/auth.log       # SSH attempts
sudo tail -f /var/log/syslog         # General system
sudo tail -f /var/log/fail2ban.log   # Fail2Ban activity

If you host on RackNerd's KVM VPS plans, you already have their dashboard for basic monitoring. Layer in fail2ban email alerts and you'll know about attack patterns within minutes.

Conclusion

Self-hosting on a VPS is rewarding, but it requires security discipline. The stack I use—SSH hardening, UFW, fail2ban, network segmentation, automatic updates, and monitoring—has kept my infrastructure untouched by compromise for over three years across dozens of services.

Start by hardening SSH (that's where 90% of attacks target), then layer on UFW and fail2ban. Once those three are solid, everything else is detail. Your next step? Audit your current VPS with these checks, then document your baseline configuration so you can replicate it across new deployments quickly.