Hardening SSH Access on Self-Hosted VPS Servers

CompactHost · March 26, 2026

When I spun up my first VPS, I thought the default SSH configuration was fine. Within days, my logs were flooded with brute-force attempts hitting port 22 from random IPs across the globe. Now, after securing dozens of servers, I've learned that SSH hardening isn't optional—it's the foundation everything else rests on. In this tutorial, I'll walk you through the exact steps I take to lock down SSH on every new VPS, from key-based authentication to fail2ban integration.

Why SSH Security Matters

SSH is your primary attack surface on any internet-facing VPS. If someone gains SSH access with valid credentials, they own your box. They can steal data, deploy ransomware, use your resources for cryptomining, or pivot into your internal network. The good news: SSH hardening is straightforward, repeatable, and immediately effective.

I recommend having a secure VPS provider as your foundation. If you're shopping for infrastructure, RackNerd offers affordable KVM VPS and hybrid dedicated servers with solid uptime for homelab and production workloads.

Step 1: Generate SSH Keys (On Your Local Machine)

The first rule of SSH hardening is ditching password authentication entirely. Keys are exponentially more secure. I generate a new 4096-bit RSA key for each critical server, keeping my main key only for Bastion access.

ssh-keygen -t rsa -b 4096 -f ~/.ssh/id_rsa_vps -C "vps-2026"

This creates two files: id_rsa_vps (private key—guard this) and id_rsa_vps.pub (public key—safe to share). When prompted for a passphrase, I always set one. It's an extra layer: even if someone steals your private key file, they need the passphrase to use it.

Step 2: Upload Your Public Key to the VPS

When you first get SSH access (usually via password or the provider's console), add your public key immediately. Log in as root (or your initial user), then:

mkdir -p ~/.ssh
echo "your_public_key_content_here" >> ~/.ssh/authorized_keys
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Now test the connection locally before closing the password session:

ssh -i ~/.ssh/id_rsa_vps [email protected]

If that works, you're safe to disable password auth. If it fails, troubleshoot before proceeding—you don't want to lock yourself out.

Step 3: Harden SSHD Configuration

The SSH daemon config lives at /etc/ssh/sshd_config. I always make a backup first, then implement these critical changes:

sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.backup

Edit the file with your preferred editor (I use nano):

sudo nano /etc/ssh/sshd_config

Find and modify these lines (or add them if missing):

# Disable root login entirely
PermitRootLogin no

# Disable password authentication
PasswordAuthentication no
PubkeyAuthentication yes

# Disable empty passwords
PermitEmptyPasswords no

# Change default port (optional but recommended)
Port 2222

# Limit authentication attempts
MaxAuthTries 3
MaxSessions 2

# Disable X11 forwarding (unless you need it)
X11Forwarding no

# Set client alive interval to detect dead connections
ClientAliveInterval 300
ClientAliveCountMax 2

# Use only modern ciphers
HostKeyAlgorithms ssh-ed25519,ecdsa-sha2-nistp256
KexAlgorithms curve25519-sha256,[email protected]
Ciphers [email protected],[email protected]
MACs [email protected],[email protected]

# Log authentication
SyslogFacility AUTH
LogLevel VERBOSE

After editing, validate the config:

sudo sshd -t

If that returns no errors, restart SSH:

sudo systemctl restart ssh

Step 4: Create a Non-Root User

I never use root for daily tasks. Create a standard user account:

sudo useradd -m -s /bin/bash deploy
sudo usermod -aG sudo deploy

Add your public key to the new user's authorized_keys:

sudo mkdir -p /home/deploy/.ssh
sudo bash -c 'echo "your_public_key_content_here" >> /home/deploy/.ssh/authorized_keys'
sudo chown -R deploy:deploy /home/deploy/.ssh
sudo chmod 700 /home/deploy/.ssh
sudo chmod 600 /home/deploy/.ssh/authorized_keys

Test login as the new user from your local machine, then disable root login and password auth permanently.

Step 5: Install and Configure Fail2Ban

Fail2ban monitors SSH logs and auto-bans IPs after repeated failed login attempts. It's a game-changer for stopping brute force attacks before they damage anything.

sudo apt update
sudo apt install fail2ban -y

Create a local config file to override defaults:

sudo nano /etc/fail2ban/jail.local

Add this configuration:

[DEFAULT]
bantime = 3600
findtime = 600
maxretry = 3
destemail = [email protected]
sendername = Fail2Ban Alert
action = %(action_mwl)s

[sshd]
enabled = true
port = 2222
logpath = %(sshd_log)s
maxretry = 3
bantime = 86400

This bans any IP that fails more than 3 times within 10 minutes for a full day. Start and enable fail2ban:

sudo systemctl start fail2ban
sudo systemctl enable fail2ban

Check the status:

sudo fail2ban-client status sshd

Step 6: Configure UFW Firewall

UFW (Uncomplicated Firewall) is Ubuntu's user-friendly wrapper around iptables. I use it to whitelist only the SSH port and any services I explicitly allow.

sudo apt install ufw -y
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 2222/tcp
sudo ufw enable

Verify the rules:

sudo ufw status verbose

Your output should show port 2222 (or 22 if you didn't change it) as ALLOW, with all other traffic denied.

Step 7: Monitor and Maintain

SSH hardening is a one-time setup, but monitoring is ongoing. I check logs weekly and keep SSH updated:

sudo tail -f /var/log/auth.log | grep sshd

This shows real-time SSH activity. You'll see failed attempts get logged and IPs banned by fail2ban.

Keep your system patched:

sudo apt update && sudo apt upgrade -y

I recommend running this monthly and enabling unattended-upgrades for security patches:

sudo apt install unattended-upgrades -y
sudo dpkg-reconfigure -plow unattended-upgrades

Bonus: SSH Config on Your Local Machine

To make connecting easier, add this to your local ~/.ssh/config:

Host my-vps
    HostName your.vps.ip.address
    User deploy
    Port 2222
    IdentityFile ~/.ssh/id_rsa_vps
    ServerAliveInterval 60
    ServerAliveCountMax 3

Now you can connect with just ssh my-vps instead of typing the full command.

Final Checklist

Before declaring your VPS secure, verify:

• ✓ Root login disabled
• ✓ Password authentication disabled
• ✓ SSH key deployed and tested
• ✓ Non-root user created and configured
• ✓ Fail2ban running and filtering
• ✓ UFW firewall enabled with SSH port open
• ✓ System fully updated
• ✓ SSH logs monitored regularly

With these steps in place, your VPS is dramatically more secure than the default configuration. Brute-force attacks will fail, compromised keys have passphrases to slow attackers, and suspicious activity is logged and auto-blocked. This is the foundation I build on for every production server—from hosting Nextcloud to running Docker stacks. Once SSH is locked down, everything else is infinitely easier to secure.

If you need reliable VPS infrastructure to practice these hardening steps, RackNerd's affordable KVM VPS plans are an excellent choice for homelabs and self-hosted projects. Start small, harden thoroughly, and scale confidently.