Every internet-facing Linux server runs SSH, and every one of those servers is being probed constantly. Within minutes of a new VPS spinning up, bots are already hammering port 22 trying default credentials. A default SSH configuration is functional but far from secure. This guide walks through the practical steps that meaningfully reduce your attack surface — from key-based authentication to fail2ban and AllowUsers restrictions.

Why Default SSH Is Risky

Out of the box, SSH allows:

Password authentication — vulnerable to brute force
Root login — if root is compromised, the attacker owns everything
All users to connect — no restriction on who can SSH in
Listening on port 22 — trivial for scanners to find

None of these defaults are catastrophic on their own, but together they create a surface that automated attack tools exploit constantly. Hardening SSH is one of the highest-leverage security improvements you can make.

Step 1: Set Up Key-Based Authentication

Key-based auth replaces passwords with a cryptographic key pair. Without the private key, password guessing is pointless.

On your local machine, generate an SSH key pair:

ssh-keygen -t ed25519 -C "your-label" -f ~/.ssh/id_ed25519

Ed25519 keys are preferred — they are smaller, faster, and more resistant to certain attacks than older RSA keys. Add a strong passphrase when prompted.

Copy your public key to the server:

ssh-copy-id -i ~/.ssh/id_ed25519.pub user@your-server-ip

Or do it manually:

# On the server
mkdir -p ~/.ssh
chmod 700 ~/.ssh
echo "YOUR_PUBLIC_KEY_HERE" >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

Verify key-based login works before disabling passwords:

ssh -i ~/.ssh/id_ed25519 user@your-server-ip

If you can connect, proceed to the next steps.

Step 2: Edit sshd_config

The SSH daemon configuration lives at /etc/ssh/sshd_config. Open it with your editor of choice (you need root):

sudo nano /etc/ssh/sshd_config

Apply these settings:

# Disable root login
PermitRootLogin no

# Disable password authentication
PasswordAuthentication no

# Disable empty passwords
PermitEmptyPasswords no

# Disable challenge-response (keyboard-interactive) auth
ChallengeResponseAuthentication no

# Only allow specific users
AllowUsers yourusername

# Use only SSH protocol 2
Protocol 2

# Change the default port (pick something above 1024)
Port 2222

# Limit authentication attempts per connection
MaxAuthTries 3

# Close idle sessions after 5 minutes
ClientAliveInterval 300
ClientAliveCountMax 0

# Disable X11 forwarding unless you need it
X11Forwarding no

# Disable TCP forwarding unless needed
AllowTcpForwarding no

# Ignore rhosts files
IgnoreRhosts yes

# Log level for auditing
LogLevel VERBOSE

AllowUsers vs AllowGroups

AllowUsers yourusername restricts SSH access to a named list of users. Even if an attacker somehow creates another system account, they cannot SSH in.

For multi-user setups, create a dedicated SSH group:

sudo groupadd sshusers
sudo usermod -aG sshusers yourusername

Then in sshd_config:

AllowGroups sshusers

After making changes, always test your config before restarting:

sudo sshd -t

This validates the syntax without applying changes. If it returns without errors, restart the daemon:

sudo systemctl restart sshd

Keep your current session open while testing a new connection in a second terminal. If the new connection fails, you still have access to fix the configuration.

Step 3: Change the Default Port

Changing from port 22 to a non-standard port does not stop a determined attacker, but it eliminates virtually all automated scanner noise. Most bots target port 22 exclusively.

The Port 2222 setting in sshd_config above handles this. After restarting SSH, connect with:

ssh -p 2222 user@your-server-ip

Update your ~/.ssh/config on the client to avoid typing the port every time:

Host myserver
    HostName your-server-ip
    User yourusername
    Port 2222
    IdentityFile ~/.ssh/id_ed25519

Now you can connect simply with ssh myserver.

Remember: if your server has a firewall, open the new port and close 22:

# UFW example
sudo ufw allow 2222/tcp
sudo ufw deny 22/tcp
sudo ufw reload

Step 4: Install and Configure fail2ban

fail2ban monitors log files and automatically bans IP addresses that show signs of brute-force attacks. Even with key-based auth, it provides a useful layer of defense.

sudo apt install fail2ban

Create a local jail configuration (do not edit jail.conf directly — it gets overwritten on updates):

sudo nano /etc/fail2ban/jail.local

[DEFAULT]
bantime  = 3600
findtime = 600
maxretry = 3

[sshd]
enabled  = true
port     = 2222
logpath  = %(sshd_log)s
backend  = %(sshd_backend)s

Key settings:

bantime: how long (seconds) to ban an offending IP — 3600 is one hour; increase for stricter banning
findtime: the window (seconds) in which maxretry failures trigger a ban
maxretry: number of failures before banning

Start and enable fail2ban:

sudo systemctl enable --now fail2ban

Check its status and view active bans:

sudo fail2ban-client status sshd
sudo fail2ban-client status sshd | grep "Banned IP"

Unban an IP if you accidentally locked yourself out:

sudo fail2ban-client set sshd unbanip YOUR_IP

Step 5: Use SSH Config for Multiple Keys

If you manage multiple servers, organize your keys:

Host vps-prod
    HostName 203.0.113.10
    User admin
    Port 2222
    IdentityFile ~/.ssh/id_ed25519_prod

Host vps-dev
    HostName 203.0.113.20
    User dev
    Port 2222
    IdentityFile ~/.ssh/id_ed25519_dev

Store your private keys with mode 600:

chmod 600 ~/.ssh/id_ed25519*

Step 6: Two-Factor Authentication (Optional, High Security)

For particularly sensitive servers, add TOTP (time-based one-time passwords) on top of key-based auth using libpam-google-authenticator:

sudo apt install libpam-google-authenticator
google-authenticator  # run as the user, not root

Then edit /etc/pam.d/sshd and add:

auth required pam_google_authenticator.so

And in sshd_config:

AuthenticationMethods publickey,keyboard-interactive
ChallengeResponseAuthentication yes

This requires both a valid private key and the TOTP code — two factors that an attacker would need to simultaneously compromise.

Hardened sshd_config Summary

Port 2222
Protocol 2
PermitRootLogin no
PasswordAuthentication no
PermitEmptyPasswords no
ChallengeResponseAuthentication no
AllowUsers yourusername
MaxAuthTries 3
X11Forwarding no
AllowTcpForwarding no
IgnoreRhosts yes
ClientAliveInterval 300
ClientAliveCountMax 0
LogLevel VERBOSE

SSH hardening is not about achieving perfection — it is about raising the cost of attack high enough that automated tools and casual attackers move on. Key-based authentication plus fail2ban plus a restricted AllowUsers list eliminates the vast majority of realistic SSH threats. Apply these settings to every server you manage.