Exposing ssh to the Internet

I’ve decided to expose ssh to the homelab server to the Internet. I had been afraid of doing so for months, but it’s about time and I think I’m ready.

As a background, my homelab server already exposes web services such as blog and Immich via Caddy. Caddy and Immich are running in docker containers on Ubuntu on Proxmox.

Before exposing ssh, I needed to fortify the server to protect it from bot and other malicious attacks.

Use a strong password

My password was weak but I was used to type it. I changed to a stronger password so that even if a hacker logged in, he/she would have hard time doing sudo.

passwd

Configure ssh settings

This step is the core of hardening ssh.

sudo vim /etc/ssh/sshd_config

Here, I changed to:

• Change ssh port from 22 to some random number between 1024 to 65535
• Disable root login
• Disable password login (ie, key login only)
• Disable keyboard-interactive auth

Port <port_number>
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no  # keep as no

Configure firewall

Ubuntu doesn’t use firewall by default. Enable it. Allow only the ssh port, 443/https and 80/http.

sudo ufw allow <ssh_port>/tcp  # allow ssh port
sudo ufw allow 443/tcp
sudo ufw allow 80/tcp
sudo ufw enable
sudo ufw reload
sudo ufw status verbose

You should see something like this:

Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), deny (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
<ssh_port>/tcp                  ALLOW IN    Anywhere
80/tcp                     ALLOW IN    Anywhere
443/tcp                    ALLOW IN    Anywhere
<ssh_port>/tcp (v6)             ALLOW IN    Anywhere (v6)
80/tcp (v6)                ALLOW IN    Anywhere (v6)
443/tcp (v6)               ALLOW IN    Anywhere (v6)

Open ssh port in the router

Then, open ssh port in the router. On my tplink router portal, go to Advanced > NAT Forwardig > Virtual Servers > Add.

Try ssh

ssh <user>@<external IP or hostname> -p <ssh_port>

If it worked, add config in ~/.ssh/config

Host <some_name>
     HostName blog.achiwa.co.uk
     Port <ssh_port>
     User <user_name>
     IdentityFile ~/.ssh/<private_keyfile>

Install and configure fail2ban

Further fortify ssh and Caddy with fail2ban. If you fail to ssh or login to Immich 5 times in 10 min, you’ll be banned for an hour (configurable).

Install fail2ban with apt.

sudo apt install fail2ban

Update a jail for ssh

ssh is covered by fail2ban by default, but create a new .local file to overwrite the setting.

sudo vim /etc/fail2ban/jail.local

[sshd]
enabled = true
port = <ssh_port>
logpath = /var/log/auth.log
maxretry = 5
findtime = 10m
bantime = 1h

Create a jail for Caddy

sudo vim /etc/fail2ban/jail.d/caddy.local

[caddy]
enabled = true
port = http,https
filter = caddy
logpath = /var/log/caddy/access.log
maxretry = 10
findtime = 10m
bantime = 1h

Create a Caddy filter

sudo vim /etc/fail2ban/filter.d/caddy.conf

[Definition]
failregex = <HOST> -.*"(GET|POST).*(wp-login|xmlrpc|admin|login).*" (401|403|404)
ignoreregex =

From this failregex setting, I understand that fail2ban doesn’t intercept incoming access to Caddy, but that it gets notified of an access attempt from the access log being updated.

Restart fail2ban

sudo systemctl restart fail2ban
sudo fail2ban-client status sshd

Well…

I also looked into cloudflare tunnel which is part of their free offering. It looked more secure as it doesn’t expose my external IP. But I decided to go without cloudflare as they might start charging the feature in the future. Maybe not, but who knows?

Let’s ask Copilot (AI) if I should do more. Here’s the prompt:

You are a linux security expert. I expose ssh, a static blog site and Immich on my homelab server. For the blog site and Immich, I use Caddy in docker. The server runs Ubuntu on Proxmox. As security measures, I did:

• Open 443, 80 and ssh_port at my wifi router
• Allow 443, 80 and ssh_port only with firewall on Ubuntu
• For ssh:
  • Change port to a random 5-digit number
  • Disable password login
  • Disable root login
• Configure fail2ban for ssh and caddy

What else should I do?

And here’s an excerpt from the response:

My honest take You’ve already done the essentials. The biggest next-level improvements are:

• Put SSH behind WireGuard or Tailscale
• Use Cloudflare as a reverse proxy
• Harden Docker (non-root, limited capabilities)
• Add monitoring + offsite backups

Maybe I’d want to go for monitoring next.