Managing SSH keys for multiple servers without losing your mind

One server, one key — easy. Ten servers, three usernames, a jump host, and a client's VPS? That's where key management either becomes a system or becomes a mess. Here's the system.

1 — Use ed25519, one key per identity (not per server)

ssh-keygen -t ed25519 -C "wes@workstation"

ed25519 is the modern default: shorter, faster, and stronger in practice than 2048-bit RSA. The -C comment matters — it's how you'll recognize the key in authorized_keys files years from now.

Per identity, not per server: one key for your workstation, one for the laptop, one for CI. If a laptop is lost, you revoke that key everywhere and the others keep working. Forty per-server keys sounds more secure but mostly guarantees you'll never rotate any of them.

2 — Copy the public key to each server

# Linux/macOS:
ssh-copy-id user@server

# Windows PowerShell (no ssh-copy-id):
type $env:USERPROFILE\.ssh\id_ed25519.pub | ssh user@server "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"

Then confirm passworded login is off for good measure (PasswordAuthentication no in sshd_config) — once every machine you need has your key.

3 — Let ~/.ssh/config remember everything

Host pi-lab
    HostName 192.168.1.42
    User pi

Host prod-web
    HostName 203.0.113.7
    User deploy
    IdentityFile ~/.ssh/id_ed25519_work

Host db-internal
    HostName 10.0.4.12
    User admin
    ProxyJump prod-web

Now ssh pi-lab just works — names instead of IPs, per-host users and keys, and ProxyJump for hosts behind a bastion. This file is the single best quality-of-life upgrade in stock SSH.

4 — Rotate and audit occasionally

• List what a server trusts: cat ~/.ssh/authorized_keys — those -C comments pay off here.
• Remove keys for machines you no longer own. Do it the day you retire the machine, not "someday."
• Passphrase-protect keys and let ssh-agent cache the unlock per session.

Where it still hurts — and the one-window answer

The config file remembers hosts, but it doesn't encrypt anything, doesn't handle the passwords legacy gear still demands, and doesn't sync to your other machine. AidaIDE (free forever) wraps this whole discipline in an AES-256 credential vault: hosts, users, keys, and passwords stored encrypted, one unlock per session, one click per connection — with tabs, the file browser, and a one-click command library on the same session. Your ~/.ssh/config knowledge maps straight onto it, jump hosts included.