How to Set Up SSH Keys for Secure Remote Server Access
SSH keys provide passwordless, encrypted access to remote servers using a public-private key pair. To set them up, generate a key pair on your local machine with `ssh-keygen -t ed25519`, copy the public key to your server using `ssh-copy-id user@server_ip`, and disable password authentication in `/etc/ssh/sshd_config` for enhanced security. This guide walks you through the entire process, from key generation to best practices for DevOps automation.
## What Are SSH Keys and Why Use Them?
SSH keys replace passwords with cryptographic authentication. A private key stays on your local machine (never shared), while the public key is installed on remote servers. When you connect, the server verifies your private key against the public key.
Benefits over passwords:
- Stronger security – 2048-bit RSA or 256-bit Ed25519 keys resist brute-force attacks
- Automation-ready – No manual password entry for scripts, CI/CD, or cron jobs
- Convenience – Set once, use forever (until you rotate keys)
- Audit trail – Each key identifies a specific user or machine

Photo by Rafael Minguet Delgado on Pexels
## Prerequisites
Before you start, ensure you have:
- Local machine: Linux, macOS, or Windows (with OpenSSH client)
- Remote server: Linux (Ubuntu, CentOS, Debian, etc.) with SSH access
- Server credentials: Root or sudo access to modify SSH configuration
- Terminal access: Command-line interface on both machines
## Step-by-Step Guide: Setting Up SSH Keys
### Step 1: Generate Your SSH Key Pair
Run this command on your local machine:
“`bash
ssh-keygen -t ed25519 -C “[email protected]”
“`
What happens:
- Creates a 256-bit Ed25519 key (faster and more secure than RSA)
- Prompts for a save location (default: `~/.ssh/id_ed25519`)
- Asks for a passphrase (recommended for extra security)
Pro tip: If your server doesn’t support Ed25519, use `ssh-keygen -t rsa -b 4096` instead.
### Step 2: Copy the Public Key to Your Server
The easiest method uses `ssh-copy-id`:
“`bash
ssh-copy-id -i ~/.ssh/id_ed25519.pub user@your_server_ip
“`
Manual alternative: If `ssh-copy-id` isn’t available, run:
“`bash
cat ~/.ssh/id_ed25519.pub | ssh user@your_server_ip “mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys”
“`
Verify the key was added:
“`bash
ssh user@your_server_ip “cat ~/.ssh/authorized_keys”
“`

Photo by Miguel Á. Padriñán on Pexels
### Step 3: Test the Connection
Try logging in without a password:
“`bash
ssh user@your_server_ip
“`
If you set a passphrase, you’ll be prompted for it once. For automated scripts, use an SSH agent to cache the passphrase:
“`bash
eval $(ssh-agent -s)
ssh-add ~/.ssh/id_ed25519
“`
## Securing Your SSH Server
### Disable Password Authentication
Once keys work, disable password logins to prevent brute-force attacks:
1. Edit the SSH daemon config:
“`bash
sudo nano /etc/ssh/sshd_config
“`
2. Set these parameters:
“`
PasswordAuthentication no
ChallengeResponseAuthentication no
PermitRootLogin prohibit-password
“`
3. Restart SSH:
“`bash
sudo systemctl restart sshd
“`
Warning: Keep a second terminal open with a root session in case you lock yourself out.
### Additional Security Hardening
| Setting | Recommended Value | Purpose |
|---|---|---|
| Port | 2222 (custom) | Reduces automated attacks |
| MaxAuthTries | 3 | Limits login attempts |
| ClientAliveInterval | 300 | Disconnects idle sessions |
| AllowUsers | your_username | Whitelists specific users |
| PubkeyAuthentication | yes | Ensures key-only access |
## Using SSH Keys for Automation and DevOps
### Automating Scripts with SSH Keys
For cron jobs, backup scripts, or CI/CD pipelines, use key-only authentication without passphrases:
“`bash
# Generate a dedicated key for automation
ssh-keygen -t ed25519 -f ~/.ssh/deploy_key -N “”
“`
Best practice: Create separate keys for different tasks:
- `~/.ssh/github_deploy` – For Git operations
- `~/.ssh/server_backup` – For backup scripts
- `~/.ssh/ansible_key` – For configuration management
### Managing Multiple Keys with SSH Config
Create `~/.ssh/config` to simplify connections:
“`
Host prod-server
HostName 192.168.1.100
User deploy
IdentityFile ~/.ssh/prod_key
Port 2222
Host dev-server
HostName dev.example.com
User developer
IdentityFile ~/.ssh/dev_key
“`
Now connect with just: `ssh prod-server`

Photo by Pixabay on Pexels
## Common Issues and Troubleshooting
### Permission Errors
Fix file permissions on the server:
“`bash
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
chmod 600 ~/.ssh/id_ed25519
“`
### Key Rejected
Check the server’s `~/.ssh/authorized_keys` file:
- Ensure the public key is on a single line
- No extra spaces or line breaks
- Verify the key matches your local `id_ed25519.pub`
### Connection Refused After Disabling Passwords
Use the recovery console or physical access to re-enable password authentication temporarily.
## Best Practices for DevOps Teams
- Rotate keys every 90 days – Automate with a script or tool like HashiCorp Vault
- Use short-lived certificates – SSH certificates (via Vault or Teleport) expire automatically
- Monitor key usage – Check `/var/log/auth.log` for failed attempts
- Never share private keys – Use `ssh-agent` forwarding instead of copying keys to servers
- Backup keys securely – Store in a password manager or encrypted USB drive
## Conclusion
Setting up SSH keys is the single most important step to secure remote server access. You’ve learned how to generate keys, copy them to servers, harden your SSH configuration, and integrate keys into automation workflows. This eliminates password vulnerabilities while enabling efficient DevOps practices.
Ready to streamline your server management? Download ready-to-use DevOps scripts, automation tools, and security templates at freefile.store – your resource hub for developer productivity.