How to Set Up SSH Keys for Secure Remote Server Access

SSH keys provide passwordless, encrypted access to remote servers using a public-private key pair. To set them up, generate a key pair on your local machine with `ssh-keygen -t ed25519`, copy the public key to your server using `ssh-copy-id user@server_ip`, and disable password authentication in `/etc/ssh/sshd_config` for enhanced security. This guide walks you through the entire process, from key generation to best practices for DevOps automation.

## What Are SSH Keys and Why Use Them?

SSH keys replace passwords with cryptographic authentication. A private key stays on your local machine (never shared), while the public key is installed on remote servers. When you connect, the server verifies your private key against the public key.

Benefits over passwords:

  • Stronger security – 2048-bit RSA or 256-bit Ed25519 keys resist brute-force attacks
  • Automation-ready – No manual password entry for scripts, CI/CD, or cron jobs
  • Convenience – Set once, use forever (until you rotate keys)
  • Audit trail – Each key identifies a specific user or machine

SSH key pair generation terminal screenshot with ssh-keygen command

Photo by Rafael Minguet Delgado on Pexels

## Prerequisites

Before you start, ensure you have:

  • Local machine: Linux, macOS, or Windows (with OpenSSH client)
  • Remote server: Linux (Ubuntu, CentOS, Debian, etc.) with SSH access
  • Server credentials: Root or sudo access to modify SSH configuration
  • Terminal access: Command-line interface on both machines

## Step-by-Step Guide: Setting Up SSH Keys

### Step 1: Generate Your SSH Key Pair

Run this command on your local machine:

“`bash

ssh-keygen -t ed25519 -C “[email protected]”

“`

What happens:

  • Creates a 256-bit Ed25519 key (faster and more secure than RSA)
  • Prompts for a save location (default: `~/.ssh/id_ed25519`)
  • Asks for a passphrase (recommended for extra security)

Pro tip: If your server doesn’t support Ed25519, use `ssh-keygen -t rsa -b 4096` instead.

### Step 2: Copy the Public Key to Your Server

The easiest method uses `ssh-copy-id`:

“`bash

ssh-copy-id -i ~/.ssh/id_ed25519.pub user@your_server_ip

“`

Manual alternative: If `ssh-copy-id` isn’t available, run:

“`bash

cat ~/.ssh/id_ed25519.pub | ssh user@your_server_ip “mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys”

“`

Verify the key was added:

“`bash

ssh user@your_server_ip “cat ~/.ssh/authorized_keys”

“`

SSH key being copied to remote server via ssh-copy-id command

Photo by Miguel Á. Padriñán on Pexels

### Step 3: Test the Connection

Try logging in without a password:

“`bash

ssh user@your_server_ip

“`

If you set a passphrase, you’ll be prompted for it once. For automated scripts, use an SSH agent to cache the passphrase:

“`bash

eval $(ssh-agent -s)

ssh-add ~/.ssh/id_ed25519

“`

## Securing Your SSH Server

### Disable Password Authentication

Once keys work, disable password logins to prevent brute-force attacks:

1. Edit the SSH daemon config:

“`bash

sudo nano /etc/ssh/sshd_config

“`

2. Set these parameters:

“`

PasswordAuthentication no

ChallengeResponseAuthentication no

PermitRootLogin prohibit-password

“`

3. Restart SSH:

“`bash

sudo systemctl restart sshd

“`

Warning: Keep a second terminal open with a root session in case you lock yourself out.

### Additional Security Hardening

Setting Recommended Value Purpose
Port 2222 (custom) Reduces automated attacks
MaxAuthTries 3 Limits login attempts
ClientAliveInterval 300 Disconnects idle sessions
AllowUsers your_username Whitelists specific users
PubkeyAuthentication yes Ensures key-only access

## Using SSH Keys for Automation and DevOps

### Automating Scripts with SSH Keys

For cron jobs, backup scripts, or CI/CD pipelines, use key-only authentication without passphrases:

“`bash

# Generate a dedicated key for automation

ssh-keygen -t ed25519 -f ~/.ssh/deploy_key -N “”

“`

Best practice: Create separate keys for different tasks:

  • `~/.ssh/github_deploy` – For Git operations
  • `~/.ssh/server_backup` – For backup scripts
  • `~/.ssh/ansible_key` – For configuration management

### Managing Multiple Keys with SSH Config

Create `~/.ssh/config` to simplify connections:

“`

Host prod-server

HostName 192.168.1.100

User deploy

IdentityFile ~/.ssh/prod_key

Port 2222

Host dev-server

HostName dev.example.com

User developer

IdentityFile ~/.ssh/dev_key

“`

Now connect with just: `ssh prod-server`

SSH config file showing multiple host configurations

Photo by Pixabay on Pexels

## Common Issues and Troubleshooting

### Permission Errors

Fix file permissions on the server:

“`bash

chmod 700 ~/.ssh

chmod 600 ~/.ssh/authorized_keys

chmod 600 ~/.ssh/id_ed25519

“`

### Key Rejected

Check the server’s `~/.ssh/authorized_keys` file:

  • Ensure the public key is on a single line
  • No extra spaces or line breaks
  • Verify the key matches your local `id_ed25519.pub`

### Connection Refused After Disabling Passwords

Use the recovery console or physical access to re-enable password authentication temporarily.

## Best Practices for DevOps Teams

  • Rotate keys every 90 days – Automate with a script or tool like HashiCorp Vault
  • Use short-lived certificates – SSH certificates (via Vault or Teleport) expire automatically
  • Monitor key usage – Check `/var/log/auth.log` for failed attempts
  • Never share private keys – Use `ssh-agent` forwarding instead of copying keys to servers
  • Backup keys securely – Store in a password manager or encrypted USB drive

## Conclusion

Setting up SSH keys is the single most important step to secure remote server access. You’ve learned how to generate keys, copy them to servers, harden your SSH configuration, and integrate keys into automation workflows. This eliminates password vulnerabilities while enabling efficient DevOps practices.

Ready to streamline your server management? Download ready-to-use DevOps scripts, automation tools, and security templates at freefile.store – your resource hub for developer productivity.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *