Every time I ssh into different servers, the dance begins: remembering which key pair belongs to which host, typing out the full -i /path/to/key flag, and occasionally facepalming when I inevitably supply the wrong key. This workflow is:

• Error-prone when managing multiple environments
• Time-consuming for frequent connections
• Downright frustrating for complex key hierarchies

The solution lives in your ~/.ssh/config file (create it if missing). This configuration file allows host-specific SSH settings, including automatic key selection:

# Basic syntax template
Host your_alias
  HostName actual.server.com
  User remote_username  
  IdentityFile ~/.ssh/id_special_key
  Port 2222  # Optional custom port

Let's examine real-world scenarios:

Production vs Staging Setup

Host production
  HostName 203.0.113.45
  User deploy
  IdentityFile ~/.ssh/prod_deploy_key
  
Host staging
  HostName staging.example.com  
  User devuser
  IdentityFile ~/.ssh/staging_ed25519

GitHub Specific Configuration

Host github.com
  User git
  IdentityFile ~/.ssh/github_rsa
  IdentitiesOnly yes  # Forces this key only

• Wildcard Hosts: Host *.aws.internal matches all AWS internal hosts
• Inheritance: Use Host * for global defaults
• Security: Set strict permissions: chmod 600 ~/.ssh/config
• Debugging: Add LogLevel DEBUG3 temporarily for connection issues

Test configuration with:

ssh -T git@github.com  # Tests GitHub connection
ssh -v production     # Verbose output for debugging

Remember that host aliases in ~/.ssh/config work with all SSH-related commands (scp, rsync, etc.).

Pro tip: Combine this with SSH agent (ssh-add) for maximum convenience without compromising security.

The simplest and most effective way to automate private key selection is by using the SSH client configuration file located at ~/.ssh/config. This file allows you to specify different configurations for different hosts or groups of hosts.

Here's how to specify which private key to use for a particular host:

Host myserver.example.com
    HostName myserver.example.com
    User myusername
    IdentityFile ~/.ssh/id_rsa_myserver
    IdentitiesOnly yes

The IdentitiesOnly yes directive tells SSH to only use the identity files explicitly configured, ignoring other keys that might be in your SSH agent.

You can use wildcards to match multiple hosts:

Host *.example.com
    User deploy
    IdentityFile ~/.ssh/id_rsa_example_deploy

For more complex setups, you can combine multiple options:

Host dev-server
    HostName 192.168.1.100
    User developer
    IdentityFile ~/.ssh/id_ed25519_dev
    Port 2222
    ForwardAgent yes
    ServerAliveInterval 60

Here's how you might organize keys for different environments:

# Production servers
Host prod-*
    User admin
    IdentityFile ~/.ssh/prod_key

# Staging servers
Host stage-*
    User stage-user
    IdentityFile ~/.ssh/stage_key

# Specific server override
Host prod-db01
    HostName db01.prod.example.com
    User dbadmin
    IdentityFile ~/.ssh/prod_db_key

Remember to set proper permissions for your SSH files:

chmod 600 ~/.ssh/config
chmod 600 ~/.ssh/id_*

If your configuration isn't working as expected, use the verbose flag to debug:

ssh -v user@hostname