Best Practices for Secure SSH Key Management in Small Dev Teams (5-50 Servers)


2 views

Many teams fall into the trap of using a single shared SSH key across all members - typically deployed to the ~/.ssh/authorized_keys file of a common admin account (like ubuntu or deploy). This creates several security blind spots:

# Bad practice example (shared key in authorized_keys):
ssh-rsa AAAAB3NzaC1yc2E... team-shared-key@company.com
  • No individual accountability: All actions logged as coming from the same identity
  • Key rotation nightmares: Compromising one device means regenerating keys everywhere
  • Termination risks: Former employees retain access until manual key removal

The optimal approach combines personal keys with infrastructure automation. Here's how we implement this:

# Good practice: Per-developer keys with comment identifiers
ssh-rsa AAAAB3NzaC1yc2E... jane.doe@company.com-laptop-macbook-pro
ssh-rsa AAAAB3NzaC1yc2E... john.smith@company.com-desktop-ubuntu

Key implementation steps:

  1. Each developer generates keys per device using ssh-keygen -t ed25519 -C "identifier"
  2. Keys are submitted via pull request to an infrastructure repository
  3. Configuration management tools deploy authorized_keys

Using Ansible as an example for automated key deployment:

# roles/ssh_keys/tasks/main.yml
- name: Deploy user SSH keys
  ansible.posix.authorized_key:
    user: "{{ remote_user }}"
    state: present
    key: "{{ item }}"
  loop: "{{ ssh_keys }}"
  tags: ssh_keys

# group_vars/all.yml
ssh_keys:
  - "ssh-ed25519 AAAAC3Nz... jane.laptop"
  - "ssh-ed25519 AAAAC3Nz... john.workstation"

For break-glass scenarios, implement a separate emergency key pair:

# Emergency access workflow
1. Key stored in encrypted vault (Hashicorp Vault/AWS KMS)
2. Requires MFA for retrieval
3. Automatically expires after 4 hours
4. Triggers alert when used

Create a cron job that forces quarterly rotations:

#!/bin/bash
# /etc/cron.quarterly/ssh_key_rotation
for user in /home/*; do
  if [ -f "$user/.ssh/authorized_keys" ]; then
    echo "Rotating keys for $(basename $user)"
    mv "$user/.ssh/authorized_keys" "$user/.ssh/authorized_keys.old"
    sendmail -t <

Implement centralized logging of SSH sessions:

# /etc/ssh/sshd_config
Match Group developers
  ForceCommand logger -p auth.info -t ssh-session
  X11Forwarding no
  AllowTcpForwarding no
  PermitTTY no

Combine this with tools like auditd for complete visibility:

# Monitor SSH key modifications
-w /etc/ssh/sshd_config -p wa -k sshd_config
-w /home/*/.ssh/ -p wa -k ssh_folder

Sharing a single SSH key across the team is equivalent to leaving your server room door unlocked with a sign saying "WELCOME HACKERS". When a team member leaves or a laptop gets stolen, you'd need to rotate keys across all servers - a nightmare scenario. Instead, implement individual key pairs:

# Bad practice (shared key)
ssh-rsa AAAAB3NzaC1yc2E... team_key@company

# Good practice (individual keys)
ssh-rsa AAAAB3NzaC1yc2E... alice@company
ssh-rsa AAAAB3NzaC1yc2E... bob@company

While using shared system accounts (like 'ubuntu') might seem convenient, it destroys audit trails. Create individual OS accounts with sudo privileges, then map SSH keys to them:

# On the server:
sudo adduser alice --ingroup developers
sudo mkdir /home/alice/.ssh
sudo chmod 700 /home/alice/.ssh

For sudo access without password:

# In /etc/sudoers.d/developers
%developers ALL=(ALL) NOPASSWD: ALL

Each team member should generate separate keys for their devices, allowing precise key revocation when needed:

# Generating device-specific keys
ssh-keygen -t ed25519 -f ~/.ssh/alice-macbook -C "alice@company-macbook"
ssh-keygen -t ed25519 -f ~/.ssh/alice-thinkpad -C "alice@company-thinkpad"

Automate key deployment using infrastructure-as-code tools. Here's an Ansible playbook snippet:

-
  name: Deploy authorized keys
  hosts: all_servers
  tasks:
    -
      name: Ensure .ssh directory exists
      file:
        path: /home/{{ item }}/.ssh
        state: directory
        mode: '0700'
        owner: "{{ item }}"
    -
      name: Deploy public keys
      authorized_key:
        user: "{{ item }}"
        key: "{{ lookup('file', 'keys/' + item + '.pub') }}"
      loop: "{{ users }}"

Maintain emergency access through encrypted, time-limited mechanisms. For AWS environments, use EC2 Instance Connect:

# Temporary SSH certificate (30-minute validity)
aws ec2-instance-connect send-ssh-public-key \
  --instance-id i-1234567890abcdef0 \
  --availability-zone us-west-2a \
  --instance-os-user ec2-user \
  --ssh-public-key file://breakglass.pub

Implement scheduled key rotation using a combination of SSH certificates and your CI/CD pipeline:

# Certificate authority setup (simplified)
ssh-keygen -t rsa -b 4096 -f ca_key
ssh-keygen -s ca_key -I user_identity -n alice -V +1d alice.pub

Enhance SSH logging in /etc/ssh/sshd_config:

LogLevel VERBOSE
PrintMotd no
PrintLastLog no
SyslogFacility AUTH

Process logs with Fail2Ban or Wazuh for real-time alerts on suspicious activity.

For high-security environments, consider YubiKey or Nitrokey integration:

# YubiKey PIV configuration
ssh-keygen -D /usr/local/lib/opensc-pkcs11.so -e \
  | sudo tee -a /home/alice/.ssh/authorized_keys