When migrating servers, preserving SSH host keys is crucial to prevent the dreaded "WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED" message that clients receive when connecting to what appears to be a different server. While the IP change is unavoidable, we can maintain cryptographic continuity by properly transferring the host keys.

On a typical Linux system, these are the essential host key files you need to move:

/etc/ssh/ssh_host_rsa_key
/etc/ssh/ssh_host_rsa_key.pub
/etc/ssh/ssh_host_ecdsa_key
/etc/ssh/ssh_host_ecdsa_key.pub
/etc/ssh/ssh_host_ed25519_key
/etc/ssh/ssh_host_ed25519_key.pub

Here's the complete procedure I've successfully used in production environments:

1. Backup original keys on source server:

sudo tar -czf ssh_host_keys.tar.gz /etc/ssh/ssh_host_*_key*

2. Transfer the archive to new server:

scp ssh_host_keys.tar.gz user@newserver:/tmp/

3. On destination server (Ubuntu 10.04):

sudo mv /etc/ssh/ssh_host_* /etc/ssh/ssh_host_*.bak  # Backup existing keys
sudo tar -xzf /tmp/ssh_host_keys.tar.gz -C /etc/ssh/
sudo chown root:root /etc/ssh/ssh_host_*_key*
sudo chmod 600 /etc/ssh/ssh_host_*_key
sudo chmod 644 /etc/ssh/ssh_host_*_key.pub
sudo service ssh restart

The move from RHEL 5 to Ubuntu 10.04 introduces some considerations:

• Key format compatibility is generally good between these versions
• Ubuntu 10.04 may support newer key types (like ED25519) that RHEL 5 didn't have
• File permissions and ownership must be verified after transfer

On a client machine that previously connected to the old server, run:

ssh-keygen -F new.server.ip

The output should show the same fingerprint as the old server. Alternatively:

ssh -o VisualHostKey=yes user@new.server.ip

The ASCII art fingerprint should match what clients previously saw.

If you can't transfer the original host keys, you can pre-populate the new server's keys in client known_hosts files:

# On new server:
ssh-keygen -lf /etc/ssh/ssh_host_rsa_key.pub

# On clients:
ssh-keygen -R old.server.ip
ssh-keygen -R new.server.ip
ssh-keyscan -t rsa new.server.ip >> ~/.ssh/known_hosts

Problem: Clients still see host key warnings
Solution: Verify the transferred keys have correct permissions (600 for private keys, 644 for public)

Problem: SSH service fails to start
Solution: Check auth.log for errors - may need to regenerate any missing key types with dpkg-reconfigure openssh-server

ssh_host_rsa_key
ssh_host_rsa_key.pub
ssh_host_dsa_key
ssh_host_dsa_key.pub
ssh_host_ecdsa_key
ssh_host_ecdsa_key.pub
ssh_host_ed25519_key
ssh_host_ed25519_key.pub

# Compress all host keys
tar czvf ssh_host_keys.tar.gz /etc/ssh/ssh_host_*

# Stop SSH service first
sudo service ssh stop

# Backup existing keys
sudo mv /etc/ssh/ssh_host_* /root/ssh_backup/

# Transfer and extract keys (via SCP/sftp)
scp user@old_server:/path/to/ssh_host_keys.tar.gz .
sudo tar xzvf ssh_host_keys.tar.gz -C /etc/ssh/

# Set proper permissions
sudo chmod 600 /etc/ssh/ssh_host_*key
sudo chmod 644 /etc/ssh/ssh_host_*pub
sudo chown root:root /etc/ssh/ssh_host_*

# Restart SSH
sudo service ssh start

# On client machine, verify fingerprint matches:
ssh-keygen -l -f /etc/ssh/ssh_host_rsa_key.pub

# Compare with what server presents:
ssh-keyscan new.server.ip | ssh-keygen -lf -