When working with dynamically provisioned EC2 instances, the SSH key verification system becomes a double-edged sword. Every time you terminate and recreate an instance with the same hostname or IP, you're greeted with that dreaded warning:
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
The SSH client maintains a ~/.ssh/known_hosts
file that stores host keys for verification. When you recreate an EC2 instance:
- A new SSH host key pair is generated
- The IP/hostname remains the same but points to new hardware
- SSH detects a mismatch between stored and presented keys
While ssh-keygen -R
removes entries by hostname, it doesn't handle these cases effectively:
- Multiple entries for the same IP (IPv4 and IPv6)
- DNS names that resolve to the same IP
- Hashed hostname entries (default in newer OpenSSH versions)
For scripting environments, here are robust approaches:
Method 1: Purge All Related Entries
#!/bin/bash
HOST="ec2-54-123-45-67.compute-1.amazonaws.com"
IP="54.123.45.67"
# Remove by hostname and IP
ssh-keygen -R "$HOST" &>/dev/null
ssh-keygen -R "$IP" &>/dev/null
# Nuclear option for hashed entries
sed -i "/$HOST/d" ~/.ssh/known_hosts
sed -i "/$IP/d" ~/.ssh/known_hosts
Method 2: Temporary Disable Strict Checking
For one-off connections in scripts:
ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null user@host
Method 3: AWS-Specific Solution
Using AWS CLI to get instance metadata:
INSTANCE_ID="i-0123456789abcdef"
HOST=$(aws ec2 describe-instances --instance-ids $INSTANCE_ID \
--query 'Reservations[0].Instances[0].PublicDnsName' --output text)
IP=$(aws ec2 describe-instances --instance-ids $INSTANCE_ID \
--query 'Reservations[0].Instances[0].PublicIpAddress' --output text)
# Clean up existing entries
ssh-keygen -f ~/.ssh/known_hosts -R "$HOST"
ssh-keygen -f ~/.ssh/known_hosts -R "$IP"
- Use EC2 Instance Connect for temporary access
- Consider maintaining a dedicated jump host with persistent keys
- Implement SSH certificate authority for dynamic environments
- Store known_hosts in memory for containerized environments:
# Dockerfile example
RUN echo "StrictHostKeyChecking no" >> /etc/ssh/ssh_config
For predictable environments, bake host keys into AMIs:
# In your Packer configuration
"provisioners": [
{
"type": "shell",
"inline": [
"ssh-keygen -A",
"sudo cp /etc/ssh/ssh_host_* /tmp/"
]
}
]
When managing dynamic EC2 instances that get frequently rebuilt, SSH host key conflicts become inevitable. The standard warning about changed host identification - while important for security - creates operational friction in automated environments.
The error persists because modern OpenSSH stores two separate entries for each host:
1. Hostname-based entry (e.g., ec2-203-0-113-25.compute-1.amazonaws.com)
2. IP-based entry (e.g., 203.0.113.25)
This one-liner removes all traces of a host from known_hosts:
ssh-keygen -R "hostname" && ssh-keygen -R "ip_address"
For AWS environments, integrate this into your instance creation script:
#!/bin/bash
INSTANCE_IP=$(aws ec2 run-instances ... | jq -r '.Instances[0].PublicIpAddress')
INSTANCE_DNS=$(aws ec2 describe-instances --instance-ids $INSTANCE_ID | jq -r '.Reservations[0].Instances[0].PublicDnsName')
# Clean previous entries
ssh-keygen -R "$INSTANCE_DNS" &>/dev/null
ssh-keygen -R "$INSTANCE_IP" &>/dev/null
# Force new key acceptance
ssh -o StrictHostKeyChecking=accept-new ubuntu@$INSTANCE_IP
Create more robust handling via ~/.ssh/config
:
Host *.amazonaws.com
StrictHostKeyChecking accept-new
UserKnownHostsFile ~/.ssh/aws_hosts
LogLevel ERROR
While automating, maintain security by:
- Using separate known_hosts files for dynamic environments
- Setting appropriate expiration with
ssh-keyscan -T 30
- Implementing host certificate authentication where possible