Troubleshooting EC2 Instance: Services Accessible But ICMP Ping Fails – AWS Networking Deep Dive


2 views

When your EC2 instance responds to HTTP/SSH requests but remains silent to ping (ICMP Echo Requests), you're likely facing one of these scenarios:

# Check if ICMP is blocked at OS level (Linux example)
sudo iptables -L -n -v | grep icmp
# Sample output showing blocked ICMP:
# 0     0 DROP       icmp --  *      *       0.0.0.0/0            0.0.0.0/0

Unlike other services that use specific ports, ICMP operates at the network layer. AWS requires explicit ICMP rules:

# Correct security group rule for ICMPv4 (AWS CLI example)
aws ec2 authorize-security-group-ingress \
    --group-id sg-903004f8 \
    --protocol icmp \
    --port -1 \
    --cidr 0.0.0.0/0

Network ACLs operate at the subnet level and can override security group permissions. Verify both inbound and outbound rules:

# Example problematic Network ACL rules
100 | ALL Traffic | ALL | 0.0.0.0/0 | DENY
*   | ALL Traffic | ALL | 0.0.0.0/0 | ALLOW
# Rule 100 would block ICMP despite the allow-all catch-all

Modern Linux AMIs often come with strict default firewall settings. For CentOS/RHEL:

# Check and modify firewalld ICMP settings
sudo firewall-cmd --list-icmp-blocks
sudo firewall-cmd --remove-icmp-block=echo-request --permanent
sudo firewall-cmd --reload

When all else fails, VPC Flow Logs can reveal whether ICMP packets reach your instance:

# Create flow log for troubleshooting
aws ec2 create-flow-logs \
    --resource-type VPC \
    --resource-id vpc-1a2b3c4d \
    --traffic-type REJECT \
    --log-destination-type cloud-watch-logs

Windows Server instances have additional firewall considerations:

# PowerShell command to enable File and Printer Sharing (includes ICMP)
Enable-NetFirewallRule -DisplayGroup "File and Printer Sharing"

# Advanced ICMP configuration
netsh advfirewall firewall add rule name="ICMP Allow incoming V4 echo request" ^
protocol=icmpv4:8,any dir=in action=allow

Recently I encountered an interesting scenario where an EC2 instance was completely functional for all TCP-based services (SSH, HTTP, etc.) but wouldn't respond to ICMP ping requests. Here's my deep dive into troubleshooting this behavior.

The server showed these symptoms:

$ ping ec2-50-19-x-x.compute-1.amazonaws.com
PING ec2-50-19-x-x.compute-1.amazonaws.com (50.19.x.x): 56 data bytes
Request timeout for icmp_seq 0
...
--- 50.19.x.x ping statistics ---
6 packets transmitted, 0 packets received, 100.0% packet loss

Yet TCP services worked perfectly:

$ curl -I http://50.19.x.x
HTTP/1.1 302 Found
Server: TornadoServer/1.2.1

In AWS EC2, security groups act as virtual firewalls. The most common reason for ping failures is missing ICMP rules. Here's how to check:

# AWS CLI command to check security group rules
aws ec2 describe-security-groups --group-ids sg-xxxxxxxx --query 'SecurityGroups[0].IpPermissions'

A proper ICMP rule should look like:

{
    "IpProtocol": "icmp",
    "FromPort": 8,
    "ToPort": 0,
    "IpRanges": [{"CidrIp": "0.0.0.0/0"}]
}

If security groups are correct, check the instance's local firewall. For Linux:

# Check iptables rules
sudo iptables -L -n -v

# Common ICMP blocking rule that would cause this
-A INPUT -p icmp --icmp-type echo-request -j DROP

To temporarily enable ping responses:

sudo iptables -A INPUT -p icmp --icmp-type echo-request -j ACCEPT

Some AMIs disable ICMP responses at the kernel level. Check:

cat /proc/sys/net/ipv4/icmp_echo_ignore_all

A value of 1 means ICMP is being ignored. To enable:

echo 0 | sudo tee /proc/sys/net/ipv4/icmp_echo_ignore_all

When ping isn't available, use these TCP-based alternatives for monitoring:

# Check SSH availability
nc -zv 50.19.x.x 22

# HTTP check with curl
curl -s -o /dev/null -w "%{http_code}" http://50.19.x.x

For infrastructure-as-code users, here's a Terraform snippet to ensure ICMP is allowed:

resource "aws_security_group_rule" "allow_ping" {
  type        = "ingress"
  from_port   = 8
  to_port     = 0
  protocol    = "icmp"
  cidr_blocks = ["0.0.0.0/0"]
  security_group_id = aws_security_group.example.id
}