Debugging OpenVPN Routing: Why ICMP Packets Reach tun0 But Fail on eth0 Forwarding in Multi-Subnet AWS VPC


3 views

When troubleshooting VPN connectivity issues, packet tracing reveals the exact failure point. In this case:

# Client side (10.10.0.6)
ping 10.10.146.8 
# Hangs with no response

# Server side capture shows packets reaching tun0
tcpdump -i tun0 icmp
10:23:45.112 IP 10.10.0.6 > 10.10.146.8: ICMP echo request
10:23:46.118 IP 10.10.0.6 > 10.10.146.8: ICMP echo request

# But target host (10.10.146.8) sees nothing:
tcpdump -i eth0 icmp
# Empty output

Amazon VPC requires explicit route table configurations for inter-subnet routing. The standard Linux forwarding checks look good:

sysctl net.ipv4.ip_forward
# Returns 1 (enabled)

iptables -L FORWARD
# Shows ACCEPT policy

But AWS needs these additional configurations:

1. VPC Route Table Configuration

Add a route in your VPC route table directing 10.10.0.0/24 traffic to your OpenVPN server instance:

Destination      Target
10.10.0.0/24    instance-id-of-openvpn-server

2. Source/Destination Check Disable

On the OpenVPN server EC2 instance:

aws ec2 modify-instance-attribute --instance-id YOUR_INSTANCE_ID --source-dest-check "{\"Value\": false}"

3. Enhanced OpenVPN Server Config

Add these directives to your server.conf:

push "route 10.10.146.0 255.255.255.0"
client-to-client
keepalive 10 120
comp-lzo no

After applying changes, test with:

# On client
traceroute 10.10.146.8

# On target host
tcpdump -i eth0 -nn icmp
# Should now show incoming packets

For persistent configurations, consider adding these iptables rules:

iptables -t nat -A POSTROUTING -s 10.10.0.0/24 -o eth0 -j MASQUERADE
iptables -A FORWARD -i tun0 -o eth0 -s 10.10.0.0/24 -j ACCEPT
iptables -A FORWARD -i eth0 -o tun0 -d 10.10.0.0/24 -j ACCEPT

Check these often-overlooked AWS settings:

  • Security groups must allow ALL traffic between VPN subnets
  • Network ACLs shouldn't block VPN traffic (UDP 1194 by default)
  • Ensure the target subnet's route table has a route back to the VPN subnet

When testing connectivity through an OpenVPN routed (non-bridged) setup between subnets 10.10.145.0/24 and 10.10.146.0/24, we observe:

# Client-side ping attempt
ping 10.10.146.8
PING 10.10.146.8 (10.10.146.8) 56(84) bytes of data.

Server-side tcpdump reveals packets reaching tun0 but failing to traverse to the destination host:

# tun0 interface shows incoming ICMP
tcpdump -i tun0 'icmp[icmptype] = icmp-echo'
10.10.0.6 > 10.10.146.8: ICMP echo request

# eth0 shows outgoing ICMP but no reply
tcpdump -i eth0 'icmp[icmptype] = icmp-echo'
10.10.0.6 > 10.10.146.8: ICMP echo request

Current routing tables show:

# Client routes
10.10.0.1       10.10.0.5       255.255.255.255 UGH       0 0          0 tun0
10.10.146.0     10.10.0.5       255.255.255.0   UG        0 0          0 tun0

# Server routes
10.10.0.0       10.10.0.2       255.255.255.0   UG        0 0          0 tun0
10.10.145.0     0.0.0.0         255.255.255.0   U         0 0          0 eth0

IP forwarding is enabled but potential NAT issues exist:

sysctl -a | grep forwarding
net.ipv4.conf.all.forwarding = 1
net.ipv4.conf.tun0.forwarding = 1

iptables -L FORWARD
Chain FORWARD (policy ACCEPT)

Add these configurations to your OpenVPN server:

# Enable IP forwarding persistently
echo "net.ipv4.ip_forward=1" >> /etc/sysctl.conf
sysctl -p

# Add NAT masquerade rule
iptables -t nat -A POSTROUTING -s 10.10.0.0/24 -o eth0 -j MASQUERADE
iptables-save > /etc/iptables.rules

# For systemd systems:
echo 'iptables-restore < /etc/iptables.rules' > /etc/network/if-pre-up.d/iptables
chmod +x /etc/network/if-pre-up.d/iptables

For AWS environments, ensure:

1. Security groups allow all traffic between the subnets
2. Route tables in VPC include proper routes
3. Source/destination check is disabled for the VPN server instance:
aws ec2 modify-instance-attribute --instance-id i-1234567890abcdef0 --no-source-dest-check

After implementing changes:

# On server:
tcpdump -i eth0 host 10.10.146.8 and icmp

# On target host:
tcpdump -i eth0 host 10.10.0.6 and icmp

# Expected output should now show bidirectional ICMP traffic

When debugging VPN routing issues:

1. Verify physical connectivity
2. Check routing tables on all hops
3. Inspect firewall rules (iptables/nftables)
4. Validate VPN configuration files
5. Test with simplest possible ruleset
6. Examine VPC/cloud provider network ACLs