“Troubleshooting TTL Exceeded and Host Unreachable Errors in Multi-Subnet XenServer Configuration”


25 views

When working with XenServer 6.0.2 configured with multiple NICs across different subnets, you might encounter two distinct ICMP error messages:

Destination Host Unreachable
From 87.117.211.46 icmp_seq=1 Time to live exceeded

The first error indicates layer 3 connectivity issues within the subnet, while the TTL exceeded message suggests routing loops or misconfiguration when trying to reach beyond the local network segment.

For a XenServer host with three NICs (eth0, eth1, eth2) serving different subnets, let's examine a typical configuration that might cause these issues:

# /etc/sysconfig/network-scripts/ifcfg-eth2
DEVICE=eth2
BOOTPROTO=static
IPADDR=87.117.211.46
NETMASK=255.255.255.0
GATEWAY=87.117.211.1
ONBOOT=yes

The critical elements to verify are:

  • Correct IP assignment matching the subnet
  • Proper gateway configuration
  • Routing table entries

First, check the routing table from the XenServer host:

ip route show
route -n

Common problematic outputs might show:

87.117.211.0/24 dev eth2 proto kernel scope link src 87.117.211.46
default via 87.117.221.1 dev eth0

This indicates that while eth2 has its local route, the default gateway is set through eth0 - a potential source of the routing loop.

The "Time to live exceeded" message typically appears when:

  1. A packet gets caught in a routing loop
  2. The TTL counter reaches zero before reaching destination
  3. Asymmetric routing paths exist

In our case, the likely scenario is that packets from the third subnet try to exit through the wrong interface, causing them to bounce between routers until TTL expires.

Here's how to properly configure multi-subnet routing in XenServer:

# Create separate routing tables for each subnet
echo "201 subnet3" >> /etc/iproute2/rt_tables

# Add rules for the new routing table
ip rule add from 87.117.211.46/32 table subnet3
ip rule add to 87.117.211.46/32 table subnet3

# Add routes to the new table
ip route add 87.117.211.0/24 dev eth2 src 87.117.211.46 table subnet3
ip route add default via 87.117.211.1 dev eth2 table subnet3

# Make changes persistent
echo "ip rule add from 87.117.211.46/32 table subnet3" >> /etc/rc.local
echo "ip rule add to 87.117.211.46/32 table subnet3" >> /etc/rc.local

After implementing the solution, verify with:

# Check specific route table
ip route show table subnet3

# Test connectivity
traceroute -n 8.8.8.8
mtr --report 87.117.221.17

You should see traffic from the third subnet properly exiting through its designated gateway without TTL issues.

For XenServer environments, also check:

  • Xen network bridges (xenbr0, xenbr1, xenbr2) configuration
  • VM interfaces assigned to correct bridges
  • Firewall rules (iptables/nftables) that might block ICMP
# Example bridge verification
brctl show
ovs-vsctl show

In a XenServer 6.0.2 environment with three NICs assigned to separate subnets, we're observing an interesting connectivity issue. The first two interfaces function normally with internet access, while the third subnet exhibits gateway communication problems.

# Current problematic ping results:
$ ping 87.117.221.17
PING 87.117.221.17 (87.117.221.17) 56(84) bytes of data.
From 87.117.211.46 icmp_seq=1 Time to live exceeded

The key observations are:

  • Intra-subnet communication works (hosts can ping each other)
  • Direct gateway pings return "Destination Host Unreachable"
  • External gateway pings show "Time to live exceeded"

First, let's verify the interface configuration on the XenServer host:

# Check interface assignments
xe pif-list host-name-label= params=uuid,device,network-uuid

# Verify network configurations
xe network-list params=uuid,name-label

# Example output parsing script:
for pif in $(xe pif-list --minimal); do
  echo "PIF: $pif"
  xe pif-param-get uuid=$pif param-name=device
  xe pif-param-get uuid=$pif param-name=network-uuid
  xe network-param-get uuid=$(xe pif-param-get uuid=$pif param-name=network-uuid) param-name=name-label
done

1. Routing Loop Detection

The "Time to live exceeded" message typically indicates packets are looping between routers. This suggests a misconfiguration in either:

  • The gateway's routing table for this subnet
  • The XenServer host's network configuration
  • Intermediate network devices

2. Gateway Configuration Verification

On the gateway device (likely a router), verify:

# Cisco example:
show ip route 87.117.211.0
show running-config interface 

# Linux router example:
ip route show table all | grep 87.117.211
iptables -L -n -v | grep 87.117.211

3. XenServer Network Stack Check

Examine the Dom0 network configuration:

# Check active routes
route -n

# Verify interface settings
ifconfig eth2  # Assuming eth2 is the problematic interface

# Check ARP table for gateway
arp -an | grep

For deeper analysis, consider packet capturing:

# On XenServer host:
tcpdump -i eth2 -nn -v icmp

# On gateway (if accessible):
tcpdump -i  -nn -v host 87.117.211.46

Here's a sample correct configuration for the third interface:

# /etc/network/interfaces on XenServer
auto eth2
iface eth2 inet static
    address 87.117.211.46
    netmask 255.255.255.0
    gateway 87.117.211.1
    dns-nameservers 8.8.8.8 8.8.4.4
    post-up route add -net 87.117.211.0 netmask 255.255.255.0 dev eth2