Linux Multi-Homing: Configuring Multiple Network Interfaces with DHCP on Same Subnet for SSL/TLS Hosting


2 views

When implementing multiple SSL/TLS sites on a single Linux host, many administrators face network policy constraints. The standard approach of IP aliasing (e.g., eth0:0) often conflicts with:

  • DHCP snooping implementations
  • MAC-IP binding security policies
  • Switch port configuration limitations

Adding a second virtual interface appears straightforward:

# Add virtual interface (KVM example)
virsh attach-interface --domain vm01 --type network --source br0 --model virtio --config --live

However, you'll encounter asymmetric routing issues where only the primary interface responds. The root causes include:

  • Default ARP behavior answering from all interfaces
  • Missing source-based routing policies
  • Incomplete route tables for multiple egress paths

First, enable strict ARP filtering:

# Temporary setting
sysctl -w net.ipv4.conf.all.arp_filter=1

# Persistent configuration
echo "net.ipv4.conf.all.arp_filter = 1" >> /etc/sysctl.conf
sysctl -p

This ensures each interface only responds to ARP requests for its assigned IP.

For a subnet 10.0.0.0/24 with gateway 10.0.0.1:

1. Define custom routing tables

# /etc/iproute2/rt_tables
1    eth0_table
2    eth1_table

2. Configure interface-specific routes

# For eth0 (10.0.0.100)
ip route add 10.0.0.0/24 dev eth0 src 10.0.0.100 table eth0_table
ip route add default via 10.0.0.1 dev eth0 table eth0_table

# For eth1 (10.0.0.101)  
ip route add 10.0.0.0/24 dev eth1 src 10.0.0.101 table eth1_table
ip route add default via 10.0.0.1 dev eth1 table eth1_table

3. Add policy routing rules

ip rule add from 10.0.0.100 lookup eth0_table
ip rule add from 10.0.0.101 lookup eth1_table

# Main table fallback (DHCP may provide this)
ip route add default via 10.0.0.1 dev eth0

Verify the configuration:

# Check ARP filtering
cat /proc/sys/net/ipv4/conf/all/arp_filter

# Test interface-specific connectivity
ping -I eth0 10.0.0.1
ping -I eth1 10.0.0.1

# Examine routing decisions
ip route get 8.8.8.8 from 10.0.0.100
ip route get 8.8.8.8 from 10.0.0.101

While this solution works for same-subnet communication, inter-subnet routing may require:

  • Additional route entries for remote networks
  • Firewall markings (fwmark) for complex scenarios
  • Network admin cooperation for asymmetric return paths

In many cases, negotiating MAC-address exceptions for IP aliasing proves more maintainable than complex multi-interface configurations.

In modern network environments, there are legitimate cases where a Linux host needs multiple IP addresses on the same subnet - for example when hosting multiple SSL/TLS sites with different IPs. The traditional approach of IP aliasing (eth0:0) often conflicts with enterprise network security policies:

  • DHCP snooping preventing static IP assignments
  • MAC-IP binding that only allows one IP per MAC address
  • Switch configurations that enforce strict address policies

Adding a second network interface seemed like the logical solution:

# Adding a virtual NIC in KVM
virsh attach-interface --domain vm01 --type network --source br0 --model virtio --config --live

However, we encountered an issue where only the first interface would respond to network traffic despite both interfaces getting DHCP leases.

After extensive testing, here's the complete configuration that works:

1. Enable ARP Filtering

# Temporary setting
sysctl -w net.ipv4.conf.all.arp_filter=1

# Permanent configuration
echo "net.ipv4.conf.all.arp_filter = 1" >> /etc/sysctl.conf
sysctl -p

2. Configure Source-Based Routing

Assuming:

  • Subnet: 10.0.0.0/24
  • Gateway: 10.0.0.1
  • eth0 IP: 10.0.0.100
  • eth1 IP: 10.0.0.101
# Add custom routing tables
echo "1 eth0" >> /etc/iproute2/rt_tables
echo "2 eth1" >> /etc/iproute2/rt_tables

# Configure routes for each table
ip route add default via 10.0.0.1 table eth0
ip route add default via 10.0.0.1 table eth1
ip route add 10.0.0.0/24 dev eth0 src 10.0.0.100 table eth0
ip route add 10.0.0.0/24 dev eth1 src 10.0.0.101 table eth1

# Add routing rules
ip rule add from 10.0.0.100 table eth0
ip rule add from 10.0.0.101 table eth1

# Main table routes (may be set by DHCP)
ip route add default via 10.0.0.1 dev eth0
ip route add 10.0.0.0/24 dev eth0 src 10.0.0.100
ip route add 10.0.0.0/24 dev eth1 src 10.0.0.101

To verify the configuration works:

# Test basic connectivity
ping -I eth0 10.0.0.1
ping -I eth1 10.0.0.1

# Check routing decisions
ip route get 8.8.8.8 from 10.0.0.100
ip route get 8.8.8.8 from 10.0.0.101

# Verify ARP responses
arping -I eth0 10.0.0.100
arping -I eth1 10.0.0.101

For more complete isolation, consider using network namespaces:

# Create namespaces
ip netns add ns1
ip netns add ns2

# Move interfaces to namespaces
ip link set eth0 netns ns1
ip link set eth1 netns ns2

# Configure each namespace independently
ip netns exec ns1 ifconfig eth0 10.0.0.100/24 up
ip netns exec ns1 route add default gw 10.0.0.1
ip netns exec ns2 ifconfig eth1 10.0.0.101/24 up
ip netns exec ns2 route add default gw 10.0.0.1

While this solution works, it's important to note that enterprise environments often have good reasons for their security policies. In our case, after demonstrating the technical need, the network team relaxed the MAC-IP binding rules allowing us to use the simpler IP aliasing approach:

# Simple IP alias configuration
ifconfig eth0:0 10.0.0.101 netmask 255.255.255.0 up

Always maintain good relationships with your network team - sometimes the technical solution isn't as important as the human one.