Linux Multi-ISP Load Balancing with NAT: Dual WAN Routing & Traffic Distribution on Debian


2 views

When dealing with multiple ISP connections, we need to establish proper routing tables and NAT rules. Your current setup with three interfaces (one LAN and two WANs) is common for dual-homed networks. The key challenges are:

  • Maintaining separate routing tables for each ISP
  • Implementing NAT for outgoing connections
  • Distributing traffic intelligently between both paths

First, ensure your kernel has these features enabled:

CONFIG_NET_MULTIPATH=y
CONFIG_IP_ADVANCED_ROUTER=y
CONFIG_IP_MULTIPLE_TABLES=y
CONFIG_IP_ROUTE_MULTIPATH=y
CONFIG_NETFILTER=y
CONFIG_NF_NAT=y

We'll create two custom routing tables (one for each ISP):

# /etc/iproute2/rt_tables
100 isp1
200 isp2

Configure the routing tables:

ip route add default via 192.168.1.1 dev eth1 table isp1
ip route add 192.168.1.0/24 dev eth1 src 192.168.1.x table isp1
ip rule add from 192.168.1.x table isp1

ip route add default via 192.168.2.1 dev eth2 table isp2
ip route add 192.168.2.0/24 dev eth2 src 192.168.2.x table isp2
ip rule add from 192.168.2.x table isp2

The NAT configuration requires careful handling to match traffic with the correct outgoing interface:

iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE
iptables -t nat -A POSTROUTING -o eth2 -j MASQUERADE

For basic round-robin load balancing between ISPs:

ip route add default scope global nexthop via 192.168.1.1 dev eth1 weight 1 \
nexthop via 192.168.2.1 dev eth2 weight 1

For more sophisticated traffic distribution, we can use iptables marks:

iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
iptables -t mangle -A PREROUTING -m mark ! --mark 0 -j ACCEPT
iptables -t mangle -A PREROUTING -m statistic --mode random --probability 0.5 -j MARK --set-mark 1
iptables -t mangle -A PREROUTING -j CONNMARK --save-mark

ip rule add fwmark 1 table isp1
ip rule add fwmark 2 table isp2

To make these changes persistent across reboots on Debian:

# /etc/network/interfaces
auto eth1
iface eth1 inet static
    address 192.168.1.x
    netmask 255.255.255.0
    post-up ip route add default via 192.168.1.1 dev eth1 table isp1
    post-up ip rule add from 192.168.1.x table isp1

auto eth2
iface eth2 inet static
    address 192.168.2.x
    netmask 255.255.255.0
    post-up ip route add default via 192.168.2.1 dev eth2 table isp2
    post-up ip rule add from 192.168.2.x table isp2

Use these commands to verify your setup:

ip route show table isp1
ip route show table isp2
ip rule show
conntrack -L
iptables -t mangle -L -v -n

When dealing with multiple ISP connections, we need to properly configure both routing and NAT to ensure seamless traffic distribution. Your setup with three interfaces is fairly standard:

eth0 (192.168.0.0/24) - LAN interface
eth1 (192.168.1.0/24) - ISP1 gateway
eth2 (192.168.2.0/24) - ISP2 gateway

First, we need to set up multiple routing tables. Add these to /etc/iproute2/rt_tables:

echo "200 isp1" >> /etc/iproute2/rt_tables
echo "201 isp2" >> /etc/iproute2/rt_tables

Then configure the routing tables:

ip route add default via 192.168.1.1 dev eth1 table isp1
ip route add 192.168.1.0/24 dev eth1 src 192.168.1.2 table isp1
ip route add default via 192.168.2.1 dev eth2 table isp2
ip route add 192.168.2.0/24 dev eth2 src 192.168.2.2 table isp2

For NAT to work properly with multiple ISPs, we need to implement policy routing with source-based rules:

iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE
iptables -t nat -A POSTROUTING -o eth2 -j MASQUERADE

Add routing rules for the outgoing traffic:

ip rule add from 192.168.0.0/24 table isp1
ip rule add from 192.168.0.0/24 table isp2

For true load balancing, we'll use iptables' statistic module:

iptables -t mangle -A PREROUTING -j MARK --set-mark 1
iptables -t mangle -A PREROUTING -m statistic --mode random --probability 0.5 -j MARK --set-mark 2
ip rule add fwmark 1 table isp1
ip rule add fwmark 2 table isp2

To make these changes permanent, add them to /etc/network/interfaces:

post-up ip route add default scope global nexthop via 192.168.1.1 dev eth1 weight 1 \
nexthop via 192.168.2.1 dev eth2 weight 1

For iptables rules, consider using iptables-persistent package or saving them to a script in /etc/network/if-up.d/

Verify your configuration with these commands:

ip route show table isp1
ip route show table isp2
iptables -t nat -L -n -v

Test connectivity through each ISP:

ping -I eth1 8.8.8.8
ping -I eth2 8.8.8.8

For better failover handling, consider adding health checks:

ip route append default via 192.168.1.1 dev eth1 metric 100
ip route append default via 192.168.2.1 dev eth2 metric 200

This setup provides a robust multi-ISP solution with load balancing and NAT capabilities on Debian Linux.