Linux Load Balancing with Dual WAN PPPoE Connections: NAT Routing and Traffic Distribution Guide

When configuring a multi-homed Linux router with two PPPoE WAN connections and one LAN interface, we need to address three key challenges:

PPPoE authentication binding to specific interfaces
Load balancing across multiple gateways
Inbound traffic routing to internal servers

First, let's properly configure PPPoE on both WAN interfaces. Create these files in /etc/ppp/peers/:

# /etc/ppp/peers/isp1
plugin rp-pppoe.so eth0
user "your_username1"
password "your_password1"
persist
maxfail 0
holdoff 5
noauth
defaultroute
usepeerdns

# /etc/ppp/peers/isp2
plugin rp-pppoe.so eth1
user "your_username2"
password "your_password2"
persist
maxfail 0
holdoff 5
noauth
nodefaultroute
usepeerdns

We'll use Linux's advanced routing capabilities to implement load balancing:

# Create custom routing tables
echo "101 isp1" >> /etc/iproute2/rt_tables
echo "102 isp2" >> /etc/iproute2/rt_tables

# Set up policy routing
ip rule add from 172.16.0.2 table isp1
ip rule add from 172.16.1.2 table isp2

# Add default routes to each table
ip route add default via 172.16.0.1 dev eth0 table isp1
ip route add default via 172.16.1.1 dev eth1 table isp2

# Main routing table
ip route add default scope global nexthop via 172.16.0.1 dev eth0 weight 1 \
nexthop via 172.16.1.1 dev eth1 weight 1

For outbound traffic NAT, we need to use source-based routing:

# Enable IP forwarding
sysctl -w net.ipv4.ip_forward=1

# NAT rules for each interface
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE

# Mark packets for routing decisions
iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
iptables -t mangle -A PREROUTING -m mark ! --mark 0 -j ACCEPT
iptables -t mangle -A PREROUTING -j MARK --set-mark 1
iptables -t mangle -A PREROUTING -m statistic --mode random --probability 0.5 -j MARK --set-mark 2
iptables -t mangle -A PREROUTING -j CONNMARK --save-mark

To route incoming traffic to specific internal servers based on public IP:

# Example: Route incoming web traffic on WAN1 to internal server
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j DNAT --to-destination 172.16.2.10:80
iptables -t nat -A POSTROUTING -o eth2 -d 172.16.2.10 -j SNAT --to-source 172.16.2.1

# For WAN2's public IP
iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 443 -j DNAT --to-destination 172.16.2.20:443
iptables -t nat -A POSTROUTING -o eth2 -d 172.16.2.20 -j SNAT --to-source 172.16.2.1

To maintain connection persistence when using multiple WAN links:

# Create connection tracking rules
iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
iptables -t mangle -A OUTPUT -j CONNMARK --restore-mark

# Route marked packets accordingly
ip rule add fwmark 1 table isp1
ip rule add fwmark 2 table isp2

Essential commands to monitor your setup:

# Check routing tables
ip route show table isp1
ip route show table isp2

# View connection marks
conntrack -L

# Test traffic distribution
tcpdump -i eth0
tcpdump -i eth1

Remember to persist these settings across reboots by adding them to /etc/rc.local or creating appropriate init scripts.

When implementing a multi-homed Linux router with dual WAN connections, we need to handle:

Two PPPoE connections on separate interfaces (eth0, eth1)
One LAN interface (eth2) with NAT for internal clients
Load balancing between WAN connections
Failover capability
Policy-based routing for specific traffic

First configure PPPoE on both WAN interfaces:

# /etc/ppp/peers/isp1
plugin rp-pppoe.so eth0
user "your_username@isp1"
password "your_password"
noipdefault
defaultroute
replacedefaultroute
hide-password
noauth
persist
mtu 1492
mru 1492

# /etc/ppp/peers/isp2  
plugin rp-pppoe.so eth1
user "your_username@isp2"
password "your_password"
noipdefault
nodefaultroute
hide-password
noauth
persist
mtu 1492
mru 1492

We'll use multiple routing tables and policy routing rules:

# /etc/iproute2/rt_tables
100 isp1
200 isp2

Create routing rules script:

#!/bin/bash
# Clear existing rules
ip rule del from all lookup main pref 32767
ip rule del from all lookup default pref 32766

# ISP1 routing
ip route add default via $(ip route show table main | grep ppp0 | awk '{print $3}') dev ppp0 table isp1
ip rule add from $(ip addr show ppp0 | grep inet | awk '{print $2}' | cut -d'/' -f1) lookup isp1

# ISP2 routing  
ip route add default via $(ip route show table main | grep ppp1 | awk '{print $3}') dev ppp1 table isp2
ip rule add from $(ip addr show ppp1 | grep inet | awk '{print $2}' | cut -d'/' -f1) lookup isp2

# Default main table
ip route add default scope global nexthop via $(ip route show table main | grep ppp0 | awk '{print $3}') dev ppp0 weight 1 \
nexthop via $(ip route show table main | grep ppp1 | awk '{print $3}') dev ppp1 weight 1

Configure iptables for NAT and load balancing:

#!/bin/bash
# Enable IP forwarding
echo 1 > /proc/sys/net/ipv4/ip_forward

# Clear existing rules
iptables -F
iptables -t nat -F

# NAT for both WAN interfaces
iptables -t nat -A POSTROUTING -o ppp0 -j MASQUERADE
iptables -t nat -A POSTROUTING -o ppp1 -j MASQUERADE

# Mark packets for load balancing
iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
iptables -t mangle -A PREROUTING -m mark ! --mark 0 -j ACCEPT
iptables -t mangle -A PREROUTING -m statistic --mode random --probability 0.5 -j MARK --set-mark 1
iptables -t mangle -A PREROUTING -j CONNMARK --save-mark

If you encounter "file exists" or "invalid device" errors:

Verify PPPoE interfaces are properly named (ppp0, ppp1)
Check for duplicate routing rules with: ip rule show
Ensure proper interface metrics are set
Verify NAT rules with: iptables -t nat -L -v -n

For persistent configuration, add the routing and firewall scripts to /etc/network/if-up.d/

To route specific internal IPs through specific WAN connections:

# Route 172.16.2.100 via ISP1
ip rule add from 172.16.2.100 lookup isp1
ip rule add to 172.16.2.100 lookup isp1

# Route 172.16.2.101 via ISP2  
ip rule add from 172.16.2.101 lookup isp2
ip rule add to 172.16.2.101 lookup isp2