Debugging Linux LACP Bonding Issues: Kernel-Level Diagnostics for Cisco Interoperability


4 views

Having spent years configuring LACP between Linux and Cisco devices, I recently encountered a perplexing case where the Linux bonding driver completely ignored incoming LACP packets from Cisco switches. The symptoms were clear:

  • TCPdump showed LACP packets arriving on bonded interfaces
  • Kernel maintained zero tx_packets counters
  • No errors in system logs or bonding module output
  • Switch eventually timed out and marked ports as stand-alone

To inspect bonding driver internals, these are the most valuable interfaces:

# Monitor bonding driver events in real-time
dmesg -wH

# Detailed bond interface status
cat /proc/net/bonding/bond0

# Check driver parameters
modinfo bonding

# Verify module loading
lsmod | grep bonding

The following bonding options significantly impact LACP behavior:

# Recommended LACP configuration sample
auto bond0
iface bond0 inet manual
    slaves eth0 eth1
    bond_mode 4
    bond_miimon 100
    bond_downdelay 200
    bond_updelay 200
    bond_lacp_rate fast
    bond_xmit_hash_policy layer2+3
    bond_ad_select 2

For thorough LACP traffic analysis:

# Capture LACP packets with detailed decoding
tcpdump -i eth0 -nn -vvv -XX -s 1500 'ether proto 0x8809'

# Alternative using SLL headers
tcpdump -i any -nn -vvv -s 1500 'port 65534'

Driver Compatibility Issues

Some NIC drivers require specific settings for proper LACP operation. Try loading the bonding driver with these parameters:

options bonding mode=4 miimon=100 lacp_rate=fast downdelay=200 updelay=200

Switch Port Configuration

The Cisco side must match Linux parameters. Verify with:

show etherchannel summary
show lacp neighbor
show interface Port-channel12

Kernel Module Debugging

Enable bonding driver debug messages:

echo 1 > /sys/module/bonding/parameters/debug

For deep inspection, use kernel probes to trace bonding driver execution:

# Trace LACP packet reception
perf probe --add 'bond_3ad_rx_indication skb'

# Monitor state machine transitions
perf probe --add 'bond_3ad_state_machine_handler'

Remember that some issues might require kernel version analysis or even driver patching. The Linux bonding driver source provides valuable insight into the LACP implementation details.

When setting up LACP (802.3ad) bonding between Linux servers and Cisco switches, you might encounter situations where the Linux kernel appears to ignore incoming LACP packets. The symptoms typically include:

# cat /proc/net/bonding/bond0
...
bond bond0 has no active aggregator
MII Status: down

While tcpdump shows LACP packets arriving from the switch, the Linux bonding driver fails to respond, resulting in zero tx_packets despite high rx_packets.

The Linux bonding driver provides several diagnostic interfaces:

# sysctl net.bonding
# ls /sys/class/net/bond0/bonding/
# cat /proc/net/bonding/bond0

Key parameters to check:

# echo 1 > /sys/class/net/bond0/bonding/lacp_active
# echo fast > /sys/class/net/bond0/bonding/lacp_rate

Verify these critical bonding parameters:

# Recommended minimal bonding config:
auto bond0
iface bond0 inet manual
    bond-mode 4
    bond-miimon 100
    bond-downdelay 200
    bond-updelay 200
    bond-lacp-rate fast
    bond-xmit-hash-policy layer2+3

Combine these tools for comprehensive analysis:

# Check interface status
ethtool eth1
ethtool -S eth1

# Capture LACP traffic
tcpdump -i eth1 -nn -v -s0 ether proto 0x8809

Ensure proper LACP configuration on Cisco devices:

interface GigabitEthernet1/0/15
 switchport trunk allowed vlan 100,101,102
 switchport mode trunk
 channel-group 12 mode active
 lacp rate fast
end

For persistent issues, try adjusting bonding module parameters:

# Add to /etc/modprobe.d/bonding.conf
options bonding max_bonds=4 miimon=100 downdelay=200 updelay=200

When standard tools fail, use these kernel debugging techniques:

# Monitor kernel messages in real-time
dmesg -wH

# Check bonding driver version
modinfo bonding | grep version

# Enable debug logging
echo 1 > /sys/class/net/bond0/bonding/debug

Use this sequence to trace LACP packet processing:

# Check if packets reach the kernel
tcpdump -i eth1 -nn -v 'ether proto 0x8809'

# Verify kernel processing
cat /proc/net/packet

# Check bonding driver state
cat /proc/net/bonding/bond0

Before declaring the issue unresolved, verify:

  • Network interface hardware compatibility
  • Kernel version and bonding driver support
  • Switch port configuration consistency
  • Physical layer connectivity (cables, SFPs)