Impact of Logical Interface MTU Settings on Physical Interfaces in Linux Network Stack


2 views

In Linux networking, logical interfaces (bond, vlan, bridge) operate at different layers of the network stack than physical interfaces. The MTU (Maximum Transmission Unit) setting follows a hierarchical relationship:

Physical NICs (eth0,eth1) → Bond interface → VLAN interfaces → Bridge interfaces

The effective MTU is determined by the lowest value in the chain. Consider this practical example:

# Physical interfaces
ip link set eth0 mtu 9000
ip link set eth1 mtu 9000

# Bond interface inherits from physical
ip link set bond0 mtu 9000

# VLAN interface (must be ≤ bond0 MTU)
ip link set vlan10 mtu 1500

# Bridge interface (must be ≤ vlan MTU)
ip link set br10 mtu 1500

When configuring MTUs in complex network stacks:

  • Jumbo frames (MTU > 1500) require consistent settings across all layers
  • Bridge interfaces cannot have MTU larger than their member interfaces
  • VLAN tagging adds 4 bytes overhead (account for this in MTU calculations)

To verify your MTU settings are properly applied:

# Check current MTU for all interfaces
ip -br link show | awk '{print $1,$4}'

# Test end-to-end MTU (from domU through all layers)
ping -M do -s 8972 10.0.0.1  # (1500 MTU: 8972 = 1500 - 28 ICMP header)

Incorrect MTU settings can cause:

  • Packet fragmentation (when higher MTU interfaces send to lower MTU interfaces)
  • TCP blackhole connections (when PMTUD fails)
  • Reduced throughput (when not using optimal jumbo frames)

For your specific bond+vlan+bridge setup:

# Set consistent MTU across all layers
for intf in eth0 eth1 bond0 vlan10 vlan20 vlan30 br10 br20 br30; do
    ip link set dev $intf mtu 9000  # Or your desired MTU
done

# Verify with
for intf in eth0 eth1 bond0 vlan10 br10; do
    echo -n "$intf: "; cat /sys/class/net/$intf/mtu
done

When encountering MTU-related issues:

# Check for silently dropped packets
ethtool -S eth0 | grep -i 'drop\|error'

# Monitor interface statistics
ip -s link show bond0

# Check kernel messages for MTU warnings
dmesg | grep -i mtu

In complex network configurations combining bonding, VLANs, and bridges, the MTU relationship between logical and physical interfaces becomes crucial for optimal performance. The typical flow looks like this:

Physical interfaces (eth0, eth1) 
  → Bond interface (bond0) 
    → VLAN interfaces (vlan10, vlan20, vlan30) 
      → Bridge interfaces (br10, br20, br30)

Here's what actually happens with MTU settings in this stack:

  • Physical interfaces have their own MTU (typically 1500 by default)
  • Bond interfaces inherit the lowest MTU of member interfaces
  • VLAN interfaces inherit from their parent interface
  • Bridge interfaces use the MTU of their first added interface

To ensure proper MTU handling:

# Example: Setting Jumbo frames consistently
# Physical interfaces first
ip link set dev eth0 mtu 9000
ip link set dev eth1 mtu 9000

# Then bond interface (auto inherits)
ip link set dev bond0 up

# VLAN interfaces (explicit setting recommended)
ip link set dev vlan10 mtu 9000

# Bridge interfaces
ip link set dev br10 mtu 9000

Common symptoms and solutions:

Symptom Diagnosis Solution
Packet drops MTU mismatch between bridge and physical Check all interface MTUs with 'ip link'
Slow performance Fragmentation occurring Standardize MTU across entire stack

For Xen domU configurations:

# In dom0 configuration (xenbr0 example)
auto xenbr0
iface xenbr0 inet manual
    bridge_ports vlan100
    bridge_stp off
    bridge_fd 0
    mtu 9000
    
# Corresponding domU config
vif = ['mac=00:16:3e:xx:xx:xx,bridge=xenbr0,mtu=9000']

Remember that virtual interfaces in domUs cannot exceed the MTU of their parent bridge in dom0.

  • VLAN tags consume 4 bytes of MTU space
  • Some NICs have hardware MTU limits
  • Routing equipment must support end-to-end MTU
  • Always test with 'ping -M do -s' to verify path MTU