When implementing NIC bonding across multiple switches, the physical topology significantly impacts redundancy and bandwidth. In your scenario with two switches and dual-NIC servers, you're essentially creating a cross-switch bond that requires careful switch configuration.

# Example ifenslave configuration for cross-switch bonding
auto bond0
iface bond0 inet static
    address 192.168.1.100
    netmask 255.255.255.0
    slaves eth0 eth1
    bond_mode active-backup
    bond_miimon 100
    bond_downdelay 200
    bond_updelay 200

For active-backup mode across switches:

• Each server's primary NIC connects to Switch A
• Secondary NICs connect to Switch B
• Inter-switch link is optional but recommended

Failure scenario: If Switch A fails, all servers failover to Switch B. Without an inter-switch link, intra-server communication would be disrupted as traffic can't cross between primary and secondary paths.

To prevent spanning tree protocol (STP) from blocking ports:

# Cisco switch configuration example
interface Port-channel1
 switchport mode trunk
!
interface GigabitEthernet0/1
 channel-group 1 mode active
 switchport mode trunk
!
interface GigabitEthernet0/2
 channel-group 1 mode active
 switchport mode trunk

For better utilization than active-backup:

• LACP (mode 4): Requires switches supporting 802.3ad
• Balance-TLB: Outgoing traffic load balancing
• Balance-ALB: Includes incoming traffic balancing

# LACP bonding configuration example
auto bond0
iface bond0 inet static
    address 192.168.1.100
    netmask 255.255.255.0
    slaves eth0 eth1
    bond_mode 4
    bond_xmit_hash_policy layer3+4
    bond_miimon 100
    bond_lacp_rate fast

In a web cluster with 10 servers and 2 ToR switches:

1. Configured LACP bonds on all servers
2. Connected odd-numbered ports to Switch A
3. Connected even-numbered ports to Switch B
4. Enabled cross-stack LACP on switches
5. Verified failover with mii-tool and switch port shutdown tests

This setup survived switch firmware updates without downtime by failing over to the alternate switch during maintenance windows.

When implementing network interface bonding across multiple switches, you're dealing with a more complex topology than single-switch bonding. The fundamental question is whether to:

1. Connect each server's NICs to separate switches without inter-switch links
2. Create an uplink between the switches
3. Implement switch stacking or MLAG technologies

For active-backup configuration (mode=1), here's a sample Linux network configuration:

# /etc/network/interfaces example
auto bond0
iface bond0 inet static
    address 192.168.1.10
    netmask 255.255.255.0
    gateway 192.168.1.1
    bond-mode active-backup
    bond-miimon 100
    bond-downdelay 200
    bond-updelay 200
    bond-primary eth0
    bond-slaves eth0 eth1

In this configuration:

• eth0 connects to Switch A (primary)
• eth1 connects to Switch B (backup)
• No direct connection between switches

The critical consideration is what happens when:

For more advanced configurations using LACP (mode=4), switch coordination becomes mandatory:

# Cisco switch configuration example
interface Port-channel1
    switchport mode trunk
    switchport trunk allowed vlan 10,20
!
interface GigabitEthernet1/0/1
    channel-group 1 mode active
!
interface GigabitEthernet1/0/2
    channel-group 1 mode active

Key requirements for multi-switch LACP:

• Switches must support MLAG or vPC technologies
• Special inter-switch control links required
• Identical switch configuration on both devices

Based on production experience, I recommend:

1. For active-backup: Keep switch interconnects minimal (no loops)
2. For LACP: Use vendor-recommended MLAG configurations
3. Always test failover scenarios under load
4. Monitor bond status via /proc/net/bonding/bond0

Common issues and diagnostic commands:

# Check bond status
cat /proc/net/bonding/bond0

# Monitor link transitions
ethtool eth0 | grep "Link detected"

# Verify switch port configuration
show etherchannel summary

Remember that spanning tree protocol (STP) behavior significantly impacts multi-switch bonding reliability. Adjust STP timers accordingly if using active-backup mode without MLAG.