On our Dell PowerEdge 840 running CentOS 6.0 64-bit, SSH sessions consistently terminate with:

Corrupted MAC on input. Disconnecting: Packet corrupt

The issue appears more frequently during X11 forwarding sessions. Initial hardware diagnostics (memtest86+, memory reseating) showed no errors, despite the error suggesting potential memory corruption.

The "MAC corruption" message indicates a Message Authentication Code verification failure during SSH packet transmission. This could stem from:

• Faulty network hardware (NIC, switches)
• Memory errors not caught by memtest
• SSH encryption algorithm incompatibilities

First, let's check active SSH algorithms:

# Check supported MACs on client
ssh -Q mac

# Check server's accepted MACs
ssh -vvv user@server 2>&1 | grep "peer mac"

Try enforcing simpler MAC algorithms in /etc/ssh/ssh_config or ~/.ssh/config:

Host problematic-server
    MACs hmac-sha1,hmac-md5
    Ciphers aes128-cbc,3des-cbc
    ForwardX11Trusted yes

For the server-side (/etc/ssh/sshd_config):

# Temporarily allow weaker but more compatible settings
MACs hmac-sha1,hmac-md5
Ciphers aes128-cbc,3des-cbc

Run packet capture during SSH sessions:

tcpdump -i eth0 -w ssh_capture.pcap port 22
# Analyze with Wireshark later

Check for Ethernet errors:

ethtool -S eth0 | grep -i error

Standard memtest might miss subtle errors. Try:

# Install and run more thorough memory tester
yum install memtester
memtester 500M 5

For DDR2 systems, consider testing with reduced speed:

# In BIOS:
# Set memory timing to 533MHz instead of 667MHz
# Increase DRAM voltage slightly (if safe)

As a test, install Dropbear as alternative SSH server:

yum install dropbear
service dropbear start
# Connect using:
ssh -p 2222 user@server

Examine kernel messages for hardware faults:

dmesg | grep -i -A10 -B10 error
dmidecode --type memory

Check for potential overheating issues:

yum install lm_sensors
sensors-detect
sensors

---

When SSH sessions fail with "Corrupted MAC on input" errors, we're dealing with Message Authentication Code verification failures during encrypted packet transmission. This typically indicates either:

• Network packet corruption
• Memory corruption during crypto operations
• CPU/motherboard issues affecting encryption

Since memtest86+ returned clean results, let's perform targeted memory stress tests focusing on crypto operations:

# Install cryptsetup for crypto benchmarking
yum install cryptsetup

# Run memory-intensive AES test
cryptsetup benchmark --cipher aes-cbc-essiv:sha256

Monitor for segmentation faults or incorrect hash outputs during the test. For more thorough testing:

# Extended memory test with openssl
for i in {1..1000}; do 
  openssl rand -base64 1024 | openssl sha256
done

Before concluding it's hardware, rule out network issues with packet capture:

tcpdump -i eth0 -s0 -w ssh_capture.pcap port 22
# Analyze with Wireshark later for TCP retransmissions

Modify /etc/ssh/sshd_config to use more resilient algorithms:

# Disable problematic MAC algorithms
MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com

# Reduce compression overhead
Compression delayed
CompressionLevel 6

Then restart SSH: service sshd restart

For Dell PowerEdge servers, check:

1. BIOS settings for memory ECC configuration
2. CPU microcode updates: yum install microcode_ctl
3. Motherboard battery voltage (low voltage can cause memory issues)

As a diagnostic measure, try different SSH implementations:

# Install dropbear as alternative
yum install dropbear
systemctl start dropbear

Connect using: dbclient user@host -p 22

Since the issue occurs more frequently with X-forwarding:

# Test with different compression levels
ssh -XC -c aes128-gcm@openssh.com user@host
ssh -x -o Compression=no user@host

Install kernel memory monitoring tools:

yum install kernel-devel
perf stat -e memory:mem_load_retired.l1_hit,memory:mem_load_retired.l1_miss \ 
  -p $(pgrep -f sshd)