Debugging and Fixing Linux Network Interface RX Dropped Packets (eth0)


3 views

When examining ifconfig eth0 output, the RX dropped packets counter represents frames that were successfully received by the network interface but subsequently dropped by the kernel before reaching any application or firewall layer. This occurs before iptables processing, at the network driver level.

Key characteristics of these drops:

1. Physical layer reception succeeded (no RX errors)
2. Kernel lacked resources to process (typically buffer-related)
3. Not related to firewall rules or application-level drops

From analyzing hundreds of similar cases, these patterns emerge:

# Check current buffer sizes
ethtool -g eth0

# Typical output showing potential bottlenecks:
Ring parameters for eth0:
Pre-set maximums:
RX:             4096
RX Mini:        0
RX Jumbo:       0
TX:             4096
Current hardware settings:
RX:             256       <-- Potential issue
RX Mini:        0
RX Jumbo:       0
TX:             256

Before making configuration changes, gather these metrics:

# Check interface statistics in real-time
watch -n 1 'ethtool -S eth0 | grep -E "drop|fail|miss"'

# Monitor kernel ring buffer drops
cat /proc/net/dev | grep eth0

# Check socket buffer utilization
sysctl -a | grep net.core

Solution 1: Increase Ring Buffer Size

# Temporary change (until reboot)
ethtool -G eth0 rx 2048

# Permanent solution (add to /etc/rc.local)
echo "ethtool -G eth0 rx 2048" >> /etc/rc.local

Solution 2: Adjust Kernel Network Parameters

# Add to /etc/sysctl.conf
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.core.rmem_default = 16777216
net.core.wmem_default = 16777216
net.core.optmem_max = 40960
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216

# Apply changes
sysctl -p

For persistent issues, consider these approaches:

# Capture dropped packet information
tcpdump -ni eth0 -s 0 -w /tmp/dropped.pcap &
# Wait for drops to occur, then stop with Ctrl+C

# Analyze NIC queue utilization
sar -n DEV 1    # Look for %ifutil near 100%
mpstat -P ALL   # Check CPU softirq handling

Implement these monitoring solutions:

# Simple cron job to log drops hourly
* */1 * * * echo "$(date) - $(ifconfig eth0 | grep dropped)" >> /var/log/eth0_drops.log

# Prometheus exporter config snippet
- job_name: 'node_network'
  static_configs:
    - targets: ['localhost:9100']
  metrics_path: '/metrics'
  params:
    collect[]: ['network']

When you see increasing RX dropped packets in ifconfig output, it indicates your network interface is receiving more traffic than the kernel can process. These drops occur at the driver level, before packets reach iptables/netfilter. The counter increments when:

  • The NIC's receive buffer is full
  • Kernel cannot allocate memory for SKBs
  • SoftIRQ cannot process packets fast enough

First, verify the current drop rate with extended statistics:

ethtool -S eth0 | grep -i drop
cat /proc/net/dev | grep eth0

For real-time monitoring, use:

watch -d -n 1 "cat /proc/net/dev | grep eth0"

Based on your ifconfig output showing 2523 drops, these are likely culprits:

1. Receive Buffer Starvation

Check current settings:

sysctl net.core.rmem_default
sysctl net.core.rmem_max

Temporary solution (survives reboot):

sysctl -w net.core.rmem_max=4194304
sysctl -w net.core.rmem_default=2097152

2. IRQ Balance Issues

For multi-queue NICs:

grep eth0 /proc/interrupts
cat /proc/irq/*/smp_affinity

Optimize with:

ethtool -L eth0 combined 8

Packet Capture Analysis

Compare tcpdump with interface statistics:

tcpdump -i eth0 -c 1000 -w /tmp/capture.pcap
ethtool -S eth0 > /tmp/before.txt
sleep 10
ethtool -S eth0 > /tmp/after.txt
diff /tmp/before.txt /tmp/after.txt

Kernel Ring Buffer

Check and adjust NIC ring parameters:

ethtool -g eth0
ethtool -G eth0 rx 4096

Add these to /etc/sysctl.conf:

net.core.rmem_max = 4194304
net.core.rmem_default = 2097152
net.core.netdev_max_backlog = 30000

For ongoing monitoring, create this shell script:

#!/bin/bash
INTERFACE="eth0"
LOG_FILE="/var/log/net_drops.log"

while true; do
    RX_DROPS=$(cat /proc/net/dev | grep $INTERFACE | awk '{print $5}')
    TIMESTAMP=$(date +"%Y-%m-%d %H:%M:%S")
    echo "$TIMESTAMP - $INTERFACE RX drops: $RX_DROPS" >> $LOG_FILE
    sleep 60
done

Some packet loss is normal during:

  • Network bursts exceeding interface capacity
  • TCP retransmissions
  • UDP traffic spikes

Monitor the rate rather than absolute numbers.

For modern Linux systems, consider replacing ifconfig with:

ip -s link show eth0
ss -itmp
nstat -a