When monitoring network interfaces in Linux, administrators often encounter puzzling discrepancies between different tools. The specific case of eth3
showing 1753 drops in /proc/net/dev
but 0 in ip -s link
output reveals important architectural differences in Linux networking statistics.
These commands pull from different kernel data structures:
// Simplified view of data flow
/proc/net/dev → net/core/net-procfs.c → dev_seq_show()
ip -s link → iproute2 package → netlink interface → rtnetlink
The divergence stems from several factors:
- Sampling Scope:
/proc/net/dev
includes all software drops whileip -s link
only shows hardware/driver-level drops - Counter Aggregation:
ip
command combines similar drop reasons that/proc
shows separately - Kernel Version Factors: Older kernels (like 2.6.32) had more fragmentation in stats collection
We can cross-validate using lower-level tools:
# Check specific drop counters
grep -H . /sys/class/net/eth3/statistics/*_drop
/sys/class/net/eth3/statistics/rx_dropped:1753
/sys/class/net/eth3/statistics/tx_dropped:0
The 1753 drops likely represent:
- Socket buffer overflows (SO_RCVBUF too small)
- Netfilter/iptables drops
- Qdisc layer drops not reflected in driver stats
For comprehensive monitoring in newer kernels:
# Combined view using ss and tc
ss -u -a -p -t -e # Socket-level stats
tc -s qdisc show dev eth3 # Queue discipline stats
ethtool -S eth3 # Driver-specific statistics
The divergence became particularly noticeable after Linux 2.6.9 when network stack architecture changed significantly. Older Debian versions (like Squeeze shown) used simpler aggregation that didn't fully reconcile all counters.
When debugging network issues on Linux, I often cross-verify statistics between /proc/net/dev
and ip -s link
. Recently, I noticed eth3 reports 1753 drops in /proc/net/dev
but shows 0 in ip -s link
. This discrepancy isn't a bug - it's about different kernel data sources:
// Kernel data collection paths
/proc/net/dev → net/core/net-procfs.c
ip -s link → net/core/rtnetlink.c
The divergence stems from how these tools access network statistics:
- /proc/net/dev: Reads directly from
dev->stats
(struct net_device_stats) - ip -s link: Uses rtnetlink to fetch
rtnl_link_stats64
(more modern accounting)
Here's a quick way to verify both sources programmatically:
#include <linux/netdevice.h>
#include <net/rtnetlink.h>
void compare_stats(struct net_device *dev) {
struct net_device_stats *proc_stats = &dev->stats;
struct rtnl_link_stats64 *rtnl_stats;
rtnl_stats = dev->netdev_ops->ndo_get_stats64(dev, rtnl_stats);
printk("Proc drops: %lu vs Rtnl drops: %llu\n",
proc_stats->rx_dropped,
rtnl_stats->rx_dropped);
}
The discrepancy typically occurs because:
- Driver implements
ndo_get_stats
but notndo_get_stats64
- Statistics get updated at different points in packet processing
- 32-bit vs 64-bit counters overflow differently
For modern kernels (3.0+), ip -s link
is generally more reliable as it uses atomic 64-bit counters. To debug further:
# Check driver implementation
grep -A10 "ndo_get_stats" /path/to/driver/source.c
# Monitor drops in real-time
watch -n1 "cat /proc/net/dev | grep eth3; ip -s link show eth3"
When I encountered this on a production server, I used this script to track the delta:
#!/bin/bash
while true; do
proc_drops=$(grep eth3 /proc/net/dev | awk '{print $4}')
ip_drops=$(ip -s link show eth3 | awk '/RX.*dropped/{print $4}')
echo "$(date) - Delta: $(($proc_drops - $ip_drops))"
sleep 5
done
The key takeaway? Neither source is "lying" - they're reporting from different layers of the network stack. For most modern troubleshooting, ip -s link
should be your primary reference, but always check both when investigating packet loss issues.