When troubleshooting network performance issues or identifying bandwidth-hungry applications, standard tools like iftop or iptraf only show interface-level statistics. In production environments, we often need to pinpoint which specific process is consuming network resources.
The most straightforward solution is nethogs, which provides real-time network usage per process:
sudo apt install nethogs # Debian/Ubuntu
sudo yum install nethogs # RHEL/CentOS
sudo nethogs
Sample output shows processes with their PID, user, and bandwidth usage:
PID USER PROGRAM DEV SENT RECEIVED
1234 www-data /usr/bin/nginx eth0 12.3KB/s 45.6KB/s
5678 mysql /usr/sbin/mysqld eth0 5.2KB/s 8.1KB/s
For more detailed monitoring, we can leverage kernel's socket accounting:
# Enable kernel monitoring
sudo sysctl -w net.netfilter.nf_conntrack_acct=1
# Check active connections per process
sudo conntrack -L
Combine ss (socket statistics) with lsof to map connections to processes:
sudo ss -tulnp
sudo lsof -i -P -n
For continuous monitoring, create this bash script:
#!/bin/bash
while true; do
echo "==== $(date) ===="
sudo ss -tunap |
awk '{print $6}' |
grep -Eo '[0-9]{1,}' |
sort | uniq -c |
sort -n
sleep 5
done
For modern Linux kernels (4.4+), BPF tools provide the most efficient monitoring:
# Install bcc-tools
sudo apt install bpfcc-tools
# Monitor network I/O
sudo execsnoop-bpfcc -T -n
Example BPF program to track socket writes:
#!/usr/bin/env python
from bcc import BPF
bpf_text = """
#include
BPF_HASH(bytes_written, u32);
int trace_sock_sendmsg(struct pt_regs *ctx) {
u32 pid = bpf_get_current_pid_tgid();
u64 *count = bytes_written.lookup(&pid);
u64 zero = 0;
if (!count) {
count = bytes_written.lookup_or_init(&pid, &zero);
}
(*count) += PT_REGS_PARM3(ctx);
return 0;
}
"""
b = BPF(text=bpf_text)
b.attach_kprobe(event="sock_sendmsg", fn_name="trace_sock_sendmsg")
while True:
print("%-8s %s" % ("PID", "Bytes"))
for k, v in b["bytes_written"].items():
print("%-8d %d" % (k.value, v.value))
time.sleep(1)
For systemd services, we can use cgroups accounting:
systemctl set-property httpd.service IOAccounting=yes
systemctl show httpd.service -p IO*
When debugging network performance issues or analyzing suspicious network activity, knowing which processes are consuming bandwidth is crucial. While traditional tools like iftop
or iptraf
show interface-level statistics, we often need process-level granularity.
The Linux kernel exposes per-process network statistics through several mechanisms:
# Using /proc/net/dev
cat /proc/net/dev
# Using ss (socket statistics)
ss -tupn
For real-time monitoring, nethogs is specifically designed for this purpose:
# Install nethogs
sudo apt install nethogs # Debian/Ubuntu
sudo yum install nethogs # RHEL/CentOS
# Run with root privileges
sudo nethogs
Example output shows processes with their sent/received bytes:
PID USER PROGRAM DEV SENT RECEIVED
1234 www-data apache2 eth0 12.3KB 45.6KB
5678 mysql mysqld eth0 89.1KB 23.4KB
Combine iftop with lsof for process identification:
# First identify connections with iftop
sudo iftop -n -P
# Then find processes using those ports
sudo lsof -i :[port_number]
For advanced users, BPF tools provide deep visibility:
# Install bcc-tools
sudo apt install bpfcc-tools
# Run TCP connection tracker
sudo tcplife
# Or network traffic by process
sudo tcptop
Here's a Python script to aggregate network I/O by process:
#!/usr/bin/env python3
import os
from collections import defaultdict
def get_process_net_io():
net_io = defaultdict(lambda: {'recv': 0, 'send': 0})
for pid in os.listdir('/proc'):
if not pid.isdigit():
continue
try:
with open(f'/proc/{pid}/net/dev') as f:
for line in f:
if ':' in line:
iface, data = line.split(':')
recv = int(data.split()[0])
send = int(data.split()[8])
net_io[pid]['recv'] += recv
net_io[pid]['send'] += send
except:
continue
return net_io
print(get_process_net_io())
For long-term tracking, consider these approaches:
- Configure
sysstat
package to collect network statistics - Use
netdata
with its per-process network plugin - Implement custom logging of nethogs output
When monitoring doesn't show expected results:
- Check for containerized processes (Docker, LXC)
- Verify you're running with sufficient privileges
- Consider kernel version differences in statistics reporting