Open Source NetFlow Generation on Linux: Tools and Implementation Guide for Multi-Interface Capture


3 views

Generating NetFlow data on Linux servers requires tools that can efficiently capture traffic from multiple interfaces and export it in a format compatible with NetFlow analyzers. While proprietary solutions like nProbe exist, open-source alternatives are preferable for cost and flexibility reasons.

Here are the most viable open-source options for NetFlow generation on Linux:

1. fprobe (http://fprobe.sourceforge.net/)  
   - Lightweight NetFlow v1/v5/v9 exporter  
   - Supports multiple interfaces  
   - Example command:  
     fprobe -i eth0 -f netflow -n 192.168.1.100:9996  

2. pmacct (http://www.pmacct.net/)  
   - Advanced flow accounting with NetFlow support  
   - Multi-interface and aggregation capabilities  
   - Sample config snippet:  
     daemonize: false  
     plugins: print[stdout], nfprobe  
     nfprobe_receiver: 192.168.1.100:9996  
     nfprobe_version: 9  
     interface: eth0,eth1  

3. softflowd (https://github.com/irino/softflowd)  
   - NetFlow v1/v5/v9 compatible  
   - Multi-interface support via multiple instances  
   - Usage example:  
     softflowd -i eth0 -n 192.168.1.100:9996 -v 9

For a robust multi-interface solution, pmacct offers the most flexibility. Here's a complete configuration example:

# /etc/pmacct/pmacctd.conf  
daemonize: true  
syslog: daemon  
plugins: nfprobe  

nfprobe_receiver: 192.168.1.100:9996  
nfprobe_version: 9  
nfprobe_engine_type: 1001  
nfprobe_engine_id: 2002  

aggregate: src_host,dst_host,src_port,dst_port,proto  
aggregate_filter: net 192.168.0.0/16  

interface: eth0  
interface: eth1  
interface: eth2

When deploying NetFlow generation on production servers:

  • Use kernel bypass techniques (AF_PACKET, XDP) for high traffic interfaces
  • Consider sampling (1:100 or 1:1000) to reduce CPU load
  • Distribute flow collection across multiple CPU cores

Common issues and solutions:

# Verify NetFlow export with tcpdump  
tcpdump -i eth0 -nn -s 0 -v port 9996  

# Check for dropped packets  
grep "dropped" /proc/net/softnet_stat  

# Test flow visibility  
nfdump -r /path/to/netflow.file -s record/bytes

Generating NetFlow data on Linux servers requires specialized tools that can capture traffic from multiple interfaces and export it in NetFlow format (typically v5 or v9). Unlike network devices with built-in NetFlow support, Linux requires additional software to perform this functionality.

After extensive testing, these are the most reliable open source options for NetFlow generation:

1. fprobe - Lightweight NetFlow probe
2. pmacct - Full-featured network monitoring toolkit
3. softflowd - Flexible flow collector and exporter
4. nfdump - Collection and processing tools

While fprobe has some reported stability issues, it remains the simplest solution for basic NetFlow generation. Here's how to configure it for multiple interfaces:

# Install fprobe on Debian-based systems
sudo apt-get install fprobe

# Start fprobe for multiple interfaces
fprobe -i eth0 -i eth1 -f ip -n 192.168.1.100:9996

# For persistent configuration (systemd)
sudo tee /etc/systemd/system/fprobe.service <<EOF
[Unit]
Description=fprobe NetFlow exporter
After=network.target

[Service]
ExecStart=/usr/sbin/fprobe -i eth0 -i eth1 -f ip -n 192.168.1.100:9996
Restart=always

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl enable fprobe
sudo systemctl start fprobe

For more robust NetFlow generation, pmacct offers better performance and features:

# Sample /etc/pmacct/pmacctd.conf
daemonize: true
pcap_interfaces: eth0, eth1
aggregate: src_host, dst_host, src_port, dst_port, proto, tos
plugins: memory
nfacctd_port: 9996
nfacctd_ip: 192.168.1.100

For servers with significant network traffic, consider these optimizations:

# Kernel parameters for better performance
net.core.rmem_max=16777216
net.core.wmem_max=16777216
net.core.netdev_max_backlog=2000

# fprobe with sampling (reduces CPU load)
fprobe -i eth0 -r 1000 -n 192.168.1.100:9996

Use nfdump to verify exported flows:

# Install nfdump
sudo apt-get install nfdump

# Capture and display flows
nfcapd -p 9996 -l /var/tmp/flows -w
nfdump -r /var/tmp/flows/nfcapd.current

If open source solutions don't meet your requirements, consider these alternatives:

  • Using iptables with ulogd and custom plugins
  • Implementing custom collection with libpcap
  • Commercial solutions like ntop's nProbe