Benchmarking NFS Share Performance: Measuring I/O Speed, Latency, and Throughput for File Operations


1 views

When working with NFS-mounted shares, three critical performance metrics matter most:

  • Throughput: Measured in MB/s, indicates data transfer speed
  • Latency: Time taken for individual operations (in milliseconds)
  • IOPS: Input/output operations per second

These metrics help identify whether performance issues stem from network limitations, server configuration, or storage hardware.

The Linux ecosystem provides several powerful tools for storage benchmarking:


# Install benchmark tools on CentOS/RHEL
sudo yum install fio iozone bonnie++ -y

FIO (Flexible I/O Tester) is the most versatile tool for thorough analysis. This example tests both sequential and random access patterns:


# Sequential read test
fio --name=seq_read --filename=/mnt/nfs_share/testfile \
    --rw=read --size=1G --direct=1 --bs=1M --numjobs=1 \
    --runtime=60 --time_based --group_reporting

# Random write test
fio --name=rand_write --filename=/mnt/nfs_share/testfile \
    --rw=randwrite --size=1G --direct=1 --bs=4k --numjobs=8 \
    --runtime=60 --time_based --group_reporting

To establish a performance baseline, run identical tests against local storage:


# Local disk benchmark
fio --name=local_test --filename=/tmp/testfile \
    --rw=rw --size=1G --direct=1 --bs=4k --numjobs=4 \
    --runtime=60 --time_based --group_reporting

Based on benchmark results, consider these tuning parameters in /etc/exports:


/mnt/share 192.168.1.0/24(rw,sync,no_wdelay,no_root_squash,no_subtree_check)

Key mount options for /etc/fstab:


server:/share  /mnt/share  nfs  rw,hard,intr,rsize=65536,wsize=65536,timeo=600  0  0

For regular monitoring, create a script that logs performance data:


#!/bin/bash
DATE=$(date +%Y-%m-%d)
LOG_FILE="/var/log/nfs_benchmark_$DATE.log"

echo "==== NFS Benchmark $DATE ====" >> $LOG_FILE
fio --name=bench --filename=/mnt/nfs_share/testfile \
    --rw=rw --size=1G --runtime=300 --output-format=json >> $LOG_FILE

# Parse and alert if thresholds exceeded
LATENCY=$(jq '.jobs[0].lat_ns.mean' $LOG_FILE)
if [ $LATENCY -gt 10000000 ]; then
    mail -s "High NFS Latency Alert" admin@example.com < $LOG_FILE
fi

Remember that NFS performance depends on multiple factors:

  • Network quality (latency, packet loss)
  • Server hardware (CPU, RAM, storage backend)
  • Concurrent access patterns
  • File sizes and access frequency

For accurate results, always test with workloads resembling your actual usage patterns.


When working with NFS-mounted shares in CentOS/RHEL systems, understanding the actual performance characteristics is crucial for system tuning and capacity planning. The key metrics to measure include:

  • Read/Write throughput (MB/s)
  • IOPS (Input/Output Operations Per Second)
  • Latency (time per operation in ms)
  • Concurrency capabilities

Here are the most effective tools for measuring NFS performance:

1. dd - Basic Sequential I/O Testing


# Write test (1GB file)
dd if=/dev/zero of=/mnt/nfs/testfile bs=1M count=1024 oflag=direct conv=fdatasync

# Read test
dd if=/mnt/nfs/testfile of=/dev/null bs=1M count=1024 iflag=direct

2. fio - Comprehensive Storage Benchmarking

The Flexible I/O Tester (fio) provides detailed performance analysis:


# Install on CentOS
yum install fio -y

# Sequential read test
fio --name=seqread --rw=read --direct=1 --ioengine=libaio --bs=1M --size=1G --numjobs=1 --runtime=60 --time_based --group_reporting --directory=/mnt/nfs/

# Random write test
fio --name=randwrite --rw=randwrite --direct=1 --ioengine=libaio --bs=4k --size=1G --numjobs=8 --runtime=60 --time_based --group_reporting --directory=/mnt/nfs/

3. nfsiostat - NFS-specific Metrics


# Install nfs-utils if needed
yum install nfs-utils -y

# Monitor NFS operations
nfsiostat /mnt/nfs 5

To establish a baseline, run the same tests on local storage:


# Local disk test with fio
fio --name=local_test --rw=randrw --direct=1 --ioengine=libaio --bs=4k --size=1G --numjobs=4 --runtime=60 --time_based --group_reporting --directory=/path/to/local/storage/

For more realistic workloads, consider these approaches:

Simulating Database Workload


fio --name=db_workload --rw=randrw --rwmixread=70 --direct=1 --ioengine=libaio --bs=8k --size=10G --numjobs=16 --runtime=300 --time_based --group_reporting --directory=/mnt/nfs/

Measuring Small File Performance


# Create test with 10,000 small files
for i in {1..10000}; do dd if=/dev/urandom of=/mnt/nfs/smallfile_$i bs=4k count=1; done

# Time file access
time find /mnt/nfs -name "smallfile_*" -exec cat {} + > /dev/null

Key indicators of NFS performance issues:

  • Latency > 10ms for simple operations
  • Throughput significantly lower than network bandwidth
  • High variation in response times
  • CPU saturation during I/O operations

If results indicate suboptimal performance:

  • Try different NFS versions (v3 vs v4)
  • Adjust rsize/wsize mount options
  • Consider TCP vs UDP protocol
  • Check network infrastructure (MTU, switches, etc.)
  • Review NFS server configuration

Remember to unmount and remount with different options for testing:


umount /mnt/nfs
mount -t nfs -o rsize=65536,wsize=65536,hard,intr server:/share /mnt/nfs