Advanced FreeBSD Kernel Tuning: Optimizing sysctl Parameters, loader.conf Settings and Network Stack for High-Performance Servers


2 views

When dealing with high-performance server environments, FreeBSD offers extensive tuning capabilities through sysctl parameters, loader.conf settings, and kernel configurations. These optimizations become crucial when handling workloads exceeding 100,000 concurrent connections.

Here's a distilled version of essential network-related parameters for modern FreeBSD systems:

# Network stack optimizations
kern.ipc.somaxconn=4096
kern.ipc.maxsockets=204800
kern.ipc.nmbclusters=262144
kern.maxfiles=204800
kern.maxfilesperproc=200000

# TCP/IP stack tuning
net.inet.tcp.rfc1323=1
net.inet.tcp.msl=5000
net.inet.tcp.maxtcptw=200000
net.inet.tcp.fast_finwait2_recycle=1
net.inet.tcp.cc.algorithm=htcp
net.inet.ip.intr_queue_maxlen=4096

These kernel module loads should be considered for optimal performance:

# Load congestion control modules
cc_htcp_load="YES"
# cc_cubic_load="YES"  # Alternative congestion control

# Increase kernel memory (adjust based on available RAM)
vm.kmem_size="3G"
vm.kmem_size_max="3G"

For web servers handling numerous small requests, these additional TCP tweaks prove valuable:

# Optimize for HTTP workloads
net.inet.tcp.recvspace=8192
net.inet.tcp.sendspace=16384
net.inet.tcp.delayed_ack=0

Conversely, for file servers or streaming applications:

# File server/streaming optimizations
net.inet.tcp.recvspace=65535
net.inet.tcp.sendspace=65535
net.inet.tcp.sendbuf_max=10485760
net.inet.tcp.recvbuf_max=10485760

For systems handling numerous files or database workloads:

# Filesystem optimizations
vfs.ufs.dirhash_maxmem=67108864
vfs.read_max=32

# Database server specific
kern.ipc.shm_use_phys=1
kern.ipc.shmmax=2147483648

Performance tuning should never compromise security. These settings provide protection without significant overhead:

# Security hardening
net.inet.ip.redirect=0
net.inet.ip.sourceroute=0
net.inet.icmp.drop_redirect=1
net.inet.tcp.drop_synfin=1
net.inet6.icmp6.nodeinfo=0

After implementing these changes, monitor system behavior with:

netstat -Lan  # Check listen queue status
netstat -m    # Monitor mbuf usage
vmstat -z     # View kernel memory allocation

Note that some parameters have evolved across FreeBSD versions:

  • HPET became default timecounter after revision 222222
  • net.inet.tcp.inflight.enable was removed in 10-CURRENT
  • Newer FreeBSD versions have better defaults for nmbclusters

These optimizations should be tested thoroughly in staging environments before production deployment. The exact combination of parameters will vary based on specific workload patterns and hardware configurations.


When tuning FreeBSD for high-performance networking, we need to understand three key configuration layers:

1. Runtime tunables (sysctl.conf)
2. Boot-time parameters (loader.conf) 
3. Kernel configuration (GENERIC or custom)

The heart of FreeBSD tuning lies in sysctl values. Here are the most impactful network-related settings:

# Socket and connection handling
kern.ipc.maxsockets=204800
kern.ipc.somaxconn=4096
kern.ipc.nmbclusters=262144

# TCP stack optimization  
net.inet.tcp.rfc1323=1
net.inet.tcp.recvspace=8192
net.inet.tcp.sendspace=16384
net.inet.tcp.msl=5000
net.inet.tcp.maxtcptw=200000
net.inet.tcp.fast_finwait2_recycle=1

These boot-time parameters affect kernel initialization:

# Enable async IO
aio_load="YES"

# Network accept filters (8.0+)
accf_http_load="YES"
accf_dns_load="YES"

# Modern storage controllers
ahci_load="YES"
siis_load="YES"

For a web server handling 50,000+ concurrent connections:

# /etc/sysctl.conf
kern.ipc.somaxconn=32768
kern.ipc.maxsockets=131072
kern.ipc.nmbclusters=196608
net.inet.tcp.recvspace=65535
net.inet.tcp.sendspace=65535
net.inet.tcp.rfc1323=1
net.inet.tcp.delayed_ack=0

# /boot/loader.conf
aio_load="YES"
accf_http_load="YES"
hw.ata.wc=0

Use these commands to validate your tuning:

# Socket usage
netstat -m
sockstat -c

# Connection queue status
netstat -Lan

# Memory buffers
vmstat -z

For specialized workloads like database servers or CDN nodes:

# Database optimization
kern.ipc.shm_use_phys=1
vm.pmap.shpgperproc=2048
vfs.read_max=32

# CDN/media servers
net.inet.tcp.cc.algorithm=htcp
net.inet.ip.dummynet.io_fast=1
vfs.zfs.prefetch_disable=0

Remember to test changes incrementally and monitor system stability. The optimal configuration varies based on hardware, workload, and FreeBSD version.