Many server administrators face this exact scenario: sendfile on improves performance but causes cache invalidation headaches when updating static files. Let's examine this at the filesystem level.

When sendfile on, Nginx uses the kernel's zero-copy mechanism:

+----------------+    +---------------+    +----------------+
| Disk Cache     | -> | Kernel Space  | -> | Network Socket |
+----------------+    +---------------+    +----------------+

This bypasses user-space buffers, but introduces these characteristics:

• 40-50% CPU reduction for static files
• 2-3x throughput improvement
• Cache coherency depends on FS notifications

For most production setups, keep sendfile on but add cache control:

location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ {
    sendfile on;
    expires 1y;
    add_header Cache-Control "public, no-transform";
    etag on;
    
    # Critical for cache invalidation
    if_modified_since exact;
}

Modern approaches to solve your versioning issue:

1. Content fingerprinting (recommended):

# During build process
main.a1b2c3d4.js

2. Query string versioning (your current approach):

# Nginx configuration to ignore query strings for cache key
location /js/ {
    sendfile on;
    try_files $uri $uri/ =404;
}

The behavior varies by storage backend:

For mission-critical deployments, consider:

# In nginx.conf
aio on;
directio 4k;
output_buffers 4 256k;

These work alongside sendfile for optimal performance while maintaining cache consistency.

Many administrators face this common scenario: After updating static files like JavaScript or CSS with cache-busting query strings (e.g., main.js?v=123), browsers still serve stale content when sendfile on is enabled in Nginx. Let's examine the technical underpinnings and solutions.

The sendfile directive enables the sendfile() system call, which performs zero-copy file transfers between disk and network sockets. Here's what happens at the kernel level:

+---------------+       +---------------+
| Nginx Process |       | Kernel Space  |
|---------------|       |---------------|
| User Space    | ----> | File Cache    |
|               |       | Network Stack |
+---------------+       +---------------+

When enabled, Nginx bypasses user-space buffers entirely. This creates the caching behavior you're observing because:

1. The kernel caches frequently accessed files
2. Cache invalidation doesn't trigger for query string changes
3. File descriptors remain open for hot files

For high-traffic servers, we can maintain sendfile benefits while solving cache issues:

# Option 1: Disable sendfile for development
sendfile off;

# Option 2: Production-optimized configuration
location ~* \.(js|css)$ {
    sendfile on;
    open_file_cache_valid 30s;
    add_header Cache-Control "no-cache, must-revalidate";
    etag off;
    if_modified_since off;
}

For maximum performance with reliable cache busting:

# 1. File versioning in filenames (recommended)
location /static {
    sendfile on;
    tcp_nopush on;
    
    # Versioned files: main.abc123.js
    location ~* \.[a-f0-9]{8}\.(js|css)$ {
        expires 1y;
        add_header Cache-Control "public";
    }
}

# 2. Query string fallback
map $request_uri $sendfile_switch {
    ~\?v= 0; # Disable sendfile for versioned requests
    default 1;
}

server {
    sendfile $sendfile_switch;
    ...
}

Testing on AWS c5.2xlarge instances shows:

The underlying filesystem affects sendfile behavior:

• Ext4/XFS: Optimal performance
• NFS: Requires sendfile_max_chunk tuning
• Encrypted volumes: May force fallback to buffered IO

# Adjust for network storage
sendfile_max_chunk 512k;
directio 4m;