How to Isolate Nginx Access Logs for Specific Locations While Maintaining Proxy Settings


2 views

When configuring Nginx as a reverse proxy, we often need to maintain different logging behaviors for different endpoints while avoiding configuration duplication. The main pain points are:

  • Keeping proxy settings DRY (Don't Repeat Yourself)
  • Maintaining clean main access logs
  • Creating specialized logs for monitoring specific endpoints

Here's how to implement this properly without duplicating proxy settings:

server {
    # Main logging configuration
    access_log  /var/log/nginx/jira.access.log full;
    error_log   /var/log/nginx/jira.error.log;
    client_max_body_size 150m;

    # Special location with separate logging
    location /special/ {
        access_log  /var/log/nginx/special.access.log full;
        
        # Inherit proxy settings from parent
        include proxy_params;
        
        proxy_pass http://dowa-02.example.com:8080;
    }

    location / {
        # Shared proxy configuration in external file
        include proxy_params;
        
        proxy_pass http://dowa-02.example.com:8080;
    }
}

1. Using Include Files for Proxy Settings
Create a separate file (e.g., /etc/nginx/proxy_params) with all shared proxy settings:

proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_redirect off;
proxy_buffering off;

2. Log Rotation Configuration
Ensure your log rotation handles both log files:

/var/log/nginx/special.access.log {
    daily
    missingok
    rotate 14
    compress
    delaycompress
    notifempty
    create 0640 www-data adm
    sharedscripts
    postrotate
        /usr/lib/nginx/nginx -s reload
    endscript
}

For more complex scenarios, you can use map directives:

map $request_uri $loggable {
    ~^/special/    0;
    default        1;
}

server {
    access_log /var/log/nginx/jira.access.log full if=$loggable;
    access_log /var/log/nginx/special.access.log full if=$loggable;
    
    # ... rest of configuration
}

When implementing multiple access logs:

  • Disk I/O increases with multiple log files
  • Consider using syslog for high-traffic applications
  • Test impact during peak traffic periods

When working with Nginx as a reverse proxy, we often need granular control over logging behavior. The default configuration writes all access logs to a single file, which becomes problematic when:

  • You need to monitor specific endpoints separately
  • Your main access log becomes too noisy
  • You want to apply different log retention policies

Nginx provides a flexible way to handle this through log directives within location blocks. Here's the proper implementation:

server {
    # Main log configuration
    access_log  /var/log/nginx/main.access.log;
    error_log   /var/log/nginx/error.log;

    location /special/ {
        # Specific log for this location
        access_log /var/log/nginx/special.access.log;
        
        # Inherit proxy settings from parent
        include proxy_params;
        
        # Or directly include your proxy configuration
        proxy_pass http://backend.example.com;
    }

    location / {
        proxy_pass http://backend.example.com;
        # Other proxy settings...
    }
}

For more sophisticated logging needs, you can define custom log formats:

log_format special_format '$remote_addr - $remote_user [$time_local] '
                          '"$request" $status $body_bytes_sent '
                          '"$http_referer" "$http_user_agent" '
                          '$request_time $upstream_response_time';

server {
    access_log /var/log/nginx/main.log;
    
    location /api/ {
        access_log /var/log/nginx/api.log special_format;
        proxy_pass http://api.backend;
    }
}

Consider these optimizations for production setups:

map $uri $loggable {
    ~^/special/ 0;  # Don't log to main log
    default     1;
}

server {
    access_log /var/log/nginx/access.log combined if=$loggable;
    
    location /special/ {
        access_log /var/log/nginx/special.log special_format;
        access_log off;  # Ensure no duplicate logging
        proxy_pass http://backend;
    }
}

When implementing this configuration, watch for:

  • Permission issues on log files (ensure nginx user has write access)
  • Log file rotation conflicts with logrotate
  • Memory usage with extremely high traffic locations