When setting up an Nginx reverse proxy for API routing, one common frustration is debugging failed requests where:

1. The client request reaches Nginx successfully
2. Nginx modifies and forwards the request
3. The upstream API responds (or fails)
4. Nginx processes and returns the response

Here's how to implement comprehensive logging at each stage:

log_format proxy_logs '$remote_addr - $remote_user [$time_local] '
                     '"$request" $status $body_bytes_sent '
                     '"$http_referer" "$http_user_agent" '
                     'UPSTREAM: "$upstream_addr" '
                     'REQ_HEADERS: "$request_headers" '
                     'RES_HEADERS: "$upstream_http_*" '
                     'RES_BODY: "$upstream_response"';

map $upstream_http_content_type $loggable {
    ~*^application/json   1;
    ~*^text/              1;
    default               0;
}

location ~ /api/(?<path>.*) {
    # Request logging
    set $request_headers "";
    more_set_input_headers 'X-Log-Request: $request';
    
    # Response logging
    more_set_headers -s '400 401 403 404 500 502 503 504' 'X-Log-Response: $upstream_response';
    
    access_log /var/log/nginx/api_access.log proxy_logs;
    error_log  /var/log/nginx/api_error.log debug;
    
    proxy_pass http://$api_route/$path$is_args$args;
    proxy_set_header X-Original-URI $request_uri;
    proxy_set_header X-Forwarded-Proto $scheme;
    
    # Capture response body when loggable
    if ($loggable) {
        set $resp_body "";
        body_filter_by_lua_block {
            ngx.var.resp_body = ngx.arg[1]
        }
    }
}

For a complete debugging setup, combine these modules:

# In main nginx.conf
load_module modules/ngx_http_headers_more_filter_module.so;
load_module modules/ngx_http_lua_module.so;

http {
    lua_shared_dict log_dict 10m;
    
    server {
        location /api {
            # Enable Lua for advanced logging
            access_by_lua_block {
                ngx.shared.log_dict:set("req_" .. ngx.var.request_id, 
                    "Method: " .. ngx.var.request_method .. 
                    " | Headers: " .. cjson.encode(ngx.req.get_headers()))
            }
            
            log_by_lua_block {
                local resp = {
                    status = ngx.var.status,
                    headers = ngx.resp.get_headers(),
                    body = ngx.var.resp_body
                }
                ngx.shared.log_dict:set("res_" .. ngx.var.request_id, cjson.encode(resp))
            }
        }
    }
}

Sample log entries will show:

• Original Request: Method, headers, body (if POST/PUT)
• Proxied Request: Final URI with parameters, modified headers
• API Response: Status code, headers, and truncated body

For production environments, consider log rotation and sensitive data masking:

logrotate -f /etc/logrotate.d/nginx

When working with Nginx as an API proxy gateway, one crucial debugging challenge is tracing the complete request/response flow between:

• The client request hitting your Nginx server
• The proxied request sent to your upstream API
• The response traveling back through Nginx to the client

Your current configuration shows timeout issues when calling /api endpoints, making detailed logging essential for troubleshooting.

Add these directives to your Nginx config for comprehensive logging:

log_format proxy_logs '$remote_addr - $remote_user [$time_local] '
                     '"$request" $status $body_bytes_sent '
                     '"$http_referer" "$http_user_agent" '
                     'upstream: $upstream_addr '
                     'upstream_status: $upstream_status '
                     'request_time=$request_time '
                     'upstream_response_time=$upstream_response_time '
                     'upstream_connect_time=$upstream_connect_time '
                     'proxy_host=$proxy_host';

server {
    # ... your existing server block ...
    access_log /var/log/nginx/api_proxy_access.log proxy_logs;
    error_log /var/log/nginx/api_proxy_error.log debug;
    
    location ~ /api/(?.*) {
        # ... your existing location block ...
        
        # Add these for request/response inspection
        proxy_set_header X-Request-Body $request_body;
        proxy_pass_request_headers on;
        proxy_pass_request_body on;
    }
}

For targeted debugging, use map directives to log specific conditions:

map $status $loggable {
    ~^[45]  1;
    default 0;
}

server {
    # ... existing config ...
    
    access_log /var/log/nginx/api_errors.log proxy_logs if=$loggable;
}

For complete request/response inspection, use Lua scripting (requires ngx_http_lua_module):

location ~ /api/(?.*) {
    # ... existing config ...
    
    set $req_body "";
    set $resp_body "";
    
    access_by_lua_block {
        ngx.var.req_body = ngx.req.get_body_data()
    }
    
    body_filter_by_lua_block {
        local resp_body = string.sub(ngx.arg[1], 1, 1000)
        ngx.ctx.resp_body = resp_body
    }
    
    log_by_lua_block {
        ngx.log(ngx.INFO, "Request Body: ", ngx.var.req_body)
        ngx.log(ngx.INFO, "Response Body: ", ngx.ctx.resp_body)
    }
}

When you encounter timeout issues, check these logs:

1. tail -f /var/log/nginx/api_proxy_error.log - Shows detailed debug info
2. grep "upstream_response_time" /var/log/nginx/api_proxy_access.log - Identifies slow upstream responses
3. journalctl -u nginx --since "5 minutes ago" - Checks system-level issues

Remember to:

• Rotate logs regularly (logrotate)
• Never log sensitive headers like Authorization
• Set appropriate permissions on log files
• Disable debug logging in production