When working with nginx as a reverse proxy, we often need to intercept and modify responses from upstream servers before delivering them to clients. While nginx provides basic header manipulation, processing response bodies requires more advanced techniques.

We'll implement this using nginx's sub_filter module for simple text replacements, then explore executing external binaries for complex processing:

server {
    listen 10.0.0.66:443;
    server_name my.example.com;

    ssl_certificate /websites/ssl/my.example.com.crt;
    ssl_certificate_key /websites/ssl/my.example.com.key;

    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $remote_addr;
    proxy_set_header Host $http_host;

    location / {
        proxy_pass https://10.0.0.100:3000/;
        
        # Enable response buffering
        proxy_buffering on;
        proxy_buffer_size 128k;
        proxy_buffers 4 256k;
        
        # For simple text replacements
        sub_filter '' '';
        sub_filter_once off;
        
        # For binary processing
        proxy_set_header Accept-Encoding "";
    }
}

For dynamic content transformation, we can use nginx's Lua module:

location / {
    proxy_pass https://10.0.0.100:3000/;
    
    header_filter_by_lua_block {
        ngx.header.content_length = nil
    }
    
    body_filter_by_lua_block {
        local resp = ngx.arg[1]
        if resp and #resp > 0 then
            -- Example: HTML minification
            local handle = io.popen("htmlcompressor --type html", "w")
            handle:write(resp)
            handle:flush()
            handle:close()
            
            -- Alternative: direct Lua processing
            resp = string.gsub(resp, "%s+", " ")
            ngx.arg[1] = resp
        end
    }
}

For complex transformations requiring external binaries:

location / {
    proxy_pass https://10.0.0.100:3000/;
    
    # Disable compression to work with raw content
    proxy_set_header Accept-Encoding "";
    
    # Store response in variable
    set $response_body '';
    
    body_filter_by_lua_block {
        local chunk = ngx.arg[1]
        if chunk ~= "" then
            ngx.var.response_body = ngx.var.response_body .. chunk
        end
        
        if ngx.arg[2] then
            -- Process complete response
            local handle = io.popen("/path/to/processor", "w")
            handle:write(ngx.var.response_body)
            handle:flush()
            handle:close()
            
            -- Alternative: read processed response back
            local processed = io.popen("/path/to/processor"):read("*a")
            ngx.arg[1] = processed
        end
    }
}

When implementing response modification:

• Always benchmark with and without processing
• Consider caching transformed responses when possible
• Set appropriate buffer sizes for expected content
• Monitor memory usage during peak loads

For production systems, consider pre-processing content at origin or using dedicated transformation services rather than doing it in the proxy layer.

When working with Nginx reverse proxies, we often need to manipulate responses from upstream servers before delivering them to clients. While Nginx offers built-in modules for simple transformations, complex processing requires integrating external processors.

We'll use Nginx's sub_filter combined with Lua scripting through OpenResty for maximum flexibility. Here's the complete approach:

server {
    listen 10.0.0.66:443;
    server_name my.example.com;

    ssl_certificate /websites/ssl/my.example.com.crt;
    ssl_certificate_key /websites/ssl/my.example.com.key;

    location / {
        proxy_pass https://10.0.0.100:3000/;
        
        # Enable response buffering
        proxy_buffering on;
        proxy_buffer_size 128k;
        proxy_buffers 4 256k;
        
        # Lua handler for response processing
        header_filter_by_lua_block {
            ngx.header.content_length = nil
        }
        
        body_filter_by_lua_file /etc/nginx/process_response.lua;
    }
}

Create /etc/nginx/process_response.lua:

local shell = require "resty.shell"

local function process_response(body)
    -- Example: HTML minification using htmlcompressor
    local ok, stdout, stderr, reason, status = shell.run([[htmlcompressor --type html]], body)
    
    if not ok then
        ngx.log(ngx.ERR, "Processing failed: ", stderr)
        return body -- Fallback to original
    end
    
    return stdout
end

ngx.ctx.buffer = (ngx.ctx.buffer or "") .. (ngx.arg[1] or "")

if ngx.arg[2] then -- eof
    ngx.arg[1] = process_response(ngx.ctx.buffer)
end

For simpler transformations, consider these built-in Nginx methods:

# 1. Simple string replacement
sub_filter '' '';
sub_filter_once off;

# 2. Gzip manipulation
gunzip on;
gzip on;
gzip_types *;

When processing responses on-the-fly:

• Enable response buffering to prevent memory issues
• Implement proper error handling in Lua scripts
• Consider caching processed responses when possible
• Monitor system resources when using external processors

Here's how to modify JSON responses using jq:

local function process_json(body)
    local ok, stdout = shell.run([[jq '.data.items[] | {id:.id, name:.name}']], body)
    return ok and stdout or body
end