How to Mirror/Duplicate Nginx HTTP Requests to Multiple Backend Servers


2 views

When building testing environments or implementing parallel processing systems, we often need identical HTTP requests (GET/POST/PUT etc.) to be processed by multiple Nginx servers simultaneously. This differs from load balancing where requests are distributed - here we need perfect replication.

1. Using Nginx Mirror Module

The simplest solution is Nginx's native mirror directive (available since 1.13.4):

server {
    listen 80;
    
    location / {
        mirror /mirror;
        mirror_request_body on;
        proxy_pass http://primary_backend;
    }

    location = /mirror {
        internal;
        proxy_pass http://replica_backend$request_uri;
        proxy_pass_request_body on;
        proxy_set_header X-Original-URI $request_uri;
    }
}

2. Lua Scripting for Advanced Control

For more complex scenarios using OpenResty:

location / {
    access_by_lua_block {
        local http = require "resty.http"
        local httpc = http.new()
        
        -- Clone request to secondary server
        local res, err = httpc:request_uri("http://replica-server", {
            method = ngx.req.get_method(),
            body = ngx.req.get_body_data(),
            headers = ngx.req.get_headers()
        })
    }
    
    proxy_pass http://primary_backend;
}
  • Request body handling: Ensure mirror_request_body on or equivalent
  • Header preservation: Use proxy_set_header to forward original headers
  • Error handling: Secondary failures shouldn't affect primary request
  • Performance impact: Async replication is preferable for production

Verify with curl:

curl -X POST http://main-nginx/endpoint \
-H "Content-Type: application/json" \
-d '{"test":"data"}'

Check logs on both primary and replica servers to confirm identical requests were received.


When you need to mirror HTTP traffic from one Nginx server to multiple backend servers (for testing, auditing, or replication purposes), simple load balancing won't work. You need request duplication where Server 1 and Server 2 receive identical copies of each request.

The ngx_http_mirror_module (available since Nginx 1.13.4) is perfect for this:

server {
    listen 80;
    
    location / {
        mirror /mirror;
        proxy_pass http://primary_backend;
    }

    location = /mirror {
        internal;
        proxy_pass http://mirror_backend$request_uri;
        proxy_pass_request_body on;
        proxy_set_header X-Original-URI $request_uri;
    }
}

upstream primary_backend {
    server 10.0.0.1:80;
}

upstream mirror_backend {
    server 10.0.0.2:80;
    server 10.0.0.3:80;
}

For proper request mirroring including POST data:

location / {
    mirror /mirror;
    mirror_request_body on;
    proxy_pass http://primary_backend;
    
    # Preserve original headers
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
}

For conditional mirroring based on traffic patterns:

split_clients "${remote_addr}${http_user_agent}" $mirror_ratio {
    50%     "1";
    *       "0";
}

server {
    location / {
        if ($mirror_ratio = "1") {
            mirror /mirror;
        }
        proxy_pass http://primary_backend;
    }
}
  • Mirrored requests are fire-and-forget - Nginx won't wait for mirror responses
  • For high traffic, consider proxy_request_buffering off to reduce memory usage
  • Always test with curl -X POST to verify POST data mirroring

For more control with OpenResty:

location / {
    access_by_lua_block {
        local http = require "resty.http"
        local httpc = http.new()
        
        -- Clone request to mirror
        local res, err = httpc:request_uri("http://mirror_backend", {
            method = ngx.req.get_method(),
            body = ngx.req.get_body_data(),
            headers = ngx.req.get_headers()
        })
    }
    
    proxy_pass http://primary_backend;
}