When trying to reuse an upstream group with different ports in multiple server blocks, Nginx throws the error: upstream "production" may not have port 1234. This occurs despite the documentation suggesting variables should work for server names and ports in proxy_pass directives.

The key misunderstanding lies in how Nginx processes variables in proxy_pass when using upstream groups. While the documentation states variables can be used, there's an important nuance:

# This WON'T work as expected with upstream groups
set $upstream_host production;
set $upstream_port 1234;
proxy_pass http://$upstream_host:$upstream_port;

When variables are used, Nginx bypasses the upstream group resolution and tries to resolve the hostname through DNS instead.

Option 1: Separate Upstream Blocks

The most straightforward solution is to define separate upstream blocks for each port:

upstream production_1234 {
    server 10.240.0.26:1234;
    server 10.240.0.27:1234;
}

upstream production_4321 {
    server 10.240.0.26:4321;
    server 10.240.0.27:4321;
}

server {
    listen 80;
    server_name some.host;
    location / {
        proxy_pass http://production_1234;
    }
}

server {
    listen 80;
    server_name other.host;
    location / {
        proxy_pass http://production_4321;
    }
}

Option 2: Rewrite with Maps

For more dynamic configurations, you can use the map module:

map $host $backend {
    some.host    "10.240.0.26:1234,10.240.0.27:1234";
    other.host   "10.240.0.26:4321,10.240.0.27:4321";
}

server {
    listen 80;
    server_name some.host other.host;
    
    location / {
        set $target_backend $backend;
        proxy_pass http://$target_backend;
    }
}

If you're using OpenResty or have the Lua module installed, you can implement more complex routing:

server {
    listen 80;
    
    set_by_lua $backend_port '
        if ngx.var.host == "some.host" then
            return "1234"
        else
            return "4321"
        end
    ';
    
    location / {
        proxy_pass http://production:$backend_port;
    }
}

Note that this requires additional module configuration and isn't suitable for all environments.

While the separate upstream blocks approach might seem redundant, it actually offers better performance:

• Nginx can optimize connection pooling per upstream group
• Health checks can be configured per port
• Load balancing metrics remain separate for different services

Watch out for these issues when implementing multi-port upstreams:

1. Ensure all servers in the upstream block are actually listening on the specified ports
2. Remember that Nginx won't automatically retry failed connections to different ports
3. Be consistent with protocol (http/https) across all servers in an upstream group