How to Proxy Multiple MySQL/TCP Connections on a Single Port Using Nginx Stream Module


14 views

Unlike HTTP traffic where we can easily virtual host multiple websites on a single port using server_name directives, TCP stream proxying presents unique challenges. The Nginx stream module doesn't support name-based virtual hosting out of the box for raw TCP connections like MySQL.

Attempting to use HTTP-style configuration for TCP streams fails because:

stream {
  upstream db1 {
    server db1.internal:3306;
    # server_name won't work here
  }
  server {
    listen 3306;
    proxy_pass db1;
  }
}

The protocol lacks HTTP headers containing host information that Nginx could inspect for routing decisions.

Here are three viable approaches:

1. Port-Based Routing

The simplest solution uses different external ports:

stream {
  server {
    listen 3306;
    proxy_pass db1.internal:3306;
  }
  server {
    listen 3307;
    proxy_pass db2.internal:3306;
  }
}

2. PROXY Protocol Implementation

For more advanced routing, consider using PROXY protocol:

stream {
  server {
    listen 3306;
    proxy_pass backend;
    proxy_protocol on;
  }

  upstream backend {
    server db1.internal:3306;
    server db2.internal:3306;
  }
}

3. Custom Lua Routing

For maximum flexibility, implement custom routing with Nginx+Lua:

stream {
  server {
    listen 3306;
    content_by_lua_block {
      local sock = ngx.req.socket()
      local data = sock:receive(5)
      if string.match(data, "^pattern1") then
        ngx.exec("@db1")
      else
        ngx.exec("@db2")
      end
    }
  }

  server {
    listen unix:/tmp/db1.sock;
    proxy_pass db1.internal:3306;
  }
  server {
    listen unix:/tmp/db2.sock;
    proxy_pass db2.internal:3306;
  }
}

When implementing these solutions:

  • TCP connection pooling can improve performance
  • Monitor connection latency between proxy and backend servers
  • Consider SSL termination overhead if using encrypted connections

For complex environments, you might consider:

  • Using HAProxy with ACL rules
  • Implementing a custom proxy in Go or Rust
  • Container-based solutions with service discovery

When working with TCP-based services like MySQL, developers often need to proxy multiple backend servers through a single entry point. While HTTP proxying is straightforward with server_name directives, TCP proxying requires a different approach.

The stream module in Nginx provides TCP/UDP proxy capabilities. Unlike HTTP proxying, it operates at the transport layer and doesn't support virtual hosting via hostnames.

# Basic stream configuration
stream {
    server {
        listen 3306;
        proxy_pass backend1:3306;
    }
}

Here are practical approaches to route multiple TCP services through one port:

1. Port Multiplexing with Different Ports

The simplest solution is to use different external ports:

stream {
    server {
        listen 3306;
        proxy_pass db1:3306;
    }
    server {
        listen 3307;
        proxy_pass db2:3306;
    }
}

2. PROXY Protocol Approach

For advanced routing, implement PROXY protocol:

stream {
    server {
        listen 3306 proxy_protocol;
        proxy_pass $upstream;
    }

    map $proxy_protocol_addr $upstream {
        192.168.1.100 db1:3306;
        192.168.1.101 db2:3306;
        default db3:3306;
    }
}

3. Using Different IP Addresses

If your server has multiple IPs:

stream {
    server {
        listen 192.168.1.100:3306;
        proxy_pass db1:3306;
    }
    server {
        listen 192.168.1.101:3306;
        proxy_pass db2:3306;
    }
}

When Nginx's limitations are problematic, consider these alternatives:

  • HAProxy with ACL rules for TCP content inspection
  • SSH port forwarding with connection routing
  • Custom middleware for protocol-specific routing

When implementing TCP proxying:

  • Enable TCP keepalive (proxy_socket_keepalive on;)
  • Adjust buffer sizes (proxy_buffer_size)
  • Monitor connection queue (listen 3306 backlog=4096;)