When implementing mutual TLS authentication in Nginx, a common challenge arises when you need different client certificate requirements for virtual hosts sharing the same IP address and port combination. The original configuration attempt shows a valid approach but misses some critical SSL handshake nuances.

The error "No required SSL certificate was sent" appears because the SSL handshake occurs before Nginx can determine which virtual host to use based on the SNI (Server Name Indication). Since client certificate verification is part of the SSL handshake, this creates a chicken-and-egg problem.

Here's a tested solution that separates the SSL contexts while maintaining shared IP functionality:

# Common SSL settings
ssl_certificate /etc/certs/server.cer;
ssl_certificate_key /etc/certs/privkey-server.pem;
ssl_client_certificate /etc/certs/allcas.pem;

server {
    listen 1443 ssl;
    server_name server1.example.com;
    root /tmp/root/server1;
    
    # Basic SSL config without client verification
    ssl_verify_client optional_no_ca;
    ssl_verify_depth 0;
}

server {
    listen 1443 ssl;
    server_name server2.example.com;
    root /tmp/root/server2;
    
    # Strict client verification
    ssl_verify_client on;
    ssl_verify_depth 2;
    
    # Additional security
    if ($ssl_client_verify != SUCCESS) {
        return 403;
    }
}

The solution leverages these Nginx features:

• optional_no_ca: Allows the client to decide whether to send a certificate for non-verified hosts
• SNI awareness: Modern Nginx versions properly handle SNI during SSL negotiation
• Post-handshake verification: The $ssl_client_verify check occurs after host determination

Use these curl commands to verify both scenarios work:

# For non-client-cert host
curl -k --resolve server1.example.com:1443:127.0.0.1 https://server1.example.com:1443/

# For client-cert required host
curl -k --resolve server2.example.com:1443:127.0.0.1 \
--cert /path/to/client.crt --key /path/to/client.key \
https://server2.example.com:1443/

If you still encounter issues, binding to different IPs remains a reliable fallback. Update the listen directives:

server {
    listen 192.168.1.10:1443 ssl;
    # ... server1 config
}

server {
    listen 192.168.1.20:1443 ssl;
    # ... server2 config with client verification
}

When configuring multiple virtual hosts on the same Nginx server with different SSL client verification requirements, we encounter a fundamental limitation in SSL/TLS protocol behavior. The client certificate verification must be negotiated during the initial SSL handshake - before Nginx can determine which virtual host (server_name) the request is intended for.

The configuration you've provided attempts to serve two virtual hosts (server1.example.com and server2.example.com) on the same IP and port (1443) with different client certificate requirements:

ssl_certificate /etc/certs/server.cer;
ssl_certificate_key /etc/certs/privkey-server.pem;
ssl_client_certificate /etc/certs/allcas.pem;

server {
    listen 1443 ssl;
    server_name server1.example.com;
    ssl_verify_client off;
}

server {
    listen 1443 ssl;
    server_name server2.example.com;
    ssl_verify_client on;
}

The "400 Bad Request" error occurs because SSL client certificate verification happens before the HTTP layer where server_name is processed. When a client connects to port 1443, Nginx must immediately determine whether to require a client certificate - it can't wait to see which hostname the client requests.

Option 1: Different Ports

The simplest solution is to use separate ports:

server {
    listen 1443 ssl;
    server_name server1.example.com;
    ssl_verify_client off;
}

server {
    listen 1444 ssl;
    server_name server2.example.com;
    ssl_verify_client on;
}

Option 2: Different IP Addresses

If you must use the same port, bind to separate IPs:

server {
    listen 192.0.2.1:1443 ssl;
    server_name server1.example.com;
    ssl_verify_client off;
}

server {
    listen 192.0.2.2:1443 ssl;
    server_name server2.example.com;
    ssl_verify_client on;
}

Option 3: Conditional Verification (Nginx 1.19.4+)

Recent Nginx versions support dynamic SSL verification:

server {
    listen 1443 ssl;
    server_name server2.example.com;

    ssl_verify_client on;
    ssl_verify_client optional;

    if ($host != "server2.example.com") {
        ssl_verify_client off;
    }
}

To verify client certificate behavior, use OpenSSL:

# Test server without client cert requirement
openssl s_client -connect example.com:1443 -servername server1.example.com

# Test server requiring client cert
openssl s_client -connect example.com:1443 -servername server2.example.com -cert client.crt -key client.key

When implementing this in production:

• Ensure your ssl_client_certificate points to all CA certificates that should be trusted
• Consider using ssl_verify_depth to control certificate chain validation
• For Option 3, be aware of potential performance implications

Remember that TLS SNI (Server Name Indication) occurs after the initial handshake, which is why these limitations exist in the first place.