How to Configure HAProxy HTTPS Health Check for Vault Server Backends


2 views

When migrating Vault servers from HTTP to HTTPS, many administrators face broken health checks in their HAProxy configurations. The TCP-mode health checks that worked perfectly for HTTP backends suddenly stop detecting server status accurately, potentially routing traffic to sealed or unhealthy Vault instances.

The original HTTP configuration uses simple HTTP checks which won't work with TLS backends because:

  • SSL/TLS requires proper handshake before HTTP communication
  • Standard httpchk doesn't account for certificate verification
  • TCP mode alone can't inspect application layer health

Here's the proper configuration for HTTPS health checks with Vault:

listen vault-https
  bind 0.0.0.0:443
  mode tcp
  balance roundrobin
  option ssl-hello-chk
  option httpchk HEAD /v1/sys/health HTTP/1.1\r\nHost:\ vault.example.com
  http-check expect status 200
  server vault1 :8200 check ssl verify none
  server vault2 :8200 check ssl verify none
  server vault3 :8200 check ssl verify none

SSL parameters: The ssl parameter enables TLS for backend connections while verify none skips certificate validation (adjust based on your security requirements).

Enhanced HTTP check: The complete HTTP/1.1 request format including Host header ensures proper routing through TLS termination points.

For production environments with strict security requirements:

backend vault-https-secure
  mode tcp
  option httpchk HEAD /v1/sys/health HTTP/1.1\r\nHost:\ vault.example.com
  http-check expect status 200
  server vault1 :8200 
    check 
    ssl 
    verify required 
    ca-file /etc/ssl/certs/vault-ca.pem
    crt /etc/ssl/certs/haproxy-client.pem
  • Use openssl s_client -connect to verify backend TLS connectivity
  • Check HAProxy logs with increased verbosity (debug mode)
  • Test health checks manually with curl: curl -k https://vault-server:8200/v1/sys/health

For high-traffic Vault clusters:

  • Set appropriate timeout values for health checks
  • Consider using rise and fall parameters to prevent flapping
  • Monitor TLS handshake overhead in your performance metrics

When migrating Vault servers from HTTP to HTTPS, many administrators encounter health check failures in HAProxy configurations. The transition breaks existing HTTP health checks while maintaining TCP-level connectivity.

The original HTTP configuration worked well:

listen vault
  bind 0.0.0.0:443
  mode tcp
  balance roundrobin
  option httpchk HEAD /v1/sys/health
  http-check expect status 200
  option tcplog
  option ssl-hello-chk
  server web1 :8200 check
  server web2 :8200 check

This checks for HTTP 200 responses but fails when Vault enforces HTTPS.

For HTTPS health checks, we need three key adjustments:

listen vault-tls
  bind 0.0.0.0:443
  mode tcp
  balance roundrobin
  option httpchk HEAD /v1/sys/health
  http-check expect status 200
  option ssl-hello-chk
  server web1 :8200 check ssl verify none
  server web2 :8200 check ssl verify none

For production environments, consider these enhanced settings:

backend vault_https
  mode tcp
  option httpchk HEAD /v1/sys/health HTTP/1.1\r\nHost:\ vault.example.com
  http-check send hdr Authorization "Bearer ${VAULT_TOKEN}"
  http-check expect status 200
  server vault1 10.0.1.10:8200 check ssl ca-file /etc/ssl/certs/ca-certificates.crt
  server vault2 10.0.1.11:8200 check ssl ca-file /etc/ssl/certs/ca-certificates.crt

While verify none works for testing, production should use proper certificate verification:

server web1 :8200 check \
  ssl ca-file /etc/haproxy/vault-ca.pem \
  sni str(vault.example.com) \
  verify required

Use these commands to verify health check behavior:

# Check server status
echo "show servers state" | socat stdio /var/run/haproxy/admin.sock

# Verify SSL connectivity
openssl s_client -connect vault-server:8200 -showcerts

HTTPS health checks add overhead. Optimize with:

option httpchk HEAD /v1/sys/health
default-server inter 10s fastinter 2s downinter 5s

When HTTP checks aren't possible, consider basic TCP checks:

option tcp-check
tcp-check connect ssl
tcp-check send GET\ /v1/sys/health\ HTTP/1.1\r\n
tcp-check expect rstring (200|429)