When migrating Vault servers from HTTP to HTTPS, many administrators face broken health checks in their HAProxy configurations. The TCP-mode health checks that worked perfectly for HTTP backends suddenly stop detecting server status accurately, potentially routing traffic to sealed or unhealthy Vault instances.
The original HTTP configuration uses simple HTTP checks which won't work with TLS backends because:
- SSL/TLS requires proper handshake before HTTP communication
- Standard httpchk doesn't account for certificate verification
- TCP mode alone can't inspect application layer health
Here's the proper configuration for HTTPS health checks with Vault:
listen vault-https
bind 0.0.0.0:443
mode tcp
balance roundrobin
option ssl-hello-chk
option httpchk HEAD /v1/sys/health HTTP/1.1\r\nHost:\ vault.example.com
http-check expect status 200
server vault1 :8200 check ssl verify none
server vault2 :8200 check ssl verify none
server vault3 :8200 check ssl verify none
SSL parameters: The ssl
parameter enables TLS for backend connections while verify none
skips certificate validation (adjust based on your security requirements).
Enhanced HTTP check: The complete HTTP/1.1 request format including Host header ensures proper routing through TLS termination points.
For production environments with strict security requirements:
backend vault-https-secure
mode tcp
option httpchk HEAD /v1/sys/health HTTP/1.1\r\nHost:\ vault.example.com
http-check expect status 200
server vault1 :8200
check
ssl
verify required
ca-file /etc/ssl/certs/vault-ca.pem
crt /etc/ssl/certs/haproxy-client.pem
- Use
openssl s_client -connect
to verify backend TLS connectivity - Check HAProxy logs with increased verbosity (
debug
mode) - Test health checks manually with curl:
curl -k https://vault-server:8200/v1/sys/health
For high-traffic Vault clusters:
- Set appropriate
timeout
values for health checks - Consider using
rise
andfall
parameters to prevent flapping - Monitor TLS handshake overhead in your performance metrics
When migrating Vault servers from HTTP to HTTPS, many administrators encounter health check failures in HAProxy configurations. The transition breaks existing HTTP health checks while maintaining TCP-level connectivity.
The original HTTP configuration worked well:
listen vault
bind 0.0.0.0:443
mode tcp
balance roundrobin
option httpchk HEAD /v1/sys/health
http-check expect status 200
option tcplog
option ssl-hello-chk
server web1 :8200 check
server web2 :8200 check
This checks for HTTP 200 responses but fails when Vault enforces HTTPS.
For HTTPS health checks, we need three key adjustments:
listen vault-tls
bind 0.0.0.0:443
mode tcp
balance roundrobin
option httpchk HEAD /v1/sys/health
http-check expect status 200
option ssl-hello-chk
server web1 :8200 check ssl verify none
server web2 :8200 check ssl verify none
For production environments, consider these enhanced settings:
backend vault_https
mode tcp
option httpchk HEAD /v1/sys/health HTTP/1.1\r\nHost:\ vault.example.com
http-check send hdr Authorization "Bearer ${VAULT_TOKEN}"
http-check expect status 200
server vault1 10.0.1.10:8200 check ssl ca-file /etc/ssl/certs/ca-certificates.crt
server vault2 10.0.1.11:8200 check ssl ca-file /etc/ssl/certs/ca-certificates.crt
While verify none
works for testing, production should use proper certificate verification:
server web1 :8200 check \
ssl ca-file /etc/haproxy/vault-ca.pem \
sni str(vault.example.com) \
verify required
Use these commands to verify health check behavior:
# Check server status
echo "show servers state" | socat stdio /var/run/haproxy/admin.sock
# Verify SSL connectivity
openssl s_client -connect vault-server:8200 -showcerts
HTTPS health checks add overhead. Optimize with:
option httpchk HEAD /v1/sys/health
default-server inter 10s fastinter 2s downinter 5s
When HTTP checks aren't possible, consider basic TCP checks:
option tcp-check
tcp-check connect ssl
tcp-check send GET\ /v1/sys/health\ HTTP/1.1\r\n
tcp-check expect rstring (200|429)