Debugging HAProxy 503 Error: Fixing “No Server Available” in Load Balancer Configuration


2 views

When setting up HAProxy with keepalived on CentOS, you're encountering the dreaded "503 Service Unavailable - No server is available to handle this request" error, despite your backend web servers (192.168.1.12 and 192.168.1.13) being manually accessible. Let's break this down systematically.

First, verify these critical configuration elements:

# Network accessibility check
ping 192.168.1.12
ping 192.168.1.13
telnet 192.168.1.12 80
telnet 192.168.1.13 80

# HAProxy process check
ps aux | grep haproxy
netstat -tulnp | grep haproxy

Your configuration shows a significant structural issue:

frontend main *:80
    # No default_backend defined!

backend app
    mode tcp
    server server1 192.168.1.12:80 check inter 2000 rise 2 fall 5
    server server2 192.168.1.13:80 check inter 2000 rise 2 fall 5

The frontend isn't routing to any backend. Fix this by adding:

frontend main *:80
    default_backend app

You're mixing HTTP and TCP modes:

defaults
    mode http  # Global HTTP mode

backend app
    mode tcp   # Backend TCP mode - conflicting!

For HTTP load balancing, use consistent mode:

backend app
    mode http
    balance roundrobin
    option httpchk GET /
    http-check expect status 200
    server server1 192.168.1.12:80 check inter 2000 rise 2 fall 5
    server server2 192.168.1.13:80 check inter 2000 rise 2 fall 5

Add proper health monitoring:

backend app
    option httpchk HEAD / HTTP/1.1\r\nHost:\ example.com
    http-check expect rstatus (2|3)[0-9][0-9]
    server server1 192.168.1.12:80 check port 80 inter 5s fastinter 1s
    server server2 192.168.1.13:80 check port 80 inter 5s fastinter 1s

Enable detailed logging to troubleshoot:

global
    log /dev/log local0 debug

defaults
    log global
    option tcplog  # For TCP mode

View logs with:

tail -f /var/log/haproxy.log

On CentOS, ensure proper permissions:

setsebool -P haproxy_connect_any=1
semanage port -a -t http_port_t -p tcp 80

Here's a verified working template:

global
    log /dev/log local0
    chroot /var/lib/haproxy
    user haproxy
    group haproxy
    daemon

defaults
    mode http
    log global
    option httplog
    timeout connect 5000
    timeout client 50000
    timeout server 50000

frontend http-in
    bind *:80
    default_backend servers

backend servers
    balance roundrobin
    server server1 192.168.1.12:80 check
    server server2 192.168.1.13:80 check

listen stats
    bind *:1936
    stats enable
    stats uri /
    stats hide-version
    stats auth admin:password

Verify with:

curl -I http://192.168.1.6

When working with HAproxy for load balancing web servers, encountering the "503 Service Unavailable - No server is available to handle this request" error typically indicates one of several configuration issues. In your specific environment where manual connections to backend servers (192.168.1.12 and 192.168.1.13) work fine but HAproxy fails to route traffic, we need to examine several critical components.

Your HAproxy configuration shows a potential mismatch between frontend and backend declarations. The frontend section doesn't properly route traffic to your defined backend. Here's the corrected version:

frontend main *:80
    default_backend app

backend app
    mode http
    balance roundrobin
    option httpchk
    server server1 192.168.1.12:80 check inter 2000 rise 2 fall 5
    server server2 192.168.1.13:80 check inter 2000 rise 2 fall 5

1. Changed mode from tcp to http in backend for proper HTTP health checks
2. Added default_backend directive in frontend section
3. Included httpchk option for better health monitoring

Before testing HAproxy, ensure basic connectivity works:

# Test backend servers directly
curl -I http://192.168.1.12
curl -I http://192.168.1.13

# Check HAproxy stats (add this to your config)
listen stats
    bind *:1936
    stats enable
    stats uri /
    stats hide-version
    stats auth admin:password
  • Firewall blocking HAproxy backend connections (check iptables/security groups)
  • SELinux preventing network connections (temporarily set to permissive mode for testing)
  • Incorrect interface binding (verify with netstat -tulpn)
  • Missing health check configuration (always include 'check' parameter)

For production environments, consider these enhancements:

backend app
    mode http
    balance leastconn
    cookie SERVERID insert indirect nocache
    option httpchk HEAD / HTTP/1.1\r\nHost:\ example.com
    http-check expect status 200
    server server1 192.168.1.12:80 check cookie s1 inter 2s
    server server2 192.168.1.13:80 check cookie s2 inter 2s