How to Configure HAProxy for Dynamic DNS Resolution of Backend Servers


2 views

When using HAProxy (particularly version 1.5.8), you might encounter a common infrastructure headache: backend servers specified by hostnames get resolved only once during startup. This becomes problematic when third-party services change their IP addresses without notice - exactly the scenario described by our user who proxies requests to external services.

The default behavior creates a single point of failure in your infrastructure. If the backend server's IP changes due to:

  • Cloud provider maintenance
  • Auto-scaling events
  • DNS failover mechanisms
  • CDN IP rotations

Your HAProxy instance will continue sending traffic to the stale IP address until manually reloaded.

For newer HAProxy versions, the solution is straightforward:


backend external_services
    server srv1 api.example.com:80 check resolvers mydns

resolvers mydns
    nameserver dns1 8.8.8.8:53
    hold valid 10s

For those stuck with older versions, we need to get creative. Here's a solution using a cron job:


#!/bin/bash
# /etc/cron.d/haproxy-dns-refresh
*/5 * * * * root current_ip=$(getent hosts api.example.com | awk '{ print $1 }') && sed -i "s/server srv1.*/server srv1 ${current_ip}:80/" /etc/haproxy/haproxy.cfg && service haproxy reload

Another approach uses template files and environment variables:


# haproxy-template.cfg
backend dynamic_backend
    server srv1 ${BACKEND_IP}:80

# refresh.sh
export BACKEND_IP=$(dig +short api.example.com)
envsubst < haproxy-template.cfg > haproxy.cfg
haproxy -f haproxy.cfg -sf $(cat /var/run/haproxy.pid)

While these workarounds function, they introduce complexity. Modern HAProxy versions offer superior solutions:

  • Built-in DNS resolution (1.6+)
  • Server templates (1.8+)
  • Dynamic server API (1.9+)

The upgrade path might ultimately be the most maintainable solution.

When working with HAProxy (particularly version 1.5.8), one frustrating limitation is its default behavior of resolving backend server hostnames only during startup. This becomes problematic when:

  • Deploying with cloud providers that change IP addresses
  • Using DNS-based load balancing
  • Working with third-party services that rotate IPs

Unlike Nginx which allows DNS caching through variables, HAProxy traditionally maintains a static resolution. Here's what happens under the hood:

# Traditional HAProxy backend configuration
backend static_backend
    server thirdparty example.com:80 check
    # ^ Resolves only at startup

While HAProxy 1.5.8 doesn't have native DNS TTL support, these approaches work:

Option 1: Runtime DNS Resolution with resolvers

Available in HAProxy 1.6+, but can be backported to 1.5.8 with some tweaks:

resolvers mydns
    nameserver dns1 8.8.8.8:53
    hold valid 10s

backend dynamic_backend
    server thirdparty example.com:80 check resolvers mydns resolve-prefer ipv4

Option 2: External DNS Watcher Script

Create a watchdog script that:

#!/bin/bash
while true; do
    new_ip=$(dig +short example.com | head -1)
    if [ "$new_ip" != "$current_ip" ]; then
        echo "set server dynamic_backend/thirdparty addr $new_ip" | socat stdio /var/run/haproxy.sock
        current_ip=$new_ip
    fi
    sleep 30
done

Option 3: Upgrade Path Considerations

For production environments, consider upgrading to modern HAProxy versions (2.0+) that include:

  • Native DNS service discovery
  • Runtime API for dynamic configuration
  • Built-in health checks with DNS re-resolution

When implementing dynamic DNS:

# Always include:
timeout connect 5s
timeout server 30s
timeout client 30s

# For resolver configuration:
defaults
    log global
    mode http
    timeout connect 5s
    timeout client 30s
    timeout server 30s
    option dontlognull

Monitor your HAProxy stats page closely after implementation to verify proper DNS rotation.