How to Dynamically Load ACL Hosts from a File in HAProxy for Large-Scale Migrations


4 views

When migrating dozens or hundreds of websites between servers, maintaining individual ACL entries in your HAProxy configuration becomes unwieldy. The standard approach of listing each domain separately:

acl is_new hdr_end(host) -i sub1.domain.com
acl is_new hdr_end(host) -i sub2.domain.com
acl is_new hdr_end(host) -i www.domain2.com

quickly becomes difficult to manage and violates the DRY (Don't Repeat Yourself) principle we value as developers.

While HAProxy doesn't natively support reading ACL patterns directly from files like it does with SSL certificates, we can leverage its file inclusion capability with some creative configuration:

frontend http_frontend
  bind *:80
  bind *:443 ssl crt /etc/haproxy/certs.d
  include /etc/haproxy/acls.d/new_hosts.acl
  mode http
  use_backend web1 if is_new
  default_backend legacy1

Create a dedicated directory for your ACL files and generate the ACL entries programmatically:

# /etc/haproxy/acls.d/new_hosts.acl
acl is_new hdr_end(host) -i sub1.domain.com
acl is_new hdr_end(host) -i sub2.domain.com
acl is_new hdr_end(host) -i www.domain2.com
# ... plus 97 more entries

For truly dynamic management, integrate with your existing automation tools:

# Example Ansible task to generate ACL file
- name: Generate HAProxy ACLs
  template:
    src: haproxy_acls.j2
    dest: /etc/haproxy/acls.d/new_hosts.acl
  notify: reload haproxy

# haproxy_acls.j2 template
{% for domain in migrated_domains %}
acl is_new hdr_end(host) -i {{ domain }}
{% endfor %}

For very large deployments (1000+ domains), consider using HAProxy's map files for better performance:

# /etc/haproxy/maps.d/new_hosts.map
sub1.domain.com 1
sub2.domain.com 1
www.domain2.com 1

# In haproxy.cfg
frontend http_frontend
  bind *:80
  use_backend web1 if { hdr_end(host) -f /etc/haproxy/maps.d/new_hosts.map -m found }
  default_backend legacy1

Always verify your configuration before applying changes:

haproxy -c -f /etc/haproxy/haproxy.cfg

Consider implementing canary testing by moving a small percentage of traffic first:

use_backend web1 if is_new rand(100) lt 5  # 5% traffic

When migrating hundreds of websites between servers using HAProxy, managing individual ACL entries in the configuration file becomes cumbersome. The traditional approach requires adding each domain manually:

frontend http_frontend
  bind *:80
  acl is_new hdr_end(host) -i sub1.domain.com
  acl is_new hdr_end(host) -i sub2.domain.com
  acl is_new hdr_end(host) -i www.domain2.com
  # ... repeat for 100+ domains
  use_backend web1 if is_new
  default_backend legacy1

HAProxy doesn't natively support loading ACL rules from external files, but we can leverage Lua scripting (available in HAProxy 1.6+) to achieve this functionality:

global
  lua-load /etc/haproxy/acl_loader.lua

frontend http_frontend
  bind *:80
  http-request lua.acl_loader
  use_backend web1 if { var(txn.host_matched) -m bool }
  default_backend legacy1

Create /etc/haproxy/acl_loader.lua:

core.register_action("acl_loader", { "http-req" }, function(txn)
    local host = txn.sf:req_fhdr("host")
    if not host then return end
    
    local file = io.open("/etc/haproxy/migrated_hosts.txt", "r")
    if not file then return end
    
    for line in file:lines() do
        line = line:gsub("%s+", "") -- remove whitespace
        if line ~= "" and host:match(line:gsub("%.", "%%.")) then
            txn:set_var("txn.host_matched", true)
            break
        end
    end
    file:close()
end)

The /etc/haproxy/migrated_hosts.txt should contain one domain per line:

sub1.domain.com
sub2.domain.com
www.domain2.com
# Additional domains...

For static configurations, HAProxy maps provide better performance:

frontend http_frontend
  bind *:80
  http-request set-var(txn.is_new) str(map_dom(/etc/haproxy/migrated_hosts.map))
  use_backend web1 if { var(txn.is_new) eq "1" }
  default_backend legacy1

Where migrated_hosts.map contains:

sub1.domain.com 1
sub2.domain.com 1
www.domain2.com 1

The Lua approach has some overhead (about 1ms per request) while maps are processed at configuration load time. For high-traffic sites, consider:

  • Using maps for production after testing
  • Keeping Lua for development/staging environments
  • Implementing caching in the Lua script