How to Block a User Agent Across All Sites on Apache Server Configuration


3 views

When your Apache server gets hammered by repetitive requests from a specific user agent like Mozilla/4.0 (compatible; ICS), it can quickly exhaust server resources. The standard approach using SetEnvIfNoCase and Deny directives in httpd.conf fails because modern Apache versions enforce context-sensitive directive placement.

The most effective way to implement server-wide blocking is through Apache's mod_rewrite module. Add this to your main server configuration or virtual host:


<IfModule mod_rewrite.c>
    RewriteEngine On
    RewriteCond %{HTTP_USER_AGENT} ^Mozilla/4\.0\ $compatible;\ ICS$ [NC]
    RewriteRule ^ - [F,L]
</IfModule>

For newer Apache installations, leverage Require directives instead of deprecated Deny:


<IfModule mod_setenvif.c>
    SetEnvIfNoCase User-Agent "Mozilla/4.0 (compatible; ICS)" badagent
</IfModule>

<IfModule mod_authz_core.c>
    <RequireAll>
        Require all granted
        Require not env badagent
    </RequireAll>
</IfModule>

When dealing with DDoS-level attacks, consider these optimizations:

  • Place rules in apache2.conf rather than .htaccess for better performance
  • Combine with mod_evasive for rate limiting
  • Use fail2ban to automatically block offending IPs

After implementation, verify using curl:


curl -A "Mozilla/4.0 (compatible; ICS)" https://yourdomain.com

You should receive a 403 Forbidden response. Monitor server logs to confirm the blocking effectiveness:


tail -f /var/log/apache2/access.log | grep " 403 "

Recently, my Apache server has been hammered by requests from a user agent identifying as Mozilla/4.0 (compatible; ICS). The sheer volume of requests was consuming all available memory, effectively creating a denial-of-service situation. While this might not be a deliberate DDoS attack, the impact was just as severe.

My first instinct was to use SetEnvIfNoCase combined with Deny in the main httpd.conf:

SetEnvIfNoCase User-Agent "Mozilla/4.0 (compatible; ICS)" bad_user
Deny from env=bad_user

However, Apache threw an error:

Syntax error on line 4 of /etc/apache2/httpd.conf: deny not allowed here

The issue here is that Deny directives can't be used in the main server context without being wrapped in a Location, Directory, or Files section. This limitation exists because access control needs to be applied to specific resources, not globally.

A more elegant solution is to use mod_rewrite in the main server configuration or virtual host files. This approach works globally without needing to modify every site's configuration:

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^Mozilla/4\.0\ $compatible;\ ICS$ [NC]
RewriteRule ^ - [F,L]

This configuration:

  • Checks if the User-Agent matches our problematic string (case-insensitive)
  • Returns a 403 Forbidden response for all matching requests
  • Applies to all sites on the server when placed in the main configuration

For Apache 2.4+, you can use the newer authorization framework:

<IfModule mod_authz_core.c>
    <If "%{HTTP_USER_AGENT} =~ /Mozilla\/4\.0 $compatible; ICS$/i">
        Require all denied
    </If>
</IfModule>

When dealing with high-volume attacks, consider these optimizations:

# Cache the regex compilation
RewriteMap ua-map prg:/path/to/user-agent-filter.pl
RewriteCond %{HTTP_USER_AGENT} ^Mozilla/4\.0\ $compatible;\ ICS$ [NC,OR]
RewriteCond ${ua-map:%{HTTP_USER_AGENT}|NOTFOUND} !=NOTFOUND
RewriteRule ^ - [F,L]

After implementation, monitor your access logs to verify effectiveness:

tail -f /var/log/apache2/access.log | grep "Mozilla/4.0 (compatible; ICS)"

Or count blocked requests:

grep -c "403 .*Mozilla/4.0 (compatible; ICS)" /var/log/apache2/access.log