How to Log Full HTTP Request Headers and Body in Apache Server Configuration


2 views

When debugging web applications or analyzing security incidents, developers often need to inspect the complete HTTP request including headers and body. Apache provides several configuration options to achieve this.

The most comprehensive solution is Apache's mod_dumpio module which logs all I/O including request bodies:


# Enable the module
LoadModule dumpio_module modules/mod_dumpio.so

# Configure logging in httpd.conf or virtual host
<IfModule dumpio_module>
    DumpIOInput On
    DumpIOOutput On
    DumpIOLogLevel debug
</IfModule>

For just headers, modify your LogFormat directive:


LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %{ALL_REQUEST_HEADERS}e" extended
CustomLog logs/access_log extended

Enable debug logging for specific modules:


LogLevel debug
# Or for specific modules
LogLevel ssl:debug rewrite:trace5

Here's a complete virtual host configuration for API debugging:


<VirtualHost *:443>
    ServerName api.example.com
    DocumentRoot /var/www/api
    
    ErrorLog /var/log/apache2/api_error.log
    CustomLog /var/log/apache2/api_access.log combined
    
    # Enable full request logging
    <IfModule dumpio_module>
        DumpIOInput On
        DumpIOLogLevel debug
    </IfModule>
    
    # Custom header logging
    LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Authorization}i\" \"%{Content-Type}i\"" api_format
    CustomLog /var/log/apache2/api_headers.log api_format
</VirtualHost>

Remember that logging complete requests may capture sensitive data like passwords or API keys. Always:

  • Restrict log file permissions
  • Consider masking sensitive headers
  • Implement log rotation and expiration

For complex scenarios, consider:

  • Using tcpdump: tcpdump -i eth0 -s 0 -w /tmp/http.pcap port 80
  • Apache Traffic Server with logging plugins
  • Specialized HTTP debugging proxies

When debugging web applications or analyzing traffic patterns, developers often need to inspect complete HTTP requests including all headers. Apache's default logging configuration only records basic information like IP address and request URI. To capture the full request including headers like User-Agent, Accept-Language, and custom headers, we need to modify the Apache configuration.

The most effective way is to use Apache's mod_dumpio module which can log both request and response data. First, ensure the module is enabled:

sudo a2enmod dumpio

Then add these directives to your virtual host configuration or apache2.conf:

DumpIOInput On
DumpIOOutput On
DumpIOLogLevel debug
LogLevel dumpio:trace7

If you prefer not to use dumpio, you can customize the log format:

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%{Host}i %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %D" custom
CustomLog ${APACHE_LOG_DIR}/access.log custom

Here's a Python script to parse the extended logs:

import re
from collections import defaultdict

def parse_apache_logs(logfile):
    pattern = r'(\S+) (\S+) (\S+) $$(.*?)$$ "(.*?)" (\d+) (\d+) "(.*?)" "(.*?)"'
    log_stats = defaultdict(int)
    
    with open(logfile) as f:
        for line in f:
            match = re.match(pattern, line)
            if match:
                ip, _, _, date, request, status, size, referrer, agent = match.groups()
                log_stats[(ip, agent)] += 1
                
    return log_stats

When logging full requests:

  • Be cautious with sensitive headers (Authorization, Cookie)
  • Consider log rotation to prevent disk space issues
  • Set appropriate file permissions on log files

To log only specific requests, use SetEnvIf:

SetEnvIf Request_URI "^/debug" debug-request
CustomLog logs/debug.log combined env=debug-request