How to Log Apache Request Time in Milliseconds for Consistent Server Metrics


2 views

When working with distributed systems, having consistent time units across all components (application servers, databases, and web servers) is crucial for performance analysis. Many teams standardize on milliseconds for request timing, but Apache's default logging uses microseconds (%D) or seconds (%T).

Apache provides two main timing directives:

# Microseconds (µs)
%D

# Seconds with decimal (s)
%T

Neither of these matches the millisecond precision that many monitoring systems expect.

The most straightforward approach is to use %D (microseconds) and convert it to milliseconds in your log format:

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %D/1000" millis_format
CustomLog logs/access_log millis_format

This divides the microsecond value by 1000 to get milliseconds. The division happens during logging, so no post-processing is needed.

For more complex scenarios, you can use expression syntax in newer Apache versions (2.4+):

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %{expr:%D / 1000}i" custom_millis

After implementing either solution, check your access log to verify the output:

192.168.1.1 - - [10/Oct/2023:15:32:45 +0000] "GET /test HTTP/1.1" 200 1234 "-" "curl/7.68.0" 42

The last value (42 in this example) represents the request duration in milliseconds.

When parsing these logs with tools like ELK, Grafana, or Splunk, you can now treat all timing data consistently:

# Sample Logstash grok pattern
grok {
  match => { "message" => "%{IP:client} %{USER:ident} %{USER:auth} $$%{HTTPDATE:timestamp}$$ \"%{WORD:verb} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}\" %{NUMBER:response} (?:%{NUMBER:bytes}|-) \"%{DATA:referrer}\" \"%{DATA:agent}\" %{NUMBER:duration_ms}" }
}

The computational overhead of the division operation is negligible, but if you're handling extremely high traffic, consider:

  • Testing with your specific workload
  • Using the simpler division (/1000) rather than expr syntax
  • Benchmarking different logging configurations

Apache's default LogFormat directives provide two main time-related variables:

%T  - Seconds with decimal portion (e.g., 0.123)
%D  - Microseconds (e.g., 123456)

While these are precise, many monitoring systems expect milliseconds for consistency across web servers, databases, and application layers.

Add this to your Apache configuration (httpd.conf or virtual host):

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %{ms}T" combined_ms
CustomLog "/var/log/apache2/access_ms.log" combined_ms

1. Using mod_log_config expressions

For older Apache versions (2.2+):

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %D/1000" combined_ms

2. Environment Variable Approach

In your virtual host:

SetEnvIf Request_Start ".*" REQUEST_START=%{ms}T
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %{ms}T" custom_ms

After restarting Apache, check your log file. You should see entries like:

192.168.1.1 - - [10/Oct/2023:15:32:45 +0000] "GET /test HTTP/1.1" 200 1234 "-" "Mozilla/5.0" 87

The last number represents milliseconds (87ms in this example).

For ELK Stack (Logstash configuration example):

filter {
  grok {
    match => { "message" => "%{IPORHOST:clientip} %{USER:ident} %{USER:auth} $$%{HTTPDATE:timestamp}$$ \"%{WORD:verb} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}\" %{NUMBER:response} (?:%{NUMBER:bytes}|-) \"%{DATA:referrer}\" \"%{DATA:useragent}\" %{NUMBER:response_time_ms}" }
  }
}

Millisecond logging adds minimal overhead (~0.2% CPU impact in benchmarks). For high-traffic sites, consider:

  • Logging to a separate file
  • Using conditional logging
  • Buffered logging (BufferedLogs On)