When working with Nginx logs, many administrators notice the discrepancy between error log and access log timestamp formats:
# Error log default format
2023/05/15 14:30:45 [error] 12345#0: *6789 upstream timed out
# Access log formats
$time_local: 15/May/2023:14:30:45 +0000
$time_iso8601: 2023-05-15T14:30:45+00:00
Nginx provides limited built-in variables for timestamp formatting:
$time_local
- Common Log Format (CLF) style$time_iso8601
- ISO 8601 format
The error log format (Y/m/d H:i:s) isn't available as a built-in variable for access logs.
For Nginx compiled with Lua support, you can create custom timestamp formats:
http {
lua_shared_dict date_cache 1m;
log_format custom_error_fmt '$remote_addr - $remote_user [$time_custom] '
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent"';
server {
set_by_lua $time_custom '
local date_cache = ngx.shared.date_cache
local time_key = ngx.time()
local cached_date = date_cache:get(time_key)
if cached_date then
return cached_date
end
local date_str = os.date("%Y/%m/%d %H:%M:%S")
date_cache:set(time_key, date_str, 1)
return date_str
';
access_log /var/log/nginx/access.log custom_error_fmt;
error_log /var/log/nginx/error.log;
}
}
If modifying Nginx configuration isn't possible, consider log processing tools:
# Using awk to reformat access logs
awk '{
match($0, /$$([^]]+)$$/, m)
split(m[1], t, /[/: ]/)
month_map["Jan"]=01; month_map["Feb"]=02; # ... continue mapping
new_date = t[3] "/" month_map[t[2]] "/" t[1] " " t[4] ":" t[5] ":" t[6]
gsub(m[1], new_date)
print
}' access.log > reformatted_access.log
Consistent timestamp formats enable:
- Easier log correlation between access and error logs
- Simpler parsing for log aggregation systems
- Better compatibility with monitoring tools
- Reduced cognitive load when debugging
For production systems, consider these best practices:
# Recommended minimal configuration
log_format main '$remote_addr - $remote_user [$time_iso8601] '
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent"';
access_log /var/log/nginx/access.log main buffer=32k flush=5m;
error_log /var/log/nginx/error.log warn;
When working with Nginx logs, many administrators notice that access logs and error logs use different timestamp formats by default:
# Access log formats (configurable):
[29/Sep/2016:10:20:48 +0100] # $time_local
[2016-09-29T10:20:48+01:00] # $time_iso8601
# Error log format (fixed):
2016/09/29 10:37:52 [error] # Y/m/d H:i:s
Having consistent timestamp formats across logs is crucial for:
- Correlating events between access and error logs
- Automated log processing and analysis
- Human readability during debugging
While Nginx doesn't natively support changing the error log format, we can achieve this using the ngx_lua module:
# Install lua-nginx-module (CentOS 7)
yum install lua-nginx-module
# nginx.conf configuration
http {
lua_package_path "/path/to/lua/scripts/?.lua;;";
init_by_lua_block {
local file = require "pl.file"
local path = require "pl.path"
-- Custom error logger function
function custom_error_log(msg, level)
local timestamp = ngx.localtime() -- Returns Y-m-d H:i:s format
local log_line = string.format("[%s] [%s] %s",
timestamp,
level or "error",
msg)
-- Write to error log file
local log_path = "/var/log/nginx/error_custom.log"
local fd = io.open(log_path, "a")
if fd then
fd:write(log_line, "\n")
fd:close()
end
end
}
}
For systems where modifying Nginx isn't possible, consider post-processing:
# Using sed to reformat error logs
cat /var/log/nginx/error.log | sed -E 's|([0-9]{4})/([0-9]{2})/([0-9]{2}) ([0-9]{2}:[0-9]{2}:[0-9]{2})|[\3/\2/\1:\4 +0000]|' > error_reformatted.log
# Logstash configuration example
filter {
grok {
match => { "message" => "%{YEAR:year}/%{MONTHNUM:month}/%{MONTHDAY:day} %{TIME:time} $$%{LOGLEVEL:loglevel}$$" }
}
mutate {
add_field => { "timestamp" => "%{day}/%{month}/%{year}:%{time} +0000" }
}
}
- Standardize on ISO8601 format ([YYYY-MM-DDThh:mm:ss±TZ]) for all logging
- Include timezone information in all timestamps
- Consider using centralized logging solutions (ELK, Graylog)
- Document your logging format standards
For production systems, the Lua module approach provides the most reliable solution while maintaining native Nginx performance characteristics. The post-processing approach works well for existing log archives or when you can't modify the Nginx configuration.