When dealing with services that maintain persistent log file handles (like CouchDB, MySQL, or custom daemons), we face a fundamental challenge: the process keeps writing to the original file descriptor even after logrotate renames the file. This creates two common solutions - copytruncate
and delaycompress
- that address different aspects of the problem.
Contrary to initial intuition, delaycompress
doesn't exist to handle the writing-after-rotation problem itself. Instead, it solves a secondary issue that occurs during compression:
# Without delaycompress:
1. service.log → service.log.1 (rotation)
2. service.log.1 → service.log.1.gz (immediate compression)
# With delaycompress:
1. service.log → service.log.1 (rotation)
2. Next cycle: service.log.1 → service.log.2
3. service.log.2 → service.log.2.gz (delayed compression)
In the CouchDB example you mentioned, using both directives makes perfect sense:
copytruncate # Handles the active file handle by copying then truncating
delaycompress # Ensures we don't compress a file that might still receive writes
Here's why this combination works:
copytruncate
creates a copy of the active log before truncating, allowing the process to continue writingdelaycompress
waits until the next rotation cycle to compress, giving any buffered writes time to complete
Consider PostgreSQL's log rotation configuration:
/var/log/postgresql/*.log {
daily
rotate 7
compress
delaycompress
missingok
notifempty
sharedscripts
postrotate
/usr/bin/pg_ctl --silent --pgdata=/var/lib/postgresql/12/main reload
endscript
}
Even with proper HUP signal handling, delaycompress
provides extra safety against race conditions during the reload.
For applications that properly handle log file reopening (like Nginx), simple compression works fine:
/var/log/nginx/*.log {
daily
rotate 14
compress
missingok
notifempty
create 0640 www-data adm
sharedscripts
postrotate
if [ -f /var/run/nginx.pid ]; then
kill -USR1 cat /var/run/nginx.pid
fi
endscript
}
- Always use
delaycompress
withcopytruncate
- For services with reload capabilities, prefer signals over
copytruncate
- Monitor compressed file sizes - sudden drops may indicate lost log entries
- Consider
maxage
withdelaycompress
to prevent stale uncompressed logs
When dealing with services that maintain persistent log file handles (like database servers or long-running daemons), we face a fundamental challenge: the process keeps writing to the original file descriptor even after log rotation occurs. This creates race conditions between the compression process and the active logging.
The delaycompress
option doesn't solve the file handle problem itself - it works in conjunction with rotation strategies to prevent data corruption. Here's the sequence:
1. Rotation occurs (logfile → logfile.1)
2. Process continues writing to logfile.1 (original FD)
3. Next rotation cycle: logfile.1 → logfile.2
4. Now logfile.2 gets compressed (safe because FD moved to logfile.1)
The CouchDB example shows defensive programming in action:
/usr/local/couchdb-1.0.1/var/log/couchdb/*.log {
weekly
rotate 10
copytruncate # Handle active file descriptors
delaycompress # Protect against slow compression
compress # Ultimately compress logs
notifempty
missingok
}
copytruncate
ensures the active log gets reset, while delaycompress
guards against:
- Compression processes locking files during rotation
- Slow compression interfering with active logging
- Resource contention on high-IO systems
For high-traffic web servers, consider this optimized configuration:
/var/log/nginx/*.log {
daily
rotate 14
delaycompress
compress
sharedscripts
postrotate
/usr/bin/killall -USR1 nginx 2>/dev/null || true
endscript
}
Note we don't use copytruncate here because Nginx supports proper log rotation via signals, while delaycompress prevents compression from affecting the active rotation cycle.
Situation | Recommended Approach |
---|---|
Process responds to SIGHUP/USR1 | Standard rotation with postrotate script |
Process ignores signals | copytruncate + delaycompress |
Very large logs (>1GB) | Always use delaycompress |
Low disk space environment | Standard rotation (no delaycompress) |