How to Properly Rotate and Compress Multiple Nginx Log Files on Ubuntu Server


2 views

When running multiple domains on a single Nginx server, each domain's access and error logs can quickly grow to unmanageable sizes. Proper log rotation is crucial for:

  • Preventing disk space exhaustion
  • Maintaining server performance
  • Complying with data retention policies
  • Facilitating log analysis

Ubuntu comes with logrotate pre-installed, making it the ideal tool for this task. Create a dedicated configuration file for Nginx:

sudo nano /etc/logrotate.d/nginx

Then add the following configuration (customize paths as needed):

/var/log/nginx/*.log {
    daily
    missingok
    rotate 14
    compress
    delaycompress
    notifempty
    create 0640 www-data adm
    sharedscripts
    postrotate
        if [ -f /var/run/nginx.pid ]; then
            kill -USR1 cat /var/run/nginx.pid
        fi
    endscript
}

For servers hosting multiple domains with separate log files, you can specify individual log paths:

/var/log/nginx/domain1_access.log
/var/log/nginx/domain1_error.log
/var/log/nginx/domain2_access.log
/var/log/nginx/domain2_error.log {
    daily
    rotate 30
    compress
    dateext
    sharedscripts
    postrotate
        [ ! -f /var/run/nginx.pid ] || kill -USR1 cat /var/run/nginx.pid
    endscript
}

Before implementing in production, test your configuration with:

sudo logrotate -d /etc/logrotate.d/nginx

For a dry run that shows what would happen without actually rotating files. To force rotation immediately:

sudo logrotate -f /etc/logrotate.d/nginx

To optimize compression (especially important for high-traffic sites), modify the compress options:

compresscmd /bin/gzip
compressoptions -9
compressext .gz

Check logrotate's own logs to verify operation:

cat /var/lib/logrotate/status
grep logrotate /var/log/syslog

Common issues to watch for:

  • Permission problems (ensure www-data can write new logs)
  • Disk space monitoring for compressed archives
  • Nginx PID file location changes

For more granular control, create separate configurations for different domains:

# /etc/logrotate.d/nginx-domain1
/var/log/nginx/domain1_*.log {
    weekly
    rotate 8
    size 100M
    ...
}

# /etc/logrotate.d/nginx-domain2 
/var/log/nginx/domain2_*.log {
    daily
    rotate 30
    ...
}

When running multiple websites on a single Nginx server, each domain typically generates its own access and error logs. Without proper rotation, these files can grow uncontrollably, consuming disk space and making log analysis difficult. Here's how to implement an efficient rotation strategy.

The most robust solution combines Ubuntu's built-in logrotate utility with proper Nginx process signaling. Here's a complete configuration example for multiple domains:

/var/log/nginx/*.log {
    daily
    missingok
    rotate 14
    compress
    delaycompress
    notifempty
    create 0640 www-data adm
    sharedscripts
    postrotate
        if [ -f /var/run/nginx.pid ]; then
            kill -USR1 cat /var/run/nginx.pid
        fi
    endscript
}

For sites requiring separate rotation policies (e.g., high-traffic domains needing weekly rotation):

/var/log/nginx/example.com.access.log {
    weekly
    rotate 8
    compress
    postrotate
        [ -f /var/run/nginx.pid ] && kill -USR1 cat /var/run/nginx.pid
    endscript
}

Test your logrotate setup without waiting for the cron job:

sudo logrotate -vf /etc/logrotate.d/nginx

Check for these success indicators:

  • Original log files moved to rotated names
  • New empty log files created with correct permissions
  • Nginx continues writing to the new files without interruption

While Ubuntu typically installs a daily cron job for logrotate, verify it exists:

sudo cat /etc/cron.daily/logrotate

For custom schedules, create a dedicated cron entry:

# /etc/cron.hourly/nginx-logrotate
0 * * * * /usr/sbin/logrotate /etc/logrotate.d/nginx >/dev/null 2>&1

Permission errors: Ensure the Nginx user (typically www-data) has write permissions to the log directory and new files.

Missing logs after rotation: Verify the postrotate signal is properly sent. The USR1 signal tells Nginx to reopen log files.

Disk space monitoring: Combine with a monitoring solution to alert when logs consume excessive space between rotations.