Recursive Log Rotation: Configuring logrotate for Directories and Subdirectories


4 views

When managing server logs, we often encounter directory structures where logs are distributed across multiple subdirectories. The default logrotate behavior requires explicit paths for each log file or directory, which becomes cumbersome with deep directory trees.

Modern logrotate versions (3.8.0+) support recursive wildcard patterns:

/var/log/**/*.log {
    daily
    missingok
    rotate 7
    compress
    delaycompress
    notifempty
    create 0640 root adm
}

For systems with older logrotate versions, we can use shell globbing:

#!/bin/sh
find /var/log -name '*.log' -exec logrotate /etc/logrotate.conf {} +

When implementing recursive log rotation:

  • Set proper permissions with create directive
  • Consider using sharedscripts for post-rotation commands
  • Be mindful of inotify watches with deeply nested directories

Here's a production configuration for Nginx/Apache logs:

/var/www/*/logs/*.log /var/www/*/*/logs/*.log {
    daily
    rotate 30
    compress
    dateext
    sharedscripts
    postrotate
        /usr/bin/systemctl reload nginx.service > /dev/null
    endscript
}

If recursive patterns don't work:

  1. Verify logrotate --version supports globbing
  2. Check for syntax errors with logrotate -d /path/to/config
  3. Ensure no duplicate entries conflict with your wildcard pattern

When managing server logs, we often encounter directory structures like:

/var/log/
├── app/
│   ├── service1.log
│   ├── service2.log
│   └── subdir/
│       └── service3.log
└── system/
    └── kernel.log

Standard logrotate configurations require explicit paths for each log file or directory. A naive approach would be:

/var/log/app/*.log /var/log/app/subdir/*.log {
    daily
    rotate 7
    compress
    missingok
}

This becomes unwieldy as your directory tree grows deeper.

Modern bash versions support recursive globbing with **:

/var/log/app/**/*.log {
    daily
    rotate 7
    compress
    missingok
}

Requires enabling globstar option in bash:

shopt -s globstar

For maximum compatibility across systems:

find /var/log/app -name "*.log" -exec logrotate /etc/logrotate.d/custom_config {} +

Create a custom config file /etc/logrotate.d/custom_config:

{
    daily
    rotate 7
    compress
    missingok
    sharedscripts
    postrotate
        /usr/bin/systemctl reload app.service
    endscript
}

Always verify with:

logrotate -d /etc/logrotate.d/your_config

For force rotation during testing:

logrotate -vf /etc/logrotate.d/your_config

For directories with thousands of log files:

  • Consider separate configurations for different log levels
  • Use maxage to automatically purge old logs
  • Implement delaycompress for busy systems