When using rsyslog's imfile module to monitor log files that get rotated, we often encounter situations where rsyslog stops forwarding logs after rotation. This typically happens because the internal state tracking gets out of sync with the actual file state.

While manually stopping rsyslog, deleting state files, and restarting does work, it's problematic because:

• It causes service interruption
• May lead to log loss during the restart period
• Creates operational complexity in production environments

Here's an optimized rsyslog configuration that handles log rotation more gracefully:

module(load="imfile" mode="inotify")
global(workDirectory="/var/spool/rsyslog")

input(
    type="imfile"
    File="/home/user/my_app/shared/log/unicorn.stderr.log"
    Tag="unicorn-stderr"
    Severity="info"
    Facility="local8"
    PersistStateInterval="1"
    reopenOnTruncate="on"
    readTimeout="10"
    addMetadata="on"
)

template(name="WithoutTimeFormat" type="string" string="[environment] [%syslogtag%] -- %msg%")

if $syslogtag contains 'apache-' then {
    action(
        type="omfwd"
        target="my_server"
        port="5000"
        template="WithoutTimeFormat"
        Protocol="tcp"
        queue.size="100000"
        queue.type="LinkedList"
        action.resumeRetryCount="-1"
        action.resumeInterval="10"
    )
}

*.* action(
    type="omfwd"
    target="my_server"
    port="5000"
    template="SyslFormat"
    Protocol="tcp"
)

Key parameters that solve the rotation issue:

• reopenOnTruncate="on": Handles copytruncate-style rotations
• readTimeout="10": Prevents hanging on file operations
• mode="inotify": Uses more efficient file monitoring
• PersistStateInterval="1": Frequently saves position state

Complement your rsyslog setup with this improved logrotate config:

/home/user/my_app/shared/log/*.log {
    daily
    missingok
    dateext
    rotate 30
    compress
    delaycompress
    notifempty
    extension gz
    copytruncate
    create 640 user user
    sharedscripts
    postrotate
        /usr/bin/systemctl kill -s HUP rsyslog.service >/dev/null 2>&1 || true
    endscript
}

To test your configuration:

# Force log rotation
logrotate -vf /etc/logrotate.d/your_config

# Check rsyslog state files
ls -l /var/spool/rsyslog/

# Monitor rsyslog processing
logger -t test "Rotation test message"
tail -f /var/log/syslog | grep rsyslog

For environments where copytruncate must be avoided:

1. Use create instead of copytruncate in logrotate
2. Configure your application to reopen log files on SIGHUP
3. Consider using Unix domain sockets instead of direct file logging

When dealing with high-volume logs:

• Adjust queue.size based on your traffic
• Monitor /var/spool/rsyslog disk usage
• Consider using RELP protocol for more reliable forwarding

When dealing with log rotation on Ubuntu 12.04 systems, many admins encounter a frustrating issue where rsyslog stops forwarding logs after rotation. The core symptoms typically include:

• TCP forwarding halts precisely after logrotate completes its job
• No errors in rsyslog's own logs about the failure
• Other non-rotated files continue forwarding normally

The root issue stems from how rsyslog's imfile module tracks file state. When using copytruncate in logrotate:

copytruncate

The module maintains an internal file position counter that becomes invalid after rotation. The state file (/var/spool/rsyslog/stat-*) doesn't properly reset, causing rsyslog to lose tracking.

Instead of stopping rsyslog and deleting state files, modify both configurations:

Updated logrotate.conf

/home/user/my_app/shared/log/*.log {
  daily
  missingok
  dateext
  rotate 30
  compress
  notifempty
  extension gz
  create 640 user user
  sharedscripts
  postrotate
    /usr/bin/pkill -HUP -u syslog rsyslog
  endscript
}

Enhanced rsyslog.conf

$ModLoad imfile

$InputFileName /home/user/my_app/shared/log/unicorn.stderr.log
$InputFileTag unicorn-stderr
$InputFileStateFile stat-unicorn-stderr
$InputFileSeverity info
$InputFileFacility local8
$InputFilePollInterval 1
$InputFilePersistStateInterval 1
$InputFileReadMode 2
$InputRunFileMonitor

Key adjustments that make this work:

1. Remove copytruncate - let logrotate move files naturally
2. Add $InputFileReadMode 2 for better rotation handling
3. Use HUP signal instead of full restart
4. Ensure proper permissions on newly created files

After implementation:

# Force log rotation for testing
sudo logrotate -vf /etc/logrotate.d/myapp

# Check forwarding status
netstat -tnap | grep rsyslog

# Monitor syslog for errors
tail -f /var/log/syslog

The solution maintains continuous logging without service interruptions while properly handling rotated files. For Ubuntu 14.04+ systems, consider upgrading to omfile module which handles rotation more elegantly.