How to Prevent Duplicate Logs in Rsyslog: Isolating local0 Facility to a Single Log File


5 views

When configuring custom logging with rsyslog, particularly using the local0 facility, a common issue arises where log entries appear in multiple destinations. In this case, PHP messages tagged with local0 are being written to:

  • The intended custom log file (/var/log/sea.log)
  • The system's primary log file (/var/log/syslog)
  • The general messages file (/var/log/messages)

This duplication occurs because of two key factors in typical rsyslog configurations:

  1. The main rsyslog.conf contains catch-all rules (like *.*;auth,authpriv.none) that process all facilities unless explicitly excluded
  2. Default Debian/Ubuntu configurations log most facilities to syslog by default

Here's how to properly configure rsyslog to handle local0 exclusively through your custom configuration:

# /etc/rsyslog.d/sea.conf - Corrected version
# First rule processes local0 to our custom log
local0.*                     /var/log/sea.log

# Then explicitly stop further processing of local0
# This prevents other rules from handling these messages
& stop

The & stop directive is crucial - it tells rsyslog to stop processing any messages that match the preceding rules. This prevents the messages from being caught by the catch-all rules in the main configuration.

For a production environment, consider these enhancements:

# Add file permissions and rotation settings
$template SeaLogFormat,"%timegenerated% %syslogtag% %msg%\n"
$template SeaLog,"/var/log/sea.log"
:syslogtag, contains, "php" ?SeaLog;SeaLogFormat
& stop

# Enable log rotation
$outchannel sea_rotation,/var/log/sea.log,104857600,/usr/local/bin/rotate_sea.sh
local0.* :omfile:$sea_rotation

After making changes:

sudo systemctl restart rsyslog
logger -p local0.info "Test message"
# Check outputs
tail /var/log/sea.log
tail /var/log/syslog
tail /var/log/messages

For more complex scenarios, you can use property-based filtering:

# Only log messages containing "[APP]" in the message text
if $msg contains "[APP]" then {
    action(type="omfile" file="/var/log/sea.log")
    stop
}

When dealing with high-volume logging:

  • Use asynchronous writing (- before filename) for better performance
  • Consider using $ActionQueueType and $ActionQueueFileName for queuing
  • For very high throughput, investigate RELP protocol or direct database logging

When configuring custom logging with rsyslog, a common issue arises where log messages appear in multiple destination files despite specific configuration. This happens because rsyslog processes rules sequentially and applies multiple matching rules unless explicitly told not to.

In the given setup, we have two relevant configuration files:

# Main rsyslog.conf contains catch-all rules:
*.*;auth,authpriv.none      -/var/log/syslog
*.=info;*.=notice;*.=warn;\\
    auth,authpriv.none;\\
    cron,daemon.none;\\
    mail,news.none      -/var/log/messages

# Custom sea.conf contains:
local0.info                     /var/log/sea.log
local0.none                     /var/log/messages
local0.none                     /var/log/syslog

The issue stems from rule processing order and pattern matching:

  1. Rsyslog processes the main configuration first
  2. The catch-all patterns (*.*) match before your specific rules
  3. The default templates in rsyslog.conf don't get overridden by your custom rules

To properly isolate local0 logs to only /var/log/sea.log, we need to:

# 1. First stop processing for local0 in main files
# Add to rsyslog.conf before any catch-all rules:
local0.none                     /var/log/syslog
local0.none                     /var/log/messages

# 2. Then in sea.conf keep just:
local0.*                        /var/log/sea.log

# 3. Add stop statement to prevent further processing
& stop

For more complex scenarios, consider using templates and filters:

# Define a custom template for sea logs
$template seaFormat,"%timestamp% %hostname% %programname% %msg%\n"

# Filter messages with property-based conditions
if $syslogfacility-text == 'local0' then {
    action(type="omfile" file="/var/log/sea.log" template="seaFormat")
    stop
}

After making changes:

# Test configuration syntax
rsyslogd -N1

# Restart rsyslog
systemctl restart rsyslog

# Generate test message
logger -p local0.info "Test message for sea.log"

When dealing with high-volume logging:

  • Use asynchronous writing (- prefix in file path)
  • Consider rate limiting for busy systems
  • Implement log rotation outside rsyslog