When running services under systemd, some applications (particularly chatty ones or those with verbose debug logging) can generate massive journal logs. I recently encountered a custom service writing 1GB/day of non-critical debug information to journald
. Here's how to surgically disable logging for just that unit.
The cleanest method uses StandardOutput
and StandardError
directives in the unit file:
[Service]
# Existing configuration...
StandardOutput=null
StandardError=null
After modifying, reload and restart:
sudo systemctl daemon-reload
sudo systemctl restart yourservice.service
If you want to keep some logs but reduce volume:
[Service]
# Limit to 100 messages per 30 seconds
RateLimitIntervalSec=30s
RateLimitBurst=100
For system-wide control, edit /etc/systemd/journald.conf
:
[Journal]
# Discard all logs from this unit
Storage=volatile
SystemMaxUse=1G
# Unit-specific override (add to bottom):
[JournalFilters]
Match_UID=systemd
Match_SYSTEMD_UNIT=yourservice.service
Action=discard
Check log reduction with:
journalctl --unit=yourservice.service --since=-1h
For persistent units, create a drop-in file at /etc/systemd/system/yourservice.service.d/nolog.conf
instead of modifying the main unit file.
- This doesn't affect application-level logging to files
- Consider adding
SyslogIdentifier=
if using external logging - For Docker containers, set
--log-driver=none
instead
When dealing with verbose systemd units that generate excessive logs (like those producing 1GB/day mentioned), it's crucial to understand the logging architecture. Systemd's journald service collects logs from all units by default, which can lead to:
- Disk space exhaustion
- Reduced system performance
- Difficulty finding relevant logs
The most surgical method is to configure journald to ignore specific units:
# /etc/systemd/journald.conf.d/override.conf [Journal] MaxLevelStore=warning SystemMaxUse=1G # Add unit-specific filters LogFilterPatterns=_SYSTEMD_UNIT=spammy.service
Note: This requires systemd v246+ and may not work on older distributions.
For more precise control, modify the unit file itself:
# Example unit override # systemctl edit spammy.service [Service] StandardOutput=null StandardError=null LogLevelMax=warning
For services that must keep stdout/stderr:
# Create a filter unit # /etc/systemd/system/spammy-filter.service [Unit] Description=Log filter for spammy service [Service] ExecStart=/bin/sh -c "journalctl -u spammy.service -f | grep -v 'DEBUG:'" StandardOutput=journal
After changes, always verify and maintain:
# Reload configuration sudo systemctl daemon-reload sudo systemctl restart spammy.service # Verify logging is reduced journalctl -u spammy.service --since "1 hour ago" -n 20
For critical production systems:
# Create private journal namespace sudo mkdir -p /var/log/journal/spammy sudo systemd-tmpfiles --create --prefix /var/log/journal/spammy # Configure namespace # /etc/systemd/journald@spammy.conf [Journal] Storage=volatile MaxLevelStore=warning