When monitoring kernel logs through dmesg
or /var/log/syslog
, you've probably encountered lines like:
[ 1234.567890] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[ 1234.567899] last message repeated 3 times
While this compression saves disk space, it's problematic for:
- Real-time monitoring scripts
- Precise event timestamping
- Log analysis pipelines
This behavior originates from the kernel's printk subsystem, specifically controlled by:
/proc/sys/kernel/printk_devkmsg
The values represent:
ratelimit
: Default compression behaviorratelimit_burst
: Number of repeats before compression kicks in
Add this to /etc/sysctl.conf
:
kernel.printk_devkmsg = on
Then apply with:
sudo sysctl -p
Alternatively for immediate effect:
echo "on" | sudo tee /proc/sys/kernel/printk_devkmsg
For specific monitoring scripts, prepend:
echo 1 | sudo tee /proc/sys/kernel/printk_devkmsg
# Your monitoring command here
echo 0 | sudo tee /proc/sys/kernel/printk_devkmsg
After changes, test with:
for i in {1..5}; do logger "Test message $i"; done
tail -n 10 /var/log/syslog
Note that some logging daemons (like rsyslog) may still perform their own deduplication. Check their configuration if needed.
For cases where you can't modify system settings:
#!/bin/bash
dmesg --follow | while read -r line; do
if [[ $line =~ "last message repeated" ]]; then
count=$(echo $line | grep -oP '(?<=repeated )\d+')
for ((i=0; i<count; i++)); do
echo "${last_line}"
done
else
last_line="$line"
echo "$line"
fi
done
When monitoring kernel logs on Debian/Ubuntu servers, you'll often encounter lines like:
kernel: [12345.67890] CPU1: Package temperature above threshold
kernel: [12345.67891] last message repeated 3 times
The rsyslog
daemon (or syslog-ng
in some cases) automatically compresses identical consecutive messages to save disk space. While useful for production systems, it's problematic for:
- Real-time monitoring scripts
- Debugging time-sensitive issues
- Log analysis tools expecting 1:1 message sequences
For systems using rsyslog
(default on Debian/Ubuntu):
sudo nano /etc/rsyslog.conf
Add or modify these directives:
# Disable repeated message reduction
$RepeatedMsgReduction off
# For older rsyslog versions (<8)
$RepeatedMessagesReduction off
Then restart the service:
sudo systemctl restart rsyslog
Add this to your GRUB configuration (/etc/default/grub
):
GRUB_CMDLINE_LINUX_DEFAULT="... log_buf_len=1M printk.always_kmsg_dump=y"
Update GRUB and reboot:
sudo update-grub
sudo reboot
For already compressed logs, use this awk script to expand repeats:
awk '{
if (/last message repeated ([0-9]+) times/) {
count = $4;
while (count-- > 0) print prev_line;
} else {
print;
prev_line = $0;
}
}' /var/log/kern.log
Test your configuration by generating repeat messages:
for i in {1..5}; do
logger -k "Test repeated message"
done
tail -n 10 /var/log/kern.log
You should now see five individual entries instead of one with a repeat count.
Disabling message compression will:
- Increase log file sizes by 10-30% in most cases
- Require more frequent log rotation
- Impact disk I/O during high-volume logging
For high-traffic servers, consider rate-limiting instead:
$SystemLogRateLimitInterval 0
$SystemLogRateLimitBurst 0