When examining system monitoring tools on Ubuntu, the sysstat package is a crucial component for collecting performance data. The error message typically appears in the cron logs:

/etc/cron.daily/sysstat:
Invalid system activity file: /var/log/sysstat//sa28

Let's first examine the directory structure where sysstat stores its data files:

ls -lai /var/log/sysstat
total 44
6601 drwxr-xr-x  2 root root 4096 Apr 29 06:48 .
6550 drwxr-xr-x 13 root root 4096 Apr 29 06:48 ..
  16 -rw-r--r--  1 root root  384 Apr 22 00:00 sa21
 510 -rw-r--r--  1 root root  384 Apr 23 00:00 sa22
 524 -rw-r--r--  1 root root  384 Apr 24 00:00 sa23
2613 -rw-r--r--  1 root root  384 Apr 25 00:00 sa24
2199 -rw-r--r--  1 root root  384 Apr 26 00:00 sa25
2745 -rw-r--r--  1 root root  384 Apr 27 00:00 sa26
2577 -rw-r--r--  1 root root  384 Apr 28 00:00 sa27
  22 -rw-r--r--  1 root root  384 Apr 29 00:00 sa28
1221 -rw-r--r--  1 root root  336 Apr 29 21:45 sa29

The error occurs when the sysstat cron job tries to process historical data files. Several potential causes exist:

• Corrupted binary data file (saXX files)
• Permission issues preventing proper file access
• Race condition during file rotation
• Incomplete data collection for that day

To diagnose the specific issue with the sa28 file mentioned in the error:

# Check file integrity
sudo sadf -d /var/log/sysstat/sa28

# Verify file permissions
stat -c "%a %U:%G %n" /var/log/sysstat/sa28

# Attempt manual processing
sudo /usr/lib/sysstat/debian-sa1 1 1

Here are three approaches to resolve this issue:

# Option 1: Reconfigure sysstat
sudo dpkg-reconfigure sysstat

# Option 2: Clean and restart collection
sudo rm /var/log/sysstat/sa*
sudo systemctl restart sysstat

# Option 3: Implement error handling in cron
if [ -s /var/log/sysstat/sa$(date +%d) ]; then
    /usr/lib/sysstat/sa1 --boot
fi

To avoid future occurrences:

1. Implement proper log rotation configuration in /etc/sysstat/sysstat
2. Add error handling to the cron job script
3. Monitor disk space where activity files are stored
4. Consider using alternative monitoring like Prometheus node exporter

Create this diagnostic script to automatically check sysstat health:

#!/bin/bash
DATE=$(date +%d)
SAFILE="/var/log/sysstat/sa$DATE"
ERROR_LOG="/var/log/sysstat_errors.log"

check_sa_file() {
    if [ ! -s "$1" ]; then
        echo "[$(date)] Error: Empty or missing file $1" >> $ERROR_LOG
        return 1
    fi
    
    if ! sadf -d "$1" &> /dev/null; then
        echo "[$(date)] Error: Corrupted file $1" >> $ERROR_LOG
        return 2
    fi
    
    return 0
}

check_sa_file "$SAFILE" || {
    sudo systemctl restart sysstat
    check_sa_file "$SAFILE" || {
        sudo rm "$SAFILE"
        sudo /usr/lib/sysstat/debian-sa1 1 1
    }
}

When examining the /var/log/sysstat directory on Ubuntu systems, you'll typically find two types of files:

sa[day_of_month] - Binary data files containing system activity records
sar[day_of_month] - Text reports generated from the binary data

The error occurs when the daily cron job (/etc/cron.daily/sysstat) tries to process these files but encounters corruption or permission issues.

Based on the directory listing provided, here are potential issues:

• Empty or zero-byte sar* files (common in this case)
• Permission mismatches (though your listing shows correct root ownership)
• File corruption during collection
• Improper rotation of log files

First, verify the file validity manually:

# Check if the binary file contains valid data
sadf -d /var/log/sysstat/sa28

# Try to generate a report
sar -f /var/log/sysstat/sa28

If these commands fail, the file is indeed corrupted. Here's how to reset the collection:

# Stop the sysstat service
sudo service sysstat stop

# Remove corrupted files (adjust date as needed)
sudo rm /var/log/sysstat/sa28 /var/log/sysstat/sar28

# Restart collection
sudo service sysstat start

Add this check to your daily maintenance scripts:

#!/bin/bash
# Verify sysstat files before processing
SAFILE="/var/log/sysstat/sa$(date +%d)"
if [ ! -s "$SAFILE" ]; then
    logger "sysstat: Invalid or empty activity file $SAFILE"
    exit 1
fi

To get more detailed output from the daily job:

# Edit the cron job with debug output
sudo nano /etc/cron.daily/sysstat

# Add these lines near the top
set -x
exec > /var/log/sysstat-cron.log 2>&1

If issues persist, consider a clean reinstall:

sudo apt-get purge sysstat
sudo rm -rf /var/log/sysstat/*
sudo apt-get install sysstat