How to Monitor and Log Execution of ‘at’ Jobs in Linux via syslog


3 views

When scheduling one-time tasks with the at command on CentOS/RHEL systems, administrators often discover there's no built-in logging mechanism to verify job execution. Unlike cron which logs to /var/log/cron, the atd daemon traditionally keeps its activities silent.

To capture atd job execution details, we need to modify its configuration to leverage syslog:


# Edit /etc/sysconfig/atd
OPTS="-l 7"  # Sets logging level to debug (value 7)

Then configure rsyslog to handle these messages by adding to /etc/rsyslog.conf:


# Custom atd logging
daemon.* /var/log/atd.log

After making these changes, restart both services:


systemctl restart atd
systemctl restart rsyslog

Now when executing test jobs:


echo "touch /tmp/testfile" | at now + 1 minute

You'll see detailed execution logs in /var/log/atd.log containing:


atd[PID]: Executing job 1 (a0000101abcdef)...
atd[PID]: Job 1 (a0000101abcdef) completed

For even richer logging, create a wrapper script (/usr/local/bin/at_wrapper.sh):


#!/bin/bash
logger -t atd_wrapper "Starting job $1"
/usr/bin/at -f "$1" "$2"
logger -t atd_wrapper "Scheduled job $1 for $2"

If logs don't appear:

  1. Verify SELinux isn't blocking: audit2allow -a
  2. Check syslog facility matches: grep -i atd /etc/rsyslog.d/*
  3. Test logger manually: logger -p daemon.info "Test atd message"

When scheduling tasks with the Linux at command, many sysadmins encounter a frustrating blind spot: there's no default logging mechanism to verify whether jobs executed successfully. Unlike cron jobs which typically log to /var/log/cron, atd (the at daemon) doesn't provide execution logs by default.

The solution lies in configuring syslog to capture atd messages. Here's how to implement it on CentOS/RHEL systems:

# Edit syslog configuration
sudo vi /etc/syslog.conf

# Add this line to log atd messages
daemon.*;authpriv.*;local5.*;local6.* /var/log/daemon.log

# Create logrotate configuration
sudo vi /etc/logrotate.d/atd

Sample logrotate configuration:

/var/log/daemon.log {
    weekly
    rotate 4
    compress
    missingok
    notifempty
    sharedscripts
    postrotate
        /bin/kill -HUP cat /var/run/syslogd.pid 2> /dev/null 2> /dev/null || true
    endscript
}

After restarting syslog (service syslog restart), you can verify the configuration by checking logs after job execution:

# Sample log entry
grep atd /var/log/daemon.log
# Expected output:
# Aug 15 09:30:01 server atd[1234]: Job 42 completed for user bob

For more detailed logging, consider wrapping your at jobs in scripts that implement logging:

#!/bin/bash
# at_wrapper.sh
LOG_FILE=/var/log/at_jobs.log

{
    echo "=== $(date) ==="
    echo "Job ID: $1"
    echo "User: $(whoami)"
    echo "Command: $2"
    
    # Execute the command
    eval "$2"
    
    echo "Exit status: $?"
    echo ""
} >> $LOG_FILE 2>&1

Usage example:

at now + 5 minutes -f at_wrapper.sh 42 "your_command"
  • If jobs aren't executing, verify atd is running: service atd status
  • Check SELinux contexts if logging fails: ls -Z /var/log/daemon.log
  • For mail-based notifications, ensure your MTA is properly configured

On modern systems using systemd, you can query journal logs:

journalctl -u atd --since "1 hour ago"