Monitoring File Deletions in Linux Directory Trees Using auditd: A Forensic Approach


2 views

When managing web applications like forum software, unexplained file deletions in attachment directories can indicate serious issues - whether from buggy scripts, malicious activity, or filesystem corruption. Traditional logging often misses these events.

While inotifywait or fanotify offer monitoring, the Linux Audit Framework (auditd) provides:

  • Kernel-level tracking with timestamps
  • PID and user context for deletions
  • Persistent logging across reboots
  • Minimal performance overhead

First, install audit tools if needed:

sudo apt install auditd audispd-plugins  # Debian/Ubuntu
sudo yum install audit audit-libs        # RHEL/CentOS

To monitor a directory tree recursively for deletions:

sudo auditctl -w /var/www/forum/attachments/ -p wa -k forum_attachments

Key parameters:

-w: Watch path

-p wa: Watch write and attribute changes (covers deletions)

-k: Custom key for search filtering

For forensic-level tracking:

# Track deletions with UID/PID details
sudo auditctl -a always,exit -F arch=b64 -S unlink -S unlinkat -S rmdir \
-F dir=/var/www/forum/attachments -F perm=w -k forum_deletions

# Make rules persistent
sudo sh -c 'auditctl -l > /etc/audit/rules.d/forum.rules'

Use ausearch to filter events:

sudo ausearch -k forum_attachments -i --start yesterday | less

Sample output shows critical forensic data:

type=SYSCALL msg=audit(03/12/24 14:23:42.123:123) : arch=x86_64 
syscall=unlinkat success=yes exit=0 items=1 pid=1234 uid=33 gid=33 
comm=php exe=/usr/bin/php8.2 key=forum_attachments
name=/var/www/forum/attachments/user345/image.jpg

Create a monitoring script (/usr/local/bin/monitor_deletions.sh):

#!/bin/bash
LOG=/var/log/attachment_deletions.log
ausearch -k forum_attachments -m DELETE --start $(date +%x -d "-5 min") | \
awk '/name=/{gsub(/"/,""); print strftime("%F %T"), $0}' >> $LOG

[ -s $LOG ] && mail -s "File deletions detected" admin@example.com < $LOG

For high-traffic directories:

  • Limit depth with -F subj_dirs if possible
  • Exclude temporary files using -F "name!~*.tmp"
  • Rotate logs aggressively (/etc/audit/auditd.conf)

If seeing audit: backlog limit exceeded errors:

# Increase kernel buffer
sudo auditctl -b 8192

# Permanent setting
echo "auditctl -b 8192" >> /etc/audit/rules.d/99-sysctl.conf

When running forum software with attachment uploads, discovering missing files can be particularly frustrating. The challenge intensifies when files are stored in multi-level directory structures, making manual monitoring impractical.

The Linux audit framework (auditd) provides kernel-level monitoring capabilities that outperform regular filesystem watchers. Unlike tools like inotify, auditd:

  • Tracks system calls at the kernel level
  • Logs complete event details including timestamps and process information
  • Maintains records even if the deleting process terminates
  • Works across all filesystem types

First, ensure auditd is installed and running:

sudo apt install auditd  # Debian/Ubuntu
sudo systemctl enable --now auditd

For RHEL/CentOS:

sudo yum install audit
sudo service auditd start
sudo chkconfig auditd on

To watch an entire directory tree for deletions, use these auditctl commands:

# Monitor /var/www/forum/attachments and all subdirectories
sudo auditctl -w /var/www/forum/attachments/ -p wa -k forum_attachments

# Alternative for recursive watching (some distros)
sudo auditctl -a exit,always -F dir=/var/www/forum/attachments/ -F perm=wa -k forum_attachments

Key parameters:

  • -w: Watch path
  • -p: Permissions to watch (w=write, a=attribute change)
  • -k: Key for searching logs

View deletion events with ausearch:

sudo ausearch -k forum_attachments -i | grep -E 'type=SYSCALL|type=PATH'

Sample output analysis:

type=SYSCALL msg=audit(1625097600.123:456): arch=c000003e syscall=263 success=yes exit=0 a0=7ffeexyz a1=0 a2=0 a3=0 items=1 ppid=1234 pid=5678 auid=4294967295 uid=33 gid=33 euid=33 suid=33 fsuid=33 egid=33 sgid=33 fsgid=33 tty=(none) comm="php" exe="/usr/bin/php7.4" key="forum_attachments"

type=PATH msg=audit(1625097600.123:456): item=0 name="/var/www/forum/attachments/user123/image.jpg" inode=12345678 dev=08:01 mode=0100644 ouid=33 ogid=33 rdev=00:00 nametype=DELETE cap_fp=0000000000000000 cap_fi=0000000000000000 cap_fe=0 cap_fver=0

For scenarios where auditd isn't available:

inotifywait Solution

inotifywait -m -r -e delete /var/www/forum/attachments/ | \
while read path action file; do
    echo "$(date): $file was $action in $path" >> /var/log/forum_deletions.log
done

fsnotify (Golang)

package main

import (
    "log"
    "github/fsnotify/fsnotify"
)

func main() {
    watcher, _ := fsnotify.NewWatcher()
    defer watcher.Close()
    
    watcher.Add("/var/www/forum/attachments")
    
    for {
        select {
        case event := <-watcher.Events:
            if event.Op&fsnotify.Remove == fsnotify.Remove {
                log.Printf("File deleted: %s", event.Name)
            }
        case err := <-watcher.Errors:
            log.Println("error:", err)
        }
    }
}

To make rules persistent, add them to /etc/audit/rules.d/forum.rules:

-w /var/www/forum/attachments -p wa -k forum_attachments

Then reload the rules:

sudo augenrules --load

When monitoring large directory trees:

  • Set proper log rotation in /etc/audit/auditd.conf
  • Consider adding filters to exclude non-deletion events
  • Monitor /var/log/audit/audit.log size
  • For high-traffic systems, use separate partitions for audit logs