When managing web applications like forum software, unexplained file deletions in attachment directories can indicate serious issues - whether from buggy scripts, malicious activity, or filesystem corruption. Traditional logging often misses these events.
While inotifywait
or fanotify
offer monitoring, the Linux Audit Framework (auditd
) provides:
- Kernel-level tracking with timestamps
- PID and user context for deletions
- Persistent logging across reboots
- Minimal performance overhead
First, install audit tools if needed:
sudo apt install auditd audispd-plugins # Debian/Ubuntu
sudo yum install audit audit-libs # RHEL/CentOS
To monitor a directory tree recursively for deletions:
sudo auditctl -w /var/www/forum/attachments/ -p wa -k forum_attachments
Key parameters:
-w
: Watch path
-p wa
: Watch write and attribute changes (covers deletions)
-k
: Custom key for search filtering
For forensic-level tracking:
# Track deletions with UID/PID details
sudo auditctl -a always,exit -F arch=b64 -S unlink -S unlinkat -S rmdir \
-F dir=/var/www/forum/attachments -F perm=w -k forum_deletions
# Make rules persistent
sudo sh -c 'auditctl -l > /etc/audit/rules.d/forum.rules'
Use ausearch
to filter events:
sudo ausearch -k forum_attachments -i --start yesterday | less
Sample output shows critical forensic data:
type=SYSCALL msg=audit(03/12/24 14:23:42.123:123) : arch=x86_64
syscall=unlinkat success=yes exit=0 items=1 pid=1234 uid=33 gid=33
comm=php exe=/usr/bin/php8.2 key=forum_attachments
name=/var/www/forum/attachments/user345/image.jpg
Create a monitoring script (/usr/local/bin/monitor_deletions.sh
):
#!/bin/bash
LOG=/var/log/attachment_deletions.log
ausearch -k forum_attachments -m DELETE --start $(date +%x -d "-5 min") | \
awk '/name=/{gsub(/"/,""); print strftime("%F %T"), $0}' >> $LOG
[ -s $LOG ] && mail -s "File deletions detected" admin@example.com < $LOG
For high-traffic directories:
- Limit depth with
-F subj_dirs
if possible - Exclude temporary files using
-F "name!~*.tmp"
- Rotate logs aggressively (
/etc/audit/auditd.conf
)
If seeing audit: backlog limit exceeded
errors:
# Increase kernel buffer
sudo auditctl -b 8192
# Permanent setting
echo "auditctl -b 8192" >> /etc/audit/rules.d/99-sysctl.conf
When running forum software with attachment uploads, discovering missing files can be particularly frustrating. The challenge intensifies when files are stored in multi-level directory structures, making manual monitoring impractical.
The Linux audit framework (auditd) provides kernel-level monitoring capabilities that outperform regular filesystem watchers. Unlike tools like inotify, auditd:
- Tracks system calls at the kernel level
- Logs complete event details including timestamps and process information
- Maintains records even if the deleting process terminates
- Works across all filesystem types
First, ensure auditd is installed and running:
sudo apt install auditd # Debian/Ubuntu
sudo systemctl enable --now auditd
For RHEL/CentOS:
sudo yum install audit
sudo service auditd start
sudo chkconfig auditd on
To watch an entire directory tree for deletions, use these auditctl commands:
# Monitor /var/www/forum/attachments and all subdirectories
sudo auditctl -w /var/www/forum/attachments/ -p wa -k forum_attachments
# Alternative for recursive watching (some distros)
sudo auditctl -a exit,always -F dir=/var/www/forum/attachments/ -F perm=wa -k forum_attachments
Key parameters:
- -w: Watch path
- -p: Permissions to watch (w=write, a=attribute change)
- -k: Key for searching logs
View deletion events with ausearch:
sudo ausearch -k forum_attachments -i | grep -E 'type=SYSCALL|type=PATH'
Sample output analysis:
type=SYSCALL msg=audit(1625097600.123:456): arch=c000003e syscall=263 success=yes exit=0 a0=7ffeexyz a1=0 a2=0 a3=0 items=1 ppid=1234 pid=5678 auid=4294967295 uid=33 gid=33 euid=33 suid=33 fsuid=33 egid=33 sgid=33 fsgid=33 tty=(none) comm="php" exe="/usr/bin/php7.4" key="forum_attachments"
type=PATH msg=audit(1625097600.123:456): item=0 name="/var/www/forum/attachments/user123/image.jpg" inode=12345678 dev=08:01 mode=0100644 ouid=33 ogid=33 rdev=00:00 nametype=DELETE cap_fp=0000000000000000 cap_fi=0000000000000000 cap_fe=0 cap_fver=0
For scenarios where auditd isn't available:
inotifywait Solution
inotifywait -m -r -e delete /var/www/forum/attachments/ | \
while read path action file; do
echo "$(date): $file was $action in $path" >> /var/log/forum_deletions.log
done
fsnotify (Golang)
package main
import (
"log"
"github/fsnotify/fsnotify"
)
func main() {
watcher, _ := fsnotify.NewWatcher()
defer watcher.Close()
watcher.Add("/var/www/forum/attachments")
for {
select {
case event := <-watcher.Events:
if event.Op&fsnotify.Remove == fsnotify.Remove {
log.Printf("File deleted: %s", event.Name)
}
case err := <-watcher.Errors:
log.Println("error:", err)
}
}
}
To make rules persistent, add them to /etc/audit/rules.d/forum.rules:
-w /var/www/forum/attachments -p wa -k forum_attachments
Then reload the rules:
sudo augenrules --load
When monitoring large directory trees:
- Set proper log rotation in /etc/audit/auditd.conf
- Consider adding filters to exclude non-deletion events
- Monitor /var/log/audit/audit.log size
- For high-traffic systems, use separate partitions for audit logs