As developers and system administrators, we often encounter situations where files mysteriously appear in directories without obvious sources. This becomes particularly problematic when:

• Debugging unexpected storage consumption
• Investigating potential security breaches
• Troubleshooting application conflicts

Linux provides several effective methods to monitor file operations:

1. Using inotifywait

The inotifywait tool from the inotify-tools package is excellent for real-time monitoring:

sudo apt-get install inotify-tools
inotifywait -m /some/dir -e create -e modify --format '%w%f %T' --timefmt '%H:%M:%S'

2. Leveraging lsof

The lsof command shows all open files and the processes using them:

sudo lsof +D /some/dir

For continuous monitoring, combine with watch:

watch -n 1 "sudo lsof +D /some/dir"

3. Auditd Framework

For persistent monitoring, configure auditd rules:

sudo auditctl -w /some/dir -p wa -k dir_monitoring
sudo ausearch -k dir_monitoring | aureport -f -i

Stracing Specific Processes

When you've identified potential suspect processes, strace can reveal file operations:

sudo strace -f -e trace=file -p PID 2>&1 | grep '/some/dir'

Using fatrace

For system-wide file activity monitoring:

sudo fatrace | grep '/some/dir'

Imagine /var/log fills up with mysterious 'app_errors.log' files. Here's how to investigate:

# Start monitoring
sudo inotifywait -m /var/log -e create |
while read path action file; do
    if [[ "$file" =~ app_errors ]]; then
        echo "New file created: $file"
        sudo lsof +D /var/log | grep "$file"
    fi
done

Create a monitoring script to log offenders:

#!/bin/bash
TARGET_DIR="/some/dir"
LOG_FILE="/var/log/dir_monitor.log"

inotifywait -m "$TARGET_DIR" -e create --format '%w%f' |
while read FILE; do
    TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S')
    PROCESS=$(sudo lsof "$FILE" | awk 'NR==2 {print $1}')
    PID=$(sudo lsof "$FILE" | awk 'NR==2 {print $2}')
    echo "[$TIMESTAMP] $FILE created by $PROCESS (PID:$PID)" >> "$LOG_FILE"
done

• Always run monitoring tools with appropriate privileges (sudo)
• Be mindful of performance impact when monitoring high-activity directories
• Consider system auditing requirements for production environments
• For containers, you may need to monitor from the host system

---

When you notice files mysteriously appearing in a directory like /some/dir, it's often crucial to identify the responsible process for system administration, debugging, or security purposes. The challenge lies in catching the process in the act of file creation.

The lsof command (list open files) is your first line of defense:

watch -n 1 lsof +D /some/dir

This continuously monitors the directory every second. When you spot a new file, note the PID in the output.

For more precise tracking, install and use inotify-tools:

sudo apt-get install inotify-tools  # Debian/Ubuntu
sudo yum install inotify-tools      # RHEL/CentOS

inotifywait -m /some/dir -e create |
while read path action file; do
    echo "File '$file' appeared via '$action' in '$path'"
    lsof $path$file | awk 'NR!=1 {print $2}' | xargs ps -fp
done

This script watches for file creation events and immediately identifies the responsible process.

For enterprise environments, configure the Linux audit subsystem:

sudo auditctl -w /some/dir -p w -k dir_write_monitor

# To view the logs:
ausearch -k dir_write_monitor | aureport -f -i

This provides detailed information including UID, PID, and command name.

For a broader filesystem audit:

sudo fatrace | grep /some/dir

This shows all file access events system-wide, filtered for your directory.

Let's say we find unexpected log files in /var/log/custom:

# Start monitoring
inotifywait -m /var/log/custom -e create |
while read; do
    lsof -t /var/log/custom | xargs ps -fp
done

# In another terminal, create a test file
touch /var/log/custom/test.log

# Output will show the responsible process
UID        PID  PPID  C STIME TTY      TIME CMD
root     12345 98765  0 14:30 ?    00:00:01 /usr/sbin/custom-daemon