Detecting Linux Server Intrusions: Key Indicators & Automated Security Audit Reporting with Shell Scripts


2 views

When your Linux box gets compromised, it often exhibits these technical symptoms:

# Check for suspicious processes
ps aux | grep -E '(wget|curl|nc|ncat|\./)' | grep -v grep

# Verify unauthorized SSH logins
last -ai | head -n 20

# Detect unexpected listening ports
ss -tulnp | grep -Ev '(ssh|systemd-resolve)'

Intruders often leave traces in these locations:

# Find recently modified config files
find /etc -type f -mtime -7 -ls

# Check for unauthorized SUID binaries
find / -perm -4000 -type f -ls 2>/dev/null

# Scan for hidden directories
ls -la / | grep '^d' | grep -vE '(\.|bin|boot|dev|etc|home)'

Create a daily audit script (/usr/local/bin/security_audit.sh):

#!/bin/bash

AUDIT_FILE="/var/log/security_audit_$(date +%Y%m%d).txt"
RECIPIENT="admin@yourdomain.com"

{
  echo "===== SECURITY AUDIT REPORT $(date) ====="
  echo -e "\n# Failed SSH attempts:"
  grep "Failed password" /var/log/auth.log | tail -n 20
  
  echo -e "\n# Active connections:"
  netstat -tuln
  
  echo -e "\n# Modified system binaries:"
  rpm -Va | grep '^.M'
  
  echo -e "\n# Suspicious cron jobs:"
  find /etc/cron* -type f -exec ls -la {} \;
} > $AUDIT_FILE

mail -s "Daily Security Audit $(hostname)" $RECIPIENT < $AUDIT_FILE

Consider implementing these security utilities:

# Install and configure AIDE (Advanced Intrusion Detection Environment)
sudo apt install aide
sudo aideinit
sudo mv /var/lib/aide/aide.db.new /var/lib/aide/aide.db
sudo aide.wrapper --check

# Set up OSSEC HIDS
sudo apt install ossec-hids-server
sudo /var/ossec/bin/ossec-control start

These log patterns indicate potential breaches:

# Check for brute force attempts
grep -i "failed" /var/log/auth.log | awk '{print $9}' | sort | uniq -c | sort -nr

# Detect privilege escalation
grep -i "sudo.*command=" /var/log/auth.log | grep -v "root"

# Find suspicious commands in history
cat /root/.bash_history | grep -E '(wget|curl|chmod|useradd|iptables)'

The first red flags often appear in system performance anomalies. Look for these technical indicators:

# Check for unexpected CPU/memory usage
top -b -n 1 | head -n 20

# Verify suspicious process trees
ps auxf | less

# Identify unauthorized cron jobs
crontab -l
ls -la /etc/cron.*

Intruders frequently leave traces in the filesystem. Conduct these forensic checks:

# Find recently modified system files
find / -type f -mtime -7 -exec ls -la {} \; | grep -v "proc\|sys"

# Check for binaries with SUID/SGID set
find / -type f \( -perm -4000 -o -perm -2000 \) -exec ls -la {} \;

# Verify critical file integrity (compare against known-good hashes)
sha256sum /bin/* /sbin/* /usr/bin/* /usr/sbin/* > current_hashes.txt
diff current_hashes.txt known_good_hashes.txt

Malicious network connections often expose compromises:

# Check established connections
ss -tulnp | grep ESTAB

# Monitor outbound traffic
iftop -n -P

# Verify listening ports against package manager records
netstat -tulnp
dpkg-query -S /usr/sbin/sshd

Implement these tools for scheduled security audits:

# Install and configure Lynis for system auditing
apt install lynis
lynis audit system --cronjob --quiet

# Set up AIDE for file integrity monitoring
apt install aide
aideinit
mv /var/lib/aide/aide.db.new /var/lib/aide/aide.db
echo "0 0 * * * /usr/bin/aide --check" | crontab -

Create a daily audit script with email notifications:

#!/bin/bash

# Generate audit report
REPORT_FILE="/tmp/security_audit_$(date +%Y%m%d).log"

{
  echo "==== SYSTEM AUDIT REPORT ===="
  date
  echo -e "\n### USER ACCOUNTS ###"
  awk -F: '($3 == 0) {print}' /etc/passwd
  echo -e "\n### CRON JOBS ###"
  ls -la /etc/cron*/*

  echo -e "\n### NETWORK STATUS ###"
  ss -tulnp

  echo -e "\n### FILE INTEGRITY ###"
  aide --check | tail -n 20
} > $REPORT_FILE

# Send email (requires mailutils configured)
mail -s "Daily Security Audit $(hostname)" admin@example.com < $REPORT_FILE

# Clean up
rm $REPORT_FILE

For enterprise environments, consider these additional measures:

# Install and configure Osquery for real-time monitoring
curl -L https://pkg.osquery.io/deb/osquery_5.8.1-1.linux_amd64.deb -o osquery.deb
dpkg -i osquery.deb

# Example osquery configuration for security monitoring
cat << EOF > /etc/osquery/osquery.conf
{
  "options": {
    "config_plugin": "filesystem",
    "logger_plugin": "filesystem",
    "logger_path": "/var/log/osquery",
    "schedule_splay_percent": 10
  },
  "schedule": {
    "process_checks": {
      "query": "SELECT * FROM processes WHERE path NOT LIKE '/usr/%' AND path NOT LIKE '/bin/%';",
      "interval": 300
    }
  }
}
EOF