How to Identify and Prevent Processes from Using Swap Memory in Linux Database Systems


2 views

html

Swap memory is a special area on your storage drive (HDD/SSD) that acts as overflow when your physical RAM becomes full. The Linux kernel moves less frequently used memory pages from RAM to this swap space, allowing active processes to continue functioning even when memory pressure is high.

While swap can prevent out-of-memory situations, excessive swapping (called "thrashing") significantly degrades performance since disk I/O is orders of magnitude slower than RAM access. This is particularly critical for database systems where low latency is essential.

Use these Linux commands to identify processes consuming swap:

# Method 1: Using smem (install with 'sudo apt install smem')
smem -s swap -r | head -n 10

# Method 2: Using /proc
for file in /proc/*/status ; do 
    awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}' $file 2>/dev/null 
done | sort -k 2 -n -r | head -n 10

# Method 3: Classic top command
top -o %SWAP

Database systems often trigger swap usage. Here's how to analyze a MySQL instance:

# First find MySQL's process ID
pgrep mysqld

# Then check its memory mapping
pmap -x [PID] | grep swap

# Alternative using smem
smem -P mysqld -s swap -r

Several approaches exist to control swap behavior:

# Temporary solution (until reboot):
sudo sysctl vm.swappiness=10

# Permanent solution (add to /etc/sysctl.conf):
echo "vm.swappiness=10" | sudo tee -a /etc/sysctl.conf
sudo sysctl -p

# Completely disable swap (not recommended for production):
sudo swapoff -a

For fine-grained control, use cgroups to restrict swap usage per process group:

# Create a cgroup
sudo cgcreate -g memory:limited_group

# Set memory limits (100MB RAM + 50MB swap)
echo "100000000" > /sys/fs/cgroup/memory/limited_group/memory.limit_in_bytes
echo "50000000" > /sys/fs/cgroup/memory/limited_group/memory.memsw.limit_in_bytes

# Assign MySQL to this cgroup
sudo cgclassify -g memory:limited_group $(pgrep mysqld)

For ongoing monitoring, consider these approaches:

# Simple bash monitoring script
while true; do
    date
    smem -P mysqld -c "name swap" -t -k | tail -n 1
    sleep 5
done

# Or use Prometheus node exporter with these metrics:
# node_memory_SwapTotal
# node_memory_SwapFree

Swap memory is a special area on your storage drive (HDD/SSD) that the Linux kernel uses when physical RAM becomes full. When your system runs out of RAM, inactive memory pages are moved to swap space to free up RAM for active processes. While this prevents crashes, excessive swap usage can significantly degrade performance - especially for database operations.

In your case where some database tables are in RAM while others are on disk, several scenarios could trigger swap usage:

  • Unexpected memory spikes in database operations
  • Insufficient RAM allocation for your working set
  • Kernel aggressively swapping due to vm.swappiness settings
  • Memory leaks in database processes

Here's a powerful command to identify processes using swap:


for file in /proc/*/status ; do awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}' $file; done | sort -k 2 -n -r | less

Alternatively, use smem for a more readable output:


sudo apt install smem
smem -s swap -r

1. Adjust swappiness: The kernel parameter vm.swappiness controls how aggressively your system uses swap. For database servers, a value between 1-10 is recommended:


sudo sysctl vm.swappiness=10
echo "vm.swappiness = 10" | sudo tee -a /etc/sysctl.conf

2. Prioritize database memory: Use cgroups to protect your database processes from being swapped:


sudo cgcreate -g memory:dbgroup
echo 4G > /sys/fs/cgroup/memory/dbgroup/memory.limit_in_bytes
echo 0 > /sys/fs/cgroup/memory/dbgroup/memory.swappiness

3. Database-specific optimizations: For MySQL/MariaDB, adjust these settings in my.cnf:


[mysqld]
innodb_buffer_pool_size = 12G  # 50-70% of available RAM
innodb_flush_method = O_DIRECT
innodb_io_capacity = 2000

Create a monitoring script to alert when swap usage exceeds thresholds:


#!/bin/bash
SWAP_THRESHOLD=10  # Percentage
current_swap=$(free | awk '/Swap/{printf "%.0f", $3/$2*100}')

if [ "$current_swap" -gt "$SWAP_THRESHOLD" ]; then
    echo "Warning: Swap usage at ${current_swap}%" | mail -s "High Swap Alert" admin@example.com
    # Optional: Log offending processes
    smem -s swap -r | head -20 >> /var/log/swap_alert.log
fi

If you consistently see high swap usage despite optimizations, consider:

  • Upgrading to faster NVMe SSDs for swap (if you must use swap)
  • Increasing total system RAM
  • Implementing a dedicated RAM disk for critical database tables