While kill -9
(SIGKILL) is considered the ultimate way to terminate a process in Linux, there are rare cases where even this nuclear option fails. This typically happens when:
# Common kill command that might fail
$ kill -9 12345
# Process still appears in ps output
$ ps aux | grep 12345
The process might be in an uninterruptible sleep state (D state in ps
output), usually due to:
- Waiting for blocked I/O operations
- Kernel-level deadlocks
- Hardware communication hangs
1. Direct Kernel Process Termination
Use the kernel's sysrq
trigger if available:
# Enable sysrq if not already enabled
$ echo 1 > /proc/sys/kernel/sysrq
# Send SIGKILL to all processes except init
$ echo f > /proc/sysrq-trigger
# Alternative: kill specific process by PID
$ echo kill <PID> > /proc/sysrq-trigger
2. Using gdb to Attach to Process
For stubborn processes that hold resources:
$ sudo gdb -p <PID>
(gdb) call exit(1)
(gdb) detach
(gdb) quit
3. Kernel OOM Killer
Force the OOM killer to target the specific process:
# Make process appear as huge memory consumer
$ sudo echo -1000 > /proc/<PID>/oom_score_adj
# Alternatively trigger OOM killer manually
$ sudo echo f > /proc/sysrq-trigger
Rebooting the System
When all else fails, consider these controlled reboot methods:
# Magic SysRq reboot sequence (safer than hard reset)
$ echo 1 > /proc/sys/kernel/sysrq
$ echo b > /proc/sysrq-trigger
For critical processes that might hang:
- Implement watchdog scripts
- Use cgroups to contain process resources
- Consider kernel hardening for better process isolation
We've all been there - you issue the infamous kill -9 PID
command, expecting instant process termination, only to find the stubborn process still running. This happens because kill -9
(SIGKILL) simply sends a termination signal to the process. If the process is in an uninterruptible sleep state (D state) or has kernel-level issues, even SIGKILL may fail.
First, verify the process state using:
ps aux | grep process_name
ps -eo pid,state,cmd | grep '^ *PID' -A 1
Look for "D" in the STATE column, indicating an uninterruptible sleep state.
1. Kernel-Level Termination
For processes stuck in kernel space:
echo 1 > /proc/sys/kernel/sysrq
echo b > /proc/sysrq-trigger # Immediate reboot (use with caution)
2. Process Tree Annihilation
Sometimes you need to kill the entire process tree:
pkill -TERM -P parent_pid
killall -9 process_name
3. Direct /proc Manipulation
For extreme cases, directly clear process resources:
gdb -p PID
(gdb) call exit(1)
(gdb) detach
For Ruby processes specifically, consider:
# Use timeout wrappers
Timeout.timeout(30) do
# risky operation
end
# Or implement signal handlers
Signal.trap("TERM") { exit }
If all else fails:
- Reboot the machine (last resort)
- Use
strace -p PID
to diagnose system calls - Check for hung I/O operations with
lsof -p PID
Remember that persistent unkillable processes often indicate deeper system issues that warrant investigation.