How to Properly Schedule Service Restarts in Linux Using Cron (httpd/mysqld Example)


2 views

Running web services on AWS EC2 micro instances (especially with limited RAM like 630MB) often leads to gradual memory consumption. As shown in the free -m output, available memory can shrink to critical levels:


             total       used       free     shared    buffers     cached
Mem:           596        573         23          0          8         71
-/+ buffers/cache:        493        103
Swap:            0          0          0

Periodic service restarts can reclaim 350MB+ of RAM, making cron automation a practical solution.

The "service: command not found" error occurs because cron executes commands in a minimal environment. Unlike your interactive shell, cron doesn't have the same PATH environment variable.

From your process list, we can see cron spawns a basic shell:


root     14664  0.0  0.2 142200  1720 ?        S    22:41   0:00 CROND
root     14665  0.0  0.2   9296  1236 ?        Ss   22:41   0:00 /bin/sh -c ./scriptName.sh

Here's the corrected approach for your restart script:


#!/bin/bash

# Use full paths to service commands
/sbin/service httpd restart
/sbin/service mysqld restart

# Alternative for systemd systems
# /bin/systemctl restart httpd
# /bin/systemctl restart mysqld

# Monitoring commands
ps aux
free -m

For root's crontab (sudo crontab -e):


# Restart services every 12 hours at 03:15 and 15:15
15 3,15 * * * /full/path/to/scriptName.sh

Consider adding logic to only restart when memory is critically low:


#!/bin/bash

THRESHOLD=50 # MB

FREE_MEM=$(free -m | awk '/Mem:/ {print $7}')

if [ $FREE_MEM -lt $THRESHOLD ]; then
    /sbin/service httpd restart
    /sbin/service mysqld restart
    echo "Services restarted due to low memory: ${FREE_MEM}MB free"
else
    echo "Memory OK: ${FREE_MEM}MB free - no restart needed"
fi

ps auxf
free -m

Instead of scheduled restarts, consider:

  • Tuning Apache's MaxRequestWorkers
  • Implementing PHP process manager (for mod_php)
  • Adding swap space as temporary relief
  • Upgrading to an instance with more RAM

Running web services on AWS EC2 micro instances (especially those with only 630MB RAM) often leads to gradual memory exhaustion. As shown in the free -m output:

             total       used       free     shared    buffers     cached
Mem:           596        573         23          0          8         71
-/+ buffers/cache:        493        103
Swap:            0          0          0

Apache's httpd processes tend to consume increasing memory over time, and manual restarts can reclaim ~350MB - a significant portion of the available memory.

The error service: command not found occurs because cron jobs execute with a minimal environment. The service command typically resides in /usr/sbin/ which may not be in cron's PATH.

Here's a more robust script example:

#!/bin/bash

# Full path to service binary
/usr/sbin/service httpd restart
/usr/sbin/service mysqld restart

# Verification commands
/usr/bin/ps aux
/usr/bin/free -m

Instead of blind restarts, consider these improvements:

#!/bin/bash

# Check memory threshold before restarting
MEM_THRESHOLD=100 # MB
CURRENT_MEM=$(free -m | awk '/Mem:/ {print $4}')

if [ "$CURRENT_MEM" -lt "$MEM_THRESHOLD" ]; then
    /usr/sbin/service httpd graceful  # Prefer graceful restart
    /usr/sbin/service mysql restart   # MySQL typically needs full restart
    
    # Log the event
    echo "$(date) - Restarted services due to low memory ($CURRENT_MEM MB free)" >> /var/log/service_restarts.log
fi

For modern Linux distributions using systemd, timer units provide better control:

# /etc/systemd/system/httpd-memory-watcher.service
[Unit]
Description=Apache HTTPD memory watcher

[Service]
Type=oneshot
ExecStart=/usr/bin/bash -c 'if [ $(free -m | awk \'/Mem:/ {print $4}\') -lt 100 ]; then systemctl restart httpd; fi'

# /etc/systemd/system/httpd-memory-watcher.timer
[Unit]
Description=Check Apache memory every 12 hours

[Timer]
OnCalendar=*-*-* 00/12:00:00
Persistent=true

[Install]
WantedBy=timers.target

For production systems, consider proper monitoring solutions:

  • AWS CloudWatch alarms triggering Lambda functions
  • Monit or Supervisor for process monitoring
  • Custom metrics with Prometheus and Grafana

While automated restarts can help, they're not a permanent solution:

  • Investigate why memory leaks occur (check Apache modules, PHP configurations)
  • Consider upgrading to a larger instance type if possible
  • Implement proper caching mechanisms
  • Review MySQL configuration (innodb_buffer_pool_size etc.)