When automating file transfers through cron jobs or scheduled tasks, we often need to ensure that only one instance of the script runs at any given time. This becomes critical when:
- Processing large files that take significant time to transfer
- Operations aren't atomic and could corrupt data if interrupted
- Resource consumption needs to be controlled
Here are three reliable methods to prevent concurrent execution:
1. File Locking Method
function isProcessRunning() {
$lockFile = '/tmp/file_transfer.lock';
// Check if lock file exists and is still valid
if (file_exists($lockFile)) {
$lockTime = filemtime($lockFile);
// Consider stale locks after 1 hour
if (time() - $lockTime < 3600) {
return true;
}
// Clean up stale lock
unlink($lockFile);
}
// Create new lock
touch($lockFile);
return false;
}
// Main script logic
if (!isProcessRunning()) {
try {
// Your file transfer logic here
// On completion, remove the lock
unlink('/tmp/file_transfer.lock');
} catch (Exception $e) {
// Ensure lock is removed even on failure
unlink('/tmp/file_transfer.lock');
error_log($e->getMessage());
}
} else {
exit("Another instance is already running\n");
}
2. Process Checking via PID
function isScriptRunning($scriptName) {
exec("ps aux | grep 'hp.*{$scriptName}'", $output);
return count($output) > 1; // More than our current process
}
if (isScriptRunning(basename(__FILE__))) {
exit("Script already running\n");
}
3. Database Flag Approach
function getDbLock($connection, $lockName = 'file_transfer') {
$stmt = $connection->prepare("
INSERT INTO process_locks (name, created_at)
VALUES (?, NOW())
ON DUPLICATE KEY UPDATE
created_at = IF(TIMESTAMPDIFF(SECOND, created_at, NOW()) > 3600, NOW(), created_at)
");
$stmt->execute([$lockName]);
return $stmt->rowCount() > 0;
}
// In your main script:
$db = new PDO('mysql:host=localhost;dbname=your_db', 'user', 'pass');
if (!getDbLock($db)) {
exit("Process is locked\n");
}
// Remember to release lock when done:
$db->exec("DELETE FROM process_locks WHERE name = 'file_transfer'");
For enterprise-level solutions, consider:
- Implementing a proper job queue system (Redis, RabbitMQ)
- Using system semaphores via
flock()
- Adding monitoring for zombie processes
- Implementing proper logging for audit trails
For more complex scenarios, these patterns might be better:
- Producer-consumer pattern with persistent workers
- Event-driven architecture using inotify
- Distributed locking with Redis for cloud environments
When dealing with scheduled file upload scripts in PHP, a common challenge arises when the script execution time exceeds the scheduled interval. If your cron job runs every 5 minutes but an upload takes 10 minutes, you'll end up with multiple instances trying to process the same files simultaneously. This can lead to:
- File corruption during concurrent uploads
- Duplicate upload attempts
- Server resource exhaustion
- Race conditions in file processing
The most reliable method is implementing a lockfile mechanism. Here's how it works:
getMessage());
}
If you're running multiple servers or need more robust locking, consider a database approach:
prepare("
INSERT INTO process_locks (process_name, locked_at, pid)
VALUES ('file_upload', NOW(), ?)
ON DUPLICATE KEY UPDATE
locked_at = IF(TIMESTAMPDIFF(MINUTE, locked_at, NOW()) > 5, VALUES(locked_at), locked_at),
pid = IF(TIMESTAMPDIFF(MINUTE, locked_at, NOW()) > 5, VALUES(pid), pid)
");
$pid = getmypid();
if ($stmt->execute([$pid]) && $stmt->rowCount() > 0) {
try {
// Your upload logic
process_files();
// Release lock
$db->exec("DELETE FROM process_locks WHERE process_name = 'file_upload'");
} catch (Exception $e) {
$db->exec("DELETE FROM process_locks WHERE process_name = 'file_upload'");
throw $e;
}
} else {
exit("Upload process already running\n");
}
For high-performance systems, PHP's semaphore functions can be used:
When setting up your cron job, consider these optimizations:
# Run every 5 minutes, but with flock to prevent overlaps */5 * * * * /usr/bin/flock -n /tmp/file_upload.lock /usr/bin/php /path/to/upload_script.php
The flock
command provides system-level file locking that's more reliable than PHP's implementation in some cases.
For particularly large files, consider these additional measures:
- Implement chunked uploads to reduce single-file processing time
- Add progress tracking to resume interrupted uploads
- Use a queue system (like RabbitMQ or database queues) for better control
- Log processing status to monitor script health