When configuring Supervisor to manage Celery processes, many developers encounter an unexpected behavior - the working directory specified in Supervisor's configuration doesn't seem to take effect. Despite setting the directory parameter in the program section of supervisord.conf, running pwd within the Celery process shows the root directory (/).

[program:celery_worker]
command=/path/to/celery -A project worker
directory=/path/to/project
autostart=true
autorestart=true

This happens because Celery, when started, may change its working directory internally. Supervisor does properly set the initial working directory, but some applications (including Celery) subsequently modify this during their initialization process.

Here are several approaches to ensure your Celery workers run in the correct directory:

1. Using Celery's --workdir Parameter

The most straightforward solution is to use Celery's built-in --workdir option:

[program:celery_worker]
command=/path/to/celery -A project worker --workdir=/path/to/project
autostart=true
autorestart=true

2. Wrapping with a Shell Script

For more complex setups, create a wrapper script:

#!/bin/bash
cd /path/to/project
exec /path/to/celery -A project worker

Then configure Supervisor to use this script:

[program:celery_worker]
command=/path/to/wrapper_script.sh
autostart=true
autorestart=true

3. Combining Supervisor directory and Celery --workdir

For maximum reliability, use both approaches:

[program:celery_worker]
command=/path/to/celery -A project worker --workdir=/path/to/project
directory=/path/to/project
autostart=true
autorestart=true

To confirm your Celery worker is running in the correct directory, add this to your task:

import os

@app.task
def check_working_dir():
    return {'working_directory': os.getcwd()}

Call this task and verify the output matches your expected directory.

---

When configuring Supervisor to manage Celery processes, many developers encounter a puzzling behavior: the directory parameter in supervisord.conf appears to have no effect on the working directory of the spawned process. Running pwd inside the script typically returns / instead of the specified directory.

Supervisor's directory option serves a specific purpose in the process lifecycle:

[program:my_celery_worker]
command=/path/to/celery -A project worker
directory=/path/to/project

The directory parameter sets the working directory before the command executes, but certain applications (like Celery) may subsequently change it during initialization.

Celery has its own workdir concept because:

• It needs to resolve relative paths in configuration
• It manages multiple worker processes that might need different working directories
• It provides process isolation between different task queues

Here are three ways to ensure proper working directory behavior:

1. Using Celery's built-in workdir

[program:celery_worker]
command=/path/to/celery -A project worker --workdir=/path/to/project

2. Wrapping with shell script

#!/bin/sh
cd /path/to/project
exec /path/to/celery -A project worker

3. Combining both approaches

[program:celery_worker]
command=/path/to/wrapper_script.sh
directory=/path/to/project

To verify the working directory at different stages:

# In your Python code:
import os
print("Current directory:", os.getcwd())

# In your bash wrapper:
echo "Starting in: $(pwd)"

The Supervisor directory parameter is still crucial for:

• Resolving relative paths in the command itself
• Managing log file locations
• Setting environment variables via .env files