When containerizing a Django application with Celery workers, one fundamental challenge emerges: how to efficiently share the same codebase between containers while maintaining Docker's isolation principles. Here are the approaches I've evaluated in production environments:

This method declares the code directory as a named volume and shares it between containers:

# docker-compose.yml
version: '3.8'

services:
  web:
    build: ./web
    volumes:
      - django_code:/usr/src/app
    
  celery:
    build: ./celery
    volumes_from:
      - web

volumes:
  django_code:

Pros:

• Single source of truth for application code
• No code duplication between containers
• Changes reflect immediately in both containers during development

For production deployments where live code reloading isn't needed, consider building a common base image:

# Dockerfile.common
FROM python:3.9-slim as builder
WORKDIR /usr/src/app
COPY requirements.txt .
RUN pip install --user -r requirements.txt
COPY . .

# Web Dockerfile
FROM builder as web
CMD ["gunicorn", "core.wsgi:application"]

# Celery Dockerfile
FROM builder as celery
CMD ["celery", "-A", "core", "worker"]

For teams using CI/CD pipelines, you can pull code during build time:

# Dockerfile.celery
FROM python:3.9-slim
RUN apt-get update && apt-get install -y git openssh-client
ARG SSH_PRIVATE_KEY
RUN mkdir -p ~/.ssh && \
    echo "${SSH_PRIVATE_KEY}" > ~/.ssh/id_rsa && \
    chmod 600 ~/.ssh/id_rsa
RUN ssh-keyscan github.com >> ~/.ssh/known_hosts
RUN git clone git@github.com:your/repo.git /app
WORKDIR /app
RUN pip install -r requirements.txt

When benchmarking different approaches in AWS ECS:

When running a Django application with separate containers for Gunicorn (web server) and Celery (task queue), you face a fundamental challenge: both services need access to the same Django codebase. The naive approach of copying the code into both containers creates maintenance headaches and potential synchronization issues.

Most developers initially consider these approaches:

• Bind Mounts: While mount -o bind works during development, it's not production-friendly and breaks container isolation
• Code Duplication: Copying the code into both containers leads to version drift and deployment complexity
• Shared Volumes: Using volumes-from creates tight coupling between containers

Here's how I solved this in production:

# Dockerfile.common
FROM python:3.9 as base
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .

# Dockerfile.gunicorn
FROM base as gunicorn
EXPOSE 8000
CMD ["gunicorn", "--bind", "0.0.0.0:8000", "myapp.wsgi"]

# Dockerfile.celery
FROM base as celery
CMD ["celery", "-A", "myapp", "worker", "--loglevel=info"]

For more complex projects, consider this structure:

project/
├── core/ (Django app)
├── workers/
│   ├── Dockerfile (references core as submodule)
└── web/
    ├── Dockerfile (references core as submodule)

Then in your Dockerfiles:

# Example using Git submodules
RUN git clone https://github.com/your/repo.git --recursive \
    && cd repo \
    && git submodule update --init

• Use .dockerignore to exclude development files
• Implement proper layer caching for faster builds
• Consider using a private package registry for your base image

For large codebases, this pattern reduces image size:

FROM python:3.9-slim as builder
COPY requirements.txt .
RUN pip install --user -r requirements.txt

FROM python:3.9-slim
COPY --from=builder /root/.local /root/.local
COPY . .
ENV PATH=/root/.local/bin:$PATH