How to Include External Files in Crontab Configuration for Deployment Automation


2 views

When managing multiple servers or deploying cron jobs across environments, maintaining identical cron configurations becomes crucial. The standard crontab format doesn't natively support file inclusion, but there are practical workarounds.

# Method 1: Using the crontab command with input redirection
crontab -l | cat - /path/to/cron_jobs.conf | crontab -

# Method 2: Concatenating multiple files
cat /etc/crontab.base /opt/app/cron/*.conf > /etc/crontab

Modern Linux systems support cron directories:

# For system-wide cron jobs
/etc/cron.d/myapp_jobs

# For user-specific jobs
mkdir -p ~/.cron.d
echo "* * * * * user /path/to/script" > ~/.cron.d/job1

Here's a complete deployment script example:

#!/bin/bash
# deploy_cron.sh

BASE_CRON="/etc/crontab.base"
APP_CRON="/opt/app/deploy/cron.conf"
TEMP_CRON="/tmp/combined_cron"

# Backup existing
crontab -l > /backup/cron_$(date +%Y%m%d).bak

# Combine files
cat ${BASE_CRON} ${APP_CRON} > ${TEMP_CRON}

# Validate syntax
crontab ${TEMP_CRON} || {
    echo "Cron syntax error detected"
    exit 1
}

# Clean up
rm ${TEMP_CRON}

For infrastructure automation:

# ansible playbook snippet
- name: Deploy cron jobs
  hosts: webservers
  tasks:
    - name: Combine cron files
      assemble:
        src: "{{ playbook_dir }}/cron_parts/"
        dest: "/etc/cron.d/application"
        mode: '0644'

Store your cron configurations in Git and use post-receive hooks:

# Sample post-receive hook
#!/bin/bash
while read oldrev newrev refname
do
    if [[ $refname = "refs/heads/master" ]]; then
        git --work-tree=/opt/app --git-dir=/opt/app/.git checkout -f
        /opt/app/scripts/deploy_cron.sh
    fi
done

Always validate cron syntax before deployment:

# Validation script example
validate_cron() {
    local file=$1
    crontab $file 2>&1 | grep -qv "errors detected" || {
        echo "Invalid cron syntax in $file"
        return 1
    }
}

When working on collaborative projects, maintaining consistent cron job configurations across multiple servers can become tedious. The standard approach of manually editing each server's crontab via crontab -e doesn't scale well and creates deployment headaches.

Unlike many Unix configuration files, crontab doesn't natively support include directives. The format you proposed:

* * * * * execsomething /blablabla/blah/
* * * * * onemore /blablabla
@include '/home/user/project/crontab.conf'

won't work because crontab parsers don't recognize @include statements.

Here are three proven approaches to achieve file inclusion in crontab:

1. Using cat in Deployment Scripts

The most straightforward method is to concatenate files during deployment:

# In your deployment script
(cat /etc/crontab.local /home/user/project/crontab.conf) | crontab -

2. Directory-based Approach (For System-wide Crontab)

Modern Linux systems often support cron directories:

# Place files in these directories
/etc/cron.d/          # System-wide
/etc/cron.d/project/  # Project-specific (custom)

Files in these directories follow the same format as crontab but can be managed individually.

3. Symbolic Link Strategy

For user-specific crontabs, create a symlink to your project file:

ln -s /home/user/project/crontab.conf /var/spool/cron/crontabs/project_jobs

When implementing any of these solutions:

  • Always include comments in your cron files indicating their source
  • Use absolute paths for all commands and includes
  • Implement file permission checks (crontab files should be 600)
  • Consider using version control hooks to validate syntax before deployment

Here's a complete example for a PHP project deployment:

#!/bin/bash
# deploy_cron.sh

PROJECT_DIR="/var/www/project"
CRON_SOURCE="${PROJECT_DIR}/config/cron.conf"
CRON_TEMP="/tmp/project_cron_$$"

# Combine existing crontab with project config
(crontab -l 2>/dev/null; cat "${CRON_SOURCE}") > "${CRON_TEMP}"

# Validate syntax before applying
if grep -q '^[^#]' "${CRON_TEMP}"; then
    crontab "${CRON_TEMP}" && echo "Crontab updated successfully"
else
    echo "Error: Invalid cron syntax detected" >&2
    exit 1
fi

rm -f "${CRON_TEMP}"

For complex scheduling needs, consider:

  • Celery Beat for Python projects
  • Sidekiq Scheduler for Ruby
  • Jenkins or Rundeck for cross-server job management