Many developers encounter credential issues when transitioning from manual CLI execution to automated cron jobs. The AWS CLI works perfectly during interactive sessions but fails with "Unable to locate credentials" in cron environments. This typically stems from environment differences between user sessions and cron's restricted environment.

Cron runs commands in a minimal environment, which means:

1. No user-specific environment variables
2. Limited PATH settings
3. Different working directory
4. No shell initialization files loaded

Many attempts like these don't work:

# Doesn't work in cron:
AWS_CONFIG_FILE="~/.aws/config"

# Also ineffective:
export AWS_SHARED_CREDENTIALS_FILE=/home/user/.aws/credentials

1. Absolute Paths for Config Files

Replace the tilde with full path:

AWS_CONFIG_FILE="/home/username/.aws/config"
AWS_SHARED_CREDENTIALS_FILE="/home/username/.aws/credentials"

2. Environment Wrapper Script

Create a wrapper that sets the environment:

#!/bin/bash
export AWS_CONFIG_FILE="/root/.aws/config"
export AWS_SHARED_CREDENTIALS_FILE="/root/.aws/credentials"
export PATH=/usr/local/bin:$PATH

/path/to/your/backup_script.sh

3. IAM Role Alternative

For EC2 instances, consider using IAM roles instead of credentials file:

# Remove credential-related lines entirely
# The CLI will automatically use the instance's IAM role

Here's a production-tested cron-compatible version:

#!/bin/bash

# Set full paths to AWS config
export AWS_CONFIG_FILE="/root/.aws/config"
export AWS_SHARED_CREDENTIALS_FILE="/root/.aws/credentials"

# Ensure PATH includes AWS CLI
export PATH=/usr/local/bin:/usr/bin:/bin:$PATH

# Main backup logic
while read -r p; do
    /usr/local/bin/aws s3 cp "$p" s3://backup-bucket/path/ \
        --recursive >> /var/log/backup.log 2>&1
done < /etc/backup/include.txt

To troubleshoot cron issues:

# Capture full environment:
env > /tmp/cronenv.log

# Test PATH availability:
which aws >> /tmp/cronpath.log 2>&1

When running as root:

• Set strict permissions on credentials files (600)
• Consider using dedicated IAM users with minimal permissions
• Regularly rotate credentials

When executing AWS CLI commands via CRON, many developers encounter the frustrating "Unable to locate credentials" error despite the same commands working perfectly in interactive shell sessions. This typically occurs because CRON runs in a minimal environment that doesn't automatically load the same configuration and credentials as your user shell.

Your current approach of setting AWS_CONFIG_FILE="~/.aws/config" has three potential problems:

1. The ~ expansion doesn't work in CRON context
2. AWS CLI might need both config and credentials files
3. CRON doesn't inherit your shell's environment variables

Here are three reliable methods to make AWS CLI work in CRON:

Method 1: Absolute Paths with Full Configuration

AWS_CONFIG_FILE="/root/.aws/config"
AWS_SHARED_CREDENTIALS_FILE="/root/.aws/credentials"
export AWS_CONFIG_FILE AWS_SHARED_CREDENTIALS_FILE

while read p; do
  /usr/local/bin/aws s3 cp "$p" s3://PATH/TO/BUCKET --recursive >> /PATH/TO/LOG 2>&1
done 

AWS_ACCESS_KEY_ID=AKIAXXXXXXXXXXXXXXXX
AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
AWS_DEFAULT_REGION=us-west-2

0 * * * * . /etc/aws_env; /path/to/your/script.sh

#!/bin/bash

# No credentials needed when using IAM roles
while read p; do
  aws s3 cp "$p" s3://PATH/TO/BUCKET --recursive >> /PATH/TO/LOG 2>&1
done 

# Add this to your script temporarily
env > /tmp/cron_env.debug
aws configure list >> /tmp/cron_env.debug