When you launch an EC2 instance with a user data script, cloud-init executes it only once during initial boot. This is controlled by state files that track execution status. The primary state file is typically located at:

/var/lib/cloud/instance/sem/config_scripts_user

To make your user data script run again on subsequent boots, you need to:

1. Remove the state file
2. Ensure cloud-init runs on next boot

Here's a complete solution you can add to your user data script:

#!/bin/bash
# Clean previous cloud-init state
rm -f /var/lib/cloud/instance/sem/config_scripts_user

# Alternative: Remove all cloud-init state (more aggressive)
# rm -rf /var/lib/cloud/instances/*/sem/*

# Ensure cloud-init runs on next boot
cloud-init clean
cloud-init init

For a more permanent solution that works across reboots, add this to /etc/rc.local:

#!/bin/bash
# Clear cloud-init user data execution flag
rm -f /var/lib/cloud/instance/sem/config_scripts_user

# Run your custom initialization script
/path/to/your/script.sh

exit 0

After implementing either method, check the cloud-init logs to verify execution:

tail -f /var/log/cloud-init-output.log

• This approach works for standard Amazon Linux and Ubuntu AMIs
• For Windows instances, the mechanism differs (check EC2Launch)
• Excessive user data script runs may impact instance performance

When working with AWS EC2 instances, cloud-init processes user data scripts only once by default during initial boot. This behavior is controlled through state tracking files that prevent re-execution on subsequent reboots. The primary state file is located at:

/var/lib/cloud/instances/[instance-id]/sem/config_scripts_user

1. Manual State File Removal

The most direct approach involves deleting the specific semaphore file:

sudo rm -f /var/lib/cloud/instances/$(curl -s http://169.254.169.254/latest/meta-data/instance-id)/sem/config_scripts_user
sudo cloud-init clean
sudo cloud-init init

2. Cloud-init Modules Configuration

Modify cloud-init configuration to enable periodic execution:

# /etc/cloud/cloud.cfg.d/99_run_userdata.cfg
cloud_final_modules:
 - [scripts-user, always]

3. Instance Metadata Service Approach

For programmatic control, use the instance metadata service:

#!/bin/bash
IMDS_TOKEN=$(curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")
INSTANCE_ID=$(curl -H "X-aws-ec2-metadata-token: $IMDS_TOKEN" http://169.254.169.254/latest/meta-data/instance-id)
sudo rm -f /var/lib/cloud/instances/$INSTANCE_ID/sem/config_scripts_user

Systemd Service Unit

Create a service to ensure user data runs on every boot:

# /etc/systemd/system/rerun-userdata.service
[Unit]
Description=Re-run EC2 user data script
After=network.target

[Service]
Type=oneshot
ExecStart=/bin/bash -c 'rm -f /var/lib/cloud/instances/$(curl -s http://169.254.169.254/latest/meta-data/instance-id)/sem/config_scripts_user && cloud-init modules --mode=config'
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

CloudFormation Template Example

For infrastructure-as-code scenarios:

Resources:
  MyInstance:
    Type: AWS::EC2::Instance
    Properties:
      UserData: 
        Fn::Base64: |
          #!/bin/bash
          # Clean previous cloud-init state
          cloud-init clean
          # Your regular user data script continues here
          yum install -y httpd
          systemctl start httpd

• Avoid frequent re-execution of potentially destructive operations
• Consider implementing idempotency checks in your scripts
• Test behavior in non-production environments first
• For Windows instances, the approach differs (check %ProgramData%\Amazon\EC2-Windows\Launch\Scripts)