When working with AWS CloudFormation, a common pain point emerges when trying to manage existing S3 buckets that weren't originally created through CloudFormation. The standard AWS::S3::Bucket resource expects to create new buckets, throwing the "already exists" error when attempting to modify pre-existing infrastructure.

1. Using Custom Resources with Lambda

This method involves creating a Lambda-backed custom resource that modifies the existing bucket:

"BucketConfigurator": {
  "Type": "Custom::S3BucketConfigurator",
  "Properties": {
    "ServiceToken": { "Fn::GetAtt": ["ConfiguratorLambda", "Arn"] },
    "BucketName": "preview.website.com",
    "AccessControl": "PublicRead",
    "WebsiteConfiguration": {
      "IndexDocument": "index.html",
      "ErrorDocument": "error.html"
    }
  }
}

The Lambda function would use the AWS SDK to update bucket properties:

const AWS = require('aws-sdk');
const s3 = new AWS.S3();

exports.handler = async (event) => {
  if (event.RequestType === 'Create' || event.RequestType === 'Update') {
    await s3.putBucketAcl({
      Bucket: event.ResourceProperties.BucketName,
      ACL: event.ResourceProperties.AccessControl
    }).promise();
    
    await s3.putBucketWebsite({
      Bucket: event.ResourceProperties.BucketName,
      WebsiteConfiguration: event.ResourceProperties.WebsiteConfiguration
    }).promise();
  }
  
  return { PhysicalResourceId: event.ResourceProperties.BucketName };
};

2. CloudFormation StackSets (For Organization-Wide Management)

If managing buckets across multiple accounts, StackSets can help maintain consistent configurations:

"BucketPolicy": {
  "Type": "AWS::S3::BucketPolicy",
  "Properties": {
    "Bucket": "preview.website.com",
    "PolicyDocument": {
      "Version": "2012-10-17",
      "Statement": [{
        "Effect": "Allow",
        "Principal": "*",
        "Action": "s3:GetObject",
        "Resource": "arn:aws:s3:::preview.website.com/*"
      }]
    }
  }
}

• Idempotency: Ensure your configuration logic can handle repeated applications
• Permissions: The execution role needs s3:PutBucket* permissions
• Drift Detection: Manual changes to the bucket may cause configuration drift

For complex scenarios, consider:

• Using AWS Config rules to enforce bucket settings
• Implementing S3 Batch Operations for bulk updates
• Creating a separate "configuration" stack that only manages existing resources

Many developers face a common dilemma when trying to manage existing AWS resources through CloudFormation. The service is primarily designed for provisioning new resources, which creates complications when you need to modify properties of already-created S3 buckets.

When you define an AWS::S3::Bucket resource in your template with an existing bucket name, CloudFormation attempts to create a new bucket rather than updating the existing one. This results in the familiar error:

The following resource(s) failed to create: [websitePreviewBucket].
preview.website.com already exists

While CloudFormation doesn't natively support direct modification of existing buckets, there are several approaches to achieve similar results:

1. Using Custom Resources with Lambda

This is the most flexible solution, allowing you to modify any existing bucket property:

"WebsiteConfigUpdater": {
  "Type": "Custom::S3BucketConfig",
  "Version": "1.0",
  "Properties": {
    "ServiceToken": {"Fn::GetAtt": ["ConfigLambdaFunction", "Arn"]},
    "BucketName": "preview.website.com",
    "AccessControl": "PublicRead",
    "WebsiteConfiguration": {
      "IndexDocument": "index.html",
      "ErrorDocument": "error.html"
    }
  }
}

2. AWS CLI in CodePipeline

For CI/CD scenarios, you can add a build step that executes AWS CLI commands:

aws s3api put-bucket-website \
  --bucket preview.website.com \
  --website-configuration file://website.json

3. Using Stack Policies with Import

CloudFormation's resource import feature combined with stack policies can help:

{
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["Update:Modify"],
      "Principal": "*",
      "Resource": "*",
      "Condition": {
        "StringEquals": {
          "ResourceType": ["AWS::S3::Bucket"]
        }
      }
    }
  ]
}

For enterprise scenarios, consider these best practices:

• Always test changes in a staging environment first
• Implement change sets to review modifications
• Use AWS Config to monitor bucket configuration drift
• Combine CloudFormation with AWS Organizations SCPs for governance

Here's a complete solution using Lambda-backed custom resources:

"Resources": {
  "ConfigLambdaFunction": {
    "Type": "AWS::Lambda::Function",
    "Properties": {
      "Handler": "index.handler",
      "Runtime": "python3.8",
      "Code": {
        "ZipFile": |
          import boto3
          import cfnresponse
          def handler(event, context):
            s3 = boto3.client('s3')
            try:
              if event['RequestType'] == 'Create':
                # Update bucket configuration
                s3.put_bucket_acl(
                  Bucket=event['ResourceProperties']['BucketName'],
                  ACL=event['ResourceProperties']['AccessControl']
                )
                # Additional configuration updates here
                cfnresponse.send(event, context, cfnresponse.SUCCESS, {})
            except Exception as e:
              cfnresponse.send(event, context, cfnresponse.FAILED, {'Error': str(e)})
      },
      "Role": {"Fn::GetAtt": ["LambdaExecutionRole", "Arn"]}
    }
  }
}