When working with AWS CloudFormation, many developers encounter this specific scenario: They set DeletionPolicy: Retain to preserve critical resources like S3 buckets during stack deletion, but then face creation failures when attempting to redeploy the stack. The system throws a "bucket already exists" error because CloudFormation's default behavior is to attempt resource creation regardless of existing infrastructure.

CloudFormation operates on a strict create/update/delete paradigm. When you specify DeletionPolicy: Retain, you're only modifying the delete behavior - not the create behavior. This is an important distinction often overlooked in documentation.

Here are three approaches to handle this situation effectively:

1. Import Existing Resources

The most robust solution is to use CloudFormation's resource import feature (available since 2019):

# First, modify your template to include the import identifier
"Resources": {
  "SomeS3Bucket": {
    "Type": "AWS::S3::Bucket",
    "DeletionPolicy": "Retain",
    "Properties": {
      "BucketName": "existing-bucket-name"
    }
  }
}

Then use the AWS CLI:

aws cloudformation create-change-set \
  --stack-name MyStack \
  --change-set-name ImportChangeSet \
  --change-set-type IMPORT \
  --resources-to-import "[{\"ResourceType\":\"AWS::S3::Bucket\",\"LogicalResourceId\":\"SomeS3Bucket\",\"ResourceIdentifier\":{\"BucketName\":\"existing-bucket-name\"}}]" \
  --template-body file://template.yaml

2. Conditional Creation with Custom Resource

For more complex scenarios, implement a custom Lambda-backed resource:

"S3BucketChecker": {
  "Type": "AWS::Lambda::Function",
  "Properties": {
    "Runtime": "python3.8",
    "Handler": "index.lambda_handler",
    "Role": { "Fn::GetAtt": ["LambdaExecutionRole", "Arn"] },
    "Code": {
      "ZipFile": {
        "Fn::Join": ["\n", [
          "import boto3",
          "def lambda_handler(event, context):",
          "    s3 = boto3.client('s3')",
          "    try:",
          "        s3.head_bucket(Bucket=event['ResourceProperties']['BucketName'])",
          "        return {'Exists': True}",
          "    except:",
          "        return {'Exists': False}"
        ]]
      }
    }
  }
}

3. Stack Policy Adjustment

For quick temporary solutions during development, you can modify the stack policy:

aws cloudformation set-stack-policy \
  --stack-name MyStack \
  --stack-policy-body '{
    "Statement" : [{
      "Effect" : "Allow",
      "Action" : "Update:*",
      "Principal": "*",
      "Resource" : "*"
    },{
      "Effect" : "Deny",
      "Action" : ["Update:Replace", "Update:Delete"],
      "Principal": "*",
      "Resource" : "LogicalResourceId/SomeS3Bucket"
    }]
  }'

• Always document retained resources in your infrastructure documentation
• Implement naming conventions that differentiate between new and existing resources
• Consider using StackSets for organization-wide resource management
• Regularly audit your retained resources to avoid "orphaned" infrastructure

Be aware of these special scenarios:

• Cross-account resource sharing
• Resources with dependencies (like bucket policies)
• Region-specific considerations
• Service-linked roles that might be affected

When working with AWS CloudFormation, the DeletionPolicy: Retain attribute is commonly used to prevent critical resources from being accidentally deleted during stack operations. However, many developers encounter a frustrating scenario where CloudFormation fails during stack recreation because it attempts to create a resource that already exists.

When you set DeletionPolicy: Retain on a resource:

• The resource persists when the stack is deleted
• CloudFormation loses track of the resource's state
• Subsequent stack creations attempt to create a new resource with the same configuration

AWS provides a resource import feature to handle this exact scenario. Here's how to modify your template:

"Resources": {
  "SomeS3Bucket": {
    "Type": "AWS::S3::Bucket",
    "DeletionPolicy": "Retain",
    "Properties": {
      "BucketName": "SomeS3Bucket"
    }
  }
}

1. Prepare your template: Ensure it exactly matches the existing resource configuration
2. Create a changeset with import operation:

aws cloudformation create-change-set \
  --stack-name MyStack \
  --change-set-name ImportChangeSet \
  --change-set-type IMPORT \
  --resources-to-import "[{\"ResourceType\":\"AWS::S3::Bucket\",\"LogicalResourceId\":\"SomeS3Bucket\",\"ResourceIdentifier\":{\"BucketName\":\"SomeS3Bucket\"}}]" \
  --template-body file://template.yaml

For more complex scenarios, you can use custom resources to check for existing resources:

"Resources": {
  "BucketExistenceChecker": {
    "Type": "Custom::BucketExistenceChecker",
    "Properties": {
      "ServiceToken": !GetAtt CheckBucketLambda.Arn,
      "BucketName": "SomeS3Bucket"
    }
  }
}

• Always document retained resources outside CloudFormation
• Consider using stack exports/imports for cross-stack references
• Implement proper naming conventions to avoid conflicts

Remember that after importing, updates to the resource configuration will still be managed by CloudFormation. Ensure your IAM permissions allow CloudFormation to modify the retained resource when needed.