Implementing Blue/Green Deployments for CloudFront: Best Practices and Workarounds


1 views

Many teams face difficulties when attempting to implement blue/green deployments with Amazon CloudFront. The platform's design makes it challenging to switch between identical distributions due to CNAME uniqueness constraints and propagation delays. Let me share practical approaches based on real-world experience.

While the initial approach of using Route53 weighted routing fails due to CloudFront's CNAME restrictions, we can implement a workaround:

// Sample CloudFormation template for DNS switch
Resources:
  ProductionDistribution:
    Type: AWS::CloudFront::Distribution
    Properties:
      DistributionConfig:
        Aliases: ["www.example.com"]
        # ... other distribution config

  StagingDistribution:
    Type: AWS::CloudFront::Distribution
    Properties:
      DistributionConfig:
        Aliases: ["stage-www.example.com"]  # Different CNAME
        # ... identical config except CNAME

  ProductionRecord:
    Type: AWS::Route53::RecordSet
    Properties:
      Name: www.example.com
      Type: A
      AliasTarget:
        HostedZoneId: Z2FDTNDATAQYW2
        DNSName: !GetAtt ProductionDistribution.DomainName

  StagingRecord:
    Type: AWS::Route53::RecordSet
    Properties:
      Name: stage-www.example.com
      Type: A
      AliasTarget:
        HostedZoneId: Z2FDTNDATAQYW2
        DNSName: !GetAtt StagingDistribution.DomainName

For environments requiring changes to origins rather than full distribution changes:

# AWS CLI command to update origin path
aws cloudfront update-distribution \
    --id E1A2B3C4D5E6F7 \
    --distribution-config file://new-config.json \
    --if-match E2A3B4C5D6E7F8

# new-config.json snippet:
{
  "Origins": {
    "Items": [
      {
        "Id": "BlueOrigin",
        "DomainName": "blue-elb.example.com",
        "CustomOriginConfig": {
          "HTTPPort": 80,
          "HTTPSPort": 443,
          "OriginProtocolPolicy": "https-only"
        }
      },
      {
        "Id": "GreenOrigin",
        "DomainName": "green-elb.example.com",
        "CustomOriginConfig": {
          "HTTPPort": 80,
          "HTTPSPort": 443,
          "OriginProtocolPolicy": "https-only"
        }
      }
    ]
  }
}

For more granular control, consider using Lambda@Edge to route traffic based on deployment phases:

// Lambda@Edge origin request handler
exports.handler = (event, context, callback) => {
    const request = event.Records[0].cf.request;
    const headers = request.headers;
    
    // Check for canary header or cookie
    const deploymentPhase = headers['x-deployment-phase'] 
        ? headers['x-deployment-phase'][0].value 
        : 'blue';
    
    // Switch origin based on deployment phase
    if (deploymentPhase === 'green') {
        request.origin.custom.domainName = 'green-elb.example.com';
    } else {
        request.origin.custom.domainName = 'blue-elb.example.com';
    }
    
    callback(null, request);
};

Create a script to monitor distribution updates:

#!/bin/bash
DISTRIBUTION_ID="E1A2B3C4D5E6F7"
MAX_RETRIES=30
RETRY_INTERVAL=60

for ((i=1; i<=$MAX_RETRIES; i++)); do
    STATUS=$(aws cloudfront get-distribution \
        --id $DISTRIBUTION_ID \
        --query 'Distribution.Status' \
        --output text)
    
    if [ "$STATUS" == "Deployed" ]; then
        echo "Distribution update complete"
        exit 0
    fi
    
    echo "Current status: $STATUS (Attempt $i/$MAX_RETRIES)"
    sleep $RETRY_INTERVAL
done

echo "Monitoring timed out"
exit 1

After switching origins, perform targeted cache invalidation:

# Create invalidation for critical paths
aws cloudfront create-invalidation \
    --distribution-id E1A2B3C4D5E6F7 \
    --paths "/index.html" "/main.js" "/assets/*"

While CloudFront excels at content delivery, its immutable nature creates deployment headaches. The core issue stems from:

  • CNAME exclusivity - Only one distribution can claim a domain
  • Slow propagation - Updates take 15-60 minutes with no progress visibility
  • Cache contamination - Mixing old and new content without proper invalidation

Here are battle-tested approaches we've used in production:

1. Origin Flipping Technique


// CloudFront Distribution JSON snippet
"Origins": {
  "Items": [
    {
      "Id": "blue-origin",
      "DomainName": "blue-elb-1234567890.us-west-2.elb.amazonaws.com"
    },
    {
      "Id": "green-origin", 
      "DomainName": "green-elb-9876543210.us-west-2.elb.amazonaws.com"
    }
  ]
}

Key implementation steps:

  1. Maintain both origins in the same distribution
  2. Use Lambda@Edge for origin selection logic
  3. Gradually shift traffic via weighted routing

2. DNS-Based Cutover

While you can't share CNAMEs, you can implement:


# Route53 Weighted Record Set
resource "aws_route53_record" "green" {
  zone_id = "${var.zone_id}"
  name    = "cdn-alias.example.com"
  type    = "A"
  
  alias {
    name                   = "${aws_cloudfront_distribution.green.domain_name}"
    zone_id                = "${aws_cloudfront_distribution.green.hosted_zone_id}"
    evaluate_target_health = false
  }

  weighted_routing_policy {
    weight = 0 # Start with 0% traffic
  }

  set_identifier = "green"
}

For complex scenarios, consider:

Path-Based Routing


// CloudFront Cache Behavior
{
  "PathPattern": "/v2/*",
  "TargetOriginId": "green-origin",
  "ForwardedValues": {
    "QueryString": true,
    "Cookies": {
      "Forward": "all"
    }
  }
}

Header-Based Switching

Implement via Lambda@Edge:


exports.handler = (event, context, callback) => {
  const request = event.Records[0].cf.request;
  
  if (request.headers['x-deploy-version']?.value === 'green') {
    request.origin = {
      custom: {
        domainName: 'green-elb.example.com',
        port: 80,
        protocol: 'http',
        path: '',
        sslProtocols: ['TLSv1', 'TLSv1.1'],
        readTimeout: 5,
        keepaliveTimeout: 5
      }
    };
  }
  
  callback(null, request);
};

Essential verification steps:

  • CloudWatch metrics for both distributions
  • Synthetic canary tests from multiple regions
  • Real-user monitoring dashboards

Key takeaways from our deployment history:

  • Always maintain forward/backward compatibility during transitions
  • Automate cache invalidation as part of deployment pipelines
  • Consider TTL adjustments before major deployments