When working with AWS, a common scenario involves EC2 instances in private subnets needing to access S3 buckets. The naive approach is to route traffic through a NAT gateway, but this introduces unnecessary costs and latency. AWS provides a more elegant solution: VPC Endpoints for S3.

VPC Endpoints for S3 allow private subnet instances to access S3 without traversing the public internet. This approach offers:

• Improved security (no exposure to public internet)
• Reduced costs (no NAT gateway charges)
• Better performance (direct AWS backbone connection)

Here's how to set up a Gateway Endpoint for S3:

# Create VPC Endpoint using AWS CLI
aws ec2 create-vpc-endpoint \
    --vpc-id vpc-12345678 \
    --service-name com.amazonaws.us-east-1.s3 \
    --route-table-ids rtb-12345678 \
    --policy-document file://policy.json

Example policy.json:

{
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": "*",
      "Action": ["s3:GetObject", "s3:ListBucket"],
      "Resource": ["arn:aws:s3:::your-bucket-name", "arn:aws:s3:::your-bucket-name/*"]
    }
  ]
}

After creating the endpoint, test connectivity from your private instance:

# Test S3 access from EC2 instance
aws s3 ls s3://your-bucket-name --region us-east-1

For more complex scenarios, consider:

1. Endpoint policies for fine-grained access control
2. Multiple endpoints for different S3 buckets
3. Combining with S3 Access Points for multi-account access

If you encounter issues:

• Verify IAM permissions on both the endpoint and instance role
• Check route tables are properly associated
• Ensure DNS resolution is enabled in your VPC

While VPC Endpoints provide direct access, remember:

• There's still a slight latency compared to public internet access
• Throughput is generally excellent for most use cases
• Consider S3 Transfer Acceleration for large file transfers

When deploying backend servers in private VPC subnets that need to interact with S3, architects face a fundamental infrastructure decision: whether to use public internet routing (via NAT) or leverage AWS's private networking capabilities. The optimal solution depends on security requirements, cost considerations, and performance needs.

AWS provides Gateway VPC Endpoints specifically for S3 access. This creates a private connection between your VPC and S3 without traversing the public internet:

aws ec2 create-vpc-endpoint \
    --vpc-id vpc-1a2b3c4d \
    --service-name com.amazonaws.us-east-1.s3 \
    --route-table-ids rtb-11aa22bb \
    --policy-document file://endpoint-policy.json

Example endpoint policy (endpoint-policy.json):

{
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": "*",
      "Action": [
        "s3:GetObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::your-bucket-name",
        "arn:aws:s3:::your-bucket-name/*"
      ]
    }
  ]
}

Internal endpoints provide:

• Lower latency (no NAT hop)
• Higher throughput (no bandwidth constraints of NAT instances)
• More consistent performance (not subject to internet congestion)

Gateway endpoints offer security advantages:

# Sample S3 bucket policy restricting access to specific VPC
{
  "Version": "2012-10-17",
  "Id": "Policy1415115909152",
  "Statement": [
    {
      "Sid": "Access-to-specific-VPCE",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::your-bucket",
        "arn:aws:s3:::your-bucket/*"
      ],
      "Condition": {
        "StringNotEquals": {
          "aws:sourceVpce": "vpce-1a2b3c4d"
        }
      }
    }
  ]
}

VPC endpoints eliminate:

• NAT Gateway data processing charges ($0.045/GB in us-east-1)
• Potential cross-AZ data transfer costs
• Only incurs standard S3 request pricing

For applications using AWS SDKs:

// Python example using boto3 with VPC endpoint
import boto3

s3 = boto3.client(
    's3',
    endpoint_url='https://bucket.vpce-xxxxxx.s3.us-east-1.vpce.amazonaws.com'
)

response = s3.list_objects_v2(Bucket='your-bucket')

• Verify route tables are properly associated with the endpoint
• Check S3 bucket policies for explicit VPC endpoint denies
• Monitor VPC endpoint connection counts (soft limit of 1024 connections)
• Use VPC Flow Logs to diagnose connection issues

Consider NAT when:

• Accessing S3-compatible non-AWS services
• Requiring static IP addresses for allowlisting
• Operating in regions without VPC endpoint support