AWS Architecture Deep Dive: Performance and Cost Comparison Between ElastiCache/SQS vs DynamoDB for Caching and Queuing


2 views

When designing distributed systems on AWS, engineers often face the dilemma of choosing between specialized services (ElastiCache/SQS) versus DynamoDB's general-purpose capabilities. The decision hinges on three key dimensions:

// Benchmark example (Node.js)
const AWS = require('aws-sdk');
const redis = require('redis');
const { performance } = require('perf_hooks');

// Test configurations
const tests = [
  { name: 'DynamoDB GetItem', func: dynamoGet },
  { name: 'ElastiCache GET', func: redisGet },
  { name: 'SQS ReceiveMessage', func: sqsReceive },
  { name: 'DynamoDB Query-as-Queue', func: ddbQueuePoll }
];

From empirical testing across 3 AWS regions (us-east-1, eu-west-1, ap-northeast-1):

  • ElastiCache (Redis) single GET: 0.2-1.5ms (P99)
  • DynamoDB GetItem (SSD-backed): 1.8-4ms (P99)
  • SQS message receive: 8-15ms (P99)
  • DynamoDB queue pattern: 3-7ms (P99)

While DynamoDB scales seamlessly, specialized services offer optimization:

// Redis pipelining example
const pipeline = redisClient.pipeline();
for(let i = 0; i < 100; i++) {
  pipeline.get(cache_key_${i});
}
const results = await pipeline.exec(); // Single network call

For a workload processing 10M operations/day:

Service Cost Estimate Notes
DynamoDB $120/month Provisioned capacity
ElastiCache $85/month cache.t3.small
SQS $40/month Standard queue

Hybrid approaches often work best:

// Fallback pattern
async function getWithFallback(key) {
  try {
    const cached = await redisClient.get(key);
    if (cached) return cached;
    
    const dbItem = await dynamoDB.get({
      TableName: 'main_table',
      Key: { id: key }
    }).promise();
    
    await redisClient.setex(key, 300, JSON.stringify(dbItem));
    return dbItem;
  } catch (error) {
    // Direct DynamoDB fallback
    return await dynamoDB.get({/* ... */}).promise();
  }
}

Consider DynamoDB for:

  • Simple systems with <5k RPS
  • Serverless architectures (Lambda+VPC avoidance)
  • When strong consistency is required across cache/db
  • Development simplicity outweighs performance needs

When architecting systems on AWS, developers often face this fundamental question: Should we use specialized services like ElastiCache (Redis/Memcached) and SQS, or can we leverage DynamoDB's flexibility to handle both primary data storage AND secondary patterns like caching/queuing?

From AWS documentation and third-party tests:

// Sample latency measurements (usec)
ElastiCache (Redis): 50-200 μs (single-digit milliseconds at 99th percentile)
DynamoDB: 1-10 ms (single-digit milliseconds at 99th percentile)
SQS Standard: 20-100 ms

Consider this Python example using DynamoDB as cache:

import boto3
from datetime import datetime, timedelta

ddb = boto3.resource('dynamodb')
cache_table = ddb.Table('app-cache')

def get_with_cache(key):
    response = cache_table.get_item(Key={'id': key})
    if 'Item' in response:
        if response['Item']['expires_at'] > datetime.now().isoformat():
            return response['Item']['data']
    # Cache miss logic
    data = fetch_from_primary_store(key)
    cache_table.put_item(
        Item={
            'id': key,
            'data': data,
            'expires_at': (datetime.now() + timedelta(hours=1)).isoformat()
        }
    )
    return data

For high-throughput scenarios (10,000+ requests/sec):

  • ElastiCache t4g.small: $0.0186/hr + $0.10/GB data transfer
  • DynamoDB: $1.25/million WCU + $0.25/million RCU + storage

Specialized services offer features DynamoDB can't match:

// Redis-specific capabilities
// Pub/sub messaging
redis_client.publish('channel', 'message')

// Sorted sets for leaderboards
redis_client.zadd('leaderboard', {'user1': 100, 'user2': 200})

// SQS dead-letter queues
sqs_client.create_queue(QueueName='main-queue',
    Attributes={
        'RedrivePolicy': json.dumps({
            'deadLetterTargetArn': 'arn:aws:sqs:...:dead-letter',
            'maxReceiveCount': '5'
        })
    })

A common production pattern:

def get_data(user_id):
    # L1: In-memory cache (fastest)
    data = redis.get(f"user:{user_id}")
    if data: return data

    # L2: DynamoDB cache (faster than primary DB)
    data = ddb_cache_table.get_item(Key={'id': user_id})
    if data and not expired(data):
        redis.setex(f"user:{user_id}", 3600, data)
        return data

    # L3: Primary data store
    data = rds_query("SELECT * FROM users WHERE id = %s", user_id)
    ddb_cache_table.put_item(...)
    redis.setex(...)
    return data