When running performance tests against an Akamai server from an AWS instance, I noticed dramatically different response times between initial and subsequent requests for a 3MB video file:
# First request (slow)
curl -v https://example.com/video.mp4 -o /dev/null
# Took ~5 seconds
# Second request (fast)
curl -v https://example.com/video.mp4 -o /dev/null
# Took ~200ms
The short answer is no, cURL doesn't have built-in response caching. The speed difference you're observing comes from other layers in the stack:
- DNS Caching: Both system-level and cURL's DNS cache
- TCP Connection Reuse: Keep-alive connections being reused
- CDN-Level Caching: Akamai's edge servers caching content
- OS-Level Caching: Filesystem caching of downloaded content
To ensure each request is truly fresh, use these cURL options:
curl -v \
-H "Cache-Control: no-cache" \
-H "Pragma: no-cache" \
--dns-servers 8.8.8.8 \
--no-keepalive \
https://example.com/video.mp4 -o /dev/null
For more accurate HTTP benchmarking:
# wrk (modern HTTP benchmarking tool)
wrk -t4 -c100 -d30s https://example.com/video.mp4
# vegeta (golang-based load tester)
echo "GET https://example.com/video.mp4" | vegeta attack -duration=30s | vegeta report
Add these headers to check caching behavior:
curl -v \
-H "X-Cache: DEBUG" \
-H "X-Cache-Key: DEBUG" \
https://example.com/video.mp4
This will often return headers showing whether the response came from cache (HIT) or origin (MISS).
During recent performance testing of an Akamai server from an AWS instance, I observed puzzling behavior with cURL requests:
# First request (slow)
curl -v https://example.com/video.mp4 -o /dev/null
# Time: ~5000ms
# Subsequent request (fast)
curl -v https://example.com/video.mp4 -o /dev/null
# Time: ~200ms
Contrary to common assumption, cURL itself doesn't maintain any caching mechanism. The speed difference points to caching occurring at different layers:
- HTTP-level caching (via Cache-Control headers)
- DNS caching at OS level
- TCP connection reuse
- CDN edge caching (Akamai in this case)
For accurate benchmarking, disable all potential caching layers:
# Disable HTTP caching
curl -H "Cache-Control: no-cache" -H "Pragma: no-cache" https://example.com/video.mp4
# Force DNS refresh and new TCP connection
curl --resolve example.com:443: --connect-timeout 5 https://example.com/video.mp4
# Comprehensive non-cached request
curl -v \
-H "Cache-Control: no-cache, no-store, must-revalidate" \
-H "Pragma: no-cache" \
-H "Expires: 0" \
--dns-servers 8.8.8.8 \
--connect-timeout 5 \
When cURL isn't sufficient, consider these specialized tools:
# wrk - modern HTTP benchmarking
wrk -t12 -c400 -d30s https://example.com/video.mp4
# siege - configurable load testing
siege -c 100 -t 1M -b https://example.com/video.mp4
# vegeta - versatile load tester
echo "GET https://example.com/video.mp4" | vegeta attack -duration=30s -rate=100 | vegeta report
When testing Akamai or other CDNs, additional headers may be needed:
# Bypass Akamai cache
curl -H "Pragma: akamai-x-get-cache-key" -H "Pragma: akamai-x-cache-on" https://example.com/video.mp4
# Cloudflare cache bypass
curl -H "CF-Cache-Status: BYPASS" https://example.com/video.mp4
Remember that many CDNs have their own cache control mechanisms that may override standard HTTP headers in their edge configurations.