When facing a DDoS situation, the first critical step is proper diagnosis. The symptoms described - bandwidth saturation (100Mbps utilization), failed SSH access, and server load spikes - indicate two distinct but related attacks:
# Sample command to identify traffic patterns
netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n
The output will show suspicious IP addresses making excessive connections. For the crashing service issue, this likely represents a targeted application-layer attack.
For network-level attacks (bandwidth saturation):
# Basic iptables rate limiting (adjust thresholds as needed)
iptables -A INPUT -p tcp --dport 80 -m connlimit --connlimit-above 50 -j DROP
iptables -A INPUT -p tcp --dport 80 -m limit --limit 50/minute --limit-burst 200 -j ACCEPT
For application-layer attacks (high server load):
# Nginx rate limiting configuration
http {
limit_req_zone $binary_remote_addr zone=one:10m rate=10r/s;
server {
location / {
limit_req zone=one burst=20 nodelay;
# Your normal config
}
}
}
For attacks exceeding your network capacity, consider:
- Cloudflare's DDoS protection (free tier available)
- AWS Shield for AWS-hosted services
- Google Cloud Armor for GCP deployments
# Example Terraform for AWS Shield Advanced
resource "aws_shield_protection" "example" {
name = "example-protection"
resource_arn = aws_lb.example.arn
}
For the crashing service scenario (likely a vulnerability exploit):
# Automated IP blocking with fail2ban
[sshd]
enabled = true
port = ssh
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
bantime = 3600
Implement real-time monitoring to detect future attacks:
# Sample Prometheus alert rule
groups:
- name: ddos.rules
rules:
- alert: HighRequestRate
expr: rate(nginx_http_requests_total[1m]) > 100
for: 5m
labels:
severity: critical
annotations:
summary: "Potential DDoS attack (instance {{ $labels.instance }})"
For mission-critical services, consider:
- Anycast network distribution
- Autoscaling with load balancers
- Edge caching (Varnish, Fastly)
# Varnish VCL snippet for attack mitigation
sub vcl_recv {
if (req.http.User-Agent ~ "(bot|spider|scraper)") {
return(synth(403, "Forbidden"));
}
if (std.ip(req.http.X-Real-IP, "0.0.0.0") ~ attackers) {
return(synth(403, "Forbidden"));
}
}
Remember that DDoS protection requires multiple layers of defense. Start with immediate mitigations, then implement more sophisticated solutions as part of your long-term infrastructure strategy.
When facing sudden traffic spikes that consume all available bandwidth (e.g., 100Mbps saturation) or cause server load averages to skyrocket from 0.25 to 20+, you're likely experiencing either:
- Volumetric DDoS: Flooding your network layer (SYN floods, UDP amplification)
- Application Layer DDoS: HTTP GET/POST floods targeting specific vulnerabilities
# Emergency iptables rules (Linux)
iptables -A INPUT -p tcp --dport 80 -m connlimit --connlimit-above 50 -j DROP
iptables -A INPUT -p tcp --dport 22 -s ! YOUR_TRUSTED_IP -j DROP
iptables -A INPUT -p tcp --syn -m limit --limit 1/s --limit-burst 3 -j ACCEPT
For AWS users:
# AWS WAF rate-based rules (CloudFormation snippet)
AWSTemplateFormatVersion: '2010-09-09'
Resources:
MyRateBasedRule:
Type: AWS::WAFv2::WebACL
Properties:
Name: DDoS-Protection-ACL
Scope: REGIONAL
DefaultAction:
Allow: {}
Rules:
- Name: RateLimitRule
Priority: 1
Action:
Block: {}
Statement:
RateBasedStatement:
Limit: 2000
AggregateKeyType: IP
VisibilityConfig:
SampledRequestsEnabled: true
CloudWatchMetricsEnabled: true
The case where a specific IP crashes your service repeatedly suggests:
- An unhandled exception in your code triggered by malicious input
- Resource exhaustion (e.g., memory leak when processing certain requests)
// Node.js express middleware example
const rateLimit = require('express-rate-limit');
const apiLimiter = rateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 100, // limit each IP
handler: function(req, res) {
res.status(429).json({
error: "Too many requests"
});
}
});
app.use("/api/", apiLimiter);
Implement real-time alerts using tools like:
- Prometheus + Grafana for metrics visualization
- Fail2ban for automated IP blocking
- Cloudflare's DDoS protection (free tier available)