When running multiple applications (like microservices) on a single EC2 instance behind an Elastic Load Balancer, the default health check mechanism presents limitations. AWS ELB only allows configuring one health check endpoint per target group, while we need visibility into each application's status.
Here are three proven approaches to solve this:
1. Health Check Aggregator Endpoint
Create a dedicated health endpoint that programmatically checks all applications:
// Node.js example using Express
const express = require('express');
const axios = require('axios');
const app = express();
const SERVICES = [
{ name: 'auth', url: 'http://localhost:3001/health' },
{ name: 'payment', url: 'http://localhost:3002/health' }
];
app.get('/aggregated-health', async (req, res) => {
const results = await Promise.all(
SERVICES.map(async service => {
try {
const response = await axios.get(service.url);
return { [service.name]: response.status === 200 };
} catch {
return { [service.name]: false };
}
})
);
const allHealthy = results.every(r => Object.values(r)[0]);
res.status(allHealthy ? 200 : 503).json({ services: results });
});
2. Custom CloudWatch Metrics with Lambda
Deploy a Lambda function that:
- Polls all application health endpoints
- Pushes custom metrics to CloudWatch
- Triggers Auto Scaling actions based on composite health
3. Service Mesh Integration
For advanced implementations, consider using AWS App Mesh:
# Example App Mesh virtual node health check config
virtualNodes:
- name: auth-service
listeners:
- healthCheck:
protocol: http
path: /health
healthyThreshold: 2
unhealthyThreshold: 2
timeoutMillis: 5000
intervalMillis: 10000
- Health check endpoints should require minimal dependencies
- Implement circuit breakers to prevent cascading failures
- Set appropriate timeouts (shorter than ELB's timeout)
- Include version information in health responses
Use this CloudFormation snippet to create alarms for multi-service health:
Resources:
MultiServiceAlarm:
Type: AWS::CloudWatch::Alarm
Properties:
AlarmDescription: "Composite health check alarm"
Metrics:
- Id: "m1"
Expression: "IF(m1 AND m2, 1, 0)"
Label: "CompositeHealth"
- Id: "m1"
MetricStat:
Metric:
Namespace: "Custom"
MetricName: "AuthServiceHealth"
Period: 60
Stat: "Minimum"
- Id: "m2"
MetricStat:
Metric:
Namespace: "Custom"
MetricName: "PaymentServiceHealth"
Period: 60
Stat: "Minimum"
ComparisonOperator: "LessThanThreshold"
Threshold: 1
EvaluationPeriods: 2
When running multiple applications on a single EC2 instance behind an Elastic Load Balancer (ELB), you'll quickly encounter a limitation: ELB health checks can only monitor one endpoint per target group. This becomes problematic when different applications have different health requirements or when you need granular visibility into each application's status.
Here are three practical approaches to implement comprehensive health monitoring:
1. Custom Health Check Endpoint
Create a dedicated health check endpoint that aggregates status from all applications:
from flask import Flask, jsonify
import requests
app = Flask(__name__)
@app.route('/health')
def health_check():
status = {
'app1': check_app1(),
'app2': check_app2(),
'overall': True
}
# If any app fails, mark overall as unhealthy
status['overall'] = all(status.values())
return jsonify(status), 200 if status['overall'] else 503
def check_app1():
try:
response = requests.get('http://localhost:8001/health', timeout=2)
return response.status_code == 200
except:
return False
def check_app2():
try:
response = requests.get('http://localhost:8002/health', timeout=2)
return response.json().get('status') == 'OK'
except:
return False
2. Application Load Balancer with Multiple Target Groups
For ALB (not classic ELB), you can create:
- Separate target groups for each application
- Different health check paths for each target group
- Routing rules based on path or host header
3. Custom Lambda Health Check
Implement a Lambda function that performs comprehensive checks and updates instance health via API:
const AWS = require('aws-sdk');
const elb = new AWS.ELBv2();
exports.handler = async (event) => {
const instanceId = event.instanceId;
const checks = {
app1: await checkApp1(),
app2: await checkApp2()
};
const isHealthy = checks.app1 && checks.app2;
await elb.setTargetHealth({
TargetGroupArn: process.env.TARGET_GROUP_ARN,
Target: { Id: instanceId },
HealthStatus: isHealthy ? 'healthy' : 'unhealthy'
}).promise();
return { status: isHealthy ? 'healthy' : 'unhealthy', checks };
};
- Timeout Handling: Set conservative timeouts for individual application checks
- Circuit Breakers: Implement fail-fast mechanisms when downstream services are unavailable
- Logging: Log detailed health check failures for troubleshooting
- Security: Protect health endpoints with proper authentication
Complement your solution with CloudWatch alarms that track:
aws cloudwatch put-metric-alarm \
--alarm-name "App1-Unhealthy-Hosts" \
--metric-name "UnHealthyHostCount" \
--namespace "AWS/ApplicationELB" \
--dimensions Name=TargetGroup,Value=target-group-arn \
--statistic "Average" \
--period 60 \
--evaluation-periods 1 \
--threshold 0 \
--comparison-operator "GreaterThanThreshold"