How to Monitor and Analyze Docker Container Restart History and Uptime Patterns


2 views

When working with Docker containers configured with restart policies like restart: unless-stopped or restart: always, it's crucial to monitor their stability. The standard docker ps output only shows current status, not historical restart events.

Docker doesn't maintain a comprehensive restart history log, but we can extract valuable information through:

# Check container restart count
docker inspect --format='{{.RestartCount}}' container_name

# Get last start time
docker inspect --format='{{.State.StartedAt}}' container_name

For systems using systemd, journalctl provides detailed container lifecycle events:

# View container start/stop events
journalctl -u docker --since "1 hour ago" | grep container_name

# Filter specifically for restart events
journalctl -u docker --grep="restarting\|started"

For production environments, consider these approaches:

#!/bin/bash
# Persistent restart counter using Docker labels
docker run -d --name my_container \
  --label restart.counter=0 \
  --restart unless-stopped \
  nginx:alpine

# Update counter after each restart
docker inspect --format='{{.State.Restarting}}' my_container && \
docker container update --label-add restart.counter=$(( $(docker inspect --format='{{index .Config.Labels "restart.counter"}}' my_container) + 1 )) my_container

For enterprise needs, consider:

  • Prometheus with cAdvisor exporter
  • Datadog Docker integration
  • New Relic infrastructure monitoring

When working with Docker containers in production environments, restart policies like unless-stopped or on-failure are commonly used. While docker ps shows the current status, tracking historical restarts requires deeper inspection.

The Docker engine maintains some restart information that can be accessed through:

docker inspect --format='{{.RestartCount}}' container_name_or_id

This returns the total number of times the container has been restarted, but doesn't provide timestamps or reasons.

For more comprehensive tracking, we need to examine Docker's logging systems:

# View container events (including restarts)
docker events --filter 'container=container_name' --since '2023-01-01'

# Alternative with formatted output
docker events --format '{{.Time}} {{.Action}} {{.Actor.Attributes.name}}' \
  --filter 'event=restart'

For production systems, consider these approaches:

# Method 1: Log all container events to file
docker events --format '{{json .}}' >> /var/log/docker-events.log

# Method 2: Use systemd journal (if Docker uses journald)
journalctl -u docker.service --since "1 hour ago" | grep -i restart

For teams needing visualization, here's a basic Prometheus setup:

# docker-compose.yml snippet for monitoring
services:
  cadvisor:
    image: gcr.io/cadvisor/cadvisor
    volumes:
      - /:/rootfs:ro
      - /var/run:/var/run:rw
    ports:
      - "8080:8080"

Popular options include:

  • Datadog Docker integration
  • New Relic Infrastructure
  • Grafana with Prometheus

When analyzing restarts, check these logs:

# Get the last 50 lines of container logs
docker logs --tail 50 container_name

# View exit codes which might indicate failure reasons
docker inspect --format='{{.State.ExitCode}}' container_name