When working with systemd's journal on Ubuntu 16.04, you might encounter situations where older logs become inaccessible despite services running continuously. The default journald configuration maintains logs in a binary format at /var/log/journal/, but several factors affect log retention:

# Check current journal disk usage
journalctl --disk-usage

# View current journal retention settings
grep -i 'SystemMaxUse\|SystemKeepFree' /etc/systemd/journald.conf

The behavior you're observing occurs because:

• Default journal retention is based on both size (10% of filesystem) and time (older entries purged when space is needed)
• Docker generates verbose output that quickly fills the journal
• Ubuntu 16.04's default configuration doesn't prioritize long-term log retention

To view historical logs that might still exist on disk:

# List available journal files
ls -lh /var/log/journal/$(cat /etc/machine-id)/

# Force journalctl to include rotated files
journalctl -u docker.service --since "2016-10-13" --merge --all

Modify /etc/systemd/journald.conf to prevent premature log rotation:

# Increase maximum journal size
SystemMaxUse=1G
# Keep logs for longer duration
MaxRetentionSec=1month
# Store more files when rotating
SystemMaxFiles=10

After modifying, reload journald:

sudo systemctl restart systemd-journald

Configure Docker to use json-file logging with rotation:

# Edit docker daemon.json
{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}

Then restart Docker:

sudo systemctl restart docker

If logs were rotated out, you might find them in archived system logs:

# Check traditional syslog rotation
sudo zgrep docker /var/log/syslog*

---

When working with journalctl on Ubuntu 16.04, you'll notice the default configuration only keeps recent logs in active storage. The system displays:

-- Logs begin at [recent_date], end at [current_date] --

This occurs because journald implements log rotation by default, moving older entries to archived state rather than maintaining them in active memory.

First verify your current settings with:

cat /etc/systemd/journald.conf | grep -v '^#' | grep -v '^$'

Key parameters affecting log retention:

• SystemMaxUse=: Maximum disk space for active journals
• SystemKeepFree=: Space to leave free on disk
• SystemMaxFileSize=: Maximum size of individual journal files
• SystemMaxFiles=: Maximum number of journal files to keep

To view older rotated logs that aren't appearing in default queries, use these techniques:

# View all available log files including rotated ones
journalctl --list-boots

# Access specific rotated log by boot ID
journalctl -b [boot_ID] -u docker.service

# Query across all stored logs (including archived)
journalctl -u docker.service --merge --since "2016-10-13" --until "2016-10-14"

For persistent log retention, modify /etc/systemd/journald.conf:

[Journal]
Storage=persistent
SystemMaxUse=1G
SystemMaxFiles=10
SystemMaxFileSize=100M

Then apply changes with:

sudo systemctl restart systemd-journald

For container-specific logs, bypass journald entirely by configuring Docker's logging driver:

# Configure Docker to use json-file logging with size limits
cat <<EOF | sudo tee /etc/docker/daemon.json
{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}
EOF

# Restart Docker
sudo systemctl restart docker

After making changes, verify you can access historical logs with:

# Check available log time range
journalctl --header | grep -E '^(Start|End)'

# Test query for older logs
journalctl -u docker.service -S "2016-10-13 18:00:00" -U "2016-10-13 22:00:00"