Many developers use the tail -f command to monitor log files in real-time, but encounter issues when log rotation occurs. The fundamental problem is that tail maintains a file descriptor to the original log file, while log rotation typically involves:

• Renaming the current log file
• Creating a new empty file with the original name
• Optionally compressing or archiving the old logs

This process breaks the file handle that tail was monitoring, causing it to stop tracking new entries.

Modern versions of tail (GNU coreutils) include the --follow=name option which handles rotation:

tail --follow=name /var/log/application.log

For systems using BSD tail (like macOS), the equivalent is:

tail -F /var/log/application.log

The capital -F flag combines -f (follow) with rotation detection.

For more sophisticated log monitoring, consider multitail:

multitail -e "error" /var/log/application.log

Key features include:

• Color highlighting
• Regular expression filtering
• Multiple log viewing in split windows
• Built-in rotation handling

For production environments, consider these robust alternatives:

1. Using inotifywait

while true; do
  inotifywait -e modify,create,move /var/log/
  tail -n 100 /var/log/application.log
done

2. Logrotate Configuration

Ensure your /etc/logrotate.d/application includes:

/var/log/application.log {
  copytruncate
  daily
  rotate 7
  compress
  missingok
}

The copytruncate method copies then truncates the original file instead of moving it, which maintains existing file handles.

Python Implementation

import time
import os

def follow(filename):
    with open(filename) as f:
        f.seek(0, os.SEEK_END)
        while True:
            line = f.readline()
            if not line:
                time.sleep(0.1)
                continue
            yield line

for line in follow('/var/log/application.log'):
    print(line, end='')

Node.js Implementation

const fs = require('fs');
const filename = '/var/log/application.log';

let fd = fs.openSync(filename, 'r');
let pos = 0;

function checkForRotation() {
    fs.stat(filename, (err, stats) => {
        if (stats.ino !== fs.fstatSync(fd).ino) {
            fs.closeSync(fd);
            fd = fs.openSync(filename, 'r');
            pos = 0;
        }
    });
}

setInterval(checkForRotation, 1000);

const stream = fs.createReadStream(filename, {
    fd: fd,
    start: pos,
    autoClose: false
});

stream.on('data', (data) => {
    pos += data.length;
    process.stdout.write(data.toString());
});

• Always use tail -F instead of tail -f for critical monitoring
• Consider implementing log rotation detection in your custom monitoring scripts
• For high-volume logs, use tools like logstash or fluentd
• Set up proper alerts for log rotation events that might indicate issues

---

When working with production systems, we frequently need to monitor log files in real-time using tail -f. However, the standard approach breaks when log rotation occurs. The fundamental issue lies in how Unix handles file descriptors:

# Basic tail command that won't survive rotation
tail -f /var/log/application.log

During rotation, three key events occur:

1. The original log file is renamed (e.g., application.log → application.log.1)
2. A new empty file is created with the original name
3. The running tail process maintains its handle to the renamed file

Option 1: Using tail with --follow=name

The most reliable native solution uses the --follow=name option (or -F shorthand):

tail -F /var/log/application.log

Key differences from -f:

• Tracks the filename rather than file descriptor
• Reopens the file when it becomes inaccessible
• Handles both rotation and recreation scenarios

Option 2: Combining with watch for Extra Resilience

For environments with aggressive log rotation:

watch -n 30 "tail -n 50 /var/log/application.log"

For monitoring multiple logs with rotation support:

# Using GNU tail with multiple files
tail -F /var/log/{app,server,error}.log

# Alternative with persistent monitoring
multitail /var/log/application.log

On modern Linux systems, consider using journalctl:

journalctl -f -u application.service

Benefits include:

• Built-in log rotation handling
• Structured logging
• Advanced filtering capabilities

For environments where logs might disappear temporarily:

while true; do
  if [ -f /var/log/application.log ]; then
    tail -n 0 -f /var/log/application.log
  fi
  sleep 1
done

This approach adds resilience but consumes more resources.

The -F option does incur slight overhead due to periodic stat calls. For high-volume logging systems:

• Consider increasing inotify limits
• Monitor system resource usage
• Evaluate specialized log shippers like fluentd or logstash