How to Log and Analyze Historical CPU/Memory Usage Data in Windows


2 views

Windows provides built-in utilities for recording system performance metrics:

# PowerShell command to start performance counter logging
Get-Counter -Counter "\Processor(_Total)\% Processor Time", "\Memory\Available MBytes" -SampleInterval 5 -Continuous | Export-Counter -FileFormat CSV -Path "C:\perflogs\system_usage.csv"

The Performance Monitor tool offers detailed logging capabilities:

  1. Open perfmon.exe
  2. Navigate to Data Collector Sets → User Defined
  3. Right-click → New → Data Collector Set
  4. Select "Create manually" → "Performance counter"
  5. Add counters for CPU and memory
  6. Set sample interval (e.g., 15 seconds)

For more advanced analysis, consider these tools:

  • Process Explorer (Sysinternals): Enhanced Task Manager replacement
  • Prometheus + Grafana: For comprehensive monitoring and visualization
  • PRTG Network Monitor: Enterprise-grade monitoring solution

Here's a Python script using psutil to log system metrics:

import psutil
import time
import csv

with open('system_usage.csv', 'w', newline='') as f:
    writer = csv.writer(f)
    writer.writerow(['timestamp', 'cpu_percent', 'memory_used'])
    
    while True:
        timestamp = time.strftime('%Y-%m-%d %H:%M:%S')
        cpu = psutil.cpu_percent(interval=1)
        mem = psutil.virtual_memory().used / (1024*1024)  # MB
        
        writer.writerow([timestamp, cpu, mem])
        time.sleep(10)  # Log every 10 seconds

For analyzing collected data, you can use pandas:

import pandas as pd
import matplotlib.pyplot as plt

df = pd.read_csv('system_usage.csv')
df['timestamp'] = pd.to_datetime(df['timestamp'])

plt.figure(figsize=(12, 6))
plt.plot(df['timestamp'], df['cpu_percent'], label='CPU %')
plt.plot(df['timestamp'], df['memory_used'], label='Memory Used (MB)')
plt.legend()
plt.show()

As developers, we often need to track system resource usage over time to identify performance bottlenecks, memory leaks, or just understand our application's behavior. While Task Manager provides real-time data, it doesn't offer historical logging capabilities. Here are several professional approaches to capture and analyze CPU and memory usage history.

The most straightforward solution is using Windows' built-in Performance Monitor (perfmon):

1. Open Performance Monitor (Win+R → "perfmon")
2. Navigate to Data Collector Sets → User Defined
3. Right-click → New → Data Collector Set
4. Select "Create manually" → Add performance counters for:
   - Processor → % Processor Time
   - Memory → Available MBytes
   - Process → % Processor Time (select specific processes)
5. Set sample interval (e.g., every 15 seconds)
6. Specify output location and start logging

For more control and automation, PowerShell is ideal. Here's a script that logs CPU and memory usage to a CSV file:

# Save as MonitorResources.ps1
$outputFile = "C:\monitoring\resource_log.csv"
$duration = 24 # hours
$interval = 5 # seconds

"Timestamp,CPU(%),AvailableMemory(MB),ProcessCPU(%),ProcessMemory(MB)" | Out-File $outputFile

$endTime = (Get-Date).AddHours($duration)
while ((Get-Date) -lt $endTime) {
    $cpu = (Get-Counter '\Processor(_Total)\% Processor Time').CounterSamples.CookedValue
    $mem = (Get-Counter '\Memory\Available MBytes').CounterSamples.CookedValue
    $processStats = Get-Process your_process_name | Select-Object CPU,WS
    
    $line = "$(Get-Date -Format 'yyyy-MM-dd HH:mm:ss'),$cpu,$mem,$($processStats.CPU),$($processStats.WS/1MB)"
    $line | Out-File $outputFile -Append
    
    Start-Sleep -Seconds $interval
}

For more advanced needs, consider these tools:

  • Process Explorer (Sysinternals): Enhanced Task Manager with logging
  • Windows Performance Recorder (WPR): Lightweight ETW-based recording
  • Prometheus + Windows Exporter: For cloud-native monitoring
  • Grafana: Visualization of collected metrics

Once you have CSV logs, you can analyze them with Python:

import pandas as pd
import matplotlib.pyplot as plt

df = pd.read_csv('resource_log.csv', parse_dates=['Timestamp'])
df.set_index('Timestamp', inplace=True)

# Plot CPU and memory
fig, (ax1, ax2) = plt.subplots(2, 1, figsize=(12, 8))
df['CPU(%)'].plot(ax=ax1, title='CPU Usage')
df['AvailableMemory(MB)'].plot(ax=ax2, title='Available Memory')
plt.tight_layout()
plt.show()

For production systems, set up alerts when thresholds are exceeded:

# PowerShell snippet for alerting
$threshold = 90 # CPU%
$current = (Get-Counter '\Processor(_Total)\% Processor Time').CounterSamples.CookedValue
if ($current -gt $threshold) {
    Send-MailMessage -To "admin@example.com" -Subject "High CPU Alert" -Body "CPU at $current%"
}