How to Effectively Ban Hardware Interrupts on Specific CPUs Using IRQBALANCE_BANNED_CPUS in Ubuntu


2 views

The IRQBALANCE_BANNED_CPUS parameter allows system administrators to exclude specific CPUs from handling hardware interrupts. This is particularly useful for:

  • Performance-sensitive applications needing dedicated cores
  • Real-time systems requiring predictable interrupt handling
  • Workload isolation in multi-core environments

To configure this in Ubuntu, edit the irqbalance configuration file:

sudo nano /etc/default/irqbalance

Add or modify these parameters:

IRQBALANCE_BANNED_CPUS=3e  # Hexadecimal mask for CPUs 1-5
ENABLED="1"                # Keep irqbalance running

The mask uses hexadecimal notation where each bit represents a CPU:

Binary: 00111110 (CPUs 1-5 excluded)
Hex:    0x3e
Decimal: 62

To exclude CPUs 2,3,4,5 specifically (leaving 0 and 1 active):

Binary: 00111100
Hex:    0x3c

After applying changes, check interrupt distribution:

watch -n 1 'cat /proc/interrupts | head -20'

Common issues and solutions:

  • Persistent interrupts on banned CPUs: Some interrupts may be pinned to specific CPUs via /proc/irq/[irq_num]/smp_affinity
  • Configuration ignored: Ensure you've restarted the service: sudo systemctl restart irqbalance

For complete control, manually set CPU affinity:

# List all IRQs and their current affinity
grep -H . /proc/irq/*/smp_affinity

# Set specific IRQ to CPU 0 only
echo 1 | sudo tee /proc/irq/22/smp_affinity

When banning multiple CPUs from interrupt handling:

  • Monitor the target CPU's load (mpstat -P ALL 1)
  • Check for increased interrupt latency (perf stat -e irq_vectors:local_timer_entry)
  • Balance between isolation and available processing power

The IRQBALANCE_BANNED_CPUS parameter in Ubuntu's irqbalance service allows you to specify which CPUs should not receive hardware interrupts. This is particularly useful for:

  • CPU isolation for real-time applications
  • Performance tuning on multi-core systems
  • Reducing latency for specific workloads

To configure banned CPUs:

  1. Edit the irqbalance configuration file:
    sudo nano /etc/default/irqbalance
  2. Add or modify the IRQBALANCE_BANNED_CPUS parameter. The value is a hexadecimal bitmask where each bit represents a CPU:
    # Example: Ban CPUs 2,3,4,5 (binary 00111110 = 0x3E)
    IRQBALANCE_BANNED_CPUS=3e
  3. Restart the service:
    sudo systemctl restart irqbalance

Check if the configuration is active:

cat /proc/interrupts

You should see interrupt counts increasing primarily on unbanned CPUs. However, note that:

  • Some interrupts (like local timer interrupts) may still appear on banned CPUs
  • The kernel may route certain critical interrupts to all CPUs regardless of settings

If interrupts still appear on banned CPUs:

  1. Check the actual CPU mask being used:
    ps aux | grep irqbalance

    Look for the --ban parameter in the command line.

  2. For complete isolation, consider disabling irqbalance entirely:
    sudo systemctl disable --now irqbalance

    Then manually assign interrupts using:

    echo "mask" > /proc/irq/[IRQ_NUMBER]/smp_affinity

To calculate the hexadecimal mask for specific CPUs:

# Python one-liner to calculate mask
python3 -c "print(hex(sum(1 << cpu for cpu in [2,3,4,5])))"

This outputs 0x3e for CPUs 2-5.

For changes to persist across reboots, create a systemd drop-in file:

sudo mkdir -p /etc/systemd/system/irqbalance.service.d
sudo nano /etc/systemd/system/irqbalance.service.d/override.conf

Add:

[Service]
Environment="IRQBALANCE_BANNED_CPUS=3e"

Then reload systemd:

sudo systemctl daemon-reload
sudo systemctl restart irqbalance