Interrupts
Overview
An interrupt is a signal from hardware or software that causes the CPU to temporarily suspend its current activity and execute a special routine called an interrupt handler (or Interrupt Service Routine, ISR). Interrupts are the fundamental mechanism that allows the OS to respond to asynchronous events — hardware completion, errors, timers, and inter-processor signals.
Motivation
Without interrupts, the CPU would have to poll (busy-wait) on every device, wasting cycles checking if work is done. Interrupts allow the CPU to do useful work and be notified only when a device needs attention.
Polling (wasteful):
CPU: "Are you done?" → Device: "No"
CPU: "Are you done?" → Device: "No"
CPU: "Are you done?" → Device: "No"
CPU: "Are you done?" → Device: "Yes!"
// CPU wasted all those check cycles
Interrupt-driven (efficient):
CPU: "Start this I/O" → Device: "OK"
CPU: [does other work]
Device: "HEY! I'm done!" → CPU: [handles result]
// CPU only involved when needed
Types of Interrupts
┌──────────────────────────────────────────────────────────────┐
│ Interrupt Taxonomy │
├──────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────┐ ┌──────────────────┐ ┌─────────────┐ │
│ │ Hardware │ │ Software │ │ Exceptions │ │
│ │ (External/Async) │ │ (Internal/Sync) │ │ (Traps) │ │
│ ├─────────────────┤ ├──────────────────┤ ├─────────────┤ │
│ │ • Device I/O │ │ • System calls │ │ • Page fault│ │
│ │ completion │ │ (int 0x80 / │ │ • Div by 0 │ │
│ │ • Timer tick │ │ syscall) │ │ • Segfault │ │
│ │ • Keyboard │ │ • Breakpoint │ │ • GPF │ │
│ │ • Network packet│ │ (int 3) │ │ • Overflow │ │
│ │ • Disk ready │ │ │ │ │ │
│ │ • Power failure │ │ │ │ │ │
│ └─────────────────┘ └──────────────────┘ └─────────────┘ │
│ Asynchronous Synchronous Synchronous │
│ (can occur at (caused by (caused by │
│ any time) instruction) instruction) │
└──────────────────────────────────────────────────────────────┘
Hardware Interrupts
Generated by external hardware devices via interrupt request (IRQ) lines.
┌──────────┐
│ Keyboard │───IRQ 1───┐
├──────────┤ │
│ Timer │───IRQ 0───┤
├──────────┤ │ ┌──────────────┐
│ Disk │───IRQ 14──┼───►│ Interrupt │───► CPU
├──────────┤ │ │ Controller │
│ NIC │───IRQ 11──┤ │ (PIC/APIC) │
├──────────┤ │ └──────────────┘
│ USB │───IRQ 9───┘
└──────────┘
Interrupt Controllers
PIC (Programmable Interrupt Controller) — Legacy (8259A):
- 8 IRQ lines per PIC, two cascaded = 15 usable IRQs
- Fixed priority (IRQ 0 highest)
- Used in single-processor systems
APIC (Advanced PIC) — Modern:
- Local APIC: one per CPU core
- I/O APIC: one per system, routes device interrupts
- Supports 256+ interrupt vectors
- Can direct interrupts to specific CPU cores
- Used in SMP/multi-core systems
┌─────────────────────────────────────────────────┐
│ Modern APIC System │
│ │
│ ┌─────────┐ ┌─────────┐ ┌─────────┐ │
│ │ CPU 0 │ │ CPU 1 │ │ CPU 2 │ │
│ │ Local │ │ Local │ │ Local │ │
│ │ APIC │ │ APIC │ │ APIC │ │
│ └────┬────┘ └────┬────┘ └────┬────┘ │
│ │ │ │ │
│ └────────────┼────────────┘ │
│ │ │
│ ┌─────┴─────┐ │
│ │ I/O APIC │ │
│ └─────┬─────┘ │
│ │ │
│ ┌───────────┼───────────┐ │
│ │ │ │ │
│ ┌────┴───┐ ┌────┴───┐ ┌────┴───┐ │
│ │ NIC │ │ Disk │ │ USB │ │
│ └────────┘ └────────┘ └────────┘ │
└─────────────────────────────────────────────────┘
Software Interrupts
Generated by software instructions:
// x86: int instruction triggers software interrupt
int 0x80; // Linux system call (32-bit)
// x86-64: syscall instruction (faster)
syscall; // Linux system call (64-bit)
// Breakpoint (used by debuggers)
int 3; // Debug breakpoint
Exceptions (Traps/Faults)
Synchronous events caused by CPU executing an instruction:
| Exception | Type | Cause |
|---|---|---|
| Page Fault (#PF) | Fault | Accessing unmapped memory |
| General Protection (#GP) | Fault | Privilege violation |
| Division by Zero (#DE) | Fault | Integer divide by 0 |
| Breakpoint (#BP) | Trap | int 3 instruction |
| Overflow (#OF) | Trap | INTO instruction with OF set |
| Invalid Opcode (#UD) | Fault | Unknown instruction |
Interrupt Handling Mechanism
┌──────────────────────────────────────────────────────────────┐
│ Interrupt Handling Flow │
│ │
│ 1. Device raises interrupt │
│ │ │
│ 2. Interrupt controller (APIC) routes to CPU │
│ │ │
│ 3. CPU finishes current instruction │
│ │ │
│ 4. CPU saves state: │
│ • Pushes flags, CS, IP onto stack │
│ • Disables further interrupts (CLI on x86) │
│ │ │
│ 5. CPU looks up handler in IDT (Interrupt Descriptor Table) │
│ │ │
│ 6. Jumps to interrupt handler (ISR) │
│ │ │
│ 7. Handler: │
│ • Saves remaining registers │
│ • Acknowledges interrupt (EOI to APIC) │
│ • Does minimal work (top half) │
│ • Schedules deferred work (bottom half) │
│ • Restores registers │
│ │ │
│ 8. IRET instruction: │
│ • Pops IP, CS, flags from stack │
│ • Resumes interrupted code │
└──────────────────────────────────────────────────────────────┘
Interrupt Descriptor Table (IDT)
┌───────┬──────────────────┬───────────┬───────────┐
│ Vector│ Handler Address │ Privilege │ Type │
├───────┼──────────────────┼───────────┼───────────┤
│ 0 │ divide_error │ Ring 0 │ Fault │
│ 3 │ debug │ Ring 0 │ Trap │
│ 13 │ general_protection│ Ring 0 │ Fault │
│ 14 │ page_fault │ Ring 0 │ Fault │
│ 32 │ timer_interrupt │ Ring 0 │ Interrupt │
│ 33 │ keyboard_interrupt│ Ring 0 │ Interrupt │
│ 128 │ system_call │ Ring 3 │ Trap │
│ ... │ ... │ ... │ ... │
│ 255 │ (last vector) │ ... │ ... │
└───────┴──────────────────┴───────────┴───────────┘
IDT register (IDTR) points to this table.
CPU uses interrupt vector number as index into IDT.
Top-Half / Bottom-Half Processing
Linux splits interrupt handling into two parts for efficiency:
┌─────────────────────────────────────────────────────────┐
│ Top Half / Bottom Half Split │
│ │
│ ┌─────────────────────────┐ │
│ │ TOP HALF │ │
│ │ (runs in interrupt │ │
│ │ context, interrupts │ │
│ │ disabled on this IRQ) │ │
│ │ │ │
│ │ • Acknowledge interrupt │ │
│ │ • Read data from device │ │
│ │ • Schedule bottom half │ │
│ │ • Return quickly │ │
│ │ │ │
│ │ ⚠ Must be fast! │ │
│ └───────────┬──────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────┐ │
│ │ BOTTOM HALF │ │
│ │ (runs in softirq/ │ │
│ │ tasklet/workqueue │ │
│ │ context, interrupts │ │
│ │ enabled) │ │
│ │ │ │
│ │ • Process received data │ │
│ │ • Wake up processes │ │
│ │ • Update statistics │ │
│ │ • Can sleep/block │ │
│ │ │ │
│ │ ✓ Can take more time │ │
│ └──────────────────────────┘ │
└─────────────────────────────────────────────────────────┘
Bottom-half mechanisms in Linux:
| Mechanism | Context | Can Sleep? | Use Case |
|---|---|---|---|
| Softirqs | Interrupt | No | Network, block I/O (high-frequency) |
| Tasklets | Softirq | No | Driver deferred work |
| Workqueues | Process | Yes | Slow work that may block |
| Threaded IRQs | Kernel thread | Yes | Modern drivers, recommended |
# View softirq statistics
cat /proc/softirqs
# CPU0 CPU1
# HI: 12345 11234
# TIMER: 98765 97654
# NET_TX: 45678 44567
# NET_RX: 87654 86543
# BLOCK: 23456 22345
# TASKLET: 5678 4567
# SCHED: 34567 33456
# RCU: 67890 66789
# View interrupt counts
cat /proc/interrupts
# CPU0 CPU1
# 0: 48 0 IO-APIC 2-edge timer
# 1: 0 2345 IO-APIC 1-edge i8042
# 9: 0 12345 IO-APIC 9-fasteoi acpi
# 16: 45678 56789 IO-APIC 16-fasteoi ehci_hcd
# NMI: 0 0 Non-maskable interrupts
Interrupt Latency
┌────────────────────────────────────────────────────────┐
│ Interrupt Latency Breakdown │
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Hardware │ │ CPU │ │ Kernel │ │ Handler │ │
│ │ Latency │ │ Response │ │ Dispatch │ │ Execution│ │
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘ │
│ │ │ │ │ │
│ Signal Finish Save Handle │
│ propagation instruction state, interrupt │
│ (ns) (ns-µs) IDT lookup (µs-ms) │
│ (ns-µs) │
│ │
│ Total: typically 1-10 µs on modern hardware │
└────────────────────────────────────────────────────────┘
Factors affecting interrupt latency:
- Interrupt controller: APIC is faster than PIC
- CPU frequency: Faster CPU = faster dispatch
- Interrupt nesting: If interrupts are disabled (e.g., in another ISR), latency increases
- Cache state: Cold cache = slower handler execution
- Power state: CPU may be in deep sleep (C-state), adding wake-up latency
MSI/MSI-X (Message Signaled Interrupts)
Modern PCIe devices use MSI instead of legacy IRQ lines:
Legacy IRQ:
Device ──── physical wire ────► IRQ pin on PIC/APIC
Limited to 15 IRQs, shared interrupts
MSI:
Device ──── memory write to APIC address ────► APIC
Up to 2048 vectors per device, no sharing
MSI-X:
Device ──── memory writes (one per vector) ────► APIC
Up to 2048 vectors, can target different CPU cores
Each vector can be independently masked
# View MSI/MSI-X status
lspci -v | grep -i msi
# Capabilities: [100] MSI: Enable+ Count=1/1 Maskable- 64bit+
# Capabilities: [180] MSI-X: Enable+ Count=32 Masked-
# NIC with multiple MSI-X vectors for multi-queue
# Each queue gets its own interrupt → can be pinned to different CPUs
Real-World Linux Examples
Interrupt Affinity
# Pin interrupt to specific CPU core
# Useful for network-intensive workloads
# View current affinity
cat /proc/irq/33/smp_affinity # Hex bitmask
# Set NIC interrupt to CPU 2
echo 4 | sudo tee /proc/irq/33/smp_affinity # 4 = 0b100 = CPU 2
# Use irqbalance daemon for automatic balancing
sudo systemctl status irqbalance
Disabling Interrupts
// Kernel code can disable interrupts
local_irq_disable(); // Disable interrupts on current CPU
// ... critical section ...
local_irq_enable(); // Re-enable
// Save and restore flags (including interrupt state)
unsigned long flags;
local_irq_save(flags);
// ... critical section ...
local_irq_restore(flags);
// ⚠ Keep disabled sections as SHORT as possible!
// Disabling interrupts increases latency for all devices
/proc Filesystem
# Detailed interrupt information
cat /proc/interrupts
# Shows per-CPU interrupt counts for each IRQ line
# Softirq statistics
cat /proc/softirqs
# IRQ statistics
ls /proc/irq/
# 0/ 1/ 2/ 3/ 4/ 5/ 6/ 7/ 8/ 9/ 10/ 11/ ...
cat /proc/irq/0/actions # What handler is registered for IRQ 0
Interview Questions
Beginner
Q: What is an interrupt? A: An interrupt is a signal that causes the CPU to pause its current execution and run a special handler routine. It allows hardware devices to notify the CPU that they need attention (e.g., I/O completion, data arrival) without the CPU having to constantly poll (check) each device.
Q: What is the difference between a hardware interrupt and a software interrupt?
A: A hardware interrupt is generated by an external device (keyboard, disk, NIC) asynchronously — it can occur at any time. A software interrupt is triggered by a CPU instruction (like int 0x80 for system calls) synchronously — it happens as part of program execution.
Intermediate
Q: Explain the top-half/bottom-half split in Linux interrupt handling. Why is it needed? A: The top half runs in interrupt context with interrupts disabled on the current CPU. It must be fast — it only acknowledges the interrupt, reads essential data, and schedules the bottom half. The bottom half runs later (as softirq, tasklet, or workqueue) with interrupts enabled, doing the heavier processing.
This split is needed because:
- Keeping interrupts disabled too long causes missed interrupts and high latency
- Other devices can’t be serviced while interrupts are disabled
- Some work (like waking processes) requires sleeping, which can’t happen in interrupt context
Q: What is interrupt affinity and why would you change it? A: Interrupt affinity controls which CPU core handles a specific interrupt. You’d change it to:
- Pin network interrupts to specific cores for cache locality
- Distribute interrupts across cores to avoid overloading one CPU
- Isolate interrupts from application cores for real-time workloads
FAANG-Level
Q: Design the interrupt handling path for a 10Gbps NIC receiving small packets (64 bytes) at line rate (14.88 Mpps). What challenges arise and how does Linux handle them?
A:
Challenge: At 14.88 Mpps, one interrupt per packet = 14.88M interrupts/second
On a 3GHz CPU, that's one interrupt every 200 cycles — too expensive!
Solution: NAPI (New API) — interrupt-driven → polling hybrid
Boot:
Packet arrives → NIC DMA to ring buffer → MSI-X interrupt
NAPI Flow:
1. First packet: interrupt fires → ISR runs
2. ISR: disable further interrupts from this NIC queue
3. ISR: schedule NAPI poll (softirq)
4. NAPI poll: process up to 64 packets from ring buffer
5. If ring buffer empty: re-enable interrupts, stop polling
6. If more packets: stay in polling mode (no interrupts)
┌──────────────────────────────────────────────┐
│ NAPI State Machine │
│ │
│ ┌──────────┐ interrupt ┌──────────┐ │
│ │ IRQ │─────────────►│ POLLING │ │
│ │ Mode │◄─────────────│ Mode │ │
│ └──────────┘ ring empty └──────────┘ │
│ (low load) (high load) │
│ Interrupts ON Interrupts OFF │
│ 1 pkt/interrupt Batch processing│
└──────────────────────────────────────────────┘
Multi-queue with RSS (Receive Side Scaling):
- NIC has multiple RX queues (e.g., 16)
- Each queue has its own MSI-X interrupt
- Each interrupt pinned to different CPU core
- Packet steering: hash(src_ip, dst_ip, src_port, dst_port) → queue
┌──────────────────────────────────────────────┐
│ NIC with 4 RX queues │
│ ┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐ │
│ │ Q0 │ │ Q1 │ │ Q2 │ │ Q3 │ │
│ └──┬──┘ └──┬──┘ └──┬──┘ └──┬──┘ │
│ │ │ │ │ │
│ IRQ 100 IRQ 101 IRQ 102 IRQ 103 │
│ │ │ │ │ │
│ CPU 0 CPU 1 CPU 2 CPU 3 │
│ (pinned affinity) │
└──────────────────────────────────────────────┘
Performance optimizations:
- Busy polling (SO_BUSY_POLL): application polls instead of sleeping
- XDP (eXpress Data Path): process packets before network stack
- io_uring: zero-copy packet reception
- Huge pages for ring buffers: reduce TLB misses
Q: What happens if an interrupt handler takes too long? How does the Linux kernel protect against this?
A:
Problems with long-running interrupt handlers:
1. Interrupts disabled on current CPU → missed interrupts
2. Other devices can't be serviced → high latency
3. Softirq starvation → network drops, timer drift
Linux protections:
1. CONFIG_HARDIRQ_TIMEOUT (kernel config):
- Warns if a handler runs > 10ms (HZ=1000)
- Prints stack trace for debugging
- Doesn't kill, just warns
2. softirq watchdog:
- Warns if softirq runs > 2ms (jiffies)
- ksoftirqd thread takes over if softirqs flood
3. NAPI budget:
- Network poll limited to 64 packets per call
- Prevents one NIC from monopolizing CPU
4. IRQ threading:
- Modern: force_irqthreads kernel parameter
- Runs all handlers in kernel threads
- Thread priority: 50 (SCHED_FIFO)
- Can be preempted by higher-priority work
5. Bottom-half limits:
- __softirq_limit: if pending softirqs > limit, defer to ksoftirqd
- ksoftirqd runs at SCHED_NORMAL priority (can be preempted)
Detection tools:
- ftrace: trace irq handler duration
- perf: profile interrupt handlers
- /proc/softirqs: check for softirq storms
Common Mistakes
- Doing too much in the top half: Keep the ISR as short as possible. Defer heavy work to the bottom half.
- Sleeping in interrupt context: You cannot call
sleep(),mutex_lock(), orkmalloc(GFP_KERNEL)in an ISR. UseGFP_ATOMICfor allocations. - Not acknowledging the interrupt: If you don’t send EOI (End of Interrupt) to the APIC, the same interrupt will fire again immediately.
- Forgetting shared IRQs: Multiple devices can share an IRQ line. The handler must check if its device actually caused the interrupt.
- Ignoring interrupt storms: A misconfigured device can flood the CPU with interrupts. Use interrupt coalescing and NAPI.
Summary
| Concept | Key Point |
|---|---|
| Interrupt | Signal that pauses CPU to handle an event |
| Hardware interrupt | External device, asynchronous |
| Software interrupt | CPU instruction, synchronous (system calls) |
| Exception | CPU error condition (page fault, div by 0) |
| IDT | Table mapping vectors to handlers |
| APIC | Modern interrupt controller (multi-core) |
| Top half | Fast ISR, interrupts disabled |
| Bottom half | Deferred work, interrupts enabled |
| MSI/MSI-X | Memory-based interrupts for PCIe devices |
| NAPI | Interrupt → polling hybrid for high-speed networking |
Cross-References
- Hardware — I/O hardware and interrupt controllers
- DMA — DMA completion uses interrupts
- Device Drivers — How drivers register interrupt handlers
- Software Layers — Where interrupts fit in the I/O stack