procps — Collection Overview
procps is the /proc reader’s toolkit: ps, top, free, vmstat,
sysctl and friends all render kernel state into human form. Every
production incident review starts with one of these binaries, and
interviews use them as a proxy for “have you actually debugged a live
system.” This collection covers all 17 binaries one page each; the hub
carries the /proc mapping ideas they share.
The /proc Contract
Everything procps shows comes from /proc — but from different files
with different semantics:
| Binary | Primary sources |
|---|---|
| ps, pgrep, pkill | /proc/<pid>/stat, status, cmdline |
| top | the above plus /proc/stat, /proc/meminfo, delta sampling |
| free | /proc/meminfo |
| vmstat | /proc/stat, /proc/meminfo, /proc/vmstat |
| uptime, w | /proc/loadavg, /proc/uptime, utmp |
| sysctl | /proc/sys/** (the tunables tree itself) |
| pmap | /proc/<pid>/maps, smaps |
| slabtop | /proc/slabinfo |
| pwdx | /proc/<pid>/cwd readlink |
Two habits transfer everywhere: values like %CPU are deltas between
samples, not absolutes (why top’s first %CPU reading is odd), and
field meanings shift across kernel versions, so tools label rather than
assume. The free-vs-available story — “available” estimates reclaimable
cache, “free” is truly idle — is the single most-asked interview question
in this collection and lives on the free page.
Inventory
Observe processes (6)
| Binary | One-liner | Page |
|---|---|---|
ps | Snapshot process list (flagship) | ps |
| top | Live process view (flagship) | top |
pgrep | Find PIDs by criteria (flagship) | pgrep |
pkill | Signal PIDs by criteria | pkill |
pidof | PIDs by exact program name | pidof |
pidwait | Wait for matching processes | pidwait |
Observe memory and kernel (5)
| Binary | One-liner | Page |
|---|---|---|
free | Memory summary | free |
pmap | Per-process memory map | pmap |
slabtop | Kernel slab usage live | slabtop |
vmstat | Virtual memory statistics | vmstat |
pwdx | Working directory of PIDs | pwdx |
Session, load, and control (6)
| Binary | One-liner | Page |
|---|---|---|
uptime | Load triple + uptime | uptime |
w | Who is on and doing what | w |
watch | Repeat a command, diff output | watch |
sysctl | Read/write kernel tunables | sysctl |
snice | Renice by criteria | snice |
tload | ASCII load graph | tload |
Shared Concepts
- POSIX vs BSD syntax duality —
ps -efvsps auxis one interface choice with deep history; ps is the canonical explanation and every other tool picks a side. - Selection grammar reuse —
pgrep/pkill/pidwait/sniceshare one matching engine (name vs-ffull cmdline,-uuser,-Pparent,-xexact); learn it once, and know that-xstill means “match the pattern exactly”, not “literal string”. - Signals as control —
pkilldefaults to SIGTERM for a reason (graceful shutdown contract); the escalation ladder TERM → KILL and the exit-code contract are on pkill. - Sampling honesty —
top -bn1%CPU is since-boot-ish, firstvmstatline is since-boot by design; batch mode for scripts needs warm-up iterations. The top and vmstat pages quantify this. - Tunables are files —
sysctl -w vm.swappiness=10is a write to/proc/sys/vm/swappiness; persistence lives in/etc/sysctl.d/. The name→path mapping rule is on sysctl.
Reading Order For Interview Prep
- ps and top — the flagship pair.
- free + vmstat — memory truth-telling.
- pgrep/pkill — safe process targeting.
- sysctl — tuning and persistence.
- The rest for breadth: pmap, slabtop, watch, w, uptime.
Related: Process Management (admin track) covers cgroups/systemd context; kill in the util-linux collection covers the signal sender itself; load-average semantics appear on both uptime and top.
Interview Questions
Q: free shows 200 MB free but 30 GB available. Is the box out of memory?
No — the opposite. “free” is memory the kernel had nothing to put in;
“available” estimates what can be reclaimed from page cache and other
reclaimable structures without hurting performance. Linux treats unused
RAM as wasted RAM, so healthy boxes have small free and large available.
The free page maps every field to /proc/meminfo.
Q: A process sits in D state and load average climbs. What does it mean and why does kill -9 not work?
D is uninterruptible sleep — the task is blocked in a syscall the kernel cannot safely abort, classically NFS or other storage I/O. Signals are delivered only when the task returns to user mode, so even SIGKILL waits. Load average includes D-state tasks on Linux, which is why the load climbs without CPU usage — the uptime and ps pages decode both halves.
Q: Why do people say “never pkill -9 by name”?
Two reasons stack: broad name patterns match more than you think
(unanchored -f matches command lines, including your own monitoring
wrapper), and SIGKILL skips the process’s cleanup contract — sockets
without FIN waits, lock files, data not fsynced. The professional
pattern: pgrep first to see targets, SIGTERM, wait, then escalate.
pkill has the full ladder and the -c/-e guardrails.
Q: What is steal time and where do you see it?
st in top’s %Cpu(s) line and vmstat’s st column: the fraction of
time the hypervisor ran other tenants while this vCPU wanted to run.
Sustained steal on a cloud VM means noisy-neighbor or undersizing —
a conversation about capacity, not the kernel. Both the
top and vmstat pages decode their full
column sets.