Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Section J — Networking Research (Topics 851–930)

Overview

This section covers cutting-edge networking research and advanced datacenter networking that frequently appears in senior/staff-level systems interviews at companies running large-scale infrastructure (Google, Meta, AWS, Microsoft, Netflix, Cloudflare). Topics range from programmable data planes (P4, SmartNICs) to emerging transport protocols and network verification.

Topic Map

graph TD
    A[Networking Advanced] --> B[Programmable Networks]
    A --> C[Advanced Congestion Control]
    A --> D[Modern Network Architecture]
    A --> E[Datacenter Topology]
    A --> F[Emerging Networks]
    
    B --> B1[P4 / Programmable Switches / ASICs]
    B --> B2[SmartNICs / DPDK / XDP]
    B --> B3[eBPF Networking Advanced / SR-IOV]
    
    C --> C1[BBR Internals / Copa / PCC]
    C --> C2[ECN / DCTCP / Incast / Bufferbloat]
    C --> C3[AQM: CoDel / PIE / Fair Queuing]
    C --> C4[Network Calculus]
    
    D --> D1[TSN / Deterministic Networking]
    D --> D2[Segment Routing / SRv6]
    D --> D3[INT / Telemetry / Tomography]
    D --> D4[SDN / P4Runtime / Verification]
    
    E --> E1[Clos / Fat-Tree / Leaf-Spine]
    E --> E2[Optical DC Networks / Photonic]
    
    F --> F1[Satellite / LEO / Edge / 5G / 6G]
    F --> F2[NFV / SFC / Container Networking]
    F --> F3[QUIC Internals / HTTP/3 / MASQUE]
    F --> F4[Encrypted Transport / DoH / DoT / ODoH]

Reading Order

#FilePrerequisitesFocus
1programmable-networks.mdeBPF basics, TCP/IPP4, SmartNICs, DPDK, XDP, SR-IOV
2congestion-control-advanced.mdTCP congestion control fundamentalsAdvanced CC algorithms, AQM, queuing
3modern-network-arch.mdSDN basics, routingTSN, SR, telemetry, verification
4datacenter-topology.mdSwitching basicsClos, fat-tree, optical DCNs
5emerging-networks.mdTCP, TLS, containersLEO, 5G/6G, NFV, QUIC, encrypted DNS

Complete Page Map

Every file in this directory, grouped by theme. Interviewers rarely ask “explain file X” — they ask the problem each file solves, so read the one-line focus as the question you should be able to answer after reading it.

Programmable data planes and fast I/O

FileOne-line focus
programmable-networks.mdHub page: P4, SmartNICs, DPDK, XDP, SR-IOV — when to move packet processing off the kernel
p4-programmable-dataplane.mdP4/PISA model: match-action pipelines, parse-match-execute, Tofino-class switches
cilium-ebpf.mdeBPF datapath for Kubernetes: XDP/TC hooks, sockmap, service mesh without sidecars
sr-iov-networking.mdSR-IOV virtual functions, VFIO passthrough, NIC sharing for NFV tenants
fdio-vpp.mdFD.io VPP vector (batch) packet processing — why batching amortizes per-packet cost
fast-failover.mdSub-second reconvergence: BFD, ECMP rehashing, SDN fast-failover groups

Congestion control, queueing, and pacing

FileOne-line focus
congestion-control-advanced.mdSurvey: BBR/Copa/PCC model-based CC, DCTCP, ECN, AQM family tree
quic-congestion-control.mdCC inside QUIC: per-stream signals, pacing, BBR over QUIC, congestion signal loss
rdma-congestion-control.mdRoCEv2: PFC pause frames, DCQCN (ECN+rate), IRN; why RDMA CC differs from TCP’s
learning-congestion-control.mdML-learned CC policies (PCC Vivace, Orca, Aurora) — reward functions, generalization
datacenter-tcp.mdDCTCP internals, incast collapse, fine-grained ECN thresholds, tuning
fair-queuing.mdScheduling theory: WFQ, DRR, guarantees and why perfect fairness costs O(log n)
fq-codel-pacing.mdFQ-CoDel + send pacing in Linux: killing bufferbloat and latency spikes
traffic-shaping.mdToken bucket vs leaky bucket, policers vs shapers, burst sizing math
tcp-outcast-problem.mdPort-level drop imbalance: last-arriving flow loses everything under drop-tail
fec-networking.mdForward error correction (Reed-Solomon, Raptor): trading bandwidth for loss recovery

Datacenter architecture

FileOne-line focus
datacenter-topology.mdClos/fat-tree math: bisection bandwidth, oversubscription ratios, cost scaling
datacenter-fabrics.mdSpine-leaf in practice: underlay/overlay routing, Equal-Cost Multi-Path design
datacenter-tcp.md(also above) transport tuning for microsecond RTTs and shallow buffers
vlan-stp.md802.1Q tagging, VLAN segmentation, STP/RSTP loop prevention and its convergence cost
mpls.mdLabel switching, LDP/RSVP-TE signaling, traffic engineering vs IP forwarding
l4-load-balancing-internals.mdMaglev-style consistent-hashing L4 LBs, DSR, state replication at millions of RPS
multicast.mdIGMP/PIM multicast: one-to-many replication for caches, market data, cluster sync

Programmable control planes and verification

FileOne-line focus
modern-network-arch.mdSurvey: TSN, segment routing, INT, SDN, formal verification — the modernization map
sdn-openflow.mdOpenFlow protocol, controller placement, flow-table limits, reactive vs proactive rules
srv6.mdSRv6 network programming: 128-bit SIDs, uSID compression, source routing in IPv6
network-verification.mdProving correctness: control-plane reachability (Batfish), data-plane header-space analysis
in-band-network-telemetry.mdINT: hop-by-hop metadata stamping in programmable pipelines, MTU/scale costs
in-band-telemetry.mdTelemetry deployment angle: INT vs streaming telemetry (gNMI), sampling vs every-packet

Deterministic networking and time

FileOne-line focus
tsn-deterministic-networking.mdIEEE 802.1 TSN: Qbv time-aware shapers, preemption, bounded-latency scheduling
tsn-time-sensitive-networking.mdTSN toolset: gPTP sync, credit-based shapers, frame replication (FRER) for loss-tolerance
time-synchronization.mdNTP vs PTP accuracy classes, hardware timestamping, why µs sync matters for trading/TSN
gnss-timing.mdGNSS time transfer, holdover oscillators, timing attacks and redundancy in datacenters

Overlays, transport extensions, emerging networks

FileOne-line focus
emerging-networks.mdHub: LEO satellites, 5G/6G, NFV/SFC, QUIC/HTTP-3, encrypted DNS
encrypted-dns.mdDoT/DoH/DoH3/ODoH: privacy gains, enterprise visibility loss, centralization debate
tcp-fast-open.mdTFO cookies: data in SYN saves 1 RTT; middlebox and idempotency pitfalls
stun-turn.mdNAT traversal: STUN discovery, TURN relay fallback, ICE candidate pairing
overlay-mesh-vpns.mdWireGuard-based mesh overlays, NAT hole-punching, identity-based access networks
geneve.mdGENEVE encapsulation: extensible tunnel metadata for overlays (used by OVN/Cilium)
diffserv-qos.mdDSCP classes, per-hop behaviors, marking/queuing policy across operator boundaries

Reading Paths by Goal

Datacenter track (backend/infra interviews at cloud companies): datacenter-topology.md → datacenter-fabrics.md → datacenter-tcp.md → congestion-control-advanced.md → rdma-congestion-control.md → l4-load-balancing-internals.md. This path answers the canonical chain: “why leaf-spine?”, “why does incast happen?”, “how do we carry storage traffic losslessly?”, “how does Google balance a front-end?”.

Low-latency track (trading, HFT, gaming, CDN edge): programmable-networks.md → sr-iov-networking.md → fdio-vpp.md → fq-codel-pacing.md → tcp-fast-open.md → time-synchronization.md → gnss-timing.md. This path follows the latency budget: bypass the kernel (DPDK/XDP), pin and isolate NIC queues (SR-IOV), batch packets (VPP), pace to avoid queueing, shave handshakes (TFO), and timestamp everything (PTP/GNSS).

Programmable track (network platforms, cloud networking teams): programmable-networks.md → p4-programmable-dataplane.md → sdn-openflow.md → srv6.md → in-band-network-telemetry.md → network-verification.md → cilium-ebpf.md. This path builds the argument arc interviewers want: what programmable ASICs enable, how control planes program them, how source routing simplifies tunnels, how telemetry observes them, and how verification replaces hope with proof.

Cross-Cutting Interview Themes

Most senior networking questions fuse two or three pages at once. Use this table to rehearse in connected chunks — each row is a real interview prompt and the files that answer it.

ThemePages to combineThe question it answers
Incast collapsedatacenter-tcp.md, tcp-outcast-problem.md, congestion-control-advanced.md“Many-to-one sync stalls at 90% — walk me through the last 10%.”
Storage over the networkrdma-congestion-control.md, fec-networking.md, datacenter-fabrics.md“How do you carry lossless-feeling traffic over Ethernet?”
Kernel-bypass decisionprogrammable-networks.md, sr-iov-networking.md, fdio-vpp.md, cilium-ebpf.md“Where should packets be processed, and what do you give up?”
Router of the futurep4-programmable-dataplane.md, sdn-openflow.md, network-verification.md“Design a switch feature that ships in weeks, not ASIC generations.”
Traffic engineering at scalesrv6.md, mpls.md, l4-load-balancing-internals.md“How do big operators steer flows without per-path state?”
Deterministic latencytsn-deterministic-networking.md, time-synchronization.md, fq-codel-pacing.md“How do you guarantee a deadline, not just a p99?”
Trust and privacy in transportencrypted-dns.md, overlay-mesh-vpns.md, stun-turn.md“Encrypt everything — what breaks operationally?”

Key Numbers to Memorize

Interview credibility in networking often comes down to quoting orders of magnitude without flinching. These recur across the section:

QuantityBallpark valueWhy it matters
Datacenter RTT (top-of-rack)~100–500 µsSets the pacing/timeout regime; 1000× smaller than WAN
10 Gbps × 1 ms BDP~1.25 MBThe buffer a switch needs to hold one RTT of one flow
DCTCP ECN threshold (K)~20 packetsShallow-marking regime that keeps queues ≈1 RTT deep
Kernel-stack forwarding ceiling~1–2 Mpps/coreThe number that justifies DPDK/XDP
DPDK polled forwardingtens of Mpps/coreThe payoff for giving up the socket API
RoCEv2 PFC riskHead-of-line blocking, deadlockWhy DCQCN (ECN+rate) or IRN is mandatory
SRv6 SID size128 bits (uSID ≈ 16–32 bit micro-SIDs)Header overhead driving compression work
PTP achievable syncsub-µs (vs ms-class NTP)What makes TSN and trading-grade timing possible

Interview Questions

  1. Why do datacenters deploy DCTCP instead of keeping loss-based TCP? Loss-based TCP needs deep buffers to avoid throughput collapse, and deep buffers add tens of milliseconds of queueing delay; they also make incast timeouts worse. DCTCP treats the ECN marking ratio as a signal of queue occupancy and scales the window down proportionally, keeping switch queues around one RTT deep while sustaining full throughput (RFC 8257). The trade-off is that every switch must mark ECN at a fine threshold, and mixed deployments with loss-based senders can starve. Interview follow-ups usually probe the marking threshold (K ≈ 20 packets at 10 Gbps) and incast behavior.

  2. What is the TCP outcast problem and how do you fix it? When a congested output port uses drop-tail, a flow whose packet happens to be enqueued last can lose every packet in a burst while others lose few — severe unfairness that shows up in incast and storage fan-in. Fixes replace tail-drop with stochastic or stateful drop: RED/ECN, fair-queuing (DRR/FQ-CoDel), or shapers that bound bursts. The general interview point is that drop policies, not TCP itself, determine per-flow fairness at the bottleneck.

  3. How does BBR differ from CUBIC, and what are BBRv1’s known weaknesses? CUBIC infers congestion from packet loss and fills buffers, while BBR explicitly estimates bottleneck bandwidth and minimum RTT and keeps roughly one BDP in flight, so it does not wait for loss to slow down. BBRv1’s weakness is that its bandwidth probe (periodic 1.25× up-ramp) can push standing queues onto loss-based flows and starve them in shallow-buffer competition; BBRv2/v3 add loss and ECN response to coexist with CUBIC. Expect follow-ups on probe intervals and why BBR loves shallow buffers but struggles with deep-buffered paths.

  4. When would you choose DPDK/XDP over the regular kernel network stack? Kernel-path forwarding peaks around 1–2 Mpps per core due to per-packet syscall, softirq, and allocation overhead, while DPDK’s user-space polling reaches tens of Mpps per core and XDP drops/redirects packets at the driver with eBPF for a middle ground. Choose DPDK for dedicated appliances (vSwitches, 5G user planes) where you own the whole port, XDP when you must coexist with normal sockets on the same NIC, and the kernel stack when throughput is not the bottleneck. The trade-offs you name — lost socket ecosystem, CPU pinning, memory management (hugepages), and onboarding cost — decide the answer.

  5. Compare MPLS and SRv6 for traffic engineering. MPLS forwards on local labels and needs a signaling plane (LDP, or RSVP-TE for explicit paths), while SRv6 encodes the path as a list of 128-bit IPv6 SIDs in a routing header, so the source node programs the end-to-end path with no per-path state in the core. SRv6 enables network programming (SID = instruction: VPN, firewall, slicing) but the 128-bit header overhead is heavy, hence the uSID/micro-SID compression deployed by large operators. If asked “why did operators move”, the answer is: fewer protocols, source-routed TE, and programmability aligned with IPv6.

  6. Your service’s p99 jumps every time a batch job runs on the same cluster. What networking knobs do you reach for? Start with queueing: batch jobs fill buffers, and filled buffers add queueing delay to interactive flows — the bufferbloat pattern, so separate the traffic with QoS classes (DSCP marking + per-class queues) or fair queuing/FQ-CoDel so one class cannot monopolize the port. Then check incast-style ECN/pacing for the batch fan-in, and confirm the fabric isn’t hashing both onto the same ECMP path. The complete answer names where each fix lives — end hosts (pacing, congestion control), switches (AQM, queues), topology (path diversity) — because a senior engineer owns all three.

Key Takeaways

  • Datacenter networking is about shallow queues + explicit signals: ECN/DCTCP and pacing beat deep buffers and loss-based reactions for both latency and throughput.
  • Topology and transport are coupled: leaf-spine gives predictable bisection bandwidth, but incast at any edge can still collapse a TCP workload without ECN or application pacing.
  • Programmable data planes (P4, DPDK, XDP, SmartNICs) trade generality for 10–100× packet-processing throughput; the decision is where each packet’s work runs, not “hardware vs software”.
  • RDMA/RoCE introduces a different congestion regime — PFC pauses can deadlock and block other traffic, so DCQCN-style ECN+rate control (or IRN) is mandatory knowledge for storage/ML clusters.
  • Segment Routing (MPLS or v6) removes per-path state from the network core; it is the default answer to “how do big operators do traffic engineering today?”
  • Telemetry evolved from SNMP polling to INT/streaming (gNMI): interviews expect you to know what you’d export per hop and the cost of doing it every packet.
  • Time is an infrastructure service: PTP/GNSS synchronization underpins TSN, trading, and even distributed databases (TrueTime-style commit waits).
  • Verification (Batfish, header-space analysis) turns “we think the config is right” into a proof — a strong differentiator answer in senior interviews.

References

Cross-References

  • TCP fundamentals: ../tcp/README.md, ../tcp/bbr.md, ../tcp/cubic.md, ../tcp/congestion-control.md
  • QUIC/HTTP3: ../http/quic.md, ../http/http3.md
  • eBPF: ../ebpf-networking.md, ../../os/kernel-advanced/ebpf-deep.md
  • Fast I/O: ../../os/advanced/fast-io.md (DPDK, io_uring)
  • Load balancing: ../load-balancing/README.md
  • Security: ../security/README.md
  • HPC networking: ../../hpc/mpi-parallelism.md (RDMA, InfiniBand, RoCE)
  • Linux kernel internals — where XDP/eBPF hooks and softirq processing live in the kernel
  • Kernel networking stack deep dive — the in-kernel packet path that DPDK/XDP bypass or short-circuit
  • Distributed systems overview — the consumers of datacenter network guarantees (partitions, latency tails)
  • HTTP/3 and QUIC page — the transport layer half of the QUIC topics introduced in emerging-networks.md