Chapter 8: Data-Plane Threading — Control, Spare, Core Loop
DP Thread Model
┌─────────────────────────────────────────────────────────┐
│ data-plane process │
│ │
│ ┌──────────────────────┐ ┌──────────────────────────┐│
│ │ Control Thread │ │ Spare Thread ││
│ │ │ │ ││
│ │ • PFCP session logic │ │ • DB I/O (db-proxy runs ││
│ │ • State machines │ │ here currently) ││
│ │ • Rule installation │ │ • Background tasks ││
│ │ • Signaling (HTTP) │ │ • Replication client ││
│ │ • Has its own EVL │ │ • Has its own EVL ││
│ └──────────────────────┘ └──────────────────────────┘│
│ │
│ ┌──────────────────────────────────────────────────────┐│
│ │ Core Loop Threads (fast path) ││
│ │ • Packet forwarding (GTP-U encap/decap) ││
│ │ • DPI, firewall, NAT ││
│ │ • No DB access, no EVL (tight poll loop) ││
│ │ • One per CPU core ││
│ └──────────────────────────────────────────────────────┘│
└─────────────────────────────────────────────────────────┘
Why This Matters for the Feature
- db-proxy currently runs on the spare thread — DB I/O happens there
- db-mux PoC was tested on control thread — different thread = unfair comparison
- Follow-up from meeting: run db-mux on spare thread for apples-to-apples comparison
- CPU metrics: "DP control CPU" and "DP spare CPU" are measured separately
Communication Between Threads
// Control → Spare: "please write this to DB"
// Uses EVL defer or mbox to cross thread boundary
evl_defer_start(spare_evl, db_write_callback, data);
// Spare → Control: "DB write complete"
evl_defer_start(control_evl, completion_callback, result);⚠️ Thread Boundary = Latency
Every cross-thread communication adds latency (wakeup + context switch). If db-mux runs on the control thread directly (no thread crossing), it could be faster for latency but uses control thread CPU. Trade-off: latency vs CPU distribution.
Key Takeaways
- DP has control thread (logic) + spare thread (DB I/O) + core loop threads (packets)
- db-proxy runs on spare thread currently
- db-mux PoC needs to be tested on spare thread for fair comparison
- Cross-thread communication adds latency (defer/wakeup)
- "DP control CPU" and "DP spare CPU" are separate metrics in test results