Chapter 12: Batching at Every Layer
The Batching Stack
Layer 5: APPLICATION BATCHING (explicit)
"Combine N logical operations into 1 Redis command (MSET)"
Impact: fewer Redis commands to parse, fewer callbacks
Layer 4: REDIS PIPELINING (implicit)
"Send N commands without waiting for replies"
Impact: 1 RTT instead of N RTTs
Layer 3: EVL ITERATION BATCHING (automatic)
"All commands queued before poll() go in one write()"
Impact: 1 write() syscall instead of N
Layer 2: TCP BATCHING (TCP_CORK or userspace buffer)
"Multiple write() calls → one TCP segment"
Impact: fewer TCP segments, less network overhead
Layer 1: TLS BATCHING (TLS cork)
"Multiple SSL_write → one TLS record → one TCP segment"
Impact: fewer TLS records (each has 29 bytes overhead)
What PCG Currently Has
| Layer | DB Path (Redis) | Signaling Path (HTTP/2) |
|---|---|---|
| Application | ❌ Individual commands | ❌ Individual frames |
| Pipelining | ✅ Fire-and-forget parallel | N/A (HTTP/2 multiplexing) |
| EVL iteration | ✅ (but broken by db-proxy defers) | ✅ |
| TCP | ❌ TCP_NODELAY, no cork | ❌ TCP_NODELAY, no cork |
| TLS | N/A (Redis not TLS in cluster) | ❌ Per-frame SSL_write |
What the Feature Adds
| Layer | DB Path Change | Signaling Path Change |
|---|---|---|
| Application | ✅ MSET (batch N keys into 1 cmd) | — |
| EVL iteration | ✅ Fixed by db-mux (no defers) | — |
| TCP | — | ✅ TCP_CORK prototype |
| TLS | — | ✅ TLS cork prototype |
Time-Based Batching (Proposed)
// Current: enable POLLOUT immediately when command queued
// → flush on next poll() (within same EVL iteration)
// Proposed: delay POLLOUT by a few microseconds
// → accumulate commands from MULTIPLE EVL iterations
// → flush when timer fires OR batch is full
// Trade-off: +latency (waiting) vs +throughput (bigger batches)
// Similar to Nagle's algorithm but at application levelKey Takeaways
- Batching happens at 5 layers — each reduces overhead differently
- Feature targets: application layer (MSET) + EVL layer (db-mux) + TCP/TLS (cork)
- Time-based batching: trade latency for throughput (proposed, not yet decided)
- Each layer compounds: MSET + db-mux + EVL batching = maximum efficiency