Chapter 10: db-proxy — How It Works and Why It's Slow

db-proxy Write Path

// Application calls db-proxy:
db_proxy_set(key, value, callback);

// Inside db-proxy:
// 1. Serialize db_key_t into string_t (HEAP ALLOCATION #1)
string_t *key_str = string_create(key);

// 2. Build argv[]/argvlen[] arrays
// 3. Call redisAsyncCommandArgv (HEAP ALLOCATION #2-3 inside hiredis)
//    hiredis calls redisFormatSdsCommandArgv:
//      - allocates SDS string
//      - snprintf for each length prefix
//      - appends to obuf (potential realloc)

// 4. Creates an evl_defer to process the write
evl_defer_start(process_write_cb);

// Total: 2-3 heap allocations per command
// At 300K/sec: 600K-900K extra allocations/sec vs db-mux

Why It's CPU-Expensive

• Heap allocations: malloc/free are expensive at high rates (cache misses, fragmentation)
• RESP formatting inside hiredis: snprintf for each field length
• Defer per write: EVL overhead for each command, callback limit can break batching
• Transaction objects: each write creates its own tracking object

Performance Numbers (from Patrik's benchmarks)

// At 300K writes/sec:
// db-proxy async: 79.9% application CPU, 76.0% Valkey CPU
// db-proxy sync:  69.6% application CPU, 53.1% Valkey CPU
// db-mux:         28.9% application CPU, 53.9% Valkey CPU
//
// db-mux is 2.5-2.8× more CPU efficient!

db-proxy async: Lower Latency, Higher CPU

db-proxy async has lower RTT (298µs vs 1015µs at 100K) because it interleaves send/receive via defers. But this costs 2.5× more CPU. For PCG where CPU is the bottleneck (not latency), db-mux wins.

Mark as Complete