Chapter 5: hiredis — The C Redis Client Library Internals

What hiredis Is

hiredis is the official C client library for Redis. Both db-proxy and db-mux use it internally. It handles:

Async Mode (What PCG Uses)

// Create async connection
redisAsyncContext *ctx = redisAsyncConnect("127.0.0.1", 6379);

// Register event loop adapter (hooks hiredis into EVL)
redisAsyncSetConnectCallback(ctx, on_connect);
redisAsyncSetDisconnectCallback(ctx, on_disconnect);

// Send a command (NON-BLOCKING)
redisAsyncCommand(ctx, reply_callback, privdata, "SET %s %b", key, val, val_len);
// This does NOT send over network immediately!
// It formats the RESP command and appends to ctx->obuf (output buffer)
// Then calls the "addWrite" hook to signal "I have data to send"

The Output Buffer (obuf)

redisAsyncCommand("SET k1 v1") → obuf: "*3\r\n$3\r\nSET\r\n$2\r\nk1\r\n$2\r\nv1\r\n" redisAsyncCommand("SET k2 v2") → obuf: "...(k1 cmd)...*3\r\n$3\r\nSET\r\n$2\r\nk2\r\n$2\r\nv2\r\n" redisAsyncCommand("SET k3 v3") → obuf: "...(k1+k2 cmds)...(k3 cmd)..." ↓ (when POLLOUT fires) write(fd, obuf, obuf_len) → ALL commands sent in ONE syscall

The addWrite/delWrite Callbacks

hiredis doesn't know about your event loop. It uses callbacks to signal when it needs to write:

// hiredis calls this when it has data to send:
void addWrite(void *privdata) {
    // "I have data in obuf, please enable POLLOUT for my fd"
    evl_fd_add_events(fd, POLLOUT);
}

// hiredis calls this when obuf is empty:
void delWrite(void *privdata) {
    // "obuf is empty, no more data to send, disable POLLOUT"
    evl_fd_del_events(fd, POLLOUT);
}
💡 The Control Point for Time-Based Batching

The addWrite callback is where you control WHEN the flush happens. If you DELAY calling evl_fd_add_events(POLLOUT), the data stays in obuf longer, accumulating more commands before the flush. This is the "time-based batching" idea from today's meeting — suppress POLLOUT briefly to batch more.

redisAsyncFormattedCommand (What db-mux Uses)

// Normal path (db-proxy uses this internally):
redisAsyncCommandArgv(ctx, cb, priv, argc, argv, argvlen);
// hiredis formats RESP internally (allocates SDS string, snprintf, etc.)
// = 2-3 heap allocations per command

// Pre-formatted path (db-mux uses this):
char *cmd = preformatted_resp_buffer;  // already in RESP format
redisAsyncFormattedCommand(ctx, cb, priv, cmd, cmd_len);
// hiredis just appends to obuf — NO formatting, NO allocations
// = 0 heap allocations per command

The Hidden API (From Today's Meeting)

Patrik mentioned a "hidden API" in hiredis to push to the output buffer without triggering a flush. This likely refers to directly appending to ctx->obuf (or using internal functions) while suppressing the addWrite callback — giving the application full control over when to flush.

Summary: Data Flow Through hiredis

Application calls redisAsyncCommand() │ ▼ hiredis formats RESP (or uses pre-formatted for db-mux) │ ▼ Appends to ctx->obuf (output buffer) │ ▼ Calls addWrite callback → evl_fd_add_events(POLLOUT) │ ▼ ... EVL continues processing other work ... │ ▼ poll() returns POLLOUT for this fd │ ▼ EVL calls write handler → hiredis flushes obuf │ ▼ write(fd, obuf, len) → TCP segment(s) sent to Redis │ ▼ Redis processes commands, sends replies │ ▼ poll() returns POLLIN → hiredis reads replies │ ▼ hiredis calls reply_callback for each command
Key Takeaways