Chapter 2: poll(), epoll, POLLIN/POLLOUT — The I/O Multiplexing Layer

What poll() Does

poll() is a syscall that asks the kernel: "which of these file descriptors are ready for I/O?"

struct pollfd {
    int   fd;       // the socket to watch
    short events;   // what we're interested in (POLLIN, POLLOUT)
    short revents;  // what actually happened (filled by kernel)
};

struct pollfd fds[3];
fds[0] = {.fd = redis_fd,  .events = POLLIN | POLLOUT};
fds[1] = {.fd = pfcp_fd,   .events = POLLIN};
fds[2] = {.fd = http_fd,   .events = POLLIN};

int ready = poll(fds, 3, 1000);  // block up to 1000ms

// After poll() returns:
if (fds[0].revents & POLLIN)   // Redis has data to read
if (fds[0].revents & POLLOUT)  // Redis socket ready for writing

POLLIN vs POLLOUT

The POLLOUT Dance (Critical for Batching)

POLLOUT is almost always ready (socket buffer is rarely full). If you always register for POLLOUT, poll() returns immediately every time → busy loop → 100% CPU. So the pattern is:

// 1. Application wants to send data to Redis
redisAsyncCommand(ctx, cb, "SET key val");
// This appends to hiredis output buffer but does NOT write() yet

// 2. hiredis calls the "addWrite" callback → EVL registers POLLOUT
evl_fd_add_events(redis_fd, POLLOUT);
// "Hey EVL, next time you poll(), include POLLOUT for this fd"

// 3. EVL's poll() returns with POLLOUT set for redis_fd
// 4. EVL calls the write handler → hiredis flushes output buffer
write(redis_fd, obuf, obuf_len);  // actual TCP send happens HERE

// 5. If buffer is now empty, UNREGISTER POLLOUT
evl_fd_del_events(redis_fd, POLLOUT);
// "Nothing more to write, don't wake me for POLLOUT anymore"

epoll vs poll

poll() scans ALL fds every time. epoll is O(1) — only returns ready fds. PCG uses poll internally via EVL but the concept is identical:

// epoll (Linux-specific, more efficient for many fds)
int epfd = epoll_create1(0);
epoll_ctl(epfd, EPOLL_CTL_ADD, redis_fd, &event);  // register once

struct epoll_event events[64];
int n = epoll_wait(epfd, events, 64, 1000);  // only returns READY fds
// n = number of ready fds (not total fds like poll)

EVL's poll_mgr_process()

In PCG's EVL, the poll happens in evl_process_poll() which runs LAST in each iteration. This is from up-common/src/evl/src/evl.c:

The "Callback Limit" Problem

EVL has a limit on how many callbacks it processes per iteration (to prevent starvation). If db-proxy creates a separate defer per write, and the callback limit is hit, some writes spill to the NEXT iteration → NEXT poll() → NEXT TCP write. This means db-proxy can produce MULTIPLE TCP writes for what should be one batch.