Chapter 16: Signaling Path — TCP/TLS Corking for HTTP/2
The Problem
With TCP_NODELAY, each SSL_write produces a separate TLS record → separate TCP segment. nghttp2 sends HTTP/2 frames one at a time. A single HTTP/2 response produces 3-5 frames → 3-5 TCP segments.
The Cork Solution
Without cork:
nghttp2 frame 1 → SSL_write → TLS record 1 → write() → TCP segment 1
nghttp2 frame 2 → SSL_write → TLS record 2 → write() → TCP segment 2
nghttp2 frame 3 → SSL_write → TLS record 3 → write() → TCP segment 3
= 3 syscalls, 3 TLS records (87 bytes overhead), 3 TCP segments
With TLS cork:
nghttp2 frame 1 → tls_write → buffered
nghttp2 frame 2 → tls_write → buffered
nghttp2 frame 3 → tls_write → buffered
uncork → SSL_write(all) → 1 TLS record → write() → 1 TCP segment
= 1 syscall, 1 TLS record (29 bytes overhead), 1 TCP segment
Implementation Layers
// The cork wraps the write callback at two levels:
tcp_on_write:
cork_tcp(); // hold kernel writes
tls_on_write:
cork_tls(); // buffer plaintext
http_session_on_write:
nghttp2_session_send:
on_send(frame1) → tls_write → buffered
on_send(frame2) → tls_write → buffered
on_send(frameN) → tls_write → buffered
uncork_tls(); // one SSL_write → one TLS record
uncork_tcp(); // one write() → one TCP segment
Prototype Gerrit Changes
evlsock: Implement cork using TCP_CORKevlsock: Implement cork using userspace bufferingTLS: Buffer plaintext when corked