Chapter 34: Observability — Tracing, Metrics, Logging
gRPC + OpenTelemetry
OpenTelemetry provides interceptors that automatically instrument gRPC calls:
- Traces: span per RPC, propagated across services
- Metrics: RPC count, latency histogram, error rate per method
- Logs: structured logs with trace context
Key Metrics to Monitor
| Metric | What It Tells You |
|---|---|
| grpc_server_handled_total | Total RPCs by method + status code |
| grpc_server_handling_seconds | Latency histogram per method |
| grpc_server_msg_received_total | Messages received (streaming) |
| grpc_client_handled_total | Client-side RPC outcomes |
Distributed Tracing
Client → Service A → Service B → Service C
[span 1] [span 2] [span 3] [span 4]
└──────────────────────────────────────────┘
trace-id: abc123
Trace context propagated via gRPC metadata:
traceparent: 00-abc123-span2-01