SRE Thailand Bootcamp

Week 4: Distributed Tracing — OpenTelemetry + Jaeger

Theme: Instrument distributed traces, understand context propagation, visualize with Jaeger.

Learning Objectives

1. Explain distributed tracing concepts (trace, span, span context, baggage)
2. Instrument Go and Python services with OpenTelemetry SDK
3. Configure context propagation (W3C Trace Context)
4. Visualize traces in Jaeger UI, identify performance bottlenecks

Reading Materials

• OpenTelemetry Overview
• OpenTelemetry Go SDK
• OpenTelemetry Python SDK
• W3C Trace Context
• Jaeger Documentation
• Google SRE Book - Chapter 14: Distributed Tracing
• OpenTelemetry Collector

Finger Exercises (choose 2)

1. Deploy Jaeger all-in-one to kind. Verify the UI.
2. Add OpenTelemetry instrumentation to the Go app: create spans for each handler, add attributes (endpoint, method).
3. Set up the OpenTelemetry Collector to receive traces and forward to Jaeger.

Project

• Make the Go HTTP server call the Python worker over HTTP for each /api/orders request.
• Instrument both services with OpenTelemetry.
• Propagate trace context via HTTP headers (W3C Trace Context).
• Deploy Jaeger, collect all traces.
• Create a Jaeger dashboard showing:
  • Service graph
  • Trace search by service, operation, tags
  • P99 latency per trace
• Find a trace where the Python worker is the bottleneck and confirm by looking at span durations.

Time Budget