Chapter 23: Distributed Transactions — Saga, 2PC, Outbox

In microservices, a single business operation may span multiple services/databases. How do you maintain consistency?

The Saga Pattern

A sequence of local transactions. If one fails, execute compensating transactions to undo previous steps.

Order Saga: 1. Order Service: Create order (PENDING) 2. Payment Service: Charge card 3. Inventory Service: Reserve stock 4. Order Service: Confirm order (CONFIRMED) If step 3 fails: Compensate step 2: Refund card Compensate step 1: Cancel order

Choreography vs Orchestration

Style	How	Pros	Cons
Choreography	Services react to events (pub/sub)	Decoupled, simple	Hard to track, debug
Orchestration	Central coordinator directs steps	Clear flow, easy to monitor	Single point, coupling to orchestrator

Transactional Outbox Pattern

Problem: you need to update DB AND publish an event atomically. If app crashes between the two, they're inconsistent.

// Solution: write event to an "outbox" table in the SAME DB transaction
BEGIN;
  INSERT INTO orders (id, status) VALUES (1, 'created');
  INSERT INTO outbox (event_type, payload) VALUES ('OrderCreated', '{...}');
COMMIT;

// Separate process polls outbox table and publishes to Kafka
// Guarantees: DB update and event are always consistent

Idempotent Consumers

Since messages may be delivered more than once, consumers must handle duplicates:

// Store processed message IDs
if message_id already in processed_messages table:
    skip (already handled)
else:
    process message
    insert message_id into processed_messages

Key Takeaways

Avoid distributed transactions if possible (design services to not need them)
Saga: sequence of local transactions + compensating actions on failure
Outbox pattern: atomically update DB + publish event
Idempotent consumers: handle duplicate messages safely
Orchestration for complex flows, choreography for simple ones

← Previous Next →