Design Exercise: Chat System (WhatsApp-like)
Requirements
- 1-on-1 and group messaging
- Online/offline status
- Message delivery guarantees (at-least-once)
- 50M DAU, average 40 messages/day per user
Estimation
Messages: 50M × 40 = 2B messages/day ≈ 23K messages/sec
Storage: 2B × 100B avg = 200GB/day, 73TB/year
Connections: 50M concurrent WebSocket connections (peak)Architecture
Client ──WebSocket──► Chat Servers (stateful, hold connections)
│
▼
Message Queue (Kafka)
│
┌───────────┼───────────┐
▼ ▼ ▼
Message DB Push Notif Presence
(Cassandra) Service Service
Key Design Decisions
- Protocol: WebSocket for real-time bidirectional
- Message routing: if recipient online → deliver via WebSocket. If offline → store + push notification
- Storage: Cassandra (write-heavy, partitioned by chat_id, sorted by timestamp)
- Message ordering: per-chat ordering via Kafka partition (partition key = chat_id)
- Presence: heartbeat-based, stored in Redis with TTL
- Group chat: fan-out on write (small groups) or fan-out on read (large groups)