SRE Thailand Bootcamp

Week 18: PostgreSQL & Redis in Production

Theme: Operate stateful databases on Kubernetes: backup, HA, migrations.

Learning Objectives

1. Deploy PostgreSQL on Kubernetes using CloudNativePG or Zalando operator
2. Configure streaming replication for HA
3. Perform zero-downtime schema migrations
4. Deploy Redis Sentinel or Redis Cluster for high availability

Reading Materials

• CloudNativePG Documentation
• Zalando PostgreSQL Operator
• PostgreSQL Streaming Replication
• golang-migrate
• Redis Sentinel
• Redis Cluster
• Redis on Kubernetes Best Practices

Finger Exercises (choose 2)

1. Deploy a PostgreSQL cluster with CloudNativePG: 1 primary + 1 replica. Connect and run a simple schema.
2. Run a schema migration using golang-migrate: add a users table, then add an index. Roll back.
3. Deploy Redis with Sentinel (3 nodes). Test failover by killing the primary.

Project

• Deploy PostgreSQL with CloudNativePG:
  • 1 primary + 1 synchronous replica + 1 asynchronous replica
  • Automated backups to S3 every 6 hours
  • Point-in-time recovery enabled
  • Connection pooling with PgBouncer (sidecar or standalone)
• Integrate with the Go app:
  • Replace in-memory storage with PostgreSQL
  • Use golang-migrate for schema migrations
  • Use connection pooling (max 20 connections per app instance)
• Deploy Redis with Sentinel:
  • 3 Sentinel nodes + 1 primary + 2 replicas
  • Used for session caching and rate limiting
• Implement backup/restore procedures:
  • Full backup every 6h, WAL archiving continuous
  • Restore test: delete the database, restore from backup, verify data integrity
• Document: DATABASE.md with connection pooling config, backup schedule, failover procedure

Time Budget