Chapter 14: Blob Storage, Object Stores, and File Systems

Large binary data (images, videos, backups, logs) doesn't belong in a database. Use object storage.

Object Storage (S3-like)

// Store: PUT object with a key
PUT /bucket/images/user123/avatar.jpg → [binary data]

// Retrieve: GET by key
GET /bucket/images/user123/avatar.jpg → [binary data]

// Properties:
// - Flat namespace (no directories, just key prefixes)
// - Immutable (overwrite = new version)
// - Virtually unlimited storage
// - 99.999999999% durability (11 nines)
// - Cheap ($0.023/GB/month for S3 standard)

Pattern: Metadata in DB, Blob in Object Store

Upload flow: 1. Client → App: "upload photo" 2. App → S3: store blob, get URL 3. App → PostgreSQL: store metadata {id, user_id, s3_url, size, created_at} 4. App → Client: "done, URL is ..." Download flow: 1. Client → App: "get photo 123" 2. App → PostgreSQL: lookup metadata → s3_url 3. Client → CDN/S3: fetch blob directly (or pre-signed URL)

Pre-Signed URLs

Let clients upload/download directly to S3 without going through your server:

// Server generates a time-limited signed URL
url = s3.generate_presigned_url('put_object', bucket='uploads', key='photo.jpg', expires=300)
// Client uploads directly to S3 using this URL
// Your server never handles the large file!
Key Takeaways
← PreviousNext →