Case study

Design Pastebin

Store and share text snippets via unique URLs with optional expiration and syntax highlighting.

blob storageTTLtext

Requirements

Functional: create paste (text + optional title, language, expiry); read by unique URL
Optional: private pastes (unlisted), burn-after-read, edit history, user accounts
Non-functional: create < 500ms; read < 200ms for 100 KB pastes
Durability: pastes persist until expiry; no silent data loss
Scale: 10M new pastes/month; avg 10 KB; 5:1 read/write

Size limits

Cap paste size (e.g. 512 KB free tier) to prevent abuse and bound storage costs. Reject oversized payloads at the edge.

Back-of-envelope Estimation

Metric	Calculation	Result
Write QPS	10M / (30 × 86,400)	~4/sec avg; ~20/sec peak
Read QPS	20 × 5	~100/sec peak
Storage/month	10M × 10 KB	~100 GB/month new data
Storage (1 yr, 50% expire)	100 GB × 12 × 0.5	~600 GB active
Bandwidth (reads)	100/sec × 10 KB	~1 MB/sec

API Design

Endpoint	Method	Description
POST /v1/pastes	POST	{ content, title?, language?, expiresIn?, visibility? } → { pasteId, url }
GET /v1/pastes/{id}	GET	Return metadata + content (or redirect to CDN URL for large blobs)
GET /raw/{id}	GET	Plain-text body for curl/wget
DELETE /v1/pastes/{id}	DELETE	Owner deletes early

Data Model

Entity	Fields	Storage
pastes	paste_id, user_id?, title, language, size, created_at, expires_at, visibility	Metadata in SQL
paste_content	paste_id → blob	S3/object store for > 4 KB; inline in DB for tiny pastes

Use paste_id as a random UUID or base62 token (8 chars). TTL index or scheduled job deletes expired metadata and blob objects.

High-level Design

Create & read flow

Create: Client ──► API ──► Object Store (content)
                         └──► SQL (metadata + pointer)

Read:   Client ──► CDN ──► API ──► Cache ──► Object Store / DB

Small pastes can live entirely in Redis with TTL matching expiry. Large pastes store content in S3; CDN caches immutable raw URLs keyed by paste_id + content hash.

Deep Dive: Expiration & Cleanup

On create, set Redis TTL = expires_at − now for hot pastes
Background sweeper (cron) queries SQL WHERE expires_at < NOW() in batches
Delete S3 objects asynchronously via deletion queue
Burn-after-read: decrement view_count in Redis; delete on first fetch atomically (Lua script)

Eventual cleanup

S3 delete is eventually consistent — orphaned objects are acceptable briefly; lifecycle rules catch stragglers.

Bottlenecks at Scale

Viral paste traffic — CDN offloads; origin only on cache miss
Spam/abuse — rate limit by IP; CAPTCHA on anonymous create
Syntax highlighting CPU — pre-render HTML at create time, serve static from CDN
Expiry sweeper lag — shard deletion jobs by paste_id range

Failure Modes & Monitoring

SLO	Target	Alert
Read latency p99	< 200ms	> 400ms for 5 min
Create success	99.9%	5xx rate > 0.1%
Expiry job lag	< 1 hour	Backlog > 100K pastes

Track storage growth rate, top languages, abuse blocks. Alert if object store egress spikes (scraping).

System Design