Case study

Design a URL Shortener

Shorten long URLs into compact codes and redirect billions of clicks with sub-100ms latency.

hashingKV storeredirect

Requirements

Separate MVP from nice-to-haves early. Redirect latency dominates the user experience — reads must be fast and highly available.

Functional: create short URL from long URL; redirect short code → original URL
Optional: custom aliases, click analytics, expiration, password protection
Non-functional: redirect p99 < 100ms; 99.99% availability on reads
Consistency: eventual consistency acceptable for analytics; strong for mapping
Scale: 100M new URLs/month; 10:1 read/write ratio

Clarify redirect semantics

302 allows changing the destination and enables click tracking server-side. 301 is cached by browsers — use only when mappings are permanent.

Back-of-envelope Estimation

Metric	Calculation	Result
Write QPS (avg)	100M / (30 × 86,400)	~40/sec
Write QPS (peak)	5× average	~200/sec
Read QPS (peak)	200 × 10	~2,000/sec
Storage (5 yr)	100M/mo × 500 B × 60 mo	~3 TB
Bandwidth (reads)	2K/sec × 500 B	~1 MB/sec

A 7-character base62 key yields 62⁷ ≈ 3.5 trillion combinations — sufficient for decades at 100M/month. Reserve custom-alias namespace separately to avoid collisions with generated codes.

API Design

Endpoint	Method	Description
POST /v1/urls	POST	Body: { longUrl, customAlias?, expiresAt? } → { shortUrl, shortCode }
GET /{shortCode}	GET	302 redirect to long URL; increment analytics async
GET /v1/urls/{code}/stats	GET	Click count, referrers (authenticated owner)
DELETE /v1/urls/{code}	DELETE	Soft-delete mapping (owner only)

Idempotency

Accept Idempotency-Key on create to prevent duplicate short links when clients retry on timeout.

Data Model

Entity	Key fields	Notes
url_mappings	short_code (PK), long_url, user_id, created_at, expires_at	Primary lookup table; partition by short_code hash
users	user_id, email, api_key	Optional for authenticated creates
click_events	event_id, short_code, ts, referrer, geo	Append-only; stream to analytics warehouse

Store mappings in a wide-column store or sharded SQL keyed by short_code. Analytics can live in a separate column store (ClickHouse) fed by Kafka to avoid slowing redirects.

High-level Design

Redirect path

Client ──► CDN/LB ──► API Servers ──► Redis (hot codes)
                              │                    miss
                              └──────────────► Sharded DB (url_mappings)

Create: API ──► ID service (Snowflake) ──► base62 encode ──► DB + cache warm

Redirect is cache-first: ~95% hit rate in Redis keeps p99 under 10ms. On miss, read DB, populate cache with 24h TTL. Create path generates a unique ID via distributed counter or Snowflake, encodes to base62, inserts with uniqueness constraint.

Deep Dive: Key Generation & Collisions

Strategy	Pros	Cons
Counter + base62	No collisions; O(1)	Predictable unless salted; needs distributed counter
Hash (MD5/SHA truncate)	Deterministic dedup of same URL	Collisions require retry loop
Random + DB check	Non-guessable	Extra read on collision (~birthday bound at scale)

Industry choice

Twitter t.co and Bitly use counter-based IDs with base62 encoding. Hash-based dedup saves storage when the same long URL is submitted repeatedly.

Bottlenecks at Scale

Hot short codes (viral links) — single Redis shard hotspot; replicate hot keys across cache nodes
Custom alias namespace collisions — unique index + friendly 409 response
Analytics write amplification — decouple via async queue; never block redirect on click logging
DB write ceiling at ~10× traffic — pre-allocate ID ranges per server to batch inserts

Failure Modes & Monitoring

SLO	Target	Alert
Redirect latency p99	< 100ms	Page if > 150ms for 5 min
Redirect availability	99.99%	Page on error rate > 0.01%
Cache hit rate	> 90%	Warn if < 85%
Create success rate	99.9%	Warn on 5xx spike

Cache stampede: use request coalescing (single-flight) on DB miss for same code
Failover: read replicas for DB; stale cache acceptable briefly on partition
Dashboards: QPS by region, top viral codes, ID generator lag

System Design