Case study

Design a Distributed Cache (Redis Cluster)

Build a horizontally scalable in-memory cache with partitioning, replication, and cache-aside patterns.

consistent hashingevictionreplication

Requirements

Functional: get/set/delete with TTL; optional CAS (compare-and-swap)
Horizontal scale to 100+ nodes; automatic sharding
Non-functional: get p99 < 5ms; 99.9% availability; LRU eviction under memory pressure
Scale: 10 TB total cache; 1M ops/sec aggregate

Back-of-envelope Estimation

Metric	Calculation	Result
Total memory	10 TB	Given
Per node	64 GB RAM	~160 nodes
Ops/sec	1M aggregate	~6.25K ops/sec per node
Key count	10 TB / 1 KB avg value	~10B keys (many small)
Network	1M × 1 KB avg	~1 TB/sec peak — needs local hot set

API Design

Command	Description
GET key	Return value or miss
SET key value EX ttl	Upsert with optional TTL
DEL key	Delete from all replicas
MGET keys[]	Batch get; client-side hash routing or proxy
CAS key expected new_value	Atomic compare-and-swap

Data Model

Concept	Implementation
Key → slot	CRC16(key) mod 16384 (Redis cluster standard)
Slot → node	Consistent hash ring; 16384 slots mapped to nodes
Replica	Primary + 1 async replica per shard for failover
Eviction	allkeys-lru when maxmemory reached

High-level Design

Cluster topology

Clients ──► Smart Client or Proxy (hash slot routing)
                    │
        ┌───────────┼───────────┐
        ▼           ▼           ▼
    Shard A     Shard B     Shard C
    (primary)   (primary)   (primary)
        │           │           │
    Replica A'  Replica B'  Replica C'
    
Gossip protocol: cluster state, failover election

Deep Dive: Consistent Hashing & Resharding

16384 hash slots distributed across nodes. Add node: migrate slots gradually (MOVED redirects during migration). Client caches slot map; refresh on MOVED/ASK responses. Minimal key movement vs naive modulo hashing.

Deep Dive: Cache-aside & Stampede

Cache-aside: app reads cache → miss → read DB → populate cache
Stampede: single-flight lock per key during rebuild
TTL jitter: prevent simultaneous expiry thundering herd
Hot key: local L1 cache on app + read replicas for that slot

Write-through vs write-back

Cache-aside is most common. Write-through simplifies consistency but adds write latency. Write-back risks data loss on crash.

Failure Modes & Monitoring

SLO	Target
Get p99	< 5ms
Failover time	< 30 sec
Hit rate	> 90% (application-dependent)

Monitor: memory usage, evictions/sec, replication lag, slot migration progress
Alert: primary down, lag > 1 sec, cluster unreachable slots

System Design