Case study

Design a News Feed (Twitter/X)

Deliver personalized home timelines to 300M DAU with hybrid fan-out and celebrity handling.

fan-outtimelinecache

Requirements

Functional: post tweets (text, media); home feed of followed users; like/retweet
Feed ordering: reverse chronological (MVP); optional ranked feed
Non-functional: feed load p99 < 500ms; post tweet < 300ms
Scale: 300M DAU; avg 200 follows; 2 posts/day per active user
Celebrity users with 50M+ followers require special handling

Back-of-envelope Estimation

Metric	Calculation	Result
Posts/day	300M × 0.5 active × 2	300M posts/day
Write QPS	300M / 86,400	~3.5K/sec avg; ~15K peak
Feed reads/day	300M × 5 loads	1.5B reads/day
Read QPS	1.5B / 86,400	~17K/sec avg; ~50K peak
Fan-out writes (naive)	15K × 200 followers	3M writes/sec — infeasible

Why hybrid fan-out

Pure fan-out on write fails for celebrities. Pure fan-out on read is slow for users following 2000 accounts. Hybrid is the industry standard.

API Design

Endpoint	Description
POST /v1/tweets	{ text, mediaIds[] } → tweetId
GET /v1/feed?cursor=&limit=20	Paginated home timeline
GET /v1/users/{id}/tweets	Profile timeline
POST /v1/tweets/{id}/like	Idempotent like

Data Model

Store	Schema	Access pattern
tweets	tweet_id, user_id, text, media_urls, created_at	Write on post; read by id
follows	follower_id, followee_id, created_at	Graph lookup
user_timeline	user_id → sorted set (tweet_id, ts)	Redis; top 800 tweet IDs
tweet_content	tweet_id → blob	Cassandra; wide rows by tweet_id

High-level Design

Post & feed flow

Post ──► Tweet Service ──► Tweet Store
              └──► Fan-out Service ──► queue ──► workers update follower timelines (Redis)

Feed ──► Timeline Service ──► merge(cached timelines + celebrity tweets on read)
              └──► Tweet Service (hydrate tweet bodies)

Deep Dive: Hybrid Fan-out

User type	Write path	Read path
Normal (< 10K followers)	Push tweet_id to each follower's Redis timeline	Read precomputed timeline
Celebrity (> 10K followers)	Skip fan-out; store in celebrity tweet list	Merge at read time
Inactive follower	Skip fan-out if not logged in 30 days	Rebuild on next login

Bottlenecks at Scale

Celebrity post triggers millions of fan-out jobs — threshold gate prevents queue flood
Timeline hydration (800 IDs → full tweets) — batch mget; CDN for media
Hot tweet (viral) — single row hotspot; replicate in cache
Feed ranking ML — async precompute scores; serve from feature store

Failure Modes & Monitoring

SLO	Target
Feed p99 latency	< 500ms
Fan-out lag	< 30 sec for 99th percentile follower
Post success	99.95%

Monitor fan-out queue depth, celebrity merge latency, timeline cache memory per user. Alert if fan-out workers fall behind post rate for > 2 min.

System Design