Design a microblogging service

01Problem and scope

A microblogging service publishes short posts and combines eligible posts into home timelines. Store each post body and its media references once; author profiles and viewer inboxes are access paths containing references to that post. For example, post p701 appears in its author’s history and may become a candidate for many followers. The central design choice is how much timeline assembly to perform during publication versus on each read.

Include follows, likes, text, media, replies and reshares, with a chronological first version and a few seconds of feed propagation delay. Use an exercise limit of 500 characters with explicit Unicode validation; this is not a current claim about a named platform. Media upload completes separately before the post can reference it. Ranked feeds can later reorder a bounded eligible candidate set without changing publication or permission authority.

Core scope is posts, follows, likes, profiles, a paginated home feed, replies and reshares. Search, trends, mentions, notifications, recommendations and curated collections are explicit derived extensions. We will explain their data inputs and limits, not pretend one timeline algorithm implements them. The central interview problem is balancing repeated feed reads against writes to follower inboxes, then recovering unfinished fanout when some authors have far more followers than others.

Read amplification is the extra candidate retrieval and comparison needed to assemble one visible page. Write-time fanout moves some of that work earlier by copying a new post's reference into followers' inboxes. It reduces repeated reads but creates more writes, especially for authors with many followers.

02Functional requirements

1. Publish: Post appears on the author's authoritative profile after accepted commit.
2. Read the home feed: Eventually includes eligible posts, with no duplicate IDs within a page/session.
3. Follow/unfollow: Durable relationship; new follow may backfill a bounded recent window.
4. Like/unlike: One logical like per user/post; repeated action is harmless.
5. Delete/restrict: Current visibility check prevents a stale candidate from granting access.
6. Search/mention/trend: Derived results may lag and are independently permission-filtered.

Publication, media and related posts

The author creates an immutable post with request key post-71. A retry returns p701 rather than publishing a duplicate. Media IDs refer only to the author's already verified READY uploads. A reshare stores the original post ID and its own actor/time; it does not copy text that would remain visible after the original is removed. Replies retain a parent/root reference and have their own publication identity.

Freshness, visibility and scope limits

Feeds can be slightly old; deletion and privacy are different promises. A failed ranking service may fall back to recency, while failed authorization cannot fall back to “show everything.” A liker can remove their own like; counts may converge asynchronously and must not be treated as a ledger. A cursor defines page continuity when new posts arrive. We exclude arbitrary retroactive editing of post text and transactional changes to every follower inbox at once. Those exclusions keep publication atomic while allowing derived views to recover independently.

03Non-functional requirements

1. Latency: Home-feed metadata p95 below 200 ms; post acceptance p95 below 300 ms.
2. Availability: 99.9% eligible feed availability inside a region. Prefer a slightly older authorized feed to a fast unauthorized one.
3. Freshness: Ordinary active followers should usually receive a new candidate within five seconds. An accepted post is immediately retrievable from its authoritative endpoint/profile path even if follower inboxes lag.
4. Durability: Accepted posts survive one node or availability-zone failure through replicated authority.
5. Regional recovery: Use a separately tested asynchronous recovery point and a one-hour recovery target. During write-authority loss, reject new posts or leave them pending.
6. Retention: Keep posts for an illustrative five years, subject to deletion policy; media retention and replication cost are separate.
7. Authorization and revocation: Check current post visibility and private membership before returning metadata. Media tokens last at most 60 seconds; already granted media delivery has that revocation bound, and downloaded content cannot be recalled.

Source and derived-view invariants

Eventual consistency is appropriate for some derived feed work. It is not a blanket permission to treat freshness, deletion and private-media disclosure as equivalent forms of staleness.

04Capacity estimates

Workload assumptions and arithmetic

Use one billion registered users, 200 million daily users, 100 million posts/day and 200 follows per account. Each daily user opens two home pages and five profile pages, each displaying twenty posts. Therefore post writes are 100M / 86,400 = 1,157/s; page requests are 200M × 7 / 86,400 = 16,204/s; item impressions are 16,204 × 20 = 324,074/s. Roughly 324,000 item impressions/s is a different unit from the approximately 16,200 HTTP page requests/s that produce them.

Worked estimates

Capacity implications and limits

Retained storage measures bytes kept over time; egress measures bytes transferred to viewers. A content delivery network (CDN) caches media near viewers so repeated requests need fewer reads from the original media store. It changes which server supplies those bytes, not how many bytes viewers consume.

Assume impression mix matches publication mix: 20% include a 200 KB photo, 10% include a 2 MB video, and one-third of encountered videos play. At 28 billion impressions/day, photo egress is 28B × 0.20 × 200 KB / 86,400 ≈ 13 GB/s; video egress is 28B × 0.10 × (1/3) × 2 MB / 86,400 ≈ 21.6 GB/s. Using 280 displayed text bytes per impression gives about 91 MB/s; the 310-byte storage record includes additional metadata. Different viewing or media mixes require a new estimate. These are illustrative averages excluding protocol/variant changes. CDN hit rate reduces origin bytes, not the total sent to users. The largest ordinary feed cost may be candidate fanout, so measure follower distribution and active-reader reuse instead of trusting the average of 200 follows.

05APIs and contracts

Request and response example

The author sends POST /v1/posts with key post-71 and {"text":"The bridge is open","mediaIds":["media91"],"visibility":"public"}. Success returns 201 {"postId":"p701","createdAt":"..."} after durable commit. Reusing the key with changed text returns 409. The server derives author identity from authentication; it does not accept arbitrary owner IDs in the payload.

Interface contracts

Validation and response semantics

Cursor tokens include a cutoff/snapshot identity and deterministic time/ID tie-breaker, not a mutable numeric offset over a changing list. The next page avoids newly inserted earlier items shifting every row. Ranking changes require a bounded session snapshot or stable score/version contract. Return 413 for oversized media bodies at the upload API, 400 for text/media validation, 429 for action limits and 503 for unavailable authority. A timeout after submission is resolved using the original request key, not by silently creating another post.

A chronological cutoff with a stable last-seen sort key is keyset pagination, not an immutable snapshot of asynchronously arriving candidates. It prevents an existing item from shifting merely because newer posts arrive, but a late fanout entry older than the cutoff can be missed until refresh. If the product needs a repeatable browsing session, materialize a bounded list of candidate IDs at the first request, bind subsequent cursors to that list and viewer, and expire it explicitly (for example after five minutes). Current deletion and authorization filtering still runs on every page; the snapshot never freezes permission.

06Data model and access patterns

Store source records separately from the feed structures rebuilt from them. Post owns content; Follow and Like own user actions; Inbox holds possible feed entries. The outbox is a durable record of work committed with a post change, allowing downstream workers to recover that change even if the immediate queue send fails.

Use an author/time index (authorId,createdAt DESC,postId DESC) for profiles and celebrity merges. An ID containing time does not let the system find every post by the author without such an access path. Inbox keys support viewer/time range queries; the unique viewer/post identity prevents repeated fanout from creating duplicates. Like records are authoritative user actions, while counts are derived from idempotent change events. A delete may leave like rows temporarily, but a hidden post cannot be exposed merely because a like still exists.

Start with post/request/outbox transactions within the same storage partition, then route by author and logical time bucket when needed. Alternatively hash primary post IDs and maintain the author index explicitly; the choice is workload-dependent. Media objects have their own immutable storage identity and READY status. Search, trends, follow suggestions and curated collections are derived stores fed from committed events. Their availability cannot decide whether p701 exists or whether viewer A is allowed to see it.

The immediate author-profile guarantee requires the owning post partition to maintain its local author/time access path in the same accepted transaction. An asynchronously rebuilt global author index is only a derived candidate source. If primary records are instead hashed by post ID, either make the authoritative author index part of the commit protocol or explicitly add a recent-write overlay / weaken immediate profile visibility. The worked author-owned layout avoids that cross-partition write dependency.

07Basic working design

Publication and pull-on-read feed

One app and one SQL database can implement the core product. The author's transaction inserts p701 and its request result. The feed query reads the viewer’s followed authors, queries their recent posts, merges by (createdAt,postId) and returns twenty. This is fanout on read: the combination work happens when a viewer asks. It avoids writing unused feeds for people who never return.

Verified media and independent derived views

Keep media upload separate and require a verified media reference before publication. A text-only post stays a small transaction. Likes use a unique relationship rather than incrementing a counter blindly; repeats do not manufacture additional likes. Profiles are indexed range reads and can often be served more cheaply than a many-author feed. A reshare points to p701 and is filtered if the original becomes unavailable.

Commit boundary and baseline limits

The baseline acknowledgement is database commit. If the author's response is lost, post-71 returns p701. If viewer A's feed request fails, retrying is a read and need not mutate anything. Backups and restore tests cover post bodies and media pointers. At small traffic and follow counts this design is easier to operate than a fleet of fanout services. We add precomputation only after calculating repeated work and deciding which viewers benefit from it.

08Find the baseline flaws

09Improve the design, step by step

1. Add replicated authority and a publication outbox. Accepted-post durability and lost fanout triggers motivate a transaction that saves post, request identity and outbox together. A relay publishes committed events and may repeat them. This survives process crashes and separates post acceptance from follower speed. It costs replication latency and event-consumer deduplication. Direct synchronous writes into every inbox are rejected because one slow follower partition would delay the author's post.

2. Prepare inbox candidates for active ordinary followers. Repeated multi-author merges trigger fanout-on-write. Workers page through followers and insert p701 references. Reads become a bounded inbox range followed by loading the corresponding post records, often called hydration. Costs are write amplification, inbox storage and rebuild logic for dormant users. Pure pull remains better for infrequent readers, new follows and small graphs. Precomputation is a materialized view, meaning a stored answer that can be rebuilt from authoritative posts and relationships.

3. Keep high-fanout authors on a pull path. The sixteen-minute celebrity calculation triggers hybrid assembly. Store their recent posts once in author lists and merge them into active viewers' inbox candidates at read time. This bounds publication work and avoids many never-read writes. It adds two candidate paths, deduplication and per-reader celebrity merge cost. A fixed threshold is only an initial policy; choose it from active follower reads and measured write/read cost.

4. Separate media delivery and add partitioned derived services. The 24 TB/day ingress and large egress trigger private object storage with CDN delivery, while metadata caches and logical partitions distribute text/history. Search and trends consume events independently. Media delivery can then grow separately from post storage. The costs are permission checks at caches, delayed indexes, object-store operations and clear responsibility while partitions move. A single replicated SQL cluster remains a valid earlier step; a NoSQL label is not a performance proof.

At each stage, keep a recency fallback and bound per-request candidate work. Faster fanout is useful only if page assembly remains authorized and within its latency budget.

10Detailed architecture

Post request and source authority

The edge routes writes to a post API and reads to a feed/profile API. The post API validates ownership, media readiness and rate limits, then routes to the post's owning partition. The storage group replicates and commits the body, request result and outbox together. Follow and like services own their respective unique relationships and publish changes for derived counts, notifications and feed maintenance.

Durable fanout and derived indexes

The outbox relay feeds a durable event stream. Fanout workers read reverse-follow pages and update viewer inbox partitions; celebrity publication updates a shared author list instead. Index workers build shared author-list caches, title/text search, trend aggregates and notification tasks. The authoritative author/time index is committed with the post; profile reads route there when the derived list has not caught up. A checkpoint describes completed follower pages, not merely an event that was fetched into worker memory.

Current visibility and media delivery

Feed APIs merge ordinary inbox candidates with followed celebrity lists, remove duplicate IDs, load post records in batches and check current visibility and membership. Hot text/author lists can be cached, but the authoritative visibility boundary remains enforced before response. The media edge validates its short-lived grant before returning cached immutable bytes or fetching origin storage.

Routing epochs and replica policy

Logical partition maps are versioned. During migration, the new storage group copies the partition and replays subsequent changes. Storage then rejects writes from the old owner using an ownership version check before the new owner accepts writes. Replicas used for public body reads may lag under a chosen policy; replicas used for current deletion/permission decisions need the protocol's required freshness. This is why adding “read replicas” cannot automatically promise both immediate revocation and arbitrary availability during isolation.

11Write path and acknowledgement

Publication commits one authoritative post and a recoverable fanout event. The example uses request post-71, post p701, event e701, and follower IDs viewerA and viewerB to demonstrate checkpoint ordering.

1. The author authenticates and submits post-71 with media91. The API validates text length, permitted media ownership and READY media state.
2. At the author's post owner, a transaction checks the request identity, assigns p701, inserts the post and outbox event e701, and records the result. Commit replication completes before returning accepted.
3. The outbox relay publishes e701. If it crashes after publish but before recording progress, it publishes e701 again; consumers use its durable identity.
4. A fanout job records p701, the chosen ordinary-author policy and a follower-page cursor. It retrieves a bounded page containing viewer A and viewer B, considering current active-user eligibility.
5. It inserts (viewerA,p701) and (viewerB,p701) with a stable sort key using conditional uniqueness. A repeated insert observes the same candidate rather than adding another row.
6. Only after all writes in the follower page complete does the job persist the next checkpoint. A failed page is repeated. A terminal checkpoint means all pages under the chosen scan policy were processed.
7. Search, notifications and trend workers independently consume e701. Their failure does not roll back the accepted post. The author's profile can read the authority while follower and search views catch up.

Follower membership can change while pages are scanned. Define new-follow backfill separately and recheck follow/privacy rules on reads. Do not claim the fanout traversed a globally frozen social graph unless the implementation actually supplies such a snapshot.

12Read and delivery path

Home-feed reads merge bounded candidate sources and recheck current visibility. This example returns up to twenty items while retaining a stable continuation boundary.

Each candidate source is already ordered. A heap, used here as a priority queue, keeps the next available item from each source and selects the newest one; after selecting it, the merge adds that source's following item. This avoids sorting every source's entire history, while still requiring explicit limits on how many sources and candidates the request examines.

1. Viewer A requests a twenty-item page under a viewer-scoped cursor. The API loads a bounded recent inbox range; a dormant or newly registered viewer may trigger a bounded rebuild from followed author histories.
2. Read recent lists for viewer A's followed high-fanout authors and merge them with inbox candidates. A heap can merge sorted lists efficiently, but candidate and author limits still matter for a user following many celebrities.
3. Deduplicate p701 if it arrives through more than one route or reshare policy. Reshares can retain their actor context while referencing one original body; the product defines whether both activities appear.
4. Batch-load the candidate post records, check current deletion/visibility and private membership, and discard ineligible IDs. A stale search/inbox/cache reference cannot override this check.
5. Apply recency ordering or a bounded ranker, return the first twenty and a stable continuation token. Reserve enough extra candidates to tolerate filtering without unbounded loops. A shorter page is preferable to exceeding the latency budget indefinitely.
6. Return scoped media grants and let the client fetch bytes through the delivery edge. Emit page and item-impression telemetry separately.

New posts do not shift a numeric offset because this API uses either stable keyset positions or a pinned bounded candidate list. A cutoff alone does not prevent late fanout from changing the candidate population; refresh includes such arrivals, or the repeatable-session option pins the candidate IDs. A refresh starts a new snapshot. Deletions may remove items between pages; the API must handle them without leaking bodies or replaying already seen IDs endlessly. Prefetching the next bounded page is an optional latency optimization, not a requirement to materialize the user's entire history.

13Correctness deep dive

Checkpoint after certified writes

Repeated follower-page interleaving

At t0 worker A inserts viewer A. At t1 it loses its lease and worker B repeats the same page. B's viewer A insert is a no-op, viewer B's insert succeeds, and B advances the checkpoint. A may wake and repeat its writes; because these effects only insert the same immutable candidate identity, duplication does not corrupt the view. Checkpoint updates still use a current job generation or monotonic compare-and-swap so an old worker cannot move progress backward or incorrectly skip a newer page.

Deletion remains a serving-time decision

Celebrity strategy migration

A fanout event for a celebrity never enters this enormous scan. Policy changes are versioned and deduplicated during migration so a post may safely appear via both candidate paths while the threshold changes.

14Failure and recovery

15Operations, security, and cost

Traffic, fanout and freshness signals

Measure accepted posts/s, page QPS, item impressions/s, feed p95/p99, candidate counts, fanout age, fraction of active readers whose inbox lacks recently eligible posts, cache byte/object hit ratios and celebrity merge cost. Keep freshness and latency separate: a 50 ms response containing yesterday's feed is not a successful five-second propagation result. Quotas cover posts, likes, follows and fanout-inducing actions, with account reputation and media validation to contain abuse.

Measured push-versus-pull cost

A rough push/pull decision compares recipient writes W with repeated merge reads R over the useful post window. If one reference write costs one unit and merging that author's candidate costs one unit per feed request, pushing to ten million mostly inactive followers can cost more than the hundred thousand actual reads it saves. Conversely, a small active community refreshing often benefits from precomputation. Measure both costs and cache effects before choosing a threshold.

Algorithms for derived features

Extended features consume committed events but need separate algorithms. Search builds an inverted index and ranks retrieved visible posts. Trends aggregate hashtags, queries, reshares or likes over explicit windows and update intervals; anti-abuse and unique-user signals prevent one bot from dominating raw frequency. Mentions/replies generate authorized notification tasks. Follow suggestions can explore bounded friends-of-friends candidates and rank mutual connections or interests, with privacy constraints. Curated “moments” group related recent posts/articles through classification or clustering and editorial policy; they are not the same as raw hashtag counts.

Fault tests and index rollout

Test fanout crashes, threshold changes, delete-during-hydration, stale replicas and replayed like events. Build a new index beside the live one and compare their query results. Then switch readers to the new version, retaining the old version for rollback. The authoritative post schema and privacy checks should not depend on a successful ranking experiment.

16Decision ledger and limitations

Identifier design is related to display ordering but does not replace storage layout. Including time can make IDs sortable; including a generator and sequence can distinguish concurrent allocations. Those fields still need allocation rules, and profile queries still need the author access path described above.

A timestamp/generator/sequence ID scheme needs unique generator assignment, overflow handling and a clock-rollback policy. An ID format with 31 timestamp bits measured in seconds and 17 sequence bits has a finite time horizon and per-second allocation limit; odd/even generators remain safe only while failover preserves disjoint allocation. Standard UUID or allocated ranges are alternatives, with sorting/index tradeoffs. No identifier scheme eliminates the database uniqueness rule or every secondary index. Three days of text may be 93 GB logically, but strings, object overhead, indexes and replicas make actual cache allocation larger.

17Interview closing

“I designed one authoritative post and multiple recoverable views. A post commits with its retry identity and outbox before acceptance. Profiles read author history; home feeds combine ordinary-author inbox references with high-fanout author lists. That asymmetry follows the workload: 16,000 average page requests per second are different from 324,000 item impressions, and fifty million recipient writes cannot meet a five-second freshness target.

“Fanout is resumable by follower page and safe to repeat through unique inbox keys. Candidate lists may lag, but current visibility filtering prevents stale IDs from authorizing deleted or private posts. Media is stored and delivered separately because its byte volume dominates text. Search, trends and recommendations are downstream products with their own quality and abuse controls.

“I accept extra derived indexes and two feed paths to avoid worst-case publication amplification. My next measurement is the push-versus-pull cost for active audiences, plus a cache-failure test on a viral post.”

If the interviewer asks for globally strict chronological order, distinguish a deterministic display sort from real-time total order across all writers. A global sequencer would add coordination and failure dependence that this feed does not require. Agree on the actual visible ordering contract before adding that bottleneck.

Recall the key ideas

Answer from memory before opening each card. Explain why the choice works and what it costs. Revisit missed cards tomorrow.