01System design: definition and purpose

System design is the process of defining components, data, interfaces and interactions that satisfy a service’s requirements. A component is a part with a specific job: an application accepts a request, a database stores searchable records, and file storage holds uploaded bytes. An interface is the agreement for asking a component to do work. The interview asks you to show how these parts cooperate and how the design behaves when traffic increases or a part fails.

You are not expected to guess a company's private architecture. You are expected to build a plausible design under stated requirements. A requirement states what the product must do or how well it must work. A tradeoff is a benefit gained by accepting a cost or limitation elsewhere. For example, keeping a second copy of an uploaded photo helps survive a disk failure, but uses more storage and requires a rule for when the upload is considered safe.

A complete answer covers requirements, workload estimates, API contracts, data ownership, component responsibilities, and failure behavior. Use one operation to verify that the proposed components form a working system. A bounded example throughout the method is a file-sharing API for PDF worksheets, with upload, download, and deletion; record 42 identifies a 2 MB PDF.

02Functional and non-functional requirements

Start by identifying actors, operations, and access rules. Clarify account requirements, file-size limits, link expiration, and revocation. These decisions determine the API and how current its permission checks must be: a public permanent link needs different read checks from a link that must stop authorizing downloads immediately after revocation.

Functional requirements: what the product does

1. Upload. An authenticated uploader can create an upload for a PDF up to 10 MB.
2. Share. Create an unlisted download link, with optional expiration. “Unlisted” means the link is hard to guess but possession permits access; private group sharing needs an additional authorization policy.
3. Download. Retrieve a permitted file through its link.
4. Delete. The owner can revoke future downloads.

These capabilities need file storage and link metadata. Collaborative editing and document grading are outside this example’s scope.

Terms used in the targets

Here, latency is the time one request takes; p95 is a threshold met by approximately 95% of the measured requests. Availability measures whether the requested operation can be used successfully. Authoritative means the component whose recorded decision is treated as truth. These terms make a requirement testable instead of merely saying “fast and reliable.”

Non-functional requirements: how well it must work

1. Performance. Start an allowed download within 200 ms at p95. Measure lookup and first-byte delay separately.
2. Correctness. A deleted link cannot start a new download. Check the authoritative deletion policy.
3. Availability. Target 99.9% successful eligible download attempts per month. Define the measurement and a failure plan.

The numeric targets here are assumptions for practice. State them, then invite the interviewer to change them.

03Workload and capacity estimates

Assume 100,000 uploading accounts, two files per account per month, a 2 MB average file, and 30 downloads per file. Use the numbers to identify the dominant work:

These figures justify separating file delivery from metadata requests. They do not establish that the metadata database needs hundreds of shards.

Interview checklist:

1. Show units. “400 GB” is storage; “400 GB/month” is growth; “1,000 requests/s” is a rate.
2. Ask about retention. Do this before multiplying monthly growth into lifetime storage.
3. Estimate to decide. Do not spend ten minutes estimating a number that will not change the architecture.

04APIs, data model and source of truth

An API is the agreement between a caller and a service. For example, use POST /worksheets with a filename, size, and expiry to create an upload session. A response returns worksheetId=42 and an upload destination. A completion call validates that the file exists before changing its state to ready. GET /links/abc123 looks up a download, and an authorized DELETE /worksheets/42 revokes future access.

Metadata is information about a file rather than the file’s own bytes. Keep one metadata row: Worksheet(id, ownerId, objectKey, objectVersion, state, expiresAt, deletedAt). The bytes live separately under an object key such as worksheets/42/v1. An object key is the storage address of the file, not the public permission to download it. The query you must support is a point lookup of one link or worksheet, so a primary-key index is a useful first choice.

A primary key uniquely identifies a database row. An index is a maintained lookup structure that helps the database find matching rows without inspecting every row. Here the link token identifies one mapping, and the worksheet ID identifies one metadata record; neither lookup needs to search the file bytes.

The initial implementation can be one application and one database plus durable file storage. Splitting APIs into many services before describing this contract adds complexity without establishing a correct request path.

The public token also needs a stored mapping: Link(token PRIMARY KEY, worksheetId). Generate an unpredictable token for an unlisted link; the short abc123 above is only a readable example, not an adequate security design. A unique owner/request-key record can make upload-session creation retryable. The file row and its request result commit together; repeating that request returns the same session rather than allocating another file.

05Worked example: upload, download and retry

Publishing a worksheet requires agreement between two stores: the metadata database and the file store. The database must not advertise a file whose upload is incomplete. An immutable object version is a particular stored version whose bytes do not change; completion verifies that version and makes the metadata point to it. The following sequence uses states to coordinate that publication.

1. An authenticated POST /worksheets validates the caller’s quota and stores row 42 with state uploading.
2. The upload writes bytes to the designated object key. The storage layer must record the complete object; an interrupted transfer does not make it downloadable.
3. The completion operation verifies a specific immutable object version, records its checksum and size, and conditionally changes row 42 from uploading to ready only if it has not been deleted. A checksum summarizes bytes so corruption can be detected. Atomic means the state transition is indivisible; another request cannot observe half of the row change.
4. GET /links/abc123 checks row 42’s readiness, expiry, and deletion state before allowing transfer.
5. An authorized delete records revocation first. Later cleanup removes unused bytes; delayed cleanup does not reauthorize the link.

The server can save a change successfully even if its response never reaches the client. Handling retries after this failure requires idempotency: repeating one logical operation must recover the same business result. If completion commits but its response is lost, retrying with the stable upload ID returns row 42’s existing ready state instead of creating another file. This failure test identifies where the deduplication record and result must be durable.

Test deletion during upload completion as well as a lost response. If deletion commits first, completion must fail its state check and leave the row deleted. If completion commits first, deletion revokes the published file. A late upload must not overwrite the verified object version. For downloads, the access check determines the order: a transfer authorized before deletion may finish; a check after deletion must reject it. If current metadata is unavailable, block new downloads. HTTP methods and object storage do not enforce these rules on their own.

06Scaling, replication and recovery

When one application cannot handle the measured peak, add stateless application instances and a load balancer, which distributes incoming requests among them. Stateless means another instance can handle the next request because essential worksheet state lives in shared durable storage. If popular public worksheets account for most bytes, a content delivery network can serve permitted cached files close to clients. Revocable/private files need a compatible authorization and cache-expiration design.

Replicate important records, decide what a successful upload promises about durability, and test restoration from backups. Replication means maintaining live copies; a backup lets you recover an older version after a bad deletion. They solve different failures.

In a 45-minute practice session, spend approximately five minutes clarifying, seven on quantities and contracts, ten drawing and tracing, fifteen on the most important bottleneck and failure, and eight reviewing. The interviewer may redirect you. Follow that signal rather than treating the time allocation as a script.

07Monoliths, modular monoliths and service boundaries

A microservices architecture separates capabilities into independently deployable services that communicate through APIs or messages. Each service controls changes to its own data; other services use its contract instead of changing its tables directly. This can help teams release independently and give a demanding component its own resources. It also adds network calls, compatibility work and more components to operate. See Martin Fowler’s discussion of microservice trade-offs.

For a concrete example, the checkout design initially keeps order creation and stock reservation in one database transaction. The application can have separate order and inventory modules without splitting that transaction across services. If browsing grows much faster than purchasing, a separate catalog search service can scale its derived product index while checkout keeps authoritative prices and stock checks together.

Splitting inventory into an independent service needs a stronger reason, such as a shared reservation capability serving several products with its own release schedule. The order and reservation would then commit separately. Define reservation expiry, retries and recovery before claiming the purchase succeeds; use the distributed-workflow lesson for that coordination. Moving code into separate processes does not make the two commits atomic.

Choose boundaries around responsibilities that can evolve independently. A diagram box may be a module, a process or a replicated service; say which you mean. Explain how a caller behaves when a service fails, because separation alone does not prevent an outage from spreading. Microsoft’s architecture guidance describes these deployment, data-ownership and failure-handling concerns.

08Interview example: explain a storage choice

Interviewer: “Why not store the PDF in the database?”

Candidate: “The database needs small records for ownership and link lookups. Our estimated 12 TB of monthly downloads is mostly file bytes, so I would put those bytes in object storage and keep their keys in the database. That allows downloads to scale independently. The added problem is publication across two stores; I handle it with uploading and ready states, and make completion retryable.”

This answer contains a choice, a workload-based reason, a new failure risk, and a concrete mechanism. If you cannot explain those four parts for a component, revisit whether it belongs in the first design.

When answering a follow-up, identify which requirement has changed before adding or replacing components in the diagram. If the interviewer changes worksheets from unlisted to private groups, the concrete new requirement is “only authorized group members may download.” Add a per-reader permission check and explain its failure behavior; the PDF storage itself does not have to change.

09The complete interview sequence

The 36 design chapters expand this method into a full practice interview. Their sections are preparation material: do not recite every paragraph or spend equal time on every stage. In a live interview, establish the complete outline, trace the central request, and use the interviewer's questions to decide which part of the design to explain in greater detail.

Draw the baseline first. When changing it, point to the failed requirement: “This cache removes repeated reads, but introduces up to 25 seconds of stale access, so it violates our original immediate-revocation promise unless we change that contract.” The change is not justified merely because a cache is conventional. A mature answer may keep the simpler design when the requirement does not pay for the added complexity.

Close in roughly 60–90 seconds: restate the requirement, describe the resulting request path, name the invariant and its mechanism, acknowledge the largest cost, and propose the next measurement. Then practise changing one requirement. Changing public file sharing to private group sharing introduces per-reader authorization and revocation; the existing object store remains useful, but the permission decision must change.

Recall the key ideas

Answer from memory before opening each card. Explain why the choice works and what it costs. Revisit missed cards tomorrow.

01API and HTTP request: definitions

An application programming interface (API) defines the agreed rules for how one program requests data or an action from another. An HTTP API represents that contract using a method, path, headers, optional request body, status code, and response body. The caller sends a request and receives a response; the contract defines the operation, input, identity, result, and failure behavior.

The edge is the service’s public entry point, often a proxy or load balancer. Authentication establishes who the caller is; authorization checks what that caller may access or change. The request lifecycle consists of name resolution, connection establishment, protocol exchange, routing, authentication, authorization, application execution, and response handling. Some stages can be cached or reused. A GET https://shop.example/orders/O17 example shows their order and distinct responsibilities; the API must identify user U9 and authorize access before returning order data.

A successful network exchange does not automatically mean a successful business operation. A server might return a valid “order not found,” an authentication error, or an infrastructure error. State what each result means before deciding which results are retryable.

02DNS and service discovery

DNS, the Domain Name System, maps names to records such as IP addresses. A client checks usable cached results or asks a recursive resolver. If necessary, the resolver follows the naming hierarchy to authoritative servers and returns an address for shop.example. A TTL specifies the record’s permitted cache lifetime under the protocol rules.

The address may point to an edge or load balancer rather than the database host. DNS does not authenticate U9 or return O17. It enables the next communication step. A cached address also explains why changing a DNS record does not instantly move every client during failover.

Internal service discovery may use DNS or a registry of healthy instances. It finds a server to contact. That server must still check the request and whether it is allowed to read or change the requested data.

03TCP, TLS, HTTP/2, and HTTP/3

For a typical HTTPS request using HTTP/1.1 or HTTP/2, the Transmission Control Protocol (TCP) provides a reliable ordered byte stream between endpoints. Transport Layer Security (TLS) authenticates the server and encrypts the conversation. The browser checks that the certificate is valid for the requested name. Existing connections may be reused, avoiding repeated setup.

HTTP/2 multiplexes requests into separate application streams on one connection. It improves reuse, but TCP packet loss can stall delivery across those streams. HTTP/3 carries HTTP over QUIC, which uses User Datagram Protocol (UDP) packets and provides reliable streams and integrated cryptographic setup. QUIC avoids that particular cross-stream TCP head-of-line blocking; it does not remove all congestion, loss, or application queueing.

In an interview, start with the transport actually needed. An HTTPS API is usually enough to describe an order read. Choose streaming, bidirectional communication, or a different transport when the user interaction requires it, rather than listing protocols without a reason.

04HTTP request lifecycle: worked order read

A simplified request contract is:

GET /orders/O17 HTTP/1.1
Host: shop.example
Authorization: Bearer <access-token>
Accept: application/json

1. The edge accepts the connection and routes /orders/O17 to the order API.
2. The API validates the credential and derives userId=U9. It never trusts a caller-provided user ID as proof of identity.
3. It performs an authorized lookup of O17 under U9’s verified scope, returning the row and version to which the access decision applies. Knowing the order ID is not authorization.
4. It serializes the permitted fields from that same authorized version and returns 200 with {"orderId":"O17","state":"paid","totalMinor":2500,"currency":"USD"}.
5. The browser parses the result and renders the order. Its total latency includes network, queueing, application, database, and rendering time.

The HTTP/1.1 notation is a readable contract illustration; HTTP/2 and HTTP/3 frame the same method/path/status semantics differently. A trace ID propagated through the servers helps diagnose where this particular request spent time.

05HTTP methods, idempotency, and status codes

The method tells the server what kind of operation the client intends. A safe method requests read-only behavior; an idempotent method has the same intended effect when repeated as when performed once. These properties help decide whether repeating an interrupted request is compatible with the API contract.

Use distinct results so the caller can decide what to do next:

The exact privacy and retry policy belongs in the API contract.

A conditional request adds a precondition about the current representation. A reader can ask whether its cached version is unchanged; an editor can require that the version it edited is still current before replacing it. Both use a server-issued version identifier rather than assuming nothing changed between requests.

Conditional requests connect HTTP to versioned data:

06Resource-oriented HTTP, RPC, and pagination

The same order operation can be exposed by naming a resource and an HTTP method, or by naming a remote procedure with typed arguments. These are interface choices: they determine how the caller expresses its request, while the service still defines ownership, permissions and success.

A resource-oriented HTTP API exposes orders and profiles through URLs and standard methods; REST is an architectural style with additional constraints, not merely a synonym for JSON. RPC means remote procedure call. An RPC API names an operation on another service, such as ReserveSeats, with typed input and output. gRPC commonly uses protocol buffers and HTTP/2 for typed calls and streaming. Browser clients and intermediaries may require compatible gateways or gRPC-Web support.

Choose a style for clients, tooling, and communication needs. A public web API benefits from familiar HTTP behavior and broad client support. Internal typed service calls may benefit from generated clients and schemas. Neither choice determines database consistency or business correctness.

Pagination contract

Bound response sizes. For order history, use a page size and cursor based on a stable ordering such as (createdAt, orderId). Validate the cursor and preserve tie-breaking semantics. Pagination is part of the API; it must match the query/index design rather than being added after the storage choice.

For descending order history, the continuation predicate is (createdAt, orderId) < (lastCreatedAt, lastOrderId) with the same ORDER BY createdAt DESC, orderId DESC and a bounded limit. Include the verified tenant, filters and ordering version in the cursor's validated scope. A cursor is not permission to switch tenants.

GraphQL: client flexibility and server cost

GraphQL provides a typed API schema and lets a client request a particular selection of fields. A query such as an order with selected item fields can reduce over-fetching and combine related reads. Field resolvers may call databases or services; one client request can still trigger many backend requests. Batch related lookups to avoid an N+1 pattern, where fetching N items adds N individual calls.

Use object/field authorization and bounded pagination. Limit expensive query shapes with complexity, depth and execution budgets or an approved operation set. A shared endpoint does not make all queries equally cheap, and authentication alone does not authorize nested objects. Prefer this flexibility when clients genuinely need different composed views; a small REST or gRPC contract is often simpler for a fixed workflow.

07API retries, deadlines, and version compatibility

A create-order timeout leaves the result unknown: the server may already have committed. A stable request key and status lookup recover the original outcome. A deadline bounds the caller’s wait; it does not roll back work committed elsewhere.

Version the contract when making incompatible changes. Additive optional fields are often easier to roll out than renaming a required field, but clients must actually tolerate unknown fields and defaults. Deploy producers and consumers in an order that supports mixed versions. For a breaking change, define an explicit migration/version policy rather than assuming all clients upgrade at once.

Spoken answer: “I define the order operation and its success meaning first. Then I trace DNS, connection, edge routing, authentication, authorization, database lookup, and response. I measure the time spent in each stage, limit request size and execution time, and make timeout recovery and compatibility part of the contract.”

Recall the key ideas

Answer from memory before opening each card. Explain why the choice works and what it costs. Revisit missed cards tomorrow.

01Capacity estimation and its units

Capacity estimation translates a workload into the resources needed to meet its targets. A workload specifies what users do, how often, how large their requests are and how concentrated the traffic becomes. The estimate should be accurate enough to choose a design; a load test must later measure the actual implementation. Begin with four separate ideas. Throughput is completed work per unit time, such as 500 uploads per second. Latency is how long one operation takes. Concurrency is the number of operations in progress at once. Storage is how much retained data exists at a point in time.

A restaurant can serve many meals an hour while one customer's meal takes a long time. A batch service can likewise have high throughput and high latency. More concurrent work helps use idle resources; once the limiting resource is fully busy, extra work mostly waits. A claim of “10,000 users” needs to say what they do and when.

QPS means queries per second; in an API discussion people often use it for requests per second, so state whether you are counting API requests or database queries. Network capacity, often called bandwidth, is the maximum data rate a link or path can carry under stated conditions, usually measured in bits/s. Network throughput is the rate actually achieved. Requests/s × bytes/request estimates the required transfer rate; provision capacity above that demand, including protocol overhead and headroom. Peak means the busiest declared interval, while an average spreads all work over the entire measured period. These are different quantities even when one calculation produces another.

02Worked example: average and peak QPS

Define a workload before estimating resources. For this photo-service example, assume one million daily active users, 20 photo views and 0.1 uploads per user per day, a 2 MB average upload, a 100 KB thumbnail, and 1 KB of metadata per photo. Use decimal units: 1 KB = 1,000 bytes, 1 MB = 1,000,000 bytes, and one day = 86,400 seconds.

The calculation is a sequence: first count actions per day, then divide by seconds per day, then apply an explicitly assumed peak factor. For these inputs, 1,000,000 × 20 = 20,000,000 image views/day, 20,000,000 ÷ 86,400 ≈ 231.5 views/s average, and 231.5 × 10 ≈ 2,315 views/s peak. We size thumbnail delivery against the last rate, then verify it against measured bursts.

03Storage, retention and network bandwidth

The photo workload creates two different demands: storage for retained objects and network capacity for repeated delivery. Logical data counts one copy of each retained object; replicas and backups consume additional physical storage. Keep those counts separate from bytes sent to viewers:

Keep these quantities separate:

1. Count other stored data separately. Thumbnail variants and backups are additional categories; the replication multiplier does not include them.
2. Bytes and metadata scale differently. Their large size difference is a reason to store them separately. The metadata database need not carry every byte transferred to viewers.
3. Convert units explicitly. Network links are often rated in bits/s: multiply bytes by eight. State whether you mean MB or MiB.

04Little’s law and latency percentiles

Request rate alone does not tell us how many connections or request buffers are occupied. A request continues using some resources while it waits for storage or another service. To size those resources, relate the completion rate to the time each request remains in the service.

If average time rises to 0.5 seconds while admitted traffic stays at 2,000/s, concurrency becomes about 1,000. The extra 900 requests need memory, sockets, and possibly database connections. An unbounded queue hides overload briefly while increasing latency. It does not create processing capacity.

The stable-system condition matters. If arrivals stay at 1,200/s while only 1,000/s complete, an unbounded backlog grows by 200 requests/s, or 12,000 requests in one minute. There is no steady finite average latency to insert into this calculation. Bound the queue and reduce admissions, or increase the bottleneck’s measured service capacity.

05Bottlenecks and failure headroom

A bottleneck is the resource that first limits the workload: for example, CPU, database writes or network transfer. Headroom is spare capacity reserved for bursts, uneven load and failures. Once a load test identifies the limiting resource, size enough instances to meet the target even with the chosen failures.

Assume a load test measures 800 requests/s per application instance while meeting the latency objective, and the target peak is 2,315 requests/s.

The notation ceil(x) means the smallest whole number at least as large as x; a partial server cannot satisfy the remaining load. For a chosen maximum of 70% of tested capacity after one failure:

1. Budget each survivor: 800 × 0.7 = 560 requests/s.
2. Find the survivors needed: ceil(2,315 / 560) = 5.
3. Add failure capacity: five survivors require six instances.

This is illustrative sizing, not a universal 70% rule. Real benchmarks, cost, autoscaling lag and failure domains determine the target.

Check downstream amplification

The database, network, and object store must support the same workload. Six application servers do not help if they all wait for one slow query. Estimate the read/write amplification: if each API call issues five database queries, 2,315 API calls/s can become 11,575 database queries/s.

Size CPU from CPU time

Compute demand has a different unit from elapsed latency. If a measured request uses 2 ms of CPU time:

1. CPU demand: 2,315/s × 0.002 CPU-seconds = 4.63 CPU-seconds/s, about 4.63 fully busy cores.
2. Utilization headroom: at a chosen 70% limit, ceil(4.63 / 0.7) = 7 usable cores before additional failure capacity.

06Cache working set and cost model

A cache stores copies of reused data. Its size depends on distinct hot entries, not total requests. Suppose 500,000 frequently viewed photo records occupy 1.4 KB each including key and bookkeeping overhead. That is about 700 MB per full cache copy. Ten million reads of those same entries do not require ten million stored entries.

A cache hit finds the requested value in the cache; a miss must fetch it from the underlying database or storage service, called the origin. The request hit rate is the fraction of cacheable requests served as hits. This rate turns the delivery estimate into an estimate of work still reaching the origin.

A 95% request hit rate reduces 2,315 cacheable lookups/s to about 2,315 × 0.05 = 116 misses/s under the same workload. But when the cache is empty, the origin can suddenly see all 2,315/s. Protect that origin and warm popular entries gradually. Track byte hit rate separately: a few missed large images may dominate bandwidth despite a high request hit rate.

For cost, write a symbolic model before using current provider prices: storage GB-month + read/write operations + delivered GB + compute time + replication/backup. A cheaper storage tier can have retrieval fees and slower access. The interview value is identifying the dominant cost and a way to measure it, not memorizing a vendor price that may change.

Recall the key ideas

Answer from memory before opening each card. Explain why the choice works and what it costs. Revisit missed cards tomorrow.

01Distributed system quality attributes: definitions

A distributed system consists of independent computers that coordinate by exchanging messages. One part can fail while others keep running; this is a partial failure. An online shop might run its request-handling application on two machines, keep live copies of orders on several database machines, and send receipts through a background worker. The customer sees one checkout experience, even though these parts can fail or respond at different times.

The qualities below answer different questions about that experience. Scalability asks whether the service can handle a larger workload while maintaining its targets. Reliability is the ability to perform the specified function correctly under stated conditions over a period of time. Availability is the degree to which the service is usable when requested, often measured as successful eligible requests divided by all eligible requests. Efficiency asks how much useful work it gets from its resources. Manageability asks how safely operators can observe, configure, operate, and change it. The related term serviceability focuses on diagnosing and repairing faults.

02Vertical scaling, horizontal scaling and serial bottlenecks

At first, one application server handles 500 checkout requests/s. Vertical scaling replaces it with a larger machine: more CPU, memory, or faster disks. It can be the simplest improvement, but hardware has practical limits and a single machine still fails as one unit.

Horizontal scaling adds machines. Put two application servers behind a load balancer, and either can handle a request if essential state is stored outside the process. This can grow application capacity and tolerate one application failure if the survivor can meet the admitted workload. It does not automatically double database write capacity.

Some work remains serialized: operations must take turns because they update the same protected state. Adding application machines does not remove that ordering requirement. This matters both for the time one checkout takes and for how many checkouts can update the same inventory record.

For a separate latency calculation, suppose one request spends 80 ms on parallelizable work and 20 ms executing a serialized operation on one inventory key, excluding queue wait. Making the first part four times faster yields 80/4 + 20 = 40 ms, a 2.5× improvement, not 4×. Even infinitely fast application work cannot eliminate the remaining 20 ms. This is the intuition behind a serial bottleneck: improve the part that limits the actual operation.

The workload also matters. Adding nodes can help independent product lookups while thousands of purchases of the same final item still contend on one record. Measure distribution, not just total QPS.

03Availability and error-budget calculations

An error budget is the amount of unsuccessful service allowed by the chosen availability target over a defined measurement window. The target supplies the permitted fraction; the number of requests or the duration of the window turns it into a count or time allowance. Choose that denominator before interpreting an outage.

At 10:00 the only order database stops responding. Automated detection fires at 10:01. An operator finishes failover and verifies writes at 10:07. Checkout was unavailable for seven minutes, not merely the six minutes spent repairing after detection. Monitoring delay is part of the user impact.

For a simple recurring up/down model, availability can be approximated by mean uptime / (mean uptime + mean downtime). Real services have partial and correlated failures, so a single formula is not a substitute for measuring user requests. Faster detection and repair can improve availability even when the underlying failure frequency is unchanged.

Dependencies also affect the result. In a deliberately simplified model, if two required dependencies are independently available 99.9% of the time, the path is available 0.999 × 0.999 = 99.8001% of the time before other failure sources. Redundant alternatives instead help only when at least one is usable and routing can reach it. Shared power, bad configuration and overload make failures correlated, so multiplying advertised service percentages is not a production reliability proof.

04Reliability, durability and failure domains

If order O17 commits but its response is lost, a retry can create O18 and charge again. Save the result under a stable request ID so a retry returns O17. Save the order and request result in one atomic database transaction: both commit or neither does. An external payment is outside that transaction. Reuse the same payment identifier under the provider’s retry rules, and check an uncertain result before issuing another charge.

Durability is retention of acknowledged data. Replicated records can survive a machine loss if the acknowledgment and recovery protocol make that promise. A backup can restore an earlier state after accidental deletion. Both require verification; merely drawing duplicate cylinders does not prove an acknowledged purchase survives.

A failure domain is a set of components that one event can disable together, such as machines sharing a power supply or deployment zone. A network partition prevents some machines from communicating even though they may still be running. Replica placement must match the failures the service is meant to survive.

The following failure sequence shows which records and identifiers must survive a lost response. (1) Purchase key K17 requests $25 and the order authority records O17. (2) Payment action charge-O17 produces confirmed provider charge C81. (3) The application response is lost. (4) Retrying K17 returns O17/C81 rather than allocating O18 or a new charge identity. If the provider response was lost instead, the charge remains unknown until lookup or the provider’s documented same-key retry resolves it. A timeout establishes uncertainty, not failure.

05Resource efficiency and communication cost

Efficiency is useful outcomes divided by the resources spent. For O17, ten internal RPCs—remote procedure calls—may each transfer a small record. One giant catalog transfer may use fewer messages but far more bytes. Count both messages and data size, then account for network distance and repeated work.

Suppose design A makes ten sequential 5 ms calls and design B makes two 20 ms calls. Their network wait contributions are about 50 and 40 ms respectively in this simplified example. A third design could batch data into one call, but might waste bytes or postpone the response. Message count alone does not identify the best design.

Mixed machine sizes, topology, and uneven load make ideal linear speedup unlikely. Compare designs with the same workload and objective.

06Manageability, monitoring and safe change

An operator should be able to answer what failed, which customers are affected, and which action is safe. Attach one request/trace ID to O17 across services, record state transitions without payment secrets, and measure both successful outcomes and latency. A health endpoint that only says the process is alive does not prove orders can commit.

Use different controls for different problems. Readiness decides whether an instance receives new requests. A restart policy decides when to restart its process. Admission control limits accepted work so existing requests can finish. If a dependency fails, accept less work where necessary; restarting otherwise healthy application processes will not fix that dependency.

Roll out a new version to a small fraction first, compare outcomes, and retain a rollback path. A database change should let old and new application versions coexist during the rollout. Stop assigning new work to a known dead instance, and bound or shed the excess traffic if survivors lack capacity. Separately, avoid ejecting or repeatedly restarting every live instance merely because a shared dependency is slow: that reaction can reduce useful capacity further. Readiness, restart policy and admission control have different jobs.

Candidate explanation: “I separate checkout availability from order correctness. I can temporarily refuse new purchases when I cannot confirm which database node is allowed to update inventory, while keeping browsing available. I add application redundancy, make retries return the original order, and measure the full checkout outcome. My recovery plan includes detection, failover, validation, and enough remaining capacity.”

Recall the key ideas

Answer from memory before opening each card. Explain why the choice works and what it costs. Revisit missed cards tomorrow.

01What is a database, data model, and transaction?

A database is an organized collection of related data. A database management system (DBMS) is the software that stores, retrieves and updates it. In everyday engineering conversation, “database” often refers to the combined system. Its data model determines whether the application thinks in tables, documents, key-value pairs, or relationships. An access pattern is a concrete query or update, such as “find recent orders for customer U7.” An invariant is a rule that must remain true, such as “available stock never becomes negative.”

A transaction groups database operations so their changes commit together or roll back together. ACID names four properties: atomicity, ACID consistency, isolation and durability. They describe what commits together, which rules remain valid, how concurrent transactions interact and which failures saved data survives. Check the database’s guarantees against your reads and writes.

One database is a sensible starting point. As the service grows, its limit might be storage, popular-item contention, history reads, or expensive analytics. The label SQL or NoSQL does not identify which limit we have. First write the questions and rules; then choose a model and implementation that support them.

Assume this purchase must create the order and allocate stock together or do neither. We have not yet introduced independent payment and warehouse services. Keeping inventory and orders in one database lets us explain a local transaction before considering a workflow that spans separate services. The quantities and prices are assumptions for this example.

Derive the model from the required operations: conditional stock allocation, order lookup by ID, and customer history ordered by time. In the example, order O81 contains two MUG9 items at an assumed $12 each. Creating the $24 order changes available stock from five to three. That joint state transition defines the required transaction boundary.

02Relational data: tables, keys, joins, and constraints

A relational database represents facts as rows in tables. Columns name fields and usually assign their types. Relationships connect records through keys. Separate shared inventory from each order’s agreed commercial terms.

OrderLine retains the purchase price if tomorrow’s catalog price changes. This deliberate duplication preserves history; eliminating every repeated field is not the goal.

SQL is a language for querying and changing relational data. A join combines O81 with its lines. An index beginning with customer and then creation order supports customer U7’s order history. Constraints such as a unique order identifier or a valid customer reference enforce specific rules within the database’s supported scope.

A useful history index is (customer_id, created_at DESC, order_id DESC), supporting a query shaped like SELECT ... FROM Orders WHERE customer_id = 'U7' ORDER BY created_at DESC, order_id DESC LIMIT 20. The final key breaks equal-time ties. It does not make an unrelated full-catalog search cheap. Store money as integer minor units or an exact decimal plus currency; the illustrative $12 unit price can be 1200 cents, and two units total 2400 cents.

03Key-value, document, wide-column, and graph models

The order, its line items and the customer relationship can be represented in several models. The choice changes which related facts are stored together and which queries need additional lookups or indexes. Compare each alternative against the same two needs: fetch O81 as a complete order and list U7’s recent orders.

A key-value store retrieves a value by a key, such as order:O81 → complete order data. That fits exact lookup. Listing every order for U7 needs another supported access path; one key does not automatically answer every question.

A document store can keep the order and its lines together: {id: O81, customer: U7, lines: [{sku: MUG9, qty: 2, price: 12}]}. This makes a complete-order read natural. Shared inventory remains separate because many orders refer to MUG9. Convenient embedding does not automatically make that cross-document rule atomic.

Wide-column systems organize application queries around partition keys, which select a group of records, and clustering keys, which order records within that group. They differ from analytical columnar engines that scan selected columns over many rows. Graph storage is useful when traversals are central, not merely because two records are related.

Choose by both fit and the operation that becomes awkward. A key-value layout needs a separate access path for customer history. A document layout makes one bounded order aggregate easy, but very large embedded arrays and shared inventory need another strategy. A wide-column layout favors planned partition-key queries and can concentrate a very large customer partition. A graph model makes multi-hop traversal expressive, but a relational foreign key alone does not justify adding a graph engine.

04SQL versus NoSQL: choose from workload and constraints

Schema is the agreed structure and meaning of records. A relational schema can enforce types and constraints. A flexible document schema can permit different shapes, but the application still needs rules for quantity, currency, and missing fields. Either model needs a compatible plan when old and new software versions coexist.

NoSQL describes a broad family, not one query language or consistency setting. Different systems expose document queries, keyed operations, graph languages, or SQL-like syntax. Some support multi-record transactions; MongoDB documents transaction support.

For this purchase, choose a relational database with appropriate transaction support. The reasons are the shared stock/order rule and useful history queries. We pay for indexes, contention on a hot product, and operating the database. We are not assuming that relational databases cannot distribute or that document stores cannot transact. If search or analytics requires another engine, treat it as a derived view of committed orders with a stated freshness delay and rebuild path.

05ACID: atomicity, consistency, isolation, durability

A transaction groups changes under specified guarantees. For O81, begin the transaction, reduce MUG9 stock by two only if at least two remain, insert the matching order and line, then commit. If a required step fails, roll back the transaction.

Keep the meanings separate

Rules involving several records may require stronger isolation or explicit locking. “ACID” does not mean every default isolation mode prevents every anomaly. The transactions-and-isolation chapter develops those traces; PostgreSQL’s isolation reference describes actual engine behavior. Application logic must still express the right invariant.

The transaction also records which logical purchase it is performing. purchase_key stays the same across retries; request_hash summarizes a consistently normalized request so the same key cannot silently mean different quantities or items. A row lock prevents competing updates to the same inventory row from proceeding simultaneously. That protection lets the database wait for an earlier updater and then test whether stock is still sufficient.

Here is the key part of a PostgreSQL-style transaction, assuming the tables and their uniqueness/foreign-key constraints already exist:

BEGIN;
INSERT INTO Orders
  (order_id, customer_id, purchase_key, request_hash, total_cents, currency)
VALUES ('O81', 'U7', 'purchase-71', 'hash-of-canonical-request', 2400, 'USD');
UPDATE Inventory
SET available = available - 2
WHERE sku = 'MUG9' AND available >= 2
RETURNING available;
-- Continue only if exactly one row was returned; otherwise ROLLBACK.
INSERT INTO OrderLine
  (order_id, sku, quantity, unit_price_cents)
VALUES ('O81', 'MUG9', 2, 1200);
COMMIT;

The comment is an application decision, not SQL that automatically aborts. Also enforce UNIQUE(customer_id, purchase_key) and a nonnegative-stock constraint. At PostgreSQL Read Committed, a competing updater waits for the row lock and rechecks its predicate against the updated row. Starting from two units, T1 changes 2 → 0 and commits; T2 then finds available >= 2 false and must roll back its transaction, removing the order it inserted earlier in that attempt. Starting from five, the single purchase changes 5 → 3. More complex multi-row rules still need the stronger strategy described above.

Claim the unique purchase key before allocating stock, as in this SQL order. A duplicate waits for the first transaction and recovers its existing outcome even if the successful purchase exhausted inventory. The new order remains uncommitted until all steps succeed; an insufficient-stock rollback removes it too. Performing the stock check first without resolving a prior purchase could incorrectly return out-of-stock for a retry of an already successful order.

06Unknown commits and changing transaction boundaries

Large images belong in storage suited to media bytes and delivery, with authoritative metadata references. Analytics can scan a derived store so monthly reports do not crowd out purchases. Add those paths when requirements and measurements justify their maintenance cost. Each derived view needs committed input, a freshness policy, and a recovery mechanism.

Two concurrency failures also require a fresh attempt. A serialization failure means the database cannot safely commit the attempted concurrent execution under its isolation rules. A deadlock occurs when transactions wait on one another’s held resources in a cycle; the database aborts an attempt to break that cycle. In either case, the application must reevaluate the purchase from fresh reads.

If the purchase key already exists, roll back the whole attempt. Then read the earlier order and its request hash in a new transaction. In this example, the duplicate is detected before stock is allocated. Return the saved result only if the normalized request matches. After a serialization failure or deadlock, retry the whole transaction with the same purchase ID and a retry limit, not just the final INSERT.

07Interview answer: choose a database for orders

Interviewer: “Would you use SQL or NoSQL for orders?”

Candidate: “I would list the queries and atomic rules first. The workload needs O81 by ID, recent orders for U7, and a purchase changing shared inventory from five to three while creating the matching order. A relational model with indexes and a suitable transaction is a straightforward starting point.

“A document can make complete-order reads convenient, but inventory is shared across many orders. I still need a supported transaction, or an explicit stock-reservation workflow, to coordinate that change with order creation. I would not claim one family always scales better or always lacks transactions. If history reads dominate later, I can add a read model. If inventory becomes an independent service, I must redesign the workflow.”

The answer connects a storage choice to the work the product performs and names what would make us reconsider. That is more useful than choosing from vendor slogans or treating a flexible schema as permission to skip data modeling.

Recall the key ideas

Answer from memory before opening each card. Explain why the choice works and what it costs. Revisit missed cards tomorrow.

01Database index: definition and tradeoff

A database index is a maintained data structure that maps searchable keys to records or contains data needed by a query. A key is the field or ordered combination of fields used for lookup, such as author and title. Think of a library catalog: to find books by Ursula Le Guin, you consult the author catalog rather than walking past every shelf. The catalog points to books; it is not a second copy of every page inside them.

Suppose our database has Book(id, author, title, publishedYear). Without a useful index, a query for one author may inspect every book row. With an author index, the engine can locate the relevant author entries and then fetch their rows. An index trades extra stored structure and write work for less work on selected reads.

02Worked example: author and title lookup

Consider a Book table containing the following four rows. We want to find Le Guin’s books and return them in title order:

The query is:

SELECT id, author, title
FROM Book
WHERE author = 'Le Guin'
ORDER BY title, id;

It returns IDs 11 and 13, in that order. Adding id makes the order deterministic if two books have the same title. For this example, assume an ordinary alphabetical collation; the database's configured collation determines the actual text ordering.

A simplified ordered index on (author, title, id) contains (Butler, Kindred, 12), (Butler, Parable..., 14), (Le Guin, A Wizard..., 11), and (Le Guin, The Dispossessed, 13).

1. The query asks for author = 'Le Guin' ordered by title and ID.
2. The database seeks to the first index entry with that author.
3. It scans the adjacent Le Guin entries in title order.
4. It reads rows 11 and 13 if the requested output needs fields unavailable from the index.
5. It stops when the author changes or the requested limit is met.

A seek navigates directly to a relevant key range; a scan then walks entries. Here the index has converted a whole-table search into a narrow seek and scan. If the table is tiny, a scan may still be cheaper; the query optimizer estimates these costs rather than treating any existing index as mandatory.

Selectivity describes how narrowly a predicate filters records. State the matched fraction to avoid terminology ambiguity: 100 matching rows out of one million is 0.01%, while 900,000 matches is 90%. The first query may avoid much table work with an index; the second may be cheaper as a sequential scan. Physical row placement, cached pages and which columns are returned still affect the decision. The mere existence of an index cannot establish the faster plan.

03B-tree, hash and inverted indexes

The book example needs both an author lookup and title ordering. Index structures organize searchable keys differently, so a structure that narrows an exact lookup may not support an ordered range or a word search. Compare the structures by how they reach the candidate records.

A B-tree index keeps search keys in sorted order inside a balanced tree of storage pages. It lets a database find a key without checking every row, and it can scan a consecutive range of keys.

To read the tree below, start with three terms:

1. A page is a block of data that the storage engine manages as a unit. One page can contain many keys.
2. The root is the entry page. Keys in internal pages act as signposts to the next page. For example, a separator at 40 sends a search for 50 to the side containing keys 40 and above.
3. A leaf is a page at the bottom. Balanced means every root-to-leaf path has the same number of levels. The example is a B+ tree, a common B-tree variant: its searchable record entries are in the leaves, which are linked for scans.

An equality query asks for one exact value, such as id = 42. A range query asks for values between bounds, such as years 2000 through 2010. A B-tree can answer both: descend to the first matching key, then follow the ordered entries if more matches are needed.

A hash index applies a hash function to a search key to choose a bucket, a group of candidate entries. Different keys can share a bucket, so the engine still checks which entry actually matches. Hash buckets group by hash value, not by the original key’s order; they do not naturally support scanning consecutive years.

An inverted index maps a term to the documents containing it. Its postings list contains document IDs and may also include counts or positions. For example, green → [D1, D3] means documents D1 and D3 contain “green.” It is called inverted because it goes from term to documents, reversing the document-to-terms view.

Interview tip: start with the query, then justify the index. Equality does not automatically make a hash index better than a B-tree; consider the database’s supported operations, measurements and other query needs.

An index does not necessarily sort the underlying table the same way. Some engines cluster table records around a primary key; others keep index entries separate from table pages. An index-only scan also depends on coverage and visibility rules in the chosen database.

Composite means the key contains several fields; it is not a competing tree algorithm. Covering means the index contains the fields needed by a query; it is not a separate universal storage structure.

04Composite indexes, key order and covering queries

The author/title example used several fields to group related records and order them within a group. Apply the same idea to customer order history: first isolate one customer, then return only that customer’s newest orders. Consider this query:

SELECT orderId, createdAt, total
FROM Orders
WHERE customerId = 'C27'
ORDER BY createdAt DESC, orderId DESC
LIMIT 20;

The leftmost prefix rule says that a composite B-tree index most directly supports lookups using its first column, or its first several columns together. It is a useful starting point for B-tree reasoning, not a universal claim that an engine can never use a later column. Optimizers may use skip scans or combine indexes depending on data distribution and implementation. In an interview, explain why your selected leading columns narrow the work directly, then inspect a plan in a real system.

A covering index includes fields needed by the query, such as total, to reduce row fetches where the engine allows it. The cost is a larger index and more updates when those fields change.

For PostgreSQL, the concrete candidate is:

CREATE INDEX orders_customer_newest
ON Orders (customerId, createdAt DESC, orderId DESC)
INCLUDE (total);

The query expression must match the access path too. An ordinary index on email does not provide the same ordered keys as lower(email). PostgreSQL supports an expression index on lower(email) when case-normalized lookup is the intended rule. That normalization has to match the product’s equality semantics; adding an index does not decide which spellings should count as the same address. See expression indexes.

05Index maintenance and write amplification

An index must stay consistent with changes to the records it describes. Write amplification is the additional physical write work created by one logical application change. Index maintenance contributes to that cost because changing one row can require updating several stored structures.

Insert book 15: (Le Guin, The Left Hand of Darkness, 1969). The database writes the row and adds entries to each maintained index: the primary-key index, author/title index, and perhaps a publication-year index. Updates of indexed fields remove or supersede old entries and install new ones; deletes must maintain the indexes too.

The engine also writes recovery logs. Index pages may split, use more cache memory and add disk writes. Ten indexes do not make every read ten times faster; a write affecting all ten must maintain ten extra structures.

Measure actual query use before removing an index: a rare month-end report or constraint may still depend on it. An index used to enforce uniqueness is part of correctness as well as read performance.

06Keyset pagination versus OFFSET

Pagination returns a bounded portion of a result instead of every matching row at once. After C27’s newest orders have been returned, the next request needs a continuation rule. An offset skips a count of earlier results; keyset pagination continues after the last ordering key that the client received.

For C27's next page, a cursor can encode the last seen (createdAt, orderId). The next query continues below that tuple in the same ordering. A cursor is a position in a chosen ordering, not necessarily a database transaction kept open between requests.

In a sharded database, first find the shard holding C27’s orders. A local index finds rows within that shard; it does not tell the client which shard to contact. Global searches need a distributed index or queries to several shards. Explain the API query, shard key and local index together.

For a compact example, use two rows per page instead of twenty. Assume createdAt is non-null and never changes, orderId is unique, and all four orders belong to C27:

After returning O402 and O400, the next PostgreSQL query is:

SELECT orderId, createdAt, total
FROM Orders
WHERE customerId = 'C27'
  AND (createdAt, orderId) <
      (TIMESTAMPTZ '2026-09-22 10:00:00+00', 'O400')
ORDER BY createdAt DESC, orderId DESC
LIMIT 2;

It returns O399 and O398. The strict tuple comparison handles the timestamp tie without repeating O400 or skipping O399. This assumes createdAt has type timestamptz and the ID comparison orders O399 before O400; use matching types and ordering in the real schema.

An order inserted with a newer timestamp belongs before this boundary and appears when the user refreshes the first page. A backdated insert may appear on a later page. That is why a stable cursor prevents shifts from newer inserts but does not freeze the dataset. See PostgreSQL's LIMIT and OFFSET documentation.

07Interview example: index customer order history

Interviewer: “How would you make customer order history fast?”

Candidate: “The request filters one customer and returns the newest twenty orders. I would use a composite index with customer first, then descending creation time and an order-ID tie breaker. The database seeks into that customer's range and reads a small ordered slice. A cursor carries the last timestamp and ID for the next page. I accept extra index writes and space, and verify the plan and latency using realistic customer sizes.”

This is more useful than saying “add a B-tree”: it explains which keys the tree contains and which work the query avoids.

A query plan describes the operations the database intends to use, such as an index seek, a table scan or a sort. Inspecting that plan tests whether the engine actually uses the access path the design relies on.

To verify the candidate in PostgreSQL, begin with EXPLAIN on the exact SELECT, including its filter, sort and limit. On a representative test workload, EXPLAIN (ANALYZE, BUFFERS) executes that query and reports actual work. Compare estimated and actual row counts, rows discarded by filters, sort work, buffers touched and table fetches. If estimates are poor, inspect statistics and skew before assuming another index is the answer. Repeat for a large customer and for cold versus warm cache conditions, then measure write cost. These observations test why the index helps; an “Index Scan” label alone is not a success criterion. See using EXPLAIN.

Recall the key ideas

Answer from memory before opening each card. Explain why the choice works and what it costs. Revisit missed cards tomorrow.

01Data model versus storage engine

Choose the logical key independently of the storage engine. A message record can be (room_id, sequence, author_id, body, version). Fetching the latest fifty messages in one room favors an ordered key beginning with room and sequence. For example, a request updates message 42 in room R7 from “Train at five” to “Train at six” and deletes message 8, whose old value is “Hello.”

One machine can store this correctly. It becomes slow when the active data no longer fits memory or disk work exceeds capacity. Before adding shards, understand which physical work each logical write creates. A write-heavy service can saturate its storage while the incoming request count appears modest.

02B-trees and B+ trees: ordered page lookup

A B-tree keeps keys ordered in a branching tree of pages. A page is a block that the engine reads or writes as a unit. Internal pages guide a search toward a child; leaf pages contain index entries, with the exact record layout depending on the engine. A B+ tree keeps record-bearing entries in leaves and supports walking adjacent leaves for ranges.

Imagine the root’s separators for R7 are message 20 and message 50. Looking up 42 follows the middle child to a leaf containing 21, 35, 42 and 49. If that index entry points to a separately stored row, fetching the message body is additional work. A latest-fifty query can seek near the end of R7’s range and walk backward through ordered entries.

Changing 42 updates the relevant data and index structures rather than scanning every message. A full leaf may split, requiring parent changes. Cached upper pages reduce physical reads, but cache misses, page splits, transaction versions, and recovery logging still matter. “Logarithmic lookup” describes growth; it does not specify a fixed number of disk operations for every product.

03Write-ahead logging: recovery and acknowledgment

A write-ahead log, or WAL, makes the recovery records for a change durable before the corresponding changed data pages are written to durable storage. That is the write-ahead ordering rule: the log reaches durable storage first. After a crash, the engine can reconstruct committed state from durable records and its persisted files. The exact protocol varies; a log is a recovery mechanism, not automatically an application event stream.

Assume the database acknowledges an edit only after its required recovery and commit records are durable under the configured local storage policy. At time 0 it logs message 42 version 2. At time 1 it acknowledges the edit. If the process crashes before the ordinary data page is flushed, recovery can replay the relevant durable information. If the service instead acknowledges only an in-memory buffer, the same crash may lose the edit.

State which failures each storage stage can survive:

Synchronization asks the storage stack to persist the necessary bytes; it does not make one local device indestructible. Group commit lets several transactions share one synchronization operation. This can improve throughput, but a transaction may wait for the group before receiving its acknowledgment.

04LSM trees: memory tables and immutable sorted files

A log-structured merge tree, abbreviated LSM, accumulates updates in a memory table and writes sorted immutable files as buffers fill. The WAL protects updates that have not yet become durable table files under the chosen configuration. Immutable means a later edit is stored as another version rather than rewriting that old file in place.

A read of message 42 must select the newest visible version according to the engine’s ordering and snapshot rules. It cannot stop at v1 merely because F1 was convenient to open. A read of message 8 encounters a deletion marker, often called a tombstone, which suppresses its older value. Range reads merge ordered streams from relevant files. They are supported, but their cost depends on how many streams and obsolete versions must be considered.

05Compaction, tombstones, and amplification

Assume an illustrative workload ingests 100 MB/s and the measured total local write amplification, including the log in this measurement, is 8. The device must sustain about 800 MB/s of writes, before adding other workloads or safety margin. This is arithmetic from assumed inputs, not a hardware guarantee. Compaction also consumes read bandwidth and CPU. Deferring it forever makes later reads and space usage worse.

Two common compaction policies move that cost differently. Leveled compaction limits overlap within deeper levels, usually reducing read and space amplification but rewriting overlapping data. Tiered compaction accumulates several sorted runs before merging them, often reducing write amplification while increasing read sources and temporary space. These are tendencies, not universal benchmark results: key order, skew, overwrite rate and tuning matter.

06Storage-engine comparison and row versus column layouts

Two different physical choices are being compared. B-trees and LSM trees organize key lookup and update work. Row-oriented and column-oriented layouts determine whether fields of one record or values of one field are stored together. These choices can be combined; select them from whether the workload fetches individual messages, scans room ranges, or analyzes a few fields across many messages.

For an assumed room-history workload dominated by appends and bounded room-range reads, I would evaluate an LSM-backed ordered store. The key (room, sequence) makes the common range explicit. I would benchmark it against an indexed relational design before assuming its extra operational complexity is worthwhile. The choice depends on latency targets, transactional requirements, updates, retention and operating experience.

A key-value API does not remove the need to design keys. Hashing every entire message key across shards scatters a room’s range; partitioning by room preserves locality but creates a hot partition for a huge room. Time buckets or subpartitions can bound growth at the cost of merging reads. Physical engine selection does not solve those ownership decisions.

Row-oriented storage places a record’s fields together, useful when fetching a message. Column-oriented analytical storage groups values by column, useful when scanning a few fields across many records. A wide-column database’s data model is not synonymous with a columnar analytics layout. Ask which query the layout accelerates rather than matching names.

A practical baseline is PostgreSQL with an ordered B-tree index for transactional room history. Evaluate RocksDB when the application needs an embedded ordered key-value engine and can own the surrounding service protocol; RocksDB alone is not a replicated database service. Its write options distinguish asynchronous WAL writes from synchronized writes. If an acknowledged edit must survive machine restart, verify that WAL is enabled and the required synchronization policy is applied rather than assuming the default write call provides it.

07Storage failure, recovery, and benchmarking

After a crash, check that every acknowledged change covered by the durability policy survived, including the new value of message 42 and the deletion of message 8. A deleted message disappearing from ordinary reads does not prove its bytes vanished from snapshots, old files, replicas or backups; physical erasure follows a separate retention and cleanup policy.

In an interview I would say: “The key supports room-history reads. An LSM may suit frequent appends, but edits and deletion markers leave versions that reads and compaction must resolve. I will state which failures saved messages survive, budget the extra reads, writes and disk space, and test range reads while background maintenance runs.”

Recall the key ideas

Answer from memory before opening each card. Explain why the choice works and what it costs. Revisit missed cards tomorrow.

01Load balancing: definition and purpose

Load balancing distributes connections or requests across eligible servers. A load balancer selects a backend for traffic addressed to one logical service. The client uses that service address; routing determines whether application instance A, B, or C receives the work.

Horizontal application scaling introduces multiple instances behind the same service address. A load balancer distributes work among them using a routing policy. Essential session state must be available to every eligible instance: rerouting a request cannot recover a cart stored only in a failed process.

02Layer 4 versus Layer 7 load balancing

“Layer” refers to the kind of network information the intermediary understands. Layer 4 (L4) is the transport layer: TCP/UDP connections or flows identified by network addresses and port numbers. A port identifies a service endpoint on a machine. L4 balancing commonly chooses a backend using this connection information. It can forward a connection without interpreting each application request. Layer 7 (L7) is the application layer. L7 balancing understands an application protocol such as HTTP, the request/response protocol used by web applications. It can route /images to an image service and /checkout to a checkout service, or use a hostname or header.

TLS termination means the encrypted client connection ends at the balancer, which can inspect the decrypted HTTP message. The balancer may then create a separately encrypted connection to the backend. If TLS passes through untouched, a transport balancer does not get the same HTTP routing information. Say where encryption ends and which network segments remain protected.

For GET /images/P7.jpg, an L7 rule can first choose the image pool; a balancing algorithm then chooses A or B inside that pool. Choosing the right service and distributing work among its instances are two separate decisions.

03Worked example: round robin, weights and least connections

After choosing the service pool, the balancer still needs a rule for selecting an instance. The main choices use a fixed schedule, a configured share of capacity, a measurement of current work, or a stable caller identity. Compare them by asking which signal best represents the work in this pool.

Assume A, B, and C are healthy and serve equal-cost short requests. Round robin visits them in order. Requests R1 through R6 go to A, B, C, A, B, C. Step one: R1 arrives and A is next. Step two: R2 advances the cursor to B. Step three: R3 advances to C. Step four: R4 wraps to A; R5 and R6 repeat B and C. Each server receives two requests, but equal counts imply equal work only under our equal-cost assumption. It is easy to operate, but it ignores work already running.

Now A has twice the tested capacity of B and C. A weighted schedule such as A, B, A, C repeats, giving A about half the requests and B/C a quarter each. Weights express capacity assumptions; they do not detect a new slow dependency.

For long-lived connections, suppose A has 10 active connections, B has 2, and C has 5. Least connections sends the next comparable connection to B. But if B's two connections each carry many expensive streams, the count may misrepresent actual load.

Implementations differ, so explain the signal instead of promising an exact universal algorithm.

Power of two choices reduces the need to compare every backend: randomly sample two eligible servers and choose the one with less measured work. With A=10, B=2 and C=5 comparable active connections, sampling A/C chooses C; sampling A/B chooses B. It need not find the global minimum to reduce imbalance. A least-request implementation counts active requests instead of transport connections, which can better match multiplexed HTTP work; neither count captures arbitrary CPU cost.

04Session affinity and persistent state

Affinity, or a sticky session, tries to send a caller back to the same backend. It can improve reuse of a local cache. IP hashing is one way; a routing cookie is another. If many students share one school's public IP through network address translation (NAT), IP hashing can concentrate them on one server.

Hashing a stable key can also place cached objects consistently. That is useful when the same key should reach the same owner, but the design must explain how keys move when servers join or leave and how it handles a key that receives unusually heavy traffic. A balancer cannot divide one expensive request simply by hashing it.

05Health checks and backend failover

At 12:00:00, B's process stops. A health check is a small probe used to decide whether B should receive new work. A readiness check asks whether it can serve new requests; a liveness check asks whether restarting the process might be necessary.

1. Requests already sent to B may fail before a health probe detects the crash. A health system does not make detection instantaneous.
2. After the configured failure threshold, the balancer removes B from new selection. A and C inherit its traffic.
3. Safe retries use a deadline and an operation identity where needed. Retrying a purchase blindly may duplicate it if B committed just before losing the response.
4. When B restarts, readiness remains false until required initialization completes. Reintroduce it gradually so a cold cache does not create a surge of database work.

A shallow probe can say “healthy” while every database query fails. An overly broad probe can remove all servers when one shared optional dependency fails. Design probes around the work each pool must actually serve.

Distinguish the source of health information. Active checks send dedicated probes even when no user traffic arrives. Passive checks infer trouble from real request failures, so an idle backend can remain untested. Thresholds reduce transient ejections but increase detection time. Neither proves future success, and an application error caused by the caller is not automatically evidence that the server is unhealthy.

06Connection draining, overload and balancer redundancy

Draining stops assigning new work while allowing existing requests to finish within a deadline. For a deployment, mark C unready, let short requests finish, then stop it. Long-lived sockets need a reconnect protocol or explicit migration; draining does not preserve in-memory conversation state by itself.

The balancer also needs a replacement if it fails. Active/passive keeps a standby and a way to redirect traffic; active/active runs several balancers. Cached DNS answers can delay redirection. Even a managed balancer needs enough surviving capacity for the failures you plan to tolerate. Existing TCP or TLS connections may break when their balancer fails: another balancer does not automatically inherit them. Clients therefore need reconnect and retry limits.

Interview answer: “For similar short HTTP calls I begin with weighted round robin over ready instances. I move essential session state out of individual servers. I calculate surviving capacity, drain during changes, and make retries safe. For long-lived or uneven work, I change the routing signal after measuring which resource is saturated.”

These policies become routing configuration in a proxy. In NGINX, an upstream group names the available backends, while proxy_pass forwards matching requests to that group. Weights and selection rules then control how the group distributes work.

A concrete starting implementation is an NGINX HTTP proxy with an upstream group, explicit backend weights and proxy_pass; choose least_conn when comparable active connections are a better signal. Its upstream module documents passive failure handling through max_fails and fail_timeout. Dedicated active HTTP checks require the documented health-check module/product support, so verify the installed edition rather than assuming all capabilities follow from the NGINX name. This configuration routes requests. The application and storage design must separately preserve carts, determine which database node may accept writes, and handle retries without losing or duplicating an operation. See the upstream reference and health-check guide.

Recall the key ideas

Answer from memory before opening each card. Explain why the choice works and what it costs. Revisit missed cards tomorrow.

01Caching: definition, hits, misses and TTL

Caching stores a reusable copy of data or a computed result so later requests can avoid a slower or more expensive operation. A cache entry is addressed by a cache key, such as product:P7. The authoritative store holds the record the application treats as the source of truth, such as the product database. A cache holds a copy. Its freshness policy states how old that copy may be, and its access policy states who may read it.

A hit means the cache has an acceptable entry. A miss means the entry is absent or cannot be used. A time to live, or TTL, is how long an entry may remain usable under the cache policy. A TTL is not the same as a guarantee that the underlying value cannot change.

Suppose P7 costs $20 and thousands of people view it each minute. Reusing a small product record can reduce database work. During checkout, however, the service must check which price applies and whether stock is available under the agreed purchase rules. The displayed cached value is not automatically permission to charge an old price or sell unavailable stock.

02Cache placement: local, shared, distributed and CDN

Memory (RAM) is fast temporary working storage; a disk retains bytes with a different access cost. An application can cache in its own memory, avoiding a network call. It can also cache on local disk: slower than RAM, but useful for larger reusable files. With several application servers, these local caches are separate. If the next request reaches another server, that server may miss or hold a different version.

A shared cache gives applications a common network-accessible cache. A distributed cache spreads that cache's keys across several machines. “Shared” describes who can use it; “distributed” describes how its capacity is placed. Neither word specifies durability or the freshness protocol.

A browser cache stores a user's copy. A reverse-proxy cache serves requests in front of the application. A content delivery network, or CDN, keeps copies at edge locations nearer users. For a product image, the first edge request misses and fetches the object from the origin, the server or storage service that supplies the original content; later allowed requests reuse it.

A separate hostname for static content, such as static.shop.example, makes a later CDN migration easier. Initially it serves image files directly. Later that hostname can point through a CDN while the object paths remain stable. Cache keys, certificates, cache headers, and private-content policy still need configuration; changing DNS alone does not define correct caching.

03Worked example: cache-aside read of P7

Cache-aside means the application manages cache lookup and refill. Start with a database record {id:P7, price:20, version:8} and an empty cache.

1. Request R1 reads P7. The application checks key product:P7 and misses.
2. It reads version 8 from the database, stores a copy with a 30-second TTL, and returns $20.
3. Request R2 reads P7 one second later. The same cache key hits; no database read is needed for that product-page request.
4. When the TTL expires, the next request needs a refresh. The cached copy did not update itself when time passed.

04Cache read and write policies

Cache placement answers where a copy lives. A cache policy answers who loads a missing copy and how a write reaches durable storage and existing cached copies. These are separate decisions: invalidation marks or removes a cached value so later readers cannot keep using it as current. For P7, the policy must explain what happens both when a page read misses and when the seller changes the price.

Read-through describes loading reads; it is not itself a write policy. Write-through can make the write path more explicit, but two independent stores are not automatically one atomic transaction. If the database commits a $25 price while the cache update fails, readers need invalidation, version checks, or a declared staleness limit.

For the public product page, use cache-aside: set a TTL and invalidate the cached price after a database update commits. Accept brief display delays. Checkout must check current price and stock in the purchase transaction. Write-back may suit disposable counters, but important orders need a way to survive cache failure before the service reports success.

05Cache invalidation and the stale-refill race

The seller changes P7 from $20 to $25. Simply deleting the cache after the database write can still race with an earlier reader:

If the business accepts up to a stated stale-display interval, a TTL may be sufficient under a specified refresh policy. If deletion or permission revocation must be immediate, verify current authorization rather than treating a stale cached record as truth. The interview answer should first state how stale a read may be and how quickly a permission change must take effect, then choose a mechanism that meets those requirements.

Suppose the database confirms version 8 at 10:00:00 with permission to reuse it until 10:00:30. A reader receiving it at 10:00:20 has only ten seconds left. Starting a fresh 30-second timer would incorrectly extend use to 10:00:50. Allow for clock differences. After expiry, obtain a newly validated value or return an error if validation fails. This limits age under the stated clock and database assumptions; it still allows stale reads before expiry and does not revoke access immediately.

Recovery also needs a way to distinguish fills started before a cache restart from fills started afterward. A cache generation is an identifier for one such cache lifetime. A refill carries the generation it started in; after recovery selects a new generation, the cache rejects results from the old one even if their requests finally resume.

06Eviction policies: FIFO, LRU, LFU and alternatives

Invalidation removes data because it is no longer acceptable. Eviction removes data because the cache needs space. A perfectly fresh entry can be evicted.

Compare the policies on one trace

Take a two-entry cache: insert A, insert B, read A, then insert C. Before C arrives, insertion order is A then B; access recency is B then A.

State the insertion convention when illustrating LIFO/MRU.

Real implementations may approximate these policies to save CPU and memory. An LRU cache can perform poorly during a large one-time scan because scan entries evict frequently reused data.

Implementation choice and data that must not be evicted

For a disposable shared product cache, one practical option is Redis with an explicit maxmemory limit and a measured choice between allkeys-lru and allkeys-lfu. Redis approximates these policies; LFU also decays old popularity. A volatile-only policy considers only expiring keys, so it is a different capacity policy. Keep durable business records and correctness metadata out of an indiscriminately evictable cache. See the Redis eviction reference; the application still owns freshness and origin-overload protection.

07Cache stampedes, negative caching and outages

When a popular key expires, 10,000 simultaneous readers may all miss and query the database. This is a stampede.

Negative caching stores a short-lived “not found” result to reduce repeated lookups of missing keys. It needs a short enough lifetime to let newly created records become visible, and must not reveal to an unauthorized user whether a private record exists.

08HTTP cache control and conditional revalidation

HTTP caches use response directives and validators to make reuse decisions. These are distinct from an application cache’s own TTL configuration.

A directive is an instruction carried in response headers, usually Cache-Control, about whether and how caches may reuse the response. A validator, such as an ETag, identifies a representation version. Revalidation asks the origin whether that cached version is still usable, which can avoid sending the full body again.

For example, an origin returns ETag: "v8". On revalidation the cache sends If-None-Match: "v8"; a 304 response confirms that its selected representation can be reused without resending the body. Vary identifies request headers that select different representations, such as language. It does not perform authorization. Cache directives do not recall bytes already downloaded or replace access checks. See RFC 9111.

Recall the key ideas

Answer from memory before opening each card. Explain why the choice works and what it costs. Revisit missed cards tomorrow.

01Proxy definition: forward versus reverse

A proxy is an intermediary that receives communication and forwards it on behalf of another party. A hop is one connection segment along that path. The same request can travel client → proxy → application, with a separate response returning through those components. The proxy can apply access or routing rules, cache a permitted response, or change how the next connection is made. A proxy adds a hop; it does not erase the need to understand the caller and the destination.

A forward proxy represents clients accessing remote services; an organizational gateway may filter destinations and record permitted outbound traffic. A reverse proxy fronts origin servers and routes incoming traffic to them. For example, shop.example can accept a public /orders/17 request and forward it to an internal order API.

The origin is the service responsible for producing the resource. A proxy may return a cached origin response when the policy allows. “Forward” and “reverse” describe whose side the intermediary serves, not whether packets travel only in one direction.

One deployed component can perform several of these jobs. Explain the responsibilities separately so a product name does not hide an authorization or failure assumption.

02Worked example: an HTTPS request through a reverse proxy

For an HTTPS request to https://shop.example/orders/17, several protocols have separate responsibilities. DNS maps a service name to a network address. HTTP carries the request and response. TLS encrypts a connection and authenticates its endpoint using certificates; HTTPS is HTTP over a protected connection. TLS termination is where that protected connection ends and the receiver can inspect its HTTP content. The following steps use those terms; protocol details are in the request-lifecycle chapter.

1. DNS resolves the public shop name to a reachable edge address.
2. The browser establishes an encrypted connection to the shop's reverse proxy and verifies its certificate for that name.
3. The proxy receives GET /orders/17, plus the session credential. That credential is evidence used to authenticate the signed-in account; the URL itself proves no identity. The proxy routes the request to the order service.
4. It opens or reuses a backend connection. If this crosses an untrusted network segment, encrypt and authenticate that hop too.
5. The order service derives the caller’s identity from a validated credential and checks permission to read order 17.
6. The response travels back through the proxy to the browser. Private order data is not placed in a broadly shared cache.

An API gateway is often a reverse proxy with additional API policy: authentication checks, quotas, routing, or request validation. A load balancer chooses among eligible backends. One product can perform both roles, but the responsibilities remain separate.

03Forwarding headers and trusted client identity

The backend connection originates at the proxy, so the backend's immediate peer address may be the proxy's address. Forwarding headers can carry earlier connection information. They must be trusted only from known proxy hops, because a public caller can forge ordinary request headers.

Suppose an attacker sends X-Forwarded-For: 127.0.0.1. A backend that treats that value as proof of an internal caller may grant unintended access. The edge should normalize forwarding metadata, and the backend should know which upstreams are authorized to supply it. Rewriting a header is a security-sensitive operation when policy depends on that field.

04Open, anonymous and transparent proxies

Forward and reverse describe whom a proxy represents. Open, anonymous and transparent describe other properties: who may use it, what identity information it reveals, or how traffic reaches it. These labels can overlap; choosing one does not answer the questions covered by the others.

An open proxy accepts use from a broad or unrestricted set of clients. A closed organizational proxy limits who may use it. Openness is an access-control property. It says nothing by itself about whether the proxy hides the client identity or inspects content.

An anonymous proxy attempts not to reveal some client-identifying information to the destination. That is not a promise of universal anonymity: accounts, cookies, behavior, or other headers can still identify a user. Explain the exact information hidden rather than using anonymity as a security guarantee.

“Transparent proxy” is overloaded. In common network terminology, a transparent or interception proxy handles traffic without explicit proxy configuration in the client. Older HTTP specifications also used transparent to mean that the proxy does not transform requests or responses beyond changes needed for proxy authentication and identification. These are different properties; name the intended meaning. Intercepting encrypted content requires an applicable trust and certificate arrangement; a device that only forwards encrypted bytes cannot arbitrarily inspect their HTTP content.

The useful interview distinction is role plus policy: who can use the proxy, which destination it represents, what it can see, and what information it forwards.

05Reverse-proxy caching and API gateway policies

A reverse proxy can cache public versioned images, compress permitted responses, terminate TLS, filter malformed requests, and route to service pools. For each feature, state which requests it applies to, what it may change, and its resource or security cost. Compression uses CPU; logging can expose sensitive data; transformation can invalidate signatures or cached representations if done incorrectly.

For the private order page, choose authorization-aware forwarding and an explicit cache policy. HTTP private prevents shared caches from storing the response but can permit a browser cache; no-store tells caches not to store it. Choose the policy required by the product instead of treating those directives as synonyms. For /images/P7/v9.jpg, a shared edge cache can reuse the same immutable public bytes. The first request misses and fetches the static origin; the next permitted request hits nearby storage. Different paths can therefore have different cache and authentication policies.

An API gateway may reject excess traffic before it reaches expensive application work, but it must identify users and quota dimensions correctly. Sending all traffic through a single unreplicated gateway creates a new failure point even if the applications behind it are redundant.

06Proxy timeouts, retries and redirect rewriting

A proxy can rewrite the path it forwards so a public URL maps to a different internal path. A redirect instead asks the client to make a new request, using the destination in the response’s Location header. The two mechanisms interact when the public and internal URL layouts differ.

Path rewriting also changes visible behavior. Suppose a public service lives under /store/ while the origin serves /. A redirect from the origin to /orders/17 can escape the public prefix unless the proxy rewrites the Location header or published links avoid the redirect. Verify both the origin address and the public path; success at one does not prove the other works.

Health-check and replicate the proxy layer, measure added latency and errors, and consider how configurations roll out. A malformed routing rule can fail every healthy backend at once. Keep configuration changes reviewable and validate representative paths, headers, uploads, and redirects.

Parsing must also agree across hops. Request framing determines where one HTTP message ends and the next begins. If a proxy and backend interpret ambiguous length information differently, they can disagree about which bytes belong to an authorized request. Prefer standards-compliant parsers, reject ambiguous framing under an explicit edge policy, and apply consistent path normalization before authorization and routing. HTTP/2 or HTTP/3 at the client does not remove this boundary when an intermediary translates to HTTP/1.1 upstream. See the HTTP/1.1 framing specification.

07Interview example: justify the proxy boundary

Interviewer: “Why do you need a reverse proxy if you already have an application?”

Candidate: “It gives the public service one controlled entry point for TLS and routing. For an order request, it forwards to an eligible order server, but the server still checks that the caller can read order 17. Public versioned images can be cached at the edge; private orders cannot share that policy. I also configure deadlines, safe forwarding headers, and redundant proxy capacity. The extra hop is useful because it performs these specific responsibilities.”

NGINX can implement this reverse-proxy role with explicit upstream routing and header policy. For a protected HTTPS backend, configure certificate verification, the expected backend name and the trusted certificate set; merely selecting an https:// upstream is not the complete identity check. The current proxy-module reference documents proxy_ssl_verify as off by default, so enable verification deliberately where required. The real-IP module separately controls which proxies may supply network-address metadata. Those options do not authenticate the application user. See the proxy module and trusted real-IP configuration.

Recall the key ideas

Answer from memory before opening each card. Explain why the choice works and what it costs. Revisit missed cards tomorrow.

01Partitioning and sharding: definitions

Data partitioning divides a dataset into smaller parts. Sharding is horizontal partitioning across separately managed storage groups. Horizontal means dividing records rather than splitting fields of each record. Each shard owns a subset of rows or records; ownership means that a designated storage group is responsible for those records and decides which writes it accepts. Replication instead creates copies of the same records. You can shard orders by customer and also replicate each shard; one choice divides ownership and the other protects each owner's data.

Suppose a shop stores one billion orders and its single database cannot meet the required storage or throughput. Before splitting, inspect inefficient queries and unused indexes; sharding adds routing, movement, and cross-shard complexity. When splitting is justified, choose a partition key: a field or combination of fields used to determine ownership.

Our example has customers C12, C27, and C44, and orders O1 through O6. Most screens ask for one customer's recent orders. That query suggests keeping a customer's records together rather than scattering every order randomly.

02Horizontal, vertical, functional and directory partitioning

Dividing a dataset involves two choices: what to separate, and how to find each part. Horizontal, vertical and functional partitioning describe what is separated. A directory describes how requests find the owner of a part.

Horizontal partitioning divides rows of the same kind: some customers' orders on shard A and others on B. Range partitioning, which groups rows by intervals of a key, is one horizontal method, not another name for all horizontal partitioning.