Design a multitenant SaaS platform

01Make tenant scope the first architectural boundary

A multitenant SaaS serves several customer organizations on shared infrastructure. A tenant is one such organization. Sharing servers does not mean sharing permissions or letting one customer consume everyone’s resources. Ask whether users may belong to several organizations and whether ordinary customers can share storage. Here they can, while selected large or constrained customers may receive dedicated placement.

T7 and T8 each own invoice 17. The local invoice number is therefore not the full record identity. A request authenticates user U7, verifies permission to act for T7, resolves T7’s placement, and reads (T7,17). A cache hit, export or download must preserve the same scope; checking SQL alone is insufficient.

The smallest complete flow is authenticate → authorize active tenant/action → execute scoped query → return only that representation. Begin with one pooled application/database. Add a cell, an independently operated slice of compute and data serving a bounded tenant set, when capacity or failure isolation justifies it. Dedicated infrastructure can isolate resources, but it cannot repair an application that trusts an attacker-supplied tenant field.

02Functional requirements

Agree on what the service must do before choosing its components.

1. Manage organizations and members. Support tenant/member lifecycle with users permitted to belong to several organizations; authorize the active organization and action on every operation.
2. Manage invoices. Create, read and update tenant-scoped invoices with safe retries and conflict handling. Cross-tenant reporting or sharing needs an explicitly privileged or consented path; ordinary invoice APIs never accept a wildcard tenant.
3. Run bounded exports. Create asynchronous exports, inspect their progress and download only authorized results. Revalidate the initiating actor before reading and exposing an export; removing a member must not leave a perpetual grant.
4. Manage tenant resources. Apply quotas, expose usage and place eligible customers in shared or dedicated capacity according to their constraints.
5. Operate the tenant lifecycle. Move, restore and delete one tenant through scoped, audited operations. Permit an explicit maintenance window during cutover rather than promising uninterrupted movement.

03Non-functional requirements

Use these hypothetical requirements for the worked interview. Confirm the assumptions with the interviewer; the numerical targets require testing and are not measured results. For latency, p95 and p99 mean that 95% and 99% of measured delays, respectively, are no greater than the reported value.

1. Workload. Assume 10,000 tenants with 100 users each, ten percent concurrently active and five requests/minute per active user. The resulting estimate is about 8,333 requests/s, with a fivefold peak near 41,667/s; skew matters more than the average tenant.
2. Latency and availability. Target invoice-read p95 below 200 ms, write p95 below 400 ms and 99.95% availability per cell. A cell is an independently operated slice of compute and data. Measure tenant cohorts so a global average cannot hide one customer’s outage; explicitly account for the agreed migration maintenance window.
3. Logical isolation and current permission. A caller cannot read or mutate another tenant’s rows, cache entries, jobs or files. Operator tools require scoped authorization and audit too.
4. Resource and placement isolation. One tenant must not exhaust shared database, CPU, queue or connection capacity. Meet its agreed region, key, administration and recovery constraints. Separate schemas alone do not reserve resources.
5. Durability and recovery. Acknowledged writes survive the selected database-node failure under the actual replication policy. For recovery from backup after a larger loss, the ordinary tier is separate: assume at most 24 hours of recoverable data loss and restoration within four hours for a 1 GB tenant; larger or stricter tenants need separately agreed recovery targets. Verify a tenant restore in an isolated environment, not by overwriting the pooled database.
6. One tenant history. Movement must not leave two active writers. Lost-response retries retain their original operation identity, and a client that just created an invoice reads from a path that includes that commit.

04Prove one pooled application before distributing it

The baseline has one application, relational database, scoped cache and private object store. A read authenticates U7, checks invoice-read in T7, looks up the T7 cache key, and on a miss executes the parameterized composite-key query. Authorization is required on both paths; a cache must not bypass it.

Invoice-creation transaction

1. Initialize scope. Begin a transaction and initialize trusted tenant context.
2. Check retry identity. Look for the scoped replay key.
3. Validate the relationship. Check customer (T7, 4).
4. Save and commit. Insert invoice (T7, 18) and its replay result together, then commit.

A database foreign key guards the relationship in addition to application checks.

Row-level security can enforce tenant policies in the database as defense in depth. Use an appropriately constrained application role, not a table owner or bypass role, and transaction-local context so pooled connections do not retain the previous request’s tenant. Arbitrary privileged SQL is outside that protection.

An export worker loads J8’s durable context, rechecks current permission, executes bounded scoped pages and writes an immutable T7-owned object. Completion records the exact object version. Download authorization happens again when requested. These checks establish logical isolation before any discussion of schemas, cells or dedicated databases; moving an insecure query into another topology would merely move the leak.

05Size for expensive tenants and operations

Assume 100 users/tenant, ten percent simultaneously active and five requests per active user/minute. That gives 100,000 active users and about 8,333 requests/s, with a fivefold peak near 41,667/s. The mean tenant uses only 0.83 requests/s, but one tenant producing 20% of base traffic already contributes about 1,667/s.

A hundred cells with about 100 tenants each would average 417 peak requests/s per cell. That count is a placement starting point, not a load-balancing proof. Use observed CPU, query cost, storage and batch demand; a single large tenant may exceed an otherwise ordinary cell.

Assume an invoice read uses 2 ms of database CPU. An export scanning one million rows at 100 microseconds each uses 100 CPU-seconds, equivalent to 50,000 such reads. Charging each operation one request token misses this disparity. Bound active exports, query time, scanned/output bytes and connections, and reserve interactive capacity.

At 1 GB of customer data per tenant, logical storage is 10 TB, or at least 30 TB for three copies before indexes/logs/backups. Moving a 1-TB tenant needs another copy plus change-log headroom. A20-MB/s write stream produces 72 GB during an hour-long copy; the destination must replay faster than arrival before the final pause can be short.

06Keep requested scope separate from trusted scope

Read an invoice

GET /tenants/T7/invoices/17

The URL names T7; it does not prove U7 belongs to T7. Middleware verifies membership and permission for the action, then passes the verified tenant and caller to downstream services. Those services validate this context instead of trusting a tenant header supplied by the user.

Create an invoice

POST /tenants/T7/invoices

Save the replay result with the invoice transaction. A duplicate identical request returns the same invoice; a conflicting payload under that key fails. Update APIs use expected row versions to prevent lost edits.

Record relationships and cache identity

Export, file and pagination records

07Repair noisy-neighbor and failure-scope limits

First introduce fair tenant scheduling and separate interactive/batch budgets. The trigger is a tenant submitting thousands of expensive exports while ordinary reads wait. Fair queues and per-tenant active-job limits preserve useful service; the cost is scheduler state and sometimes idle reserved capacity. Adding application replicas without database budgets can worsen contention.

Next introduce cells when one pooled database’s capacity or failure impact becomes too large. A tenant directory maps T7 to cell C2, region, tier and placement version 8. Each cell has its own compute and replicated data capacity, so many failures and rollouts affect a bounded customer set. Shared identity and directory services still require their own availability design.

Dedicated placement becomes justified by sustained resource pressure or explicit residency, key, recovery or administrator constraints. Separate schemas primarily organize namespaces; they do not reserve CPU. Dedicated databases and compute add stronger administrative/resource boundaries at higher fleet and utilization cost.

Keep the same application contract across pooled and dedicated placements. A cell should not become a special bypass of tenant authorization. Movement, restoration and deletion are controlled lifecycle operations with durable state and audit, not ad hoc routing edits or unrestricted copies between databases.

08Trace scoped reads, writes and exported bytes

U7’s create request resolves T7’s current cell and supplies the original operation key. The cell checks that T7 is active there, initializes trusted tenant context, and commits the invoice with its replay result. A placement change returns a retriable error so the router can refresh its directory entry. Reusing the key avoids creating a new invoice after a lost response.

A read resolves the placement, authorizes the exact object and returns only scoped fields. For the agreed read-your-write requirement after invoice creation, use the primary or a replica known to include that commit. A random lagging replica can make the new record appear missing. This is why the agreed contract cannot treat all replicas as interchangeable.

Export workers carry durable tenant and actor context. Before reading and exposing results, they recheck the current permission and placement. Their database work uses the same resource limits and scoped access as ordinary requests.

For downloads, choose an authenticated gateway that checks current tenant membership and permission for the immutable object version before serving a new request. This matches the baseline’s repeated download authorization. A short-lived presigned URL is an alternative with a weaker expiry-bound contract: possession may permit access until expiry after membership changes. A tenant prefix or a user name embedded in a transferable URL does not supply that enforcement.

09Move a tenant with an explicit maintenance pause

The chosen interview design permits a maintenance pause when moving a tenant. Transfer T7 in this order:

1. Freeze the source. Stop T7’s mutations at C2.
2. Drain admitted work. Wait for already admitted writes and background updates to finish.
3. Copy stable data. Transfer the now-stable tenant state.
4. Verify the destination. Check C5 before activation.
5. Switch authority. Change the directory and enable T7 at C5.

Reads can continue from the frozen source if the product accepts that view.

Every writer must encounter the freeze check in storage or in a shared write path. Removing a frontend route is insufficient: old workers and cached routes may still reach C2. Keep C2 write-disabled after the move, so a stale request gets a placement-change error instead of changing the old copy.

Copy invoices, related rows, completed operation keys and durable jobs. Verify scoped counts, checksums and relationships before activation. A copied replay result lets a client retry an invoice whose successful response was lost. The directory changes only after the destination is ready.

For a large tenant, the pause may be too long. A future refinement copies a consistent snapshot while the source remains active, then catches up its changes before a short final freeze. That requires a reliable change log and recovery protocol. Present it as an extension; the simple design makes a longer, explicit availability tradeoff instead of promising zero-pause movement.

10Check writes around the maintenance boundary

Follow one invoice through the maintenance pause. If U7’s write was admitted before the freeze, the move waits for it to commit or abort. A committed invoice and its replay result are therefore present in the stable copy. If the request arrives after the freeze, C2 rejects it without a mutation. The client later retries against C5 under the same operation key.

This reasoning depends on including background writers. An export-completion update or administrative import cannot continue changing T7 behind the maintenance boundary. The freeze is a tenant lifecycle state enforced by the mutation path, not a flag that only the public API happens to check.

Record movement progress durably so a coordinator restart can inspect what completed. When activation is uncertain, keep the source frozen and investigate or finish the destination transition. A timeout does not prove that the destination accepted no writes. Reopening both sides would exchange a visible outage for divergent customer records.

Once C5 accepts writes, pointing the directory back to the old copy would lose those changes. Recovery must repair C5 or perform another controlled transfer. Full coordinator fencing and continuously replicated migration are advanced protocols. The interview design accepts a recoverable maintenance window and establishes one active writer before it claims service is restored.

11Test isolation where shortcuts usually hide

Create a fixture with T7 and T8 both owning invoice 17 and customer 4. Exercise it through real cache keys, pooled connections, background jobs, object access and operator tooling. Test forged tenant headers, missing SQL scope, retained connection context and canceled/revoked exports. A helper-unit test cannot cover the full path.

Measure tenant/cohort latency, database time, scanned/output bytes, active jobs, queue age, denied scope violations, migration lag and epoch mismatches. Control metric-label cardinality while preserving authorized investigation paths for one tenant. Global averages should not hide a starved customer.

For the ordinary backup tier, take at least daily recoverable backups and measure the four-hour restore goal on a 1 GB tenant. Restore a tenant in a separate environment, validate scope and import deliberately; restoring the pooled database in place overwrites other customers. Deletion covers rows, derived indexes, caches, exports and documented backup treatment. Backups must retain placement fences and replay outcomes, not merely invoice content.

Roll schema changes cell by cell with compatible readers/writers and stop on failures. Drill a paused write during migration, reverse the ordering, then crash the coordinator during the freeze, copy and activation stages. Compare destination results and current authority before reopening traffic. The next economic decision is whether observed tenant skew or policy requirements justify a dedicated cell, not whether a customer carries an “enterprise” label.

12Check the design against its requirements

Before closing, check the final design against the agreed requirements. FR means functional requirement and NFR means non-functional requirement; the numbers refer to the lists above. These are proposed validation checks, not test results.

13Rapid revision

Rehearse the numbered functional requirements and non-functional targets first. Use this table to recall the mechanisms, then close with the requirements check above.

Remember: Same invoice number; different tenant ownership.

Close with: “I pool ordinary tenants while enforcing trusted scope end to end. Fair budgets protect shared resources; cells and dedicated placement follow measured needs. During a move, every writer respects the source freeze, and the destination activates only after a verified copy. I accept a maintenance pause to keep one authoritative tenant history.”

Recall the key ideas

Answer from memory before opening each card. Explain why the choice works and what it costs. Revisit missed cards tomorrow.