AI Architecture Pattern Reference

A design pattern is a reusable approach to a recurring problem under stated conditions. It gives you a starting structure and known tradeoffs. It does not establish that your application needs the pattern or that a particular implementation will work.

This Learnastra reference connects requirements to choices you can defend in an interview. Follow a link for the full mechanism, implementation considerations and practice. Reviewed September 24, 2026.

Pattern selection guide

1. Define the result: what must the user be able to do, and what would count as success?
2. Set constraints: latency, quality, availability, privacy, cost and supported load.
3. Draw a baseline: start with the smallest design that can meet the requirements, including a non-AI approach when appropriate.
4. Identify a measured or reasoned failure: missing evidence, stale data, queue growth, unsafe action or excessive cost.
5. Add one justified mechanism: explain what it fixes, what it costs and what new failure it introduces.
6. Close with verification and operations: test the claim, define rollback and name who handles unresolved outcomes.

flowchart TD
  R[Requirements and constraints] --> B[Simplest viable baseline]
  B --> E[Measure representative tasks and load]
  E --> F{Which constraint fails?}
  F -->|Missing evidence| D[Retrieval and data quality]
  F -->|Invalid actions| A[Authorization and workflow controls]
  F -->|Slow or expensive| C[Capacity, caching or routing]
  F -->|No constraint fails| K[Keep baseline and monitor]
  D --> V[Evaluate benefit and new risks]
  A --> V
  C --> V
  V --> E

Recall: requirement → baseline → failure → change → evidence. This is a study aid, not a named industry standard.

Retrieval patterns

flowchart LR
  Q[Question and authenticated identity] --> P[Resolve scope and policy]
  P --> L[Lexical candidates with ACLs]
  P --> V[Vector candidates with ACLs]
  L --> F[Fuse and deduplicate]
  V --> F
  F --> R[Optional bounded reranker]
  R --> X[Fetch current authorized passages]
  X --> G[Generate with provenance]
  G --> C[Check citations and claim support]
  C --> O[Answer, clarify or abstain]

Interview tip: ask where the failure occurs. Improving generation cannot recover a document that was never indexed. Adding a reranker cannot fix an unauthorized cache hit. Trace ingestion, retrieval and synthesis separately.

Generation patterns

For a current model, confirm supported combinations in the model-selection guide. Do not assume every API accepts temperature, manual reasoning budgets or forced tool selection in every mode.

Agent patterns

stateDiagram-v2
  [*] --> Ready
  Ready --> Decide: budget and deadline available
  Decide --> Validate: proposed action
  Validate --> AwaitApproval: approval required
  Validate --> Escalate: action rejected
  AwaitApproval --> Validate: exact action approved
  AwaitApproval --> Escalate: denied or expired
  Validate --> Execute: authorized, valid and required approval current
  Execute --> Record: confirmed outcome
  Execute --> Reconcile: timeout or lost response
  Reconcile --> Record: outcome resolved
  Reconcile --> Escalate: cannot resolve safely
  Record --> Ready: more work required
  Record --> Complete: completion checks pass
  Decide --> Escalate: no safe next step
  Ready --> Escalate: budget or deadline exhausted
  Complete --> [*]
  Escalate --> [*]

For a refund, bind approval to customer, order, amount, currency and operation version. Recheck authorization before execution. An expired approval or changed amount returns for review. An unknown payment result enters reconciliation; it is not silently converted into a second refund request.

Agentic coding patterns

Choose editor, CLI, hosted worker or SDK based on the workflow. The tool comparison explains the boundaries. No single product name establishes autonomy, repeatability or safe execution.

Reliability patterns

A deadline example: an interview requires a complete response within 5 seconds. Reserve 0.2 seconds for admission/routing, at most 2.5 seconds for a primary attempt, up to 0.2 seconds for backoff, at most 1.5 seconds for a compatible fallback and 0.3 seconds for final validation/return. That is 4.7 seconds, leaving 0.3 seconds of headroom. These are allocated limits, not a prediction of p95 latency. Shorten or skip a step if the actual remaining deadline is insufficient. Do not give each nested retry a fresh 5-second budget.

Caching patterns

Hit rate comes from the workload and key design; cache categories do not have inherent “low,” “medium” or “high” hit rates. A 50% hit rate can be harmful if a small fraction of hits are wrong. Measure valid-hit rate, invalidation lag and the consequence of stale answers.

Tip: start with safely reusable deterministic work when evidence supports it. Semantic response caching is often inappropriate for personal balances, mutable entitlements and irreversible decisions without additional validation.

Security patterns

Evaluation patterns

Cost optimization patterns

Worked decision: cheaper model routing

Functional requirements

1. Classify and answer 100,000 support requests per month.
2. Escalate cases outside the model's approved scope to a person.
3. Preserve the conversation and resolution outcome for review.

Non-functional requirements

1. Meet the same task-quality and data-handling criteria as the baseline.
2. Keep the agreed interactive latency target on each important traffic slice.
3. Reduce total monthly cost without exceeding available reviewer capacity.

Baseline: use one qualified model for every request, then review 2% of requests. Candidate: a router sends 60% to a smaller model and 40% to the baseline model. The following are hypothetical loaded costs, not provider prices or measured outcomes.

The flaw: model and router spend falls by $3,400, but additional review and operating costs make the candidate $800 more expensive. Review workload rises from about 133.3 to 200 hours per month. The team must account for that additional capacity even if headcount cannot change immediately.

Repair: restrict small-model routing to validated categories. Re-evaluate quality, traffic shares, review rate and latency; do not assume the original 60/40 traffic split survives. If that split does remain and review returns to 2%, the candidate totals $15,000, saving $2,200 per month. With all other assumptions fixed, break-even review workload is about 2,733 requests, or 2.73%. Any routing-induced harm not captured by review also belongs in the decision.

Closing: proceed only if a held-out evaluation and limited rollout support the revised assumptions. Retain the baseline route for rollback, log the policy version and compare cost per correctly resolved request. A lower token bill is evidence about one line item, not the final business case.

Anti-patterns to avoid

Interview practice

1. Reranker or hybrid retrieval? A relevant part number never appears in the vector candidate set. Start by improving candidate recall, for example lexical retrieval and fusion; a reranker cannot score a missing candidate.
2. RAG or fine-tuning? A policy changes daily. Retrieve a versioned authoritative policy; fine-tuning is not a dependable daily fact store.
3. Agent or workflow? Every invoice follows four known approval states. A workflow with bounded model extraction is a strong baseline; justify any model-selected transitions.
4. Retry or reconcile? A refund request times out after transmission. Resolve its status using the same operation identity before initiating another effect.
5. One model or two? The current model already meets quality, latency and availability targets. A second model needs evidence of incremental benefit exceeding cost and complexity.
6. Cache or recompute? Two users ask the same question but have different permissions. Similar text is insufficient; use permission-scoped validity checks or recompute.
7. Parallel agents or sequential work? Two changes modify the same schema and depend on its final form. Resolve that dependency and ownership first; parallel work is not automatically independent.
8. Human approval or automation? An action is approved, then its amount changes. Invalidate approval and seek authorization for the actual action.
9. More samples or a better verifier? Five answers repeat the same unsupported claim. Inspect shared evidence and the acceptance test before buying more votes.
10. Higher throughput or lower latency? A larger batch improves GPU utilization but grows the queue. Measure the end-to-end latency distribution under the arrival pattern, not only kernel throughput.
11. Cheaper inference or cheaper service? Inference savings cause more correction work. Include review, rework, incidents and operations in the comparison.
12. Better benchmark or better product? A public score rises while domain performance falls. Prefer the application's representative held-out tasks and required slices for that deployment decision.
13. More logging or better debugging? Full prompts expose customer secrets. Preserve scoped identifiers, versions, timings and safely retained samples needed for diagnosis.
14. Longer context or better retrieval? Relevant evidence is buried among distractors. Compare both on matched tasks and budgets; capacity alone is not evidence-use quality.
15. Timeout or cancellation? The caller returns an error while a worker continues. Persist job state, stop further scheduling where possible and reconcile any external action already issued.

Final summary and notes

End an interview by naming the chosen design, the assumption most likely to fail, the metric that reveals it and the safe fallback. Continue with the complete pattern lesson, anti-pattern lesson or whiteboard interviews.