Pricing and Costs

Pricing is the schedule of charges for a service; cost is the charge incurred by a measured workload. Total cost of ownership (TCO) includes the infrastructure, people and operating work within an explicitly stated accounting boundary. An API rate is one input to that calculation.

Rate-card review: September 24, 2026. Unless another unit is shown, table prices are USD per million tokens (MTok), using direct-provider Standard list rates. They exclude taxes, negotiated discounts, marketplace terms and regional/service modifiers unless stated. Current prices and announced future prices are separate. Workload sizes, staffing and comparison costs are interview assumptions.

Read tokenization, capability assessment and model selection alongside the calculations.

Token units and rates

A token is a unit produced by a model's tokenizer, not a fixed number of words. Two models can tokenize the same text differently. A price quoted per million tokens must be multiplied by the measured token count divided by one million. Input and output can have different rates, and the provider's usage fields determine which categories are billable.

Use invented round numbers to practice. At $2 per million input tokens, 2,000 input tokens cost 2,000 / 1,000,000 × $2 = $0.004. At $10 per million output tokens, 500 output tokens cost $0.005. The call costs $0.009 before tools, storage, or other charges. One hundred thousand such calls cost $900. That is a call forecast; if each user task makes five calls, the task volume is different.

Partition cached usage without double-counting

Suppose a provider reports 10,000 total input tokens, of which 8,000 are eligible cache reads. In an illustrative contract, uncached input costs $2 per million, cache reads cost $0.20, and 1,000 output tokens cost $10 per million. The input is 2,000 uncached tokens plus 8,000 cached tokens, not 10,000 uncached plus another 8,000 cached. The bill is $0.004 + $0.0016 + $0.01 = $0.0156 before any separate cache creation or storage charge.

Some providers report usage categories differently, so map the actual fields to non-overlapping billed quantities. Cache writes may be a separate category or priced under a documented multiplier. Reasoning usage may already be included in output billing. Adding it again overstates cost. The exact contract belongs beside the calculation, not in an assumption hidden inside a spreadsheet.

Provider prompt caching reuses eligible prompt processing. An answer cache returns a previous answer. The latter requires an additional correctness decision: do the query, user permissions, source versions, and freshness make reuse appropriate? Similar wording alone is not enough.

From a rate card to a workload forecast

Multiply by the distribution of tasks, not one unusually short example. Include tool calls, retries, escalations, long-context tiers, modalities, and service options that apply. Separate assumptions about present prices from announced future changes and from rates that could not be verified. A promotional price is not a permanent architectural guarantee.

Then add non-model costs: ingestion, embeddings, indexes, storage, network, observability, human review, and operating capacity. Define which costs belong to the business comparison. A self-hosted model's raw accelerator rental does not include all of those responsibilities, and an API's token bill does not include all application operations either.

The rate tables below are reference material. For an interview, memorize the arithmetic and the cost drivers. Explain which provider facts you would verify before making a purchasing or design decision.

Table of Contents

• What Appears on the Bill
• Current Text API Pricing
• Specialist APIs and Tool Fees
• Cost Calculation
• Worked Examples
• Context Caching Economics
• Cost Optimization
• Self-Hosting Economics
• Total Cost of Ownership and FinOps
• API versus self-hosted worksheet
• Forecast boundary cases
• Full FinOps design interview
• Interview questions
• Official Sources

What Appears on the Bill

A production request can create several independently billed quantities:

The general monthly model is:

$$\begin{aligned} \text{Monthly cost} = \sum_{\text{model calls}} (&I_uP_i + I_cP_c + I_wP_w + OP_o) \\ &+ \text{tool fees} + \text{media fees} + \text{storage fees} \end{aligned}$$

where token quantities are measured in millions and the non-token fees are monthly totals counted once. Media converted to billed model tokens belongs in the token term; use separate media fees only for distinct charges. Add serving infrastructure, observability, human review, support, and incident costs when comparing total cost of ownership.

There is no universal enterprise discount schedule. Volume discounts, committed capacity, support, regional routing, and cloud-marketplace terms are provider- and contract-specific. Use a written quote for a procurement model.

Current Text API Pricing

OpenAI text models

For these models, prompts above 272K input tokens use whole-request long-context rates: input, cache read and cache write are 2× the short-context rates; output is 1.5×. For example, GPT-6 Sol becomes $4 / $0.40 / $5 / $15 in the table’s column order. This is not a surcharge on only the excess tokens.

Batch and Flex use half the listed token rates; Fast uses twice Standard. Check availability as well as price: GPT-6 Astra, Sol and Luna support EU data residency only with Standard processing. Eligible regional processing adds 10%. GPT-5.6 Sol’s older promotional rate is guaranteed at least through November 21, 2026; that is not an announced future increase.

Astra requires reasoning; billed reasoning can exceed the visible answer. Its short-context input/output rates are 5× GPT-6 Sol’s, but different token consumption and retry behavior determine the actual task cost. OpenAI rate card.

Anthropic Claude

Cache reads are 2.5% of base input for Fable 5.1, 5% for Opus 5.5, and 10% for Sonnet 5/Haiku 4.5. Restricted-access Mythos 5.1 shares Fable’s rates. Do not apply one cache multiplier to every Claude model.

Batch halves input/output rates. Claude 4.6-and-later long context has no higher token tier. Opus 5.5 Fast costs $8 input / $40 output, is first-party-only and does not combine with Batch. US-only inference_geo on eligible models adds 10% to token categories. Newer tokenizers can change counts for the same text. Claude pricing.

Google Gemini

Google's output price includes thinking tokens. Batch and Flex input/output rates are half Standard for the models above; check their separate cache and storage columns rather than applying one blanket multiplier. Gemini 3.8 is the latest stable Flash; 3.7 remains stable at the same listed price. Paid-tier content is not used to improve Google's products; the free tier has different data-use terms.

gemini-3.1-pro-preview has a two-tier standard rate: $2 input / $12 output for prompts up to 200K tokens, and $4 / $18 above 200K. It is a preview model, so its lifecycle risk belongs in the decision alongside price.

xAI, Mistral, and DeepSeek

Grok 4.7 has a 500K context window. At 200K prompt tokens or more, its whole-request rates become $4 input / $1 cached input / $12 output. xAI's current rate card now publishes both tiers explicitly. The US regional endpoint adds 10% to token rates.

DeepSeek has separate peak and off-peak rates:

Off-peak is half of every listed rate. Peak windows are Monday–Friday 01:00–04:00 and 06:00–10:00 UTC, excluding Chinese public holidays. Legacy Flash names route to V4.1 Flash; V4 Pro remains separately listed. Record the actual version, billing window and usage. DeepSeek rate card.

Mistral figures above use its global Standard schedule; use the actual regional and service contract when budgeting. Mistral pricing.

Why the cheapest token row may lose

Rate tables do not normalize:

• provider tokenizers or media-to-token conversion;
• reasoning-token consumption;
• tool-schema and system-prompt overhead;
• successful-task rate, retries, or human correction;
• latency, capacity, and service tier;
• safety, governance, support, or data placement.

The decision metric should usually be cost per successful task at the required quality and service level, not cost per token.

Specialist APIs and Tool Fees

Do not compare unlike units as if they were all token prices.

GPT-Image-2.5 text input costs $5/MTok ($1.25 cached), separate from image tokens. Do not assume every image model supports Batch because an earlier model did. Use its explicit rate card and output settings to estimate each image.

The Videos API and Sora 2 models have a September 24, 2026 shutdown date and are excluded from new-workload budgets. A historical rate does not establish availability. OpenAI’s self-serve fine-tuning platform is also winding down and closed to new users; do not assume a current text model supports new fine-tuning jobs. OpenAI deprecations.

Embedding bills cover embedding computation, not the vector database, index rebuilding or document ingestion. Training/adaptation budgets additionally need data preparation, label review, training runs, evaluation and serving; see fine-tuning.

Common tool charges can be large enough to change model routing decisions:

OpenAI notes that eligible containers are billed per minute with a five-minute minimum, not necessarily one full 20-minute unit. Check actual session usage.

X Search is now billed by fetched items: $5/1,000 posts and $10/1,000 profiles. Parent and quoted posts count. It is no longer a flat per-call charge. xAI tool pricing.

Google Search grounding provides 5,000 free search requests per month shared across Gemini 3.x models on the paid tier, then charges $14 per 1,000 search queries. One customer request may trigger multiple queries, so log the billed query count. Other providers also charge for some server-side tools; verify the specific tool version and deployment path. Google pricing.

Cost Calculation

Use the provider's usage fields for metering and reconcile them with the invoice. Normalize provider-specific fields into non-overlapping billed categories; some APIs report cached input as a subset of total input. Keep rates in a dated configuration instead of hard-coding them throughout the application.

from dataclasses import dataclass
from decimal import Decimal


@dataclass(frozen=True)
class TokenRates:
    input_per_mtok: Decimal
    cached_input_per_mtok: Decimal
    cache_write_per_mtok: Decimal
    output_per_mtok: Decimal

    def __post_init__(self):
        for value in vars(self).values():
            if not isinstance(value, Decimal) or not value.is_finite() or value < 0:
                raise ValueError("rates must be finite non-negative Decimals")


def token_cost(rates, *, uncached_input=0, cached_input=0,
               cache_write=0, output=0, extra_fees=Decimal("0")):
    if not isinstance(rates, TokenRates):
        raise ValueError("a validated rate card is required")
    counts = (uncached_input, cached_input, cache_write, output)
    if any(type(value) is not int or value < 0 for value in counts):
        raise ValueError("token counts must be non-negative integers")
    if (not isinstance(extra_fees, Decimal)
            or not extra_fees.is_finite() or extra_fees < 0):
        raise ValueError("fees must be a finite non-negative Decimal")
    return (
        uncached_input * rates.input_per_mtok
        + cached_input * rates.cached_input_per_mtok
        + cache_write * rates.cache_write_per_mtok
        + output * rates.output_per_mtok
    ) / Decimal("1000000") + extra_fees


GPT_6_SOL = TokenRates(*(Decimal(x) for x in ("2", "0.20", "2.50", "10")))
cost = token_cost(GPT_6_SOL, uncached_input=2_600, output=300)
print(f"USD {cost:.6f}")  # USD 0.008200

This function covers token categories and explicitly supplied fees. It does not infer long-context tiers, service modifiers, cache-storage duration, media conversion, taxes, or contract terms. Select the correct rate card before calling it. Create decimals from strings, accumulate sub-cent usage, and round at the documented billing boundary. A production meter also needs bounded inputs, currency, immutable rate revisions and reconciliation records.

For an agent, calculate every turn and tool invocation. In this formula, model-call cost excludes the separately summed tool fees:

$$\text{Agent-run cost} = \sum\_{t=1}^{T}\text{model-call cost}\_{t} + \sum\_{j=1}^{J}\text{tool cost}\_{j}$$

Also record abandoned runs. Excluding failures makes the cost per successful task look artificially low.

Worked Examples

RAG chatbot

Assume each request has 2,600 uncached input tokens and 300 total billed output tokens, including any billed reasoning tokens. There are no cache, tool, retry, or service-modifier charges. Actual reasoning runs may use much more output.

This is arithmetic, not a model recommendation. If a lower-rate model needs more retries, produces more output, or fails the quality threshold, its cost per accepted answer can be higher.

Document summarization

For 8,000 uncached input tokens and a 500-token output using GPT-6 Sol:

$$(8{,}000 \times 2 + 500 \times 10) / 1{,}000{,}000 = 0.021\ \mathrm{USD}$$

That is $21 for 1,000 documents or $210 for 10,000 documents, before storage, retrieval, retries, and review.

Tool-using agent

Assume one run makes eight GPT-6 Sol calls totaling 20,000 uncached input tokens, 25,000 cached input tokens, and 4,000 output tokens, plus five OpenAI web-search calls:

At 1,000 runs per day, the illustration is $4,050 for a 30-day month. It excludes the initial cache write, search-content tokens, containers, failed runs, and any regional or service-tier modifier.

Context Caching Economics

Caching is valuable when a sufficiently large prefix repeats within the cache lifetime. It is not automatically valuable merely because a prompt is long.

For a reusable prefix of one MTok, let:

• $B$ be the ordinary input price;
• $W$ be the cache-write price;
• $R$ be the cache-read price;
• $N$ be total uses, including the initial write.

Ignoring storage fees, caching wins when:

$$W + (N-1)R < NB$$

when $B > R$:

$$N > \frac{W-R}{B-R}$$

For GPT-6/GPT-5.6 and Sonnet/Haiku 5-minute caches, $W=1.25B$ and $R=0.1B$. Fable/Mythos 5.1 use $R=0.025B$; Opus 5.5 uses $R=0.05B$. In these cases, the 1.25× write pays back on the second total use: one write plus one read. Claude's 1-hour cache uses $W=2B$ and pays back on the third total use. These conclusions assume full-prefix hits, no storage fee, and reuse within the actual provider TTL; OpenAI's cache duration is not Claude's five minutes.

Google's explicit caching also charges token-hours of storage, so add:

$$\text{cached tokens in MTok} \times \text{hours stored} \times \text{storage rate}$$

Measure actual cache-hit tokens, reuse count, TTL expiry, prefix churn, and the cost of cache misses. Put stable instructions and shared documents before request-specific data when the provider's caching rules are prefix-based.

Cost Optimization

Optimize only after measuring quality and usage. The highest-leverage controls usually are:

1. Measure cost per successful task. Join usage, tool fees, retries, human review, latency, and outcome quality by request or agent run.
2. Shorten what is repeatedly sent. Remove redundant instructions, trim tool descriptions, summarize state, and retrieve only relevant context.
3. Cache stable prefixes. Validate the realized hit rate and include cache writes and storage in the calculation.
4. Use Batch or Flex for deferrable work. OpenAI, Anthropic, and Google publish 50%-lower token rates for eligible asynchronous/deferred paths, but their latency and feature contracts differ.
5. Route with evaluations. Send easy work to a lower-cost model only after measuring routing errors, escalation frequency, and cost per passing result.
6. Bound agents. Cap model turns, output, reasoning effort, tool calls, elapsed time, and dollars per run. Detect repeated state and tool loops.
7. Use specialist endpoints. OCR, transcription, moderation, embeddings, and media generation often have better pricing and behavior than a general model forced into the task.
8. Control failure amplification. Use bounded retries with jitter, idempotent tools, circuit breakers, and failure-aware fallbacks.
9. Clean up billed storage. Expire unused vector stores, files, caches, and container sessions according to retention policy.

Avoid promising a fixed percentage saving from routing, caching, quantization, or prompt compression. The realized saving depends on traffic and quality.

Self-Hosting Economics

There is no universal query-count or GPU-utilization threshold at which self-hosting becomes cheaper. Compare equal-quality systems under the same latency, availability, safety, and governance requirements.

Capacity model

A first approximation is:

$$\text{monthly token capacity} = \text{tokens/second while serving} \times \text{serving duty fraction} \times \text{seconds/month}$$

Here the first rate is measured during active serving and the duty fraction is the portion of wall-clock time serving this workload. If your rate is already averaged over the full interval, do not multiply by duty fraction again. GPU utilization is not the same as useful token throughput.

Benchmark the exact model, quantization, prompt-length distribution, output length, batch size, tensor parallelism, and serving stack. Prefill and decode have different bottlenecks; one average tokens-per-second number can hide tail latency and concurrency failures.

Reserved, serverless and interruptible capacity

No option has infinite burst capacity, and no universal 40% utilization crossover exists. Moving work to a cheaper region may violate residency requirements or cost more after egress and model loading. Use interruptible capacity for restartable work only when the recovery plan meets the deadline.

Memory floor: 70 billion parameters at two bytes each require about 140 GB (130.4 GiB) for weights alone. A nominal four-bit representation has a 35 GB (32.6 GiB) weight floor before scales, metadata and unquantized layers. Add KV cache, activations, runtime buffers and fragmentation. In a mixture-of-experts model, fewer active parameters per token do not mean only those weights must be stored. Benchmark the complete deployment; see quantization.

Full self-hosted TCO

Include:

• accelerator and CPU hours, including idle, failover, and capacity headroom;
• storage, networking, load balancing, orchestration, and observability;
• inference-server and model licenses;
• quantization or fine-tuning work and quality regression testing;
• security patching, abuse controls, incident response, and on-call labor;
• deployment, autoscaling, upgrades, and model migrations;
• downtime, capacity shortages, and disaster recovery.

A defensible break-even condition is:

$$\begin{aligned} \text{API cost at required quality and SLO} &> \text{self-hosted monthly TCO at the same quality and SLO} \end{aligned}$$

Use current quotes for the intended hardware, provider, region, commitment, and network path. A public on-demand GPU price from another region is not a reliable production estimate.

Self-hosting can still be preferable when weight access, customization, data placement, or operational control matters more than the raw cost comparison.

Total Cost of Ownership and FinOps

FinOps is an operating practice for making technology spending accountable to business value through collaboration between engineering, finance and business teams. It includes allocation, forecasting and optimization; reducing spend alone is not the goal. FinOps Foundation framework.

Track the economic unit the product actually sells or operates:

$$\text{Cost per successful task} = \frac{\text{model + tools + infrastructure + human + incident costs}} {\text{successful tasks}}$$

Useful dimensions include tenant, feature, environment, model and version, prompt version, service tier, region, cache status, tool, success result, and retry/fallback reason.

Operational controls should include:

• budgets and alerts by tenant, feature, model, and environment;
• per-request usage and estimated-cost logs reconciled with invoices;
• anomaly detection for token growth, cache misses, loops, and retry storms;
• quotas and graceful degradation instead of one global hard stop;
• canary and shadow-traffic budgets;
• a dated rate-card registry with an owner and review cadence;
• invoice reconciliation for rounding, free allowances, taxes, and negotiated terms.

Forecast from the joint workload distribution, preserving relationships between long inputs, output length, tool use and retries. Report P50/P95 for planning, but do not multiply independent P95 values and call the result the P95 bill. Segment totals and replayed scenarios give a more defensible forecast. Run sensitivity analysis for price changes, growth, cache-hit rate, model migration, and quality regression.

Manager interview practice

Mental arithmetic: 2,000 input tokens at an illustrative $2/MTok plus 500 billed output tokens at $10/MTok costs $0.004 + $0.005 = $0.009 per call. Five such calls cost $0.045 before tools, retries, and review. These are invented rates to practice units, not a provider quote.

Recall checks: Are cache writes being counted twice? Does output already include reasoning? Which context/service tier applies? Are failed attempts in the numerator? What happens to unit economics if review doubles?

Define the successful-outcome denominator with product and the accounting boundary with finance. Maintain a rate version and forecast range rather than a permanent hard-coded model price.

Hypothetical API versus self-hosted TCO worksheet

Assume one million completed tasks/month of equal accepted quality. For teaching arithmetic only, suppose the API costs $0.008/task for model calls and the self-hosted serving fleet can support this workload at its measured latency target. These are invented rates, not current vendor prices.

At 500,000 tasks, assume API model spend halves while its other lines stay fixed: $10,000 total, or $0.020/task. If the self-hosted fleet cannot shrink, $21,000 becomes $0.042/task. At two million tasks, the API becomes $22,000. Self-hosting is $0.0105/task only if the existing fleet can actually handle that demand and failure headroom; otherwise add the required capacity. The apparent break-even under fixed non-model assumptions is (21,000 − 6,000) / 0.008 = 1.875 million tasks/month, conditional on that capacity.

Idle and reserved capacity are already paid for in the GPU lines; do not add “idle cost” a second time. Measure utilization to explain why cost per task rises at low volume. Add one-time migration, compliance, tooling, and retraining costs over an explicit amortization period when relevant. For unequal quality, compare total cost per verified outcome rather than equal raw request counts.

Forecast the workload and test the boundary cases

A rate change can affect the whole request. With no cache or other fees, GPT-6 Sol at 272,000 input tokens and 1,000 billed output tokens costs 272000 × 2/1M + 1000 × 10/1M = USD 0.554. At 272,001 input tokens, the long-context rates apply: 272001 × 4/1M + 1000 × 15/1M = USD 1.103004. One added token crosses a billing boundary; do not estimate this as one extra token at the old rate.

For caching, use the strict inequality in the earlier section. At illustrative rates B=2, W=2.50, R=0.20 per MTok, a reusable one-MTok prefix costs:

These are invented rates for isolating cache arithmetic, not a one-MTok short-context provider quote. Add storage and rewrite costs when they apply. If B ≤ R and W ≥ B, caching cannot reduce this token bill; the break-even expression requiring B > R is not applicable.

Traffic mix example: suppose 80% of tasks cost USD 0.005 and 20% cost USD 0.050, including their own calls and tools. Mean variable cost is 0.8 × .005 + 0.2 × .050 = USD 0.014, so 100,000 tasks cost USD 1,400. If the expensive slice rises to 40%, cost becomes USD 2,300, an increase of about 64.3%, with no rate or volume change. Preserve this correlation in the forecast.

Interview: design a cost-control service

Scope: build metering, budgets and allocation for an application using several model APIs. This is an internal cost ledger, not a payment processor. The provider’s invoice remains the reconciliation authority for its charges.

Functional requirements

1. Attribute each model/tool execution to an authenticated tenant, product feature, environment and task.
2. Select a versioned rate contract by provider, model, service, region and effective time.
3. Reserve budget before admitting work and settle against observed usage.
4. Include retries, failed tasks, storage, evaluations and human-review allocations in reports.
5. Reconcile delayed usage and invoices without silently rewriting history.

Non-functional requirements

1. Concurrent workers cannot each spend the same unreserved balance.
2. A duplicated usage event cannot charge the internal ledger twice; distinct provider attempts remain distinct.
3. Missing usage stays unknown and visible; it does not become zero after a deadline.
4. Metering records exclude prompt bodies and credentials; access is tenant-scoped.
5. Target p95 admission overhead below 20 ms and 99.9% admission-service availability at the illustrative peak below; validate both under hot-tenant load. Define bounded queues and behavior during ledger outages.

Initial design: each worker reads a tenant balance from a cache, calls the model, then subtracts an estimate. This is easy to prototype. Two workers can both read the same balance, a crash can lose usage, and a stream can end before final counters arrive. A nightly spreadsheet discovers overruns only after spending occurs.

Detailed design

flowchart TB
    U["Authenticated task and bounded execution plan"] --> A["Admission service: tenant budget + policy"]
    R[("Versioned rates and billing contracts")] --> A
    A --> B[("Atomic reservation and task ledger")]
    B -->|Reservation committed| E["Execute bounded model / tool attempt"]
    E --> P["Provider / external tool"]
    P --> M["Usage adapter: disjoint units + actual tier"]
    M --> Q["Durable usage event stream"]
    Q --> S["Idempotent settlement / adjustment"]
    S --> B
    E -->|Outcome or usage unknown| H["Hold reservation; reconcile with provider"]
    H --> S
    I["Invoice / usage export"] --> C["Reconciliation and discrepancy review"]
    C --> S
    B --> D["Tenant allocation, forecasts and alerts"]
    D --> O["Reviewed policy / routing change"]
    O --> A

Data and API contract

Concurrency example: the limit is USD 10, settled spend USD 9.50 and existing reservations USD 0.10. Available balance is USD 0.40. Two workers each ask for USD 0.30. An atomic conditional update admits one; the other must be rejected or delayed. A read-then-write cache cannot enforce this invariant. Use a transaction or equivalent atomic conditional operation for the authoritative ledger. In a multi-region design, route a tenant budget to its authority or allocate bounded regional budgets; asynchronous replication alone can overspend a shared limit.

Settlement example: reserve USD 0.30, then observe USD 0.18 actual usage. Settle USD 0.18 and release USD 0.12. If the request outcome is unknown, retain the reservation until evidence resolves it. A lease expiry establishes worker ownership, not that the provider did no work. Stop further expensive actions when evidence is missing or the next action cannot be funded. Refunds or billing corrections are new adjustment entries, not deletion of an inconvenient charge.

An estimate is a strict spend cap only when every possible charge is bounded by the execution contract. Enforce maximum input/output, loops, tools, duration and approved services, and reserve conservatively for unknown cache hits. External pricing, delayed records or unbounded tools can invalidate that guarantee. Model cancellation can reduce further work; it does not undo completed billing. Keep a separate documented response for an actual charge exceeding the reservation.

Capacity: assume 100 admitted tasks/second at peak and three model attempts/task. This creates 300 attempts/second, before retries or shadow work. At least one reservation and one settlement mutation per attempt gives roughly 600 ledger mutations/second, plus events and reconciliation. Size partitions and indexes from the busiest tenant, not just total traffic. At nine million monthly attempts and an illustrative one-kilobyte usage record, raw records are roughly nine GB/month before replicas, indexes and event envelopes. Keep prompt traces under a separate privacy/retention policy.

Full cost and benefit calculation

For a smaller product example, assume 100,000 tasks/month. These rates and workloads are invented:

xychart-beta
    title "Illustrative monthly total cost"
    x-axis [Baseline, Proposal]
    y-axis "USD per month" 0 --> 25000
    bar [20200, 22600]

The model/tool bill falls 50%, yet TCO rises USD 2,400/month because reviewer work and new operating costs outweigh the saving. If both workflows produce 95,000 verified accepted outcomes after review, cost per 1,000 accepted outcomes rises from about USD 212.63 to USD 237.89. Equal accepted quality is an assumption to verify, not a consequence of human review.

Holding other lines fixed, the proposal breaks even at 5,200 reviewed tasks, or a 5.2% review rate. At 5%, it costs USD 19,600 and saves USD 600/month. That narrow margin needs production validation. The meter may still be justified for attribution and budget control even when a particular model-routing change fails its economic test; evaluate those decisions separately.

Closing remarks: “I would launch the meter with accurate attempt identity, atomic reservations and explicit unknown usage, then reconcile it with invoices. I would approve routing changes only after matched-quality tests show a reduction in full cost per outcome. Our example shows why token savings alone are insufficient. I would watch reviewer workload, hot-tenant admission latency and unresolved charges, with an owner for each discrepancy and a reversible routing policy.”

Interview questions with developed answers

Q1: How would you optimize a high-volume RAG application's cost?

Q2: When would you recommend self-hosting instead of APIs?

Q3: How do you avoid double-counting cache and reasoning tokens?

Q4: Why can a cheaper per-token model be more expensive overall?

Q5: How do you budget when provider prices change?

Q6: The API reports 10,000 input tokens including 8,000 cached tokens. What is billed?

Q7: One extra prompt token nearly doubles an example bill. Is that a calculation bug?

Q8: A prefix is used once before expiry. Does a 90% read discount save money?

Q9: Why can two small requests exceed a tenant’s remaining budget?

Q10: A stream disconnects without final usage. What cost should the report show?

Q11: A usage stream reports cumulative counts 100, 250 and 400. Is the total 750?

Q12: Model and tool spending falls by half. Why might finance reject the optimization?

Q13: When do nine million usage records require more than nine GB?

Q14: A serverless GPU is cheaper per hour. Why can a reserved fleet still win?

Q15: How would you forecast a bill after the traffic mix changes?

Final recall table and notes

Interview tip: state units aloud, show one calculation and then challenge its assumptions. Keep quoted prices separate from workload assumptions. A rate table answers “what is charged”; a forecast answers “what will this workload cost”; a TCO comparison answers “which acceptable system is worth operating.”

60-second interview answer

I price the whole task, including every model turn, cache category, tool call, retry, and human review. I use the rate card for the actual model, context tier, service tier, region, and contract, then reconcile estimates with billed usage. I compare cost per successful outcome at the required quality and latency. The cheapest token price can lose if it needs more calls or creates more rework. I forecast a range and name the assumptions that would change the decision.

Remember: Rate × measured usage + tools + rework + operations.

Official Sources

Next checks

Text and specialist entries above were checked September 24, 2026:

• OpenAI API pricing
• Anthropic Claude API pricing
• Google Gemini Developer API pricing
• xAI API pricing
• Mistral API pricing
• DeepSeek models and pricing

Next: Model Selection Guide