How to Calculate AI API Token Costs: Input, Output, Cache

AI APIs are rarely priced as a flat amount per question. Every request is metered in tokens, split between input and output at different rates, and some models add separate prices for cache reads, cache writes, images, or tool use. Cost control therefore starts in an unglamorous place: taking your real requests apart and calculating what each component costs — not hunting for the lowest headline number on a pricing page.

The Basic Formula

The core calculation for a single request:

request cost = (input tokens  / 1,000,000) x input price per 1M
             + (output tokens / 1,000,000) x output price per 1M

Input and output are billed at different rates, and output is typically the more expensive side. The classic mistake is applying the input rate to all tokens, which understates real cost significantly for generation-heavy work.

Here is the same arithmetic as a reusable function, with deliberately hypothetical example rates:

def request_cost(input_tokens, output_tokens,
                 input_rate=1.0, output_rate=5.0):
    """Rates are per 1M tokens. Example rates only —
    always read current prices from Model Hub."""
    return (input_tokens / 1_000_000) * input_rate \
         + (output_tokens / 1_000_000) * output_rate

print(request_cost(20_000, 2_000))  # 0.03 at the example rates

Notice the shape of that result: at these example rates, 2,000 output tokens cost a third as much as 20,000 input tokens. Rates change over time and vary by model, so read current per-model prices from Model Hub rather than carrying numbers from old articles in a spreadsheet.

What Counts as Input

Input is far more than the last sentence the user typed. A single request can include:

• System instructions
• The full conversation history
• Uploaded or retrieved documents
• Repository files read by Claude Code
• Tool results fed back into the model
• Context resent on every retry

This is why typing "continue" deep in a long session costs many times what it costs in a fresh one: the model must be shown enough prior context to continue correctly, and every one of those tokens is billed as input again.

What Counts as Output

Output covers generated prose, code, and some tool-call arguments. Asking for complete files, exhaustive explanations, or many alternatives inflates output tokens directly — at the higher rate.

Effective output control is specific, not a vague "be brief":

• Return a patch or diff instead of restating the whole file
• Cap the number of table columns or list items
• Ask for the conclusion first, with detail only on request
• Require structured JSON for batch jobs, which trims filler automatically

How Prompt Cache Changes the Math

Some models and protocols support prompt caching: a repeated prefix — a long system prompt, a shared document — is written to cache once, then billed at a lower cache-read rate on subsequent requests. The first write may carry its own price.

Caching pays off when all four hold:

If the prompt opening changes every request, or a job runs once, caching adds a write cost and returns nothing. Current support and rates are documented in the billing guide.

Why Long Context Blows Budgets

A large context window means the model can accept more material — not that every request should carry all available material. The recurring offenders:

• Resending an entire repository on every request
• Never trimming conversation history
• Duplicated retrieval results stacked into the prompt
• Raw logs and documents uploaded without filtering

A cheaper pattern: run a low-cost model such as gemini-3.5-flash to classify or summarize first, then pass only the relevant sections to the target model. Since a gateway key covers both tiers, this is a routing decision rather than an integration project — the cost optimization guide treats this tiering in depth.

Estimating a Monthly Budget

Build the estimate from real samples, not one optimistic request:

1. Pull 20 to 50 representative calls from your usage log
2. Compute average input and output tokens separately
3. Multiply by expected requests per day
4. Multiply by working days per month, then add headroom for retries and growth

monthly tokens = average tokens per call x calls per day x usage days

Then convert with current rates from Model Hub. For teams, split the estimate by member, project, and key, and give each key its own cap — a runaway test script should hit its own limit, not drain the team balance. Teams supports exactly this: shared balance, independent per-member keys, and multi-dimensional budgets.

Usage Data Beats Estimates

Estimates set the range; production data sets the priorities. Review weekly:

• Which model consumes the most
• The input-to-output ratio per workload
• The fastest-growing key or member
• Unexpected retry storms
• Premium models quietly handling trivial tasks

Then fix the largest line item first. If repeated context dominates, invest in retrieval and caching. If output dominates, tighten response formats. Fixing the biggest term of the equation always beats shaving every prompt by five percent.

FAQ

Q: How do I know how many tokens my text uses? A: The API response reports exact input and output token counts per request, and the usage log aggregates them. As a rough planning figure, one token is around four characters of English text, but always calibrate against your own logged requests.

Q: Do failed or retried requests cost money? A: A request the model actually processed bills its tokens even if your application discarded the answer, and each retry resends full context as fresh input. Retry storms are one of the most common surprise line items — watch for them in the usage log.

Q: Is cheaper input or cheaper output more important for me? A: Check your ratio. Context-heavy workloads like coding agents are dominated by input cost; generation-heavy workloads flip the balance toward output. The ratio decides whether caching or output formatting is your bigger lever.

Q: Does a gateway change how tokens are counted? A: No. Token accounting matches the underlying model; what changes is the unit price and having one consolidated usage log across 100+ models — which is exactly what makes this kind of analysis practical.

Wrap-Up

There is no universal token cost — only your prices multiplied by your usage. The reliable loop: confirm current rates in Model Hub, compute average per-call cost from real requests, route simple work to cheap models, trim repeated context and padded output, cap every key, and review weekly. Create a key to get a live usage log to calculate from; the current accounting rules are always in the billing documentation.