How to Calculate AI API Costs

The basic formula

For each request, multiply input tokens by the provider’s input rate and output tokens by its output rate, then divide by the provider’s pricing unit—commonly one million tokens. Multiply the per-request amount by request volume for daily or monthly estimates.

Input and output rates are often different, so averaging all tokens into one number produces misleading forecasts. Keep the two sides separate.

Model real traffic

Averages are useful but hide expensive tails. Split traffic into a few representative groups, such as short interactive requests, document analysis, and long-form generation. Estimate tokens and volume for each group, then combine the totals.

Include retries, tool-call messages, system prompts, structured-output repair attempts, and background evaluations. These are easy to omit and can become material at scale.

Caching and batch discounts

Some providers offer lower rates for eligible cached input or asynchronous batch processing. Apply discounted rates only to the share of traffic that actually meets the provider’s rules. Cache hit rate should be measured, not assumed.

Operate with guardrails

Track actual usage by feature, environment, customer, and model. Set alerts for unusual token growth and compare invoices with internal telemetry. Revisit the forecast whenever prompts, context limits, model routing, or provider prices change.