Tokens and usage

What counts as input and output, how to read the usage object, and what your credit is spent on.

Every model call reports the tokens it used in usage. Those counts are what your account is charged for, so the totals you log match what Avey records.

What counts

Input	Output
Messages, system prompt and `instructions`	The answer in `content`
The text of every file you reference	The reasoning in `reasoning_content`
A short instruction per user message when `reasoning_effort` is set	Tool call arguments Olive writes
Search results Olive reads during a file search

Reasoning is output. A request with a long reasoning trace and a short answer still uses many output tokens.

Read the usage object

{
  "usage": {
    "prompt_tokens": 1840,
    "completion_tokens": 612,
    "total_tokens": 2452,
    "prompt_tokens_details": {
      "cached_tokens": 1536
    }
  }
}

Chat completions	Responses	Meaning
`prompt_tokens`	`input_tokens`	Input tokens, cached ones included
`completion_tokens`	`output_tokens`	Output tokens, reasoning included
`total_tokens`	`total_tokens`	The sum
`prompt_tokens_details.cached_tokens`	`input_tokens_details.cached_tokens`	Input tokens served from the prompt cache

A field with a zero count can be left out. Read each one with a default of zero.

On a stream, usage arrives once, on the final chunk. See Streaming.

When a request starts with the same tokens as an earlier one, such as the same system prompt, guideline text or file, Olive reuses the work for that prefix. The reused tokens show in cached_tokens, and the request starts answering sooner. Cached tokens do not count toward your per-minute token limit.

The cache is per account. Another account sending the same prompt never hits your cache.

To get more cache hits, put what stays the same first (system prompt, reference text, files) and what changes last (the question). Keep reasoning_effort the same across a conversation: the effort instruction is written into every user message, so changing it changes the prefix.

Requests that make several model calls

A Responses request with tools can call the model more than once: Olive asks for a tool, reads the result, and answers. Each call is counted. The usage on the response is the sum over all of them.

Count before you send

POST /responses/input_tokens returns the input token count of a request, files included, without running the model. It is free. Use it to check a prompt fits the 65,536-token context window, or to warm a file you just uploaded.

What is not charged

A request rejected before the model runs, for a bad parameter, a rate limit or a key problem.
A request that fails on Avey's side.
Counting tokens with POST /responses/input_tokens.

A client that disconnects from a stream is charged for the tokens generated before it left.

Credit

Model calls are paid from your account's credit balance. The API platform shows the balance, adds credit, and lists usage per day and per request.

When the balance runs out, requests return 429 with "type": "insufficient_quota". Retrying does not help. See Rate limits.