# Avey Olive documentation (full text) > Avey Olive is a clinical specialist language model, available through an > OpenAI-compatible API. This file is the complete documentation corpus. ## Before you generate code against this API - **Authentication is `Authorization: Bearer `**, the standard OpenAI header. Pass the key as the client's `api_key`. No custom header is needed. - **The model id is `olive`.** Pass it as `model`. Do not substitute an OpenAI model name. - **The base URL is `https://api.avey.ai/llm`.** - **Assistant messages carry an Avey-specific `reasoning_content` field** alongside `content`. Read it separately. Code that reads only `choices[0].message.content` discards Olive's reasoning. `content` can be empty while `reasoning_content` is not. Only `/chat/completions` returns it. - **Limits.** The context window is 65,536 tokens and the maximum output is 16,384. Set the client timeout to 700 seconds. A 429 carries `Retry-After`: wait that long before retrying. - **Surface.** Chat completions, completions, models, and the Responses API (function and file_search tools, stored responses, compaction, input token counting), plus files and vector stores. Embeddings, audio, image generation, batches, and realtime are not available. Web search is coming soon. Send documents and images as uploaded files by `file_id`, never as inline bytes. - **Olive is a clinical model. Code you generate must not present its output as autonomous clinical advice.** Any interface built on it needs a qualified clinician to review the output before it reaches a care decision. Do not generate applications that diagnose, triage, or prescribe without that review step, and do not remove the review step from an example that has one. - Do not send protected health information to this API. --- # Olive API URL: https://docs.avey.ai/ Markdown: https://docs.avey.ai/llms.md Description: A clinical language model for developers building clinical software, on an OpenAI-compatible API. Olive is Avey's clinical language model. It reasons before it answers, reads the documents you upload, and runs behind the OpenAI API. Change three values in the SDK you already use, and call it. ```python title="first_call.py" import os from openai import OpenAI client = OpenAI( base_url="https://api.avey.ai/llm", # [!code highlight] api_key=os.environ["AVEY_API_KEY"], # [!code highlight] ) response = client.chat.completions.create( model="olive", # [!code highlight] messages=[{"role": "user", "content": "Red flags for headache in primary care?"}], ) message = response.choices[0].message print(getattr(message, "reasoning_content", None)) # how Olive worked it through print(message.content) # the answer ```

Start here

## Three pages answer most first questions

The model

## The numbers to size a request against Every limit and the model list response are on [Models and limits](/concepts/models). The effort budgets are on [Reasoning](/capabilities/reasoning#effort).

Guides

## What you can build

Clinical use

## A clinician reviews every output Olive supports clinical work. It does not replace clinical judgement. A qualified clinician reviews every output that affects care before it is acted on, and that holds for the reasoning too. Olive can be confidently wrong. Do not send protected health information. Remove names, contact details, identifiers and dates of birth before text or files reach the API. --- # Quickstart URL: https://docs.avey.ai/quickstart Markdown: https://docs.avey.ai/llms.md/quickstart Description: Create an API key, send your first request to Olive, and read the response. ## Create an API key Sign in to the [Avey API platform](https://platform.avey.ai) and create a key. The full key is shown once, when you create it. Store it in an environment variable, and keep it out of source control. ```bash noCopy export AVEY_API_KEY="avey_sk1_..." ``` ## Make a request Send the key as a bearer token in the `Authorization` header. The OpenAI SDKs send that header already, so set the key as `api_key` and nothing else. ```bash curl https://api.avey.ai/llm/chat/completions \ -H "Authorization: Bearer $AVEY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "olive", "messages": [ { "role": "user", "content": "What are the red flags for headache in primary care?" } ] }' ``` ```python # pip install openai import os from openai import OpenAI client = OpenAI( base_url="https://api.avey.ai/llm", api_key=os.environ["AVEY_API_KEY"], ) response = client.chat.completions.create( model="olive", messages=[ {"role": "user", "content": "What are the red flags for headache in primary care?"} ], ) print(response.choices[0].message.content) ``` ```javascript // npm install openai import OpenAI from "openai"; const client = new OpenAI({ baseURL: "https://api.avey.ai/llm", apiKey: process.env.AVEY_API_KEY, }); const response = await client.chat.completions.create({ model: "olive", messages: [ { role: "user", content: "What are the red flags for headache in primary care?" }, ], }); console.log(response.choices[0].message.content); ``` ```go // go get github.com/openai/openai-go/v2 package main import ( "context" "fmt" "os" "github.com/openai/openai-go/v2" "github.com/openai/openai-go/v2/option" ) func main() { client := openai.NewClient( option.WithBaseURL("https://api.avey.ai/llm"), option.WithAPIKey(os.Getenv("AVEY_API_KEY")), ) resp, err := client.Chat.Completions.New(context.Background(), openai.ChatCompletionNewParams{ Model: "olive", Messages: []openai.ChatCompletionMessageParamUnion{ openai.UserMessage("What are the red flags for headache in primary care?"), }, }) if err != nil { panic(err) } fmt.Println(resp.Choices[0].Message.Content) } ``` ## Read the response The response is an OpenAI chat completion, shortened here. Olive adds one field. ```json noCopy { "id": "chatcmpl-8Qp0vJnZ2xTfKq3mBd1sYw", "object": "chat.completion", "created": 1753660800, "model": "olive", "choices": [ { "index": 0, "message": { "role": "assistant", "reasoning_content": "The question is asking which features of a headache presentation raise the probability of a secondary cause...", "content": "Screen for these red flags in any patient presenting with headache:\n\n1. **Thunderclap onset**: maximal intensity within a minute..." }, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 18, "completion_tokens": 412, "total_tokens": 430 } } ``` * **`message.reasoning_content`** holds Olive's reasoning, kept apart from the answer in `content`. An SDK that does not know the field may drop it. See [Reasoning](/capabilities/reasoning). * **`finish_reason`** is `stop` when Olive finished. It is `length` when the response hit the `max_tokens` budget, and the answer may be cut short. See [Token budget](/concepts/models#token-budget). * **`usage`** holds the token counts. On a stream, it arrives on the final chunk. ## If the first request fails * **Every request returns `401`.** When you leave `api_key` unset, an OpenAI SDK sends a key it finds in your environment instead. That key can belong to another project, and Avey answers `401`. Pass your Avey key explicitly. * **`content` is empty and `finish_reason` is `length`.** Reasoning used the whole `max_tokens` budget. Raise `max_tokens`, or leave it unset. See [Token budget](/concepts/models#token-budget). * **Any other status.** See [Errors](/concepts/errors) for what each status means and what to do. ## Next steps --- # Models and limits URL: https://docs.avey.ai/concepts/models Markdown: https://docs.avey.ai/llms.md/concepts/models Description: The model id, every limit that shapes a request, what each feature supports, and how to read the limits from the API. API keys call one model, Olive. Send its id, `olive`, as `model`. A version suffix such as `olive@` returns `404`. ## Olive | | `olive` | | ------------------------------------------------------ | --------------------------------------------------------------------------------------------------------- | | Input | Text. Documents and images are read as text after you upload them. See [Files](/capabilities/files) | | Output | Text. Chat completions also return the reasoning, in `reasoning_content` | | Context window | 65,536 tokens, prompt and output together | | Maximum output | 16,384 tokens, reasoning included | | Output when you set no limit | Up to 16,384 tokens on chat completions and completions. 4,096 tokens per model call on the Responses API | | [Reasoning effort](/capabilities/reasoning#effort) | `none`, `minimal`, `low`, `medium`, `high`, `xhigh` | | [Streaming](/capabilities/streaming) | Chat completions, completions, Responses | | [Function tools](/capabilities/tools) | Responses API. Untested on chat completions | | [File search](/capabilities/vector-stores) | Responses API, over your own vector stores | | [Structured outputs](/capabilities/structured-outputs) | Ask for JSON in the prompt, then parse and validate it. `response_format` is untested | | [Web search](/capabilities/web-search) | Coming soon | | Embeddings, audio, images out | Not available | An unknown model id returns `404` with the message `the model "" does not exist`. ## Read the limits from the API [`GET /models`](/api-reference/listModels) lists the models your key can call, with the context window and output ceiling of each. Read the limits from here rather than hardcoding them. ```json noCopy { "object": "list", "data": [ { "id": "avey/olive", "object": "model", "created": 1762560000, "owned_by": "avey", "context_length": 65536, "max_output_tokens": 16384 } ] } ``` The list prefixes the id with `avey/`. Requests accept either form. ## Token budget The context window holds the prompt and the output together. The output alone is capped at 16,384 tokens. Reasoning is output. It counts toward `max_tokens` (`max_output_tokens` on Responses) along with the answer. If the budget runs out while Olive is still reasoning, the answer is cut off or missing: | Endpoint | What you see | | ---------------- | ------------------------------------------------------------- | | Chat completions | `finish_reason` is `length`. `content` can be empty or `null` | | Responses | `status` is `incomplete` | When you set a limit, leave room for both. Leave `max_tokens` unset on chat completions and the ceiling of 16,384 applies. On the Responses API, set `max_output_tokens` yourself: without it, each model call stops at 4,096 tokens. A prompt that does not fit returns `400`. Files you reference count as input, and their text must fit the window too. Count a prompt before you send it with [`POST /responses/input_tokens`](/api-reference/countResponseInputTokens). The count is free. A large `max_tokens` holds headroom in your per-minute token limit while the request runs. See [Rate limits](/concepts/rate-limits#token-limits). Long prompts take longer to answer. Stream long requests and set your client timeout as described in [Errors](/concepts/errors#timeouts). ## Updates Avey may update the model behind `olive`. The same prompt can then get a different answer. Watch the [Changelog](/changelog), and re-run your evaluation set when it records a model change. --- # OpenAI compatibility URL: https://docs.avey.ai/openai-compatibility Markdown: https://docs.avey.ai/llms.md/openai-compatibility Description: Which OpenAI endpoints, request fields, and response fields Avey Olive supports, and which it does not. Avey Olive implements a subset of the OpenAI API. To point an OpenAI SDK at Olive, change three values: The [Quickstart](/quickstart) shows that pattern in each SDK, and [Frameworks](/frameworks) shows it in the libraries that wrap them. ## Check your integration Most integrations that send text and read `content` work unchanged. If your code does any of the following, change it first. | If your code | Change | | ---------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | | Sends images as `image_url` or files as inline data | Upload with [`POST /files`](/api-reference/createFile) and send the `file_id`. Inline data returns `400` | | Uses `response_format` for JSON | Untested. Ask for JSON in the prompt and validate it. See [Structured outputs](/capabilities/structured-outputs) | | Sends `tools` on chat completions | Untested. Move function calling to the Responses API. See [Tools](/capabilities/tools) | | Sends `stop` as a string | Wrap it in an array. A bare string returns `400` | | Reads only `message.content` | Also read `message.reasoning_content` if you want the reasoning. See [Reasoning](/capabilities/reasoning) | | Calls the Responses API without `max_output_tokens` | Set it. Without it, each model call stops at 4,096 tokens | | Retries every `429` | Stop on `insufficient_quota`. See [Rate limits](/concepts/rate-limits#a-429-that-is-not-a-rate-limit) | | Calls embeddings, audio, images, batches or realtime | Not available. See the last table | | Records `system_fingerprint` | It is empty or absent. Track model changes in the [Changelog](/changelog) | Fields not in the tables below are untested. Write code as though they do nothing. The limits are on [Models and limits](/concepts/models). ## Supported | Field or endpoint | What changes in your code | | --------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `POST /chat/completions` | Streams with `stream: true`. The reference is [here](/api-reference/createChatCompletion). | | `POST /completions` | Streams. Returns no reasoning field. | | `POST /responses` | Streams. Tools, stored responses, and file search live here. See [Tools](/capabilities/tools). | | `POST /responses/compact` | Shrinks a stored conversation. See [Conversation state](/capabilities/conversation-state). | | `POST /responses/input_tokens` | Counts input tokens before you send. | | `GET` and `DELETE /responses/{id}`, `GET /responses/{id}/input_items` | Reads or deletes a stored response. | | `GET /models` | Lists the model ids your key can use. | | `/files` | Uploads documents for use by `file_id`. See [Files](/capabilities/files). | | `/vector_stores` | Indexes files for search. See [Vector stores](/capabilities/vector-stores). | | `model` | Required. Send `olive`. An unknown id returns 404. See [Models](/concepts/models). | | `messages` | Required. Send the whole history on every call. Chat completions keep no state. | | `max_tokens` | Up to 16,384, which is also the default. A larger value is lowered to 16,384. Reasoning counts against it. | | `temperature` | 0 to 2. | | `top_p` | 0 to 1. Set this or `temperature`, not both. | | `stream` | Server-sent events, ending with `data: [DONE]`. | | `stop` | An array of up to 4 strings. The matched sequence is not returned. | | `response_format` | Untested. See [Structured outputs](/capabilities/structured-outputs) for what is verified. | | `reasoning_effort` | `none`, `minimal`, `low`, `medium`, `high`, or `xhigh`. Each sets a thinking budget. Any other value returns 400. See [Reasoning](/capabilities/reasoning#effort). | | `n` | Each answer is generated and billed. The rate limit reserves `max_tokens` for each one. | | Responses `input` | A string, or a list of items. Refer to uploaded files by `file_id` in `input_file` parts. | | Responses `instructions` | Sets Olive's role and constraints. | | Responses `max_output_tokens` | Up to 16,384. Without it, each model call stops at 4,096. A response that stops here has `status: "incomplete"`. | | Responses `temperature` | 0 to 2. | | Responses `previous_response_id` | Continues a stored response. Send only the new input. An id that is not yours, or does not exist, returns 404. | | Responses `store` | `true` keeps the response until you delete it. | | Responses `reasoning.effort` | The same values as `reasoning_effort`. | | Responses `tools`: `function` | You run the function and return its output as a `function_call_output` item. See [Tools](/capabilities/tools). | | Responses `tools`: `file_search` | Searches vector stores you own. See [Vector stores](/capabilities/vector-stores). | | Responses `tool_choice` | `auto` or `required`. | | Responses `include` | `file_search_call.results` adds the matched chunks to each `file_search_call` item. | | Chat response `choices[].message.content` | Always present. Can be `null` or empty when `finish_reason` is `length`. | | Chat response `choices[].message.reasoning_content` | Present when Olive reasoned. Show or log it. Do not parse it. | | Chat response `usage` | On a stream, only on the final chunk. `prompt_tokens_details.cached_tokens` counts prompt tokens served from your account's cache. | | Responses `status` | `incomplete` when generation stopped at `max_output_tokens`. Check it before you use the answer. | | Responses `output` | A plain answer is one `message` item. Tool calls appear as `function_call` or `file_search_call` items. | | Message `content` as an array of `file` parts | Allowed in `user` messages. Reference an uploaded file by `file_id`. | ## Accepted and ignored | Field | What changes in your code | | ---------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `stream_options.include_usage` | Accepted. Usage is always on the final chunk, set or not. | | `user` | Accepted. Avey does not read it. Keep your own tracking. | | `metadata` | Accepted. Chat completions have no retrieval endpoint, so you cannot read it back. | | `store` on chat completions | Accepted. To keep a response, use `store` on `/responses`. | | `service_tier` | Accepted. Olive has one tier. | | Responses `truncation: "disabled"` | Accepted. An over-long conversation can lose its oldest turns and then fail with 500. Count tokens first with [`/responses/input_tokens`](/api-reference/countResponseInputTokens). | ## Rejected or not available | Field or endpoint | What changes in your code | | ----------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | | `stop` as a bare string | Returns 400. Wrap it: `stop=["\n\n"]`. | | Responses `reasoning.summary` | Returns 400. | | Responses `background` | Returns 400. | | Responses `context_management` | Returns 400. Compact with [`POST /responses/compact`](/api-reference/compactResponse). | | Responses `tools` of any other type, including `mcp` | Returns 400. | | `web_search` tool and `web_search_options` | Coming soon. See [Web search](/capabilities/web-search). | | Message content with `image_url` parts or inline file data | Returns 400. Upload the file with [`POST /files`](/api-reference/createFile) and send its `file_id`. | | Response field `refusal` | Always `null`. | | Response fields `logprobs` and `service_tier` | Untested. Do not rely on them. | | `system_fingerprint` | Empty or absent. Do not read it. | | Reasoning on `/responses` and `/completions` | Not returned as a separate field. Only `/chat/completions` returns it. | | `POST /embeddings` | Returns 404. | | Audio, image generation, batches, and the Conversations API | Returns 404. Chain turns with `previous_response_id` instead of Conversations. | | Realtime (WebSocket mode) | Returns 400. | | `/vector_stores/{id}/file_batches` | Returns 403. Attach files one at a time. See [Vector stores](/capabilities/vector-stores). | | Any other path | Returns 404. | ## Next steps --- # Authentication URL: https://docs.avey.ai/concepts/authentication Markdown: https://docs.avey.ai/llms.md/concepts/authentication Description: Send your API key with each request, and handle keys safely. Send your API key as a bearer token in the `Authorization` header of every request. ```bash curl https://api.avey.ai/llm/models \ -H "Authorization: Bearer $AVEY_API_KEY" ``` ## Key format Keys start with `avey_sk1_`, then 52 characters, `_`, and a 7-character checksum. ```text avey_sk1_2A6J5GM2LAXHXHIOU2M77OPFOP2XC4LE3XGORIXVSO2JHGCVZCXQ_FS5FY7Q ``` A key that was cut short or altered when you copied it returns `401` with `invalid API key`. ## Create and manage keys Create keys on the **API keys** page of the [API platform](https://platform.avey.ai). Give each key a name and an expiry: 30 days, 90 days, 180 days, one year, or a date you pick. The full key is shown once, when you create it. The platform keeps only a hash, so a lost key cannot be recovered. Create a new one. | Action | What happens to the old key | | ------------- | ----------------------------------------------------------------------- | | Regenerate | Stops working. The new key has the same name. Update your integrations. | | Revoke | Stops working. It stays in the list as revoked. This cannot be undone. | | Delete | Stops working and leaves the list. This cannot be undone. | | Let it expire | Returns `403` with `API key expired` from the expiry date on. | Every key on your account shares the same rate limits and the same credit balance. A second key does not add capacity. See [Rate limits](/concepts/rate-limits). ## Use the OpenAI SDKs The OpenAI SDKs send this header already. Pass the key as `api_key` and set nothing else. ```python import os from openai import OpenAI client = OpenAI( base_url="https://api.avey.ai/llm", api_key=os.environ["AVEY_API_KEY"], ) ``` ## Handle keys safely * Call Avey from your server. A key in browser or mobile code is a public key. * Keep keys out of source control. Store them in environment variables or a secrets manager. * Use one key per application, so you can revoke one without affecting the others. * Rotate a key if it may have been exposed. ## Authentication errors A missing key returns `401` with `missing credential`. A malformed or unknown key returns `401` with `invalid API key`. A revoked or expired key, or an account without API access, returns `403`. Both bodies use the simple `{ "message": ... }` shape. The messages, and what to do about each, are in [Errors](/concepts/errors). --- # Frameworks URL: https://docs.avey.ai/frameworks Markdown: https://docs.avey.ai/llms.md/frameworks Description: Connect LangChain, LlamaIndex, the Vercel AI SDK, or another OpenAI-compatible client to Olive. Point a framework at Olive with three values: the base URL, your API key, and the model id `olive`. Clients built on an OpenAI client already send the `Authorization: Bearer` header, so no custom header is needed. Pass the key explicitly in every client. The [Quickstart](/quickstart) explains why. Each framework names these settings differently. Look for "base URL" and "API key". ## LangChain `ChatOpenAI` takes the base URL and key directly. In JavaScript, the base URL goes under `configuration`. ```python # pip install langchain-openai import os from langchain_openai import ChatOpenAI llm = ChatOpenAI( model="olive", base_url="https://api.avey.ai/llm", api_key=os.environ["AVEY_API_KEY"], ) print(llm.invoke("What are the red flags for headache in primary care?").content) ``` ```javascript // npm install @langchain/openai import { ChatOpenAI } from "@langchain/openai"; const llm = new ChatOpenAI({ model: "olive", apiKey: process.env.AVEY_API_KEY, configuration: { baseURL: "https://api.avey.ai/llm" }, }); ``` ## LlamaIndex Use `OpenAILike` for an OpenAI-compatible endpoint. Set `is_chat_model=True` so LlamaIndex calls `/chat/completions`, which returns Olive's [reasoning](/capabilities/reasoning) in `reasoning_content`. ```python # pip install llama-index-llms-openai-like import os from llama_index.llms.openai_like import OpenAILike llm = OpenAILike( model="olive", api_base="https://api.avey.ai/llm", api_key=os.environ["AVEY_API_KEY"], is_chat_model=True, ) ``` ## Vercel AI SDK `@ai-sdk/openai-compatible` takes a base URL and key. ```javascript // npm install ai @ai-sdk/openai-compatible import { createOpenAICompatible } from "@ai-sdk/openai-compatible"; import { generateText } from "ai"; const avey = createOpenAICompatible({ name: "avey", baseURL: "https://api.avey.ai/llm", apiKey: process.env.AVEY_API_KEY, }); const { text } = await generateText({ model: avey("olive"), prompt: "What are the red flags for headache in primary care?", }); ``` ## Other OpenAI clients The OpenAI Agents SDK and Instructor take a client object that you build. Build the client once and pass it in. ```python # pip install openai-agents (or: pip install instructor) import os from openai import AsyncOpenAI from agents import OpenAIChatCompletionsModel, set_default_openai_client client = AsyncOpenAI( base_url="https://api.avey.ai/llm", api_key=os.environ["AVEY_API_KEY"], ) set_default_openai_client(client) model = OpenAIChatCompletionsModel(model="olive", openai_client=client) ``` ```javascript // npm install @openai/agents openai import OpenAI from "openai"; import { setDefaultOpenAIClient } from "@openai/agents"; setDefaultOpenAIClient( new OpenAI({ baseURL: "https://api.avey.ai/llm", apiKey: process.env.AVEY_API_KEY, }), ); ``` Where a framework lets you choose the endpoint, use `/chat/completions` to get Olive's reasoning in `reasoning_content`. Function tools, files, and stored responses are on `/responses`. A framework that wraps those features needs the Responses API, so check that it accepts a custom base URL there. See [OpenAI compatibility](/openai-compatibility). ## Tools with only a base URL and an API key Hosted evaluation platforms, IDE plugins, and chat UIs often have two fields and no way to add a header. Those two fields are all Avey needs. | Field | Value | | -------- | --------------------------------------- | | Base URL | `https://api.avey.ai/llm` | | API key | your key, which starts with `avey_sk1_` | | Model | `olive` | If the tool has a provider list, choose "OpenAI-compatible" or "Custom". ## Timeouts Long answers can run for minutes. Set the client's request timeout as described in [Errors](/concepts/errors#timeouts). ## Next steps --- # Build with AI tools URL: https://docs.avey.ai/ai-tools Markdown: https://docs.avey.ai/llms.md/ai-tools Description: Give a coding assistant these docs in a form it reads well, so the code it writes uses the right model id, key header and reasoning field. A coding assistant that has not read these docs writes OpenAI code. It sends an OpenAI model name, reads only `message.content`, and drops Olive's reasoning. Point it at one of the files below before it writes anything. ## Files for assistants | File | What it holds | Use it when | | ---------------------------------- | ------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- | | [`/llms.txt`](/llms.txt) | A short brief on the API, then a linked index of every page | The assistant can follow links and you want it to pick the pages it needs | | [`/llms-full.txt`](/llms-full.txt) | Every page in one file, in sidebar order | You paste or attach one file and want the whole picture | | `/llms.md/` | One page as Markdown. `/quickstart` is also at [`/llms.md/quickstart`](/llms.md/quickstart) | You want one topic, such as streaming, without the rest | Each file opens with the same brief: the bearer header, the model id `olive`, the base URL, the `reasoning_content` field, the limits, and what is not supported. Values are filled in, so the assistant never sees a placeholder. ## From a page Every page has two buttons under its title. * **Copy Markdown** copies the page as Markdown, ready to paste into a chat. * **Open** sends the page to ChatGPT, Claude, Cursor or Scira AI, or shows it as plain Markdown. ## A prompt that works Paste this, with the full-text file attached or its URL given: ```text Read the Avey Olive docs in llms-full.txt before writing code. Use the OpenAI SDK with the base URL, model id and bearer key it gives. Print message.reasoning_content and message.content separately. Read the key from the AVEY_API_KEY environment variable. ``` ## Clinical review stays in The brief tells assistants that Olive is a clinical model and that a qualified clinician reviews output before it reaches a care decision. If generated code sends Olive's answer straight to a patient, or to a decision with no review step, change it before you ship it. Do not paste patient records into an assistant to build test prompts. Write synthetic cases. --- # Reasoning URL: https://docs.avey.ai/capabilities/reasoning Markdown: https://docs.avey.ai/llms.md/capabilities/reasoning Description: Read Olive's reasoning apart from its answer, and set how much it reasons. Olive returns its reasoning in `reasoning_content`, separate from the answer in `content`. Set `reasoning_effort` to change how much it reasons. ## When to use it * A clinician reviews how Olive reached an answer. * You log reasoning to trace an answer that looks wrong. * A hard case needs more room, or a simple question should return sooner. The SDK samples assume the `client` from the [Quickstart](/quickstart). ```bash curl https://api.avey.ai/llm/chat/completions \ -H "Authorization: Bearer $AVEY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "olive", "reasoning_effort": "low", "messages": [ { "role": "user", "content": "62-year-old with exertional chest pain relieved by rest. What is the differential?" } ] }' ``` ```python response = client.chat.completions.create( model="olive", reasoning_effort="low", messages=[{ "role": "user", "content": "62-year-old with exertional chest pain relieved by rest. What is the differential?", }], ) message = response.choices[0].message print(getattr(message, "reasoning_content", None)) # how Olive worked through it print(message.content) # the answer ``` ```javascript const response = await client.chat.completions.create({ model: "olive", reasoning_effort: "low", messages: [ { role: "user", content: "62-year-old with exertional chest pain relieved by rest. What is the differential?" }, ], }); const message = response.choices[0].message; console.log(message.reasoning_content); // how Olive worked through it console.log(message.content); // the answer ``` ```go resp, err := client.Chat.Completions.New(context.Background(), openai.ChatCompletionNewParams{ Model: "olive", ReasoningEffort: shared.ReasoningEffortLow, Messages: []openai.ChatCompletionMessageParamUnion{ openai.UserMessage("62-year-old with exertional chest pain relieved by rest. What is the differential?"), }, }) if err != nil { panic(err) } message := resp.Choices[0].Message fmt.Println(message.JSON.ExtraFields["reasoning_content"].Raw()) // how Olive worked through it fmt.Println(message.Content) // the answer ``` The Go samples reuse the Quickstart `client`. Add the imports each sample uses. The response carries both fields on the message: ```json noCopy { "choices": [ { "index": 0, "finish_reason": "stop", "message": { "role": "assistant", "reasoning_content": "Exertional pain relieved by rest suggests a demand-related...", "content": "The most likely diagnosis is stable angina..." } } ] } ``` ## Effort `reasoning_effort` sets a thinking budget. Olive is told to think for at most that many tokens before it answers. On the Responses API, send the same values as `reasoning.effort`. | `reasoning_effort` | Thinking budget | | ------------------ | ----------------------------------------------------------- | | `none` | 256 tokens | | `minimal` | 512 tokens | | `low` | 1,024 tokens | | `medium` | 2,048 tokens | | `high` | 4,096 tokens | | `xhigh` | 6,144 tokens | | Not set | No budget. Olive reasons as long as the output limit allows | * `none` is the smallest budget, not "off". Olive always reasons before it answers. * The budget is an instruction, not a hard cap. Use `max_tokens` for a hard limit. * Any other value returns `400`: `Invalid value: 'x'. Supported values are: 'none', 'minimal', 'low', 'medium', 'high', 'xhigh'.` * `reasoning.max_tokens` with no effort picks the largest level whose budget fits under it. An explicit effort wins. * `reasoning.summary` on Responses returns `400`. Reasoning counts toward `max_tokens` with the answer. If it uses the whole budget, a chat completion ends with `finish_reason` `length` and `content` can be empty. A Responses request ends with `status` `incomplete`. Leave room for both. On chat completions you can leave `max_tokens` unset. On Responses, set `max_output_tokens`. ## Multi-turn conversations Chat completions keep no state. Send the whole conversation on every call. For each earlier assistant turn, send its `content`. Keep the reasoning in your own logs. ```python messages = [{"role": "user", "content": "62-year-old with exertional chest pain relieved by rest. What is the differential?"}] first = client.chat.completions.create(model="olive", messages=messages, reasoning_effort="medium") messages.append({"role": "assistant", "content": first.choices[0].message.content}) messages.append({"role": "user", "content": "Which test would you order first, and why?"}) second = client.chat.completions.create(model="olive", messages=messages, reasoning_effort="medium") ``` Keep `reasoning_effort` the same for the whole conversation. The budget is written into every user message, so changing it on a later turn changes the whole prompt. The cached prefix then misses, and that turn is slower. To keep the conversation on the server instead, use the Responses API with `previous_response_id`. See [Conversation state](/capabilities/conversation-state). ## Things to know * Reasoning is output. It counts in `usage.completion_tokens`, or `output_tokens` on Responses, and is billed as output. * Only chat completions return the reasoning text. Responses and completions do not return it as a separate field. * Each effort budget adds a short instruction to every user message. Those tokens count as input. * Reasoning streams before the answer. See [Streaming](/capabilities/streaming). * Show reasoning to clinicians, not patients. A clinician reviews the answer, with its reasoning, before it affects care. * Reasoning is free text. Read `content` for the answer, and see [Structured outputs](/capabilities/structured-outputs) before you parse anything. --- # Streaming URL: https://docs.avey.ai/capabilities/streaming Markdown: https://docs.avey.ai/llms.md/capabilities/streaming Description: Receive Olive's answer as it is generated, as server-sent events. Set `stream: true` to receive the answer one chunk at a time. Chat Completions streams and Responses streams both work this way. ## When to use it * You show the answer as it arrives. * A request may take minutes, for example a long answer or a large file. A stream stays open while Olive works, where a request that waits for the whole answer can time out. * You want to stop a long answer early. You are billed for the tokens generated up to the moment you disconnect. The SDK samples assume the `client` from the [Quickstart](/quickstart). ```bash curl https://api.avey.ai/llm/chat/completions \ -H "Authorization: Bearer $AVEY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "olive", "messages": [{ "role": "user", "content": "Summarise the management of stable angina." }], "stream": true }' ``` ```python stream = client.chat.completions.create( model="olive", messages=[{"role": "user", "content": "Summarise the management of stable angina."}], stream=True, ) for chunk in stream: if not chunk.choices: continue delta = chunk.choices[0].delta if delta.content: print(delta.content, end="", flush=True) ``` ```javascript const stream = await client.chat.completions.create({ model: "olive", messages: [{ role: "user", content: "Summarise the management of stable angina." }], stream: true, }); for await (const chunk of stream) { process.stdout.write(chunk.choices[0]?.delta?.content ?? ""); } ``` ```go stream := client.Chat.Completions.NewStreaming(context.Background(), openai.ChatCompletionNewParams{ Model: "olive", Messages: []openai.ChatCompletionMessageParamUnion{ openai.UserMessage("Summarise the management of stable angina."), }, }) for stream.Next() { chunk := stream.Current() if len(chunk.Choices) > 0 { fmt.Print(chunk.Choices[0].Delta.Content) } } if err := stream.Err(); err != nil { panic(err) } ``` The Go samples reuse the Quickstart `client`. Add the imports each sample uses. ## Chat Completions chunks Each chunk carries the new text in `choices[0].delta.content`. The stream ends with `data: [DONE]`. Reasoning arrives in `delta.reasoning_content` before the content. Usage arrives once, on a last chunk after the content, whether or not you set `stream_options.include_usage`. Its `choices` holds one entry with an empty `delta`, not an empty array as on OpenAI. ```python for chunk in stream: if chunk.usage: print(f"\ntokens: {chunk.usage.total_tokens}") if not chunk.choices: continue delta = chunk.choices[0].delta reasoning = getattr(delta, "reasoning_content", None) if reasoning: print(reasoning, end="") if delta.content: print(delta.content, end="") ``` ## Responses streams Set `stream=True` on `responses.create`. Each event has a `type`. The answer text arrives in `response.output_text.delta` events. ```bash curl -N https://api.avey.ai/llm/responses \ -H "Authorization: Bearer $AVEY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "olive", "input": "Summarise the management of stable angina.", "stream": true }' ``` ```python stream = client.responses.create( model="olive", input="Summarise the management of stable angina.", stream=True, ) for event in stream: if event.type == "response.output_text.delta": print(event.delta, end="", flush=True) ``` ```javascript const stream = await client.responses.create({ model: "olive", input: "Summarise the management of stable angina.", stream: true, }); for await (const event of stream) { if (event.type === "response.output_text.delta") { process.stdout.write(event.delta); } } ``` ```go stream := client.Responses.NewStreaming(context.Background(), responses.ResponseNewParams{ Model: "olive", Input: responses.ResponseNewParamsInputUnion{ OfString: openai.String("Summarise the management of stable angina."), }, }) for stream.Next() { event := stream.Current() if event.Type == "response.output_text.delta" { fmt.Print(event.Delta) } } if err := stream.Err(); err != nil { panic(err) } ``` | Event | What it carries | | ---------------------------- | ----------------------------------------------------------------------------------- | | `response.output_text.delta` | The next piece of the answer, in `delta` | | `response.completed` | The finished response. The stream ends here. | | `response.incomplete` | The response, stopped early, for example at the output limit. The stream ends here. | | `error` | A failure after the stream started. The stream ends here. | A Responses stream has no `data: [DONE]` line. It ends with one of the last three events. Read `type` and ignore the events you do not handle. ## Heartbeats During gaps between events, for example while a file is read or a tool runs, the server sends an SSE comment line, `: heartbeat`, about every second. It keeps the connection open. The SDKs and standard SSE parsers skip comments. If you parse the stream yourself, skip every line that starts with `:`. ## Limits | Limit | Value | | -------------- | -------------------------------------------------------------- | | Context window | 65,536 tokens for the input, reasoning and answer together | | Output | 16,384 tokens at most | | Read timeout | 700 seconds without a byte. Heartbeats keep a stream under it. | ## Errors An error before the first event is a normal HTTP error with a JSON body. See [Errors](/concepts/errors). An error after the stream started cannot change the status code, so it arrives in the stream: | API | What you receive | | ---------------- | ------------------------------------------------------------------------- | | Chat Completions | A last `data:` line holding an `error` object. No `data: [DONE]` follows. | | Responses | An `event: error` event. | The message is generic, such as `the request failed; retry later`. Treat the text received so far as incomplete, and retry. ## Things to know * A response that stops at the output limit has status `incomplete`. * Rate limits apply when the stream opens. A request over a limit returns 429 before any event is sent. See [Rate limits](/concepts/rate-limits). --- # Structured outputs URL: https://docs.avey.ai/capabilities/structured-outputs Markdown: https://docs.avey.ai/llms.md/capabilities/structured-outputs Description: Parse and validate Olive's JSON before your code stores or acts on it. Olive returns its answer as text in `content`. If your prompt asks for JSON, parse the answer and validate it before your code uses it. ## When to use it * You extract codes or fields from a note into a form or a database row. * Your code reads the same keys from every answer. The SDK samples assume the `client` from the [Quickstart](/quickstart). ```bash curl https://api.avey.ai/llm/chat/completions \ -H "Authorization: Bearer $AVEY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "olive", "messages": [ { "role": "system", "content": "Extract diagnoses as JSON with a diagnoses array. Return only the JSON." }, { "role": "user", "content": "45-year-old woman, two days of dysuria and urinary frequency. No fever, no flank pain. Urinalysis positive for nitrites and leukocyte esterase." } ] }' ``` ```python import json response = client.chat.completions.create( model="olive", messages=[ {"role": "system", "content": "Extract diagnoses as JSON with a diagnoses array. Return only the JSON."}, {"role": "user", "content": "45-year-old woman, two days of dysuria and urinary frequency. No fever, no flank pain. Urinalysis positive for nitrites and leukocyte esterase."}, ], ) choice = response.choices[0] if choice.finish_reason == "length": raise RuntimeError("The answer was cut off at the output limit.") data = json.loads(choice.message.content) if not isinstance(data.get("diagnoses"), list): raise ValueError("The answer has no diagnoses array.") ``` ```javascript const response = await client.chat.completions.create({ model: "olive", messages: [ { role: "system", content: "Extract diagnoses as JSON with a diagnoses array. Return only the JSON." }, { role: "user", content: "45-year-old woman, two days of dysuria and urinary frequency. No fever, no flank pain. Urinalysis positive for nitrites and leukocyte esterase." }, ], }); const choice = response.choices[0]; if (choice.finish_reason === "length") { throw new Error("The answer was cut off at the output limit."); } const data = JSON.parse(choice.message.content); if (!Array.isArray(data.diagnoses)) { throw new Error("The answer has no diagnoses array."); } ``` ```go resp, err := client.Chat.Completions.New(context.Background(), openai.ChatCompletionNewParams{ Model: "olive", Messages: []openai.ChatCompletionMessageParamUnion{ openai.SystemMessage("Extract diagnoses as JSON with a diagnoses array. Return only the JSON."), openai.UserMessage("45-year-old woman, two days of dysuria and urinary frequency. No fever, no flank pain. Urinalysis positive for nitrites and leukocyte esterase."), }, }) if err != nil { panic(err) } choice := resp.Choices[0] if choice.FinishReason == "length" { panic("the answer was cut off at the output limit") } var data struct { Diagnoses []json.RawMessage `json:"diagnoses"` } if err := json.Unmarshal([]byte(choice.Message.Content), &data); err != nil { panic(err) } if data.Diagnoses == nil { panic("the answer has no diagnoses array") } ``` The Go samples reuse the Quickstart `client`. Add the imports each sample uses. ## Limits | Limit | Value | | ---------------- | -------------------------------------------------------------------------------------------------------------------------- | | Output | 16,384 tokens at most, with reasoning included | | Truncated answer | `finish_reason` is `length` on Chat Completions. On Responses, `status` is `incomplete`. A cut-off JSON answer is invalid. | ## Things to know * Validate every field. JSON that parses can still hold a wrong code or dose. A clinician checks the values before they affect care. * Reasoning uses the same output budget as the JSON. Leave room for both, or see [Reasoning](/capabilities/reasoning). * `response_format` is not covered on this page. Olive's support for its JSON modes is untested, so do not rely on it yet. --- # Tools URL: https://docs.avey.ai/capabilities/tools Markdown: https://docs.avey.ai/llms.md/capabilities/tools Description: Let Olive call your functions, or search your vector stores, in a Responses request. Define a function tool, run it in your code, and send the result back so Olive can finish its answer. Send tools with `POST /responses`. ## When to use it * A value must come from your code, not from the model's text. For example, a BMI calculated from weight and height. * Olive should search your [vector store](/capabilities/vector-stores) with `file_search`. | Tool type | Who runs it | Status | | ------------- | ------------------------ | ----------------------------------------------------------------------------- | | `function` | Your code | Available | | `file_search` | Avey, inside the request | Available | | `web_search` | Avey | Coming soon. Returns 400 for now. See [Web search](/capabilities/web-search). | | `mcp` | None | Not available. Returns 400. | Every other tool type returns 400. The SDK samples assume the `client` from the [Quickstart](/quickstart). ## Define a function Describe the function with a name and a JSON schema. Olive never runs it. It asks for a call when it needs the result. ```bash curl https://api.avey.ai/llm/responses \ -H "Authorization: Bearer $AVEY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "olive", "input": "Calculate the BMI for a patient who weighs 82 kg and is 1.75 m tall.", "tools": [{ "type": "function", "name": "calculate_bmi", "description": "Body mass index from weight in kilograms and height in metres.", "parameters": { "type": "object", "properties": { "weight_kg": { "type": "number" }, "height_m": { "type": "number" } }, "required": ["weight_kg", "height_m"] } }] }' ``` ```python tools = [{ "type": "function", "name": "calculate_bmi", "description": "Body mass index from weight in kilograms and height in metres.", "parameters": { "type": "object", "properties": { "weight_kg": {"type": "number"}, "height_m": {"type": "number"}, }, "required": ["weight_kg", "height_m"], }, }] response = client.responses.create( model="olive", input="Calculate the BMI for a patient who weighs 82 kg and is 1.75 m tall.", tools=tools, ) ``` ```javascript const tools = [ { type: "function", name: "calculate_bmi", description: "Body mass index from weight in kilograms and height in metres.", parameters: { type: "object", properties: { weight_kg: { type: "number" }, height_m: { type: "number" }, }, required: ["weight_kg", "height_m"], }, }, ]; const response = await client.responses.create({ model: "olive", input: "Calculate the BMI for a patient who weighs 82 kg and is 1.75 m tall.", tools, }); ``` ```go tools := []responses.ToolUnionParam{{ OfFunction: &responses.FunctionToolParam{ Name: "calculate_bmi", Description: openai.String("Body mass index from weight in kilograms and height in metres."), Strict: openai.Bool(false), Parameters: map[string]any{ "type": "object", "properties": map[string]any{ "weight_kg": map[string]any{"type": "number"}, "height_m": map[string]any{"type": "number"}, }, "required": []string{"weight_kg", "height_m"}, }, }, }} resp, err := client.Responses.New(context.Background(), responses.ResponseNewParams{ Model: "olive", Input: responses.ResponseNewParamsInputUnion{ OfString: openai.String("Calculate the BMI for a patient who weighs 82 kg and is 1.75 m tall."), }, Tools: tools, }) if err != nil { panic(err) } ``` The Go samples reuse the Quickstart `client`. Add the imports each sample uses. ## Return the result When Olive wants the function, `output` holds a `function_call` item. Its `arguments` is a JSON string. Run the function, then send the result as a `function_call_output` item with the same `call_id`. Chain it to the response that asked for the call, and send `tools` again. ```python import json call = next((item for item in response.output if item.type == "function_call"), None) if call is not None: args = json.loads(call.arguments) result = {"bmi": round(args["weight_kg"] / args["height_m"] ** 2, 1)} response = client.responses.create( model="olive", previous_response_id=response.id, input=[{ "type": "function_call_output", "call_id": call.call_id, "output": json.dumps(result), }], tools=tools, ) print(response.output_text) ``` Olive can ask for more than one call, or for another call after your result. Loop until a response has no `function_call` item. Olive writes the arguments as it writes any text. They can be malformed JSON, miss a required field, or carry a plausible wrong value. Validate them against your schema and check clinical values for range. A function that changes a record, places an order or sends a message needs a clinician's confirmation, not only Olive's request. ## File search Pass `file_search` with the IDs of the stores to search. Avey runs the search inside the request, so your code does not answer a call. ```python response = client.responses.create( model="olive", input="What is the recommended first-line antiplatelet therapy?", tools=[{"type": "file_search", "vector_store_ids": [store.id]}], ) ``` Only stores you own are searched. Citations come back as `file_citation` annotations on the output text. See [Vector stores](/capabilities/vector-stores). ## Errors | Status | Message | Cause | | ------ | --------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------- | | 400 | `tool type "" is not available` | A tool type other than `function` or `file_search`, such as `web_search` or `code_interpreter`. | | 400 | `unknown tool "