Conversation state
Continue a conversation with previous_response_id, and keep a long conversation inside the context window.
Continue a conversation by sending the new message and the id of the response it follows. You do not resend the earlier turns.
When to use it
- A consultation runs over several turns, and you want the history kept on the server.
- A conversation is close to the context window, and you want to shorten it with compaction.
The SDK samples assume the client from the Quickstart.
curl https://api.avey.ai/llm/responses \
-H "Authorization: Bearer $AVEY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "olive",
"previous_response_id": "resp_...",
"input": "What changes if she is also on warfarin?"
}'The Go samples reuse the Quickstart client. Add the imports each sample uses.
Olive reads the whole chain on every turn, so the input grows with the conversation.
Set max_output_tokens on Responses
Without max_output_tokens, each model call a Responses request makes stops at
4,096 output tokens, reasoning included. A long answer, or a
high reasoning effort, can be cut off. Set max_output_tokens, up to
16,384, on every Responses request.
Stored responses
| Operation | Request |
|---|---|
| Retrieve | GET /responses/{id} |
| List input items | GET /responses/{id}/input_items |
| Delete | DELETE /responses/{id} |
A stored response keeps its full input
That includes the text of any file the request referenced. A response stays until you delete it. Deleting a file does not remove its text from responses that already used it.
Compaction
POST /responses/compact shortens a long conversation. It keeps every user message word for word. It replaces Olive's replies, tool results and file text with one summary item. Call it yourself between turns. Continue from the compacted response:
compacted = client.responses.compact(
model="olive",
previous_response_id=second.id,
)
response = client.responses.create(
model="olive",
previous_response_id=compacted.id,
input="Does his dose need adjusting?",
)The answer has object: "response.compaction". Its output holds the user messages of the compacted range, then one item of type compaction with the summary.
The summary is told to keep each dose in full, warning signs with their thresholds, allergies, and standing instructions quoted exactly. It is written in the language most of the user's messages are in. It is capped at 4,096 tokens, and max_output_tokens does not change that cap.
Limits
| Limit | Value |
|---|---|
| Context window | 65,536 tokens for the input, reasoning and answer together |
| Output per model call | 4,096 tokens unless you set max_output_tokens, up to 16,384 |
| Compaction summary | 4,096 tokens |
| Automatic compaction | Not available. context_management returns 400. |
Errors
| Status | Cause | What to do |
|---|---|---|
| 404, or 400 that names the ID | previous_response_id belongs to another user, or does not exist. | Use a response ID your key created. |
| 400 | The request sends context_management. The message is background, context_management and reasoning.summary are not available. | Remove context_management, and call POST /responses/compact yourself. |
Things to know
- The summary is written by the model. It can leave out or change a fact. Keep your own record of doses and allergies, and have a clinician check the summary before anyone relies on it.
- After compaction, Olive no longer has the exact text of the files in the conversation. Attach the file again if a question needs its wording.
- Compaction keeps every user message, so a long user side still grows.
- Compaction is an extra model call. Run it between turns, not while a user waits for an answer.
- Count the input with Count input tokens and compact well before the context window fills.