AI API conversation history must reach the next request. Learn how stateless chat, response IDs and session affinity differ on Token360.
CONVERSATION STATE · BLOG 164
By Token360 · Editorial review: November 5, 2026
An AI API does not necessarily remember previous messages just because you reuse the same API key, model or chat window. In a stateless workflow, your application supplies the relevant conversation history with each request. A stateful workflow instead requires a documented, supported mechanism for referencing stored context. Check which contract your endpoint actually provides.
This question grows from OpenRouter’s FAQ about Chatroom storage. As checked November 5, 2026, it describes locally stored conversations and export/import controls. That is a chat-interface behavior; it does not establish that an arbitrary API request can recover those messages from the same account.
This guide separates history storage, request assembly and routing. The JSON examples are constructed text-only request examples, not captured responses or a live compatibility test. No model calls or conversation-recall measurements were performed.
Read first: Token360’s endpoint contracts identify the protocol your application must follow. Similar-looking SDK methods do not establish identical state-management features.
Where does conversation history actually live?
Conversation history can live in a browser, an application database or a supported server-side conversation service. The model’s next input is a separate object. Trace how stored messages enter that input before calling the behavior “memory.” A visible chat transcript proves that the interface has records, not that a subsequent API request includes them.
Layer
What it does
What it does not establish
Chat interface
Displays prior exchanges.
Every displayed turn was sent again.
Application history store
Keeps records for a particular user and conversation.
The request builder selected the correct record.
Request context
Supplies messages or supported context references.
The model will interpret every detail correctly.
Routing preference
Influences where eligible requests are served.
The route stores and reloads the conversation.
For example, a UI may display yesterday’s chat after loading it from a database while a new backend endpoint sends only today’s latest question. The interface appears continuous, but the request lacks the previous facts. Debugging only the browser transcript would miss this difference.
OpenAI’s conversation-state guide distinguishes manually supplied history from its supported conversation and response-chaining mechanisms. Those are documented ways to provide context, not evidence that every compatible gateway implements the same persistence features. Treat each platform’s contract as authoritative for that platform.
What should a stateless second request contain?
A stateless second request needs the context required to understand the follow-up, along with the new user message. For a simple text-only chat, that often means earlier user messages and completed assistant replies in order. Preserve roles and applicable instructions. Do not send only an ambiguous follow-up and expect account identity to recover its meaning.
Consider a fictional shipping conversation. The first user message supplies a destination; the next asks for that destination. The following Chat Completions-shaped request contains the relevant exchange. The model name is an explicit placeholder, and the assistant reply was authored for this example.
{
"model": "REPLACE_WITH_SUPPORTED_MODEL_ID",
"messages": [
{"role": "user", "content": "The destination is Porto."},
{"role": "assistant", "content": "The destination is Porto."},
{"role": "user", "content": "Which city did I specify?"}
]
}
Compare that with a request containing only Which city did I specify?. In the constructed request, the city is absent. Reusing the same credential does not insert it into the message list. This is an input observation, not a claim about the exact answer a particular model would produce.
Copy second-turn-with-history.json:
{
"model": "REPLACE_WITH_SUPPORTED_MODEL_ID",
"messages": [
{
"role": "user",
"content": "The destination is Porto."
},
{
"role": "assistant",
"content": "The destination is Porto."
},
{
"role": "user",
"content": "Which city did I specify?"
}
]
}
and
Copy second-turn-without-history.json:
{
"model": "REPLACE_WITH_SUPPORTED_MODEL_ID",
"messages": [
{
"role": "user",
"content": "Which city did I specify?"
}
]
}
. Neither file includes credentials, executes a call or claims compatibility with a specific model. Replace the placeholder only after checking the chosen endpoint and model.
In a real integration, use the actual completed assistant message instead of inventing an acknowledgement to fill a gap. If no usable response was received, keep that state explicit. Inserting a fabricated assistant reply can make the next turn appear to rely on a statement the model never made.
Can Token360 use a previous response ID instead?
Do not assume that a response ID can be used to continue a conversation. That requires a supported stateful endpoint feature, not merely an ID in a response. Check the current Token360 reference before adopting a server-managed history pattern; an SDK accepting a parameter does not establish that the service implements it.
As checked November 5, 2026, Token360’s Create response documentation marks previous_response_id, conversation and store: true as unsupported stateful operations returning 501 unsupported_feature. It specifies that store must be false or omitted. This is a documented limitation, not a newly executed API test.
Accordingly, a design that assumes those fields will reload earlier context needs to be changed or validated against a newly supported contract before deployment. Keep your application’s conversation record independent of an unsupported continuation field. Use input structures appropriate to the endpoint you selected; do not move a Chat Completions message object into another protocol without checking its schema.
An identifier may still be useful for tracing or support even when it cannot retrieve a stored conversation. Name these roles separately in your application: a request reference identifies an operation; your conversation identifier identifies a history record; a supported continuation reference, where available, has its own documented behavior.
Is session affinity the same as conversation memory?
No. Session affinity concerns routing, while conversation history supplies the content for the next turn. Keeping requests on a preferred eligible route does not replace sending context or using a supported continuation mechanism. Give these two concerns separate fields, tests and failure handling so that a stable route does not disguise missing input.
Token360’s routing documentation describes session_id and x-session-id as routing inputs. It states that the body field is used for routing and is not forwarded to the upstream provider. The documented preference can help compatible multi-turn workflows, but it does not describe a transcript-retrieval API.
Keep the history record, submitted context and routing identifier separate.
Use an opaque per-conversation or per-run identifier when the routing feature calls for one. Do not use an email address or credential as a convenient conversation label. More fundamentally, the identifier should not substitute for authorization: your application still needs to verify which user may read and continue the associated history.
This distinction also helps diagnose a model switch. A new model needs a compatible representation of the relevant context; a remembered route cannot translate unsupported message fields or reconstruct omitted records. Validate the protocol and selected model rather than treating continuity as a routing-only setting.
What changes when the conversation includes tools or streaming?
A conversation containing tools or streamed responses needs more than a list of visible chat bubbles. Preserve the protocol-required assistant fields, tool-call relationships and completion state. A partial display is not automatically a completed assistant turn. Build history from the structured response contract, with an explicit policy for interrupted or failed operations.
Token360’s capability guide describes appending the assistant tool-call message and a tool result with the matching call ID. It also requires retaining the fields needed by reasoning workflows. Saving only the final prose can discard information required for the next supported turn.
For streaming, collect the supported deltas according to the endpoint and track whether a terminal event arrived. If a connection closes unexpectedly, record the partial result as partial. A policy may allow the user to keep useful text, but do not silently represent an unfinished tool call as a completed operation.
Our two downloadable examples intentionally exclude these cases. They demonstrate a simple history omission, not a universal serialization format. Before extending them to a tool-enabled or reasoning model, validate a complete second turn using the exact fields returned by that model and protocol. Do not expose hidden reasoning merely because the application needs to preserve an opaque field.
How should an application test conversation continuity?
Test continuity at three boundaries: the correct history is loaded, the expected context is submitted, and the answer uses it appropriately. Record the first two independently of model output. A correct answer alone can be accidental, while an incorrect answer does not identify whether history storage, request assembly or model behavior caused the failure.
Start with a harmless, distinctive fictional fact and a follow-up that depends on it. Inspect the outgoing request in a controlled environment with secrets removed. Confirm the fact appears in the intended role and conversation. Then assess the answer. This is a proposed validation procedure; no recall result is reported here.
Proposed case
Inspect before assessing the answer
Expected application property
Normal second turn
Prior fact and actual assistant reply.
Relevant history is included once and in order.
New conversation
Fresh record and outgoing context.
Unrelated prior history is not attached.
Two different users
Authorized history lookup and selected records.
Each request contains only permitted context.
Page reload or backend restart
Chosen storage and reconstruction behavior.
Continuity matches the product’s stated behavior.
Interrupted assistant response
Completion state and subsequent input.
Partial output is not silently promoted to a finished turn.
For concurrent messages, define whether your product serializes turns or branches them. Two requests assembled from the same old snapshot can each omit the other’s exchange. A model cannot reason over a turn that was not in its input; ordering and versioning belong in the conversation system.
Keep diagnostic records proportional to what you need. Full conversation dumps may contain sensitive information. The fact that an application manages history does not determine its retention or training policy, and this guide makes no platform-wide privacy promise. Follow the documented data-handling boundary and your application’s own controls.
Conversation continuity depends on the context mechanism, not on a model appearing to remember a prior answer. Keep interface history, application records and supported API state distinct. The answers below summarize integration boundaries; they do not establish retention guarantees, recall accuracy or support for features that the selected endpoint has not documented.
Does using the same API key preserve a conversation?
An API key authenticates requests. Do not treat it as a history-retrieval instruction. Use the endpoint’s supported context mechanism and your application’s authorized conversation record.
Why does a Playground remember something my API call forgets?
The interfaces may assemble different inputs. Compare the actual context sent by your integration rather than assuming that the model or account alone explains the difference.
Should I resend every message forever?
No. Select sufficient authorized context within the model’s limits and your product’s policy. How to shorten history without losing conditions is a separate preservation problem, not evidence of automatic server memory.
Does switching models erase the history in my database?
Not inherently. Database persistence is an application concern. Whether the next model can consume the stored history depends on the format, capabilities and context your application supplies.
Can a response ID be both a trace reference and a continuation ID?
Some APIs document continuation behavior for response IDs. Others do not. Verify the specific endpoint; an ID’s existence or name alone is insufficient.
Developer Guide
API
Build faster with one AI API.
Use Token360 to call video, image, audio, and text models with one key and one bill.
Guides
Guides
Guides