HTTP 200 does not prove an AI stream finished. Check terminal events, preserve partial output, and separate retry attempts in your Token360 client.
STREAMING · RELIABILITY · BLOG 167
By Yuki Yu · Editorial update: November 6, 2026
An AI stream can return HTTP 200 and still fail before the answer finishes. The status arrives before the full stream has been received. Your application needs to inspect subsequent events, the completion reason and the protocol’s terminal signal before treating partial text as a completed result.
This topic starts with OpenRouter’s FAQ question about how streaming works. The practical integration question is what to do when the response starts successfully but the user receives only part of the answer.
This article combines official documentation with an executed, eight-case local event-state fixture. The fixture uses synthetic events; it is not a Token360 outage test, a network benchmark or an estimate of model reliability. No paid inference requests were made.
Read first: Token360’s response protocol compatibility guide identifies the contract your parser should follow. The endpoint determines that contract, rather than the upstream provider selected behind it.
Why can a stream fail after HTTP 200?
The initial HTTP status and the final outcome describe different moments. Once response headers have been sent, a later failure cannot replace that status. The application may instead receive an error event or lose the connection. A successful start therefore needs a separate record from a successfully completed generation.
OpenRouter’s streaming documentation explicitly describes errors after the response is committed. Its error event format depends on the endpoint. That establishes a real integration hazard, but it does not establish that another gateway uses the same error fields. Implement against the selected service’s documented contract.
Observation
What it establishes
What remains unknown
HTTP 200 received
The response began with a successful HTTP status.
Whether generation and delivery will finish.
Text appears
Some content reached the application.
Whether it is complete or usable.
Connection closes
No more data is arriving on that connection.
Whether closure followed the expected terminal signal.
Normal terminal signal and acceptable completion reason
The client has evidence of protocol completion.
Whether the answer meets the task’s quality requirements.
For example, an assistant may begin “The three required documents are…” and disconnect after listing one. The text is meaningful and the HTTP status is successful, yet saving it as the final checklist would hide a missing deliverable.
What should a Token360 client inspect?
Track the stream using the endpoint’s documented events and fields, then translate them into your application’s own states. Keep received text separate from completion evidence. An adapter should record errors, finish information and the terminal signal, rather than treating every event containing text as progress toward an automatically successful response.
Token360 documents Chat Completions-compatible SSE chunks, Responses-compatible events and Messages-compatible events. Its protocol guide says Chat Completions clients should wait for [DONE], tolerate optional fields and avoid assumptions about chunk boundaries. A parser for one endpoint should not be reused unchanged for another.
The application design proposed here has three separate questions: did the transport end normally, did the model finish the expected kind of response, and did the result satisfy the user’s task? A tool handoff can be a legitimate next step without being the final user-facing answer. A length-limited response needs a different label from an ordinary completed answer.
Recommended application checkpoints. These labels are internal design choices, not Token360 API field names.
Do not let a terminal marker erase an earlier error. Preserve all relevant observations until classification. Equally, do not conclude that a well-formed, completed answer is factually correct: business validation remains a separate step.
How can you test completion handling without spending tokens?
Feed deliberately constructed event sequences into the part of your client that decides whether an answer is finished. Include successful endings, errors, interruptions and non-answer handoffs. This isolates a client-side decision rule without asking a live model to fail on demand or confusing model behavior with application logic.
We ran the following eight cases locally on November 6, 2026. Every sequence represents an already-open response. Events are normalized labels such as text, finish and terminal, not captured provider payloads. The fixture deliberately does not implement HTTP, SSE framing, UTF-8 decoding, concurrent requests or tool execution. It assumes one response stream with one output choice and normalized events in valid order. It does not validate event ordering or separate interleaved choices; those require adapter-level tests.
Synthetic sequence
Actual local classification
Recommended UI treatment
Text → stop → terminal
complete_candidate
Allow task-level validation.
Text → connection ends
incomplete
Label the fragment incomplete.
Error before text
failed
Show failure, not a blank success.
Text → error → terminal
failed
Keep the error despite the terminal signal.
Text → length limit → terminal
limited
Explain that output hit a limit.
Tool handoff → terminal
tool_handoff
Continue the approved tool workflow.
Heartbeat → connection ends
incomplete
Do not count a heartbeat as an answer.
Text → terminal, no finish information
needs_review
Apply the endpoint-specific policy.
All eight returned their expected classifications. That verifies these eight assertions in this illustrative classifier; it is not an eight-out-of-eight model success rate or proof of production readiness. The last case is deliberately conservative: your actual adapter must account for the completion evidence the chosen endpoint provides.
Copy the
Copy stream-state-fixture.py:
"""Synthetic normalized events; NOT a wire parser or captured API traffic."""
import json
from pathlib import Path
def classify(events):
text = ''; terminal = False; reason = None; error = False
for kind, value in events:
if kind == 'text': text += value
elif kind == 'finish': reason = value
elif kind == 'error': error = True
elif kind == 'terminal': terminal = True
if error: state = 'failed'
elif not terminal: state = 'incomplete'
elif reason == 'tool_calls': state = 'tool_handoff'
elif reason == 'length': state = 'limited'
elif reason == 'stop' and text: state = 'complete_candidate'
else: state = 'needs_review'
return state
cases = [
('normal text', [('text','Hello'),('finish','stop'),('terminal',None)], 'complete_candidate'),
('text then disconnect', [('text','Partial')], 'incomplete'),
('error before text', [('error','upstream')], 'failed'),
('text then error then terminal', [('text','Partial'),('error','upstream'),('terminal',None)], 'failed'),
('limit then terminal', [('text','Partial'),('finish','length'),('terminal',None)], 'limited'),
('tool handoff', [('finish','tool_calls'),('terminal',None)], 'tool_handoff'),
('heartbeat then disconnect', [('heartbeat',None)], 'incomplete'),
('terminal without finish', [('text','Maybe'),('terminal',None)], 'needs_review'),
]
results=[]
for name, events, expected in cases:
actual=classify(events)
assert actual==expected, (name,actual)
results.append({'case':name,'expected':expected,'actual':actual})
Path(__file__).with_name('fixture-results.json').write_text(json.dumps(results,indent=2))
print(json.dumps(results,indent=2))
. Run it with python3 stream-state-fixture.py; it uses only the Python standard library and makes no network calls. Extend it with sanitized real events before using the design in production.
How should the interface handle partial output?
Keep partial output visible only with a clear unfinished state, and prevent downstream actions that assume a completed answer. A user can still inspect or copy a fragment, but the application should not silently approve, export or execute it as final. Define those restrictions according to the consequence of the task.
In a drafting tool, a useful message is “Generation interrupted; this draft may be incomplete.” In a procurement workflow, leave the recommendation unapproved. In a structured extraction workflow, avoid inserting a truncated record into the operational database simply because its opening fields were valid.
Use separate records for the conversation turn and each generation attempt. For instance, turn T-17 can contain attempts A-1 and A-2, each with its own output, completion state and correlation IDs. That makes a retry reviewable without replacing the evidence of the original interruption. These identifiers are illustrative application IDs, not prescribed API fields.
Should you retry and append the new text?
Do not append a fresh generation to a failed fragment by default. A new attempt can produce different wording, assumptions or structure, so joining the outputs may create an answer that neither attempt actually produced. Choose explicitly between replacing the draft, asking for a continuation or escalating the failure for review.
Token360’s interrupted-stream guidance says to treat closure before the normal terminal event as incomplete, label or discard partial output, and consider duplicate work and cost before retrying. It also recommends bounded retry behavior and retaining correlation information. That is a recovery policy, not a guarantee that retrying fixes the cause.
For a simple text draft, replacement may be acceptable. For an agent that can change external state, first determine whether any action already happened. A second model request does not roll back a tool action, and an interrupted answer is not evidence that all earlier steps failed.
If you offer “Continue,” disclose that it starts another generation using the context you supply. Do not describe it as byte-exact resumption of the original stream unless the endpoint explicitly provides and verifies that capability.
What should you record to investigate an interruption?
Record enough lifecycle evidence to distinguish a response that never delivered text from one that stopped midway. Keep the HTTP status, endpoint, public model, attempt ID, available request correlation IDs, observed finish information and whether a terminal signal arrived. Add timestamps without treating any one timestamp as proof of completion.
A compact record can include headers_received_at, first_text_at, last_event_at, terminal_seen, error_seen and application_state. These are proposed internal fields. Use a consistent clock for elapsed durations and retain a UTC wall-clock time for support correlation.
Keep usage accounting separate from stream classification. If an interrupted connection prevents the client from receiving usage data, record usage as unknown pending reconciliation. Do not replace it with zero. Avoid including authorization headers or raw sensitive prompts in a diagnostic export.
For an enterprise deployment through Token360, the practical acceptance check is whether the application can distinguish started, interrupted and accepted work. The platform’s enterprise focus does not remove the need to verify those client behaviors for the endpoint and model used.
Does a complete-looking sentence prove that streaming finished?
No. A fragment can end at a sentence boundary by chance. Use the completion evidence supplied by the protocol, then assess whether the requested task was fulfilled.
Does every interrupted stream require another request?
No. A user may cancel, retain a clearly labeled fragment or ask for human review. Retrying is a product decision constrained by cause, budget and possible duplicate actions.
Can these local tests tell me which model disconnects least?
No. They test a small application classifier with synthetic events. Comparing real reliability requires actual requests, a defined workload and a consistent measurement window.
Prompt Library
Guides
Guides