Routing and Reliability

Token360 resolves each public model name to an eligible upstream route. The platform applies authentication, model configuration, route health, metering, and account policy before forwarding a request.

What the model name means

Use the public model name shown in the Token360 catalog. It is a stable product-facing identifier, not an upstream account, deployment, or API-key name.

A model can have one or more eligible service routes. Availability and parameter compatibility are evaluated against the configured route. Token360 does not promise that every request will use the same upstream provider unless you use session affinity or your enterprise configuration explicitly pins that behavior.

Route selection

The platform may consider:

• whether the route and its credentials are active;
• recent provider health and rate-limit state;
• model and parameter compatibility;
• tenant model-access and data-residency policy;
• capacity and routing policy configured for the account.

Token360 supports the session_id and x-session-id routing controls described below. Other provider-routing fields, such as arbitrary provider ordering, provider allowlists, price ceilings, or cross-model fallback, are available only when they appear in the selected model's published schema.

Session affinity

For multi-turn conversations and agent workflows, use a stable session identifier to prefer the same available upstream provider and credential after the first successful request. This gives prompt caches a chance to be reused from the start of a session and helps workflows that replay provider-specific state, including encrypted reasoning returned by some GPT models.

Session affinity is a routing preference, not a hard availability guarantee. Policy, model compatibility, data residency, cooldown, capacity, and provider health are always evaluated first.

Send a session identifier

You can provide the identifier in either place:

The value must be valid UTF-8 and no longer than 256 characters. Token360 uses it only for routing, derives a protected internal key from it, and does not forward the body field to the upstream provider.

Use a unique, opaque value for each conversation or agent run. Do not put API keys, access tokens, email addresses, or other secrets in it.

1curl https://api.token360.ai/v1/chat/completions \
2  -H "Authorization: Bearer $TOKEN360_API_KEY" \
3  -H "Content-Type: application/json" \
4  -d '{
5    "model": "gpt-6-astra",
6    "session_id": "agent-run-7f3a9c",
7    "messages": [
8      {"role": "user", "content": "Continue this task."}
9    ]
10  }'

The equivalent header form is:

1curl https://api.token360.ai/v1/chat/completions \
2  -H "Authorization: Bearer $TOKEN360_API_KEY" \
3  -H "Content-Type: application/json" \
4  -H "x-session-id: agent-run-7f3a9c" \
5  -d '{
6    "model": "gpt-6-astra",
7    "messages": [
8      {"role": "user", "content": "Continue this task."}
9    ]
10  }'

allow_fallbacks

Optional top-level boolean. Default true. Token360 consumes the field for routing and does not forward it upstream.

When no affinity preference exists yet (first request, expired idle window, or miss), allow_fallbacks=false does not change routing: the request still explores the normal candidate set. A successful response may establish the preference for later turns.

1curl https://api.token360.ai/v1/chat/completions \
2  -H "Authorization: Bearer $TOKEN360_API_KEY" \
3  -H "Content-Type: application/json" \
4  -H "x-session-id: agent-run-7f3a9c" \
5  -d '{
6    "model": "gpt-6-astra",
7    "allow_fallbacks": false,
8    "messages": [
9      {"role": "user", "content": "Continue this task."}
10    ]
11  }'

Lifecycle and scope

• The first successful request establishes the preferred route, even when no prompt-cache hit has occurred.
• Each later successful request refreshes a 10-minute inactivity window.
• Affinity is isolated by Token360 account, public model, and session identifier.
• The same session can carry its preference across Chat Completions, Responses, and Messages when the selected model and provider support those APIs.
• A failed request does not replace the remembered route.
• If the remembered provider or credential is no longer usable and allow_fallbacks is not false, Token360 fails open to another eligible route. The next successful route becomes the new preference.
• An explicit provider or route selection takes precedence over session affinity when that control is supported by the selected API and model.

Without session_id or x-session-id, existing prompt-based affinity remains unchanged. Token360 may derive a conversation fingerprint from the opening system/developer message and first user message, but applications must not rely on that implicit behavior for provider-specific encrypted state. allow_fallbacks=false still applies when that implicit affinity has already been applied.

This behavior follows the same client contract as OpenRouter provider sticky routing, with Token360's own routing, policy, and availability checks remaining authoritative.

Failure behavior

When no compatible route is available, the request returns a standard error. A route change is not a reason to change the public model name in your application.

Automatic route handling does not replace client resilience:

• retry only transient errors;
• cap attempts and total elapsed time;
• add exponential backoff and jitter;
• do not retry invalid requests or insufficient balance without correcting the cause;
• take extra care with asynchronous create operations, because an upstream operation might have started before the client observed a timeout.

Response and audit signals

Where available, capture:

• response model name;
• provider or route metadata;
• request or generation ID;
• X-Request-Id and X-Trace-ID response headers;
• status, latency, usage, and cost.

Use the console request detail and generation metadata endpoints to investigate the route that served a request. When contacting support, include correlation IDs and timestamps, but never include the API-key secret.

Data policy and routing

Enterprise data-residency settings constrain eligible routing. A preferred or required region is a routing policy; it does not by itself describe every retention, logging, or training behavior. Review Security and Data Handling for the full data path.

Design for continuity

• Keep public model names in application configuration rather than source code.
• Validate a replacement model before switching production traffic.
• Use streaming for interactive long responses when supported.
• Use Async Chat or Batch Jobs for work that should survive a client connection ending.
• Monitor success rate, P95 latency, and error categories by model.
• Keep enough spendable balance or credit for expected bursts.

What Token360 does not guarantee

• A fixed upstream provider for an unpinned route.
• Identical output across providers, model revisions, or retries.
• That retrying a non-deterministic request returns the same result.
• Cross-model fallback unless it is part of an explicitly contracted route.