Rate Limits, Timeouts, and Retries

Production clients should distinguish capacity limits, account policy, billing stops, and invalid requests. They can share an HTTP status while requiring different actions.

Limit dimensions

Token360 can enforce limits across several dimensions:

RPM — requests per minute.
TPM — input and output token throughput per minute.
Concurrency — requests currently in flight.
API-key spend limit — lifetime, daily, weekly, or monthly spend for a key.
Daily spend protection — account-level UTC-day protection.
Credit or wallet balance — available funds for new work.
Provider capacity — temporary upstream saturation or rate limiting.

Your commercial agreement and console configuration determine the applicable values. Do not assume a limit from another account or model.

Handling `429`

Inspect the error body, not only the HTTP status:


`rate_limit_exceeded` or temporary capacity limit	Retry with exponential backoff and random jitter. Honor `Retry-After` when present.
`insufficient_quota` or no spendable balance	Add funds, increase approved credit, or wait for the applicable budget reset. Repeated retries will not help.
API-key spend limit reached	Use the console to review the key's limit and reset period. Do not silently switch to an unrestricted key.
Daily spend protection active	An account owner must raise the cap above current UTC-day usage or deliberately disable protection.

Retry policy

A safe starting policy for synchronous inference is:

Retry transient 429, 500, 502, 503, and 504 responses.
Use exponential backoff with jitter, for example 1 s, 2 s, then 4 s plus randomness.
Set a small maximum attempt count and a total elapsed-time budget.
Stop immediately for validation, authentication, permission, balance, or unsupported-operation errors.
Log the request ID from every attempt.

Adjust this policy to the latency and duplication risk of your application.

Timeouts

Use separate connection and overall request timeouts. Interactive chat, large multimodal inputs, and video generation have different latency profiles.

For streaming, an application timeout must allow time for the first event and for progress between events.
For video and other asynchronous operations, timeout the submission request, then poll the returned resource instead of keeping one connection open.
A client timeout does not prove that upstream work never started.

Idempotency and duplicate work

Do not blindly retry create or submit operations after an ambiguous timeout. First check the request history or resource list using your correlation data. Webhook receivers must be idempotent because delivery can be retried.

For Batch Jobs, use a unique custom_id for every JSONL line so results can be reconciled without relying on order.

Interrupted streams

If an SSE connection closes before the normal terminal event:

treat the completion as incomplete;
discard or explicitly label partial output;
retry only if duplicate model work and cost are acceptable;
preserve the original request and correlation IDs for investigation.

When to contact support

Contact support when transient failures persist beyond your retry budget or affect multiple requests. Include the UTC time range, public model name, endpoint, HTTP status, error code, request ID, and trace ID. Remove prompts, files, authorization headers, and API-key secrets unless support explicitly requests a secure transfer.