Evaluate GPT-5 reasoning with a checkable scheduling task. Count rejected attempts, output usage and elapsed time before comparing accepted-task cost.
By Yuki Yu · November 4, 2026 · No live model test claimed
Evaluate GPT-5 reasoning cost against accepted tasks, not answer length. Define a checkable result, record usage and charges for every attempt, and include rejected answers in the cost calculation. A response that sounds convincing but violates a constraint has not completed the task.
Read first. The model evaluation framework explains how to define acceptance criteria. This article applies that idea to one constrained scheduling problem and its complete attempt cost.
What does GPT-5’s model status mean for this guide?
Treat this as an evaluation guide for an existing GPT-5 integration, not a recommendation to choose an older model for a new product. Confirm lifecycle status and your serving route first. A model can be useful as a historical baseline even when a newer candidate deserves the production evaluation.
As checked November 4, 2026, OpenAI’s GPT-5 reference marks GPT-5 Deprecated and lists minimal, low, medium and high reasoning effort. That upstream list does not establish which parameters a Token360 route accepts. Check the selected model schema before sending optional controls.
The historical catalog ID for this series is gpt-5. No successful GPT-5 run was supplied for this article. The scheduling result is derived locally, and all cost examples are explicitly illustrative.
Which task gives you a checkable reasoning answer?
Use a task with a small enough solution space to verify independently. Scheduling is useful when dependencies and resource limits are explicit: an answer can satisfy the dependency graph while still violating worker capacity. Write both rules into the prompt so the evaluator is not grading hidden requirements.
One worker completes three non-preemptive tasks.
A takes 20 minutes. B takes 15 minutes. C takes 10 minutes.
A must finish before either B or C starts.
Tasks cannot overlap. Work starts at minute 0.
Return a feasible order, each start/end time, and the minimum
completion time. Explain the resource constraint in one sentence.
With one worker and no overlap, the work requires 20 + 15 + 10 = 45 minutes. The valid orders are A → B → C and A → C → B. The second order is equally correct; an evaluator that accepts only the first would misclassify a valid response.
The answer key was checked by enumerating all six permutations locally. Two obey the dependency rule, and both finish at 45 minutes when scheduled without gaps. This verifies the fixture, not GPT-5’s ability to solve it.
Synthetic fixture and independently checked answer. This is not a model-generated schedule.
Why can an apparently good answer still fail?
Inspect the schedule, the stated completion time and the explanation as separate artifacts. A model might name a valid order but report the wrong duration, or calculate 35 minutes by overlapping tasks that share one worker. Reject contradictions that would cause the downstream application to execute an impossible plan.
Candidate output
Decision
Reason
A 0–20; B 20–35; C 35–45
Accept
Dependencies and worker capacity hold
A 0–20; B 20–35; C 20–30
Reject
B and C overlap on one worker
B 0–15; A 15–35; C 35–45
Reject
B starts before A finishes
A 0–20; C 20–30; B 30–45
Accept
Alternative valid ordering
These candidates are editorial examples. They are not observed model responses. Extend the test set with a second worker, changed dependencies or an impossible deadline, but create a fresh answer key for each variation. Do not assume that “45” remains correct after changing the problem.
Which usage belongs in the reasoning cost?
Use the API’s reported usage and reconciled charges, not the visible word count. Reasoning and final-answer accounting depend on the model and serving contract. Retain detailed fields where exposed, but avoid adding a reasoning-token detail to an output total that already includes it.
OpenAI’s reasoning-model guide explains that reasoning tokens consume output capacity and are billed as output tokens in its API. For a Token360 request, verify the returned usage structure and applicable billing record rather than assuming identical reporting fields.
Record one row per attempt: logical task ID, attempt ID, model, requested effort, acceptance outcome, elapsed seconds, input usage, output usage and charge. Keep incomplete responses and errors in the log. A failure with no confirmed charge should be marked unresolved rather than silently entered as free.
Accounting framework; no measured GPT-5 performance or price is shown.
How should you compare two candidate configurations?
Apply the same task set and acceptance rule to both configurations, then compare accepted results, total billed spend and complete-response time. Report the number of logical tasks and attempts separately. A cheaper single call can be the more expensive configuration if it needs more retries or produces more rejected answers.
In an invented ten-task comparison, configuration A costs $0.20 across all attempts and accepts eight tasks: $0.025 per accepted task. Configuration B costs $0.30 and accepts all ten: $0.030 per accepted task. A has the lower accepted-task cost, but its two unresolved tasks may make it unsuitable for the product.
Decide the minimum acceptable completion coverage before viewing the results. Otherwise, a configuration can appear economical simply by failing the difficult work. Report review labor and tool charges separately when they are part of the actual workflow.
What should happen when GPT-5 fails the constraint check?
Preserve the rejected result and identify the failed constraint before retrying. A bounded repair prompt can state the specific violation, but the repaired answer must pass the same validator. Keep repair attempts under the original task so their time and spend remain visible.
For this fixture, “B and C overlap on one worker; return a non-overlapping schedule” is actionable feedback. Repeating “think harder” does not identify the error. If a configuration repeatedly fails a mandatory condition, route the task to review or a separately validated alternative rather than retrying without a limit.
Before running a paid pilot, establish a working request through the Token360 quickstart. Use the public model ID and only parameters supported by the selected route. Do not put a secret key into a shared prompt, screenshot or article example.
Score the schedule without trusting its explanation
A plausible explanation cannot rescue an infeasible schedule. Evaluate the returned intervals before reading the model’s prose: A must last twenty minutes, B fifteen and C ten, one worker cannot overlap tasks, and A must precede both other tasks. These are checks derived from the stated fixture.
Authored candidate
Result
A 0–20, B 20–35, C 35–45
Feasible, finishes at 45
A 0–20, C 20–30, B 30–45
Also feasible; do not reject a different valid order
A 0–20, B 20–35, C 20–30
Reject: B and C overlap on one worker
A 0–20, B 25–40, C 40–50
Feasible but not minimum completion time
All candidates here are human-authored. They expose two distinct failures: violating a constraint and missing the optimum. When testing an actual GPT-5 route, preserve both scores rather than reducing every outcome to whether it printed “45.” Charge all attempts to the task, including a corrected answer, and report the first-pass result separately from the eventual accepted schedule.
Frequently asked questions
Is a longer answer evidence of more useful reasoning?
No. Score the returned schedule against the constraints. Extra explanation can repeat the problem without repairing an invalid result.
Is 45 minutes a GPT-5 benchmark result?
No. It is the independently derived minimum for the synthetic one-worker fixture. The article does not report a GPT-5 run.
Should failed attempts count in cost?
Count any billed attempts in total spend, including rejected outputs. Reconcile uncertain charges instead of inferring billing from an error alone.
Can I use the same fixture to compare another model?
Yes. Keep task wording and the grader fixed, record configuration differences, and include harder representative tasks before making a production decision.
Comparison
Guides
Guides