Reasoning models spend extra computation while answering, often to work through a task before returning the visible response. That extra work can help with multi-step math, complex code, planning, and problems with several constraints. It can also make a response slower and use more tokens. For a routine rewrite or a one-field extraction, the extra effort may buy you nothing.
The useful question is not “Is this the smartest model?” It is “Does this task benefit enough from extra reasoning to justify the cost and wait?” This guide explains test-time compute, what a thinking budget does (and does not mean), where reasoning helps, and how to compare it with a faster general model.
What are reasoning models?
A reasoning model is a model or model mode designed to apply additional computation to harder tasks before it gives a final answer. Providers may describe that work through internal reasoning tokens, extended thinking, an effort setting, or another model-specific control.
“Reasoning” and “thinking” are convenient product terms. They do not mean the system has human awareness or experiences thoughts. In practical terms, the model has more opportunity to process the problem, consider intermediate steps, or use a longer internal generation before it returns the answer. The implementation and controls depend on the provider.
If you need a quick refresher on the basic next-token model, start with what an LLM is before comparing reasoning modes.
This happens at inference time, after training. Our earlier article, How LLMs Learn, covers how training shapes a model. Test-time compute is different: it is extra work during a particular request, not another training run that permanently changes the model.
What does test-time compute mean?
Think of ordinary generation as a model producing a response from its prompt. A reasoning-capable model may use more computation before settling on the final output. Depending on the system, it may generate internal tokens that are not shown as ordinary answer text, or the product may expose a setting that changes how much reasoning effort to apply.
That extra work can help when the problem has several linked steps. For example, a coding task may require reading a specification, spotting edge cases, choosing an approach, and checking whether the code meets the constraints. A short sentiment label usually does not need the same amount of problem-solving.
But more compute is not a truth detector. A model can spend longer and still make an incorrect assumption, misread a source, or produce a confident error. Reasoning effort changes the process; it does not replace evidence, tests, or human review.
Where extra reasoning can help
Reasoning modes are worth testing when a task has multiple dependencies, ambiguous instructions, or constraints that must all be satisfied. Common examples include:
- Multi-step mathematics: the answer depends on carrying intermediate results through several operations.
- Complex coding: a change touches multiple files, has hidden edge cases, or must fit an existing design.
- Planning: a useful plan must sequence actions while respecting time, resource, or policy limits.
- Constraint-heavy analysis: the model must compare several documents or facts and satisfy a detailed set of requirements.
- Ambiguous requests: the model may need to identify missing information rather than rush into a guess.
These are starting points, not guarantees. A strong general model may already handle a particular task well. A smaller reasoning model may fail on it. The model’s label is not your evaluation result.
When reasoning is probably unnecessary
Simple, well-defined tasks often work with a faster general model or a low-effort setting. Examples include rewriting a sentence, extracting a named field, assigning a sentiment label, or summarizing a short passage. If the task has one clear operation and a narrow answer, paying for a long internal process may only add delay.
For repetitive tasks, it can be better to improve the prompt or use a structured-output feature than to ask the model to reason harder. For instance, if an extractor keeps returning extra prose, specify the schema or output format and validate the result. More “thinking” is not a substitute for a clear interface.
There is also a middle ground. Use a fast model for routine cases, then send uncertain or high-impact cases to a reasoning model. That routing pattern can keep simple requests quick while reserving more compute for cases that need it. The trade-off depends on the cost of a mistake and the cost of waiting.
What is a thinking budget or reasoning effort?
A thinking budget usually refers to a limit or allowance for internal reasoning work. Reasoning effort usually selects a level, such as low or high, without promising an exact number of internal tokens. These terms sound similar, but they are not interchangeable across APIs.
For example, OpenAI’s Responses API documentation shows a model-specific reasoning.effort setting. Its supported values and defaults depend on the model; the field is a control, not a guaranteed fixed amount of computation. Anthropic documents manual thinking-token budgets for supported models, while its current documentation marks that mode as legacy and points to adaptive thinking for newer models. A setting copied from one model version may not work for another.
Other APIs have their own names and rules. A numeric budget may be a target, a cap, or one part of a larger output limit. Some products choose how much reasoning to use automatically. Always check the current documentation for the exact provider, model, endpoint, and version you call. Our guide to choosing OpenClaw model providers covers the broader decision of matching a provider to an application.
The quality, cost, and latency trade-off
Extra reasoning can improve the quality of some difficult answers, but it tends to require more work. That can mean more time before the final answer and higher token usage, depending on the model and billing rules. A fast general model can be the better tool when a task is simple and the answer needs to arrive quickly.
Measure the trade-off instead of relying on a marketing label. For a given task, track at least three things:
- Quality: Did the response meet the task’s requirements? Check correctness, completeness, and failure cases.
- Latency: How long did it take to return a usable result?
- Cost or usage: What did the request consume under the provider’s current pricing and token rules?
Do not compare unrelated examples. Use the same prompt, input, model version where possible, and success criteria. If the model can use tools, decide whether tool access is part of the test. A longer answer may look more thoughtful without being more correct, so score the result against a rubric or known answer rather than its tone.
How to prompt a reasoning model
“Think step by step” is rarely the most useful instruction on its own. Give the model a clear target, the evidence it may use, the constraints it must respect, and the format of a good answer. If important information is missing, tell it to identify the gap instead of guessing.
For a planning task, for example, describe the goal, available resources, deadlines, and forbidden actions. Ask for a plan that names assumptions and checks each constraint. For a code task, include the expected behavior, relevant files or interfaces, and tests that should pass. This helps the model focus its extra computation on the parts that matter.
Then inspect the result. Run the code, verify calculations, or compare document claims to their sources. A reasoning model can help produce a better first answer, but it does not remove the need to validate work that matters.
A practical way to decide whether it is worth it
- Choose a real task. Use work your application actually needs, not a toy prompt chosen to flatter one model.
- Build a small representative set. Include ordinary examples, edge cases, and a few known failures.
- Run a fast baseline. Record the model, prompt, output, quality, latency, and usage.
- Try a reasoning model or higher effort. Keep the task and success criteria the same.
- Compare the difference. Did it solve failures the baseline missed? Was the improvement worth the added time and cost?
- Route by need. Keep the simpler option for easy cases if it meets the bar; reserve extra reasoning for tasks where the comparison justifies it.
Even a small evaluation beats an intuition based on one impressive demo. If you cannot tell what “better” means for the task, define that before you increase the reasoning setting.
FAQ
Are reasoning models more accurate?
They can do better on some difficult, multi-step tasks, but they are not always more accurate. Test them on representative examples from your own workload.
Does more thinking always improve an answer?
No. Extra computation can add latency and usage without fixing a bad prompt, missing information, or model limitation.
Is a thinking budget the same as a reasoning-effort setting?
Not necessarily. A budget may refer to token allowance, while an effort level may be a qualitative control. Meanings, behavior, and support vary by API and model.
Should I ask a reasoning model to “think step by step”?
Give it the goal, relevant context, constraints, and desired output. A generic instruction to think step by step does not explain what success requires.
When should I use a regular model instead?
Use a faster general model when the task is clear and simple, such as a short rewrite or straightforward extraction, provided it meets your quality bar.
Try the comparison
Pick a planning, coding, or multi-step question you care about. Run it once through a fast general model and once through a reasoning model. Keep the prompt and success criteria the same. Check which answer actually meets the constraints, then compare how long each took and what each request used.
That is the key lesson: use extra reasoning when it earns its keep. In the next roadmap chapter, we’ll look at the context window and why a bigger prompt does not always mean a better answer.
Featured illustration generated with AI.