Browse notes
Reasoning
Some models let you adjust how much work they spend on a request before producing an answer or choosing the next action. APIs and agent harnesses often expose this as reasoning effort, with levels such as low, medium, or high.
The setting does not describe the model’s internal process step by step. It gives more or less room to tasks that require connecting evidence, testing hypotheses, or revising a decision after observing a result.
When to increase effort
Extracting one value from JSON or classifying a straightforward input usually needs little reasoning. Investigating an ambiguous failing test may require reading several files, relating a recent change, running the test, and interpreting the next error.
For tasks like that, additional effort can improve the investigation, while also increasing unnecessary exploration. A practical way to choose a level is to test representative work and see where the result begins to stabilize.
If medium consistently solves the tasks that matter, high is useful when it improves something measurable: correctness, success rate, or the number of interventions required.
Latency, tokens, and cost
More reasoning commonly increases latency and usage. Accounting varies by API: a provider may include that work in output usage, expose it separately, or report different metrics.
For a useful comparison, record at least:
- Whether the task completed correctly.
- How long it took.
- How many tokens or calls it consumed.
The difference may be small for one request. In an agent, it is paid across many inferences and can grow further through retries or subagents. API Pricing covers that calculation in more detail.
Reasoning inside the agent loop
Agent reasoning happens between observations, not only before the final answer. The model can read a file, inspect the result, change its hypothesis, and run a test; the new output then shapes the next decision.
For example, a coding agent may start by suspecting the implementation, read the test, and discover that the expected value is stale. That evidence changes the next step in the agent loop.
Higher effort can change both analysis and the tool calls the model chooses. Evaluate the behavior of the complete run rather than the length of its visible explanation.