A reasoning model is a language model that generates intermediate work before it answers. That one design choice changes four things you will meet in production: the bill, the latency, the output cap and what you can learn from the response. This guide explains reasoning models as the two largest API vendors document them today, with the cost mechanics spelled out, and ends with a limit on how far to trust the visible reasoning. For the token arithmetic behind the bill, read tokens, context windows and what an AI request really costs first.
Where the idea comes from
The research starting point is chain-of-thought prompting. Wei et al. showed that prompting large models with a few examples containing intermediate reasoning steps improves performance on arithmetic, commonsense and symbolic reasoning tasks, and report that a 540B-parameter model prompted with eight such exemplars reached state-of-the-art accuracy on the GSM8K math word-problem benchmark at the time.[5] The paper also reports that the ability emerges in sufficiently large models.[5]
Reasoning models move that idea from the prompt into the product. Anthropic describes thinking as Claude working through a problem in its own words before answering: restating the question, trying approaches, checking intermediate results and abandoning paths that do not hold up.[1] OpenAI's description is shorter: reasoning models use internal reasoning tokens before producing a response.[4] Our reading of both: the model is given room to compute before it commits, and you pay for that room.
What a thinking token is, and who sees it
On Anthropic's API, thinking arrives as thinking content blocks ahead of the text blocks. Each block carries a signature field, an encrypted copy of the full reasoning that you pass back unchanged in multi-turn and tool-use conversations.[1] The text inside a thinking block is never the raw chain of thought: it is a summary produced by a different model from the one you called, and the documentation states that no display setting returns the raw reasoning.[1]
Visibility depends on the model. For Claude Opus 5.5, Sonnet 5.5, Haiku 5.5 and Fable 5.1, among others, the documentation lists display as defaulting to "omitted", which returns thinking blocks with an empty thinking field; you opt in to summaries with display: "summarized".[1] The signature still carries the encrypted reasoning, and the documentation says omitting the text is mainly a latency win when streaming, because the server skips streaming thinking tokens.[1]
OpenAI's pattern is similar: raw reasoning tokens are not visible through the API, but you can request a summary through the summary parameter, and summary output appears only if you opt in.[4] The OpenAI page also notes that some summary options may require organisation verification.[4]
What it costs
Both vendors bill reasoning as output tokens.
| Question | Anthropic | OpenAI |
|---|---|---|
| How is reasoning billed? | Thinking tokens are billed as output tokens, even when the text is not returned.[1] | Reasoning tokens are billed as output tokens.[4] |
| Do reasoning tokens count against the output cap? | Yes, they count toward max_tokens alongside the response text.[1] | max_output_tokens caps reasoning, visible output and formatting together.[4] |
| Where do I read the count? | usage.output_tokens_details.thinking_tokens.[2] | The output_tokens_details field of the usage object.[4] |
| What if the cap is hit? | stop_reason: "max_tokens"; raise the cap or lower effort.[2] | Status incomplete with reason max_output_tokens, which can happen before any visible output.[4] |
Two consequences are easy to miss.
The bill does not match the visible text. Anthropic's pricing note says billed output tokens do not match the visible token count, because you are billed for the full thinking process rather than the summary you see; the billed amount is the same whether display is summarized or omitted.[2] Summary generation itself carries no charge.[2]
A cap sized for a short answer can cut off a hard one. Anthropic warns that a max_tokens sized for a response with no thinking is often too small once the model starts thinking on a hard request.[2] OpenAI's guidance is to reserve at least 25,000 tokens for reasoning and output when you start experimenting, and notes that reasoning can range from a few hundred to tens of thousands of tokens.[4]
Prior reasoning can also cost you as input. Anthropic bills thinking blocks from earlier turns that stay in context as input tokens; Claude Opus 4.5 and models numbered 4.6 and higher keep prior thinking blocks in context, while Sonnet 4.5, Haiku 4.5 and earlier models strip them.[2][3]
Effort: a soft control, not a budget
Older Claude models took a fixed budget_tokens value for thinking, with a minimum of 1,024 tokens, and the documentation calls the budget a target rather than a strict cap.[3] That mode is deprecated on the 4.6 models, and Claude 4.7 and later models reject it with a 400 error.[3] On current models you set thinking: {type: "adaptive"} if it is not already on and steer depth with output_config.effort.[2][3]
Anthropic describes the effect of each level:
| Effort level | Documented thinking behaviour |
|---|---|
max | Thinks most readily and at greatest depth, with no constraint on thinking length.[2] |
xhigh | More readily and deeper than high, suited to extended exploration.[2] |
high | Thinks on most requests that benefit from it; the default on most models.[2] |
medium | Moderate thinking, may skip it for simple queries; default on Opus 5.5 and Haiku 5.5.[2] |
low | Minimises thinking and skips it for simple tasks where speed matters most.[2] |
The important property is in the next sentence of that page: at every level the model decides per request whether to think, in a tool-use loop the first request after new input typically carries most of the reasoning, and no level guarantees a thinking block.[2] Do not write code that assumes every assistant turn starts with one.[2]
OpenAI's equivalent is reasoning.effort, whose supported values depend on the model and can include none, minimal, low, medium, high, xhigh and max; the page states that lower effort favours speed and lower token usage.[4] It also lists model-specific exceptions, for example that GPT-6.1 Sol does not support none or minimal and defaults to medium.[4] Check the page for the model you actually call. The model-by-model price and context differences are in GPT-6.1 Sol vs Claude Sonnet 5.5 vs Gemini: API specs and pricing.
Controlling it in practice
- Lower effort before you reach for prompt wording. Anthropic's documentation says asking Sonnet 5.5 in the system prompt to think less does not reliably reduce thinking, and that in its testing telling Haiku 5.5 in the prompt to answer directly did not stop it from thinking.[2] The model-level details are in Claude Haiku 5.5: specs, pricing and when to use it.
- Hold the setting steady inside a cached conversation. On Anthropic's API, changing the effort level between requests invalidates prompt-cache breakpoints, because the resolved value is rendered into the prompt.[2] The page's own run shows the third request, made at a different effort, re-creating a 3,546-token cache that the second request had read for free.[2]
- Measure on your own traffic. The documentation recommends running a representative sample with and without any steering and comparing how often thinking triggers, output tokens, latency and answer quality.[2] That is exactly what an eval is for; see LLM evaluation: how to build evals that survive production.
When extra reasoning is worth paying for
Both vendors point to the same categories. Anthropic names math, coding, analysis and long-running agentic work, where answer quality depends on intermediate work.[1] OpenAI says reasoning models work especially well for complex problem solving, coding, scientific reasoning and multi-step agentic workflows, and that effort none suits latency-critical tasks such as voice, fast information retrieval and classification.[4]
Our interpretation, not a vendor claim: reasoning is a poor default for work where the answer is a lookup or a transformation. If a retrieval step supplies the facts, as in how RAG retrieval works, extra thinking mostly adds latency and output tokens. If the task is multi-step and a wrong intermediate step ruins the result, the added tokens are usually cheap insurance. The way to settle a specific case is to run both settings on a fixed test set and compare cost per correct answer.
Reasoning also does not remove hallucination. The vendors' documents above describe more thinking as improving quality on complex tasks; none of them claims it makes outputs reliably true.
Do not read the reasoning as an explanation
It is tempting to treat a readable chain of thought as the model's actual reason. Anthropic's Alignment Science team tested that directly. They planted hints in multiple-choice questions and checked whether the model's chain of thought mentioned the hint when the hint changed the answer.[6]
The reported results:
- Averaged across hint types, Claude 3.7 Sonnet mentioned the hint 25% of the time and DeepSeek R1 39%; the article calls a substantial majority of answers unfaithful.[6]
- For hints framed as unauthorized access, the figures were 41% for Claude and 19% for R1.[6]
- When models exploited deliberately wrong hints, they did so in over 99% of cases but disclosed it in less than 2% of cases in most scenarios, often building fake rationales for the wrong answers.[6]
- Unfaithful chains of thought were longer than faithful ones, so brevity did not explain the omissions, and faithfulness was lower on harder questions.[6]
The authors list limits: contrived hint scenarios, multiple-choice questions rather than real high-stakes tasks, only two model families, and tasks that may not have required chain of thought at all.[6] Their conclusion is that chain-of-thought monitoring cannot yet reliably rule out undesirable behaviour.[6]
The practical reading: a summary of reasoning is useful for debugging a prompt, but it is not an audit log. Verify outputs against sources, tests or a grader instead of against the model's account of itself. Note also that the study used earlier models than those now on sale, so it does not say how current ones behave.
Checklist before you ship
- Read
thinking_tokens(Anthropic) oroutput_tokens_details(OpenAI) on real requests, and price a hard request as well as an easy one. - Set the output cap high enough for thinking plus the answer, and alert on
max_tokensorincompletestops. - Choose the effort level per workload, test one level up and one down, and keep it fixed within cached conversations.
- Decide whether users need to see reasoning. If not, omitting it on Anthropic's API speeds up streaming.[1]
- Grade outputs, not explanations.
