Papers
arxiv:2611.16961

The Value of Thinking: Learning When to Reason in Hybrid Decision Models

Published on Nov 12
· Submitted by
Zhengxu Yu
on Nov 13
Authors:
,
,
,
,
,

Abstract

A hybrid decision model routes between implicit single-forward-pass answering and full autoregressive reasoning via a self-routing block trained with calibrated decision policy optimization, reducing token and latency costs while preserving accuracy.

Existing adaptive-reasoning methods shorten the explicit reasoning a model writes, or skip it by switching modes; either way the answer is still written out token by token. Rather than deciding how much to think, we explore a different paradigm, zero-or-T thinking: a single model either answers from the implicit thinking inside one forward pass, generating no token, or performs full autoregressive reasoning at a cost of T tokens when that is worth paying. We realise it in a hybrid decision model (HDM), where one frozen backbone serves both modes and a lightweight self-routing block chooses between them. The block estimates the value of thinking, the utility the slow mode is expected to add over the fast one, from what that implicit thinking has already produced, and the slow mode runs only where that value exceeds a price λ set per request. Where no per-decision label exists, as in episodic agentic tasks, we introduce Calibrated Decision Policy Optimisation (CDPO), a reinforcement-learning recipe that trains the block from episode-level returns. We evaluate HDM on MMLU-Pro, AGIEval, GPQA-Diamond, JevBench, BANKING77 and MiniWoB. Specifically, on JevBench HDM-4B scores 0.857 against Jev's 0.851 (+0.7%) at a median 0.179\,s against 0.652 (-72.5%). On the agentic venue MiniWoB HDM-27B cuts token cost by 80.1%.

Community

Paper submitter

HDM, a hybrid decision model: one that decides for itself whether an answer needs autoregressive generation at all, so a question costs either no token or a full reasoning pass.

Letting the model answer from one forward pass and only paying for T tokens when the value of thinking exceeds a price is a clean way to frame it. Pricing it per request with lambda is the bit I like. How stable is the router when the price changes at test time without retraining?

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend

Deciding when thinking is worth paying for is an estimator's problem. You don't do a full takeoff on every lead. You glance, and only the jobs that might pay off get the long look. Setting the price per request is a nice touch. I'd be curious how often the quick answer was wrong on cases the router skipped.

The routing block worries me more than the reasoning does. The errors are asymmetric: send a hard query down the zero-token path and you get a confident wrong answer with nothing in the logs to flag it, while the reverse mistake just burns tokens. So average token savings is the wrong headline number — I'd want per-route accuracy and the router's calibration right at the boundary, where it's essentially flipping a coin.

Before anything gates production traffic I'd run it in shadow mode — log the route it would have taken next to what the full-reasoning baseline actually answered, for a few weeks, and diff them. One frozen backbone serving both modes is the part that would get me to try it at all: same weights, same eval suite, one deployment to babysit.

·

The router can be wrong in two ways, and I agree they do not cost the same. If it skips reasoning on a question that needed it, the answer comes back wrong with no warning. If it sends a question to reasoning that did not need it, the only cost is tokens. So the number of tokens saved is not enough on its own, and below is the accuracy of each route separately, on the larger backbone and on subjects the router was not trained on. The tables show four things. First, the questions answered without reasoning are right almost as often as they would be with reasoning, so few wrong answers come from skipping it. Second, the questions sent to reasoning are mostly ones the model gets wrong without it, and nearly every question that reasoning fixes is among them. Third, when reasoning is made more expensive, the questions that stop being sent to it are mainly ones the model answers correctly either way, so near the threshold the choice of route rarely changes the answer. Fourth, and this is a weakness: the router also sends most of the questions that reasoning cannot fix, or makes worse, to reasoning, and those tokens are wasted. All of this is one run on one benchmark, so I agree that logging the router's choices on real traffic before it controls anything is the right next step.

Qwen3.8-27B, MMLU-Pro, 346 held-out questions, reasoning budget 4096 tokens. Fast only scores 65.0%, always reasoning 90.2%. Of the 346, reasoning rescues 96 (fast wrong, reasoning right), 216 are right either way, 25 are wrong either way, and reasoning spoils 9 (fast right, reasoning wrong).

Which questions get sent to reasoning (count, and share of that group)

Price of reasoning λ Sent to reasoning (of 346) Rescued by reasoning (of 96) Right either way (of 216) Wrong either way (of 25) Spoiled by reasoning (of 9)
0.1 161 (46.5%) 89 (92.7%) 50 (23.1%) 15 (60.0%) 7 (77.8%)
0.05 167 (48.3%) 90 (93.8%) 55 (25.5%) 15 (60.0%) 7 (77.8%)
0 219 (63.3%) 94 (97.9%) 97 (44.9%) 19 (76.0%) 9 (100%)

Accuracy on each route, next to what the other route scores on the same questions (taken from the fast-only and always-reasoning runs)

Price of reasoning λ Overall accuracy Questions answered without reasoning Their accuracy Accuracy of those same questions if they had been sent to reasoning Questions sent to reasoning Their accuracy Accuracy of those same questions if answered without reasoning
0.1 88.7% 185 (53.5%) 90.8% 93.5% 161 (46.5%) 86.3% 35.4%
0.05 89.0% 179 (51.7%) 91.1% 93.3% 167 (48.3%) 86.8% 37.1%
0 89.6% 127 (36.7%) 93.7% 95.3% 219 (63.3%) 87.2% 48.4%

Hi authors, thanks for the work. A question on Table 2: routed HDM-4B has a JevBench P50 of 0.179 s, below fast-only HDM-4B at 0.226 s, although routing always runs the fast pass first. Were the two rows timed differently? Details in this GitHub issue: https://github.com/zhengxuyu/HDM-code/issues/4

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2611.16961
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 1

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.16961 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.16961 in a Space README.md to link it from this page.

Collections including this paper 2