Test-time Compute and Verifiers

Spending the same amount of compute on the training stage versus on every request looks like the same accounting on the surface, but actually they are two different product decisions: one is a fixed cost paid once before deployment, and the other is a marginal cost incurred on every request.

Training-time scaling invests more compute into weight learning. This cost is paid before the model is deployed and is thereafter shared by all requests. Regardless of whether later problems are hard or easy, the capability improvements brought by weight updates take effect for every user, and the marginal cost per request remains almost unchanged. Training compute directly changes the model parameters themselves, yielding a more capable but already frozen model.

Inference-time scaling, by contrast, invests this compute on a per-request basis. The model parameters remain unchanged; the extra compute is used to change how the model uses its existing parameters and external tools: generating more candidate answers, performing search, calling tools, and verifying candidates. Its core advantage is that the budget can be dynamically routed according to difficulty—spending more budget on hard problems and less on simple problems. The costs are equally direct: latency, expense, and peak energy consumption for each request all rise accordingly. More importantly, test-time compute can only amplify the knowledge already present in the weights and context; it cannot create facts that are completely absent from them out of thin air.

When comparing the two together, the inputs include training budget, request difficulty, inference budget, latency cost, and knowledge boundary, and the output is the choice between training-time scaling and inference-time scaling. The causal chain is: training compute changes the weights and is shared by all requests; test-time compute adds candidates, search, tools, and verification on a per-request basis, which can be dynamically routed but directly increases service costs. The common boundary of both is that no matter how much compute is invested, there is no guarantee of creating facts that are completely absent from training and context.

The extra computation from inference-time scaling is not a homogeneous resource: extra tokens, extra candidates, search nodes, and tool calls provide different forms of benefit. Spending budget on the wrong form increases computation while accuracy stays unchanged.

Longer serial trajectories let the model gradually decompose, compute, check, and correct within a continuous reasoning process. It excels at tasks with strong dependencies—the next step depends on the intermediate result of the previous step, and one wrong step will propagate along the chain. At this point, adding parallel candidates does not help, because branches have already forked at the source; what is really needed is to make the single chain deeper.

Parallel candidates solve the coverage problem. When a problem has multiple relatively independent solution paths, a single chain may choose the wrong direction from the start; generating multiple candidates simultaneously and then selecting among them can significantly increase the probability of hitting the correct solution. It is suitable for tasks where dependencies between solutions are weak, and each candidate can advance independently.

Explicit search turns the reasoning process into a stateful process: save expanded nodes, expand candidates according to a policy, and prune using a scoring function. It lies between serial and parallel—preserving multiple paths while using scores to decide which paths are worth continued investment. It is suitable for tasks where the state space is branch-structured and requires systematic exploration rather than luck.

Tool execution obtains new evidence beyond the model weights by calling a code interpreter, retrieval system, math solver, or interactive environment. Its benefit differs from the previous three: the first three do more processing on the model's existing knowledge, while tool execution introduces external information to make up for facts missing from the model or context.

Which form to choose depends on the task's verifiability. Math and code often have strong checkers: whether a candidate answer is correct and whether tests pass provide clear, cheap feedback signals, and spending more computation can almost always improve in the right direction. Open-ended writing has only vague preference feedback, with no objective criterion to tell you which candidate is better; extra search can easily optimize the wrong objective—the more computation, the further you may get from the true intent.

Therefore, diagnose the bottleneck first, then choose the computation form: the inputs are the task's dependency depth, solution coverage, state space, external verifiability, and tool availability; the output is one of the four forms. Use serial for deep dependencies, parallel for diverse solutions, search for stateful branching, and tools when new evidence is missing. Misjudging the bottleneck is the most common source of waste in inference-time scaling.

The benefit of parallel candidates is not linear. With each additional candidate, the improvement in coverage is smaller than the previous candidate; the reason can be derived starting from the most ideal case.

Assume each candidate is fully independent, and the probability of success on a single attempt is p; then the probability that a single candidate fails is 1 − p. For all k candidates to fail, these k independent attempts must fail simultaneously, with probability (1−p)ᵏ. Taking the complement, the probability that at least one candidate succeeds is:

P_coverage = 1 − (1−p)ᵏ

The coverage gain from adding the kth candidate equals the probability that the first k−1 candidates all fail and the kth candidate succeeds:

P(k) − P(k−1) = (1−p)ᵏ⁻¹ × p

Since 0 ≤ 1−p < 1, this gain decays exponentially as k increases. The first few candidates cover most of the easily covered success cases, and subsequent candidates can only cover an increasingly small remaining failure space.

This formula is an ideal upper bound, and in reality it is almost never reached. Real multiple candidates share the same model and the same prompt, and their errors are highly correlated: if the model has a systematic weakness on a certain type of problem, all candidates often fail together in similar places, and the independence assumption breaks down. Second, coverage is only a necessary condition; the system must also identify which candidate is the successful one; if the verifier selects wrong, no matter how high the candidate pool coverage is, the accuracy cannot be realized. Therefore, increasing k does not automatically bring all the theoretical benefits, and actual benefit is usually below the upper bound.

The inputs of candidate coverage are the single-try success probability p, the number of candidates k, and candidate correlation; the output is the ideal probability P_coverage that at least one candidate succeeds. Under the independence assumption, P_coverage = 1 − (1−p)ᵏ, and the marginal coverage gain of additional candidates decreases as k increases; real candidates share the model and prompt, are usually correlated, and fall below this upper bound. This formula only estimates the candidate pool's own ability to cover the correct answer and does not include whether the verifier can choose correctly.

After candidate generation, search itself cannot tell you which one is correct; it is the verifier that determines where the search goes.

Verifiers can check only the final answer, or they can score intermediate steps. Outcome verification suits tasks with a well-defined terminal state—where whether an answer is correct can be judged by an objective terminal state, such as whether tests pass or whether an equation holds. Process verification, in contrast, gives feedback on each reasoning step and can prune an erroneous branch before it expands to the endpoint, saving downstream computation. Its cost is that process annotation is much more expensive than terminal-state annotation, and the scorer may mistake a particular problem-solving style for correctness: steps that are neatly written and follow common patterns in the training data do not necessarily mean the reasoning is actually correct.

The more independent a verifier's signal is, the more trustworthy it is. Compilers and formal provers are usually more independent than model self-evaluation—they execute mechanical rules rather than the same model judging its own output, so they are less likely to make the same mistakes as the generation process. But they still need to check coverage, version, and cross-domain drift: a compiler can only verify the languages and features it supports, and a prover only provides guarantees within its formal system; in uncovered domains, signal strength drops.

The verifier is also an attack surface. As long as scoring rules have loopholes, search will actively find high-scoring but incorrect outputs—what the scorer rewards, search will concentrate on producing, a phenomenon known as reward hacking. The larger the candidate space and the more ample the search budget, the more lethal a systematically exploited loophole becomes.

The verifier's inputs are candidate answers or processes, an explicit correctness contract, test rules, version, and domain; its outputs are pass, fail, score, or corrective feedback. Outcome verification checks the terminal state, process verification prunes early, and compilers and formal rules are usually more independent than same-source self-evaluation; but every verifier has coverage gaps. Search actively exploits scoring loopholes, so a high score does not equal true correctness.

Inference-time scaling is easily mistaken for "letting the model write more thinking text." Stateful exploration and verbose rewriting of the same error are two different things; distinguishing them depends on whether each expansion changes the search state.

Effective computation requires four things: state, actions, stopping conditions, and feedback. State records where the search has reached, actions are the legal next steps, stopping conditions determine when to stop investing further, and feedback tells the search whether these actions are good or bad. Under this framework, every expansion should produce a new candidate, fill an evidence slot, or change the verification state. If a token sequence merely repeats the same mistake at greater length—the same erroneous conclusion, the same blocked path—it brings no new information, is not new exploration, and only consumes budget through verbose rewriting.

Publicly displaying the reasoning trace to users also does not make the internal computation more reliable. There is no causal relationship between display length and computation quality: a long public text may only be low-information repetition, while truly valuable exploration may be reflected only in a few changes to internal state. A system can retain necessary intermediate state, use hidden drafts or structured tool calls to complete exploration, and at the same time provide users with a concise verifiable explanation—explanation is responsible for letting users verify conclusions, exploration is responsible for finding conclusions, and the lengths of the two need not match.

State search takes as input the current state, legal actions, new evidence, stopping conditions, and feedback, and outputs traceable state changes and candidates. Each expansion should fill in evidence, change verification state, or produce a different candidate; repeatedly generating verbose text with the same error does not count as new exploration. The concise verifiable explanation provided to users and the length of internal computation are inherently two different things.

Search does not know by itself when to stop. The question budget allocation must answer is: at what point does the expected benefit of continuing to search fall below the added latency and cost it brings? This requires a stopping policy, rather than unconditionally using up the budget.

The first step in allocation is estimating difficulty. First use a lightweight model to make a cheap difficulty or uncertainty judgment about the request, then decide on the investment tier: simple requests are answered directly, while difficult requests are escalated to a larger model or more candidates. This step reserves expensive computation for the requests that truly need it.

Stopping, in turn, depends on verification signals. As soon as strong verification evidence appears—tests pass, a proof is closed, a checker confirms—you can stop early instead of mechanically running through the preset maximum number of steps. Conversely, to guard against worst-case scenarios, you must also set hard upper limits: token count, number of tool calls, wall-clock time, and monetary caps, to prevent a single request from running out of control.

The quality of a routing policy should be evaluated using curves sliced by task: for each task type, how quality rises with cost and how latency rises with cost. A good routing policy optimizes the quality gain per unit cost, rather than having every request use the maximum budget—the latter only raises average quality while raising average cost to a meaningless level.

There is also an easily overlooked failure mode here: the router may lower average cost by incorrectly downgrading difficult requests, while the average metrics still look good. Therefore, record the difficult samples that were incorrectly downgraded and examine their performance separately, to prevent the router from learning only "spend less" rather than "spend money correctly".

The inputs to a budget policy are difficulty estimates, calibrated uncertainty, verification results, tokens, call counts, wall-clock time and monetary caps, and task value; the outputs are budget tier, early stopping, escalation, or reporting unknown. Simple requests are answered briefly; difficult and verifiable requests get more candidates; stop when strong evidence appears, while recording difficult examples that were incorrectly downgraded. The goal is quality gain per unit cost, not having every request use the maximum budget.

To connect the earlier stages: for test-time compute to be converted into quality gains, a complete causal chain must be traversed; if any one link breaks, the invested computation spins without effect.

The starting point of the chain is the candidate distribution produced by the base model. The distribution only indicates which answers are more likely, not that a correct answer is necessarily among them—the first step of coverage depends on whether the model has placed the correct solution within the range that can be sampled. Additional sampling or search expands the explored solution space: more candidates, more branches, more paths. But expanding the solution space is meaningful only when diversity holds; if newly generated candidates are highly similar to one another and repeatedly fall into the same kind of error, then the additional compute does not bring different attempts, only repeated sampling of the same failure region.

Next, the verification signal is responsible for distinguishing more reliable branches from worse ones. Without this layer, search does not know which direction to move; however many candidates there are, they just pile up. Finally, selection, correction, and early stopping convert the verification signal into real quality gains: choose the highest-scoring branch, correct the branch close to correct, and stop investing when strong evidence appears.

Diminishing returns come from three places: correlated errors among candidates destroy diversity, weak verifiers provide incorrect discriminative signals, and budget limits cut off paths that could have continued to improve. If any of the three takes effect, it lowers the marginal quality of additional compute.

The inputs to this causal chain are the base candidate distribution, exploration diversity, verification signal, selection and correction, and budget limit; its output is an explanation of "why additional compute did or did not produce quality gains". Additional compute first expands coverage, then is identified by the verifier, and finally is realized by the stopping strategy. Its purpose is diagnosis: when inference-time scaling does not bring the expected improvement, examine the chain layer by layer to find the broken link. It does not by itself constitute a new correctness guarantee.

A numerical example of code repair can make the preceding chain concrete. Assume the model's single-repair success rate is 35%, and the system generates 4 candidates for each problem. "At least one candidate is correct" and "the system ultimately selects the correct one" are two different events, corresponding to two different probabilities. The whole flow is shown in Figure 1: from difficulty routing into four-candidate generation, then through compilation and test verification, and early stopping when strong evidence appears.

The extra computation in Figure 1 realizes gains through "expanding the candidate set + independent verification": parallel sampling only increases the chance that a correct candidate appears; the verifier determines whether the system can find it. Step by step:

First, under the independence assumption, use 1 − (1−0.35)⁴ to get 82.1%; this is an upper bound. Real candidates share the same model, so errors are correlated, and measured coverage is only 68%—the four candidates often get stuck together on the same blind spot, so the independence formula clearly overestimates. Next, the verifier can only make a selection when the candidate pool actually contains a solution, and its selection accuracy is 85%. End-to-end success requires both events to occur simultaneously: a candidate appears and is selected, so it is 0.68 × 0.85 = 57.8%.

Going from 35% to 57.8% is a valuable improvement, but the cost is approximately 4 times candidate generation plus verification overhead. The gain is extremely sensitive to verifier quality: if selection accuracy is only 55%, end-to-end becomes 0.68 × 0.55 = 37.4%, and the gain is almost entirely eaten away; if candidates are highly homogeneous, increasing k will not approach the upper bound from the independence formula. The direction for improvement is therefore to optimize diversity and verification together, not merely to expand tokens.

The failure boundary is equally clear: a verifier can only check the contract it encodes. When unit tests miss concurrency isolation, search will more efficiently find patches that "pass the tests but still have vulnerabilities"—search is optimizing the test score, and the test score does not fully represent true correctness. The countermeasure is to retain hidden tests, protect acceptance and permission boundaries, and let the verification contract cover the properties that truly need to be guaranteed. The code case inputs are a single-repair success rate of 35%, k=4 candidates, measured coverage of 68%, and verifier selection accuracy of 85%; outputs are candidate coverage, selection rate, and end-to-end success of 57.8%. First use the independence formula to obtain an 82.1% upper bound, then substitute measured coverage, and finally compute 0.68 × 0.85; candidate appearance and final correct selection are two events.

If all requests are fixed to use the maximum number of candidates and the longest trajectories, then the vast majority of simple requests are paying for the marginal benefit of a very small number of difficult requests. The value of budget routing lies precisely in treating them differently according to signals:

Tasks with strong external verification and cheap candidates are ideal scenarios for scaling up k, but only if the verifier really covers the properties to be guaranteed; misjudgment in difficulty estimation can cause simple problems to be incorrectly downgraded; open-ended preference tasks, if only using model self-evaluation, easily fall into sycophancy and mode collapse, requiring clear rubric and human evaluation as a backstop; high-risk irreversible actions must first be simulated in a sandbox and obtain approval before execution, and majority voting from sampling does not constitute authorization; when the budget is exhausted, honestly reporting unknown is more honest than forcing an answer.

The learning path can be layered according to dependencies. Prerequisite concepts are sampling parameters, probability, reasoning models, and code execution. The core of this page is parallel candidates, sequential trajectories, search, verifiers, early stopping, and budget routing. Closely adjacent extension concepts include self-consistency, tree of thoughts, reflection, and post-training. Engineering-level extensions are model routing, evaluation, reward hacking, observability, and human-in-the-loop.

Post-launch evaluation should plot the quality–cost–p95 latency frontier and slice by difficulty, domain, and verifier type. To judge whether a task needs depth or coverage, compare one long trajectory with multiple short candidates under the same total tokens; looking only at overall success rate can mask the router incorrectly downgrading difficult domains or minority domains. The inputs to budget routing are task difficulty, verification strength, candidate cost, risk reversibility, SLO, and historical slice performance; outputs are k, trajectory length, model choice, sandbox approval, or stop and report unknown. Strong-verification low-cost tasks can scale up k, high-risk actions should only be simulated in a sandbox and authorized before execution, and open-ended preference tasks use clear rubric and human evaluation. After launch, report the quality-cost-p95 frontier and incorrect downgrading of difficult slices.