WorksComputer Science
LProp: Propagating Latent Uncertainty through Evidence-Conditioned Belief Revision in LLM Agents
Abstract
When LLM agents pass intermediate judgments downstream without uncertainty, later decisions may treat tentative assumptions as facts. Effective uncertainty propagation must preserve information for correction and update inherited beliefs as new evidence arrives. Yet scalar scores omit information needed for revision, while detailed descriptions and reflection incur additional costs. We introduce Latent Uncertainty, a latent-space estimator that preserves uncertainty-relevant information without lengthy descriptions. Building on this, we propose LProp (Latent Uncertainty Propagation), which revises inherited answer distributions using latent representations and new evidence without additional trajectory sampling. We evaluate both methods across multiple-choice, yes/no, open-ended question answering, and mathematical reasoning tasks. Our approach outperforms baselines on most metrics, while LProp achieves the lowest mean inference-token usage in all eight propagation settings. Matched feature ablations further demonstrate that inherited latent content improves error recovery beyond answer probabilities or model capacity alone.
Introduction
Nowadays, LLM agents are increasingly involved in people’s work and daily lives, such as serving as personal assistants [32, 27, 53]. However, this growing reliance on LLM agents makes overconfidence consequential, as acting on unchecked assumptions can cause financial loss or data damage rather than merely produce an incorrect response [35]. To address this risk, uncertainty can signal insufficient support for an agent’s judgment and guide it to seek further information or assistance before committing [18 Shi et al.].
Preserving detail is costly. Unlike single-response LLM tasks, agent tasks involve successive steps
in which earlier results inform later decisions, a transfer we refer to as a handoff [52, 45]. When a handoff preserves the result but omits its uncertainty, subsequent steps may treat tentative judgments as reliable premises, underestimating the uncertainty inherited from earlier decisions [13, 37] (Zhao et al. 2025). As a result, agents may proceed with unwarranted confidence and miss opportunities to verify assumptions inherited from earlier steps [54]. Therefore, uncertainty should be propagated throughout the agent’s trajectory. However, uncertainty propagation faces an information tax: semantic entropy and verbalized confidence summarize uncertainty as scores [24, 14, 43], but passing these scores alone does not convey the underlying alternatives needed for later revision (Figure 1a). To reduce this loss, an agent can instead transmit detailed, structured descriptions of unresolved assumptions, alternative interpretations, and supporting evidence. Yet retaining and extending these descriptions across steps creates a growing history that subsequent steps must process, increasing token cost and inference time [21]. The challenge is therefore to preserve uncertainty-relevant information without continually expanding the communication and processing burden.

Figure 1. Two gaps in uncertainty propagation. (a) Compact representations lose uncertainty details, while detailed descriptions increase input cost. (b) Risk estimation alone does not specify how to revise beliefs, while reflection requires additional generation, motivating efficient and timely uncertainty updates.
Revision can be costly and unreliable. Moreover, uncertainty propagation must determine not only how much uncertainty is inherited, but also how it should change with new evidence. Existing methods account for inherited uncertainty through decomposition or situation-aware weighting [13] (Zhao et al. 2025). However, these approaches primarily assess propagated uncertainty rather than actively revising inherited beliefs as new evidence arrives. To support such revision, AUQ retains confidence and explanations in memory and triggers corrective reflection before execution [54]. Yet accumulated textual memory may introduce additional uncertainty as stored information becomes stale or inconsistent, while memory-based reflection incurs additional context-processing and generation costs [41, 54]. The challenge is therefore to develop a reliable and efficient evidence-conditioned update that revises inherited beliefs as new evidence arrives, without relying on potentially stale memory or costly reflection (Figure 1b).
To address these challenges, we introduce Latent Uncertainty, a latent-space uncertainty estimator that preserves uncertainty-relevant information across steps without lengthy textual descriptions. Building on this, LProp learns an efficient, evidence-conditioned update that revises inherited uncertainty and adjusts its influence on subsequent decisions. We evaluate our approach on multiple-choice, yes/no, open-ended question-answering, and mathematical reasoning tasks. The results show consistently strong performance across task types, with improvements over the compared baselines on most evaluation metrics. Meanwhile, LProp also achieves the lowest mean inference-token usage in every evaluated propagation setting. Our contributions are as follows.
We introduce Latent Uncertainty, a latent-space uncertainty estimator, and validate its ability to predict answer errors, providing an empirical foundation for uncertainty propagation through latent representations.
Building on this foundation, we propose LProp, an efficient propagation method that preserves inherited latent information and updates beliefs as new evidence arrives. Experiments demonstrate substantial error recovery across diverse tasks, with the lowest mean inference-token usage in all eight propagation settings.
Through matched feature ablations, we demonstrate that inherited latent content improves downstream error recovery beyond answer probabilities or model capacity alone, establishing its contribution to evidence-conditioned belief revision.
Related Work
Uncertainty Carrier
The representation used to carry uncertainty determines what information remains available for downstream decisions. Existing methods make uncertainty explicit through probability-derived scores or verbalized confidence [25, 43]. Token likelihoods and predictive entropy provide probability-based estimates, while semantic entropy groups sampled answers by semantic equivalence to quantify uncertainty over meanings [25, 24, 14]. Verbalized confidence instead asks models to report their confidence in text [43, 38]. These outputs communicate the degree of uncertainty, but passing a score or brief confidence statement alone does not necessarily preserve the context needed to determine what to verify or revise.
Latent Information and Communication
A separate line of work explores communication through hidden representations to preserve information beyond explicit outputs. CIPHER [33] passes probability-weighted embeddings between agents, Interlat [12] transmits hidden states with learned compression, and LatentMAS (Zou et al., 2025) maintains shared latent working memory. Related work also explores extracting uncertainty-related signals from hidden representations. Azaria and Mitchell [2] train classifiers on hidden activations to predict statement truthfulness, while Semantic Entropy Probes [23] estimate semantic entropy from a single generation’s hidden states. These findings motivate our latent-space uncertainty estimator, while our approach further introduces an answer-indexed carrier to preserve candidate-specific information for downstream evidence-conditioned revision.
Uncertainty Propagation
In multistep decision-making, uncertainty in earlier results can affect later decisions, making local confidence alone insufficient. SAUP (Zhao et al., 2025) aggregates step uncertainties using situation-aware weights based on reasoning drift and observation alignment. To capture isolated failure signals that averaging can dilute, TRACER [42] combines local uncertainty with repetition and coherence indicators, emphasizing high-risk episodes. While these methods aggregate observed risk, UProp [13] samples trajectories and alternative decisions to distinguish intrinsic from inherited uncertainty. Risk estimation alone, however, does not specify how to revise beliefs with new evidence. AUQ [54] supports such revision by retaining confidence and explanations in memory and triggering reflection before execution. However, stored assessments may become stale or inconsistent, introducing further uncertainty, while memory-based reflection adds context-processing and generation costs. These limitations motivate us to develop an efficient, evidence-conditioned update that revises inherited beliefs in latent space without generating textual reflections.
Methodology
An uncertainty score can flag a need for verification, but revising an answer also requires information about competing hypotheses. Latent Uncertainty therefore combines a reader that estimates the risk of a fixed answer with a separate carrier that retains candidate-specific content and support. LProp uses subsequent evidence to revise this support through a shared answer-distribution interface. Figure 2 illustrates the two components. The backbone remains frozen throughout.

Figure 2. Overview of Latent Uncertainty and LProp. (a) The latent risk reader estimates answer-failure risk from hidden states and generation statistics, while an answer-indexed carrier retains candidate-specific content and support. (b) LProp updates latent content and candidate probability mass with new evidence, enabling answer revision and uncertainty estimation across subsequent handoffs.
Latent Uncertainty Carrier
Reading answer risk. Given a trajectory with hidden states and a fixed answer, , we construct step features from projected states and observable generation statistics. Projection and feature normalization are fitted on training data and frozen before evaluation. A learned map of produces a row-normalized transition matrix and a positive observation vector . Starting from a learned prior , a conditional filter integrates these observations and reads out answer-failure risk:
Here is a distribution over latent risk components, normalizes a nonnegative vector to sum to one, and denotes elementwise multiplication. The transition carries earlier observations forward, while the observation vector incorporates the current step. Fixed, ordered anchors map the component probabilities to a continuous risk estimate. The reader is trained with binary cross-entropy on labels indicating whether is incorrect, without changing the answer. This supervision concerns answer correctness and does not constrain the answer space. Additional reliability observations can refine the same estimate; feature construction and risk prediction are detailed in Appendix § 6.§ 1.
Retaining answer-specific information. Risk estimation alone does not retain the alternatives needed for revision. We therefore construct a carrier over a support available at the source:
where identifies a hypothesis, retains its projected source content of dimension , and records its relative support. Initial weights are derived from source answer scores, with a source readout adjusting them where enabled. The risk components describe the reliability of a fixed answer, while the carrier entries describe competing answer hypotheses; the two distributions serve distinct purposes.
Candidate construction uses neither future evidence nor gold answers. Candidate identities remain fixed during propagation, and an aggregate residual category can retain probability mass outside the explicitly represented answers. The receiver obtains the carrier, the question, and later public evidence without the source’s raw private context. The carrier’s numerical representation contains values, retaining candidate-specific information for revision over the source support. Construction and initialization details appear in Appendix § 6.§ 2.
LProp: Uncertainty Propagation through Latent Carrier
Evidence-conditioned support revision. At handoff , LProp revises a distribution over the retained support. Let denote the inherited distribution supplied to the update. Pooling parameterizations additionally use , the distribution obtained from public evidence over the same support. The inherited distribution and candidate content may be carried forward from the preceding revision or retained from the source. We express support revision in the common form
where controls reliance on public-evidence probabilities and is a candidate-specific evidence residual. The mixture determines how inherited and public support are combined, while the residual adjusts individual hypotheses using retained content and current evidence. Normalization covers the entire support, including any residual category.
This form admits both direct residual revision, with , and adaptive probability pooling. In the latter, a gate evaluates candidate content, inherited mass, distribution entropy, and evidence-change features; averaging its values over yields . These parameterizations share the carrier and revised-distribution interface, while differing in how they incorporate evidence. Propagation updates and task-specific parameterizations are summarized in Appendix § 6.§ 3.
Revising latent content and support. Candidate content can remain anchored to the source or evolve as evidence is incorporated. For a recurrent reference-difference update, let collect the candidate’s latent content, answer identity, and inherited support features. A shared function predicts content and log-mass changes under current evidence and reference evidence :
Holding the inherited state fixed in both evaluations measures the function’s change in response to evidence relative to the chosen reference. The reference may encode the question alone or the preceding evidence state. The resulting increments jointly revise content and support:
where scales the bounded content residual, bounds the updated coordinates, and is an optional correction derived from changes in the public model’s relative answer support. This mass update instantiates Equation (4) with , , and . Revised content and masses then enter the next recurrent state. Task-specific reference choices and update parameterizations are summarized in Appendix § 6.§ 3.
Learning and readout. Propagation is supervised at available handoff views using
where identifies the target entry in the retained support and is a Brier penalty. Views are grouped by question to prevent unequal view counts from dominating training, and model choices use development data. At inference, candidate probabilities are averaged across independently trained heads before selecting the admissible answer with the largest mass. For a valid selected answer, uncertainty is one minus its probability in the full distribution, a readout distinct from the reader’s fixed-answer risk estimate . Target construction, training, and inference details are given in Appendix § 7.§ 2.3. LProp encodes new public evidence and revises inherited support through the learned update, avoiding textual reflections and repeated trajectory sampling at each handoff.
Experiments
We evaluate two requirements separately. The first experiment tests whether the Latent Uncertainty Estimator identifies errors in fixed agent answers. The second tests whether LProp improves answer quality and uncertainty assessment after evidence handoffs, and examines the token cost required to obtain these improvements.
Latent Uncertainty Estimator
Experimental Setting
Benchmarks. We evaluate the Latent Uncertainty Estimator on HotpotQA [51] with Qwen3-8B [50] and Qwen2.5-7B (Qwen et al., 2025) and on StrategyQA [17] with Qwen3-8B, using the released 1,024-question evaluations with 512 training and 128 development questions per setting.
Compared methods. We compare against Token Entropy [15], Mean token NLL [15], Sequence NLL [25], Predictive Entropy [25], Length-normalized Predictive Entropy [25], Semantic Entropy [24], and Semantic Entropy Probe [23]. The probe predicts semantic entropy and uses this prediction as its uncertainty score; its remaining evaluation settings match those of Semantic Entropy. Detailed evaluation protocols and baseline configurations appear in Appendix § 7.§ 1.
Metrics. We report AUROC for error discrimination, AUARC for retained-answer accuracy across coverage levels, @30 for accuracy after rejecting the highest-risk 30% of answers, and Brier for the squared error of predicted failure probabilities. Higher values are better for the first three metrics, while lower Brier scores are better.
Results
Error discrimination. Figure 3 shows that the Latent Uncertainty Estimator achieves the best point estimates for AUROC, AUARC, @30, and calibrated Brier in every evaluated setting. Because all estimators score the same fixed answers, these gains reflect better uncertainty estimation rather than changes in answer quality. The AUROC improvements are accompanied by higher AUARC and @30, indicating that better error discrimination also translates into more reliable retained answers across coverage levels.
Figure 3. Latent uncertainty as an error-risk estimator. Reader metrics are averaged over five training seeds. Rows show settings; columns show metrics. Gray circles: baselines; blue diamonds: Ours. Axes are zoomed and matched within each metric. Values and calibration details appear in Table 3 and Appendix § 7.1.4.
Probability estimates. Lower calibrated Brier point estimates complement the ranking improvements by assessing the quality of predicted failure probabilities. The clearest gains occur on HotpotQA. Together, these results support latent representations as a useful basis for assessing answer reliability. Appendix § 7.1.5 provides the detailed comparisons.
LProp
Experimental Setting
Benchmarks. We evaluate LProp using Qwen3-8B and Mistral-7B-Instruct-v0.3 (Mistral AI, [28]) on BoolQ (Clark et al., [7]), StrategyQA, MMLU mathematics (Hendrycks et al., [19]), and HotpotQA, covering yes/no reasoning, mathematics, and open-ended multi-hop QA. We convert each QA instance into a controlled source–receiver episode. The source forms candidate support from private evidence; the receiver obtains the handoff and successive public observations without access to the raw private context. Data splits and protocol details appear in Appendices § 7.§ 2.1 and § 7.§ 2.2. A recorded recovery episode illustrating this setting and LProp’s updates appears in Appendix § 10.§ 1.
Compared methods. We compare with four representative uncertainty propagation methods: SAUP (Zhao et al., 2025), TRACER (Tayebati et al., [42]), AUQ (Zhang et al., [54]), and UProp (Duan et al., [13]). Detailed protocols and baseline configurations appear in Appendix § 7.§ 2.
Metrics. We report F1, AUROC, AUARC, @20, @30, and @50, with higher values indicating better performance. F1 measures token overlap with reference answers. AUROC measures error discrimination, while AUARC summarizes retained-answer accuracy across coverage levels. @20, @30, and @50 measure accuracy after rejecting the highest-risk 20%, 30%, and 50% of answers, respectively.
Main Results
Propagation performance. Table 1 shows that LProp achieves the highest AUROC in all eight evaluated settings, indicating stronger discrimination between correct and erroneous answers after evidence handoffs. This consistent advantage spans the evaluated task types and backbones. LProp also achieves the highest AUARC and all reported retained-correctness scores in every setting. The joint gains in AUARC and individual rejection-rate metrics show that its selective-reliability advantage extends beyond a single operating point. Moreover, LProp can recover from initially incorrect judgments by revising inherited beliefs as new evidence arrives, achieving the highest F1 in all eight settings. Appendix § 10 provides a case study illustrating this recovery. These results support the effectiveness of the complete propagation system; Appendix § 7.2.6 details gains by setting.
| Method | \multicolumn{6}{c}{Qwen3-8B} | \multicolumn{6}{c}{Mistral-7B-Instruct-v0.3} | ||||||||||
| F1↑ | AUROC↑ | AUARC↑ | @20↑ | @30↑ | @50↑ | F1↑ | AUROC↑ | AUARC↑ | @20↑ | @30↑ | @50↑ | |
| BoolQ | ||||||||||||
| SAUP | 0.7630 | 0.5902 | 0.8347 | 0.8000 | 0.8114 | 0.8340 | 0.6990 | 0.5794 | 0.7136 | 0.7225 | 0.7343 | 0.7540 |
| TRACER | 0.7570 | 0.5076 | 0.7927 | 0.7838 | 0.7886 | 0.7920 | 0.6790 | 0.4730 | 0.6575 | 0.6912 | 0.6857 | 0.6820 |
| AUQ | 0.7480 | 0.7038 | 0.8520 | 0.7989 | 0.8164 | 0.8489 | 0.6650 | 0.4191 | 0.5383 | 0.6687 | 0.6588 | 0.6272 |
| UProp | 0.7870 | 0.5581 | 0.7976 | 0.8162 | 0.8143 | 0.8080 | 0.7190 | 0.5757 | 0.7066 | 0.7349 | 0.7414 | 0.7560 |
| LProp | 0.8310 | 0.7961 | 0.9407 | 0.8950 | 0.9186 | 0.9460 | 0.8010 | 0.7656 | 0.9178 | 0.8600 | 0.8829 | 0.9200 |
| StrategyQA | ||||||||||||
| SAUP | 0.6417 | 0.5068 | 0.6528 | 0.6406 | 0.6399 | 0.6583 | 0.5875 | 0.5068 | 0.5497 | 0.5938 | 0.6101 | 0.5917 |
| TRACER | 0.6563 | 0.5396 | 0.6533 | 0.6510 | 0.6548 | 0.6875 | 0.5924 | 0.5344 | 0.5653 | 0.6146 | 0.6190 | 0.6083 |
| AUQ | 0.6750 | 0.6005 | 0.7209 | 0.7253 | 0.7195 | 0.7201 | 0.5667 | 0.3979 | 0.4300 | 0.5632 | 0.5476 | 0.5063 |
| UProp | 0.6438 | 0.4619 | 0.6037 | 0.6484 | 0.6369 | 0.6167 | 0.5896 | 0.5191 | 0.5487 | 0.6042 | 0.6071 | 0.6208 |
| LProp | 0.7250 | 0.6865 | 0.8372 | 0.7682 | 0.7946 | 0.8417 | 0.6500 | 0.6160 | 0.7415 | 0.6797 | 0.6905 | 0.7375 |
| MMLU | ||||||||||||
| SAUP | 0.8854 | 0.6481 | 0.9263 | 0.9127 | 0.9182 | 0.9236 | 0.4586 | 0.3696 | 0.2967 | 0.4643 | 0.4455 | 0.3885 |
| TRACER | 0.8885 | 0.5006 | 0.8946 | 0.8770 | 0.8909 | 0.8854 | 0.4554 | 0.3099 | 0.2749 | 0.4246 | 0.4045 | 0.3376 |
| AUQ | 0.8917 | 0.6669 | 0.8882 | 0.9339 | 0.9409 | 0.9326 | 0.4682 | 0.3651 | 0.3036 | 0.4718 | 0.4504 | 0.3826 |
| UProp | 0.8885 | 0.6425 | 0.9186 | 0.9206 | 0.9318 | 0.9299 | 0.4586 | 0.4496 | 0.3329 | 0.4960 | 0.4955 | 0.4459 |
| LProp | 0.8949 | 0.8102 | 0.9651 | 0.9484 | 0.9500 | 0.9809 | 0.5892 | 0.7248 | 0.7634 | 0.6468 | 0.6773 | 0.7516 |
| HotpotQA | ||||||||||||
| SAUP | 0.4573 | 0.6477 | 0.4810 | 0.3902 | 0.4067 | 0.4531 | 0.2532 | 0.4301 | 0.1372 | 0.2546 | 0.2551 | 0.2382 |
| TRACER | 0.4294 | 0.4837 | 0.3613 | 0.3561 | 0.3343 | 0.3281 | 0.2677 | 0.4773 | 0.1486 | 0.2664 | 0.2624 | 0.2626 |
| AUQ | 0.5127 | 0.5927 | 0.6177 | 0.5761 | 0.5760 | 0.6017 | 0.2808 | 0.6013 | 0.2235 | 0.3264 | 0.3337 | 0.3337 |
| UProp | 0.4319 | 0.6528 | 0.4791 | 0.3927 | 0.4067 | 0.4336 | 0.2903 | 0.5811 | 0.1832 | 0.2929 | 0.2996 | 0.3306 |
| LProp | 0.5253 | 0.7468 | 0.6638 | 0.5829 | 0.6156 | 0.6602 | 0.4024 | 0.8503 | 0.4539 | 0.4630 | 0.4933 | 0.5637 |
Table 1. Uncertainty propagation main results. LProp uses a three-head probability ensemble, with metrics computed after answer selection. All metrics prefer higher values. HotpotQA uses answer-token F1 and exact-match correctness for AUROC, AUARC and retained-answer metrics. The best values use boldface and darker shading; second-best distinct values use lighter shading. Approximate 95% bootstrap interval half-widths are reported in Appendix § 7.2.7, Table 5.
Cost Analysis
Figure 5 compares mean inference tokens per question, including generation inputs and outputs, backbone encoding, and auxiliary scoring. LProp uses the fewest tokens in all eight settings, with unweighted mean per-setting reductions of 61.2% relative to AUQ and 71.1% relative to UProp. This efficiency stems from reusing candidate representations and applying learned updates to new evidence, avoiding additional reflection or trajectory sampling at each handoff. These savings accompany improved uncertainty assessment across all eight settings. Appendix § 9 provides the accounting protocol and detailed comparisons.
Figure 5. Inference token cost in the LProp propagation experiment. Values report mean tokens per question for LProp and the compared baselines. The reported token costs are averaged over three runs.
Ablation Study
We examine the contribution of inherited information and receiver updates, sensitivity to model capacity, and reliability across rejection rates. Details appear in Appendix § 11, Appendix § 8.§ 1, and Appendix § 8.§ 2.
Feature ablations. We freeze Qwen3-8B and group the controls by handoff representation (probability-only and 128-token text summaries), latent content (frozen content and Gaussian noise), and receiver design (matched MLP and log-linear fusion). Figure 6 contrasts recovery of initially wrong answers with regression of initially correct answers. Full LProp achieves the highest final correctness on all four benchmarks and recovers more

Figure 6. Correctness gains and losses in matched receiver ablations. Bars show recovered errors (right) and introduced errors (left), each as a percentage of all questions; diamonds show net gains. Axes vary by benchmark.
errors than both alternative handoff representations. On StrategyQA, freezing content recovers slightly more errors but introduces substantially more regressions, highlighting the importance of selective revision. Appendix F gives the grouped protocol and counts.
Sensitivity to model capacity. Figure 4 shows that on BoolQ with Qwen3-8B, latent dimension 64 achieves the best ACC and AUARC, while hidden widths 64 and 256 achieve the best ACC and AUARC, respectively. Detailed ablations and configuration selection appear in Appendix C.1.
Figure 4. Sensitivity to model capacity on BoolQ with Qwen3-8B. Top: latent dimension; bottom: hidden width, with the other parameter fixed at 64. Horizontal axes use a log scale.
Rejection-rate sensitivity. Figure 7 compares LProp with AUQ across rejection rates on Qwen3-8B. LProp achieves higher retained-answer correctness at 35 of 36 operating points, supporting its advantage across rejection budgets. Appendix C.2 provides the protocol and complete values.
Figure 7. Rejection-rate sensitivity on Qwen3-8B. Lines show LProp minus AUQ in retained-answer correctness. Positive values favor LProp; the dashed line denotes equal performance.
Conclusion
We introduced Latent Uncertainty and validated its ability to predict answer errors, providing an empirical foundation for uncertainty propagation through latent representations. Building on this foundation, LProp preserves candidate-specific latent information and performs efficient, evidence-conditioned belief updates, enabling substantial recovery from inherited errors without additional trajectory sampling or textual reflection at each handoff. Matched feature ablations further demonstrate that inherited latent content improves downstream error recovery beyond answer probabilities or model capacity alone. Across diverse tasks, LProp achieves the highest performance and the lowest mean inference-token usage among the compared methods in all settings.
Limitations
Our approach requires access to backbone hidden states and the answer scores used to initialize candidate support. Freezing the backbone removes the need to update its parameters, but does not remove this access requirement. Models exposed only through text-generation or token-probability interfaces cannot directly supply the latent features required by the reader and carrier; compatibility therefore depends on internal-state access rather than model size alone.
The latent interface is also tied to the backbone’s representation space. Source and evidence projections, feature normalization, and learned heads are fitted using training data for the selected setting. Their parameters and serialized carriers are not inherently interchangeable across backbones, even when the models share an answer format. Changing the backbone or its hidden-state representation may therefore require refitting the associated components.
Our efficiency analysis reports inference-token usage under the specified accounting protocol rather than hardware-independent speedups. Public evidence still requires backbone encoding, while hidden-state extraction, carrier storage, and learned updates introduce computational and storage requirements that token counts do not fully characterize. Actual latency and device-memory usage depend on hardware, model size, batching, and serving implementations. Token savings should therefore not be interpreted as proportional reductions in runtime or memory consumption.
Ethical Considerations
This work aims to improve reliability assessment and evidence-based revision in LLM agents. It uses existing benchmarks and pretrained language models, whose reuse remains subject to applicable access and redistribution terms. Consequential applications should include appropriate validation and human oversight. Applications involving sensitive data should include safeguards for access, retention, and reuse. AI was used for writing refinement and code development under human guidance and supervision.
Implementation Details
Our approach is motivated by the hypothesis that latent representations retain uncertainty-relevant information beyond explicit model outputs. We first validate this premise by demonstrating that latent-state-informed uncertainty estimation can reliably predict errors in fixed answers. Building on this evidence, we develop LProp to preserve candidate-specific latent information across agent handoffs and revise inherited beliefs as new evidence arrives. Matched receiver ablations further establish that this latent information contributes to downstream error recovery beyond answer probabilities or model capacity alone. This section details the implementation; training and evaluation are described in Appendix § 7.
Latent Uncertainty Estimation
Feature construction. At each trajectory step, the reader combines a 64-dimensional PCA projection of the backbone hidden state with mean and maximum token negative log-likelihood, normalized output length, and an action-parsing indicator, yielding 68 features. For HotpotQA, two additional token statistics increase the dimension to 70.
Risk prediction. The conditional filter in Equation (1) uses a learned prior and a two-layer network with hidden width 32 and tanh activation. An additive bias in the transition logits encourages state persistence. We use four latent risk components with fixed anchors evenly spaced between 0.001 and 0.999. Their posterior probabilities produce a continuous failure-risk estimate through Equation (2).
Carrier Construction and Handoff
Candidate support. The carrier in Equation (3) retains admissible answers for BoolQ, StrategyQA, and MMLU mathematics. For HotpotQA, four source samples construct the candidate set; duplicate normalized answers are merged, and an OTHER category represents answers outside the retained candidates. Candidate construction uses neither future evidence nor gold answers. Initial probability masses are derived from source answer logits using temperature scaling, with an optional learned source correction.
Latent representation. For binary and multiple-choice tasks, candidate-specific representations are extracted by appending each admissible answer token to the source prompt and reading its final hidden state. Source and public-evidence representations use separate PCA projections. Projected features are standardized and clipped to , with 64 dimensions retained per candidate.
The carrier stores candidate identities, probability masses, and candidate-specific latent content, requiring numerical values for candidates of dimension . At each handoff, the receiver obtains the carrier, question, and subsequent public evidence without accessing the source’s raw private context. Unlike scalar or probability-only handoffs, this representation preserves information about how the source supports competing hypotheses.
Answer support and decoding. Candidate identities remain fixed during propagation. On HotpotQA, OTHER participates in probability normalization but is excluded from concrete-answer selection. The selected answer is the admissible candidate with the highest probability, and its uncertainty is one minus that probability in the full distribution, without renormalizing after excluding OTHER. Recovery is therefore restricted to correct answers already present in the source support.
Evidence-Conditioned Propagation
LProp applies a common evidence-conditioned update to inherited candidate support. As formulated in Equation (4), the update combines an inherited distribution with public-evidence support and applies candidate-specific residuals:
Here controls reliance on public evidence, while adjusts individual hypotheses using inherited latent content and current evidence. The resulting distribution is normalized over the complete retained support.
Task-specific adaptation. The same update framework accommodates different answer formats and evidence schedules. For binary QA, candidate support is revised through learned evidence-dependent residuals, with . The residuals can incorporate changes in public answer log odds, while latent content is updated through the bounded transition in Equation (6). Evidence differences are evaluated relative to either the question-only state or the preceding evidence state, as defined in Equation (5).
For multi-candidate tasks, a learned gate determines from candidate content, inherited probabilities, distribution entropy, and evidence-change features. Candidate-specific residuals further adjust the resulting support. Depending on the task, updates either carry forward revised content and probabilities or retain the source state while incorporating cumulative public evidence. These adaptations share the same carrier, candidate support, and probability-update interface; they differ only in how evidence is incorporated into the common revision mechanism.
Training and Evaluation Details
Latent Uncertainty Estimation
Evaluation Protocol
Agent execution. Each question is evaluated using a fixed greedy ReAct trajectory containing tool observations and the final answer, with at most six interaction steps. All estimators assess the same fixed answers, while auxiliary sampled trajectories provide additional uncertainty observations without altering the evaluated answers.
Data splits. Table 2 reports the training, development, and evaluation sizes for each setting. Training data are used to fit the reader, while development data support model selection and calibration.
| Dataset | Backbone | Train | Dev. | Eval. |
| HotpotQA | Qwen3-8B | 512 | 128 | 1,024 |
| HotpotQA | Qwen2.5-7B | 512 | 128 | 1,024 |
| StrategyQA | Qwen3-8B | 512 | 128 | 1,024 |
Table 2. Question counts for the Latent Uncertainty Estimator. Each row represents one dataset–backbone setting.
Reader Training
The reader is trained with binary cross-entropy against answer-error labels while keeping evaluated answers fixed. This supervision directly assesses whether latent-state-informed features can predict answer failures independently of answer format. PCA and feature normalization are fitted on training data and frozen during evaluation. Backbone weights remain frozen. The reader is trained with five seeds: 17, 29, 43, 71, and 101. Each trained reader is evaluated independently, and the reported metrics are averaged across the five runs.
Baseline Configurations
Token Entropy. Token Entropy averages the entropy of the next-token probability distribution across generation positions, measuring how diffuse the model’s token predictions are [15].
Mean token NLL. Mean token NLL averages the negative log-probabilities assigned to the generated tokens, providing a length-normalized measure of how unlikely the observed generation is [15].
Sequence NLL. Sequence NLL sums the negative log-probabilities of the generated tokens to score the entire sequence without length normalization [25].
Predictive Entropy. Predictive Entropy estimates entropy over output sequences by averaging the negative log-probabilities of sampled generations [25, 24]. Unlike Sequence NLL, it assesses uncertainty across sampled outputs rather than scoring a single generation.
Length-normalized Predictive Entropy. This variant divides each sampled sequence’s negative log-probability by its token length before averaging, reducing the influence of generation length on the uncertainty estimate [25].
Semantic Entropy. Semantic Entropy groups sampled answers by semantic equivalence and estimates entropy over the resulting meaning-level distribution [24]. It distinguishes disagreement in answer meaning from variation in wording.
The three sampling-based baselines each use ten sampled trajectories.
Semantic Entropy Probe. Semantic Entropy Probe [23] predicts the Semantic Entropy score and uses the predicted value as its uncertainty score. It is denoted as Semantic Probe in the figure and table. The prediction target is semantic entropy, whereas our latent reader is trained on answer-error labels. AUROC, AUARC, and @30 use the probe score to rank uncertainty; Brier uses the failure probability obtained from the development-fitted sigmoid calibration in Appendix § 7.1.4. The remaining evaluation settings, including the fixed evaluated answers and calibration protocol, are the same as for Semantic Entropy.
Probability Calibration
Figure 3 and Table 3 report development-calibrated Brier scores. All methods use the same two-parameter sigmoid calibration with a positive slope. Inputs are standardized using development data, with native risk scores for the baselines and failure-probability logits for the latent reader.
| Method | HotpotQA / Qwen3-8B | HotpotQA / Qwen2.5-7B | StrategyQA / Qwen3-8B | |||||||||
| AUROC ↑ | AUARC ↑ | @30 ↑ | Brier ↓ | AUROC ↑ | AUARC ↑ | @30 ↑ | Brier ↓ | AUROC ↑ | AUARC ↑ | @30 ↑ | Brier ↓ | |
| Semantic Ent. | .7534 | .7994 | 80.56 | .1677 | .8296 | .7335 | 70.43 | .1579 | .6923 | .7528 | 74.48 | .1946 |
| Semantic Probe | .6903 | .7781 | 75.12 | .1824 | .7884 | .7012 | 68.53 | .1722 | .6703 | .7392 | 72.18 | .2136 |
| Token Ent. | .7128 | .7937 | 76.15 | .1812 | .7984 | .7118 | 68.76 | .1703 | .6599 | .7456 | 72.25 | .2018 |
| Predictive Ent. | .7005 | .7758 | 76.43 | .1866 | .7901 | .7054 | 67.92 | .1718 | .6935 | .7673 | 73.92 | .1972 |
| Len.-norm. PE | .7199 | .7902 | 76.43 | .1822 | .7943 | .7077 | 68.06 | .1689 | .6789 | .7584 | 72.38 | .2008 |
| Mean token NLL | .7039 | .7890 | 76.15 | .1830 | .7912 | .7093 | 68.48 | .1714 | .6532 | .7420 | 71.41 | .2030 |
| Sequence NLL | .6890 | .7760 | 75.31 | .1871 | .7963 | .7128 | 68.90 | .1704 | .6664 | .7523 | 72.25 | .2005 |
| Ours | .8079 | .8317 | 83.38 | .1533 | .9032 | .7911 | 73.44 | .1172 | .7189 | .7837 | 74.67 | .1942 |
Table 3. Complete uncertainty estimation results. Bold values indicate the best result within each setting and metric. @30 is a percentage; other metrics use their original scales. Brier uses development-calibrated failure probabilities.
Calibration uses 118 development examples for HotpotQA/Qwen3, 103 for HotpotQA/Qwen2.5, and 123 for StrategyQA/Qwen3. The calibration parameters are fitted on development data and frozen before evaluation. Each reader seed is calibrated independently, and its Brier score is included in the five-run average.
The calibration objective is mean binary log loss with a slope penalty of , where is the sigmoid slope and the intercept is unpenalized. Both parameters are optimized using L-BFGS-B. The positive slope preserves the original risk rankings and ties, leaving AUROC, AUARC, and @30 unchanged while enabling Brier evaluation on a common probability scale.
Evaluation Metrics and Results
Table 3 contains all values behind Figure 3, using the 1,024-question evaluation cohorts in Table 2. Calibration follows Appendix § 7.1.4.
Reading the fixed-answer comparison. The estimators share the same evaluated answers, which holds answer correctness fixed when comparing uncertainty estimates. AUROC measures the separation of erroneous and correct answers by estimated risk; AUARC summarizes retained-answer accuracy across coverage levels. The two metrics provide complementary views of ranking quality: AUROC compares error–correct answer pairs, while AUARC evaluates the subsets selected at successive coverage levels. A change in the ordering can consequently affect AUROC and AUARC differently, because they summarize different aspects of the same selection problem. The @30 result identifies a particular operating point, retaining the lowest-risk 70% of answers. Calibrated Brier scores evaluate the quality of the failure probabilities obtained through development-set calibration. Together, these metrics characterize error discrimination, selective correctness, and probability prediction.
Figure 3 shows that the Latent Uncertainty Estimator achieves the best point estimates across all four metrics in all three settings. On HotpotQA, AUROC reaches 0.8079 with Qwen3 and 0.9032 with Qwen2.5, compared with 0.7534 and 0.8296 for Semantic Entropy. On StrategyQA, it reaches 0.7189, compared with 0.6935 for Predictive Entropy.
The improved rankings also retain more correct answers at a fixed rejection budget. HotpotQA @30 increases from 80.56% to 83.38% on Qwen3 and from 70.43% to 73.44% on Qwen2.5 relative to Semantic Entropy. AUARC improves from 0.7994 to 0.8317 and from 0.7335 to 0.7911, respectively, showing that the benefit extends across coverage levels. On StrategyQA, AUARC rises from the strongest baseline’s 0.7673 to 0.7837, while @30 improves from 74.48% to 74.67%.
Calibrated Brier scores complement these ranking results. On HotpotQA, Brier decreases from 0.1677 to 0.1533 with Qwen3 and from 0.1579 to 0.1172 with Qwen2.5. Table 3 reports the complete Brier results alongside the discrimination and retention metrics.
Uncertainty Propagation
Data Splits and Evaluation Cohorts
Table 4 reports the training, development, and evaluation question counts for each dataset–backbone setting. All compared methods use the same evaluation questions within each setting.
| Dataset | Backbone | Train | Dev. | Eval. |
| BoolQ | Qwen3-8B | 4,096 | 512 | 1,000 |
| Mistral-7B | 4,096 | 512 | 1,000 | |
| StrategyQA | Qwen3-8B | 512 | 128 | 480 |
| Mistral-7B | 512 | 128 | 480 | |
| MMLU math | Qwen3-8B | 512 | 128 | 314 |
| Mistral-7B | 512 | 128 | 314 | |
| HotpotQA | Qwen3-8B | 512 | 128 | 512 |
| Mistral-7B | 512 | 128 | 512 |
Table 4. Question counts for the LProp main experiments. Mistral-7B denotes Mistral-7B-Instruct-v0.3. For both backbones on MMLU math, training and development use MMLU-Pro math.
Training and development. Both backbones use the same split settings within each dataset. BoolQ training and development questions come from the official training split. Questions or passages overlapping official validation are excluded, and duplicates are grouped before deterministic hash-based splitting. StrategyQA uses 512 training and 128 development questions. The mathematics heads are trained and selected on MMLU-Pro math ([44]) before evaluation on MMLU mathematics. HotpotQA training and development sets are disjoint, question-type-stratified subsets of the official training split, with normalized duplicate questions and overlap with official validation excluded. Model fitting and feature transformations use training data, while model selection uses development data.
Evaluation cohorts. BoolQ uses 1,000 questions from official validation per backbone. StrategyQA uses the 480 questions with subsequent public evidence from a collected set of 512. Both backbones share the same 314-question MMLU mathematics cohort, comprising 191 official test questions and 123 official development/validation questions across abstract algebra, college mathematics, elementary mathematics, high-school mathematics, and high-school statistics. This cohort is deduplicated against the previously used MMLU-Pro math training, development, and evaluation questions. The 123 official development/validation questions are used for evaluation, not head selection, which uses the separate MMLU-Pro development set. HotpotQA uses 512 question-type-stratified questions from official distractor validation, shared across backbones.
QA-to-Multi-Step Agent Construction
Episode structure and information boundary. Each QA instance is organized into an ordered source–receiver episode while retaining its original question and answer criterion. The source processes the question and private evidence or a reasoning draft to construct candidate support. It then transmits a carrier containing candidate identities, probability masses, and candidate-specific latent content. The receiver processes subsequent public evidence without accessing the source’s raw private text, future evidence, or reference answer. Source and receiver are roles separated by this information boundary within the same backbone setting.
Each subsequent public evidence event defines an update stage. The frozen backbone encodes the available public context, and LProp uses these evidence features and inherited latent information to revise support over the fixed candidate set.
BoolQ: controlled passage release. The original passage is divided into three contiguous blocks at approximately one third and two thirds of its sentence sequence. Passages with fewer than three sentences are divided by words instead. The first block is private to the source, while the second and third blocks are revealed sequentially. The first receiver update uses the question and block 2; the second uses the question and blocks 2 and 3. Block 1 is represented only through the carrier. Splitting preserves the original order and uses neither answer labels nor sentence relevance. Candidate support remains throughout.
StrategyQA: replayed retrieval observations. A Qwen3-8B collector generates greedy ReAct trajectories using Search and Lookup over a fixed Wikipedia corpus. Search attempts exact-title matching followed by BM25 retrieval, while Lookup returns the next matching sentence from the first passage in the latest search result. Retrieval prompts exclude gold facts and decompositions.
The first parsed, nonterminal retrieval observation becomes private source evidence, and the source is encoded after receiving it. Later observations become public events in their original order; the collector’s final answer is not included as evidence. Both backbones receive the same frozen observation sequence and encode it separately. Public contexts accumulate the observations revealed so far while excluding the private observation. Repeated observations are deduplicated, and questions without new content after deduplication remain in the evaluation cohort. MMLU mathematics: draft, derivation, and check. Each question retains its original answer options and follows three information stages: a private generated draft, a separately generated public derivation, and a public check of that derivation. The source processes the private draft and constructs an option-indexed carrier. The public derivation and check then arrive as successive evidence events, and the receiver revises support over the same options after each arrival. The reasoning segments are generated greedily and reused across compared methods. Derivations and checks are treated as fallible evidence, while reference answers are reserved for supervision and scoring.
HotpotQA: staged passage release. A question-conditioned BM25 retriever ranks the distractor passages without using answer or supporting-fact labels. For passages, the top form the source’s private context. The remaining passages are divided into two disjoint, nearly equal public groups, with the first group larger when the remainder is odd. These groups are revealed sequentially; empty groups cause no update. A ten-passage question therefore has five private passages followed by public groups of three and two passages.
The source generates four answers from its private context. Normalized duplicates are merged into retained candidates, and an OTHER category represents answers outside this support. At each public arrival, LProp updates candidate support using the available evidence. Private passages are not forwarded to the receiver, and the final answer is selected from the retained concrete candidates.
Execution and readout. The carrier is serialized and deserialized before receiver-side processing. We evaluate the final receiver state after all available public evidence has been processed. Answer selection follows Appendix § 6.§ 2.
LProp Training
Supervision. LProp is trained on available handoff views using the objective in Equation (7), which combines negative log-likelihood of the target candidate with a Brier penalty. For binary QA, the Brier term measures squared error on the yes probability; for categorical tasks, it measures squared distance to the one-hot target. HotpotQA uses the matching source candidate or OTHER as its target. Minibatches are organized by question to prevent unequal numbers of evidence views from dominating training. Where enabled, an unrelated-evidence consistency penalty discourages unjustified changes to inherited support.
Optimization and inference. The backbone remains frozen, and propagation heads are optimized with AdamW using weight decay 0.01 and gradient-norm clipping at 1.0. Training consists of initial fitting followed by lower-learning-rate refinement. Source and public-evidence PCA projections, feature normalization, and the source probability temperature are fitted on training data and frozen before evaluation. Development data are used for model selection. At inference, candidate probabilities from three independently trained heads are averaged before answer selection. Public evidence is encoded by the frozen backbone, and the learned update revises inherited support without additional trajectory sampling or textual reflection at each handoff.
Baseline Configurations
We implement each baseline according to its original formulation and released procedures, applying training only where required by the method.
SAUP. SAUP ((Zhao et al., 2025)) aggregates step-level uncertainty using situation-aware weights that account for the agent’s reasoning and its alignment with observations. We use its direct-distance weighting configuration.
TRACER. TRACER ([42]) combines token-level surprisal with repetition and coherence indicators, using tail-focused aggregation to emphasize critical high-risk episodes. We use its QA encoder adaptation.
UProp. UProp ([13]) decomposes decision uncertainty into intrinsic uncertainty and uncertainty inherited from preceding decisions. Its distribution-level estimator uses sampled trajectories and alternative decisions to estimate the inherited component.
AUQ. AUQ ([54]) retains verbalized confidence and explanations in memory and uses uncertainty-triggered reflection to revise tentative decisions. We preserve its original memory and reflection procedure.
Shared protocol and uncertainty-guided reflection. All methods are evaluated on the same questions and evidence schedule, with required auxiliary inference included in token accounting. SAUP, TRACER, and UProp provide post-hoc uncertainty-guided reflection. estimates rather than an answer-revision mechanism. We therefore append one final reflection step to each method: the agent receives its trajectory-level uncertainty estimate and reflects once before producing the final answer used for evaluation.
Metrics and Answer Handling
F1 is the mean answer-token F1 across questions. For canonical yes/no and option answers, it equals correctness. On HotpotQA, token F1 awards partial credit for overlap with the reference answer, while AUROC, AUARC, and retained-answer correctness use exact match. Erroneous answers are the positive class for AUROC. AUARC averages retained-answer correctness across coverage levels; @20, @30, and @50 report correctness after rejecting the highest-risk 20%, 30%, and 50% of answers, respectively.
All questions, including those without a matching source candidate on HotpotQA, remain in the evaluation denominator.
Detailed Result Comparisons
Table 1 shows that LProp achieves the highest AUROC, AUARC, and all three retained-correctness metrics in every dataset–backbone setting. It also achieves the highest F1 in all eight settings. Relative to the strongest baseline in each setting, AUROC gains range from 0.0816 to 0.2752 and AUARC gains from 0.0388 to 0.4305. These results indicate improvements in both answer quality and selective reliability across the evaluated settings.
On BoolQ, LProp raises F1 from the strongest baseline’s 78.70% to 83.10% with Qwen3 and from 71.90% to 80.10% with Mistral. The corresponding @30 values reach 91.86% and 88.29%, exceeding the strongest baselines by 10.22 and 14.14 percentage points. On MMLU mathematics with Mistral, F1 increases from 46.82% to 58.92%, while AUARC rises from 0.3329 to 0.7634. With Qwen3, @50 reaches 98.09%, compared with 93.26% for the strongest baseline.
On HotpotQA/Mistral, LProp improves F1 over AUQ from 28.08% to 40.24% and AUARC from 0.2235 to 0.4539. On HotpotQA/Qwen3, LProp achieves the highest AUROC, AUARC, and reported selective-correctness scores; for example, @30 is 61.56%, compared with 57.60% for AUQ. These results extend the selective-reliability gains to open-ended multi-hop QA.
Backbone access and model scale. LProp requires white-box access to backbone hidden states and candidate answer scores. We therefore use locally deployed Qwen3-8B and Mistral-7B-Instruct-v0.3, whose sizes fit our available computational resources. This choice reflects deployment constraints rather than an inherent model-size requirement of LProp. The consistent gains across these backbones show that useful latent uncertainty can support evidence-conditioned revision even at this scale. These findings motivate the expectation that larger backbones can also benefit from the same mechanism, provided that their internal states are accessible and the projections and propagation heads are fitted to their representations.
Bootstrap Intervals
Table 5 reports half-widths , where is the bootstrap standard error. For the corresponding main-table estimate , the approximate 95% interval is . These symmetric normal-approximation intervals are not constrained to .
| Qwen3-8B | Mistral-7B-Instruct-v0.3 | |||||||||||
| Method | F1 | AUROC | AUARC | @20 | @30 | @50 | F1 | AUROC | AUARC | @20 | @30 | @50 |
| BoolQ | ||||||||||||
| SAUP | ±0.0251 | ±0.0428 | ±0.0303 | ±0.0272 | ±0.0287 | ±0.0325 | ±0.0217 | ±0.0377 | ±0.0333 | ±0.0317 | ±0.0337 | ±0.0387 |
| TRACER | ±0.0234 | ±0.0432 | ±0.0346 | ±0.0283 | ±0.0301 | ±0.0349 | ±0.0287 | ±0.0381 | ±0.0378 | ±0.0319 | ±0.0344 | ±0.0405 |
| AUQ | ±0.0265 | ±0.0318 | ±0.0214 | ±0.0272 | ±0.0272 | ±0.0254 | ±0.0293 | ±0.0294 | ±0.0299 | ±0.0304 | ±0.0306 | ±0.0306 |
| UProp | ±0.0250 | ±0.0462 | ±0.0363 | ±0.0266 | ±0.0287 | ±0.0341 | ±0.0235 | ±0.0376 | ±0.0344 | ±0.0316 | ±0.0321 | ±0.0368 |
| LProp | ±0.0234 | ±0.0340 | ±0.0133 | ±0.0218 | ±0.0215 | ±0.0211 | ±0.0241 | ±0.0341 | ±0.0169 | ±0.0242 | ±0.0248 | ±0.0239 |
| StrategyQA | ||||||||||||
| SAUP | ±0.0416 | ±0.0548 | ±0.0590 | ±0.0485 | ±0.0515 | ±0.0601 | ±0.0236 | ±0.0533 | ±0.0590 | ±0.0484 | ±0.0530 | ±0.0626 |
| TRACER | ±0.0311 | ±0.0543 | ±0.0585 | ±0.0474 | ±0.0516 | ±0.0571 | ±0.0419 | ±0.0526 | ±0.0579 | ±0.0493 | ±0.0520 | ±0.0618 |
| AUQ | ±0.0426 | ±0.0528 | ±0.0515 | ±0.0449 | ±0.0483 | ±0.0557 | ±0.0331 | ±0.0471 | ±0.0505 | ±0.0458 | ±0.0489 | ±0.0562 |
| UProp | ±0.0237 | ±0.0568 | ±0.0634 | ±0.0476 | ±0.0520 | ±0.0637 | ±0.0436 | ±0.0531 | ±0.0610 | ±0.0498 | ±0.0517 | ±0.0603 |
| LProp | ±0.0397 | ±0.0507 | ±0.0391 | ±0.0432 | ±0.0442 | ±0.0483 | ±0.0426 | ±0.0506 | ±0.0497 | ±0.0464 | ±0.0496 | ±0.0573 |
| MMLU | ||||||||||||
| SAUP | ±0.0314 | ±0.0555 | ±0.0371 | ±0.0357 | ±0.0362 | ±0.0419 | ±0.0565 | ±0.0644 | ±0.0748 | ±0.0630 | ±0.0664 | ±0.0798 |
| TRACER | ±0.0322 | ±0.0478 | ±0.0463 | ±0.0417 | ±0.0416 | ±0.0503 | ±0.0539 | ±0.0647 | ±0.0790 | ±0.0632 | ±0.0675 | ±0.0796 |
| AUQ | ±0.0350 | ±0.0403 | ±0.0261 | ±0.0264 | ±0.0261 | ±0.0274 | ±0.0548 | ±0.0426 | ±0.0612 | ±0.0632 | ±0.0652 | ±0.0639 |
| UProp | ±0.0354 | ±0.0636 | ±0.0411 | ±0.0339 | ±0.0365 | ±0.0417 | ±0.0511 | ±0.0524 | ±0.0722 | ±0.0647 | ±0.0670 | ±0.0830 |
| LProp | ±0.0346 | ±0.0317 | ±0.0209 | ±0.0279 | ±0.0293 | ±0.0283 | ±0.0550 | ±0.0550 | ±0.0560 | ±0.0609 | ±0.0632 | ±0.0751 |
| HotpotQA | ||||||||||||
| SAUP | ±0.0310 | ±0.0494 | ±0.0582 | ±0.0449 | ±0.0494 | ±0.0583 | ±0.0217 | ±0.0215 | ±0.0251 | ±0.0209 | ±0.0209 | ±0.0259 |
| TRACER | ±0.0315 | ±0.0531 | ±0.0592 | ±0.0454 | ±0.0483 | ±0.0556 | ±0.0275 | ±0.0267 | ±0.0297 | ±0.0232 | ±0.0250 | ±0.0303 |
| AUQ | ±0.0360 | ±0.0443 | ±0.0526 | ±0.0376 | ±0.0394 | ±0.0391 | ±0.0295 | ±0.0334 | ±0.0342 | ±0.0346 | ±0.0347 | ±0.0347 |
| UProp | ±0.0396 | ±0.0491 | ±0.0594 | ±0.0459 | ±0.0486 | ±0.0582 | ±0.0238 | ±0.0303 | ±0.0272 | ±0.0251 | ±0.0279 | ±0.0336 |
| LProp | ±0.0378 | ±0.0406 | ±0.0513 | ±0.0423 | ±0.0431 | ±0.0450 | ±0.0345 | ±0.0352 | ±0.0612 | ±0.0425 | ±0.0450 | ±0.0563 |
Table 5. Approximate 95% interval half-widths for Table 1. All half-widths use the same 0–1 scale as the corresponding main-table estimates.
Ablation Studies
BoolQ Parameter Ablations
Sweep Settings
We evaluate the sensitivity of LProp to latent dimension and hidden width on BoolQ using Qwen3-8B, with 4,096 training, 512 development, and 1,000 evaluation questions. Both sweeps consider values of 32, 64, 128, 256, and 1,024. The latent-dimension sweep fixes hidden width at 64, while the hidden-width sweep fixes latent dimension at 64.
Training configurations remain fixed across each sweep, with projections and normalization fitted on training data only. Predictions are ensembled over three training seeds (17, 29, and 43). We report accuracy (ACC) and AUARC, where ACC equals answer F1 on BoolQ.
Detailed Results
Table 6 presents the complete results.
| Latent dimension | Hidden width | ||||
| Value | ACC (%) | AUARC | ACC (%) | AUARC | |
| 32 | 83.20 | 0.9356 | 82.60 | 0.9353 | |
| 64 | 84.70 | 0.9369 | 84.70 | 0.9369 | |
| 128 | 82.50 | 0.9318 | 82.60 | 0.9382 | |
| 256 | 82.40 | 0.9337 | 83.10 | 0.9407 | |
| 1024 | 79.50 | 0.9126 | 83.00 | 0.9360 |
Table 6. BoolQ parameter ablations on Qwen3-8B. The latent-dimension sweep fixes hidden width at 64, while the hidden-width sweep fixes latent dimension at 64. Higher values are better for both metrics. Bold indicates the best result within each sweep and metric.
Latent dimension. A dimension of 64 achieves the highest ACC (84.70%) and AUARC (0.9369). Compared with dimension 1,024, it improves ACC by 5.20 percentage points and AUARC by 0.0243, indicating that compact latent representations perform better under the evaluated configuration.
Hidden width. Width 64 achieves the highest ACC (84.70%), while width 256 achieves the highest AUARC (0.9407), with ACC of 83.10%. These results show that full-coverage accuracy and selective reliability favor different network capacities.
Model Selection
Configurations are selected using development-set ranks across AUROC, ACC, AUARC, @20, @30, and @50. The selected configuration uses latent dimension 64 and hidden width 256. This configuration is also used in the main BoolQ propagation experiment, evaluated on the same 1,000-question cohort. Its ACC of 83.10% and AUARC of 0.9407 therefore appear in both the parameter ablation and main results.
Sensitivity to the Rejection Rate
We compare LProp and AUQ on Qwen3-8B across BoolQ, StrategyQA, MMLU mathematics, and HotpotQA. The sweep covers rejection rates from 10% to 90% in increments of 10 percentage points, yielding 36 paired operating points. We reuse the main experiment’s frozen predictions and risk scores without retraining, regenerating answers, or recalibrating scores at individual rates.
At @k, each method retains the lowest-risk answers, using expected correctness to resolve boundary ties. Each method is evaluated on its own final answers and risk scores. Correctness follows the main protocol; HotpotQA uses exact match rather than answer-token F1. Table 7 reports all values, including @20, @30, and @50 from the main table. Rates within a benchmark reuse the same predictions and are not independent replications.
| Benchmark | Method | @10 | @20 | @30 | @40 | @50 | @60 | @70 | @80 | @90 |
| BoolQ | AUQ | 77.70 | 79.89 | 81.64 | 82.99 | 84.89 | 87.73 | 92.46 | 94.71 | 94.29 |
| LProp | 86.78 | 89.50 | 91.86 | 93.50 | 94.60 | 96.25 | 98.33 | 99.00 | 100.00 | |
| StrategyQA | AUQ | 70.52 | 72.53 | 71.95 | 72.72 | 72.01 | 72.53 | 73.64 | 75.87 | 76.06 |
| LProp | 73.84 | 76.82 | 79.46 | 80.56 | 84.17 | 86.46 | 88.89 | 92.71 | 91.67 | |
| MMLU math | AUQ | 92.11 | 93.39 | 94.09 | 93.67 | 93.26 | 92.66 | 91.66 | 89.62 | 83.73 |
| LProp | 93.99 | 94.84 | 95.00 | 95.77 | 98.09 | 98.41 | 97.89 | 98.41 | 96.88 | |
| HotpotQA | AUQ | 56.07 | 57.61 | 57.60 | 58.66 | 60.17 | 62.39 | 66.09 | 68.46 | 68.46 |
| LProp | 53.58 | 58.29 | 61.56 | 64.61 | 66.02 | 67.80 | 70.78 | 74.76 | 78.85 |
Table 7. Qwen3-8B rejection-rate results. Retained-answer correctness (%) at each rejection rate. LProp values are underlined. HotpotQA uses exact match.
Figure 7 plots LProp minus AUQ in percentage points. LProp has higher retained-answer correctness at 35 of 36 operating points: all nine rates on BoolQ, StrategyQA, and MMLU mathematics, and @20–@90 on HotpotQA. These results extend the comparison beyond the three rates in the main table. Answer handling follows Appendix § 7.2.5; boundary ties are handled as specified above.
Inference Token Cost Details
Token Accounting
Table 8 reports mean inference-token positions per question. The total sums generation inputs and outputs, non-generative backbone encoding inputs, and required auxiliary-scoring inputs. It includes source generation, method-specific samples and reflections, and unsuccessful attempts. Repeated context is counted on each generation call.
| Method | Qwen3-8B BoolQ | Qwen3-8B StrategyQA | Qwen3-8B MMLU math | Qwen3-8B HotpotQA | Mistral-7B-Instruct-v0.3 BoolQ | Mistral-7B-Instruct-v0.3 StrategyQA | Mistral-7B-Instruct-v0.3 MMLU math | Mistral-7B-Instruct-v0.3 HotpotQA |
| SAUP | 1,633.0 | 4,071.1 | 11,681.3 | 10,734.7 | 1,824.7 | 4,539.8 | 9,026.2 | 12,710.5 |
| TRACER | 1,747.3 | 4,184.5 | 11,753.1 | 10,868.1 | 1,953.3 | 4,673.2 | 9,223.6 | 12,765.5 |
| AUQ | 4,732.7 | 15,833.3 | 10,855.1 | 13,919.3 | 4,487.5 | 12,253.7 | 15,827.4 | 15,564.7 |
| UProp | 4,489.1 | 9,472.8 | 21,313.3 | 19,002.4 | 5,278.2 | 11,260.5 | 19,150.3 | 23,548.5 |
| LProp | 577.7 | 1,930.9 | 8,976.7 | 8,260.3 | 597.9 | 2,119.7 | 6,582.5 | 11,222.7 |
Table 8. Mean inference tokens per question (). Includes generation input/output, backbone encoding, and auxiliary scoring; excludes training and padding. Bold indicates the lowest value within each setting.
Reusable encodings are counted once within each method and question; LProp’s three heads share these encodings. No cache discount is transferred across methods or questions. Training, development, padding, and counterfactual evaluation probes are excluded. For LProp, source construction and public-evidence encoding remain part of the accounting even though the learned propagation update requests no additional language tokens.
Additional Cost Comparisons
For each dataset–backbone setting, the percentage reduction divides the saved tokens by the comparator’s token count. We then average these reductions with equal weight across the eight settings. Relative to AUQ, LProp reduces token usage by 17.3–87.8%, with an unweighted mean reduction of 61.2%. Relative to UProp, the reductions range from 52.3% to 88.7%, with an unweighted mean of 71.1%.
On BoolQ/Qwen3, LProp uses 577.7 tokens per question, compared with 4,732.7 for AUQ and 4,489.1 for UProp, while achieving higher F1, AUROC, and AUARC than either method. On MMLU mathematics/Mistral, LProp reduces tokens by 58.4% relative to AUQ and 65.6% relative to UProp, alongside the improvements in answer quality and selective reliability reported in the main text.
Performance and Token Efficiency
The cost advantage accompanies higher F1 in all eight settings and higher selective-correctness scores in all eight. On HotpotQA/Qwen3, LProp uses 40.7% fewer tokens than AUQ and achieves higher AUARC.
As described in § 4.2.3, source construction supplies the inherited candidates and backbone encoding supplies the public evidence. The learned update revises candidate support directly, without additional textual reflection or trajectory sampling at each handoff. These comparisons quantify inference-token efficiency under the accounting protocol above.
An Empirical Case Study
The recorded episode below shows how LProp revises an inherited error as public reasoning and checking arrive. This single empirical case illustrates both the agent setting and the resulting propagation behavior.
A Recorded LProp Recovery Episode
We use a recorded Qwen3-8B episode to connect the multi-step construction in Appendix § 7.§ 2.2 with an observed answer recovery. The question comes from MMLU high-school statistics. A private solver produces the source draft; a public solver receives the question and options; a checker subsequently reviews the public derivation. These roles execute in the prescribed order. The receiver obtains the source carrier and successive public encodings, with private draft content retained through the carrier. The probabilities below are LProp’s answer distributions at the corresponding stages, rather than the public solver’s confidence scores.
How uncertainty changes during recovery. LProp revises both the inherited answer and its uncertainty as evidence arrives. The derivation transfers support from D to the retained alternative B; the subsequent check strengthens B. Selected-answer uncertainty first rises when the initial confidence is revised, then falls after confirmation. Freezing the update preserves the initial error under the same source state and answer-selection rule. The public derivation and check follow the fixed evidence schedule; their execution is not triggered by the displayed uncertainty values. This control isolates enabling the update for this recorded carrier and episode. The separately trained public-evidence-only control in Appendix § 11 tests the contribution of inherited content.
Recorded probability distributions. The workflow highlights B and D; the complete distributions are given below. Uncertainty is one minus the selected-answer probability.
| BoolQ | StrategyQA | MMLU mathematics | HotpotQA | |||||||||
| Initial | 291 / 709 | 167 / 313 | 33 / 281 | 267 / 245 | ||||||||
| 1,000 | 480 | 314 | 512 | |||||||||
| Receiver | Final | Final | Final | Final | ||||||||
| Full LProp | 182 | 51 | 840 | 73 | 25 | 361 | 13 | 7 | 287 | 21 | 3 | 263 |
| Handoff representation | ||||||||||||
| Probability-only | 144 | 72 | 781 | 33 | 26 | 320 | 8 | 7 | 282 | 1 | 4 | 242 |
| Text summary | 169 | 53 | 825 | 63 | 24 | 352 | 12 | 8 | 285 | 11 | 2 | 254 |
| Latent content | ||||||||||||
| Frozen content | 165 | 91 | 783 | 76 | 61 | 328 | 7 | 7 | 281 | 3 | 1 | 247 |
| Gaussian noise | 130 | 92 | 747 | 21 | 55 | 279 | 10 | 33 | 258 | 0 | 11 | 234 |
| Receiver design | ||||||||||||
| Matched MLP | 153 | 43 | 819 | 55 | 34 | 334 | 8 | 13 | 276 | 6 | 4 | 247 |
| Log-linear fusion | 95 | 38 | 766 | 19 | 16 | 316 | 5 | 3 | 283 | 1 | 1 | 245 |
Table 9. Matched receiver ablation counts with frozen Qwen3-8B. and denote wrong and correct answers. The two transition columns count recovered errors and introduced errors. Gray shades mark the best and second-best final-correct counts within each benchmark; full-model values are underlined.

Figure 6 (PDF p. 22). Case Study: Recovery through Propagation. Question. What does the difference between the 60th and 40th percentile scores measure? Options. A: central tendency; B: variability; C: symmetry; D: skewness. 1. Private draft: D (incorrect) Interprets the percentile gap as skewness. D: 91.56% B: 5.11%. Selected-answer uncertainty: 8.44%. 2. After public derivation: B (correct) Explains that the gap measures spread. D: 21.72% B: 76.93%. Selected-answer uncertainty: 23.07%. 3. After public check: B (correct) Confirms that spread means variability. D: 11.23% B: 87.32%. Selected-answer uncertainty: 12.68%. Same model, updates disabled Source D Final D (incorrect). Support for D stays at 91.56%. Latent handoff + evidence update. Evidence update. Qwen3-8B. Solid arrows: LProp updates; dashed branch: frozen-update control. Texts are condensed from the recorded episode.
| Stage | Answer | ||||
| Source | .0065 | .0511 | .0268 | .9156 | D |
| Derivation | .0045 | .7693 | .0090 | .2172 | B |
| Check | .0069 | .8732 | .0076 | .1123 | B |
| Frozen update | .0065 | .0511 | .0268 | .9156 | D |
Table 9 (PDF p. 22).
Case selection. This qualitative example is selected retrospectively from the evaluation records. Values average the development-selected heads with seeds 17, 29, and 43. The frozen-update control uses the same model with public updates disabled.
Matched Receiver Ablations
Do latent content and its update improve recovery?
Question and rationale. We test whether recovery depends on the handoff representation, informative latent content and its updating, or the receiver design.
Setting. We freeze Qwen3-8B and compare seven receiver variants using the main experiment’s splits. Questions, source-private evidence, candidates, supervision, and public evidence remain fixed, including each benchmark’s multi-step schedule (Appendix § 7.§ 2.2). Full LProp is the retrained reference and receives both source probabilities and latent content .
Controls. We design different control type:
Handoff representation: probability-only retains and zeros source ; its BoolQ receiver can still construct states from public evidence. Text summary transfers a 128-token summary of the private evidence.
Latent content: frozen content disables revision of source while allowing probability updates; Gaussian noise replaces source with a Gaussian-noise carrier.
Receiver design: matched MLP uses source probabilities and public-evidence features without source , with parameters within 1% of the full receiver; log-linear fusion combines source and public probabilities using two learned coefficients. The probability-only, frozen-content, matched-MLP, and fusion controls are retrained for their respective interfaces.
Metrics. Figure 6 shows recovery , regression , and net gain , where is the number of questions. Correctness follows the main protocol, including HotpotQA exact match. Table 9 reports counts; final correct equals initial correct plus recovered minus regressed.
Results. Full LProp has the highest final correctness in every setting. It recovers more errors than probability-only and text-summary handoffs. Gaussian noise reduces final correctness across all benchmarks; on StrategyQA, frozen content recovers slightly more errors but introduces substantially more regressions. Full LProp also outperforms matched MLP and log-linear fusion, supporting the complete receiver design.
Additional Related Work
Large language models (LLMs) have advanced rapidly in recent years [53, 47 b]. Building on this progress, LLM-driven agents and agentic reinforcement learning have gained increasing attention for their ability to plan, interact with external environments, and solve complex tasks with limited human oversight [54, 10 a]. In this section, we provide a broader review of our related work.
Uncertainty, interpretation, and commitment. Measuring disagreement among possible answers and deciding what to communicate are distinct reliability problems. Kernel Language Entropy addresses the former by replacing hard semantic clusters with a positive-semidefinite kernel that captures graded similarities and by computing its von Neumann entropy [30]. SAGE uses answer-conditioned targets to calibrate verbal expressions of uncertainty, while GCPO combines geometry-aware disagreement with reward-based calibration to guide policy optimization [38, 54]. Linguistic calibration places the target at the reader, training the model to generate responses that improve the reader’s probabilistic forecasts [3]. Conformal methods use calibration to control what the system returns. Conformal generation uses sampling and rejection to construct candidate sets with a coverage guarantee for an acceptable response [34]. Conformal factuality controls the risk of unsupported assertions by backing off to less specific claims [29]. These guarantees depend on the calibration data and the distributional assumptions of each method.
Detecting and filling information gaps. The signal that flags a possible information gap and the source that can fill it play different roles. Self-CheckGPT compares sampled responses to flag potentially inconsistent factual statements [26]. When external evidence can address a gap, FLARE uses low-confidence predicted tokens to trigger retrieval [22], while Self-RAG learns when to retrieve and uses reflection tokens to critique retrieved passages and generated text [1]. Once evidence has been retrieved, CRAG evaluates it to choose among refinement, web search, and a combination of the two [49]. CiteAudit adds source verification to the evidence pipeline [36]. Clarification addresses gaps in what the user has communicated. Drift-Bench studies clarification across input faults and user personas [4, 54], while masked-user-reply experiments motivate assigning training credit based on representations of unmet requirements [5].
Allocating computation and context. The timing of an allocation decision determines which signals it can use. Before generation, HybridLLM predicts query difficulty to balance quality and cost [8], while RouteLLM routes queries using learned model preferences [31]. Once answers are available, FrugalGPT scores completed responses and escalates through a model cascade when scores fall below optimized thresholds [6], while EvolveRouter uses answer agreement to decide when to stop [20]. Allocation also extends to the agents, context, and evidence used to solve a task. AgentRouter and NG-Router organize selection around graph-structured agent–context relations and evidence subgraphs, respectively [54, 39]. Across repeated tasks, LLMA-Mem shows that reusing experience can let smaller teams outperform larger ones [46].
Preserving information for revision. What a downstream stage can revise depends on the information it receives. RARR takes a completed generation, retrieves evidence, and edits unsupported passages while seeking to preserve the rest [16]. Multiagent debate revises proposed answers through successive rounds of exchanged answers and reasoning, with agreement emerging from deliberation among model instances [11]. Both approaches operate on generated text. LProp studies what candidate-specific information a handoff must preserve so that new evidence can update both the latent content of individual candidates and the probability mass assigned to them.
References
- [1]Akari Asai, Zeqiu Wu, Yizhong Wang, Avi Sil, and Hannah Hajishirzi. 2024. Self-rag: Learning to retrieve, generate, and critique through self-reflection. In International conference on learning representations, volume 2024, pages 9112–9141.arxiv.org/abs/2310.11511
- [2]Amos Azaria and Tom Mitchell. 2023. The internal state of an llm knows when it’s lying. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 967–976.arxiv.org/abs/2304.13734
- [3]Neil Band, Xuechen Li, Tengyu Ma, and Tatsunori Hashimoto. 2024. Linguistic calibration of long-form generations. arXiv preprint arXiv:2404.00474.arxiv.org/abs/2404.00474
- [4]Han Bao, Zheyuan Zhang, Pengcheng Jing, Zhengqing Yuan, Kaiwen Shi, and Yanfang Ye. 2026a. Drift-bench: Diagnosing cooperative breakdowns in llm agents under input faults via multi-turn interaction. arXiv preprint arXiv:2602.02455.arxiv.org/abs/2602.02455
- [5]Han Bao, Zheyuan Zhang, Kaiwen Shi, and Yanfang Ye. 2026b. Not every user says it all: Training tool-using agents beyond ideal simulated users. arXiv preprint arXiv:2610.24294.arxiv.org/abs/2610.24294
- [6]Lingjiao Chen, Matei Zaharia, and James Zou. 2023. FrugalGPT: How to use large language models while reducing cost and improving performance. arXiv preprint arXiv:2305.05176.arxiv.org/abs/2305.05176
- [7]Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019. Boolq: Exploring the surprising difficulty of natural yes/no questions. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2924–2936.arxiv.org/abs/1905.10044
- [8]Dujian Ding, Ankur Mallick, Chi Wang, Robert Sim, Subhabrata Mukherjee, Victor Rühle, Laks Lakshmanan, and Ahmed H Awadallah. 2024. Hybrid llm: Cost-efficient and quality-aware query routing. In International Conference on Learning Representations, volume 2024, pages 41348–41366.arxiv.org/abs/2404.14618
- [10]Guangting Dong, Hangyu Mao, Kai Ma, Licheng Bao, Yifei Chen, Zhongyuan Wang, Zhongxia Chen, Jia-zhen Du, Huiyang Wang, Fuzheng Zhang, et al. 2025b. Agentic reinforced policy optimization. arXiv preprint arXiv:2507.19849.arxiv.org/abs/2507.19849
- [11]Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch. 2023. Improving factuality and reasoning in language models through multi-agent debate. arXiv preprint arXiv:2305.14325.arxiv.org/abs/2305.14325
- [12]Zhuoyun Du, Runze Wang, Huiyu Bai, Zouying Cao, Xiaoyong Zhu, Yueh-Cheng Yu, Bo Zheng, Wei Chen, and Haochao Ying. 2026. Enabling agents to communicate entirely in latent space. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 27106–27129.
- [13]Jinhao Duan, James Diffenderfer, Sandeep Madireddy, Tianlong Chen, Bhavya Kailkhura, and Kaidi Xu. 2025. Uprop: Investigating the uncertainty propagation of llms in multi-step agentic decision-making. arXiv preprint arXiv:2506.17419.arxiv.org/abs/2506.17419
- [14]Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. 2024. Detecting hallucinations in large language models using semantic entropy. Nature, 630(8017):625–630.DOI
- [15]Marina Fomicheva, Shuo Sun, Lisa Yankovskaya, Frédéric Blain, Francisco Guzmán, Mark Fishel, Nikolaos Aletras, Vishrav Chaudhary, and Lucia Specia. 2020. Unsupervised quality estimation for neural machine translation. Transactions of the Association for Computational Linguistics, 8:539–555.arxiv.org/abs/2005.10608
- [16]Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, et al. 2023. Rarr: Researching and revising what language models say, using language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 16477–16508.arxiv.org/abs/2210.08726
- [17]Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. 2021. Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies. Transactions of the Association for Computational Linguistics, 9:346–361.arxiv.org/abs/2101.02235
- [18]Jiuzhou Han, Wray Buntine, and Ehsan Shareghi. 2024. Towards uncertainty-aware language agent. In Findings of the Association for Computational Linguistics: ACL 2024, pages 6662–6685.
- [19]Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300.
- [20]Jiatian Huang, Zheyuan Zhang, Kaiwen Shi, Yanfang Ye, and Chuxu Zhang. 2026. Evolverouter: Co-evolving routing and prompt for multi-agent question answering. arXiv preprint arXiv:2604.05149.arxiv.org/abs/2604.05149
- [21]Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yujing Yang, and Lili Qiu. 2023a. Llmlingua: Compressing prompts for accelerated inference of large language models. In Proceedings of the 2023 conference on empirical methods in natural language processing, pages 13358–13376.arxiv.org/abs/2310.05736
- [22]Zhengbao Jiang, Frank F Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie
- [23]Jannik Kossen, Jiatong Han, Muhammed Razzak, Lisa Schut, Shreshth Malik, and Yarin Gal. 2024. Semantic entropy probes: Robust and cheap hallucination detection in llms. arXiv preprint arXiv:2406.15927.arxiv.org/abs/2406.15927
- [24]Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. 2023. Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation. arXiv preprint arXiv:2302.09664.arxiv.org/abs/2302.09664
- [25]Andrey Malinin and Mark Gales. 2020. Uncertainty estimation in autoregressive structured prediction. arXiv preprint arXiv:2002.07650.arxiv.org/abs/2002.07650
- [26]Potsawee Manakul, Adian Liusie, and Mark Gales. 2023. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. In Proceedings of the 2023 conference on empirical methods in natural language processing, pages 9004–9017.
- [27]Meta. 2026. Introducing Muse: The world's first personal AI agent built for everyone. Meta Newsroom.
- [28]Mistral AI. 2024. Mistral-7B-Instruct-v0.3 model card.
- [29]Christopher Mohri and Tatsunori Hashimoto. 2024. Language models with conformal factuality guarantees. arXiv preprint arXiv:2402.10978.arxiv.org/abs/2402.10978
- [30]Alexander Nikitin, Jannik Kossen, Yarin Gal, and Pekka Marttinen. 2024. Kernel language entropy: Fine-grained uncertainty quantification for llms from semantic similarities. Advances in Neural Information Processing Systems, 37:8901–8929.arxiv.org/abs/2405.20003
- [31]Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E Gonzalez, Mohammed Kadous, and Ion Stoica. 2025. Routellm: Learning to route llms from preference data. In International Conference on Learning Representations, volume 2025, pages 34433–34448.arxiv.org/abs/2406.18665
- [32]OpenAI. 2025. Introducing ChatGPT agent: Bridging research and action.
- [33]Chau Pham, Boyi Liu, Yingxiang Yang, Zhengyu Chen, Tianyi Liu, Jianbo Yuan, Bryan Plummer, Zhaoran Wang, and Hongxia Yang. 2024. Let models speak ciphers: Multiagent debate through embeddings. In International Conference on Learning Representations, volume 2024, pages 51899–51928.arxiv.org/abs/2310.06272
- [34]Victor Quach, Adam Fisch, Tal Schuster, Adam Yala, Jae Ho Sohn, Tommi Jaakkola, and Regina Barzilay. 2024. Conformal language modeling. In International Conference on Learning Representations, volume 2024, pages 11654–11681. Qwen,; An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, and 25 others. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115.
- [35]Yangjun Ruan, Honghua Dong, Andrew Wang, Silviu Pitis, Yongchao Zhou, Jimmy Ba, Yann Dubois, Chris Maddison, and Tatsunori Hashimoto. 2024. Identifying the risks of lm agents with an lm-emulated sandbox. In International Conference on Learning Representations, volume 2024, pages 27031–27098.arxiv.org/abs/2309.15817
- [36]Kaiwen Shi, Weixiang Sun, Zheyuan Zhang, Lichao Sun, Nitesh V Chawla, and Yanfang Ye. 2026a. Citeaudit: You cited it, but did you read it? a benchmark for verifying scientific references in the llm era. arXiv preprint arXiv:2602.23452.
- [37]Kaiwen Shi, Zheyuan Zhang, Han Bao, Colby Nelson, and Yanfang Ye. 2026b. Confidence laundering in agent systems: Why uncertainty needs a latent carrier. arXiv preprint arXiv:2606.20662.
- [38]Kaiwen Shi, Zheyuan Zhang, and Yanfang Ye. 2026c. Sage: Answer-conditioned uncertainty targets for verbal uncertainty alignment. arXiv preprint arXiv:2606.11512.
- [39]Kaiwen Shi, Zheyuan Zhang, Zhengqing Yuan, Keerthiram Murugesan, Vincent Galassi, Chuxu Zhang, and Yanfang Ye. 2026d. Ng-router: Graph-supervised multi-agent collaboration for nutrition question answering. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7508–7527.
- [41]Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. Advances in neural information processing systems, 36:8634–8652.
- [42]Sina Tayebati, Divake Kumar, Nastaran Darabi, Davide Ettori, Ranganath Krishnan, and Amit Ranjan Trivedi. 2026. Tracer: Trajectory risk aggregation for critical episodes in agentic reasoning. arXiv preprint arXiv:2602.11409.
- [43]Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher D Manning. 2023. Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5433–5442.
- [44]Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. 2024. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. Advances in Neural Information Processing Systems, 37:95266–95290.DOI
- [45]Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, et al. 2023. Autogen: Enabling next-gen llm applications via multi-agent conversation. arXiv preprint arXiv:2308.08155.
- [46]Shanglin Wu, Yuyang Luo, Yueqing Liang, Kaiwen Shi, Yanfang Ye, Ali Payani, and Kai Shu. 2026. Scaling teams or scaling time? memory enabled lifelong learning in llm multi-agent systems. arXiv preprint arXiv:2604.03295.
- [47]Siheng Xiong, Oguzhan Gungordu, James K Kerce, and Faramarz Fekri. 2026a. Adaptive information control for search-augmented llm reasoning. arXiv preprint arXiv:2602.01672.
- [49]Shi-Qi Yan, Jia-Chen Gu, Yun Zhu, and Zhen-Hua Ling. 2024. Corrective retrieval augmented generation. arXiv preprint arXiv:2401.15884.arxiv.org/abs/2401.15884
- [50]An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengyuan Huang, Chenxu Lv, et al. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.arxiv.org/abs/2505.09388
- [51]Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 conference on empirical methods in natural language processing, pages 2369–2380.
- [52]Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629.
- [53]Yanfang Ye, Zheyuan Zhang, Tianyi Ma, Zehong Wang, Yiyang Li, Shifu Hou, Weixiang Sun, Kaiwen Shi, Yijun Ma, Wei Song, et al. 2025. Llms4all: A review of large language models across academic disciplines. arXiv preprint arXiv:2509.19580.
- [54]Jiaxin Zhang, Prafulla Kumar Choubey, Kung-Hsiang Huang, Caiming Xiong, and Chien-Sheng Wu. 2026a. Agentic uncertainty quantification. arXiv preprint arXiv:2601.15703.arxiv.org/abs/2601.15703



