betaFeedback

WorksComputer Science

When Does Evidence-Graph Structure Predict Answer Correctness? Label-Free Abstention for Structured Retrieval

Abstract

Retrieval systems return scored answers, but a high score does not show that an answer should be trusted. The proposed training-free abstention method applies the boundary cut to the selected evidence in an evidence graph. The signal is the weighted cut between selected and unselected candidates and requires no correctness labels or confidence-model training. On MemoryAgentBench (MAB) FactConsolidation multi-hop, with unresolved queries ranked last for every selector, it raises area under the precision-coverage curve from 0.736 for the best score baseline to 0.785 (+0.049, 95% CI [+0.008, +0.092]); under the native evaluation, where score baselines keep their unresolved values, the gap is +0.108. A zero-cut subset contains 71 of 400 queries at 100% observed precision. In this pipeline, the subset corresponds to structural closure; controlled analyses attribute much of the remaining separation to resolution status, with little benefit from continuous edge weights. Across external benchmarks, usefulness varies with graph construction, edge semantics, and relation; typed structure alone does not ensure useful confidence. A supervised gradient-boosted selector achieves 0.814 versus the boundary cut's 0.785 when the retrieval stack matches. These findings provide empirical guidance for using available evidence graphs for lightweight, auditable post-retrieval abstention without additional model inference.

Introduction

Retrieval systems often decide whether to trust an answer from the retrieval score alone. But a high score does not tell us whether the selected evidence is uncontested.

Consider the query: What country is Alice’s birthplace in? A retrieval system may select the facts “Alice was born in Paris” and “Paris is in France” and return France. A score-based confidence rule asks whether these selected facts received high scores. It does not ask whether the same local evidence graph also contains a competing birthplace, a conflicting path, or other candidates connected to the selected support.

This distinction matters in structured retrieval systems. Agent-memory, knowledge-graph, and provenance-aware pipelines often already organize retrieved evidence as typed graphs. These graphs contain information that a scalar retrieval score discards: how the selected evidence is connected to plausible alternatives.

We ask whether this structure can be used directly to decide when a retrieved answer should be trusted.

Existing query performance prediction methods estimate retrieval quality from scores or relationships among ranked items (§ 6). We instead study whether the boundary of the evidence actually selected to support an answer provides a query-level signal for abstention in evidence graphs.

Our starting point is simple. Let PP be the evidence selected for the returned answer, and let the local evidence graph contain both PP and nearby unselected candidates. The boundary measures how strongly the selected evidence remains connected to unselected candidates in the evidence graph. A larger boundary indicates that the selected support is less structurally isolated; whether this separation predicts correctness depends on the semantics of the graph edges.

The resulting abstention rule can be added to a structured retrieval pipeline without training a confidence model. It leaves the retrieval and answer-selection process unchanged, then uses a sparse edge sum over the available local evidence graph to decide whether to return the answer or abstain. This separates answer selection from answer acceptance, and offers lightweight post-retrieval abstention without additional model inference.

The central question is:

When does the structure of retrieved evidence provide information about whether the returned answer is correct?

Our experiments show that the answer depends on what the graph represents. The signal is useful when edges encode support compatibility: two connected candidates are expected to support the same answer hypothesis. Typed entity–relation structure can provide this property, whereas temporal adjacency, co-occurrence, or generic semantic similarity often do not. Our experiments identify alignment with answer-support semantics as a key condition for useful structural confidence.

We study the weighted boundary between the evidence selected for an answer and the remaining candidates. We call this boundary statistic the boundary cut; it is the Dirichlet energy of the selected-support indicator (§ 3.2), and it appears as ‘coherence’ in table labels, figure legends, and released artifacts.

On MAB multi-hop, ranking unresolved queries last for every selector gives the boundary cut AUPCC 0.785 versus the strongest score baseline’s 0.736, a status-matched advantage of +0.049 (95% CI [+0.008, +0.092]). The native gap is +0.108, explicitly including the difference in unresolved-status handling. The zero-cut subset contains 71/400 evaluations, all correct. A matched-coverage score threshold selects a substantially different set and overlaps with only 14 of these queries.

The experiments also reveal why the effect occurs. On MAB, the 100%-precision regime corresponds to structural closure: the selected support exhausts its local typed neighborhood. Structural controls match the weighted boundary signal on MAB, but its apparent advantage outside closure largely disappears after unresolved queries are excluded. External benchmarks then show the limit of this result: the same boundary statistic is not universally useful, and its behavior changes with graph construction, candidate-space structure, and edge semantics.

These results identify deployment tradeoffs between structural abstention and learned confidence. GBT has a higher in-distribution point estimate, but its paired difference from the boundary cut is not statistically resolved. The structural rule is recomputed from the deployed evidence graph and requires neither correctness labels nor confidence-model retraining.

Contributions

  1. Structural abstention. A training-free acceptance rule on selected-support boundaries includes a structural-closure regime with 100% observed precision at approximately 18% coverage on MAB multi-hop. This regime is stable across the evaluated encoder, scorer, and γ\gamma settings and degrades gracefully under extraction noise. Closure is produced by the upstream resolver; it is an empirical reliability indicator, not a certificate of correctness.

  1. Mechanism. An empirical decomposition separates closure, resolution state, boundary topology, and continuous edge weights. On MAB, the graded signal outside closure largely reflects resolution-state composition, and continuous weights are unnecessary for the observed selective behavior.

  1. Applicability. We identify structural characteristics associated with effective abstention and investigate how they vary across retrieval tasks and answer relations. Cross-benchmark and relation-level evidence links usefulness to graph semantics and candidate-space structure; support compatibility is an explanatory hypothesis rather than a validated predictive criterion.

  1. Deployment tradeoffs. On the regenerated population, the boundary cut exceeds four local score-distribution QPP adaptations with paired CIs excluding zero. GBT has the higher in-distribution point estimate, not statistically resolved, and portability across retrieval stacks is mixed. The structural rule needs no correctness labels.

Method

Candidates, Scores, and the Evidence Graph

For a query qq, a retrieval system returns candidate items N={o1,…,on}\mathcal{N} = \{o_1,\ldots,o_n\}. A candidate may be a sentence, a knowledge-graph triple, or an entity, depending on the task. Each candidate ii has a representation vi∈Rd\mathbf{v}_i \in\mathbb{R}^d and a retrieval score bib_i. We write the score vector as b∈Rn\mathbf{b} \in\mathbb{R}^n. The abstention rule also uses a weighted graph GN=(N,EN,w)G_{\mathcal{N}} = (\mathcal{N}, E_{\mathcal{N}}, w) over the retrieved candidates. Its adjacency matrix is A=[wij]A = [w_{ij}], with wij≥0w_{ij} \ge0, and its Laplacian is L=D−AL = D - A, where Dii=∑jwijD_{ii} = \sum_j w_{ij}. An edge does not mean that two candidates are generally similar. It means that, for the current query, the two candidates should support the same answer. A boundary edge therefore connects selected support to an unselected candidate expected to support the same answer hypothesis; it does not by itself encode a contradiction or a competing answer. The method does not prescribe a specific encoder, scorer, or graph-construction implementation, subject to the support-compatibility requirement below; the abstention rule itself requires only the resolved selected support PP and the weighted graph. Retrieval scores are upstream inputs, used to resolve PP and for the score-based baselines.

Graph construction and support compatibility The abstention rule does not prescribe how the graph is constructed, but its usefulness depends on what the edge weights mean. In an existing knowledge graph, raw weights may encode similarity, proximity, or relation strength. For abstention, these weights can be preserved but gated by a query-dependent support function: wijverify=sijm(q,oi,oj)w^{\mathrm{verify}}_{ij} = s_{ij}m(q,o_i,o_j), where sij≥0s_{ij} \ge0 is the original edge weight and m(q,oi,oj)∈[0,1]m(q,o_i,o_j) \in[0,1] measures whether candidates oio_i and ojo_j should receive similar support scores for the current answer. More generally, the abstention rule can use any nonnegative application-specific edge function wij=g(q,oi,oj,vi,vj)≥0w_{ij} = g(q,o_i,o_j,\mathbf{v}_i,\mathbf{v}_j) \ge0. We investigate whether assigning edge weight to candidate pairs whose support scores should agree is associated with informative structural confidence.

To make this condition explicit, let yiy_i be the latent answer-support class of candidate ii: yi=yjy_i = y_j means that candidates ii and jj support the same underlying answer hypothesis. For a graph with nonzero total edge weight, define

ρ(GN;y)=∑i<j: yi≠yjwij∑i<jwij\rho(G_{\mathcal{N}}; \mathbf{y}) = \frac{\sum_{i<j:\,y_i\ne y_j} w_{ij}} {\sum_{i<j} w_{ij}}

A graph constructor is more support-compatible when this fraction is smaller: most edge weight lies within answer-support classes rather than between classes whose scores should differ. The latent classes yiy_i are not observed at deployment and are not inputs to the abstention rule. They describe a structural characteristic for interpreting graph constructors; this ratio is not a validated predictive criterion.

Boundary Cut and Abstention

Let PP denote the evidence selected for the returned answer and zi=1[i∈P]z_i = \mathbf{1}[i \in P] its binary indicator on the local neighborhood VV. Classical Dirichlet energy is then exactly the weighted boundary cut of the selected support:

E(z)=zTLz=∑i∈P, j∉Pwij.(1)E(\mathbf{z}) = \mathbf{z}^{\mathsf{T}}L\mathbf{z} = \sum_{i\in P,\,j\notin P} w_{ij}. \tag*{(1)}

Undirected edges are counted once. On MAB, PP is the selected support produced by the benchmark resolver: the resolved multi-hop evidence path or the selected single-hop supporting fact with its induced local support. In the other evaluated settings, PP is the selected top-kk set (§3). Thus a low cut means the selected support is weakly coupled to unselected candidates, and exact zero means no positive-weight edge leaves the selected support. We later find that, on MAB, continuous edge magnitudes are not necessary for the selective effect: replacing them with an unweighted boundary count gives equivalent performance (§4.2; see §4.3).

Unresolved queries are abstained from before evaluating the boundary rule. For precision–coverage evaluation, they receive a worst-confidence sentinel and are ranked last.

The abstention rule does not change the retrieved answer; it decides whether to trust the top-scoring answer. The threshold τ\tau is swept to produce precision–coverage curves; the zero-cut result corresponds to τ=0\tau= 0. Because the cut scales with the edge weights, fixed absolute thresholds are weight-scale dependent; our evaluation therefore uses threshold sweeps rather than a universal threshold. Computing the score requires no correctness labels, but choosing a threshold to meet a target precision requires separate validation. The zero-cut rule supplies a fixed structural criterion; its relationship to correctness is empirical. Laplacian-smoothed scores, b~=(I+γL)−1b\tilde{\mathbf{b}} = (I + \gamma L)^{-1}\mathbf{b}, are not part of the abstention rule; they are included only as a score-based baseline (Appendix I).

After the selected support and the graph are available, abstention is a single sparse edge sum. It costs O(∣EN∣)O(|E_{\mathcal{N}}|) time. If the edge list is streamed or already stored, the computation uses O(1)O(1) working memory beyond the graph and node field; materializing the sparse Laplacian costs O(∣EN∣)O(|E_{\mathcal{N}}|) memory. The dense worst case is O(n2)O(n^2). Graph construction is

Algorithm 1 Boundary-cut abstention for one query

Require: query qq, candidate set N={o1,…,on}\mathcal{N} = \{o_1,\ldots,o_n\}, candidate representations {vi}i=1n\{\mathbf{v}_i\}_{i=1}^{n}, retrieval scores b\mathbf{b}, graph constructor G\mathcal{G}, selection/resolution rule S\mathcal{S}, threshold τ\tau

Ensure: COMMIT or ABSTAIN

1: GN←G(q,N,{vi}i=1n)\mathcal{G}_{N} \gets\mathcal{G}(q,\mathcal{N},\{\mathbf{v}_i\}_{i=1}^{n}) 2: (P,oa)←S(q,N,b,GN)(P,o_a) \gets\mathcal{S}(q,\mathcal{N},\mathbf{b},\mathcal{G}_{N}) ▷ S\mathcal{S} may ignore GN\mathcal{G}_{N} 3: if the resolver reports NOT_FOUND then 4: &nbsp;&nbsp;&nbsp;&nbsp;return ABSTAIN 5: end if 6: zi←1[i∈P]z_i \gets1[i \in P] for i=1,…,ni = 1,\ldots,n ▷ hard selected-support field 7: E←∑i∈P,j∉PwijE \gets\sum_{i\in P,j\notin P} w_{ij} ▷ boundary cut of PP 8: if E≤τE \le\tau then 9: &nbsp;&nbsp;&nbsp;&nbsp;return COMMIT(oa,E)(o_a,E) 10: else 11: &nbsp;&nbsp;&nbsp;&nbsp;return ABSTAIN(oa,E)(o_a,E) 12: end if

separate from the abstention rule: a typed graph uses available entity–relation structure, while a semantic kkNN graph has the cost of the nearest-neighbor routine used to build it.

The method applies classical Dirichlet energy to the binary selected-support indicator, yielding the weighted boundary statistic in Eq. (1). It measures how strongly selected evidence is connected to unselected support and supplies an abstention signal for the retrieved support neighborhood.

Boundary Interpretation

Remark 2.1 (Binary selected-support boundary). For a binary selected-support indicator and nonnegative edge weights, zero cut means that no positive-weight edge crosses from PP to its complement. Equivalently, the indicator is constant on each positive-weight connected component, and PP is a union of such components. Increasing a crossing edge’s weight increases the cut by the same amount; increasing an internal edge’s weight leaves it unchanged. These are immediate consequences of the boundary sum in Eq. (1) and do not establish answer correctness.

Remark 2.1 explains why graph construction matters. Increasing weight on a pair increases its contribution in proportion to the squared field disagreement. For the selected-support indicator zi=1[i∈P]z_i = 1[i \in P], (zi−zj)2=1(z_i-z_j)^2 = 1 exactly when one endpoint lies in PP and the other does not, so increasing the weight of such an edge increases the selected-support boundary cut; this is the primary interpretation of the remark in our experiments. Adding weight is useful when the pair should agree under the current answer hypothesis, but harmful when the pair is only temporally adjacent, co-occurring, or generically similar. A support gate m(q,oi,oj)m(q,o_i,o_j) is therefore useful because it preserves raw graph weights only on pairs that should support the same answer.

Why Graph Structure Matters

The preceding properties hold for whatever abstention weights the constructor produces, so their practical consequence is a requirement on graph construction: edges should connect candidates whose support scores should agree under the current answer hypothesis, and large weights on support-incompatible pairs (candidates that are merely temporally adjacent, co-occurring, or generically similar) add boundary cut unrelated to correctness. Appendix § 10 gives a continuous latent-class model that makes this requirement precise by decomposing Dirichlet energy into within-class and across-class terms; it is motivational intuition only and is not instantiated in any reported experiment (every reported boundary cut result verifies the hard selected-support indicator of Eq. (1)).

Experimental Setup

Experimental rationale and baselines The experiments test whether the boundary cut is a label-free abstention signal when the evidence graph is support-compatible. MAB provides the positive setting. We compare the boundary cut to three kinds of alternatives. Score-based confidence baselines (raw score, smoothed score, and score margin) test whether the boundary cut adds information beyond retrieval confidence alone. Search-style baselines (leaf MCTS, chain-conditioned MCTS, and path-score reranking) test whether additional post-retrieval computation solves the same failures. Learned confidence baselines (logistic regression and a shallow gradient-boosted tree ensemble) test how a zero-fit graph signal compares with supervised confidence models trained on retrieval-time features. In addition, graph ablations and retrieval-stack shifts test the graph and the encoder/scorer dependence (below).

Statistical unit and metric. Tables and figure legends label the boundary-cut signal ‘Coherence’ for continuity with the released artifacts. The reconstructed mapping (Appendix M) contains 374 distinct exact question texts across 400 evaluations at four context scales; resampling and folds group all occurrences of each text together. The mapping uses positional lookup into the available dataset snapshot; the historical run did not record its dataset revision (Appendix M). Selective prediction is summarized by AUPCC: predictions are ordered most- to least-confident, and at prefix kk we take precision (correct among the top kk, over kk) and coverage k/nk/n; AUPCC is the trapezoidal area under this curve. AUPCC is the primary metric because the boundary cut is an abstention signal—it changes which predictions are trusted, not the base retrieval answer at full coverage—so we report both retrieval accuracy and precision–coverage quality, with bootstrap 95% CIs over queries for key comparisons. The boundary cut is oriented so that a lower cut is more confident; raw, smoothed, and margin scores use higher-is-confident; predictions are sorted by argsort(−confidence) and exact ties are resolved in the evaluator’s native row order (no semantic secondary key). Two implementations are used: the headline Table 1 evaluator subsamples the nn prefixes to 50 points before integration, while the learned-baseline and bootstrap comparisons use all nn prefixes; on the headline signals the two agree to within 0.0024 AUPCC. Continuous signals (boundary cut, raw, smoothed) are effectively insensitive to tie ordering; discrete or constant structural-control signals (e.g. the closure indicator, complement size, or unweighted boundary count on the P≠VP \ne V subset) can be tie-order-sensitive, so their absolute AUPCC values are interpreted descriptively and compared as paired deltas under a shared ordering. Unresolved (NOT_FOUND) queries take a worst-confidence sentinel for the boundary cut that ranks them last; the score baselines carry their native small values and are not forced last, and the few missing score-derived features are zeroed by the historical implementation.

RegimeBase AccuracyCoherenceAUPCCStatus-matched smoothedGap
Multi-hop0.420–0.7000.7850.736+0.049
Single-hop0.900–0.9600.975——

Table 1. Aggregate results with the status-matched multi-hop comparison primary: unresolved queries are ranked last for every selector. Native comparisons are reported below the table; resolved-only results are in § 5.1.

Primary benchmark. MemoryAgentBench (MAB) FactConsolidation [13] is the primary positive test because it provides retrieved candidate facts, explicit multi-hop evidence, and typed entity–relation structure. The multi-hop regime tests whether the boundary cut helps when evidence must be composed across facts; the single-hop regime tests whether the signal remains useful when base retrieval accuracy is already high. MAB has four context scales (6k–262k tokens), 100 queries per split, and 400 queries per regime. On MAB, PP is one such graph constructor: a typed entity–relation mask multiplied by squared embedding similarity, wij=Mij(viTvj)2w_{ij} = M_{ij}(\mathbf{v}_i^{\mathsf{T}}\mathbf{v}_j)^2. For MAB, Mij=1M_{ij} = 1 when the retrieved candidate facts share the same extracted entity id, and 0 otherwise. These entity ids are available in candidate metadata at retrieval time; the mask does not use gold answers, correctness labels, or calibration data.

Node field. In every experiment the node field is a binary selected-support indicator; no experiment computes the cut of a continuous score field. On MAB, PP is the resolver’s selected support as defined in §2.2; on the transfer benchmarks (CoDeX-S, MetaQA-3hop, LongMemEval, LoCoMo), PP is the top-kk retrieved set under the smoothed ranking.

MAB resolver and neighborhood. The MAB multi-hop pipeline runs in order: (1) context parsing/triple extraction into a per-example knowledge base; (2) question decomposition into a hop chain; (3) per-hop candidate scoring by smoothed similarity; (4) path resolution, where each hop’s object seeds the next hop, yielding the resolved support path PP (NOT_FOUND if a hop fails); (5) construction of the local neighborhood VV as PP together with its typed one-hop neighbors, so P⊆VP \subseteq V by construction; (6) typed edges over VV; (7) the boundary cut of the selected-support indicator; (8) correctness scoring against the gold answer. Gold answers are used only in step 8; they do not enter the construction of PP or VV.

Transfer and graph ablations. To test whether the boundary cut depends on a particular retrieval stack, we evaluate two additional multi-hop configurations: a linear scorer qTvi\mathbf{q}^{\mathsf{T}}\mathbf{v}_i instead of the default (qTvi)2(\mathbf{q}^{\mathsf{T}}\mathbf{v}_i)^2, and a MiniLM encoder (d=384d=384) instead of the default Qwen encoder (d=128d=128), with the typed graph held fixed (Table 4). To isolate graph construction itself, we cache the retrieved MAB candidates, node scores, embeddings, and correctness labels from one run and recompute the boundary cut post hoc under alternative graphs: the typed entity–relation graph, a semantic kNN graph, and hybrid graphs that interpolate the two. Because the candidates and scores are fixed, this ablation isolates the graph effect; its absolute AUPCC values are not directly comparable to the end-to-end results in Table 1 and Table 13.

SignalAUPCCAUROC
GBT0.8140.889
Coherence0.7850.807
Smoothed score0.6770.645
Raw score0.6530.611
LR0.6480.620
Score margin0.5470.413

Table 13. Learned and training-free confidence on headline MAB multi-hop (N=400N = 400), using the reconstructed mapping (Appendix § 21). GBT and LR use five-fold cross-validation grouped by exact question; paired uncertainty resamples exact question texts. Native row order; per-fold StandardScaler. CIs condition on fixed out-of-fold predictions.

Use of AI tools Experiment code, statistical analyses, and reproducibility audits were implemented with LLM-based coding agents (Claude Code and Codex) under the author’s direction. All protocols, eligibility rules, and hypotheses were specified by the author before results were computed; all reported numbers were verified by the author against frozen artifacts; artifact identifiers and protocol records are available from the author. The fact-extraction and question-decomposition model inside the evaluated pipeline is gpt-4.1-mini-2025-04-14 (§ 4). LLM tools were also used for writing assistance.

Results

The Boundary Cut Improves Aggregate Precision–Coverage on MAB

Table 1 summarizes both regimes (per-split detail in Appendix § 9).

In the multi-hop regime, 71 of 400 rows have a zero boundary cut and all 71 are correct. Under the reconstructed mapping (Appendix § 21), these rows represent 65 distinct questions, all correct. The exact two-sided 95% Clopper–Pearson interval over distinct questions is [0.945, 1.000]. Observed row coverage is 17.75% (71/400), with 95% CI grouped by exact question [0.138, 0.219]. The single-hop regime shows that the signal is not limited to low-accuracy retrieval: even with 90–96% base accuracy, the boundary cut gives the best aggregate AUPCC. On one split (SH 32k), smoothed score narrowly leads (0.951 vs. 0.934); this is the only exception across both regimes. Per-split single-hop precision–coverage curves are shown in Figure 1.

Figure 1. Single-hop precision–coverage. The boundary cut helps even at 90–96% base accuracy.

This zero-cut subset is not just a repackaging of easy high-score examples. The full overlap analysis (Table 3) compares it with matched-coverage and top-decile score thresholds. A raw-score threshold matched to the zero-cut coverage attains only 67.6% precision and overlaps with the zero-cut set on just 14 queries (Jaccard 0.11). At matched coverage, the boundary cut selects 57 correct queries that the raw-score threshold misses.

SubsetSizePrecisionOverlap w/ zero-energy
Zero-energy (E=0E=0)711.000—
Matched-coverage raw (top 71)710.67614 (Jaccard 0.110)
Top 10% raw400.775—
Top 10% smoothed400.825—

Table 3. Zero-energy overlap analysis on multi-hop MAB.

A gamma sweep and a linear-scorer ablation confirm that the abstention rule is not a smoothing artifact: the boundary cut remains the strongest zero-fit selector, and the zero-cut subset is unchanged (Appendix § 17, § 15).

Changing the scorer or encoder changes the retrieval scores, and hence can change path resolution and the continuous edge weights; the abstention rule is then recomputed on the resulting selected-support field and graph. Table 4 reports boundary cut performance under three configurations on the 400-query MAB multi-hop set: empirically, the same 71 queries have a zero boundary cut in all three, and all 71 are correct. This is a structural result about the selected supports and their boundaries, not exact invariance of continuous neural scores. Figure 5 shows the per-context-length aggregate.

Figure 5. AUPCC by context length. Boundary cut vs. the smoothed-score baseline across both regimes. The boundary cut has higher AUPCC at every multi-hop context length and at three of four single-hop context lengths; smoothed score is higher at single-hop 32k.

Native evaluation (score baselines retain their unresolved values): multi-hop smoothed AUPCC 0.677, gap +0.108; single-hop smoothed AUPCC 0.941, gap +0.034. No status-matched single-hop result is available in the frozen tables, so its status-matched entries are marked —; its boundary cut AUPCC is the unchanged native value.

SplitAccuracyCoherenceSmoothedRawMargin
6k0.9100.9800.9280.9140.718
32k0.9000.9360.9510.9500.813
64k0.9600.9770.9400.9280.875
262k0.9000.9710.9100.8980.862
All0.9170.9750.9410.9320.823

Table 2. Per-split single-hop AUPCC (n=100n=100 per split). Smoothed score leads on 32k.

ScorerEncoderddCoh. AUPCCNative smoothedGap
(qTv)2(\mathbf{q}^{\mathsf{T}}\mathbf{v})^2Qwen1280.7850.677+0.108
qTv\mathbf{q}^{\mathsf{T}}\mathbf{v}Qwen1280.7750.690+0.085
(qTv)2(\mathbf{q}^{\mathsf{T}}\mathbf{v})^2MiniLM3840.7880.680+0.108

Table 4. Transfer across scorers and encoders (multi-hop, 400 queries each). Same 71 zero-energy queries, same 100% precision in all configurations.

What produces the zero-cut cases The zero-cut rule identifies an empirically high-precision structural-closure regime. Closure is determined entirely by the upstream resolver and neighborhood construction, without correctness labels. On MAB, this event selects 71 of 400 evaluations, all correct. Thus closure serves as a practical indicator of reliability in this pipeline, although it is not a logical certificate that the returned answer is true.

We audited the topology of the zero-cut neighborhoods. The original run retained per-query scalar results but not graph snapshots, so the audit uses a regeneration with the pinned extraction model (gpt-4.1-mini-2025-04-14) and the original retrieval configuration, which yields 70 zero-cut queries, 67 overlapping the reported 71. None is trivial or disconnected: every regenerated zero-cut neighborhood is a single small connected component with internal edges. The mechanism is hard selection: the typed one-hop neighborhood induced by the resolved path contains only the selected path nodes in all 70 cases, but not in any of the 330 non-zero cases, so the indicator field is constant and its cut boundary is empty. This closure is produced upstream by neighborhood construction and path resolution; the abstention rule detects it, it does not create it. The zeros mean no unselected candidate remains in the constructed local neighborhood—not that no competing evidence exists in the world, and not equality of continuous retrieval scores: a continuous score field over the same neighborhoods yields no exact zeros in 400 queries, and the zero set contains no duplicate candidates. Base accuracy is 56.8% among the 396/400 regenerated queries with a non-trivial connected component (the 4 trivial queries are all incorrect), versus 70/70 in the zero-cut subset, so zero cut marks a high-precision region within the non-trivial population.

Controlling for unresolved retrieval Boundary cut ranks unresolved (NOT_FOUND) queries last through its cut sentinel, whereas native score baselines retain their (overlapping) scores, so the native full-population comparison partly reflects unresolved-status handling. We report three comparisons grouped by exact question (10,000 replicates, unit = exact question text): (A) native full (Table 1), boundary cut 0.785 vs. smoothed 0.677, with Δ=+0.108\Delta= +0.108 (95% CI [+0.062, +0.156]); (B) status-matched full, a post-hoc diagnostic that gives each score baseline the same unresolved-last placement while leaving resolved-row scores untouched, boundary cut 0.785 vs. smoothed 0.736, Δ=+0.049\Delta= +0.049 ([+0.008, +0.092]); and (C) resolved-only (N=320), boundary cut 0.823 vs. smoothed 0.762, Δ=+0.061\Delta= +0.061 ([+0.010, +0.114]). The advantage is not a sentinel artifact: it remains positive with intervals excluding zero under both status matching and resolved-only evaluation. On the regenerated population used for the structural controls and QPP comparison, the same boundary-cut-versus-smoothed difference is +0.027 with a CI including zero (Table 19), so the aggregate score-baseline advantage is population-sensitive; the regenerated analysis retains the closure regime and an advantage over all four QPP adaptations, although QPP was evaluated only on that population. The stricter resolved non-closure analysis below nevertheless removes the raw-boundary advantage.

SignalStatus-matched (primary) AUPCCStatus-matched (primary) AUROCNative AUPCCNative AUROCResolved-only AUPCCResolved-only AUROC
Coherence0.7820.8100.7820.8100.8220.635
Raw score0.7550.8140.7550.8130.7880.643
Smoothed score0.7540.8150.7540.8140.7870.643
Smoothed full-vector margin0.6660.7290.5930.5590.6750.479
NQC-local0.6690.7300.5580.5200.6790.481
WIG-local0.6990.7420.5430.4950.7190.504
σmax⁡\sigma_{\max}0.6610.7190.5470.4930.6690.459
σ50%\sigma_{50\%}0.6760.7260.5650.5080.6880.473
Paired AUPCC differences [95% bootstrap CI grouped by exact question]
Coherence − Raw score0.027 [-0.021, +0.076]0.027 [-0.021, +0.076]0.034 [-0.026, +0.094]
Coherence − Smoothed score0.027 [-0.019, +0.074]0.028 [-0.019, +0.074]0.035 [-0.024, +0.093]
Coherence − Smoothed full-vector margin0.116 [+0.073, +0.158]0.189 [+0.143, +0.234]0.147 [+0.092, +0.198]
Coherence − NQC-local0.113 [+0.065, +0.158]0.224 [+0.169, +0.276]0.143 [+0.083, +0.200]
Coherence − WIG-local0.083 [+0.037, +0.133]0.238 [+0.170, +0.288]0.103 [+0.047, +0.168]
Coherence − σmax⁡\sigma_{\max}0.121 [+0.068, +0.170]0.235 [+0.176, +0.290]0.153 [+0.088, +0.216]
Coherence − σ50%\sigma_{50\%}0.106 [+0.056, +0.153]0.216 [+0.158, +0.270]0.134 [+0.073, +0.193]
Best primary QPPWIG-localσ50%\sigma_{50\%}WIG-local
Best QPP − smoothed-0.055 [-0.114, -0.001]-0.189 [-0.249, -0.128]-0.069 [-0.146, +0.003]

Table 19. Frozen regenerated population: minimum attempted-hop aggregation. NQC and WIG use a local-background score proxy. All scores are recomputed on the same rows.

Several analyses intentionally use different preserved or cached populations; their absolute AUPCC values answer different questions and are not interchangeable with the end-to-end 0.785 headline.

The full-prefix evaluator gives 0.7849 on the same headline run; Table 1’s 50-point evaluator rounds to 0.785.

Structural Controls: Closure, Boundary Topology, and Resolution State

The zero-cut subset coincides exactly with structural closure P=VP = V (§ 5.1). This raises a sharper question: outside that binary closure, does the graded boundary carry selective information, and in particular do the continuous edge magnitudes matter beyond unweighted boundary topology? Holding the constructed graph and the selected support fixed, we compare the weighted boundary cut Ew(P)=∑i∈P, j∉PwijE_w(P) = \sum_{i\in P,\,j\notin P} w_{ij} against structural controls: the unweighted boundary count E1(P)=∣{(i,j):i∈P, j∉P, wij>0}∣E_1(P) = |\{(i,j): i \in P,\,j \notin P,\,w_{ij} > 0\}|, the selected fraction ∣P∣/∣V∣|P|/|V|, the complement size ∣V∖P∣|V \setminus P|, and the binary closure indicator 1[P=V]1[P = V]. Each is scored as a selective-prediction signal with the same AUPCC evaluator. The verified regenerated population contains 374 distinct exact question texts across 400 evaluations; E1 and E1A resample exact-question groups. The analysis runs on the regenerated population of § 5.1 (the only per-query artifact that retains VV, PP, and edge weights), on which the weighted cut reproduces AUPCC 0.7819 (vs. the paper’s 0.785; the −0.0031 gap is the documented 70-vs-71 zero-count difference; Table 5). Full definitions, hypotheses, and protocol are in Appendix D.

AUPCCAnalysisRelation to 0.785
0.785Original end-to-end MAB (pooled 400, d128)headline
0.782Regenerated mechanism audit (400; 70 zero-energy)different regeneration
0.765Graph ablation (cached candidates/path; graph varied)within-table only
0.765–0.766Gamma sweep (separate cached harness; path re-resolved per γ\gamma)within-sweep only
0.750Edge perturbation (own cached seed/protocol)perturbation baseline

Table 5. Provenance of MAB boundary-cut AUPCC values across analyses.

On the full 400-query set (Table 6), the weighted cut (AUPCC 0.7819) is statistically indistinguishable from the unweighted boundary count (0.7828; paired Δ=−0.0009\Delta= -0.0009, 95% CI [−0.004, +0.003]), the selected fraction (0.7832), and the complement size (0.7811). It does materially exceed the binary closure indicator alone (0.7199; Δ=+0.0620\Delta= +0.0620, 95% CI [+0.030, +0.092]): the graded boundary carries information beyond closure, but that information does not require the continuous weights.

Population / signalAUPCC
Full (N=400N = 400)
Weighted cut0.782
Unweighted cut0.783
Resolved non-closure (N=246N = 246)
Weighted cut0.600
Unweighted cut0.610
Boundary-weight fraction0.697
Boundary-edge fraction0.701

Table 6. Boundary-signal AUPCC on the regenerated MAB multi-hop population: full (N=400N = 400) and resolved non-closure (N=246N = 246). Paired intervals in the accompanying analysis resample exact question texts.

Restricting to the 330 queries with P≠VP \ne V removes the exact closure cases (base accuracy 0.4697). The weighted cut retains selective value above base (AUPCC 0.5837, +0.11 over base) but again does not beat the unweighted count (0.5900; paired Δ=−0.0064\Delta= -0.0064, 95% CI [−0.020, +0.026]). This does not eliminate all apparent selective signal, but this experiment alone does not establish why that residual separation occurs: the P≠VP \ne V subset still mixes resolved and unresolved (chain-broken) states, and we do not attribute the residual to boundary topology per se.

These controls establish that the MAB advantage does not depend on continuous Laplacian weights. In this pipeline, inexpensive measures of selected-set topology recover essentially the same selective behavior. The practical implication is that the abstention statistic can be simplified when the graph structure supports it. This is an empirical finding about the MAB graph construction; the external results show that edge weights matter in other regimes (§ 5.3).

A normalized variant (the boundary-weight fraction) separates more strongly, but our preregistered controls did not isolate normalization from the weights themselves, so we develop it as a hypothesis below.

Resolution state: where the residual comes from (this regenerated structural-control population has 84 unresolved, versus 80 in the original headline run; see § 4). Within this regenerated analysis all 84 are incorrect and receive the structural-control family’s worst-confidence sentinel, so every structural-control signal ranks them last and gains an easy failure-state distinction; this concerns the boundary signals of this analysis, not the headline score baselines. Of the 84, 80 carry a partial resolved prefix and 4 have an empty path. Stratifying by resolution state, on the same population and evaluator, isolates the effect.

Restricting to resolved non-closure queries (resolved and P≠VP \ne V; N=246N = 246, base accuracy 0.6301), the raw boundary signals fall to or below base accuracy: the weighted cut reaches AUPCC 0.6000 and the unweighted count 0.6104, both under 0.6301. The apparent raw-boundary advantage outside P=VP = V does not survive this stricter control; much of the earlier residual separation is consistent with resolution-state composition (failure cases ranked last) rather than a graded raw-boundary confidence signal among resolved answers.

Normalizing the boundary by the local graph’s total edge weight or count changes the picture. The weighted normalized boundary (the boundary-weight fraction, AUPCC 0.6968) and its unweighted analogue (the boundary-edge fraction, E1(P)/(total positive edge count)E_1(P)/(\text{total positive edge count}), AUPCC 0.7005) both sit about +0.07 above base accuracy. Weighted normalization improves the point estimate, but its paired interval includes zero; the unweighted normalization interval excludes zero (weighted normalized vs. raw weighted Δ=+0.0968\Delta= +0.0968, CI [−0.0002, +0.196]; unweighted normalized vs. raw unweighted Δ=+0.0901\Delta= +0.0901, CI [+0.007, +0.201]; intervals are wide at N=246N = 246).

Continuous weights again add nothing: the weighted and unweighted normalized fractions are statistically indistinguishable (Δ=−0.0037\Delta= -0.0037, 95% CI [−0.024, +0.016]), the point estimate trivially favoring the unweighted fraction). This generates a narrower, MAB-specific hypothesis, not an established mechanism: relative boundary exposure, the fraction of the local graph that crosses the selected/unselected boundary, rather than raw cut magnitude, may be the useful structural quantity. Because the unweighted normalized boundary was motivated by the same data it is evaluated on, external validation is required before any general claim, which the next subsection supplies.

Cross-Benchmark Test of the Normalization Hypothesis

We test the MAB-specific normalization hypothesis of §4.2 outside MAB, on existing structured-retrieval pipelines with their own graphs, selectors, and correctness definitions (configurations in Table 7). This is an external-validation test, not a further within-MAB optimization.

Benchmarkkkγ\gammaRetrieval / graphReproduction
LongMemEval temporal50.100session chain, continuousreproduced
LongMemEval semantic50.100mutual-kNN(5), continuousreproduced
MetaQA-3hop100.100typed answer-relation + type filterreproduced
CoDEx-S100.100co-occurrence / typed maskexploratory
LoCoMo50.100temporal chainblocked

Table 7. External-benchmark evaluation configurations and reproduction status.

Confirmatory interpretation is restricted to benchmark lanes whose original published result we could reproduce exactly. LongMemEval temporal and semantic both reproduced (boundary cut AUPCC 0.7356 and 0.6818; gate PASS), as did MetaQA-3hop (reproduced boundary cut−raw Δ=+0.0198\Delta= +0.0198 vs. published +0.023; PASS). CoDEx-S failed its reproduction gate (the existing ComplEx training is unseeded and the published initialization is unrecoverable), so it is reported as gate-failed exploratory only (Appendix § 14). The MAB single-hop and LoCoMo lanes were provenance-blocked (Appendix § 14) and are not interpreted.

The normalization improvement seen on MAB does not replicate. On the LongMemEval temporal graph the normalized unweighted boundary is no better than the raw count (AUPCC 0.7363 vs. 0.7451; Δ=−0.0088\Delta= -0.0088, 95% CI [−0.026, +0.015]), and on MetaQA-3hop it is numerically harmful (0.2831 vs. 0.3238; Δ=−0.0407\Delta= -0.0407, CI [−0.095, +0.022], worse on every predefined answer-relation). Across the two confirmatory external families the normalized boundary never beats the raw count (0/2 positive point estimates).

The LongMemEval semantic graph behaves differently. There the raw weighted cut is weak (0.6818) while the raw unweighted count is strong (0.7793), and among the normalized signals the weighted normalized boundary (0.8528) materially exceeds the unweighted normalized boundary (0.7744): Δ=+0.0785\Delta= +0.0785, 95% CI [+0.048, +0.111]. The MAB finding that continuous weights are unnecessary does not hold universally: continuous weights can matter in at least one external graph regime. Task-relevant edge-weight semantics (here, relevance-weighted similarity) is a plausible, regime-specific explanation; this interpretation is exploratory and post-hoc.

No single raw-versus-normalized or weighted-versus-unweighted rule generalized across the evaluated external regimes: normalization helped only on MAB, was null on the temporal graph, harmful on MetaQA, and on the semantic graph the advantage came from weights rather than from normalization. Performance depends on the interaction among candidate-space structure, graph construction, normalization, and edge-weight semantics. The result is a regime-dependent structural diagnostic, not a universal replacement for the raw boundary cut.

Typed Structure, Not Generic Similarity, Drives the Signal

This ablation asks which edge set (graph construction) works, holding candidates and scores fixed; it is distinct from the structural controls of §4.2, which hold the graph fixed and ask whether continuous edge magnitudes matter beyond unweighted boundary topology.

Within this cached ablation (which holds the resolved path PP and scores fixed and rebuilds only the edge weights) the typed graph gives AUPCC 0.765, compared with 0.716 for semantic kNN and 0.748 for the best hybrid construction; full graph-ablation results are in Table 8. The typed graph is also the only variant that preserves the perfect zero-cut subset: semantic kNN produces more zero-cut queries, but seven are wrong, reducing its zero-cut precision to 91.7%.

VariantAgg.6k32k64k262kZero-energy precision
Typed entity–relation0.7650.8660.7430.7540.5921.000
Semantic kNN0.7160.8270.6830.6550.5980.917
Hybrid (α=0.250\alpha= 0.250)0.7450.8550.7220.7230.5670.986
Hybrid (α=0.500\alpha= 0.500)0.7470.8580.7220.7380.5600.986
Hybrid (α=0.750\alpha= 0.750)0.7480.8570.7240.7370.5640.986

Table 8. Full MAB graph ablation on cached local neighborhoods (multi-hop).

The result supports Remark 2.1: generic semantic edges add disagreement without adding correctness information, so the typed mask is valuable because of what it excludes. The boundary cut does not need a particular encoder or scorer, but it does need graph edges aligned with answer support (robustness to edge corruption in Appendix § 16).

Mean boundary cut separates correct from incorrect answers most sharply for the typed graph (mean E=13.25E = 13.25 on correct vs. 29.48 on incorrect), and least for semantic kNN (2.28 vs. 4.02), with the hybrids intermediate (α\alpha=0.25: 5.03 vs. 9.91; α\alpha=0.50: 7.77 vs. 15.79; α\alpha=0.75: 10.51 vs. 21.68).

Support Compatibility Is Relation-Level, Not Dataset-Level

On LongMemEval, LoCoMo, and CoDEx-S, the boundary cut underperforms score baselines at the dataset level (Table 9). Per-relation analysis shows a finer pattern. On CoDEx-S and MetaQA-3hop, each dataset contains both support-compatible and support-incompatible relations, so the aggregate result reflects a mixture rather than a single dataset-level verdict. Neither dataset provides a positive aggregate benchmark for the method; the observed variation is relation-dependent. The RmedianR_{\mathrm{median}} gate correctly rejects both at the dataset level (Appendix K). Relation-level behavior is heterogeneous and we treat it as exploratory: on the reproduced MetaQA evaluation only in_language has a paired bootstrap interval excluding zero, and the CoDEx-S per-relation results are artifact-conditional (their exact historical per-example predictions were not persisted). Illustrative relation-level results in both directions, with paired bootstrap intervals, are in Table 11 and Table 12. This secondary diagnostic consumes correctness labels; its threshold was chosen on the original evaluated families, not independently validated as a general predictive criterion.

Benchmark / graphCoherenceBest comparatorDiagnosis
LongMemEval / temporal0.7360.887temporal adjacency
LongMemEval / semantic kNN0.6820.890generic similarity
LoCoMo / temporal0.2860.430temporal adjacency
CoDEx-S / typed entity–relation0.3730.543heterogeneous, App. L

Table 9. Boundary-condition experiments. The boundary cut fails when the graph does not encode support-compatible structure.

PIDLabelNNAccAUPCCcoh_{\mathrm{coh}}AUPCCraw_{\mathrm{raw}}RmedR_{\mathrm{med}}ZE precZE nn
P101field of work200.8000.7660.8961.400—0
P463member of2700.5800.7000.62452.0000.76489
P1303instrument710.5600.5770.7191.4700.60020
P27citizenship1050.6000.4850.7950.6900.0002
P530diplomatic relation2870.2400.2560.390N/A0.244287
P106occupation5930.2500.2150.3130.8801.0001
P136genre1020.2200.1780.2680.7500.0004

Table 11. Per-relation boundary-cut performance on CoDEx-S with typed entity–relation mask. RmedianR_{\mathrm{median}}: ratio of median cut on incorrect vs. correct answers (reported as N/A when both medians are zero). Relations ordered by boundary-cut AUPCC. This secondary diagnostic consumes correctness labels; its threshold was chosen on the original evaluated families, not independently validated as a general predictive criterion.

Answer relationNNBase accAUPCCcoh_{\mathrm{coh}}AUPCCraw_{\mathrm{raw}}Δ\Delta (95% CI)
in_language460.4350.5460.317+0.230 [+0.030, +0.407]
release_year410.3410.3670.306+0.062 [−0.120, +0.246]
has_genre560.3040.3520.326+0.026 [−0.128, +0.157]
starred_actors310.2260.1680.141+0.027 [−0.137, +0.192]
directed_by390.2820.2870.333−0.046 [−0.292, +0.160]
written_by370.2970.2090.271−0.062 [−0.256, +0.137]

Table 12. Illustrative relation-level results on MetaQA-3hop (n=250n=250 stratified), boundary cut vs. raw score, recomputed from the persisted per-query evaluation artifact. Both directions shown; selection is illustrative, not statistically representative. Intervals are paired bootstrap 95% CIs using the per-query sampling unit (10,000 replicates, seed 42), descriptive and uncorrected for multiple comparisons; small-NN relations have wide intervals. Only in_language has an interval excluding zero.

The negative-transfer results clarify the conditions under which graph structure provides useful confidence information. In these experiments, misaligned graph edges yield weak or misleading signals, while the positive MAB result is associated with edges that connect candidates supporting the same answer.

As a practical diagnostic, we compute three cheap metrics across six graph families to characterize when the boundary cut is useful (Table 10): RmedianR_{\mathrm{median}}, the ratio of median cut on wrong vs. correct answers (higher means better separation); AUROC of −E-E as a correctness discriminator; and AUPCC. This probe is not a prospective predictor of when the boundary cut will work (it requires ground-truth labels), but it provides a useful post-hoc sanity check for new graph constructions. This secondary diagnostic consumes correctness labels; its threshold was chosen on the original evaluated families, not independently validated as a general predictive criterion.

Graph familyNNAccRmedianR_{\mathrm{median}}AUROCAUPCCZE prec
MAB typed4000.5608.1500.7550.7651.000
MAB semantic kNN4000.5602.7500.6920.7160.917
LongMemEval temp.5000.7401.0100.4880.736—
LongMemEval kNN5000.7500.5100.4140.682—
LoCoMo temporal19860.2701.0100.5170.286—
CoDEx-S co-occ.18280.3300.6700.3320.2150.083

Table 10. Support-compatibility probe across six graph families. High RmedianR_{\mathrm{median}} and AUROC describe observed correctness separation; values near 1.0 or below 0.5 indicate negative transfer. This secondary diagnostic consumes correctness labels; its threshold was chosen on the original evaluated families, not independently validated as a general predictive criterion. ZE denotes zero-energy.

The pattern is clear: MAB’s typed entity–relation graph produces large cut separation (RmedianR_{\mathrm{median}}=8.15, AUROC 0.755) and a perfect zero-cut subset. Temporal adjacency produces RmedianR_{\mathrm{median}}≈1 (no separation). Semantic kNN and co-occurrence show intermediate or inverted ratios. These results are consistent with the support-compatibility hypothesis: the graph must connect candidates that should agree under the current query. This secondary diagnostic consumes correctness labels; its threshold was chosen on the original evaluated families, not independently validated as a general predictive criterion.

Note: For P530, the typed mask produces disconnected local subgraphs, so zero cut is structural rather than informative.

The pattern tracks candidate-space structure: closed-vocabulary or structurally dense answer types (organizational membership, languages, release years) yield small, connected typed neighborhoods in which incorrect answers cross a non-empty boundary, whereas open-vocabulary or sparse types (person names; arbitrary-tail relations such as P530) yield high-cardinality, sparse or disconnected neighborhoods where the cut carries little information. On MetaQA the answer-type candidate filter performs most of this narrowing and the typed edge mask is nearly a no-op. Table 11 and Table 12 give the per-relation results; Appendix § 20 details the mechanism and candidate-filter versus edge-mask comparison.

Comparison with Learned Confidence

A shallow gradient-boosted tree ensemble (GBT; depth 3, 50 trees) trained via five-fold CV grouped by exact question on five retrieval-time features achieves AUPCC 0.814, versus the boundary cut’s 0.785 (Table 13). Under the reconstructed mapping (Appendix § 21), the main native-order, training-fold-standardized protocol gives boundary cut minus GBT Δ=−0.029\Delta= -0.029 (95% CI [−0.062-0.062, +0.005+0.005]), so the difference is not statistically resolved. L2-regularized logistic regression achieves 0.648. A separate native-order unscaled sensitivity yields GBT AUPCC 0.812 and boundary cut minus GBT Δ=−0.027\Delta= -0.027 (95% CI [−0.060-0.060, +0.008+0.008]). This sensitivity is distinct from the main estimate; both protocols and their grouping history are documented in Appendix § 21.

Label-efficiency table Table 14 reports GBT performance trained on subsamples of training data (1%–100%) within each CV fold grouped by exact question, alongside the constant zero-fit boundary cut baseline. GBT first exceeds the boundary cut by point estimate at 20% labels; the paired aggregate intervals include zero at 20%, 50%, and 100%. Figure 4 plots the aggregate curve and the 262k point estimates. All headline-derived results use the reconstructed mapping (Appendix § 21).

Labels (%)GBT (all)Coherence (all)GBT (262k)Coherence (262k)
10.6130.7850.4300.661
50.6650.7850.5390.661
100.7510.7850.6310.661
200.7900.7850.6600.661
500.7860.7850.6600.661
1000.8140.7850.6560.661

Table 14. Label-budget comparison on headline MAB multi-hop (N=400N = 400 overall; N=100N = 100 at 262k). Budgets are percentages of training rows within folds grouped by exact question; paired intervals resample exact question texts. Question identity uses the reconstructed mapping (Appendix § 21).

Figure 4. Label-efficiency curve on multi-hop MAB. Boundary cut (zero-fit, horizontal line) vs. GBT trained on subsampled labels. Top: aggregate AUPCC. Bottom: 262k point estimates. GBT first exceeds the boundary cut in aggregate by point estimate at 20% labels; paired intervals include zero at 20%, 50%, and 100%. Question identity uses the reconstructed mapping (Appendix § 21).

AUROC and cross-scale generalization details are in Appendix § 11.

Setup and maintenance tradeoff Supervised selection attains a higher in-distribution point estimate when correctness labels and suitable confidence features are available, though the paired difference is not statistically resolved on 400 queries. Graph-boundary abstention offers a different operating point: a training-free rule computed from deployment-time evidence structure, without maintaining a separate learned confidence model. We test how the GBT point-estimate comparison changes with the retrieval stack. We hold the 400 multi-hop MAB queries fixed and use three configurations: BASE (Qwen, (qTv)2(q^{\mathsf{T}}v)^2), MINILM (MiniLM, (qTv)2(q^{\mathsf{T}}v)^2; encoder shift), and LINEAR (Qwen, qTvq^{\mathsf{T}}v; scorer shift). For each (source, target) pair we train GBT on source-config features+labels under 5-fold CV grouped by exact question text, and evaluate on target-config features against target-config labels (no query overlap, no target labels used for fitting). Tables 15 and 16 give the transfer matrix and per-cell bootstrap confidence intervals; Appendix § 21 gives the AUROC matrix and feature-scale distributions.

Under cross-validation grouped by exact question with the reconstructed mapping (Appendix § 21) (Table 16), the MINILM and LINEAR matched-stack comparisons favor GBT, while the BASE matched-stack difference is unresolved. Among off-diagonal cells, BASE→LINEAR favors GBT; MINILM→BASE and LINEAR→MINILM favor the boundary cut; the remaining three differences are unresolved. In particular, BASE→BASE\mathrm{BASE}\to\mathrm{BASE} has a 95% CI of [−0.0000, +0.065], MINILM→BASE\mathrm{MINILM}\to\mathrm{BASE} [−0.097, −0.012], MINILM→MINILM\mathrm{MINILM}\to\mathrm{MINILM} [+0.005, +0.073], and LINEAR→BASE\mathrm{LINEAR}\to\mathrm{BASE} [−0.084, +0.004]. Feature-scale shift remains a plausible explanation for transfer degradation.

CellGBTCoherenceΔ\Delta95% CIDirection
BASE→BASE\mathrm{BASE}\to\mathrm{BASE}0.8180.7850.033[-0.0000, +0.065]unresolved
BASE→MINILM\mathrm{BASE}\to\mathrm{MINILM}0.7710.788-0.017[-0.050, +0.024]unresolved
BASE→LINEAR\mathrm{BASE}\to\mathrm{LINEAR}0.8250.7750.050[+0.016, +0.087]positive
MINILM→BASE\mathrm{MINILM}\to\mathrm{BASE}0.7390.785-0.046[-0.097, -0.012]negative
MINILM→MINILM\mathrm{MINILM}\to\mathrm{MINILM}0.8280.7880.041[+0.005, +0.073]positive
MINILM→LINEAR\mathrm{MINILM}\to\mathrm{LINEAR}0.7630.775-0.012[-0.059, +0.029]unresolved
LINEAR→BASE\mathrm{LINEAR}\to\mathrm{BASE}0.7420.785-0.043[-0.084, +0.004]unresolved
LINEAR→MINILM\mathrm{LINEAR}\to\mathrm{MINILM}0.6960.788-0.092[-0.132, -0.034]negative
LINEAR→LINEAR\mathrm{LINEAR}\to\mathrm{LINEAR}0.8380.7750.063[+0.031, +0.096]positive

Table 16.

Table 15: GBT transfer AUPCC across retrieval configurations on headline MAB multi-hop (N=400N = 400 per target). Rows identify training configurations and columns deployment configurations. Folds and paired resampling use exact question texts under the reconstructed mapping (Appendix § 21).

Train / testBASEMINILMLINEAR
BASE0.8180.7710.825
MINILM0.7390.8280.763
LINEAR0.7420.6960.838

Table 15.

Bootstrap 95% confidence intervals (10,000 resamples grouped by exact question) Table 16 compares each GBT cell against the target-configuration boundary cut using 10,000 paired exact-question bootstrap replicates, seed 42, conditional on the fixed out-of-fold predictions. The archived correction used 2,000 replicates despite the earlier manuscript’s 10,000 description; the repaired intervals use 10,000 throughout.

Table 16: Paired AUPCC differences between transferred GBT and the target-configuration boundary cut on headline MAB multi-hop (N=400N = 400 per target). Intervals use 10,000 exact-question bootstrap replicates, conditional on out-of-fold predictions, under the reconstructed mapping (Appendix § 21).

Values are rounded to three decimals; a fourth decimal preserves the sign of near-zero CI endpoints. The displayed −0.0000 is negative at full precision, so that interval includes zero. Significance is determined before rounding.

A reconstructed post-hoc population-rank diagnostic substantially mitigates transfer degradation: LINEAR→MINILM\mathrm{LINEAR}\to\mathrm{MINILM} increases from 0.696 to 0.771. This result concerns ranks across queries within each configuration, not within-query candidate ranks. The boundary cut is recomputed on the deployment path and remains perfectly preserved under encoder shift (BASE↔MINILM\mathrm{BASE}\leftrightarrow\mathrm{MINILM} Pearson ρ=1.000\rho= 1.000) and partially preserved under scorer shift (ρ=0.41\rho= 0.41).

Reconstructed post-hoc percentile-rank diagnostic Table 17 reports the archived population-rank transformation under repaired exact-question folds and the reconstructed mapping (Appendix § 21). Each feature column is ranked across all 400 rows within its configuration before cross-validation, including held-out rows, with positional tie breaks. This is a transductive post-hoc diagnostic. Eight of nine cells reach or exceed the target-configuration boundary cut; the exception is LINEAR→MINILM\mathrm{LINEAR}\to\mathrm{MINILM} (0.771 versus 0.788). Literal within-query candidate percentiles are unavailable because the headline features omit candidate-score vectors. The eight-of-nine population-rank result therefore does not establish a within-query normalization claim.

Train / testBASEMINILMLINEAR
BASE0.8270.8050.824
MINILM0.8080.8200.792
LINEAR0.8080.7710.831

Table 17. GBT transfer AUPCC after transductive population-rank normalization on headline MAB multi-hop (NN = 400 per configuration). Exact-question folds and paired resampling use the reconstructed mapping (Appendix § 21). Feature ranks include held-out rows; this diagnostic is not within-query normalization.

Search baselines We evaluated search methods only on multi-hop 6k (100 queries; Table 18; results at other scales may differ.

MethodAccuracyΔ\Delta
Baseline0.660—
Leaf MCTS0.660+0.000
Chain MCTS0.580−0.080
Path reranking0.580−0.080

Table 18. Search baselines on multi-hop 6k. Search does not improve accuracy in this setting; chain methods hurt.

In this setting, the dominant failure mode is error compounding: an incorrect first hop produces a corrupted query that drives later hops into an incorrect subtree with high confidence. The first hop is not the bottleneck (98/100 queries have a unique first-hop candidate, so first-hop reranking can affect at most 2% of queries); the path-reranking and chain-MCTS lanes lose 8 points because their chain-aware greedy descent at subsequent hops follows a cumulative-log-score signal that prefers locally-plausible but globally-incorrect paths over the deterministic smoothed-lookup choice. When the Transductive per-feature ranks across 400 rows per configuration. This does not implement within-query candidate ranks.

failure mode is local scoring ambiguity rather than incomplete exploration, abstention is the more effective use of post-retrieval compute.

Figure 2. Multi-hop precision–coverage (6k/32k/64k/262k). The boundary cut (red) attains the highest AUPCC on all four multi-hop context lengths.

Figure 3. Graph-representation ablation on cached multi-hop MAB neighborhoods. Results are computed post hoc on cached local neighborhoods and are not directly comparable to the end-to-end AUPCC values in Table 1 and Table 13. Typed entity–relation structure outperforms semantic kNN and hybrid alternatives. The typed graph is the only variant with a perfect zero-cut subset (71 queries, 100% precision). Adding semantic edges dilutes the signal, consistent with Remark 2.1.

Score-distribution QPP comparison Table 19 reports the Stage 3 comparison on the regenerated population, using minimum attempted-hop aggregation. NQC-local, WIG-local, σmax⁡\sigma_{\max}, and σ50%\sigma_{50\%} are local score-distribution adaptations, not corpus-score QPP evaluations. The boundary cut exceeds all four in AUPCC, with 95% CIs grouped by exact question excluding zero in the status-matched, native, and resolved-only settings. All four adaptations underperform the paper’s smoothed baseline by point estimate on these same rows. The best-QPP-minus-smoothed interval excludes zero in the primary status-matched and native settings, but includes zero in resolved-only evaluation; best-QPP selection is post hoc. The boundary cut’s difference from smoothed is unresolved on this regenerated population. Formulas, provenance, and final-hop sensitivity results are available from the author.

N=400N = 400 for native/status-matched and N=316N = 316 for resolved-only. CIs use 10,000 paired bootstrap draws, seed 42, unit exact question text (verified from the frozen intermediate bundle). Best-QPP selection is post hoc; its intervals are descriptive. Smoothed scores use (I+0.1L)−1b(I + 0.1L)^{-1}b at the frozen selected candidate. The margin uses the two highest smoothed scores over the full local vector. No historical headline baseline values are imported.

Related Work

Predicting retrieval quality without relevance judgments is the subject of query performance prediction (QPP). Post-retrieval predictors have relied on the distribution of retrieval scores or on the coherence of the top-ranked documents, and score-based predictors remain competitive with supervised ones [5, 6, 21, 22, 23, 25, 29, 34]. Diaz [7] predicts performance from the autocorrelation of scores across similar documents, the nearest precedent for graph-smoothness signals over retrieved items. QPP has also been extended to conversational search and to retrieval-augmented generation [9, 14, 20], and to dense retrieval under corpus growth [11]. Abstention has been studied for reranking in black-box settings [12], and calibration for knowledge-graph RAG has been proposed because such systems are often overconfident when retrieved subgraphs are incomplete [19]. Several QPP methods are also unsupervised. The distinction studied here is the boundary between evidence selected to support a returned answer and unselected candidates in a typed evidence graph, used as an answer-abstention signal. The name is unrelated to coherence-based QPP, which measures similarity within the top-ranked set.

Agent memory work [2, 4, 27, 28, 31, 32] organizes and retrieves long-term memory; we study when a retrieved answer should be trusted. Selective prediction [8, 10] uses pointwise signals (softmax, margin); the boundary cut is structural, using graph-neighbor agreement. The closest precedents are neighbor-agreement confidence methods: Trust Score [15] and deep-kNN confidence [18] trust a prediction when its metric neighborhood, estimated from a labeled reference set, agrees with it. The boundary cut shares this intuition but uses a typed evidence graph and no labels; the semantic-kNN ablation (§4.4) shows that generic metric neighborhoods do not reproduce the typed-graph signal. Conformal prediction [1, 26, 33] provides distribution-free coverage via calibration; the boundary cut is training-free and could serve as a nonconformity score within a conformal pipeline. Laplacian regularization is classical in semi-supervised learning [3, 35]. Test-time compute work [24] shows abstention-guided selection outperforms extended search; we ask the analogous question for structured retrieval.

Spectral-summary work [16, 17] uses weight-matrix spectra as data-free quality proxies; our signal is a Laplacian quadratic form over an evidence graph, with the support-compatibility view of § 3 clarifying when it should work.

Limitations

Single positive benchmark family All positive aggregate results use MAB FactConsolidation. (Per-relation lifts on CoDEx-S and MetaQA-3hop are exploratory; § 5.5.) We treat support compatibility as a deployment condition, not as a universal property of structured retrieval benchmarks.

Breadth of external validation The confirmatory cross-benchmark test (§ 5.3, Appendix § 14) spans only two independent external benchmark families, LongMemEval and MetaQA; the LongMemEval temporal and semantic lanes are two graph regimes within one family, not two independent families. The CoDEx-S lane failed its reproduction gate (unseeded upstream ComplEx training) and is exploratory only; the MAB single-hop and LoCoMo lanes were provenance-blocked. Conclusions about cross-benchmark (non-)generalization should be read against this limited external sample. The reported analyses use multiple preserved evaluation populations, whose provenance is distinguished in Table 5. Some historical extraction and model states are unavailable, limiting exact reconstruction of those runs.

Learned confidence The learned selector attains a higher point estimate, not statistically resolved in-distribution (GBT 0.814 vs. boundary cut 0.785 with full in-distribution labels; § 5.6); portability across retrieval stacks is mixed (Table 16). Percentile-rank features substantially mitigate but do not eliminate cross-stack degradation. Two points are specific to this limitation. First, after purging repeated question texts from training, GBT has the higher point estimate on three of four held-out scales (Appendix § 11). Second, a post-hoc complementarity check found no statistically resolved evidence that adding the boundary cut to the GBT’s five score-derived features improves out-of-fold AUPCC (Δ=+0.005\Delta= +0.005, 95% CI grouped by exact question [−0.005,+0.016][-0.005, +0.016]); the two methods differ in that the GBT requires correctness labels while the boundary cut does not.

Support-compatibility boundary Negative results on LongMemEval, LoCoMo, and CoDEx-S co-occurrence show that the boundary cut is not a generic graph confidence criterion: temporal adjacency, semantic kNN, and raw co-occurrence fail. The 50-example Rmedian>2R_{\mathrm{median}} > 2 probe matches the observed usefulness labels 95% of the time across eight families, including two prospectively-probed rejections with within-dataset heterogeneity (Appendices K, § 20). The gate is a practical heuristic, not a validated predictor, and it requires labels. Label-free score computation should therefore be distinguished from validating whether a graph construction is suitable for deployment. Candidate-space cardinality, answer-space closedness, and typed-mask density are computable at deployment without labels and serve only as heuristic diagnostics (the closed-vs-open-vocabulary mechanism and the MetaQA candidate-filter/typed-mask substitution are detailed in § 5.5). This secondary diagnostic consumes correctness labels; its threshold was chosen on the original evaluation families, not independently validated as a general predictive criterion.

Other limitations and broader impact The mask depends on upstream entity extraction, and the perfect zero-set precision is a clean-extraction property: under entity-fragmentation noise on the 6k split (three seeds per rate), zero-set precision degrades from 1.000 to 0.907, 0.874, and 0.715 at 5%, 10%, and 20% corrupted mentions, while retaining a +0.25–0.31 precision lift over base accuracy at every rate. Structural closure (P=VP = V) is pipeline-dependent: it reflects the specific extraction, candidate-neighborhood construction, and resolution procedure, and may change under different such procedures. The boundary cut does not improve base retrieval accuracy; the zero-cut subset covers only 17.75% of multi-hop queries; the search comparison is limited to multi-hop 6k (Table 18); and the method applies to retrieved-item settings, not free-form text. Because the cut and zero-cut decisions are deterministic functions of the selected support and the graph weights (retrieval scores enter upstream through path resolution), the boundary cut improves auditability without additional model inference, but abstention criteria should not be overtrusted. Deployment also requires candidate-level retrieval scores, a graph constructor that produces support-compatible edges, and a downstream policy for abstained queries such as re-retrieval, escalation, or clarification.

Conclusion

We study when typed evidence-graph structure predicts retrieval reliability and present a training-free abstention rule on selected-support boundaries. On MAB multi-hop, the rule identifies an empirically precise structural-closure regime produced by the upstream resolver. The decomposition separates closure, resolution state, topology, and edge weights: continuous weights add little on MAB, and the apparent raw-boundary signal outside closure largely reflects resolution-state composition. Cross-benchmark and relation-level analyses identify characteristics associated with useful abstention rather than a validated predictive criterion; the MAB normalization finding does not replicate externally. The status-matched score-baseline advantage is population-sensitive, while the regenerated population retains the closure regime and an advantage over the local QPP adaptations. GBT has a higher in-distribution point estimate with an unresolved paired difference, and transfer across retrieval stacks is mixed. These findings give an explicit abstention method and empirical guidance on its applicability and deployment tradeoffs without requiring correctness labels for the structural score. The central lesson is that the reliability of graph-based confidence depends on what the graph represents, not merely on its connectivity.

Per-Split Results

The tables and figure below summarize the per-split results.

SplitAccuracyCoherenceSmoothedRawMargin
6k0.7000.8610.7400.7300.670
32k0.5600.7670.6210.5990.554
64k0.5900.7710.7620.7360.672
262k0.4200.6610.4950.4680.355
All0.5680.7850.6770.6530.547

Table 20. Per-split multi-hop AUPCC (n=100n=100 per split).

SplitQwen d=128d=128MiniLM d=384d=384Winner
6k0.8610.855Qwen (+0.006)
32k0.7670.769MiniLM (+0.002)
64k0.7710.776MiniLM (+0.005)
262k0.6610.669MiniLM (+0.008)

Table 21. Per-split boundary-cut AUPCC by encoder (multi-hop, scorer held fixed at (q⊤v)2(\mathbf{q}^{\top}\mathbf{v})^2).

Scope of the Continuous Latent-Class Model

This appendix develops the continuous latent-class model motivated in § 3.4 and clarifies its scope. The model is qualitative intuition for support-compatible graph construction only, and it is not instantiated in the reported experiments.

Latent-class decomposition The abstention properties are deterministic and apply to the weights wijw_{ij} of Eq. (1), whether they are constructed directly or derived from raw graph weights sijs_{ij} through a support gate wij=sijm(q,oi,oj)w_{ij}=s_{ij}m(q,o_i,o_j); the energy only sees the final weights. To connect this to empirical behavior, consider a simple latent-class model. Suppose each candidate belongs to an answer-support class yiy_i, and its score is

zi=μyi+εi,(2)z_i = \mu_{y_i} + \varepsilon_i, \tag*{(2)}

where μyi\mu_{y_i} is the class-level support and εi\varepsilon_i is local noise. Then

E(z)=∑(i,j):yi=yjwij(εi−εj)2+∑(i,j):yi≠yjwij(μyi−μyj+εi−εj)2.(3)E(\mathbf{z}) = \sum_{(i,j):y_i=y_j} w_{ij}(\varepsilon_i-\varepsilon_j)^2 + \sum_{(i,j):y_i\ne y_j} w_{ij}(\mu_{y_i}-\mu_{y_j}+\varepsilon_i-\varepsilon_j)^2. \tag*{(3)}

The first term is score variation within the same support class. The second term is score variation across different support classes. A support-compatible graph constructor keeps little abstention weight in the second term: it assigns large weights mostly to candidates whose support scores should agree. A temporal graph, co-occurrence graph, or generic similarity graph may instead assign large weights across support classes, making the energy less informative for correctness.

Continuous-field concentration (intuition only) The latent-class model is stated for a continuous score field. On a connected component, zero score-field energy forces all observed scores to be equal; if that component contains candidates from two support classes separated by Δ=∣μa−μb∣\Delta= \lvert\mu_a-\mu_b\rvert, the noise must cancel the class-level gap. For independent σ\sigma-sub-Gaussian noise,

Pr⁡(∣εi−εj∣≥Δ)≤2exp⁡(−Δ24σ2).\Pr(\lvert\varepsilon_i-\varepsilon_j\rvert\ge\Delta) \le2\exp\left(-\frac{\Delta^2}{4\sigma^2}\right).

We use this only as intuition for the generic continuous-field model. Real retrieval scores are correlated through the shared encoder, scorer, and graph construction, so the independence assumption does not hold.

Why the model does not explain the experiments Every reported boundary cut result verifies a binary selected-support indicator (§ 4), so no reported energy is a continuous-field energy generated under Eq. (2): we do not use the reported energies to estimate the model’s parameters, and we do not use the model to explain the MAB zero-cut precision. For the selected-support indicator the relevant quantity is instead the boundary cut of Eq. (1): what matters is which edges cross from the selected support to its complement. The latent-class partition is not, in general, the same partition as path membership (an unselected candidate can share the answer-support class of a selected one), so Eq. (3) motivates graph construction rather than explains the experimental zero set. The observed MAB mechanism is instead deterministic and structural: in the regenerated population every zero-cut case satisfies P=VP = V, so the indicator field is constant and its positive-weight boundary cut is empty (§ 5.1), and Remark 2.1 requires no concentration argument.

We therefore retain the latent-class model only as motivation for support-compatible graph construction (a constructor should avoid placing weight on support-incompatible pairs), not as an account of the experimental zero-cut subset. The paper does not claim any population-level probability guarantee.

Learned-Confidence Comparison: Extended Analysis

Per-split bootstrap On the 262k split, boundary cut minus main-protocol GBT is Δ=+0.005\Delta= +0.005 (95% exact-question bootstrap CI [−0.073-0.073, +0.085+0.085]), so the difference remains unresolved. The score ratio has mean impurity importance 0.691 across the five corrected GBT folds. These results use the reconstructed mapping (Appendix § 21).

AUROC On the same 400-query multi-hop set, AUROC is higher for GBT in aggregate (0.889 vs. 0.807), reflecting that the boundary cut is a precision-oriented selector strongest in the high-precision, low-coverage regime emphasized by AUPCC; AUROC weights all operating points equally.

Cross-scale generalization Training on the three shorter scales and testing on 262k, after purging 2 training rows whose question texts appear in the test set, yields GBT AUPCC 0.673 versus the boundary cut’s 0.661. The corresponding within-distribution 262k GBT estimate is 0.656. The 6k, 32k, and 64k held-out evaluations purge 21, 18, and 11 training rows, respectively. After these purges, GBT exceeds the boundary cut by point estimate on three of four held-out scales. All four paired intervals include zero. Details of the reconstructed mapping (Appendix § 21) and training protocol are available from the author.

Table 22: Question-disjoint leave-one-scale-out evaluation on headline MAB multi-hop (N=100N = 100 test rows per scale). Training rows sharing test-question text are purged. GBT-minus-boundary-cut intervals resample exact question texts under the reconstructed mapping (Appendix § 21).

Test scalePurgedGBTCoherenceΔ\Delta95% CI
6k210.8590.861-0.001[-0.077, +0.067]
32k180.7940.7670.027[-0.047, +0.095]
64k110.7950.7710.024[-0.057, +0.109]
262k20.6730.6610.012[-0.075, +0.094]

Table 22.

Table 23: Mean GBT impurity importance across five exact-question folds of headline MAB multi-hop (N=400N = 400), using the reconstructed mapping (Appendix § 21). These are descriptive fold means, without bootstrap resampling.

FeatureImportance
Raw score0.058
Smoothed score0.062
Score margin0.139
Second-highest score0.050
Score ratio0.691

Table 23.

Structural-Control Decomposition

This appendix reports the preregistered structural-control decomposition (E1) summarized in § 5.2. The specification was frozen before any comparative control outcome was computed.

Question Does the boundary cut contain selective-prediction information beyond structural closure P=VP = V and simple graph-topology controls (boundary existence, boundary-edge count, selected fraction, complement size)?

Population and provenance The original graph-ablation cache no longer exists on disk; the only per-query artifact retaining VV, PP, edges, and weights is the regenerated zero-cut audit published in § 5.1 (pinned extraction model gpt-4.1-mini-2025-04-14; N=400N = 400; 316 resolved, 84 unresolved chain-broken queries, all incorrect, held at worst-confidence sentinels mirroring the boundary cut’s convention; 70 zero-cut, 67/71 overlap with the original). This regenerated population resolves 316 of 400 queries, whereas the original headline run resolves 320; the four-query difference is a property of the regenerated extraction/resolution artifact (a different pinned extraction pass), not an inconsistency in any headline number. The AUPCC evaluator is unchanged; the paired bootstrap now groups by verified exact question text. Reproduction gate: the weighted cut reproduces AUPCC 0.7819 on this population (vs. the paper’s 0.785; −0.0031, the documented 70-vs-71 difference), with per-query parity of recomputed vs. recorded energies to 5.7×10−145.7 \times10^{-14}.

Signals S0S_0 weighted cut Ew(P)=∑i∈P, j∉PwijE_w(P) = \sum_{i \in P,\,j \notin P} w_{ij}; S1S_1 closure indicator 1[P=V]\mathbf{1}[P = V]; S2S_2 boundary existence; S3S_3 unweighted boundary count E1(P)E_1(P); S4S_4 selected fraction ∣P∣/∣V∣|P|/|V|; S5S_5 complement size ∣V∖P∣|V \setminus P|; S6S_6 boundary-weight fraction (cut normalized by total neighborhood edge weight). Unresolved queries receive, for every signal, a value strictly less confident than any resolved value.

Populations and protocol PA1: full N=400N = 400. PA2a (primary, literal P≠VP \ne V): N=330N = 330, base accuracy 0.4697. PA2b (secondary, resolved ∧P≠V\land P \ne V): N=246N = 246, base accuracy 0.6301. The paired bootstrap resamples question groups (cross-scale instances of a question together), 10,000 replicates, seed 42; Δ=AUPCC⁡(S0)−AUPCC⁡(control)\Delta= \operatorname{AUPCC}(S_0) - \operatorname{AUPCC}(\text{control}). The structural-control estimates are consolidated in Table 24.

PopulationNNAccuracyWeighted cutUnweighted cutBoundary-weight fractionBoundary-edge fraction
Full4000.5620.7820.7830.8130.813
Resolved3160.7120.8220.8190.8610.861
Non-closure3300.4700.5840.5900.6570.662
Resolved non-closure2460.6300.6000.6100.6970.701
Additional controlFull (N=400N = 400)Non-closure (N=330N = 330)
Selected fraction0.7830.579
Complement size0.7810.561
Closure indicator 1[P=V]1[P = V]0.7200.476

Table 24. Structural-control AUPCC on regenerated MAB multi-hop: full (N=400N = 400), resolved (N=316N = 316), non-closure (N=330N = 330), and resolved non-closure (N=246N = 246). The second panel retains the additional full and non-closure controls. Paired intervals resample exact question texts.

Preregistered hypotheses

  • H1 (SUPPORTED). Full set: AUPCC(S0S_0) = 0.7819 vs. AUPCC(S1S_1) = 0.7199; Δ\Delta = +0.0620, 95% CI [+0.030, +0.092], meeting the ≥+0.02\ge+0.02 rule with the CI excluding 0. The graded boundary carries information beyond the binary closure.

  • H2 (SUPPORTED). P≠VP \ne V (PA2a): AUPCC(S0S_0) = 0.5837 vs. base 0.4697; margin +0.114 ≥+0.05\ge+0.05. Boundary information survives removal of the closure cases.

  • H3 (NOT SUPPORTED). P≠VP \ne V (PA2a): Δ(S0−S3)\Delta(S_0 - S_3) = −0.0064, 95% CI [−0.020, +0.026]; the point estimate is ≤0\le0, failing the preregistered rule. The weighted cut does not outperform the unweighted boundary count.

The overall verdict is partial support: the graded signal is not purely a closure artifact (H1, H2), but the continuous edge magnitudes do not add value over unweighted boundary topology (H3), and the selected fraction and complement size reproduce the weighted cut’s performance nearly everywhere. E1 is specific to MAB and does not speak to generalization; it also does not establish the source of the residual P≠VP \ne V separation, which the mixed resolution states of that subset leave open.

Post-hoc observation (not confirmatory) The boundary-weight fraction S6S_6 (cut normalized by total neighborhood edge weight) numerically exceeds every other signal on every population (full 0.8131; P≠VP \ne V 0.6575) and has a higher point estimate than the raw weighted cut on PA2a, but the paired interval includes zero (Δ=−0.0738\Delta= -0.0738, 95% CI [−0.146, +0.001]). E1 preregistered no test isolating whether this advantage comes from the weights or from the normalization structure, so it is reported only as a hypothesis-generating observation.

Resolution-State and Normalization Diagnostic (E1A)

This appendix reports the preregistered resolution-state and normalization diagnostic (E1A) summarized in §4.2. Its specification was frozen (2026-08-13) before any E1A comparative outcome was computed. E1A reuses the E1 per-query artifact and evaluator unchanged; a parity gate reproduced the E1 values exactly (full weighted cut 0.7819; P≠VP \ne V weighted 0.5837, unweighted 0.5900, boundary-weight fraction 0.6575). No LLM/API calls; no artifacts regenerated.

Question How much of E1’s non-closure separation survives conditioning on resolution state, and does E1’s post-hoc S6S_{6} advantage come from the continuous weights or from normalization itself (tested by the unweighted analogue S8S_{8} = boundary-edge count / total edge count)?

Populations F (all, N=400N = 400); R (resolved, N=316N = 316); N (literal P≠VP \ne V, N=330N = 330); RN (primary: resolved ∧ P≠VP \ne V, N=246N = 246, base accuracy 0.6301); U (unresolved, N=84N = 84, descriptive only: all incorrect, so any AUPCC on U is degenerate).

Unresolved-query audit All 84 unresolved queries lie in P≠VP \ne V; 80 carry a partial resolved prefix and 4 have an empty path. Raw cuts take the sentinel 10910^{9} and the normalized fractions the sentinel 2.0; all 84 are incorrect, so every structural-control signal ranks them last. On literal P≠VP \ne V the raw weighted cut sits +0.114 above base (0.5837 vs. 0.4697), but on resolved P≠VP \ne V it sits below base (0.6000 vs. 0.6301): the raw-cut residual in E1’s literal population is attributable to the unresolved-failure composition.

Signals and the 2×22 \times2 S0S_{0} weighted cut and S3S_{3} unweighted cut are the raw column; S6S_{6} boundary-weight fraction and S8S_{8} boundary-edge fraction are the normalized column. S8S_{8} is a post-E1 signal generated by E1’s post-hoc S6S_{6} observation; its formula and all comparisons were frozen before any E1A outcome, but any S8S_{8} (A3/A4) finding is a within-dataset follow-up, not external validation.

Paired bootstrap on RN All intervals below use 10,000 replicates over question groups (seed 42). Each difference is the AUPCC of the first signal minus that of the second.

ComparisonΔ\Delta95% CI
Weighted cut - unweighted cut-0.010[-0.027, +0.035]
Boundary-weight fraction - weighted cut0.097[-0.0002, +0.196]
Boundary-edge fraction - unweighted cut0.090[+0.007, +0.201]
Boundary-weight fraction - boundary-edge fraction-0.004[-0.024, +0.016]

Table 25. Paired AUPCC differences on regenerated MAB resolved non-closure queries (RN, N=246N = 246). Intervals use 10,000 paired bootstrap replicates grouped by exact question text.

Benchmark / graphNNReproduced vs publishedGate
LongMemEval / temporal5000.736 vs 0.736PASS
LongMemEval / semantic5000.682 vs 0.682PASS
MetaQA-3-hop / typed250Δ=+0.020\Delta=+0.020 vs +0.023+0.023PASS
CoDEx-S / co-occurrence18280.237 vs 0.373FAIL (exploratory)
MAB single-hop——BLOCKED (provenance)
LoCoMo——BLOCKED (provenance)

Table 26. Table 26: E2 reproduction gates. Confirmatory lanes reproduce the published result; CoDEx-S fails (unseeded upstream training); two lanes are provenance-blocked.

Preregistered hypotheses (frozen labels and rules)

  • A1 (residual raw signal survives the resolution filter): NOT SUPPORTED. AUPCC(S0S_0 | RN) = 0.6000 ≤\le base 0.6301.

  • A2 (raw weights still do not beat raw topology): SUPPORTED. Δ(S0−S3)=−0.0104\Delta(S_0 - S_3) = -0.0104, CI [−0.027-0.027, +0.035+0.035] (∣Δ∣<0.02|\Delta| < 0.02, CI includes 0).

  • A3 (normalization improves the weighted signal): NOT SUPPORTED (within-dataset follow-up). Δ(S6−S0)=+0.0968\Delta(S_6-S_0) = +0.0968, CI [−0.0002-0.0002, +0.196+0.196].

  • A4 (weights vs. normalized topology): INDISTINGUISHABLE. Δ(S6−S8)=−0.0037\Delta(S_6 - S_8) = -0.0037, CI [−0.024-0.024, +0.016+0.016], ∣Δ∣<0.02|\Delta| < 0.02.

Confirmatory analyses are A1 and A2 (raw signals, E1-defined); A3 and A4 are frozen but within-dataset follow-ups involving the post-E1 signal S8S_8. The combined E1+E1A reading: continuous edge weights are unnecessary (raw and normalized); the strict 100%-precision regime is structural closure P=VP = V; the non-closure residual is largely resolution-state composition; and among resolved non-closure queries a normalized boundary fraction (relative boundary exposure) carries a moderate signal (∼+0.07\sim+0.07 over base) that is an MAB-specific hypothesis requiring external validation. Caveats: single dataset, N=246N = 246, wide CIs, and S6/S8S_6/S_8 evaluated on their motivating data.

Provenance E1A operates on the same audited per-query artifact as E1 (verified by an identical integrity hash), with no re-extraction or recomputation.

Cross-Benchmark Validation (E2)

This appendix reports the preregistered cross-benchmark validation (E2) summarized in §4.3. Its specification was frozen (2026-08-13) after the provenance/reproduction gates but before any comparative S8/S6/S3/S0S_8/S_6/S_3/S_0 outcome was computed. No LLM/API/network calls were made (HF_HUB_OFFLINE); no E1/E1A artifacts were modified.

Question and frozen hypotheses Does the MAB-generated hypothesis that normalized unweighted boundary topology S8S_8 (boundary-edge count / total edge count) is a transferable zero-fit confidence signal generalize outside MAB? Frozen hypotheses: E2-H1 (normalization generalizes: S8S_8 beats raw S3S_3 across external families); E2-H2 (S8S_8 exceeds base accuracy on ≥1\ge1 confirmatory external population); E2-H3 (weights not required in the abstention statistic: no confirmatory instantiation shows Δ(S6−S8)≥0.02\Delta(S_6 - S_8) \ge0.02 with CI excluding 0); E2-H4 (methodological: report full vs. resolved wherever a failure state exists).

Reproduction-gate protocol Each lane recomputes the benchmark’s own published boundary cut result from its existing pipeline (selector, graph, kk, correctness) through the paper’s evaluator; a lane is confirmatory only if this reproduces the published value. Weight conclusions concern the post-selection abstention statistic only; every external pipeline uses graph-smoothed selection upstream.

PopulationWeighted cutUnweighted cutBoundary-weight fractionBoundary-edge fraction
LongMemEval temporal0.7360.7450.7330.736
LongMemEval semantic0.6820.7790.8530.774
MetaQA-3hop0.3240.3240.2830.283

Table 27. Confirmatory E2 AUPCC on external populations: LongMemEval temporal and semantic (N=500N = 500 each) and MetaQA-3hop (N=250N = 250). Paired intervals resample each benchmark's external query identifier. These populations are separate from headline and regenerated MAB.

Confirmatory signal AUPCC On each gate-passing lane’s full population (base accuracy in parentheses): raw weighted S0S_0, raw unweighted S3S_3, normalized weighted S6S_6, normalized unweighted S8S_8.

Paired bootstrap per lane The following differences use 10,000 replicates (seed 42) and 95% percentile confidence intervals.

LaneComparisonΔ\Delta95% CI
LongMemEval temporalboundary-edge fraction − unweighted cut-0.009[-0.026, +0.015]
LongMemEval temporalboundary-edge fraction − weighted cut0.001[-0.043, +0.044]
LongMemEval temporalboundary-weight fraction − boundary-edge fraction-0.003[-0.045, +0.039]
LongMemEval semanticboundary-edge fraction − unweighted cut-0.005[-0.017, +0.007]
LongMemEval semanticboundary-edge fraction − weighted cut0.093[+0.041, +0.146]
LongMemEval semanticboundary-weight fraction − boundary-edge fraction0.078[+0.048, +0.111]
MetaQA-3hopboundary-edge fraction − unweighted cut-0.041[-0.095, +0.022]
MetaQA-3hopboundary-edge fraction − weighted cut-0.041[-0.095, +0.022]
MetaQA-3hopboundary-weight fraction − boundary-edge fraction0.000[+0.000, +0.000]
CoDEx-Sboundary-edge fraction − unweighted cut-0.001[-0.009, +0.007]
CoDEx-Sboundary-edge fraction − weighted cut-0.003[-0.010, +0.006]
CoDEx-Sboundary-weight fraction − boundary-edge fraction0.003[-0.003, +0.009]

Table 28. External E2 paired AUPCC differences: LongMemEval temporal and semantic (N=500N = 500 each), MetaQA-3hop (N=250N = 250), and exploratory CoDEx-S (N=1828N = 1828). Intervals use 10,000 replicates grouped by external query identifier. CoDEx-S failed its reproduction gate.

CoDEx-S remains exploratory because its reproduction gate failed.

Frozen hypothesis verdicts

  • E2-H1 (normalization generalizes): NOT SUPPORTED. Both confirmatory external families have Δ(S8−S3)≤0\Delta(S_8 - S_3) \le0 by point estimate (temporal −0.0088, semantic −0.0049, MetaQA −0.0407); none positive.

  • E2-H2 (S8S_8 above base somewhere external): SUPPORTED (weakly). On LME semantic, S8=0.7744S_8 = 0.7744, above base accuracy 0.748, but the raw count S3S_3 (0.7793) and S6S_6 (0.8528) are both higher, so S8S_8 is above base without being the operative signal; elsewhere S8S_8 is at or below base.

  • E2-H3 (weights not required in the abstention statistic): NOT SUPPORTED. On the semantic-graph instantiation S6S_6 beats S8S_8 by +0.0785, CI [+0.048, +0.111] (material, CI excludes 0).

  • E2-H4 (failure-state separation): NOT APPLICABLE IN SAMPLE. No evaluated benchmark produced an unresolved/failure state (MetaQA: 0/250; others resolve by construction), so full ≡ resolved everywhere.

Frozen cross-benchmark verdict: MAB-specific. The MAB normalization hypothesis did not generalize across benchmarks; behavior is regime-dependent; one external semantic regime shows that continuous weights can matter.

External-sample caveat Confirmatory evidence spans two independent external families (LongMemEval, MetaQA); LongMemEval temporal and semantic are two graph regimes within one family, not two independent families.

Exploratory (gate-failed) CoDEx-S Reported for completeness only; the reproduction gate failed (0.2373 vs published 0.3727; unseeded ComplEx initialization), so no confirmatory conclusion is licensed. Full-population AUPCC: base 0.3255, S0S_0 0.2373, S3S_3 0.2355, S6S_6 0.2370, S8S_8 0.2344; Δ(S8−S3)=−0.0011\Delta(S_8-S_3)=-0.0011 [−0.009, +0.007]. Mild positive S8S_8 deltas appear on a few closed/dense relations, directionally reminiscent of the closed-vocabulary pattern, but this run does not reproduce the published pipeline and licenses no conclusion.

Provenance-blocked lanes MAB single-hop: the pinned LLM-extraction cache covers multi-hop splits only, so per-query graphs are unavailable and regeneration would require new LLM calls (prohibited). LoCoMo: the session-content file is absent from disk (only questions present) and re-download is prohibited. Neither lane is interpreted as E2 evidence.

Confirmatory vs. post-hoc Confirmatory: the frozen E2-H1–H4 adjudications above. Post-hoc / hypothesis-generating: the mechanistic reading of the LME-semantic S6>S8S_6>S_8 result (that weight semantics may matter when edge weights encode task-relevant compatibility), which is a single-instantiation observation in the opposite direction of the E1A weights finding and cannot rescue the preregistered hypothesis.

Provenance The evaluator is unchanged; E2 bootstrap clusters use the external query identifier directly, with 10,000 replicates and seed 42.

Linear-Scorer Ablation

We ablate the default squared scorer (qTv)2(\mathbf{q}^{\mathsf{T}}\mathbf{v})^2 by recomputing all selectors on the linear scorer qTv\mathbf{q}^{\mathsf{T}}\mathbf{v}. Table 29 reports boundary cut and three score-only baselines on this linear-scorer field. The boundary cut remains the strongest zero-fit selector. This is a scorer-function ablation, not a smoothing ablation: E(z)=zTLzE(\mathbf{z})=\mathbf{z}^{\mathsf{T}}\mathbf{L}\mathbf{z} depends only on the graph and the node field; γ\gamma enters at most upstream through path resolution, and Appendix § 17 shows the signal is unchanged across γ\gamma.

SignalAggregate AUPCC262k point estimate
Coherence0.7750.624
Smoothed score0.6880.518
Raw score0.6670.490
Margin0.6120.435

Table 29. Selector comparison on multi-hop MAB under the linear scorer qTv\mathbf{q}^{\mathsf{T}}\mathbf{v} (alternative to the default (qTv)2(\mathbf{q}^{\mathsf{T}}\mathbf{v})^2).

Typed-Edge Robustness

Table 30 reports how the boundary cut changes when the typed graph is deliberately corrupted. Missing edges hurt more than added spurious edges, but performance degrades gracefully rather than catastrophically. The uncorrupted baseline AUPCC (0.7495) reported here comes from this perturbation experiment’s own seed/protocol and differs by ∼0.016 from the typed-graph entry in Table 8 (0.7652); deltas within Table 30 are internally consistent against this baseline.

PerturbationAUPCCDelta
Baseline0.750—
Drop 10% edges0.716 ± 0.005−0.034
Drop 20% edges0.717 ± 0.005−0.033
Drop 30% edges0.720 ± 0.002−0.029
Add 10% spurious0.743 ± 0.001−0.007
Add 20% spurious0.743 ± 0.001−0.006

Table 30. Robustness of the boundary cut to graph corruption on cached multi-hop MAB neighborhoods.

Gamma Invariance

The boundary cut E(z)=zTLzE(\mathbf{z})=\mathbf{z}^{\mathsf{T}}\mathbf{L}\mathbf{z} depends on the graph Laplacian and the node field; the smoothing hyperparameter γ\gamma does not enter the cut directly and can affect it at most upstream through path resolution, so the abstention signal is expected to be stable in γ\gamma. Table 31 confirms this empirically. The smoothed-score baseline shifts with γ\gamma, but the boundary cut AUPCC and the zero-cut subset are nearly unchanged.

γ\gammaCoherence AUPCCZE queriesZE precision
0.0000.766711.000
0.0500.765711.000
0.1000.765711.000
0.2000.765711.000

Table 31. Gamma sweep on multi-hop MAB (400 queries). Boundary-cut AUPCC is numerically stable across the tested γ\gamma values (0.765–0.766), with the same 71 zero-energy queries at 100% precision throughout.

AUROC Comparison

Table 32 reports AUROC for all selectors, complementing the AUPCC values in Table 13. AUROC measures overall discrimination between correct and incorrect queries without the coverage-weighting of AUPCC. GBT achieves the highest AUROC (0.889), followed by the boundary cut (0.807). The boundary cut’s AUROC advantage over score baselines is consistent with its AUPCC advantage: the graph signal provides correctness information beyond what scores alone capture.

SignalAll6k32k64k262k
GBT0.8890.9120.8880.8850.850
Coherence0.8070.7530.7930.7920.845
Smoothed score0.6450.5860.6050.7320.617
Raw score0.6110.5730.5840.6830.578
LR0.6200.5860.5920.6550.605
Score margin0.4130.4580.4470.5080.351

Table 32. AUROC for headline MAB multi-hop (N=400N = 400 overall; N=100N = 100 per context scale). GBT and LR use folds grouped by exact question under the reconstructed mapping (Appendix § 21); paired uncertainty uses exact-question resampling.

RmedianR_{\mathrm{median}} Gate: A Small Labeled Diagnostic

This gate is a small labeled diagnostic (an audit for whether the boundary cut is likely to help on a new graph construction), not part of a label-free deployment mechanism: it consumes ground-truth labels in the probe set. To test whether support compatibility can be assessed prospectively from such a probe, we repeatedly sample tiny labeled probe sets (20, 30, and 50 examples) from each of eight graph families and apply a simple decision rule: predict “boundary cut likely useful” if Rmedian>2R_{\mathrm{median}} > 2, where RmedianR_{\mathrm{median}} is the ratio of median cut on incorrect vs. correct answers within the probe. We compare this prediction against the full-run ground truth (Table 33). This secondary diagnostic consumes correctness labels; its threshold was chosen on the original evaluated families, not independently validated as a general predictive criterion.

Family (truth)Probe 20Probe 30Probe 50
AccMed RmR_mAccMed RmR_mAccMed RmR_m
MAB typed (useful)0.8608.1400.9208.5800.9507.530
MAB sem. kNN (useful)0.5502.2500.6102.3900.6802.500
LongMemEval temp. (not)0.9701.0000.9800.9701.0000.990
LongMemEval kNN (not)0.8500.6500.9200.6500.9700.590
LoCoMo temp. (not)1.0001.0201.0001.0301.0001.030
CoDEx-S co-occ. (not)0.9900.5801.0000.5901.0000.630
CoDEx-S typed (not)0.9701.0100.9800.9901.0001.010
MetaQA-3hop typed (not)0.9201.0300.9701.0401.0001.000
Overall0.8900.9200.950

Table 33. Prospective RmedianR_{\mathrm{median}} gate (1,000 trials per cell, threshold = 2). Decision accuracy is the fraction of trials where the gate matches the full-run ground truth. The gate is conservative: false-positive rate < 1% at probe size 50, meaning it rarely approves a graph that should be rejected. This secondary diagnostic consumes correctness labels; its threshold was chosen on the original evaluated families, not independently validated as a general predictive criterion.

At probe size 50, the gate achieves 95% overall accuracy across eight families with a false-positive rate of 0.5%. The only substantial source of error is the borderline case (MAB semantic kNN, full-run RmedianR_{\mathrm{median}}=2.75, just above the threshold). The four clearly misaligned graphs in the original probe set are rejected at ≥97% accuracy even at probe size 20. This secondary diagnostic consumes correctness labels; its threshold was chosen on the original evaluated families, not independently validated as a general predictive criterion.

This is a heuristic, not a validated theorem. The threshold of 2 and the probe sizes were chosen post hoc on the original six families used for evaluation. Nevertheless, the result suggests that a small labeled probe can provide a useful go/no-go signal for deployment of the abstention rule.

We add CoDEx-S with a typed entity–relation mask and MetaQA-3hop as the seventh and eighth prospectively probed families. Both are correctly rejected: CoDEx-S typed has mean RmedianR_{\mathrm{median}} = 1.01 (std 0.23 across five probe-50 seeds), MetaQA-3hop has mean RmedianR_{\mathrm{median}} = 1.04, and decision accuracy is ≥92%\ge92\% at every probe size. This extends the diagnostic to two previously unprobed families: the threshold of 2 was chosen on the original six evaluated families. It does not establish a general predictive criterion. The per-relation breakdown of both rejected datasets (Appendix § 20) shows that aggregate rejection masks within-dataset heterogeneity. CoDEx-S contains a relation that would individually pass the threshold-2 gate (P463 “member of” with RmedianR_{\mathrm{median}} = 52); MetaQA-3hop is uniformly subthreshold (max RmedianR_{\mathrm{median}} = 1.46 for in_language) yet still contains relations with positive boundary-cut lifts. This secondary diagnostic consumes correctness labels; its threshold was chosen on the original evaluated families, not independently validated as a general predictive criterion.

Support Compatibility as a Per-Relation Property

Both CoDEx-S typed and MetaQA-3hop are correctly rejected by the RmedianR_{\mathrm{median}} gate at the aggregate level (Appendix K), yet both contain relations on which the abstention rule works. This appendix reports per-answer-relation breakdowns for each dataset and identifies the structural conditions under which the cut signal is informative. This secondary diagnostic consumes correctness labels; its threshold was chosen on the original evaluated families, not independently validated as a general predictive criterion.

CoDEx-S Typed

The aggregate CoDEx-S result (boundary cut AUPCC 0.3727 vs. raw score 0.5430) masks substantial heterogeneity across relations. Table 11 reports boundary cut performance for the seven most-populated relations in the CoDEx-S typed evaluation.

For CoDEx-S typed, compatible pairs are constructed from relation ids and hand-specified compositional relation families. The mask uses relation metadata only; it is not learned from correctness labels and does not read test-time answer labels.

Three patterns emerge:

Support-compatible relations P463 (“member of”) achieves AUPCC 0.700 with RmedianR_{\mathrm{median}} = 52 and a zero-cut subset of 89 queries at 76.4% precision. This relation behaves like the MAB typed graph: organizational membership creates dense, support-compatible neighborhoods where the boundary cut separates correct from incorrect answers. This secondary diagnostic consumes correctness labels; its threshold was chosen on the original evaluated families, not independently validated as a general predictive criterion.

Sparse-mask collapse P530 (“diplomatic relation”) assigns zero cut to all 287 queries because the typed mask produces disconnected local subgraphs: every neighborhood is a single node or pair with no typed edges. Zero-cut precision equals base accuracy (0.244), confirming that the cut signal is trivially uninformative. This is a graph-construction failure, not a boundary-cut failure: the mask is too restrictive for this relation’s neighborhood structure.

Dense but misaligned P106 (“occupation,” 593 queries) has dense typed neighborhoods from the Person-Role family, but the boundary cut (0.215) actively underperforms raw score (0.313) with Rmedian=0.88<1R_{\mathrm{median}} = 0.88 < 1. The typed edges connect candidates whose support should not agree under occupation queries: a person’s employer and field of work do not reliably predict whether a specific occupation candidate is correct. This secondary diagnostic consumes correctness labels; its threshold was chosen on the original evaluated families, not independently validated as a general predictive criterion.

These patterns refine the support-compatibility thesis from a binary dataset-level property to a continuous, relation-level property. A dataset can contain both support-compatible and support-incompatible relations; the aggregate result reflects the mixture. Among the seven relations above, only P463 exceeds the gate threshold of 2, and it is the only relation where the boundary cut outperforms raw score. These CoDEx-S per-relation values are artifact-conditional: the exact historical ComplEx initialization is not reproducibly recoverable and the original per-example predictions were not persisted, so per-relation bootstrap intervals could not be reconstructed; the point estimates are reported as illustrative exploratory evidence only.

MetaQA-3hop

MetaQA-3hop [30] provides an independent test of the per-relation pattern: a different domain (movie KG with 9 relations), a different scoring channel (MiniLM bi-encoder, d=384d=384), and a different retrieval setup (per-question subgraph instead of trained KG embedding scorer). We sample 250 queries stratified by extracted question–relation set and apply the same boundary-cut pipeline with an answer-type candidate filter (entities appearing as objects of the inferred answer relation in the subgraph are scored).

The aggregate result mirrors CoDEx-S typed: the boundary cut narrowly improves over raw score by +0.023 AUPCC (95% bootstrap CI [−0.068, +0.104], 2,000 iterations), Rmedian=1.10R_{\mathrm{median}} = 1.10, and the gate at threshold 2 correctly rejects deployment in aggregate. Per-relation, behavior is heterogeneous, though the per-relation intervals are wide and only in_language excludes zero (Table 12). This secondary diagnostic consumes correctness labels; its threshold was chosen on the original evaluated families, not independently validated as a general predictive criterion.

For in_language, the boundary-cut lift is +0.230 AUPCC at base accuracy 0.435. Language candidates per query have median 4, compared with medians 15–33 for person-name candidates.

Candidate-type filtering and edge-type masking are substitutes The MetaQA pipeline applies an answer-type candidate filter upstream: only entities appearing as objects of the inferred answer relation in the subgraph are scored. Typed and untyped graphs produce identical top-1 picks on 243 of 250 queries; the aggregate AUPCC delta is −0.003.

Question Identity and Cross-Configuration Evaluation

The earliest transfer harness assigned a separate group to each pair (context scale, positional query index). A later correction grouped rows by the positional index alone, assuming that the same index referred to the same question at all four scales. Inspection of the actual question lists contradicts that assumption. The reconstructed headline mapping contains 374 distinct exact question texts across 400 evaluations. There are 26 repeated texts, each appearing twice, including repetitions at different indices. Neither the composite row identifier nor the positional index is the appropriate identity key for those repetitions. The previous claim that index grouping had corrected question leakage was therefore unsupported; its reported 99/100 leakage count measured index reuse, not verified question reuse.

The repaired evaluation uses five-fold cross-validation grouped by exact question text, keeping every occurrence together across scales and configurations. Existing features, labels and estimator hyperparameters are retained. transfer harness uses numeric sorting by (context_length,query_index) and no feature standardization. This ordering was verified by reproducing all 18 archived raw/population-rank cells under their original index grouping. The main learned-confidence experiment instead uses native CSV order and training-fold StandardScaler. The separate native-order unscaled sensitivity (historically 0.8200, repaired to 0.812) is retained as a protocol sensitivity, not a second primary estimate. The main comparison uses its own repaired paired interval; the separate unscaled evaluation is retained only as a protocol sensitivity (details available from the author). Row order and scaling remain protocol differences, but neither justifies calling the earlier folds question-disjoint.

The mapping from historical feature rows to question text uses the reconstructed mapping described in this appendix, obtained by (context_length,query_index) from the available dataset snapshot. Its exact data content and question-list hashes are recorded, and agree with the verified regenerated bundle. The original headline run did not record a dataset revision, so this does not establish independently authenticated historical identity.

Every repaired paired interval uses 10000 exact-question bootstrap replicates with seed 42, conditional on the fixed out-of-fold predictions. The archived transfer correction report specifies 2000 replicates, while the earlier manuscript described 10000; the repair resolves this discrepancy prospectively and does not relabel the old run.

The archived rank-normalization diagnostic ranked each feature column across all 400 queries within each configuration, before cross-validation, with positional tie breaks. This is a transductive population-rank transformation, not a within-query rank. Its repaired result is reported separately. Literal within-query candidate percentiles cannot be computed from the headline scalar-feature artifacts, which lack candidate-score vectors. Ranking five heterogeneous scalar features against one another would be a different, unsupported definition. The 8-of-9 population-rank finding must not be presented as evidence for within-query normalization.

Transfer data. The repaired AUROC matrix and the unchanged feature-scale distributions accompany the grouping history.

Train / testBASEMINILMLINEAR
BASE0.8910.8410.857
MINILM0.7610.9090.753
LINEAR0.7980.7440.868

Table 34. GBT transfer AUROC on headline MAB multi-hop (N=400N = 400 per target). Rows identify training configurations and columns deployment configurations. Folds and paired resampling use exact question texts under the reconstructed mapping (Appendix § 21).

ConfigurationMeanStdRangeMedian
BASE0.0080.002[0.000, 0.009]0.008
MINILM0.0030.001[0.000, 0.003]0.003
LINEAR0.3900.131[0.000, 0.625]0.406

Table 35. raw_score distribution per configuration (n=400n=400 queries).

References

  1. [1]Anastasios N. Angelopoulos and Stephen Bates. 2023. Conformal Prediction: A Gentle Introduction. Foundations and Trends in Machine Learning 16, 4 (2023), 494–591. doi:10.1561/220000001
  2. [2]Petr Anokhin, Nikita Semenov, Artyom Sorokin, Dmitry Evseev, Andrey Kravchenko, Mikhail Burtsev, and Evgeny Burnaev. 2025. AriGraph: Learning Knowledge Graph World Models with Episodic Memory for LLM Agents. In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence. 12–20. doi:10.24963/ijcai.2025/2DOI
  3. [3]Mikhail Belkin, Partha Niyogi, and Vikas Sindhwani. 2006. Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples. Journal of Machine Learning Research 7, 85 (2006), 2399–2434.DOI
  4. [4]Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. 2025. MemO: Building Production-Ready AI Agents with Scalable Long-Term Memory. In ECAI 2025 – 28th European Conference on Artificial Intelligence, 25–30 October 2025, Bologna, Italy, Including 14th Conference on Prestigious Applications of Intelligent Systems (PAIS 2025) (Frontiers in Artificial Intelligence and Applications). IOS Press, 2993–3000. doi:10.3233/FAIA251160
  5. [5]Steve Cronen-Townsend, Yun Zhou, and W. Bruce Croft. 2002. Predicting query performance. In Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval. Association for Computing Machinery, New York, NY, USA, 299–306. doi:10.1145/564376.564429
  6. [6]Suchana Datta, Debasis Ganguly, Derek Greene, and Mandar Mitra. 2022. Deep-QPP: A Pairwise Interaction-based Deep Learning Model for Supervised Query Performance Prediction. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining. Association for Computing Machinery, New York, NY, USA, 201–209. doi:10.1145/3488560.3498491DOI
  7. [7]Fernando Diaz. 2007. Performance prediction using spatial autocorrelation. In Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval. Association for Computing Machinery, New York, NY, USA, 583–590. doi:10.1145/1277741.1277841DOI
  8. [8]Ran El-Yaniv and Yair Wiener. 2010. On the Foundations of Noise-free Selective Classification. Journal of Machine Learning Research 11 (2010), 1605–1641.DOI
  9. [9]Guglielmo Faggioli, Nicola Ferro, Cristina Ioana Muntean, Raffaele Perego, and Nicola Tonellotto. 2023. A Geometric Framework for Query Performance Prediction in Conversational Search. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, New York, NY, USA, 1355–1365. doi:10.1145/3539618.3591625DOI
  10. [10]Yonatan Geifman and Ran El-Yaniv. 2017. Selective Classification for Deep Neural Networks. In Advances in Neural Information Processing Systems, Vol. 30. Curran Associates, Inc. https://proceedings.neurips.cc/paper_files/paper/2017/file/4a8423d5e91fda00bb7e46540e2b0cf1-Paper.pdfarxiv.org/abs/1705.08500
  11. [11]Kanishka Ghosh Dastidar, Michael Danzinger, Laura Caspari, Jelena Mitrovic, and Michael Granitzer. 2026. Query Performance Prediction under Corpus Growth in Dense Retrieval. In Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, New York, NY, USA, 3733–3737. doi:10.1145/3805712.3809960DOI
  12. [12]Hippolyte Gisserot-Bouchlef, Manuel Faysse, Emmanuel Malherbe, Céline Hudelot, and Pierre Colombo. 2024. Towards Trustworthy Reranking: A Simple yet Effective Abstention Mechanism. Transactions on Machine Learning Research 2024 (2024). https://openreview.net/forum?id=iMKUMWfRIj
  13. [13]Yuanzhe Hu, Yu Wang, and Julian McAuley. 2026. Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions. In International Conference on Learning Representations. Poster.arxiv.org/abs/2507.05257
  14. [14]Oz Huly, David Carmel, and Oren Kurland. 2025. Predicting RAG Performance for Text Completion. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, New York, NY, USA, 1283–1293. doi:10.1145/3726302.3730062DOI
  15. [15]Heinrich Jiang, Been Kim, Melody Y. Guan, and Maya Gupta. 2018. To Trust Or Not To Trust A Classifier. In Advances in Neural Information Processing Systems, Vol. 31. Curran Associates, Inc. https://proceedings.neurips.cc/paper_files/paper/2018/file/7180cffd6a8e829dacf2a31b3f72ece-Paper.pdfarxiv.org/abs/1805.11783
  16. [16]Charles H. Martin and Michael W. Mahoney. 2021. Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning. Journal of Machine Learning Research 22, 165 (2021), 1–73.
  17. [17]Charles H. Martin, Tongshu Serena Peng, and Michael W. Mahoney. 2021. Predicting Trends in the Quality of State-of-the-Art Neural Networks Without Access to Training or Testing Data. Nature Communications 12 (2021), 4122. doi:10.1038/s41467-021-24025-8DOI
  18. [18]Nicolas Papernot and Patrick McDaniel. 2018. Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning. arXiv preprint arXiv:1803.04765 (2018).arxiv.org/abs/1803.04765
  19. [19]Jing Ren, Bowen Li, Ziqi Xu, Xikun Zhang, Haytham Fayek, and Xiaodong Li. 2026. When to Trust: A Causality-Aware Calibration Framework for Accurate Knowledge Graph Retrieval-Augmented Generation. In Proceedings of the ACM Web Conference 2026 (WWW ’26). Association for Computing Machinery, New York, NY, USA, 3859–3869. doi:10.1145/3774904.3792358DOI
  20. [20]Alireza Salemi and Hamed Zamani. 2024. Evaluating Retrieval Quality in Retrieval-Augmented Generation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, New York, NY, USA, 2395–2400. doi:10.1145/3626772.3657957DOI
  21. [21]Anna Shtok, Oren Kurland, and David Carmel. 2010. Using statistical decision theory and relevance models for query-performance prediction. In Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval. Association for Computing Machinery, New York, NY, USA, 259–266. doi:10.1145/1835449.1835494DOI
  22. [22]Anna Shtok, Oren Kurland, and David Carmel. 2016. Query Performance Prediction Using Reference Lists. ACM Transactions on Information Systems 34, 4, Article 19 (2016), 34 pages. doi:10.1145/2926790DOI
  23. [23]Anna Shtok, Oren Kurland, David Carmel, Fiana Raiber, and Gad Markovits. 2012. Predicting Query Performance by Query-Drift Estimation. ACM Transactions on Information Systems 30, 2, Article 11 (2012), 35 pages. doi:10.1145/2180868.2180873DOI
  24. [24]Charlie Victor Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2025. Scaling LLM Test-Time Compute Optimally Can Be More Effective Than Scaling Parameters for Reasoning. In International Conference on Learning Representations. Oral.arxiv.org/abs/2408.03314
  25. [25]Maria Vlachou and Craig Macdonald. 2024. Coherence-based Query Performance Measures for Dense Retrieval. In Proceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retrieval. Association for Computing Machinery, New York, NY, USA, 15–24. doi:10.1145/3664190.3672518DOI
  26. [26]Vladimir Vovk, Alexander Gammerman, and Glenn Shafer. 2005. Algorithmic Learning in a Random World. Springer, New York, NY. doi:10.1007/b106715DOI
  27. [27]Weizhi Wang, Li Dong, Hao Cheng, Xiaodong Liu, Xifeng Yan, Jianfeng Gao, and Furu Wei. 2023. Augmenting Language Models with Long-Term Memory. In Advances in Neural Information Processing Systems, Vol. 36. Curran Associates, Inc., 74530–74543. doi:10.52202/075280-3259
  28. [28]Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, and Yongfeng Zhang. 2025. A-Mem: Agentic Memory for LLM Agents. In Advances in Neural Information Processing Systems. Poster.arxiv.org/abs/2502.12110
  29. [29]Hamed Zamani, W. Bruce Croft, and J. Shane Culpepper. 2018. Neural Query Performance Prediction using Weak Supervision from Multiple Signals. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. Association for Computing Machinery, New York, NY, USA, 105–114. doi:10.1145/3209978.3210041DOI
  30. [30]Yuyu Zhang, Hanjun Dai, Zornitsa Kozareva, Alexander J. Smola, and Le Song. 2018. Variational Reasoning for Question Answering with Knowledge Graph. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32. 6069–6076.arxiv.org/abs/1709.04071
  31. [31]Yujie Zhang, Weikang Yuan, and Zhuoren Jiang. 2025. Bridging Intuitive Associations and Deliberate Recall: Empowering LLM Personal Assistant with Graph-Structured Long-term Memory. In Findings of the Association for Computational Linguistics: ACL 2025. Association for Computational Linguistics, Vienna, Austria, 17533–17547. doi:10.18653/v1/2025.findings-acl.901DOI
  32. [32]Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. 2024. MemoryBank: Enhancing Large Language Models with Long-Term Memory. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 19724–19731. doi:10.1609/aaai.v38i17.29946DOI
  33. [33]Hang Zhou and Hans-Georg Müller. 2025. Conformal Inference for Random Objects. The Annals of Statistics 53, 3 (2025). doi:10.1214/25-AOS2495DOI
  34. [34]Yun Zhou and W. Bruce Croft. 2006. Ranking robustness: a novel framework to predict query performance. In Proceedings of the 15th ACM international conference on Information and knowledge management. Association for Computing Machinery, New York, NY, USA, 567–574. doi:10.1145/1183614.1183696DOI
  35. [35]Xiaojin Zhu, Zoubin Ghahramani, and John Lafferty. 2003. Semi-Supervised Learning Using Gaussian Fields and Harmonic Functions. In Proceedings of the Twentieth International Conference on Machine Learning. 912–919.

Paper details

vlnc:W_najkx4cs
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 8

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents