betaFeedback

WorksComputer Science

CascadeFilter: Cost-Efficient Semantic Filtering with Joint Threshold Selection

Abstract

Semantic filtering selects records that satisfy a natural-language predicate and is an important operation in data analytics based on large language models (LLMs). Executing an expensive model on every document incurs substantial inference cost. Existing approaches reduce this cost by accepting predictions from cheaper models and forwarding uncertain records to an oracle. Task Cascades further varies document portions and operations, and provides an accuracy-guaranteed extension. However, its threshold calibration follows a prescribed adjustment sequence after cascade assembly, rather than minimizing cost over alternative combinations of stage thresholds. Motivated by this, we propose CascadeFilter, a three-stage semantic filtering method that jointly searches the thresholds for a small language model operating on extracted evidence and full documents, with unresolved records forwarded to the oracle. For a fixed cascade, we formulate threshold selection as minimizing token-aware cost subject to an oracle-relative error constraint, and select the least-cost certified configuration in a pre-specified grid. A finite-population calibration procedure bounds oracle-relative error after selection, using fixed-sequence tests on threshold tracks to allocate the multiple-testing budget. We further combine lexical and dense retrieval, reranking, and neighborhood-supported confidence to improve evidence extraction and routing. These components enhance the cascade while leaving its certification interface unchanged. Across six datasets and 13 evaluation settings, the reported results have the highest benchmark accuracy and macro-averaged F1 score in 12 settings each. These comparisons describe the reported operating points; the accuracy guarantee concerns agreement with the oracle.

Introduction

Semantic filtering identifies the records in a corpus that satisfy a natural-language predicate. Given a collection of documents and a query such as “does this review mention a billing issue?”, the operation returns the documents for which the predicate is true. Large language models (LLMs) make such predicates available in data systems without requiring users to specify extraction rules or train a classifier for each query [1, 2, 3]. Applications include selecting customer reviews by aspect, identifying dialogue outcomes, and checking whether textual evidence supports a claim. These applications require semantic decisions over large collections, making efficient execution of the filtering operation an important problem. Unlike a predicate over a structured attribute, a semantic predicate may depend on paraphrases, relationships between sentences, or the context in which a statement appears. A review can describe a billing problem without using that exact phrase, while a document containing those words need not report a problem. Retrieving a relevant passage helps identify evidence, but does not by itself replace the final semantic decision.

A direct execution strategy invokes a capable LLM on every document. Although simple, this strategy repeatedly pays for processing the entire input and generating a decision, even when a short passage is sufficient to answer the predicate. Its cost increases with both the number and length of documents. A cheaper small language model (SLM) can reduce inference cost, but accepting all its predictions can reduce accuracy. A model cascade therefore uses the cheaper model first and forwards uncertain records to a more expensive reference model, called the oracle. The objective is to reduce the total execution cost while meeting a user-specified accuracy target. Here accuracy is certified relative to the oracle; agreement with benchmark ground truth is evaluated separately because the oracle can itself make mistakes. Two decisions determine execution cost: how much text to process and which model to invoke. A short input can omit necessary evidence; a full document can supply that evidence without requiring a stronger model. Conversely, some records remain difficult even with full context. The execution policy should distinguish these cases and account for both input and output tokens.

Existing work addresses this cost–accuracy tradeoff through model routing, proxy prediction, and cascade execution. FrugalGPT and RouteLLM select among models with different costs and capabilities [4, 5]. LOTUS supports approximate semantic execution, while BARGAIN provides variants with statistical guarantees on accuracy, precision, or recall [1, 6]. ScaleDoc uses document representations and lightweight proxy models to reduce expensive predicate evaluations [7]. These methods motivate using inexpensive computations to resolve easy records. Task Cascades extends this idea by varying the model, document portion, and operation within a cascade [8]. Thus, exploiting shorter inputs and providing an accuracy guarantee are established directions; our focus is how to select coupled routing thresholds under an explicit cost objective.

The State-of-the-art. Task Cascades [8] generates candidate tasks using alternative models, document portions, and simplified or related operations. It first sets class-specific thresholds for individual tasks on development data, then greedily assembles a cascade from these tasks. Its accuracy-guaranteed extension subsequently adjusts the assembled cascade’s thresholds on a validation split. In particular, it moves the thresholds together along a prescribed sequence of score-based offsets and tests the resulting cascade, stopping when certification fails [8]. This procedure already accounts for cascade-level accuracy and should not be characterized as merely calibrating each stage independently. However, the guarantee adjustment is a post-processing step rather than a cost optimization over threshold combinations. Changing an early threshold changes both the number and composition of records reaching later stages. Consequently, different combinations can have different costs even when they meet the same accuracy target. This motivates jointly searching thresholds for a fixed cascade structure and comparing certified candidates by their total token cost. For example, relaxing the first threshold accepts additional records cheaply, but changes the error budget available downstream. Tightening the second threshold can compensate by forwarding more records to the oracle. Whether this exchange saves cost depends on which records move between stages and their processing costs. Evaluating each threshold separately cannot in general determine the best exchange. This comparison concerns threshold selection; our fixed structure does not subsume the broader task-generation capabilities of Task Cascades.

Our Idea. We propose CascadeFilter, which processes each record using an SLM on extracted evidence, then the SLM on the full document if needed, and finally the oracle for unresolved records. We formulate cost-minimizing threshold selection for this fixed cascade. The objective and feasible set are generally nonconvex, but error decreases with the second-stage threshold when the first is fixed. This structure motivates searching the two acceptance thresholds jointly over a grid fixed before calibration. For each pair, we account for errors across the complete cascade and certify oracle-relative accuracy using a sample drawn without replacement from the deployment population. We organize candidates into monotone tracks and apply fixed-sequence testing within each track, allocating the failure budget across tracks. We then select the least-cost certified candidate; this need not recover the true feasible optimum even within the grid. To improve the inputs and routing scores, we combine Best Matching 25 (BM25) lexical retrieval [9] with dense retrieval using reciprocal rank fusion (RRF), rerank the resulting chunks, and incorporate embedding-neighborhood support into model confidence. These established retrieval components support the threshold-search design rather than constitute separate claims of new retrieval algorithms. The resulting method couples cost-aware selection with finite-population certification while retaining flexibility in evidence extraction and routing. The candidate grid and routing scores are frozen before calibration, preventing calibration labels from being reused to construct new policies. For each fixed first-stage threshold, second-stage candidates form a conservative-to-lenient sequence, allowing testing to stop at the first uncertified candidate. Searching across tracks retains alternative first-stage choices. If no candidate certifies, the method uses the oracle. The guarantee applies to the specified population and requires fresh calibration for deployment on a new population.

Contributions. We summarize our contributions as follows:

  • Joint Threshold Selection (Section III). We develop a three-stage semantic filtering method and jointly search its coupled thresholds, accounting for both context length and model cost. The search returns the least-cost certified configuration in a pre-specified grid.

  • Finite-Population Certification (Section IV). We instantiate a without-replacement betting bound for cascade error and combine monotone tracks with fixed-sequence testing to control oracle-relative error after threshold selection. The betting method and fixed-sequence testing are existing tools; our contribution is their application to the coupled search.

  • Evidence and Routing Enhancements (Section III). We integrate hybrid retrieval, reranking, batched evidence inference, and neighborhood-supported confidence into the cascade. These components improve the candidate policies evaluated by the certification procedure.

  • Experimental Analysis (§ 5). We evaluate six datasets with 13 settings and study cascade structure, routing components, and calibration behavior. The main table reports the highest reported benchmark accuracy and macro-averaged F1 score in 12 settings each, at the evaluated operating points.

Due to space limitations, proofs and supplementary details are provided in Appendix A, which serves as the technical report for this version. Code is available at https://github.com/oceanusXXD/cascade.

Preliminaries

Task: Semantic Filtering

Semantic filtering selects the documents in a corpus that satisfy a natural-language predicate. Given a query qq and NN documents D={d1,…,dN}D = \{d_1,\ldots,d_N\}, the task returns D∗={d∈D:f∗(q,d)=1}D^* = \{d \in D : f^*(q,d) = 1\}, where f∗(q,d)∈{0,1}f^*(q,d) \in\{0,1\} indicates whether dd satisfies qq. Predicates such as “does this article support the given claim?” are semantic judgments that neither keyword matching nor embedding similarity can reliably make [1].

The ground truth f∗f^* is the evaluation target but is unavailable at corpus scale. We therefore drive the cascade with a frontier LLM ML\mathcal{M}_L (e.g. GPT-5.5) as a reference model, the oracle: its disagreement with f∗f^* is accounted for separately below, while all headline accuracies are against f∗f^*. Served through an application programming interface (API) returning a decision rather than calibrated probabilities, ML\mathcal{M}_L is used only as a binary label ML(q,d)∈{0,1}\mathcal{M}_L(q,d) \in\{0,1\}. We further assume an open-source small language model (SLM) MS\mathcal{M}_S exposing token-level log probabilities for confidence-based routing.

Problem Definition

We fix the cascade structure, models, evidence extractor, and routing-score functions. Stage 1 applies the SLM to extracted evidence, Stage 2 applies it to the full document, and Stage 3 invokes the oracle. The variables to optimize are the two acceptance thresholds ϕ=(λ1,λ2)\phi= (\lambda_1,\lambda_2). For record i∈I={1,…,N}i \in\mathcal{I} = \{1,\ldots,N\}, let z~i(s)\tilde{z}_i^{(s)} and y^i(s)\hat{y}_i^{(s)} denote its fixed score and prediction at stage s∈{1,2}s \in\{1,2\}. Define

Di(s)(λs)=1[z~i(s)<λs].D_i^{(s)}(\lambda_s) = \mathbf{1}[\tilde{z}_i^{(s)} < \lambda_s].

A record stops at the first stage that accepts it. Stage 2 predictions need only be computed for records that can reach that stage under an admissible threshold.

Execution cost. Let csi≥0c_{si} \ge0 be the fixed processing cost of record ii at stage s∈{1,2,L}s \in\{1,2,L\}, with fixed batching at Stage 1. A token-based cost is csi=psintsiin+psouttˉsioutc_{si} = p_s^{\mathrm{in}}t_{si}^{\mathrm{in}} + p_s^{\mathrm{out}}\bar{t}_{si}^{\mathrm{out}}, where psp_s denotes token prices, tsiint_{si}^{\mathrm{in}} the input length, and tˉsiout\bar{t}_{si}^{\mathrm{out}} an output-length estimate fixed without calibration labels. Thus this objective is a modeled execution cost, not necessarily an observed invoice.

With C1=∑ic1iC_1 = \sum_i c_{1i},

C(ϕ)=Cost⁡(ϕ)=C1+∑iC2iDi(1)+∑iCLiDi(1)Di(2).(1)C(\phi) = \operatorname{Cost}(\phi) = C_1 + \sum_i C_{2i}D_i^{(1)} + \sum_i C_{Li}D_i^{(1)}D_i^{(2)}. \tag*{(1)}

The first threshold controls which records incur second-stage cost; both thresholds jointly determine oracle cost. Per-record costs retain variation in document lengths rather than assuming that routed records have the population-average length.

Oracle-relative error. Let yi=ML(q,di)y_i = \mathcal{M}_L(q,d_i), ai=1[y^i(1)≠yi]a_i = \mathbf{1}[\hat{y}_i^{(1)} \ne y_i], and bi=1[y^i(2)≠yi]b_i = \mathbf{1}[\hat{y}_i^{(2)} \ne y_i]. Since oracle fallback has zero error relative to yiy_i, the population error is

E(ϕ)=1N∑i[(1−Di(1))ai+Di(1)(1−Di(2))bi].(2)E(\phi) = \frac{1}{N}\sum_i\left[(1-D_i^{(1)})a_i + D_i^{(1)}(1-D_i^{(2)})b_i\right]. \tag*{(2)}

Unlike routing and modeled cost, this quantity requires oracle labels.

Theorem 1 (Nonconvexity and partial monotonicity). For fixed scores, predictions, and nonnegative per-record costs, CC and EE are constant on regions inducing the same routing decisions. In general, CC is not convex and {ϕ:E(ϕ)≤ϵ}\{\phi: E(\phi) \le\epsilon\} is not convex. Nevertheless, CC is nondecreasing in either threshold, and, for fixed λ1\lambda_1, EE is nonincreasing in λ2\lambda_2. Error need not be monotone in λ1\lambda_1.

If the grid has no feasible pair, use the all-oracle fallback, whose oracle-relative error is zero. Models, stage order, retrieval, and score functions are not optimization variables in this definition.

Nonconvexity and Threshold Structure

To describe the geometry, first allow thresholds to vary continuously over a rectangle; the actual search restricts them to Φ\Phi. Convexity refers to these continuous functions and their feasible set, rather than to the finite grid itself.

The proof is given in Appendix A–F.

Implications for search. For each fixed λ1\lambda_1, the feasible Stage 2 thresholds form an upper interval, possibly empty, or a suffix of the ascending grid. The smallest feasible grid threshold minimizes cost on that track, possibly with ties. Hence, if population errors were available, enumerating first-stage thresholds and locating each track’s feasibility boundary would suffice. With NN records there are at most (N+1)2(N+1)^2 distinct threshold-induced acceptance configurations, and direct evaluation on a grid of L1L2L_1L_2 candidates takes O(NL1L2)O(NL_1L_2) time. Nonconvexity therefore does not imply hardness of unrestricted cascade assembly for this fixed two-threshold problem. The central difficulty is finding a low-cost feasible pair without labeling the entire population with the oracle.

Selection with Limited Oracle Labels

We draw a simple random sample without replacement (SRSWOR), S⊆IS \subseteq\mathcal{I} with ∣S∣=n|S| = n, and construct a certified set C(S)⊆Φ\mathcal{C}(S) \subseteq\Phi. The population, grid, predictions, scores, and modeled cost are fixed independently of the calibration draw. The sample retains its uniform random draw order. The algorithm returns ϕ^=A(S)∈arg⁡min⁡ϕ∈C(S)C(ϕ)\hat{\phi} = A(S) \in\arg\min_{\phi\in\mathcal{C}(S)} C(\phi), with all-oracle fallback when this set is empty. Its statistical requirement is

Pr⁡S{E(A(S))≤ε}≥1−δ.(3)\Pr_{S}\{E(A(S)) \leq\varepsilon\} \geq1-\delta. \tag*{(3)}

This requirement concerns the sample-dependent selection algorithm. For a fixed threshold pair, population error is deterministic. Certification may exclude truly feasible pairs, so the minimum over C(S)\mathcal{C}(S) need not equal the optimum in Definition 1. We establish selection validity and minimum cost within the certified set, not an approximation ratio or recovery of ϕ∗\phi^*. The monotonicity of true error motivates tracks; it does not itself imply monotonicity of the sample-based certification statistic or justify binary search with repeated tests.

The guarantee is relative to the oracle. For benchmark truth f∗f^*,

Acc⁡f∗(ϕ)≥Acc⁡ML(ϕ)−Δ,(4)\operatorname{Acc}_{f^*}(\phi) \geq\operatorname{Acc}_{\mathcal{M}_L}(\phi) - \Delta, \tag*{(4)}

where Δ=N−1∑i1[yi≠f∗(q,di)]\Delta= N^{-1}\sum_i \mathbf{1}[y_i \ne f^*(q,d_i)]. Thus a ground-truth guarantee requires an upper bound on oracle disagreement; a measured signed accuracy gap alone does not provide one.

Selected-plan cost excludes search overhead. Appendix A-§ 9.11 details same-batch accounting and prediction reuse.

tbl:V in the appendix summarizes the notation.

Calibration Principle

The deployment corpus is a fixed finite population; only the calibration sample is random. We use the Waudby-Smith and Ramdas (WS&R) betting framework [10] to test cascade error without labeling the entire corpus. Because labels are sampled without replacement, the test accounts for the remaining population after each draw. Section IV explains how these tests remain valid when selecting among threshold pairs; Appendix A-I gives the construction.

CascadeFilter: System Design

Overview

CascadeFilter separates preparation, threshold selection, and execution (Algorithm 1). Preparation computes and caches the two small-model predictions and routing scores. Selection uses one oracle-labeled sample to find a low-cost certified threshold pair. Execution returns the first accepted small-model answer, or the oracle answer if both stages reject. The models, retrieval rules, and scores are fixed before calibration; only the two thresholds are selected.

Algorithm 1 CascadeFilter: Prepare, Select, and Execute
Require: Documents, predicate, fixed models and score rules;
         τ, δ, n
Ensure: One prediction per record
 1: Extract evidence; cache Stage 1 predictions and scores.
 2: Form Λ₁; identify records any candidate can defer.
 3: Cache Stage 2 predictions and scores for these records.
 4: Form Λ₂; freeze the grid Φ and candidate costs.
 5: Sample n records without replacement in uniform random
    order; obtain oracle labels.
 6: Select φ̂ with Algorithm 2.
 7: if no pair certifies then
 8:     Return oracle answers for all records; stop.
 9: end if
10: for each record i do
11:     if Stage 1 score ≥ λ₁ then
12:         Return its cached Stage 1 prediction.
13:     else if Stage 2 score ≥ λ₂ then
14:         Return its cached Stage 2 prediction.
15:     else
16:         Return the oracle label, reusing it if already cached.
17:     end if
18: end for

Three Processing Stages

Stage 1: short evidence and the small model. We split a document into overlapping sentence windows, combine BM25 and dense retrieval using RRF, and rerank the candidates to retain compact evidence. Several query–evidence pairs are processed in one small-model call. Each record receives its own prediction and score. A record is accepted when its score reaches λ1\lambda_1; malformed outputs are deferred. Window sizes, batching, and score extraction are specified in Appendix A-G.

Stage 2: full document and the small model. A record rejected by Stage 1 is reconsidered with the complete document. It is accepted if its new score reaches λ2\lambda_2. During search, we precompute this result for every record that any candidate first-stage threshold could defer. All candidate pairs reuse these predictions; changing a threshold does not rerun the model.

Stage 3: oracle fallback. A record rejected by both stages is answered by the oracle. Calibration labels are reused when available. This stage has zero error relative to the oracle by definition, but the oracle can disagree with ground truth. Modeled selected-plan cost and actual same-batch cost with prediction reuse are distinguished in Eq. (12).

Routing Scores

The routing score combines the model’s own confidence with agreement among nearby records in embedding space. Similar neighbors with matching predictions increase the score; disagreement lowers it. Sparse or distant neighborhoods receive less weight. This is a heuristic for ranking records by reliability, not an accuracy guarantee. The exact weighting and log-odds formula appear in Appendix A-H.

Figure 2. Three-stage overview: evidence and full-document SLM stages precede oracle fallback. Dollar values and routing counts are illustrative, not measured. Panel C adapts scores per record; the selected thresholds are shared. Calibration selects the least-cost certified pair, with oracle-relative validity under Theorem 3; it need not certify every grid point.

The statistical procedure only requires a fixed scalar score and acceptance rule z~i(s)≥λs\tilde{z}_i^{(s)} \ge\lambda_s. Neighbor predictions may be used to build the score, but calibration oracle labels may not. Keeping scores fixed preserves the nested acceptance sets needed by threshold search.

Joint Threshold Selection

We build a finite grid from score quantiles and evaluate complete threshold pairs. Changing the first threshold changes which records reach the second stage, so independent stage targets need not minimize total cost. Section IV tests the combined cascade error, then chooses the cheapest certified pair. A pair may be truly feasible yet fail certification; the method therefore does not claim to recover the true feasible optimum.

A Threshold-Selection Example

Consider a record whose fixed routing scores are z~(1)=0.85\tilde{z}^{(1)} = 0.85 and z~(2)=0.90\tilde{z}^{(2)} = 0.90. For illustration, suppose its stage costs are c1=1c_1 = 1, c2=3c_2 = 3, and cL=20c_L = 20 in arbitrary units. The same cached predictions support three different execution paths:

Threshold pairAccepted atPlan cost
(0.80, 0.85)Stage 11
(0.90, 0.85)Stage 21+3=41 + 3 = 4
(0.90, 0.95)Oracle1+3+20=241 + 3 + 20 = 24

These costs are illustrative. Lowering the first threshold can avoid both later stages; changing the second affects only records that reach Stage 2. Candidate cost sums these path costs over the population.

Additional interpretation of this example appears in Appendix A-L.

We certify threshold pairs from one ordered calibration sample and select the cheapest certified pair. Certification takes O(∣Φ∣n)O(|\Phi|n) arithmetic operations on cached predictions; Theorem 4 includes candidate-cost computation. The guarantee applies after selection and concerns oracle-relative error ε=1−τ\varepsilon= 1-\tau. All-oracle fallback is feasible but may be expensive.

For ϕ=(λ1,λ2)\phi=(\lambda_1,\lambda_2) write Ai(s)=1[z~i(s)≥λs]A_i^{(s)}=\mathbf{1}[\tilde{z}_i^{(s)}\geq\lambda_s] and Di(s)=1−Ai(s)D_i^{(s)}=1-A_i^{(s)} for the stage-ss accept and defer indicators. A record errs only if an SLM stage accepts it with a label differing from the oracle label yi=ML(qi,di)y_i=\mathcal{M}_L(q_i,d_i), Stage 3 being zero-loss by definition; the three paths are mutually exclusive, so

ei(ϕ)=Ai(1)1[y^i(1)≠yi]+1[i∈I2max⁡]Di(1)Ai(2)1[y^i(2)≠yi]∈{0,1}.(5)e_i(\phi)=A_i^{(1)}\mathbf{1}[\hat{y}_i^{(1)}\ne y_i] +\mathbf{1}[i\in\mathcal{I}_2^{\max}]D_i^{(1)}A_i^{(2)}\mathbf{1}[\hat{y}_i^{(2)}\ne y_i]\in\{0,1\}. \tag*{(5)}

Writing E(ϕ)=1N∑iei(ϕ)E(\phi)=\frac{1}{N}\sum_i e_i(\phi) for the population error rate, we have E(ϕ)≤ε⟺Acc⁡ML(ϕ)≥τE(\phi)\leq\varepsilon\Longleftrightarrow\operatorname{Acc}_{\mathcal{M}_L}(\phi)\geq\tau, so bounding E(ϕ)E(\phi) certifies accuracy. The two routing rates that price a candidate, rreach2(ϕ)=1N∑i∈I2max⁡Di(1)r_{\mathrm{reach2}}(\phi)=\frac{1}{N}\sum_{i\in\mathcal{I}_2^{\max}}D_i^{(1)} and r3(ϕ)=1N∑i∈I2max⁡Di(1)Di(2)r_3(\phi)=\frac{1}{N}\sum_{i\in\mathcal{I}_2^{\max}}D_i^{(1)}D_i^{(2)}, are computable exactly on the full population without any oracle query.

Bounding a Single Candidate

For a fixed threshold pair, compute its disagreement with the oracle on the calibration sample. We use a without-replacement betting test [10], denoted Mn+(ε;ϕ)M_n^+(\varepsilon;\phi), to test whether population error exceeds ε\varepsilon. A candidate passes at level β\beta when Mn+(ε;ϕ)≥1/βM_n^+(\varepsilon;\phi)\geq1/\beta. Larger evidence is required for smaller β\beta. Appendix A-I specifies the statistic and its residual-population centering.

Theorem 2 (Single-candidate coverage). Condition on the fixed population and candidate family. For any candidate ϕ\phi with E(ϕ)>εE(\phi) > \varepsilon, under the uniformly ordered draw and finite predictable bets of Appendix A-I, P(Mn+(ε)≥1/β)≤β\mathbb{P}(M^{+}_{n}(\varepsilon) \ge1/\beta) \le\beta. Hence the probability of certifying a candidate whose accuracy falls below τ\tau is at most β\beta.

The proof is given in Appendix A-A.

Searching the Grid under Selection

Because ϕ^\hat{\phi} is chosen by its own calibration outcome, per-candidate bounds do not suffice: the search must control the risk of selecting an infeasible configuration. Rather than pay a flat multiplicity price we exploit the structure of Φ\Phi. The error rate is monotone along λ2\lambda_2 but not along λ1\lambda_1, since relaxing λ1\lambda_1 moves records to the chunk view, which may raise or lower their error depending on the predicate’s locality. We therefore organize Φ\Phi into L1L_1 tracks, one per λ1\lambda_1, ordering λ2,1>⋯>λ2,L2\lambda_{2,1} > \cdots> \lambda_{2,L_2} within each.

Lemma 1 (Monotonicity along a track). Fix λ1\lambda_1 and let Em=E(λ1,λ2,m)E_m = E(\lambda_1,\lambda_{2,m}). Then E1≤⋯≤EL2E_1 \le\cdots\le E_{L_2}.

The proof is given in Appendix A-B.

A pre-specified order and stopping at the first failed test give a valid fixed-sequence scan: test from the most stringent λ2,1\lambda_{2,1} toward the most lenient, certify while the test passes, and stop at the first failure, spending the same level ηf\eta_f at every step. Across tracks we split δ\delta by Bonferroni, ηf=δ/L1\eta_f = \delta/L_1. The certified set C\mathcal{C} is the union of the L1L_1 certified prefixes. Monotonicity gives this ordering its cost–error interpretation; the first-error proof itself only requires a fixed order and stopping rule.

Lemma 2 (Certified-set validity). For a fixed family of L1L_1 nonempty tracks, fixed within-track orders, valid marginal tests, and stopping at the first failed test, with ηf=δ/L1\eta_f = \delta/L_1, P(∃ϕ∈C:E(ϕ)>ε)≤δ\mathbb{P}(\exists\phi\in\mathcal{C}: E(\phi) > \varepsilon) \le\delta.

The proof is given in Appendix A-C.

The error-budget saving is that one false-certification event, at the first infeasible index mk⋆m_k^{\star}, covers an entire track: the search spends δ/L1\delta/L_1 per track instead of δ/(L1L2)\delta/(L_1L_2) per candidate, so refining Λ2\Lambda_2 costs no confidence budget at all — the reason the joint search can afford a fine grid. Flat grid-wide Bonferroni remains valid with smaller per-test levels. Fixed-sequence validity does not require monotone errors, but nonmonotone tracks can lose feasible candidates after an early failure. Score parameters ψs\psi_s extend the grid to Λ1×Λ2×Ψ\Lambda_1 \times\Lambda_2 \times\Psi with L1→L1∣Ψ∣L_1 \to L_1|\Psi| throughout; as the budget scales as 1/∣Ψ∣1/|\Psi| we keep ∣Ψ∣≤3|\Psi| \le3.

Algorithm 2 Certified Joint Threshold Search
Require: Grid Φ\Phi, each track ordered λ2,1>⋯>λ2,L2\lambda_{2,1} > \cdots> \lambda_{2,L_2}; sample S∼SRSWOR⁡(I,n)\mathcal{S} \sim\operatorname{SRSWOR}(\mathcal{I},n) with oracle labels {yi}i∈S\{y_i\}_{i \in\mathcal{S}}; ε=1−τ\varepsilon= 1-\tau; failure budget δ\delta; label-free Cost⁡(⋅)\operatorname{Cost}(\cdot)
Ensure: Certified set C\mathcal{C}, deployed ϕ^\hat{\phi}
1: C←∅\mathcal{C} \leftarrow\varnothing; ηf←δ/L1\eta_f \leftarrow\delta/L_1 ▷ Bonferroni across L1L_1 tracks
2: for each track k=1,…,L1k = 1,\ldots,L_1 (one per λ1,k\lambda_{1,k}) do
3: &nbsp;&nbsp;for m=1,…,L2m = 1,\ldots,L_2 (stringent →\to lenient) do
4: &nbsp;&nbsp;&nbsp;&nbsp;W←Mn+(ε;λ1,k,λ2,m)W \leftarrow M_n^{+}(\varepsilon;\lambda_{1,k},\lambda_{2,m}) ▷ O(n)O(n) sweep, Thm. 2
5: &nbsp;&nbsp;&nbsp;&nbsp;if W<1/ηfW < 1/\eta_f then
6: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;break ▷ fixed sequence, Lem. 1
7: &nbsp;&nbsp;&nbsp;&nbsp;end if
8: &nbsp;&nbsp;&nbsp;&nbsp;C←C∪{(λ1,k,λ2,m)}\mathcal{C} \leftarrow\mathcal{C} \cup\{(\lambda_{1,k},\lambda_{2,m})\}
9: &nbsp;&nbsp;end for
10: end for
11: ϕ^←arg⁡min⁡ϕ∈CCost⁡(ϕ)\hat{\phi} \leftarrow\arg\min_{\phi\in\mathcal{C}}\operatorname{Cost}(\phi), or full Stage-3 fallback if C=∅\mathcal{C} = \varnothing
12: return C,ϕ^\mathcal{C},\hat{\phi} ▷ P(Acc⁡ML(ϕ^)≥τ)≥1−δ\mathbb{P}(\operatorname{Acc}_{\mathcal{M}_L}(\hat{\phi}) \ge\tau) \ge1-\delta

Table 2.

End-to-End Guarantee

Theorem 3 (Composable accuracy guarantee). Assume (1) the population, cached predictions, score family, grids, test rules, track orders, and δ\delta-allocation are fixed independently of the calibration draw; (2) 1≤n≤N1 \le n \le N is fixed and S\mathcal{S} retains its uniform without-replacement draw order; (3) Stage 2 covers T2max⁡T_2^{\max} and the tests and scan satisfy Theorem 2 and Lemma 2; and (4) the selected pair minimizes cost in C\mathcal{C}, with all-oracle

fallback if C=∅\mathcal{C} = \varnothing. Oracle fallback must return the same fixed labels yiy_i that define the target. Then P(Acc⁡ML(ϕ^)≥τ)≥1−δ\mathbb{P}(\operatorname{Acc}_{\mathcal{M}_L}(\hat{\phi}) \ge\tau) \ge1-\delta, and when C≠∅\mathcal{C} \ne\varnothing, ϕ^\hat{\phi} is the cheapest certified configuration.

The proof is given in Appendix A-D.

The guarantee applies after threshold selection and is relative to the oracle. The score determines which candidates can save cost; certification determines whether there is sufficient evidence to accept them. Extensions to other loss functions are discussed in Appendix 1.

Cost-optimal selection. (1) is a weighted sum of routing indicators, computable without oracle labels using fixed per-record cost estimates. After certification, scanning C\mathcal{C} returns its least-cost member. This does not establish minimum cost over all feasible members of Φ\Phi: finite-sample conservatism and track stopping may omit feasible candidates. Costs need not be proportional to oracle-call counts when record lengths differ. Same-batch execution with reused predictions requires the separate accounting in (12).

Theorem 4 (Computational complexity). Let NN be the population size, n≤Nn \le N the calibration size, and G=L1L2≥1G = L_1L_2 \ge1 the number of threshold pairs for one fixed score family. Suppose cached per-record routing and cost contributions can be evaluated in constant time and each betting update takes constant time. Using sorting to construct the quantile grids and a direct population scan to price each candidate, Algorithm 1 requires

O(Nlog⁡N+G(N+n))O(N \log N + G(N+n))

arithmetic operations and O(N+G)O(N+G) auxiliary scalar storage. Its certification phase alone costs O(Gn)O(Gn) time. These bounds exclude retrieval, score construction, model inference, and storage of documents, embeddings, and model parameters. If RR is the deployed oracle residual, caching requires ∣S∪R∣≤min⁡{N,n+∣R∣}|\mathcal{S} \cup R| \le\min\{N,n+|R|\} oracle evaluations in total.

The proof is given in Appendix A-E. The total runtime additionally includes preprocessing and model execution, whose cost depends on document lengths, retrieval backbones, and batching. The algorithm evaluates Stage 1 on NN records and Stage 2 on at most NN records; these predictions are reused across all GG candidates. Thus the grid search multiplies scalar computation, rather than requiring a new model-inference pass per candidate. The theorem is an upper bound for the stated direct-scan algorithm, not an NP-hardness claim for the restricted threshold problem.

Scope and assumptions. The single-point certificate is conservative, and the claim holds within the family and procedure frozen before calibration (Theorem 3, assumption 1): CascadeFilter returns the least-cost certified candidate, not a global optimum. Exact r3,rreach2r_3,r_{\mathrm{reach}2} need one small-model Stage-2 pass over I\mathcal{I}; its computation is incurred once, and its cached predictions must not be charged again as selected-plan invocations. This distinction is explicit in Eq. (12). Every statement is finite-population over the specific I\mathcal{I}, with no i.i.d. assumption; a stream is handled by recalibrating per window from a fresh draw.

The theorem concerns one calibration run. Simultaneous validity across multiple deployments requires an additional allocation of failure probabilities whose sum is bounded by the overall budget.

Experiments

Our evaluation addresses five research questions (RQs). RQ1 (accuracy and cost): how do the reported operating points compare across methods? RQ2 (cascade structure): how does removing either small-model stage affect the measured results? RQ3 (components): how do Neighbor Support and evidence retrieval affect those operating points? RQ4 (calibration): what do repeated draws reveal about certificate behavior? RQ5 (sensitivity): how do the target accuracy, failure budget, sample size, and model backbones affect the selected policy? The results below are reported measurements; unresolved run-provenance and accounting issues are stated in § 5-E and V-H.

Experimental Setup

Datasets and predicates. We evaluate CascadeFilter on six datasets spanning 13 evaluation settings and three document genres (Table I), each classified along the two axes the design targets. Locality (Loc.) asks whether a single retrievable span decides the predicate (H), whether the answer is local but needs corroboration (M), or whether evidence is spread across the document (L); this is the axis Stage 1 exploits (Figure 1a). Balance (Bal.) marks a rare positive class (skew), the regime an accuracy-only target leaves unpoliced. Both labels are assigned a priori, from the predicate wording alone and before any run. Since most public suites ship only a handful of fixed queries, we build the Yelp benchmark ourselves: 50 hand-authored aspect queries over 2,000 cleaned Yelp texts form 100K (q,d)(q,d) records. The finite-population argument permits certification over this pooled population without a homogeneity assumption. Ground truth is the consensus of

Figure 1. Motivating examples: (a) compact evidence, (b) token-aware cost, and (c) routing by record difficulty. The labels illustrate these scenarios, not limitations of all prior cascades; numerical values are illustrative. Panel (c) motivates record-dependent scores, while the selected thresholds are shared across records.

IDPredicateLoc.Bal.
Yelp (ours)
Yelp50 aspect templates (e.g., complaints about wait time)M–
Codebase [12]
CB-Q1Does the user’s ⟨\langleAboutMe⟩\rangle contain a social-media link?Hskew
CB-Q2Does ⟨\langleAboutMe⟩\rangle indicate experience in or interest in computer science?M–
CB-Q3Does ⟨\langleAboutMe⟩\rangle focus on factual identification details?L–
IMDB-Review [13]
RV-Q1Is the overall review sentiment positive?M–
RV-Q2Does the review explicitly recommend that others watch the movie?H–
RV-Q3Does the review focus on factual descriptions, such as technical details?Lskew
AirDialogue [14]
AD-Q1Does the dialogue have outcome ⟨\langlebook⟩\rangle?M–
AD-Q2Does the dialogue have outcome ⟨\langlecancel⟩\rangle?Mskew
AD-Q3Does the dialogue have outcome ⟨\langleno_flight⟩\rangle?M–
AD-Q4Does the dialogue have outcome ⟨\langleno_reservation⟩\rangle?M–
Twitter Hate [15]
TCDoes the tweet contain hate speech or offensive language?M–
FEVER [16]
FEVERIs the claim supported by the provided evidence?M–

Table I. Evaluation workload (13 settings; Yelp aggregates 50 aspect templates). Non-Yelp predicate definitions follow [11]. Loc.: qualitative a priori evidence locality (H/M/L), not a measured context requirement. Bal.: class-balance annotation; skew: rare positive class; –: not flagged as rare-positive.

three independent labelers (Claude Sonnet 4.6, ChatGLM 5.2, DeepSeek-V4-Pro), keeping only records on which all three agree; on a random 1K sample these match human annotation at Cohen’s κ=0.94\kappa= 0.94 (98.5% raw agreement). The cited workload uses model-generated reference labels for Codebase and RV-Q2/Q3; their provenance in our runs needs confirmation. The public code supplies a three-record example, not the full benchmark or experiment traces.

Baselines. We compare seven baselines. Full SLM applies the small model to every record. LOTUS [1] uses proxy-based semantic execution; BARGAIN [6] selects a proxy-oracle routing threshold with quality guarantees. UniVote/SimVote denote the uniform and similarity-weighted voting variants of Clustering-Sampling-Voting (CSV) [11], which infer tuple labels from sampled cluster members and support semantic filtering. ScaleDoc [7] trains a query-specific proxy on pre-computed document representations. Task Cascades +G adds accuracy-guaranteed threshold adjustment to an assembled task cascade [8]. BARGAIN supports accuracy, precision, and recall variants, and Task Cascades +G provides an oracle-relative accuracy guarantee. These targets differ from the benchmark-label metrics reported here. The reported comparisons use common small and oracle models at one operating point per method; baseline-specific parameters, prompts, label budgets, and source revisions still require verification from run manifests. The tables therefore describe these operating points and do not establish performance at matched accuracy or dominance over each baseline’s attainable tradeoff curve.

Models and calibration. MS\mathcal{M}_S uses Qwen3-4B-Instruct-2507 [17] (cSin=cSout=$0.2c_S^{\mathrm{in}}=c_S^{\mathrm{out}}=\$0.2 per million tokens) for inference, Qwen3-Embedding-4B [18] for retrieval, and Qwen3-Reranker-0.6B [18] as the cross-encoder, reported as running on a single A100-80GB graphics processing unit (GPU). Stage 3 dispatches to the frontier API model ML\mathcal{M}_L, GPT-5.5 [19] (cLin=$5c_L^{\mathrm{in}}=\$5, cLout=$30c_L^{\mathrm{out}}=\$30 per million tokens), 25–150× costlier per token than MS\mathcal{M}_S, with thinking mode disabled to minimize its cost. The self-hosted SLM is assigned a nominal rate of $0.2 per million input or output tokens. This normalizes the token-cost comparison; it is not a measured GPU operating cost or a verified provider tariff. The stated oracle rates match its documented standard text-token rates. Temperature is set to zero; the theorem additionally requires fixed potential outputs independent of calibration sampling (Appendix A-I). Two structural hyperparameters are fixed across all 13 settings and never tuned per predicate: the Stage-1 batch token budget Bmax⁡=8,192B_{\max}=8{,}192 (Section A-G3), packing 20–40 records per call at the observed evidence lengths, and the neighborhood size K=20K=20 (Section III-C). Accuracy and Macro-F1 are against benchmark reference labels; the oracle is invoked only for Stage 3 fallback and for labeling SS. Unless noted, all main-table results use τ=0.9\tau=0.9, δ=0.05\delta=0.05, and n=max⁡{ROUND_HALF_UP⁡(0.01N),n0(N,τ,δ)}n=\max\{\operatorname{ROUND\_HALF\_UP}(0.01N),n_0(N,\tau,\delta)\}, with the necessary zero-error feasibility floor n0n_0 discussed in Section V-E.

Certification grid. The grids are empirical quantiles of the frozen routing scores, formed before SS: up to 32 points for Λ1\Lambda_1 and 64 for Λ2\Lambda_2 (duplicates removed), so ∣Φ∣≤2048|\Phi|\leq2048, with ∣Ψ∣=1|\Psi|=1 and ηf=δ/L1\eta_f=\delta/L_1 per track. Validity holds at any resolution; resolution and δ\delta-allocation shift only how many lenient configurations certify, hence the realized r3r_3. Label budgets run from 119 to 3,618, so the smallest corpora bind on power (Section V-E). For certificate validation, we repeat calibration and deployment over 20 independent draws of SS per setting, using the default calibration-size rule.

Cost accounting. The reported cost model follows Eq. (1), using input-token and invocation counts and mean output lengths. The cost in Figure 3 is described as including all three stages, calibration labeling, the label-free Stage-2 search pass over II (Section IV-C), and embedding, BM25, reranking, and indexing compute. These reported totals require reconciliation with Eq. (12) to verify that cached Stage-2 predictions and reused calibration labels are not charged twice; they should not yet be interpreted as audited end-to-end bills. The reported equal-weight mean cost across the 13 settings is $2.968×10−4\$2.968\times10^{-4} per predicate–record. The token panel counts only oracle tokens.

Figure 3. Reported nominal cost, oracle calls, and oracle-token usage per predicate–record across 13 settings; cost accounting remains unaudited; AVG denotes the equal-weight arithmetic mean across the 13 settings.

Main Results (RQ1)

Table II reports Accuracy and Macro-F1 across all 13 evaluation settings. CascadeFilter has the highest reported Accuracy and Macro-F1 in 12 of 13 settings, averaging 0.938/0.887; the only losses are on AD-Q4 (BARGAIN 0.902/0.866 vs. our 0.901/0.846), explained below. CascadeFilter’s benchmark Accuracy standard deviations are 0.013 on AD-Q2 and 0.006 on AD-Q4 (Table III). The point differences on AD-Q2 (0.907 vs. 0.903 against Task Cascades) and AD-Q4 (0.901 vs. 0.902 against BARGAIN) are small. Per-draw spreads for those baselines are not reported, so these comparisons alone do not establish statistical significance. The comparisons below describe the table without attributing differences to a single mechanism.

Accuracy and oracle routing. On CB-Q3, Full SLM reaches 0.614 accuracy and the cascade reaches 0.951, a difference of 33.7 percentage points (pp), with 29.9% of records assigned to Stage 3. The cascade also changes the input to Stage 1 and its routing scores, so this difference cannot be attributed to oracle calls alone. Stage 3 routing is highest on RV-Q3 (31.2%) and lowest on CB-Q1 (2.5%). The corresponding Full-SLM accuracies are 0.736 and 0.766; Full SLM is not already near ceiling on CB-Q1. The stage-level error breakdown needed to isolate these effects is not reported.

Routing and annotated evidence locality. Table I labels CB-Q1 and RV-Q2 as high-locality (H), CB-Q3 and RV-Q3 as low-locality (L), and the remaining settings as medium-locality (M). From Table III, their mean Stage-1 acceptance shares are 69.1%, 37.7%, and 59.4%, respectively. These descriptive averages are consistent with greater early acceptance for more local predicates, but do not isolate a causal effect. CB-Q1 asks about social-media links, and RV-Q2 asks whether a review recommends the movie. RV-Q1 and AD-Q1 are medium-locality settings in the workload definition. The oracle-routing rates range from 2.5% to 31.2%, with an equal-weight mean of 14.6%.

Differences at the reported operating points. UniVote and Sieve reach 0.726/0.733 Accuracy on AD-Q2 and 0.729/0.739 on AD-Q4. BARGAIN reaches 0.898 on AD-Q3, compared with 0.919 for CascadeFilter. These comparisons do not identify the source of the differences: the systems differ in label propagation, calibration, and routing, in addition to their scores. Across these 13 settings, CascadeFilter’s lowest reported Accuracy is 0.901 on AD-Q4. This observation does not establish robustness on other predicates or prove that Neighbor Support causes the improvement; the component-removal study provides the more direct comparison.

Accuracy and class imbalance. The certificate controls overall oracle-relative error. It does not guarantee Macro-F1, precision, recall, or minority-class performance; Appendix 1 explains why these require additional analysis. The three settings marked skew in tbl:1 are CB-Q1 (0.975/0.905 Accuracy/Macro-F1), RV-Q3 (0.947/0.809), and AD-Q2 (0.907/0.594). Their metric gaps illustrate why overall Accuracy alone is insufficient to describe class-wise performance. AD-Q4 is not marked skew in that table. It is the one setting where BARGAIN has higher reported Accuracy and Macro-F1 than CascadeFilter (0.902/0.866 versus 0.901/0.846). No significance claim follows from these point estimates.

Reported cost and quality. Figure 3 reports nominal cost, oracle-call counts, and oracle-token counts; Figure 4 groups the measured operating points by dataset. The oracle has a higher assigned per-token price, but its contribution also depends on the lengths of the routed records. Full SLM, UniVote, and SimVote have lower reported cost and lower mean benchmark Accuracy (0.716, 0.738, and 0.742). These benchmark-label comparisons cannot determine whether an oracle-relative constraint is met. CascadeFilter has higher reported mean Accuracy than the other baselines, but the cost ratios still require a common accounting ledger for preprocessing, calibration, caching, and deployment. We therefore do not infer a matched-accuracy advantage, an audited end-to-end saving, or a Pareto-frontier dominance result from these plots. The modeled-cost selection theorem concerns candidates within our fixed cascade, not optimality over other systems.

Accuracy–cost tradeoff

Figure 4. Accuracy–cost tradeoff, each point a method’s mean over the settings in a dataset group. The Accuracy axis increases to the left, so the lower-left corner (high accuracy, low cost) is best.

(a) Accuracy
MethodYelpCB-Q1CB-Q2CB-Q3FEVERTCRV-Q1RV-Q2RV-Q3AD-Q1AD-Q2AD-Q3AD-Q4Avg
Full SLM0.7430.7660.6790.6140.6940.6250.8260.7200.7360.7490.7310.7040.7200.716
LOTUS0.8140.9300.7710.7170.8040.7280.9050.7990.8460.8200.8330.8460.8000.816
BARGAIN0.8950.9280.8780.8750.8920.8820.9130.8850.9010.8750.8960.8980.9020.894
ScaleDoc0.8400.8810.8480.8040.8380.8490.8410.7680.8700.8000.8200.7520.8230.826
UniVote0.7380.8560.7000.6330.6680.6680.8090.6890.8510.8210.7260.7120.7290.738
SimVote0.7570.8730.7040.6610.6660.6990.7910.6640.8140.8240.7330.7190.7390.742
Task Cascades0.8980.9120.9080.8770.8990.8870.9110.9150.8910.8990.9030.8960.8990.900
CascadeFilter0.9430.9750.9560.9510.9430.9310.9550.9370.9470.9310.9070.9190.9010.938
(b) Macro-F1
MethodYelpCB-Q1CB-Q2CB-Q3FEVERTCRV-Q1RV-Q2RV-Q3AD-Q1AD-Q2AD-Q3AD-Q4Avg
Full SLM0.6290.6120.6470.5800.6870.5370.8230.6780.5190.7360.4680.6650.6140.630
LOTUS0.7540.7740.7690.7010.8020.6530.9050.7810.6300.8200.5220.8070.6870.739
BARGAIN0.8840.7760.8760.8540.8920.7840.9130.8680.6310.8750.5430.8650.8660.817
ScaleDoc0.8240.6720.8470.7360.8380.7320.8400.7100.5920.8000.5120.7070.7790.738
UniVote0.6230.5340.6980.6130.6560.5530.8080.6740.4870.8190.4670.6370.6160.630
SimVote0.6490.4940.7000.6410.6530.5980.7900.6480.5060.8230.4710.6420.6260.634
Task Cascades0.8890.7410.9060.8460.8990.7890.9100.9030.7040.8980.5790.8620.8650.830
CascadeFilter0.9340.9050.9560.9410.9430.8920.9550.9260.8090.9310.5940.8960.8460.887

Table 2 (PDF p. 9). Accuracy and Macro-F1 across 13 evaluation settings. Avg denotes the equal-weight arithmetic mean across the 13 settings. The best value in each column is highlighted.

Cascade-Structure Ablation (RQ2)

Each variant changes which decisions are available to the cascade and is described as being recalibrated at the same oracle-relative target. Figure 5(a,Figure 5b) reports the resulting benchmark Accuracy and nominal cost. Removing Stage 1 changes the full method’s mean Accuracy from 0.938 to 0.923 and its cost from 2.9682.968 to 5.570×10−45.570 \times10^{-4} per predicate–record (1.88×\times). Removing Stage 2 gives Accuracy 0.914 and cost 4.871×10−44.871 \times10^{-4} (1.64×\times). These measurements support retaining both stages for this workload and configuration. They do not prove that both stages are necessary for every predicate, or that a ground-truth Accuracy above 0.9 validates the oracle-relative certificate.

Figure 5. Component-removal ablation: (a,b) pipeline and search variants; (c,d) routing-score variants. The panels report benchmark Accuracy and nominal cost; averages are equal-weight arithmetic means across settings.

Component Ablation (RQ3)

Two components change the selected operating point: Neighbor Support (NS) and evidence retrieval. Figure 5(c,Figure 5d) reports Accuracy 0.901 without NS, versus 0.938 for the full method, and nominal cost 6.733 ×10−4\times10^{-4} versus 2.968 ×10−4\times10^{-4}.

SettingNNS1S2S3AccAccML\mathrm{Acc}_{\mathrm{ML}}Δ^\hat{\Delta}VVMgn
Yelp100K66.825.67.6.943 ±\pm .006.940 ±\pm .006-.0020–
CB-Q19.4K86.810.72.5.975 ±\pm .011.976 ±\pm .011.0010–
CB-Q29.4K61.232.76.1.956 ±\pm .004.963 ±\pm .004.0070–
CB-Q39.4K38.431.729.9.951 ±\pm .007.953 ±\pm .007.0010–
FEVER165.4K52.126.521.4.943 ±\pm .010.944 ±\pm .010.0020–
TC24.8K45.133.221.7.931 ±\pm .009.937 ±\pm .010.0060–
RV-Q150K82.613.63.8.955 ±\pm .004.959 ±\pm .004.0030–
RV-Q250K51.334.214.5.937 ±\pm .012.938 ±\pm .012.0010–
RV-Q350K36.931.931.2.947 ±\pm .011.948 ±\pm .011.0010–
AD-Q1361.8K83.712.83.5.931 ±\pm .009.933 ±\pm .009.0020–
AD-Q2361.8K51.633.215.2.907 ±\pm .013.907 ±\pm .013.0006–
AD-Q3361.8K56.035.88.2.919 ±\pm .019.922 ±\pm .020.0023–
AD-Q4361.8K35.939.524.6.901 ±\pm .006.910 ±\pm .006.0090–
Summary1.9M57.627.814.6.938 ±\pm .003.941 ±\pm .003.0039/260–

Table 3. Per-setting reference deployment and oracle-agreement profile of CascadeFilter (τ=0.9\tau= 0.9, δ=0.05\delta= 0.05). The routing split describes the reference operating point. Acc is CascadeFilter’s ground-truth accuracy; AccML\mathrm{Acc}_{\mathrm{ML}} is its agreement with the oracle, not the oracle’s own ground-truth accuracy. Both are shown as mean ±\pm sample standard deviation over 20 realizations per setting. Δ^=AccML−Acc\hat{\Delta} = \mathrm{Acc}_{\mathrm{ML}} - \mathrm{Acc} is the signed difference between the means. VV counts oracle-relative target violations among those realizations. Mgn is unsupported without corresponding wealth-test logs. Statistics are computed before rounding. The summary totals NN (predicate–record instances) and VV; its routing split is the unweighted setting mean, and its accuracy statistics use the per-realization macro-averages over settings.

Removing NS changes scores, grids, and subsequently selected thresholds, so its effect on deferral is not determined by the sign of the score adjustment alone. Removing RRF or the reranker gives Accuracy 0.934 or 0.926 and cost 3.326×10−43.326 \times10^{-4} or 3.502×10−43.502 \times10^{-4}, respectively. These are observations for the displayed ablations, not evidence that every extraction or routing replacement improves every record.

The two search ablations in Figure 5(a,Figure 5b) are stage-wise tuning (w/o Joint Search) and flat grid-wide Bonferroni (w/o Fixed-Sequence). Their displayed Accuracy values are 0.906 and 0.908, and nominal costs are 7.866×10−47.866 \times10^{-4} and 7.135×10−47.135 \times10^{-4}: 3.2 and 3.0 pp lower Accuracy and 2.65×2.65 \times and 2.40×2.40 \times higher cost than the full method. A flat correction tests each grid point at δ/(L1L2)\delta/(L_1L_2); fixed-sequence testing uses δ/L1\delta/L_1 and stops each track after its first failed test. The larger per-test budget does not establish uniform power dominance because the stopping rules visit different candidates. The stage-wise variant’s allocation and selection procedure require a separate validity check from its implementation.

Calibration Diagnostics (RQ4)

The results above summarize operating points, but the guarantee concerns the distribution over calibration draws. We repeat calibration and deployment on 20 independent draws of SS per setting at the default τ\tau, δ\delta, and calibration size nn. The intended protocol draws a fresh sample, obtains its fixed oracle labels, reruns Algorithm 2, and evaluates the selected configuration. Re-querying a stochastic oracle would change the target population and would need separate treatment.

Validity. The primary check is the frequency with which oracle-relative accuracy falls below τ\tau. The current routing table records six violations on AD-Q2 and three on AD-Q3, totaling 9/260 (3.46%). Earlier narrative values of 3/260 and one violation per setting conflict with this table and are not used here. In particular, the reported AD-Q2 rate is 6/20; pooled results cannot establish the per-setting guarantee. Per-draw logs are needed to reconcile these counts and assess the implementation before claiming empirical validation of the certificate. Differences of 0.001 between subtracting rounded means and the signed-gap column can arise from independent rounding. They are not, by themselves, contradictions; the unrounded means and per-draw results are needed to verify that column and its uncertainty summaries.

Power and the label floor. Certifying a track requires wealth L1/δ=640L_1/\delta= 640 at L1=32L_1 = 32 and δ=0.05\delta= 0.05. The correct finite-population zero-error envelope is the product in Eq. (11), not a general nn KL bound. For Codebase (N=9378N = 9378), its values are 165.633 at n=94n = 94, 616.076 at n=118n = 118, and 650.788 at n=119n = 119. Thus 119 is a necessary zero-error feasibility floor under the stated betting cap. It does not ensure that the implemented adaptive bet reaches 640: actual bets may lie below the cap. The manuscript reports n0=119n_0 = 119 when the nominal 1% budget is smaller. However, the released code at revision 9a9adc74 yields wealth 463.601 on 119 zero errors and first reaches 640 at 125 zero errors. Its split script also defaults to min⁡(N,max⁡(500,⌈0.01N⌉))\min(N,\max(500,\lceil0.01N\rceil)). These are different rules; the reported runs cannot be identified with that revision without their manifests and wealth traces. Appendix A-I1 gives the derivation and a counterexample to the previous KL claim.

Calibration Sensitivity (RQ5)

Figure 6 varies the accuracy target τ\tau, failure budget δ\delta, and calibration size nn. These parameters play different roles: τ\tau determines the admissible error rate, δ\delta determines the strength of evidence required for certification, and nn determines how many oracle labels are available to provide that evidence.

Figure 6. Sensitivity to the target failure probability δ\delta, target accuracy τ\tau, and calibration ratio nn.

Accuracy target. The reported operating points generally move toward higher accuracy and higher cost as τ\tau increases. A stricter target makes it harder to accept small-model predictions, so more records may reach the oracle. This is a tendency of the selected configurations rather than a claim that every individual routing threshold changes monotonically. Ground-truth accuracy in the plot is a measured outcome; the theorem concerns oracle-relative accuracy.

Failure budget. Across the reported range δ∈[0.005,0.10]\delta\in[0.005, 0.10], the operating points change less than in the accuracy-target sweep. The test threshold depends on log⁡(L1/δ)\log(L_1/\delta), while selected plans change only when a candidate enters or leaves the certified set. A range of failure budgets can therefore select the same plan. This observation does not establish that δ\delta is unimportant or that the certificate has been empirically validated.

Calibration size. A larger sample provides more evidence for certification, but requires additional oracle labels. Consequently, a lower deployment fallback rate need not imply a lower total cost. The reported sweep illustrates this tradeoff; any end-to-end comparison must count calibration labels and cached predictions once, using the accounting in Appendix A-K. We also examine how replacing the small model, embedding model, and reranker changes the selected operating point (§ 5-G).

Model-Backbone Robustness

Measured model-replacement behavior. Table IV replaces the SLM, embedding, and reranking backbones while holding the cascade and operating point (τ=0.9\tau=0.9, δ=0.05\delta=0.05, and nn under the default calibration rule) fixed, re-certifying each replacement. Every backbone achieves measured ground-truth Accuracy of at least 0.901, with the lowest value observed for Qwen3/0.6B and Llama-3.2/1B on RV-Q1. The maximum within-dataset Accuracy spread is 6.1 percentage points. The main difference is in Stage 3 routing. On CB-Q2, Qwen3/0.6B routes 54.1% of records to the oracle, compared with 6.1% for Qwen3/4B. Replacing the default embedding backbone increases the CB-Q2 fallback rate from 6.1% to 12.1–13.8%. The trend is capability-dependent but not strictly monotone: Qwen3/8B reduces fallback on Yelp (6.2% vs. 7.6%) and FEVER (10.3% vs. 21.4%), while increasing it on CB-Q2 (12.7% vs. 6.1%) and RV-Q1 (7.8% vs. 3.8%). Changes in prediction confidence and the resulting quantile

TypeModelYelp A/FYelp S3(%)CB-Q2 A/FCB-Q2 S3(%)FEVER A/FFEVER S3(%)RV-Q1 A/FRV-Q1 S3(%)
SLMQwen3/0.6B [20]0.904/0.89046.80.904/0.90154.10.903/0.90234.20.901/0.90132.7
SLMQwen3/1.7B [20]0.916/0.90330.20.917/0.91436.80.912/0.91217.40.924/0.92412.6
SLMQwen3/4B [20]0.943/0.9347.60.956/0.9566.10.943/0.94321.40.955/0.9553.8
SLMQwen3/8B [20]0.939/0.9326.20.958/0.95712.70.941/0.94110.30.958/0.9587.8
SLMLlama-3.2/1B [21]0.920/0.88849.60.903/0.89960.90.902/0.90142.10.901/0.90129.4
SLMLlama-3.2/3B [21]0.909/0.89539.70.909/0.90645.40.907/0.90631.80.905/0.90528.9
Emb.Qwen3/4B [20]0.943/0.9347.60.956/0.9566.10.943/0.94321.40.955/0.9553.8
Emb.Qwen3/0.6B0.921/0.91011.60.930/0.92813.80.922/0.92223.40.919/0.9195.9
Emb.BGE-M3 [22]0.918/0.90710.90.939/0.93712.90.925/0.92522.60.913/0.9135.2
Emb.E5-L [23]0.924/0.91311.30.935/0.93312.10.922/0.92223.20.924/0.9246.6
Rer.Qwen3/0.6B0.943/0.9347.60.956/0.9566.10.943/0.94321.40.955/0.9553.8
Rer.Qwen3/4B0.948/0.9419.10.961/0.9608.50.951/0.95124.10.962/0.9624.2
Rer.BGE-M3 [22]0.925/0.91511.20.930/0.92713.30.928/0.92825.40.915/0.9154.7

Table IV. Model replacement study. A/F denotes Accuracy/Macro-F1; S3 denotes the Stage 3 fallback rate. Bold marks the best A/F and the lowest S3 within each type. BGE-M3 in the Rer. block is a bi-encoder similarity control without cross-encoder reranking.

grid influence the selected routing thresholds. The theorem remains applicable after recalibration only when the fixed-output and sampling assumptions hold; the displayed ground-truth metrics do not check those assumptions. Overall Accuracy can nevertheless mask differences in Macro-F1: on Yelp, replacing Qwen3/4B with Llama-3.2/1B reduces Macro-F1 from 0.934 to 0.888 while Accuracy changes from 0.943 to 0.902.

Threats to Validity

Oracle as ground-truth proxy. The certificate is, strictly, oracle-relative: theorems bound P(Acc⁡ML(ϕ^)≥τ)≥1−δ\mathbb{P}(\operatorname{Acc}_{\mathcal{M}_L}(\hat{\phi}) \ge\tau) \ge1-\delta, and transfer to f∗f^* uses Eq. (5), which needs an upper bound on oracle–ground-truth disagreement while the absolute signed

accuracy gap supplies only a lower bound (§ 5-E); we clear the GT target by measurement rather than by proof. The current experiment certifies accuracy against ML\mathcal{M}_L, not f∗f^*. Changing calibration labels to ground truth changes the loss at every stage, including oracle fallback. A mean-loss test can still be constructed, but an end-to-end guarantee requires a safe or separately certified fallback, or a no-decision outcome. It does not follow from changing the label source or multiplicity correction alone (Appendix 1).

Benchmark and cost accounting. The Yelp ground truth keeps only records on which three labelers agree, which removes ambiguous cases, making the benchmark easier for every method and affecting the observed oracle-relative gap there: the absolute Yelp numbers and the relative ranking may therefore differ on an unfiltered population. The local-compute portion requires a documented resource and pricing ledger before it can be repriced. Because we compare systems at their delivered operating points rather than sweeping each baseline’s own knob, the plotted cost gaps are not matched-accuracy comparisons or a frontier comparison. The guarantee is over the finite population I\mathcal{I}, so each deployment draws a fresh sample; cross-corpus transfer without recalibration is not covered.

Certification conservatism and operating point. The single-point test is one-sided, so a feasible configuration can fail certification and be dropped from CC; CascadeFilter returns the cheapest certified candidate, not the global optimum (Section IV-C). The reported ground-truth Accuracy of CascadeFilter meets 0.9 in every setting, and Figure 4 summarizes the measured accuracy–cost operating points. Finally, the nominal token costs do not establish runtime, throughput, GPU-hours, or latency savings. Batching changes the invocation pattern; its wall-clock effect requires measurement. Batch composition is fixed by document order before calibration, so it does not interact with the sampling argument, but a throughput-oriented deployment would want the wall-clock accounting we omit. The violation-count discrepancy and cached-cost accounting still require verification against run logs; the present results do not establish empirical validity of the certificate.

Related Work

Semantic filtering and query approximation. Approximate predicate evaluation predates LLMs [24]; recent systems expose LLM-powered operations as declarative primitives: LOTUS [1] introduces semantic operators with statistical guarantees on expected loss against a gold algorithm; Palimpzest [2], DocETL [3], and MOAR [25] optimize analytical pipelines; SUQL [26], ZenDB [27], and UQE [28] query structured and unstructured data; and Abacus [29], TAG [30], and TASTI [31] add cost-based optimization, an operator interface, and semantic indexes. Closer to our cost focus, UniVote/SimVote [11], Fast Semantic Filtering [32], and AI Query Approximation [33] cut LLM calls under an accuracy target, and TRAQ [34] guarantees answer-set coverage; these works provide related approaches to reducing semantic execution cost.

Model routing and cascades. FrugalGPT [4], RouteLLM [5], AutoMix [35], Hybrid LLM [36], SMART [37], ThriftLLM [38], and Universal Model Routing [39] study cost-aware model selection; C3PO [40] and UCCI [41] add probabilistic cost constraints and calibrated uncertainty to cascade routing; SUPG [42] gives proxy-based selection with precision/recall guarantees; Streaming [43] and Task Cascades [8] study streaming and generalized cascades. BARGAIN [6] is closely related: it selects a routing threshold for a cheap-model/oracle cascade and provides task-specific quality guarantees. Our formulation fixes an evidence/full-document/oracle structure and compares pairs of thresholds using per-record token costs. This is a difference in the optimization studied here, rather than a claim that prior cascades cannot account for token prices or accept alternative input representations.

Finite-population calibration. Our guarantee builds on the Waudby-Smith and Ramdas (WS&R) betting framework for bounded means [10]. Conformal Risk Control (CRC) [44] controls expected loss. Selective CRC [45] combines sample selection with risk control, including a calibration-only variant with probably approximately correct (PAC) guarantees, while Adaptive Conformal Inference [46] targets online distribution shift. The closest methodology is Learn-then-Test (LTT) [47], which, like us, selects configurations from a grid under family-wise error control; we differ in the sampling model and in what is being selected. LTT provides a general framework for calibrating collections of configurations and is not limited to scalar rules. We instantiate related multiple-testing tools for a fixed finite deployment population sampled without replacement, using residual-population centering for finite-sample validity (Section IV-A). Our candidates are coupled threshold pairs whose Stage-2 populations depend on λ1\lambda_1 (Section IV-B); the test can be conservative and does not estimate error exactly.

Task Cascades already combines varying document portions with cascade-level accuracy certification. Its guarantee adjustment follows a prescribed threshold sequence after assembly; our fixed three-stage design instead compares certified threshold pairs by token-aware cost. Thus neither shorter inputs nor statistical certification alone distinguishes our contribution.

Conclusion

We presented CascadeFilter, which selects two thresholds for a fixed three-stage semantic filter to reduce modeled execution cost under an oracle-relative error constraint. The optimization is generally nonconvex, but its error is monotone along each fixed-first-threshold track. Our method uses this structure to certify candidates from a finite calibration sample and selects the cheapest certified pair, without claiming recovery of the true feasible optimum. Experiments cover six datasets and 13 settings; the main table reports the highest benchmark accuracy and Macro-F1 in 12 settings each. The reported violation counts and end-to-end cost accounting still require reconciliation. Future work includes other semantic operators and recalibration under distribution shift.

References

Appendix A: Technical Report: Proofs and Supplementary Details

This appendix contains the technical-report material referenced in the main text.

Proof of Theorem 2

Proof. Fix the entire population, its potential model outputs, and all candidate-specific test rules independently of the sample draw. Write θ=N−1∑iei(ϕ)\theta= N^{-1}\sum_i e_i(\phi) and use the filtration and bets in Appendix A-I. Uniform ordered sampling gives Eq. (9); in particular its right-hand side is in [0,1][0,1] on every history of positive probability.

For u≤θu \le\theta, the unclipped quantity (Nu−St−1)/(N−t+1)(Nu-S_{t-1})/(N-t+1) is no larger than μt=E[Xt∣Ft−1]\mu_t = \mathbb{E}[X_t \mid\mathcal{F}_{t-1}]. Clipping to [0,1][0,1] preserves qt(u)≤μtq_t(u) \le\mu_t. For qt<1q_t < 1, the admissible bet ensures 1+κt(qt−Xt)≥1−κt(1−qt)≥1/21+\kappa_t(q_t-X_t) \ge1-\kappa_t(1-q_t) \ge1/2. For qt=1q_t=1, any finite nonnegative bet gives a factor at least one; under the null, μt=1\mu_t=1 and Xt=1X_t=1 almost surely, so the factor equals one. No infinite bet is permitted.

Since κt\kappa_t and qtq_t are measurable before observing XtX_t,

E[Mt+(u)∣Ft−1]=Mt−1+(u){1+κt(u)(qt(u)−μt)}≤Mt−1+(u).\mathbb{E}[M_t^+(u)\mid\mathcal{F}_{t-1}] = M_{t-1}^+(u)\{1+\kappa_t(u)(q_t(u)-\mu_t)\} \le M_{t-1}^+(u).

Thus Mt+(u)M_t^+(u) is a nonnegative supermartingale with initial value one. At the fixed sample size, E[Mn+(u)]≤1\mathbb{E}[M_n^+(u)]\le1. Markov’s inequality at u=ε<θu=\varepsilon<\theta yields Pr⁡(Mn+(ε)≥1/β)≤β\Pr(M_n^+(\varepsilon)\ge1/\beta)\le\beta. This proves the stated fixed-time bound; Ville’s inequality also bounds crossing at any time. If the fixed candidate family is itself random independently of the sampling permutation, the same argument holds conditionally on that family, and averaging preserves the bound.

Proof of Lemma 1

Proof. The scores z~i(2)\tilde z_i^{(2)} are frozen before calibration, so the accept indicator is nested, Ai(2)(λ2,m)≤Ai(2)(λ2,m′)A_i^{(2)}(\lambda_2,m)\le A_i^{(2)}(\lambda_2,m') for m<m′m<m'. With λ1\lambda_1 fixed, the first term of (6) is unchanged and the second equals giAi(2)(λ2)g_iA_i^{(2)}(\lambda_2) with the λ2\lambda_2-free factor gi=1[i∈I2max⁡]Di(1)[1{y^i(1)≠yi}]≥0g_i=1[i\in\mathcal{I}_2^{\max}]D_i^{(1)}[1\{\hat y_i^{(1)}\ne y_i\}]\ge0, so each eie_i is non-decreasing in mm.

Proof of Lemma 2

Proof. Condition on the fixed population, candidate family, orders, and test rules. On track kk, let mk⋆=min⁡{m:Ek,m>ε}m_k^\star=\min\{m:E_{k,m}>\varepsilon\}, if that set is nonempty. This index is fixed with respect to calibration sampling, even though its value is unknown to the algorithm. If there is no such index, that track cannot certify an infeasible candidate.

Otherwise, certifying any infeasible candidate requires certifying the first infeasible candidate mk⋆m_k^\star: every earlier candidate must pass because the scan stops at its first failure. Therefore

{track k certifies an infeasible pair}⊆{Mn+(ε;ϕk,mk⋆)≥1/ηf}.\{\text{track }k\text{ certifies an infeasible pair}\} \subseteq \{M_n^+(\varepsilon;\phi_{k,m_k^\star})\ge1/\eta_f\}.

The latter event has probability at most ηf\eta_f by Theorem 2. Summing over the L1L_1 tracks gives at most L1ηf=δL_1\eta_f=\delta. Independence between tests or tracks is not needed, so all candidates may reuse the same sample. Monotonicity is not needed for this first-error inclusion; it explains why a stringent-to-lenient order is useful for threshold search. Reordering or restarting tests after observing calibration labels is outside this proof.

Proof of Theorem 3

Proof. Let G={∀ϕ∈C:E(ϕ)≤ε}\mathcal{G}=\{\forall\phi\in\mathcal{C}:E(\phi)\le\varepsilon\}, so P(G)≥1−δ\mathbb{P}(\mathcal{G})\ge1-\delta by Lemma 2 under the theorem assumptions. On G\mathcal{G}, either C≠∅\mathcal{C}\ne\varnothing and ϕ^∈C\hat\phi\in\mathcal{C} gives E(ϕ^)≤εE(\hat\phi)\le\varepsilon, or the fallback emits yiy_i on every record and E(ϕ^)=0E(\hat\phi)=0; either way the target holds, so P(Acc⁡M(ϕ^)<τ)≤P(Gc)≤δ\mathbb{P}(\operatorname{Acc}_{\mathcal{M}}(\hat\phi)<\tau)\le\mathbb{P}(\mathcal{G}^c)\le\delta. Because G\mathcal{G} controls every member of C\mathcal{C} simultaneously, validity does not depend on which member the label-free cost rule selects, and the arg min over the finite C\mathcal{C} is attained. □

Proof of Theorem 4

Proof. There are at most NN routing scores at each SLM stage, so sorting them and selecting grid quantiles costs O(Nlog⁡N+G)O(N\log N+G) time, including grid materialization. A direct scan evaluates each candidate’s routing decisions and sums its cost contributions in O(N)O(N) time, giving O(GN)O(GN) for all candidates. Each candidate’s certificate uses at most nn constant-time betting updates, and at most GG candidates are tested. Early stopping within tracks can reduce this work but does not increase its O(Gn)O(Gn) worst-case bound. Selecting the cheapest certified candidate costs O(G)O(G), and producing final routing decisions costs O(N)O(N). Cached scalar scores, predictions, and labels occupy O(N)O(N) space; candidate costs and certification results occupy O(G)O(G). Processing one candidate at a time avoids storing an N×GN\times G error matrix. Uniform ordered sampling can be implemented with a partial permutation in O(N)O(N) time and space. The nonempty-grid condition makes O(N+G)O(N+G) additive work dominated by the stated time bound (including N=1N=1). Finally, calibration labels are queried for SS, and deployment queries only the uncached members of RR, yielding the stated oracle count; for all-oracle fallback take R=IR=\mathcal{I}. The bound counts real-arithmetic operations and excludes arbitrary-precision cost explicitly. An empty grid can instead return the fallback immediately. If the Stage 2 pool is empty, a singleton dummy second-stage grid suffices, since all candidates stop at Stage 1. □

Proof of Theorem 1

Proof. Routing indicators change only when a threshold crosses a score, proving the piecewise-constant property. For nonconvex cost, take one record with z~(1)=0.5\tilde z^{(1)}=0.5, set c2=1c_2=1, and fix λ2\lambda_2 so Stage 2 accepts. After subtracting C1C_1, cost at λ1=0.4,0.8\lambda_1=0.4,0.8 are 0,10,1, whereas their midpoint 0.60.6 has cost 1>1/21>1/2, violating convexity.

For a nonconvex feasible set, take one record with both scores 0.50.5, a=0a=0, b=1b=1, and ε=0.1\varepsilon=0.1. The pair (0.4,0.2)(0.4,0.2) accepts the correct Stage 1 answer; (0.8,0.6)(0.8,0.6) uses the oracle. Both have zero error. Their midpoint (0.6,0.4)(0.6, 0.4) accepts the incorrect Stage 2 answer and has error one, so is infeasible. Increasing λ1\lambda_1 changes a record’s cost from c1ic_{1i} to c1i+c2i+Di(2)cLic_{1i} + c_{2i} + D_i^{(2)}c_{Li}; increasing λ2\lambda_2 can only add oracle cost. Both changes are nonnegative. For fixed λ1\lambda_1, increasing λ2\lambda_2 decreases 1−Di(2)1-D_i^{(2)}, whose coefficient Di(2)biD_i^{(2)}b_i in Eq. (2) is nonnegative, so error cannot increase. In contrast, crossing a Stage 1 score changes error by (1−Di(2))bi−ai(1-D_i^{(2)})b_i-a_i, which can have either sign. For an explicit example, take two records with Stage 1 scores 0.30.3, 0.70.7, Stage 2 scores both 0.80.8, a=(0,1)a=(0,1), b=(1,0)b=(1,0), and λ2=0.5\lambda_2=0.5. Their mean errors at λ1=0.2\lambda_1=0.2, 0.50.5, 0.80.8 are 1/21/2, 11, 1/21/2, respectively. Thus error need not be monotone in the first threshold.

Evidence Extraction and Stage Details

Evidence locality

The chunk view reduces input length when a short passage contains sufficient evidence. Whether it improves a prediction is data-dependent; it is evaluated separately from the full-document view.

Adaptive Evidence Extraction

Predicates rely on different signals and relevant content varies in density across records, so a fixed top-kk rule does not fit. CascadeFilter sizes the evidence set per record with a two-cutoff retrieval pipeline. The document did_i is split into sentences (Bling-Fire) and formed into overlapping three-sentence windows (stride 1). Candidates are ranked by reciprocal rank fusion (RRF) [48] of BM25 and dense (Qwen3-4B-Emb [18], [49]) scores,

RRF⁡(c)=1k0+rank⁡BM25(c)+1k0+rank⁡emb(c)(k0=60),\operatorname{RRF}(c)=\frac{1}{k_0+\operatorname{rank}_{\mathrm{BM25}}(c)}+\frac{1}{k_0+\operatorname{rank}_{\mathrm{emb}}(c)} \quad(k_0=60),

combining exact lexical hits with semantic matches. A first, recall-favoring cutoff keeps candidates up to the largest gap in the sorted RRF scores, plus a buffer of five; a cross-encoder reranker (Qwen3-0.6B-Reranker [18], [50]) rescores the survivors, and a second, precision-favoring cutoff (no buffer, capped at 5 chunks) keeps them up to the largest reranker-score gap. Selected chunks are merged in document order, overlapping sentences deduplicated, into echunk(di)e_{\mathrm{chunk}}(d_i).

Batched Inference and Confidence Scoring

CascadeFilter packs multiple query–evidence pairs into one batched context with explicit DocID delimiters,

CB=concat⁡[⟨DocID=i,qi,echunk(di)⟩]i∈B,C_B=\operatorname{concat}\left[\langle\mathrm{DocID}=i,q_i,e_{\mathrm{chunk}}(d_i)\rangle\right]_{i\in B},

where a token budget Bmax⁡B_{\max} caps batch size. The SLM emits one binary label per document identifier (DocID) in a single pass; any malformed output is marked defer, which can only shrink the accept set, preserving the guarantee. For each record, CascadeFilter reads the log-probability margin at its label position, ℓi=log⁡p(yes∣CB,prefixi)−log⁡p(no∣CB,prefixi)\ell_i=\log p(\mathrm{yes}\mid C_B,\mathrm{prefix}_i)-\log p(\mathrm{no}\mid C_B,\mathrm{prefix}_i), predicts y^i(1)=1[ℓi>0]\hat{y}_i^{(1)}=\mathbf{1}[\ell_i>0], and, since the margin scale shifts with batch length, maps it to a base confidence bi(1)=σ(ℓi/Ttemp)∈[0,1]b_i^{(1)}=\sigma(\ell_i/T_{\mathrm{temp}})\in[0,1] (Ttemp=15T_{\mathrm{temp}}=15), comparable across batch sizes; logistic compression is preferred over verbalized self-assessment, poorly calibrated for routing [51]. This base score is fused with neighborhood evidence into the adaptive routing score z~i(1)\tilde{z}_i^{(1)} (Section III-C), and Stage 1 accepts when z~i(1)≥λ1\tilde{z}_i^{(1)}\geq\lambda_1, with λ1\lambda_1 selected by certification.

Stage 2: Full-Document Filtering

Records that any candidate λ1\lambda_1 could defer form the maximal Stage-2 subset

T2max⁡={i∈T:z~i(1)<λ1,max⁡},λ1,max⁡=max⁡Λ1,(6)\mathcal{T}_2^{\max}=\{i\in\mathcal{T}:\tilde{z}_i^{(1)}<\lambda_{1,\max}\},\qquad\lambda_{1,\max}=\max\Lambda_1, \tag*{(6)}

which depends only on the fixed λ1,max⁡\lambda_{1,\max}, not on any candidate λ1\lambda_1, so running Stage 2 on T2max⁡\mathcal{T}_2^{\max} covers every record that could reach it. This full pass is a one-time search computation supplying the Stage-2 scores for each candidate’s label-free routing counts, not deployment routing, since under ϕ^\hat{\phi} a record accepted at Stage 1 is never sent to Stage 2; it is a constant offset across candidates, counted in full in the reported cost (Section IV-C). Stage 2 re-evaluates each record individually, without batch packing, removing the Stage 1 batch error modes (cross-document attention, DocID–label misalignment, evidence truncation), and accepts when z~i(2)≥λ2\tilde{z}_i^{(2)}\geq\lambda_2, with the score recomputed from full-document embeddings.

Stage 3: Oracle Fallback

The residual set (records deferred by both SLM stages under ϕ^\hat{\phi}) holds the records that require the oracle. CascadeFilter submits each to ML\mathcal{M}_L with the full document and predicate, returning the binary label and a short justification for auditability. The Stage 3 rate reflects predicate difficulty without tuning, from 2.5% on CB-Q1 to 31.2% on RV-Q3 (Table III). Because Stage 3 records are scored against the oracle label yi=ML(qi,di)y_i=\mathcal{M}_L(q_i,d_i), they contribute zero loss to the certification target by construction; the tier is warranted where SLM capacity runs out, as on CB-Q3 where the full-SLM reaches only 61.4% accuracy (§ 5-B).

Routing-Score Equations

The idea. Two records can share the same base confidence bi(s)b_i^{(s)} while only one deserves acceptance (Figure 1c), so we want each record’s effective threshold to track its difficulty, with no per-record tuning and no held-out labels. A free second cue makes this possible: a record whose embedding neighbors received the same prediction is likelier correct than one that disagrees, so CascadeFilter nudges the base confidence by how strongly a record’s neighbors agree with its prediction.

Neighbor support. Let vi(s)v_i^{(s)} be the unit-normalized Stage-ss embedding and NK(s)(i)\mathcal{N}_K^{(s)}(i) its top-KK cosine neighbors within the stage’s own pool (T\mathcal{T} at Stage 1, T2max⁡\mathcal{T}_2^{\max} at Stage 2), with weights wij(s)=max⁡{⟨vi(s),vj(s)⟩,0}w_{ij}^{(s)}=\max\{\langle v_i^{(s)},v_j^{(s)}\rangle,0\} and total mass Wi(s)=∑jwij(s)W_i^{(s)}=\sum_j w_{ij}^{(s)}. Neighbor support NSi(s)∈[0,1]NS_i^{(s)}\in[0,1] is the similarity-weighted fraction of those neighbors whose prediction matches ii’s own y^i(s)\hat{y}_i^{(s)} (set to 1/21/2 when Wi(s)=0W_i^{(s)}=0); it reads neighbor predictions, never labels.

Shrinkage. A neighborhood with few or distant neighbors carries little evidence, so we trust NSi(s)NS_i^{(s)} only in proportion to that evidence, shrinking it toward the uninformative value 1/21/2:

NS~i(s)=ρi(s)NSi(s)+(1−ρi(s))12,\widetilde{NS}_i^{(s)}=\rho_i^{(s)}NS_i^{(s)}+\left(1-\rho_i^{(s)}\right)\frac{1}{2},
ρi(s)=Wi(s)Wi(s)+σ02∈[0,1].(7)\rho_i^{(s)}=\frac{W_i^{(s)}}{W_i^{(s)}+\sigma_0^2}\in[0,1]. \tag*{(7)}

An evidence-rich neighborhood keeps NS~i(s)≈NSi(s)\widetilde{NS}^{(s)}_i \approx NS^{(s)}_i, an evidence-poor one returns it to 12\frac{1}{2}. Fixing σ0(s)\sigma^{(s)}_0 to the median of W(s)W^{(s)} over that pool needs no tuned constant and makes NS~i(s)\widetilde{NS}^{(s)}_i invariant to global rescaling of the similarities.

Score. The routing score adds a bounded neighbor adjustment to the base confidence in log-odds space,

z~i(s)=lgt⁡(bi(s))+γs(2NS~i(s)−1),(8)\widetilde{z}^{(s)}_i=\operatorname{lgt}(b^{(s)}_i)+\gamma_s\left(2\widetilde{NS}^{(s)}_i-1\right), \tag*{(8)}

where lgt⁡(b)=log⁡b+εℓ1−b+εℓ\operatorname{lgt}(b)=\log\frac{b+\varepsilon_\ell}{1-b+\varepsilon_\ell} is a smoothed log-odds link (εℓ=10−3\varepsilon_\ell=10^{-3}) and the neighbor term 2NS~i(s)−1∈[−1,1]2\widetilde{NS}^{(s)}_i-1\in[-1,1] is positive under agreement, negative under dissent, and zero when uninformative; γs≥0\gamma_s\ge0 (default 1; γs=0\gamma_s=0 recovers the pure base score) sets its weight.

Interface to the guarantee. Because NS~i(s)\widetilde{NS}^{(s)}_i depends on the weights and predictions but not on λs\lambda_s, each z~i(s)\widetilde{z}^{(s)}_i is frozen as λs\lambda_s varies, so one grid threshold acts per-record while the accept set stays nested in λs\lambda_s. Certification needs only that the score be admissible: computed independently of the calibration draw and its labels, scalar, and applied through a single threshold Ai(s)(λs)=1[z~i(s)≥λs]A^{(s)}_i(\lambda_s)=\mathbf{1}[\widetilde{z}^{(s)}_i\ge\lambda_s], which the default score meets (Section IV).

Single-Candidate Betting Test

Condition on a fixed finite population of records and potential outputs. Decoding, batching, retrieval, score construction, candidate grids, and test specifications must be fixed independently of the calibration draw. In particular, seeing the sampled indices before choosing the family is not permitted, even if their labels have not yet been revealed. Temperature zero alone is not a mathematical guarantee of deterministic API responses; the oracle target must be a fixed function, or fixed cached potential labels generated independently of the draw.

Let π1,…,πn\pi_1,\ldots,\pi_n be the first nn entries of a uniform random permutation of I\mathcal{I}, with fixed 1≤n≤N1\le n\le N. Preserve this order, or uniformly reshuffle the selected sample independently; do not sort it by record identity, confidence, or observed labels. For a fixed candidate define Xt=eπt(ϕ)X_t=e_{\pi_t}(\phi), St−1=∑k<tXkS_{t-1}=\sum_{k<t}X_k, and Ft=σ(π1,X1,…,πt,Xt)\mathcal{F}_t=\sigma(\pi_1,X_1,\ldots,\pi_t,X_t), including the fixed background information. Since the next index is uniform among the remaining indices,

E[Xt∣Ft−1]=Nθ−St−1N−t+1,θ=1N∑iei(ϕ).(9)\mathbb{E}[X_t\mid\mathcal{F}_{t-1}]=\frac{N\theta-S_{t-1}}{N-t+1},\qquad\theta=\frac{1}{N}\sum_i e_i(\phi). \tag*{(9)}

For a target u∈(0,1)u\in(0,1), set qt(u)=clip⁡[0,1]{(Nu−St−1)/(N−t+1)}q_t(u)=\operatorname{clip}_{[0,1]}\{(Nu-S_{t-1})/(N-t+1)\} and

Mt+(u)=∏k=1t[1+κk(u)(qk(u)−Xk)],M0+(u)=1.(10)M_t^+(u)=\prod_{k=1}^{t}\left[1+\kappa_k(u)(q_k(u)-X_k)\right],\qquad M_0^+(u)=1. \tag*{(10)}

Every bet must be finite, nonnegative, and Ft−1\mathcal{F}_{t-1}-measurable. Require κt(u)(1−qt(u))≤1/2\kappa_t(u)(1-q_t(u))\le1/2; when qt=1q_t=1, require a finite bet explicitly rather than evaluating 1/[2(1−qt)]1/[2(1-q_t)]. A proposed predictable nonnegative raw bet κ~t\widetilde{\kappa}_t can be clipped to min⁡{κ~t,1/[2(1−qt)]}\min\{\widetilde{\kappa}_t,1/[2(1-q_t)]\} for qt<1q_t<1, and left finite for qt=1q_t=1.

The original experimental description names approximate growth-rate adaptive (aGRAPA) betting [10]; the theorem covers any rule satisfying these conditions, but its power and sample-size behavior depend on the actual rule. The audited released implementation (revision 99adc74) uses mt=(1/2+St−1)/tm_t=(1/2+S_{t-1})/t, vt=(1/4+∑j<t(Xj−mj)2)/tv_t=(1/4+\sum_{j<t}(X_j-m_j)^2)/t, and raw bet κ~t=max⁡{0,(qt−mt)/[vt+(qt−mt)2]}\widetilde{\kappa}_t=\max\{0,(q_t-m_t)/[v_t+(q_t-m_t)^2]\}. These quantities use only preceding outcomes, and vt>0v_t>0 keeps the raw bet finite even at qt=1q_t=1. This identifies the current code, not necessarily the implementation used for the reported experiments. Tuning bets using later calibration outcomes is not predictable and is not covered.

Certify a candidate at level β\beta if Mn+(ϵ)≥1/βM_n^+(\epsilon)\ge1/\beta. This certifies the candidate through a fixed-time rejection of an excessive-error null; it does not estimate its error exactly. All proof statements are conditional on the fixed setup and therefore remain valid after integrating over independent preprocessing randomness.

Zero-error feasibility envelope

For an all-zero sample, qt(u)=Nu/(N−t+1)q_t(u)=Nu/(N-t+1) as long as t<N(1−u)+1t<N(1-u)+1. Maximizing each factor over the allowed bet interval gives

Un(u)=∏t=1n(1+qt(u)2(1−qt(u))),Mn+(u)≤Un(u).(11)U_n(u)=\prod_{t=1}^{n}\left(1+\frac{q_t(u)}{2(1-q_t(u))}\right),\qquad M_n^+(u)\le U_n(u). \tag*{(11)}

The bound follows because a zero-error factor 1+κtqt1+\kappa_tq_t is nondecreasing in the nonnegative bet. It is attained on this path by using the clipping cap at every step, but an implemented adaptive rule may use smaller bets. The formula applies only where qt<1q_t<1. It is not a universal envelope after clipping reaches one.

For N=9378N=9378, u=0.1u=0.1, and the required wealth L1/δ=640L_{1/\delta}=640, direct evaluation gives U94=165.632781U_{94}=165.632781, U118=616.076398U_{118}=616.076398, and U119=650.788159U_{119}=650.788159. Hence 119 is the first sample size at which even the allowed zero-error envelope reaches 640. It is a necessary feasibility floor for a zero-error certificate under this cap, not sufficient for any particular aGRAPA implementation.

The earlier expression log⁡Mn≤n KL(θ^∥u)\log M_n\le n\,\mathrm{KL}(\hat{\theta}\mathbin\Vert u) is not a general bound for this without-replacement process. For example, N=10N=10, u=0.1u=0.1, n=8n=8, an all-zero sample, and cap bets give log⁡U8≈0.854115\log U_8\approx0.854115, exceeding 8log⁡(1/0.9)≈0.8428848\log(1/0.9)\approx0.842884. Residual-population centering must be retained in the calculation.

Extensions of the Certification Target

Remark 1 (Conditions for extending the target). The single-candidate proof extends to a fixed population of losses ℓi(ϕ)∈[0,1]\ell_i(\phi)\in[0,1]: its conditional-mean identity and nonnegative-factor argument use boundedness, not binary labels. For a fixed finite family and a fixed within-track order, the first-error proof also controls false certification of these mean-loss constraints. Monotone losses are useful for the search interpretation but not required by that proof. However, an end-to-end guarantee additionally requires a fallback satisfying the new target, or separately certified with an allocated failure budget. Oracle fallback is zero-loss only for disagreement with that same oracle; arbitrary losses and ground-truth errors need not vanish there. Alternatively, the procedure may report no certified decision.

Class-conditional precision and recall are ratios with different target populations, not automatic instances of the overall-accuracy theorem. For a fixed subgroup of known size, independent of calibration sampling, conditioning on the number of sampled subgroup members gives a uniform without-replacement sample within it. The same test is then valid using that subgroup size and its random-order subsequence, conditional on a positive sample count. With zero sampled members, it must not certify a nontrivial bound from data. The zero-size target group also needs an explicit metric convention.

For recall, the positive-label population size is generally unknown without additional oracle information. For precision, the predicted-positive population changes with the candidate and may itself depend on oracle fallback labels. The current algorithm supplies neither all such population sizes nor simultaneous ratio guarantees. These extensions require an explicitly valid conditional-population or ratio procedure and appropriate error-budget allocation; they are not established here merely by substituting a loss or adding a Bonferroni split.

Cost Accounting with Cached Predictions

Search overhead and prediction reuse. (1) prices execution of a selected plan without pre-existing prediction caches. It is distinct from the total cost of searching and executing on the same batch. If search has already evaluated Stage 1 on all records and Stage 2 on I2max⁡\mathcal{I}_{2}^{\max}, these predictions are reused. With R(ϕ)={i:Di(1)Di(2)=1}R(\phi)=\{i:D_i^{(1)}D_i^{(2)}=1\}, same-batch modeled cost is

Cbatch(ϕ;S)=Cprep+C1+∑i∈I2max⁡c2i+∑i∈S∪R(ϕ)cLi,(12)C_{\mathrm{batch}}(\phi;S)=C_{\mathrm{prep}}+C_1+\sum_{i\in\mathcal{I}_{2}^{\max}}c_{2i}+\sum_{i\in S\cup R(\phi)}c_{Li}, \tag*{(12)}

where CprepC_{\mathrm{prep}} accounts for retrieval, indexing, and search computation. Each invocation is charged once. Optimizing CC and optimizing CbatchC_{\mathrm{batch}} can select different pairs. Our definition uses CC; reported end-to-end costs must separately account for the actual cached execution. The certification argument also permits selecting by CbatchC_{\mathrm{batch}} among certified candidates, but this is a different cost objective.

Interpreting the Threshold-Selection Example

The example in Section III uses fixed scores and illustrative stage costs. If this record’s short-evidence prediction is wrong but its full-document prediction is correct, the second pair removes the error for three additional cost units. If both predictions are wrong, oracle fallback removes it. The reverse is also possible: the short-evidence prediction can be correct while the full-document prediction is wrong. More context does not guarantee a better answer.

Calibration tests aggregate disagreement under one shared threshold pair; it neither selects per-record thresholds nor requires every accepted answer to be correct. The cheapest pair is selected only among certified candidates. Cached predictions are reused, so the table describes plan cost rather than additional charges for cached execution.

Notation

SymbolMeaning
yi,y^iny_i,\hat{y}_i^{\mathrm{n}}oracle label ML(q,di)\mathcal{M}_L(q,d_i); prediction of plan Π\Pi
cost⁡S,cost⁡L\operatorname{cost}_S,\operatorname{cost}_Ltoken-level invocation cost functions for MS,ML\mathcal{M}_S,\mathcal{M}_L
zi(s)z_i^{(s)}adaptive routing score at Stage ss (Eq. 9)
Λ1,Λ2\Lambda_1,\Lambda_2Stage-1/2 threshold grids; Φ=Λ1×Λ2\Phi=\Lambda_1\times\Lambda_2
ϕ=(λ1,λ2)\phi=(\lambda_1,\lambda_2)a candidate cascade configuration
I2max⁡\mathcal{I}_2^{\max}maximal subset reaching Stage 2 (Eq. 7)
ei(ϕ),E(ϕ)e_i(\phi),E(\phi)per-record error; population error rate 1n∑iei(ϕ)\frac{1}{n}\sum_i e_i(\phi)
r3(ϕ)r_3(\phi)Stage-3 oracle-call ratio (Section IV)
S,Mn+S,M_n^+SRSWOR calibration sample (∣S∣=n|S|=n); betting wealth
C,ϕ^C,\hat{\phi}certified set; its cost-optimal member

Table V. Primary notation. Certification threshold ϵ=1−τ\epsilon=1-\tau.

References

  1. [1]L. Patel, S. Jha, M. Pan, H. Gupta, P. Asawa, C. Guestrin, and M. Zaharia, “Semantic operators and their optimization: Enabling LLM-based data processing with accuracy guarantees in LOTUS,” Proceedings of the VLDB Endowment, vol. 18, no. 11, pp. 4171–4184, 2025. [Online]. Available: https://www.vldb.org/pvldb/vol18/p4171-patel.pdfDOI
  2. [2]C. Liu, M. Russo, M. J. Cafarella, L. Cao, P. B. Chen, Z. Chen, M. J. Franklin, T. Kraska, S. Madden, R. Shahout, and G. Vitagliano, “Palimpzest: Optimizing AI-powered analytics with declarative query processing,” in 15th Con- ference on Innovative Data Systems Research, CIDR 2025, Amsterdam, The Netherlands, January 19-22, 2025. www.cidrdb.org, 2025. [Online]. Available: https://vldb.org/cidrdb/2025/papers/p32-palem optimizing-ai-powered-analyti cs-with-declarative-query-processing.html
  3. [3]S. Shankar, T. Chambers, T. Shah, A. G. Parameswaran, and E. Wu, “DocETL: Agentic query rewriting and evaluation for complex document processing,” Proceedings of the VLDB Endowment, vol. 18, no. 9, pp. 3053–3048, 2025. [Online]. Available: https: //www.vldb.org/pvldb/vol18/p3035-shankar.pdfarxiv.org/abs/2410.12189
  4. [4]L. Chen, M. Zaharia, and J. Zou, “FrugalGPT: How to use large language models while reducing cost and improving performance,” Transactions on Machine Learning Research, 2024, featured Certification. [Online]. Available: https://openreview.net/forum?id=cSimKw5p6R
  5. [5]I. Ong, A. Almahairi, V. Wu, W.-L. Chiang, T. Wu, J. E. Gonzalez, M. W. Kadous, and I. Stoica, “RouteLLM: Learning to route LLMs from preference data,” in The Thirteenth International Conference on Learning Representations, 2025. [Online]. Available: https://openreview.net/forum?id=8sSqNntaMrarxiv.org/abs/2406.18665
  6. [6]S. Zeighami, S. Shankar, and A. Parameswaran, “Cut costs, not accuracy: LLM-powered data processing with guarantees,” Proc. ACM Manag. Data, vol. 3, no. 6, Dec. 2025. [Online]. Available: https://doi.org/10.1145/3769776
  7. [7]H. Zhang, Y. Hui, Y. Liu, and H. Zhang, “ScaleDoc: Scaling LLM-based predicates over large document collections,” Proc. ACM Manag. Data, vol. 4, no. 3, May 2026. [Online]. Available: https://doi.org/10.1145/3802106
  8. [8]S. Shankar, S. Zeighami, and A. Parameswaran, “Task Cascades for efficient unstructured data processing,” Proc. ACM Manag. Data, vol. 4, no. 1, Apr. 2026. [Online]. Available: https://doi.org/10.1145/3786702
  9. [9]S. Robertson and H. Zaragoza, “The probabilistic relevance framework: BM25 and beyond,” Foundations and Trends in Information Retrieval, vol. 4, no. 1–2, pp. 1–174, 2009. [Online]. Available: https: //doi.org/10.1561/1500000019
  10. [10]I. Waudby-Smith and A. Ramdas, “Estimating means of bounded random variables by betting,” Journal of the Royal Statistical Society Series B: Statistical Methodology, vol. 86, no. 1, pp. 1–27, Feb. 2024. [Online]. Available: https://doi.org/10.1093/jrsssb/qkad009
  11. [11]N. Hou, K. Zhao, J. Xie, and J. X. Yu, “Beyond linear LLM invocation: An efficient and effective semantic filter paradigm,” arXiv preprint arXiv:2603.04799, 2026. [Online]. Available: https: //arxiv.org/abs/2603.04799arxiv.org/abs/2603.04799
  12. [12]J. Li, B. Hui, G. Qu, J. Yang, B. Li, B. Li, B. Wang, B. Qin, R. Geng, N. Huo, X. Zhou, C. Ma, G. Li, K. C. Chang, F. Huang, R. Cheng, and Y. Li, “Can LLM already serve as a database interface? a BIg bench for large-scale database grounded text-to-SQLs,” in Advances in Neural Information Processing Systems, vol. 36. Red Hook, NY, USA: Curran Associates, Inc., 2023. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2023/hash/83fc8fab1710363050bbd1d4b8cc0021-Abstract-Datasets_and_Benchmarks.html
  13. [13]A. L. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y. Ng, and C. Potts, “Learning word vectors for sentiment analysis,” in Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, D. Lin, Y. Matsumoto, and R. Mihalcea, Eds. Portland, Oregon, USA: Association for Computational Linguistics, Jun. 2011, pp. 142–150. [Online]. Available: https://aclanthology.org/P11-1015/
  14. [14]W. Wei, Q. Le, A. Dai, and J. Li, “AirDialogue: An environment for goal-oriented dialogue research,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, E. Riloff, D. Chiang, J. Hockenmaier, and J. Tsuji, Eds. Brussels, Belgium: Association for Computational Linguistics, Nov. 2018, pp. 3844–3854. [Online]. Available: https://aclanthology.org/D18-1419/DOI
  15. [15]T. Davidson, D. Warmsley, M. Macy, and I. Weber, “Automated hate speech detection and the problem of offensive language,” Proceedings of the International AAAI Conference on Web and Social Media, vol. 11, no. 1, pp. 512–515, 2017. [Online]. Available: https://ojs.aaai.org/index.php/ICWSM/article/view/14955
  16. [16]J. Thorne, A. Vlachos, C. Christodoulopoulos, and A. Mittal, “FEVER: a large-scale dataset for fact extraction and VERification,” in Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), M. Walker, H. Ji, and A. Stent, Eds. New Orleans, Louisiana: Association for Computational Linguistics, Jun. 2018, pp. 809–819. [Online]. Available: https://aclanthology.org/N18-1074/
  17. [17]Qwen Team, “Qwen3-4B-Instruct-2507,” Hugging Face model card, 2025, accessed: 2026-07-02. [Online]. Available: https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507
  18. [18]Y. Zhang, M. Li, D. Long, X. Zhang, H. Lin, B. Yang, P. Xie, A. Yang, D. Liu, J. Lin, F. Huang, and J. Zhou, “Qwen3 embedding: Advancing text embedding and reranking through foundation models,” 2025. [Online]. Available: https://arxiv.org/abs/2506.05176
  19. [19]OpenAI, “GPT-5.5 model,” OpenAI API documentation, 2026, accessed: 2026-07-02. [Online]. Available: https://developers.openai.com/api/docs/models/gpt-5.5
  20. [20]A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, C. Zheng, D. Liu, F. Zhou, F. Huang, F. Hu, H. Ge, H. Wei, H. Lin, J. Tang, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Zhou, J. Lin, K. Dang, K. Bao, K. Yang, L. Yu, L. Deng, M. Li, M. Xue, M. Li, P. Zhang, P. Wang, Q. Zhu, R. Men, R. Gao, S. Liu, S. Luo, T. Li, T. Tang, W. Yin, X. Ren, X. Wang, X. Zhang, X. Ren, Y. Fan, Y. Su, Y. Zhang, Y. Zhang, Y. Wan, Y. Liu, Z. Wang, Z. Cui, Z. Zhang, Z. Zhou, and Z. Qiu, “Qwen3 technical report,” 2025. [Online]. Available: https://arxiv.org/abs/2505.09388
  21. [21]Meta, “Llama 3.2: 1b and 3b text model card,” Hugging Face model card, 2024, accessed: 2026-11-05. [Online]. Available: https: //huggingface.co/meta-llama/Llama-3.2-1B-Instruct
  22. [22]J. Chen, S. Xiao, P. Zhang, K. Luo, D. Lian, and Z. Liu, “M3-Embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation,” in Findings of the Association for Computational Linguistics: ACL 2024, L.-W. Ku, A. Martins, and V. Srikumar, Eds. Bangkok, Thailand: Association for Computational Linguistics, 2024, pp. 2318–2335. [Online]. Available: https://doi.org/10.18653/v1/findings-acl.137
  23. [23]L. Wang, N. Yang, X. Huang, B. Jiao, L. Yang, J. Jiang, R. Majumder, and F. Wei, “Text embeddings by weakly-supervised contrastive pre-training,” 2022. [Online]. Available: https://arxiv.org/abs/2212.03533
  24. [24]Y. Lu, A. Chowdhery, S. Kandula, and S. Chaudhuri, “Accelerating machine learning inference with probabilistic predicates,” in Proceedings of the 2018 International Conference on Management of Data, ser. SIGMOD ’18. New York, NY, USA: Association for Computing Machinery, 2018, pp. 1493–1508. [Online]. Available: https://doi.org/10.1145/3183713.3183751
  25. [25]L. L. Wei, S. Shankar, S. Zeighami, Y. Chung, F. Ozcan, and A. G. Parameswaran, “Multi-objective agentic rewriters for unstructured data processing,” 2025. [Online]. Available: https://arxiv.org/abs/2512.02289
  26. [26]S. Liu, J. Xu, W. Tjagnaka, S. Semnani, C. Yu, and M. Lam, “SUQL: Conversational search over structured and unstructured data with large language models,” in Findings of the Association for Computational Linguistics: NAACL 2024, K. Duh, H. Gomez, and S. Bethard, Eds. Mexico City, Mexico: Association for Computational Linguistics, Jun. 2024, pp. 4535–4555. [Online]. Available: https://aclanthology.org/2024.findings-naacl.283/arxiv.org/abs/2311.09818
  27. [27]Y. Lin, M. Hulseb os, R. Ma, S. Shankar, S. Zeighami, A. G. Parameswaran, and E. Wu, “Querying templatized document collections with large language models,” in 2025 IEEE 41st International Conference on Data Engineering (ICDE), 2025, pp. 2422–2435. [Online]. Available: https://ieeexplore.ieee.org/document/11112850DOI
  28. [28]H. Dai, B. Y. Wang, X. Wang, B. Dai, S. Yang, A. Noya, P. Yin, P. M. Phothilimthana, C. Sutton, and D. Schuurmans, “UQE: A query engine for unstructured databases,” in Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, Eds., vol. 37. Red Hook, NY, USA: Curran Associates, Inc., 2024, pp. 29 807–29 838. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2024/file/34b3a40ec9752c1ae48fe85ef8e8dc-Paper-Conference.pdfarxiv.org/abs/2407.09522
  29. [29]M. Russo, C. Liu, S. Sudhir, G. Vitagliano, M. Cafarella, T. Kraska, and S. Madden, “Abacus: A cost-based optimizer for semantic operator systems,” Proceedings of the VLDB Endowment, vol. 19, no. 5, pp. 1060–1073, 2026. [Online]. Available: https://www.vldb.org/pvldb/vol19/p1060-russo.pdfDOI
  30. [30]M. J. S. Jia, C. Guestrin, M. Zaharia, J. E. Gonzalez, A. Kamsetty, S. Liu, and L. Patel, “Text2SQL is not enough: Unifying AI and databases with TAG,” in 15th Conference on Innovative Data Systems Research, CIDR 2025, Amsterdam, The Netherlands, January 19-22, 2025. www.cid rdb.org, 2025. [Online]. Available: https://vldb.org/cidr b/2025/text2sql-is-not-enough-unifying-ai-and-databases-with-tag.htmlarxiv.org/abs/2408.14717
  31. [31]D. Kang, J. Guibas, P. D. Bailis, T. Hashimoto, and M. Zaharia, “TASTI: Semantic indexes for machine learning-based queries over unstructured data,” in Proceedings of the 2022 International Conference on Management of Data, ser. SIGMOD ’22. New York, NY, USA: Association for Computing Machinery, 2022, pp. 1934–1947. [Online]. Available: https://doi.org/10.1145/3514221.3517897
  32. [32]K. Kim, M. Cathel and A. Ailamaki, “Fast LLM-based semantic filtering: From a unified framework to an adaptive two-phase method,” 2026. [Online]. Available: https://arxiv.org/abs/2606.08090
  33. [33]Y. Chung, R. Desai, J. He, Y. Xiao, T. Hothiller, Y.-L. Kom Samo, P. Khadilkar, X. Chen, S. Idicula, F. Ozcan, A. Halevy, and Y. Papakonstantinou, “100x cost & latency reduction: Performance analysis of AI query approximation using lightweight proxy models: experiments & analysis,” Proc. ACM Manag. Data, vol. 4, no. 3, May 2026. [Online]. Available: https://doi.org/10.1145/3802002
  34. [34]S. Li, S. Park, I. Lee, and O. Bastani, “TRAQ: Trustworthy retrieval augmented question answering via conformal prediction,” in Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), K. Duh, H. Gomez, and S. Bethard, Eds. Mexico City, Mexico: Association for Computational Linguistics, Jun. 2024, pp. 3799–3821. [Online]. Available: https://aclanthology.org/2024.naacl-long.210/arxiv.org/abs/2307.04642
  35. [35]P. Aggarwal, A. Madaan, A. Anand, S. P. Potharaju, S. Mishra, P. Zhou, A. Gupta, D. Rajagopal, K. Kappaganthu, Y. Yang, S. Upadhyay, M. Faruqui, and M. Mausam, “AutoMix: Automatically mixing language models,” in Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, Eds., vol. 37. Red Hook, NY, USA: Curran Associates, Inc., 2024, pp. 131 000–131 034. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2024/file/ecda225cb187b40ea8edc1f46b03ffda-Paper-Conference.pdfarxiv.org/abs/2310.12963
  36. [36]D. Ding, A. Mallick, C. Wang, R. Sim, S. Mukherjee, V. Rühle, L. V. S. Lakshmanan, and A. H. Awadallah, “Hybrid LLM: Cost-efficient and quality-aware query routing,” in The Twelfth International Conference on Learning Representations, 2024. [Online]. Available: https://openreview.net/forum?id=02f3mUtqnMarxiv.org/abs/2404.14618
  37. [37]S. Jo and I. Trummer, “SMART: Automatically scaling down language models with accuracy guarantees for reduced processing fees,” 2024. [Online]. Available: https://arxiv.org/abs/2403.13835
  38. [38]K. Huang, Y. Shi, D. Ding, Y. Li, Y. Fei, L. Lakshmanan, and X. Xiao, “ThriftLLM: On cost-effective selection of large language models for classification queries,” Proceedings of the VLDB Endowment, vol. 18, no. 11, pp. 4410–4423, 2025. [Online]. Available: https://www.vldb.org/pvldb/vol18/p4410-huang.pdfarxiv.org/abs/2501.04901
  39. [39]W. Jitkrittum, H. Narasimhan, A. S. Rawat, J. Juneja, C. Wang, Z. Wang, A. Go, C.-Y. Lee, P. Shenoy, R. Panigrahy, A. K. Menon, and S. Kumar, “Universal model routing for efficient LLM inference,” in The Fourteenth International Conference on Learning Representations, 2026. [Online]. Available: https://openreview.net/forum?id=ka82fvJ5fJ
  40. [40]A. Valkan as, S. Pal, P. Rumiantsev, Y. Zhang, and M. Coates, “C3PO: Optimized large language model cascades with probabilistic cost constraints for reasoning,” in Advances in Neural Information Processing Systems, D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen, Eds., vol. 38. Red Hook, NY, USA: Curran Associates, Inc., 2025, pp. 84 481–84 522. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2025/file/79f03e5d0a6a4e34b38ddcc7dd204897-Paper-Conference.pdfarxiv.org/abs/2511.07396
  41. [41]V. Rotte, “UCCI: Calibrated uncertainty for cost-optimal LLM cascade routing,” 2026. [Online]. Available: https://arxiv.org/abs/2605.18796
  42. [42]D. Kang, E. Gan, P. Bailis, T. Hashimoto, and M. Zaharia, “Approximate selection with guarantees using proxies,” Proceedings of the VLDB Endowment, vol. 13, no. 11, pp. 1990–2003, 2020. [Online]. Available: https://www.vldb.org/pvldb/vol13/p1990-kang.pdfarxiv.org/abs/2004.00827
  43. [43]P. Liskowski and K. Schmaus, “Streaming model cascades for semantic SQL,” 2026. [Online]. Available: https://arxiv.org/abs/2604.00660
  44. [44]A. N. Angelopoulos, S. Bates, A. Fisch, L. Lei, and T. Schuster, “Conformal risk control,” in The Twelfth International Conference on Learning Representations, 2024. [Online]. Available: https://openreview.net/forum?id=33XGfHLtZg
  45. [45]Y. Xu, W. Guo, and Z. Wei, “Selective conformal risk control,” 2025. [Online]. Available: https://arxiv.org/abs/2512.12844
  46. [46]I. Gibbs and E. Candes, “Adaptive conformal inference under distribution shift,” in Advances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, Eds., vol. 34. Curran Associates, Inc., 2021, pp. 1660–1672. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2021/file/0d441de75945e5acbc850406fc9a2359-Paper.pdfarxiv.org/abs/2106.00170
  47. [47]A. N. Angelopoulos, S. Bates, E. J. Candès, M. I. Jordan, and L. Lei, “Learn then test: Calibrating predictive algorithms to achieve risk control,” The Annals of Applied Statistics, vol. 19, no. 2, pp. 1641–1662, 2025. [Online]. Available: https://doi.org/10.1214/24-AOAS1998
  48. [48]G. V. Cormack, C. L. A. Clarke, and S. Buettcher, “Reciprocal rank fusion outperforms condorcet and individual rank learning methods,” in Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, ser. SIGIR ’09. New York, NY, USA: Association for Computing Machinery, 2009, pp. 758–759. [Online]. Available: https://doi.org/10.1145/1571941.1572114
  49. [49]Qwen Team, “Qwen3-Embedding-4B,” Hugging Face model card, 2025, accessed: 2026-07-02. [Online]. Available: https://huggingface.co/Qwen/Qwen3-Embedding-4B
  50. [50]——, “Qwen3-Reranker-0.6B,” Hugging Face model card, 2025, accessed: 2026-07-02. [Online]. Available: https://huggingface.co/Qwen/Qwen3-Reranker-0.6B
  51. [51]M. Xiong, Z. Hu, X. Lu, Y. Li, J. Fu, J. He, and B. Hooi, “Can LLMs express their uncertainty? an empirical evaluation of confidence elicitation in LLMs,” in The Twelfth International Conference on Learning Representations, 2024. [Online]. Available: https://openreview.net/forum?id=gjeQKFxFpZarxiv.org/abs/2306.13063

Paper details

vlnc:W_n6ax395y
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 14

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents