betaFeedback

WorksComputer Science

Retriever-Free Retrieval: Navigating Documents with Decision Models

Abstract

While language models have become dramatically more capable, retrieval still largely relies on lexical matching or similarity in fixed embedding spaces. This creates a capability asymmetry: what the retriever fails to find, even the strongest generator cannot recover. We propose DecisionRAG, which brings the capabilities of pretrained language models to retrieval using decision models that jointly evaluate candidate branches in a single forward pass, without generating any tokens, and show that judging candidates in a shared context makes their relevance estimates comparable. Like a person browsing a library, DecisionRAG navigates the corpus's inherent structure from broad topics down to individual passages, deciding at each level which branches are worth exploring. Without task-specific training or a first-stage retriever, DecisionRAG outperforms dense retrieval with an 8B reranker by 13.8 R@1 points on MultiDoc2Dial and 17.4 Hit@1 points on Natural Questions. Across four benchmarks, including a corpus of 2.68 million Wikipedia paragraphs, retrieval takes just 0.7-1.8 seconds per query.

Introduction

Retrieval-augmented generation (RAG) has become a core component of modern language-model pipelines ([24, 16, 12]). By fetching evidence from an external corpus at inference time, RAG gives a model access to up-to-date information it never saw during training ([24] (Vu et al. 2024)) and grounds its responses in verifiable sources ([39, 11]).

Yet, while the language generation models have become dramatically more capable ([1] (OpenAI 2026)), the retrievers that decide what they see have not kept pace.

Most retrieval systems still score each passage via heuristics based on term frequencies like BM25 (Robertson & Zaragoza [36]), or by nearest neighbors in a fixed embedding space ([20, 17, 51]). This creates a striking capability asymmetry between the retriever and the generation model (Figure 1): whatever the retriever misses, no amount of generator capability can recover ([3]).

Capability asymmetry: a simple retriever bottlenecks a powerful LLM; decision models level the playing field

Figure 1. Capability asymmetry: a simple retriever bottlenecks a powerful LLM (a); decision models level the playing field (b).

Ideally, a model as capable as the generator would decide what to retrieve. Agentic approaches let an LLM navigate the corpus step by step ([50, 5, 19]), but generating many tokens at every step in an agentic loop makes them too slow and costly for many applications ([18, 49]). Rerankers offer a cheaper middle ground ([30, 41, 35]), but they can only reorder what a first-stage retriever has already returned.

Decision models have emerged as a new paradigm to leverage the power of pretrained language models quickly without sampling any tokens (TypeSafe AI, 2026; OpenAI, 2026; Li & Wagle, 2026). Given a context and a set of questions, a decision model such as Jey (TypeSafe AI, 2026) returns, for each question, a probability distribution over its answer options (for a yes/no question, P(yes)P(\mathrm{yes})), answering many questions in parallel against a shared context in a single forward pass. This makes them a natural fit for retrieval, bringing the capability of LLMs to the choice of what to retrieve. Furthermore, scoring the candidates jointly makes their decisions comparable with one another ([7, 14]). Using decision models to choose what to retrieve thus promises to level the playing field between the retriever and generator.

We propose DecisionRAG, which retrieves by navigating instead of searching (Figure 2). Most corpora have an inherent structure of topics, documents, sections, and passages, which humans exploit to find information efficiently [34]. DecisionRAG leverages this structure through hierarchical decisions [13, 27]: at each level, a decision model jointly evaluates candidates to identify the most promising branches, progressively narrowing the search from broad categories (e.g. topics or document titles) down to individual passages. Each decision requires only a single forward pass without generating any tokens, keeping retrieval fast even for large corpora. DecisionRAG thus brings the capabilities of pretrained language models directly to retrieval, without a first-stage retriever at query time, a precomputed index, or task-specific training.

Figure 2. Retrieval by matching versus retrieval by decisions. (a) BM25 and (b) dense retrieval match query terms or embeddings and can miss the relevant page. (c) DecisionRAG walks the corpus hierarchy: a decision model judges all candidates of a level in one forward pass, keeping the top kk documents and then the top rr passages; pruned documents are never expanded. Term frequencies, IDF values, BM25 scores, and rankings in panel (a) are illustrative.

We evaluate DecisionRAG on four benchmarks spanning multi-turn dialogue [9, 21], scenario-based questions [40], and open-domain search over 2.68 million Wikipedia paragraphs [22, 46]. Without task-specific training, DecisionRAG outperforms dense retrieval with an 8B reranker by 13.8 R@1 points on MultiDoc2Dial and 17.4 Hit@1 points on Natural Questions. Judging candidates in a shared context contributes to these gains, improving accuracy by up to 7.7 points over independent judgments (§ 6.1). Despite harnessing the capabilities of a language model at every stage, DecisionRAG retrieves in just 0.7–1.8 seconds at a cost of 0.2–2.3 cents per query. Compared with an LLM that generates navigation decisions over the same hierarchy token-by-token, it achieves comparable accuracy with 34–145× lower latency (§ 6.3). Training a dedicated 8B decision model on in-domain data further raises R@1 on MultiDoc2Dial to 60.7, surpassing the best compared published task-trained retriever by 5.7 points and suggesting substantial headroom as decision models improve.

Related Work

Traditional retrieval methods, including BM25 [36] and dense retrievers [20], rank passages using lexical matching or learned embedding similarity. Neural reranking methods refine these rankings using language-model-based relevance estimates. Cross-encoders evaluate individual query–passage pairs [31], while LLM-based rerankers perform listwise ranking by generating permutations [41] or iteratively selecting candidates from subsets [53] (Li et al. 2026). Although reranking can improve retrieval quality, it remains limited by the candidate recall of the initial retrieval stage.

An alternative line of work integrates language models directly into the retrieval process. Agentic approaches interleave reasoning with the use of search-engines to iteratively identify relevant evidence [50, 19]. Other methods use hierarchical representations to guide retrieval. MemWalker navigates a hierarchy of summaries constructed from long documents [5], while PageIndex uses document tables of contents for LLM-guided navigation [44]. LATTICE extends hierarchical navigation to corpus-level retrieval, constructing a semantic tree and using LLM-generated relevance assessments to select branches at each level [15]. These approaches require generating tokens sequentially at each retrieval step, which can incur substantial latency when navigation involves multiple rounds of reasoning.

Recent work has explored more efficient approaches to language-model-based ranking. FIRST derives listwise rankings from the logits of a single generated identifier [35], while ICR and BlockRank extract ranking signals from attention patterns over jointly presented candidates [7, 14]. Decision models offer another approach to evaluating multiple candidates without generating tokens (TypeSafe AI, 2026; OpenAI, 2026). Given a shared context and a set of questions, these models produce probability distributions over the answer options of each question in a single forward pass. This enables parallel relevance judgments over multiple candidates, with each candidate evaluated against the same contextual information.

Hierarchical retrieval has also been studied as a means of reducing the search space in large document collections. DocQN learns to navigate document trees using reinforcement learning [13], while dense hierarchical retrieval progressively selects documents and passages using trained encoders [27]. RAPTOR constructs a hierarchy of recursively generated summaries to support retrieval at different levels of abstraction [38]. Related work has also highlighted the importance of document-level context for passage retrieval [46, 52] (Xu et al. 2025).

DecisionRAG combines hierarchical retrieval with non-generative relevance judgments. It uses existing document structure where available and constructs document groups otherwise (§ 4). Unlike conventional reranking pipelines, DecisionRAG does not require a first-stage retriever to construct a candidate pool. Unlike LLM-guided hierarchical search, it evaluates candidates at each level through parallel relevance judgments in a single forward pass, without generating tokens.

Method

Problem Formulation

We consider a corpus D={D1,…,DN}\mathcal{D} = \{D_1, \ldots, D_N\}, where each document DiD_i consists of passages di1,…,diMid_{i1}, \ldots, d_{iM_i}. Given a query qq, which may include the preceding dialogue turns, our goal is to retrieve the passages containing the evidence needed to answer it. (Figure 2) More generally, we view the corpus as a tree whose leaves are passages and whose internal nodes group them at increasing granularity, such as help centers, topic clusters, documents, or section headings. Each node has a short textual description: the text itself for a passage, the title for a document, and a one-sentence summary for a cluster. We call each depth of the tree a level and each round of requests to the decision model a stage; most levels take one stage, and the final reranking is a stage of its own. As a running example, we use the query

Who is the main character in Christopher Nolan’sfilm from the year 2000?(1)\textit{Who is the main character in Christopher Nolan’s}\\ \textit{film from the year 2000?} \tag*{(1)}

which can be answered by the Wikipedia corpus.

Relevance Judgments with Decision Models

Decision models such as Jev ([43]) and OpenAI’s decision model (OpenAI, 2026) evaluate questions against a context and return probabilities over predefined answer options, without autoregressive generation. We use two of their inference modes, parallel binary judgments and multiple-choice judgments, both against a shared context ss (Figure 3).

Two judgment modes of a decision model, with parallel binary judgments and multiple-choice reranking

Figure 3. The two judgment modes of a decision model, each against a shared context that the model reads once.

Parallel binary judgments. Given a shared context ss and a set of binary questions Q={Q1,…,Qn}Q = \{Q_1, \ldots, Q_n\}, the decision model evaluates all questions in parallel in a single forward pass. For each question QiQ_i, it produces a probability of answering “Yes”,

pibin=P(Yes∣s,Qi).(2)p_i^{\mathrm{bin}} = P(\mathrm{Yes} \mid s, Q_i). \tag*{(2)}

Each question is evaluated against the same context, but its answer probabilities are normalized only over “Yes” and “No”, not across questions. Thus, the resulting scores need not sum to one. Binary judgments therefore impose no limit on the number of candidates beyond the model’s context length; larger levels are split across parallel requests (§ 3.4).

Multiple-choice judgments. Given a context ss, a question QQ, and a set of answer options A={a1,…,am}A = \{a_1, \ldots, a_m\}, the decision model produces a probability distribution over the options,

pimul=P(ai∣s,Q,A),∑i=1mpimul=1.(3)p_i^{\mathrm{mul}} = P(a_i \mid s, Q, A), \qquad\sum_{i=1}^{m} p_i^{\mathrm{mul}} = 1. \tag*{(3)}

Unlike parallel binary judgments, multiple-choice judgments evaluate the answer options as competing alternatives within a single decision. The number of answer options is limited (255 for Jev), so we apply this mode only to the top candidates from the binary judgments.

Judging candidates in a shared context. Because every question sees the same context, each probability is formed in view of the alternatives: a candidate that would look relevant on its own receives a lower P(Yes)P(\mathrm{Yes}) when a clearly better one is present. The shared context thus puts the individual probabilities into perspective, making them comparable across candidates. Our ablations support this: scoring candidates together rather than separately improves R@1 on MultiDoc2Dial by 7.7 points and Hit@1 on Natural Questions by 6.5 points (§ 6.1 and § 6.2). They also show where the comparison matters: splitting a pool into smaller requests lowers Hit@1 but leaves R@10 unchanged. Scores from separate requests therefore suffice for screening, where recall counts, whereas the final ranking benefits from one shared request.

Hierarchical Retrieval

Rather than scoring every passage in the corpus individually, DecisionRAG progressively narrows the search space by traversing a corpus hierarchy. At each level, a decision model evaluates the candidates under consideration and retains the most promising ones for further exploration. Candidates not selected are pruned, and their descendants are never evaluated.

We illustrate this process in Figure 2 (c) using the running example in Equation (1).

Document screening. We first evaluate candidate documents using parallel binary judgments (Equation (2)). The shared context sDs_D contains the query and descriptions of all candidate documents, while each binary question asks whether a particular document is worth exploring for the query. As illustrated in Figure 3a, the model evaluates all candidates in a single forward pass, assigning each a relevance score. In our running example, the pages for Christopher Nolan and Memento (film) are retained, while Gladiator (2000 film) is discarded despite the latter’s keyword overlaps with the query. We retain the top kk documents according to their relevance scores and prune the remaining branches.

Passage scoring. We next expand the retained documents into their passages and evaluate them using parallel binary judgments (Equation (2)). The shared context sPs_P now contains the query and all candidate passages, allowing the model to assess their relevance in view of one another, so that their probabilities are directly comparable (§ 3.2). We rank the passages by their binary relevance scores and keep the top rr across all selected documents. On Natural Questions, where the paragraph stage spans 30 pages, each paragraph’s score is first multiplied by its page’s (Appendix § 11.§ 5). In our running example, a passage from Memento (film) describing its protagonist, Leonard Shelby, provides the evidence needed to answer the query.

Optional passage reranking. To refine the final ranking, we optionally apply a multiple-choice judgment over the top m≤rm \le r passages selected by binary scoring (Equation (3)). The choice context lists only these mm passages (Appendix § 11.§ 6). Their choice probabilities determine their order, while the remaining passages retain their binary ranking. We use m=20m = 20, except on Natural Questions, where m=5m = 5. On MTRAG, we instead promote only the highest-probability choice to rank one, preserving the relative order of all other passages. As illustrated in Figure 3b, the model compares the shortlisted passages as competing alternatives

Algorithm 1 DecisionRAG

Require: query qq; corpus tree with LL levels; budgets k1,…,kL−1k_1,\ldots,k_{L-1}; number of returned passages rr; reranking shortlist size m≤rm \le r

  1. C←\mathcal{C} \leftarrow children of the root

  2. for ℓ=1,…,L−1\ell= 1,\ldots,L-1 do

  3. pbin←BinaryJudge⁡(q,C)\mathbf{p}^{\mathrm{bin}} \leftarrow\operatorname{BinaryJudge}(q,\mathcal{C}) {Equation (2)}

  4. C←⋃c∈top-⁡kℓ(C,pbin)children⁡(c)\mathcal{C} \leftarrow\bigcup_{c \in\operatorname{top-}k_\ell(\mathcal{C},\mathbf{p}^{\mathrm{bin}})} \operatorname{children}(c)

  5. end for

  6. pbin←BinaryJudge⁡(q,C)\mathbf{p}^{\mathrm{bin}} \leftarrow\operatorname{BinaryJudge}(q,\mathcal{C})

  7. R←top-⁡r(C,pbin)R \leftarrow\operatorname{top-}r(\mathcal{C},\mathbf{p}^{\mathrm{bin}})

  8. if multiple-choice reranking is enabled then

  9. pmul←ChoiceJudge⁡(q,R[1:m])\mathbf{p}^{\mathrm{mul}} \leftarrow\operatorname{ChoiceJudge}(q,R[1:m]) {Equation (3)}

  10. reorder R[1:m]R[1:m] by decreasing pmul\mathbf{p}^{\mathrm{mul}}

  11. end if

  12. return RR

within a single decision.

Binary judgments suit the inner levels: they have no option limit, run in parallel, and favor recall, which matters while the search narrows. The multiple-choice judgment is reserved for the end, where it compares the top passages from the binary judgments directly and sharpens the top rank (Section 6.4). Algorithm 1 summarizes the procedure for a hierarchy of any depth.

Budgets and pruning. Pruning is irreversible: a branch dropped at one level is not revisited for this query. The budgets kℓk_\ell therefore lean towards recall, trading coverage for cost (§ 6.6). A budget can also be a probability threshold, e.g., keeping a runner-up help center whenever the top score is below 0.8 (Appendix § 11.§ 2).

Hierarchy Construction and Scaling

Higher-order hierarchies. The document–passage hierarchy is only the simplest instance of DecisionRAG. For larger corpora, documents can be organized into higher-order groups, such as topics, categories, or products, using existing corpus structure or grouping related documents. The same procedure then applies recursively: at each level, parallel binary judgments identify promising groups, and only their children are evaluated at the next level. For example, when searching the full Wikipedia corpus, one could traverse categories related to films and Christopher Nolan before reaching the individual page for Memento. Thus, DecisionRAG extends to deeper hierarchies without requiring all documents to be evaluated at once. When a corpus lacks such structure, a hierarchy can be built offline by clustering document embeddings, e.g., with recursive kk-means (Nistér & Stewenius, [29]) or scalable agglomerative clustering ([4, 28, 8]), as we do for Natural Questions (§ 4.1). The tree need not be uniform: on MTRAG, a heading level is inserted only below pages with more than 60 passages.

Levels too large for one request. A level may hold more candidates than fit into one request; on Natural Questions, the 150 retained clusters contain about 17,500 page titles. We then split the level into parallel requests, each with its own shared context. This costs little: the benefit of the shared context levels off at about 50 candidates per request (§ 6.2), and pooled scores suffice for screening. Where the top candidates of a split level should be compared more closely, we rescore them in one request, as for the top 50 pages on MTRAG (Appendix § 11.§ 4).

Combining with a retriever. DecisionRAG needs no first-stage retriever, but it can use one: adding the top passages of a dense retriever to the final candidates recovers evidence that pruning discarded (+ dense candidates; § 5.1).

Scaling with corpus size. Because all candidates of a level are judged in parallel, latency depends on how many levels the hierarchy has, not on how many passages the corpus holds. Assume a roughly balanced hierarchy in which every node groups at most bb children, e.g., topics of 100 documents with 100 passages each. Every additional level then multiplies the number of reachable passages by up to bb, so a corpus of NN passages needs about ⌈log⁡bN⌉\lceil\log_b N\rceil levels. Each level adds one sequential decision, so the latency per query is

T(N)≈treq⌈log⁡bN⌉,(4)T(N) \approx t_{\mathrm{req}} \lceil\log_b N\rceil, \tag*{(4)}

where treq≈0.2t_{\mathrm{req}} \approx0.2 s is the latency of one request. With b=100b = 100, even 10910^9 passages need only five decisions, about one second (Figure 4). Cost, in contrast, grows with the number of judgments: bb children for each of the kk nodes kept per level. Equation (4) bounds the sequential depth rather than the total compute, but because each level keeps only its top kk nodes, no level scores more than k⋅bk \cdot b candidates.

Figure 4. Analytical model: sequential stages grow with the depth of the hierarchy, not the size of the corpus. Theoretical latency per query (Equation (4), b=100b = 100), assuming a balanced hierarchy and enough serving capacity to judge every level in parallel: each 100×100\times more passages adds one level, i.e., one more sequential decision (≈0.2\approx0.2 s).

Experimental Setup

We evaluate DecisionRAG on four benchmarks, ranging from help-center dialogues over 413 pages to open-domain questions over 108,626 Wikipedia pages (Table 1). Unless stated otherwise, we use the pretrained Jev decision model (TypeSafe’s Jev 1.13, typesafe/jev-1.13) (TypeSafe AI, [43]), without task-specific fine-tuning or a conventional first-stage retriever. We also evaluate task-trained, open-weight decision models in § 5.3.

DatasetCorpus hierarchyEvaluation setGold evidence
MultiDoc2Dial4 centers / 413 pages / 4,110 passages4,201 turns1 passage
ConditionalQA652 pages / 58,483 paragraphs285 questions (271 labeled)1–22 paragraphs
MTRAG (IBM Cloud)172 products / 8,578 pages / 72,442 passages205 turns (188 labeled)1–7 passages
Natural Questions (DAPR)1,000 clusters / 108,626 pages / 2.68M paragraphs500 test questions1–3 paragraphs

Table 1. Overview of the four retrieval benchmarks: MultiDoc2Dial [9], ConditionalQA [40], MTRAG [21] and Natural Questions [22] in its DAPR version [46].

Datasets and Corpus Construction

MultiDoc2Dial (Feng et al., [9]) contains goal-oriented dialogues grounded in US government documentation. Con- ditionalQA (Sun et al., [40]) contains scenario-based questions about UK government guidance, often requiring evidence from multiple paragraphs. MTRAG (Katsis et al., [21]) evaluates conversational retrieval over technical documentation; we use its IBM Cloud subset. Natural Questions (Kwiatkowski et al., [22]) contains real search queries grounded in Wikipedia.

For the first three datasets, we construct corpus hierarchies from the existing document structure and metadata. For Natural Questions, whose Wikipedia corpus lacks a hierarchy above the page, we cluster pages into 1,000 groups using k-means over e5-base-v2 embeddings [47] of page titles and first sentences. Each cluster is described by a one-sentence summary generated by Qwen3-8B [42] from its page titles.

Retrieval Configurations

We select dataset-specific navigation hierarchies and retention budgets using separate design sets (Appendix § 10.§ 5). For conversational benchmarks, we include the last six dialogue turns and maintain the notepad described in Appendix § 11.

On MultiDoc2Dial, we navigate a help center →\to page →\to passage hierarchy, retaining the top ten pages and scoring their passages alongside those from up to two notepad pages.

On ConditionalQA, we use a page →\to paragraph hierarchy, retaining the top three pages and jointly scoring their paragraphs.

On MTRAG, we use a page →\to heading →\to passage hierarchy, scoring all page titles in parallel, jointly rescoring the top 50 pages, and retaining the top ten. For long pages, we select relevant headings before scoring their passages.

On Natural Questions, we navigate a cluster →\to page →\to paragraph hierarchy, retaining 150 of the 1,000 clusters, evaluating paragraphs from the top 30 pages.

All DecisionRAG (Jev) configurations finish with multiple-choice ranking of the top 20 passages (top five for Natural Questions); on MTRAG, only the selected passage is promoted to rank one. The + dense candidates variant adds the dense top 20 passages (top ten on ConditionalQA) to the final candidate pool. Additional routing rules and implementation details are provided in Appendix § 11.

We also evaluate task-trained decision models based on Qwen3-Base at 1.7B, 4B, and 8B parameters (§ 5.3). Architecture and training details are provided in Appendix § 16.

Baselines

We compare DecisionRAG with retrieval methods used without task-specific fine-tuning, as well as task-trained methods where available. The baselines are grouped as follows:

First-stage retrieval. We compare against BM25 with stemming and stopword removal, SPLADE++ [10], and Qwen3-Embedding-8B [51]. On ConditionalQA and MTRAG, the appendix also reports SPLADE-v3 with a DistilBERT encoder [23] (Table 11 and Table 12).

Table 11. ConditionalQA development set, retrieval from all 652 pages (%). Page metrics use 285 questions and paragraph metrics the 271 questions with evidence. The paragraph column shows Hit@1 above R@1 / R@5 / R@10. nDCG@10 uses DAPR’s relevance judgments [46]. Dense and BGE as in Table 10.

MethodPage Hit@1 / 3 / 10Paragraph Hit@1R@1 / 5 / 10nDCG@10 (DAPR)
Conventional retrieval pipelines
BM25 + headings51.6 / 74.0 / 89.124.78.9 / 24.6 / 35.027.5
SPLADE-v3 (DistilBERT)62.5 / 80.4 / 94.026.912.1 / 26.9 / 34.128.1
SPLADE++61.4 / 81.4 / 91.928.412.1 / 28.3 / 38.331.2
Dense74.7 / 88.4 / 96.147.221.0 / 43.2 / 54.047.6
Dense top-100 + BGE72.3 / 88.8 / 96.553.122.1 / 48.9 / 60.653.5
Dense top-100 + Qwen3-Reranker68.8 / 78.9 / 95.464.629.3 / 54.8 / 67.262.2
LLM-based listwise reranking
Dense top-20 + GPT-5–73.430.4 / 57.6 / 64.063.4
Task-trained retrieval
DecisionRAG (8B trained), 3 pages–75.328.4 / 66.7 / 78.875.7
DecisionRAG (8B trained), 5 pages–77.129.9 / 67.7 / 80.277.3
Decision-based retrieval (ours)
DecisionRAG (Jev)–69.727.3 / 59.4 / 71.967.5
DecisionRAG (Jev), 5 pages74.0 / 90.2 / 97.571.628.3 / 61.3 / 73.2–
DecisionRAG (Jev) + dense candidates–73.429.9 / 63.2 / 76.371.2

caption: ConditionalQA development set, retrieval from all 652 pages (%). Page metrics use 285 questions and paragraph metrics the 271 questions with evidence. The paragraph column shows Hit@1 above R@1 / R@5 / R@10. nDCG@10 uses DAPR’s relevance judgments [46]. Dense and BGE as in Table 10. The top-100 rerankers rerank the dense top 20 pages for the page column. Cost and latency are in Table 5 and Table 15.

MethodR@1R@5R@10
Conventional retrieval pipelines
BM25 ([9])219.641.950.8
BM25 (ours)20.646.856.4
SPLADE++24.4–58.8
Dense27.960.470.9
Dense top-100 + BGE22.852.863.9
Dense top-100 + Qwen3-Reranker33.671.881.1
LLM-based listwise reranking
Dense top-20 + GPT-5s^{s}47.573.778.6
Task-trained retrieval
DPR fine-tuned ([9])249.072.380.0
DPR + cross-encoder ([52])255.078.383.2
DecisionRAG (8B trained), 3 pages60.782.785.8
DecisionRAG (8B trained), 10 pages60.886.190.0
Decision-based retrieval (ours)
DecisionRAG (Jev)47.476.283.6
DecisionRAG (Jev) + dense candidates48.577.684.7

Table 10. MultiDoc2Dial validation (4,201 turns), passage recall by passage ID (%). Dense: Qwen3-Embedding-8B; BGE: bge-reranker-v2-m3. 2Published results, not rerun.

SystemR@5nDCG@5
DecisionRAG + dense candidates36.665.0
DecisionRAG34.262.3
Dense + Qwen3-Reranker, last turn32.556.5
Dense top-20 + GPT-531.859.4
Dense + Qwen3-Reranker, six turns28.450.9
SPLADE++, last turn27.847.1
SPLADE-v3, last turn27.447.8
ELSER (released, rewritten query)26.646.4
Dense, last turn26.046.5

Table 12. MTRAG, 188 labeled turns, against gold and judged-relevant passages (%); identical texts count once. The remaining nine systems (BM25, dense, SPLADE and BGE variants) trail DecisionRAG by 10 to 20 R@5 points.

Reranking. We evaluate bge-reranker-v2-m3 [6, 2] and Qwen3-Reranker-8B on the top 100 passages retrieved by Qwen3-Embedding-8B. We also evaluate GPT-5 [33] with low reasoning effort, reranking the top 20 dense candidates using a RankGPT-style prompt [41].

Published reference results. We include the MultiDoc2Dial authors’ fine-tuned DPR [20, 9] and DPR with cross-encoder [52], as well as the ELSER retrieval results released with MTRAG [21]. These results are included as references rather than rerun under our evaluation setup.

Evaluation Metrics and Protocol

We denote Recall@k as R@k and report conventional retrieval metrics on corresponding benchmarks in Table 2. Additional metrics are reported in the appendix (Appendix § 12).

MethodMultiDoc2Dial R@1MultiDoc2Dial R@10ConditionalQA Hit@1ConditionalQA R@10MTRAG{sup}‡` R@5MTRAG{sup}‡` nDCG@5Natural Questions Hit@1Natural Questions R@10Natural Questions nDCG@10
Conventional retrieval pipelines
BM2520.656.424.735.018.830.916.442.328.4
SPLADE++24.458.828.438.327.847.136.074.654.5
Dense27.970.947.254.026.046.546.287.266.0
Dense top-100 + BGE22.863.953.160.614.525.354.888.771.7
Dense top-100 + Qwen3-Reranker33.681.164.667.232.556.548.089.069.0
LLM-based listwise reranking
Dense top-20 + GPT-547.578.673.464.031.859.460.888.974.6
Decision-based retrieval (ours)
DecisionRAG (Jev)47.483.669.771.934.262.365.486.776.0
DecisionRAG (Jev) + dense candidates48.584.773.476.336.665.066.488.977.6

Table 2. Retrieval without task-specific training on four benchmarks (%).

To assess downstream question answering, we use Qwen3.5-27B to answer each query from a system’s top five passages. We report exact match on ConditionalQA and Natural Questions and LLM-judged answer acceptance on MTRAG, whose references are long replies (Appendix § 12.§ 1).

Confidence intervals are paired bootstrap 95% intervals (10,000 resamples of whole dialogues, conversations or questions). We also measure per-query cost and online latency (Appendix § 15).

Dense: Qwen3-Embedding-8B; BGE: bge-reranker-v2-m3. Rerankers rerank the dense top 100, GPT-5 the dense top 20. + dense candidates adds the dense top 20 (top 10 on ConditionalQA) to its last stage. Natural Questions: all 2.68M paragraphs are searched.

{sup}‡`MTRAG: gold and GPT-5-judged relevant passages, identical texts counted once (Appendix A.3); conventional baselines use their best input, the last turn (BGE: six turns).

Results

We evaluate DecisionRAG across four benchmarks, comparing its retrieval quality against conventional retrieval and reranking pipelines. We further assess downstream answer quality, the effects of task-specific training, and online latency and cost.

Retrieval Effectiveness

Without task-specific training or a conventional retrieval index, DecisionRAG substantially outperforms dense retrieval with an 8B reranker (Table 2): On MultiDoc2Dial, it improves R@1 by 13.8 percentage points, and on Natural Questions, it improves Hit@1 by 17.4 points.

Natural Questions also shows that DecisionRAG needs no ready-made hierarchy: Wikipedia’s 108,626 pages, far too many for one top level, are simply clustered into 1,000 groups (§ 4.1). DecisionRAG then reads only 16% of the page titles. Any corpus can be given a hierarchy this way, with a number of sequential stages that grows only logarithmically with its size (Equation (4) and § 6.6).

Even when using a strong language model such as GPT-5 to rerank the dense retrieval results, DecisionRAG achieves comparable or higher rank-one accuracy on MultiDoc2Dial and Natural Questions (65.4 versus 60.8). It also improves ConditionalQA R@10 (71.9 versus 64.0) and MTRAG R@5 (34.2 versus 31.8), although GPT-5 achieves higher ConditionalQA Hit@1 (73.4 versus 69.7). On MTRAG, whose annotated evidence was collected with ELSER, we judge the top five passages of every system, and DecisionRAG ranks first among all eighteen, including last-turn and rewritten-query baselines (R@5 34.2 versus 32.5 for the best baseline; Appendix § 10.§ 3 and Table 12).

DecisionRAG can also complement dense retrieval by incorporating dense-retrieved passages into its final ranking stage. This combination improves R@10 from 83.6 to 84.7 on MultiDoc2Dial, from 71.9 to 76.3 on ConditionalQA, and from 86.7 to 88.9 on Natural Questions. These gains suggest that the two approaches recover complementary evidence and can be effectively combined.

Retrieval-Augmented Generation

We next ask whether the retrieved evidence improves downstream answers. We provide each method’s top five passages to the same generator model, Qwen3.5-27B (§ 4.4 and Table 3). The largest improvement occurs on MTRAG, where DecisionRAG increases judged answer correctness from 39.4% with Qwen3-Reranker to 44.1%. Adding dense candidates raises correctness further to 47.3%, compared with 34.0% for GPT-5 reranking. This configuration also achieves the highest exact match on Natural Questions (54.8%) and ConditionalQA (47.6%), where it approaches the performance ceiling set by the gold passages (49.4%).

RetrievalConditionalQA EMMTRAG JudgedNQ EM
None1.86.426.9
Dense43.531.453.0
Dense top-100 + Qwen3-RR45.439.452.4
Dense top-20 + GPT-544.334.054.5
DecisionRAG (Jev)46.944.154.2
+ dense candidates47.647.354.8
Gold passages49.460.165.3

Table 3. Answer quality of one generator (Qwen3.5-27B) given each system’s top five passages (%): exact match, and on MTRAG the share of replies judged correct (188 labeled turns). Qwen3-RR: Qwen3-Reranker-8B; NQ: Natural Questions.

Learning a Retrieval Judge

The retrieval framework also supports task-trained, open-weight decision models. We adapt Qwen3-8B using MultiDoc2Dial training data (Appendix § 16). The resulting model achieves 60.7 R@1 on MultiDoc2Dial, exceeding the strongest published task-trained retriever by 5.7 points (Table 4). This demonstrates that the decision-based retrieval framework is not limited to the pretrained Jev model and can benefit from task-specific adaptation.

MethodMultiDoc2Dial
R@1R@10
Task-trained conventional retrieval
DPR ([9])349.080.0
SFRe2g ([52])355.083.2
Task-trained decision-based retrieval (ours)
DecisionRAG (8B trained)60.785.8

Table 4. Retrieval with task-trained methods (%).

Latency and Cost

Finally, we measure retrieval latency and API cost (Table 5 and Appendix § 15). Under the measured serving configurations, DecisionRAG completes retrieval in 0.68–1.76 seconds per query, compared with 3.3–14.3 seconds for dense retrieval with Qwen3-Reranker-8B on one GPU and 11.8–15.0 seconds for GPT-5 reranking.

MD2DCQAMTRAGNQ
Median latency (s)
Dense top-100 + Qwen3-RR7.53.314.35.8
Dense top-20 + GPT-511.814.412.215.0
DecisionRAG (Jev)0.680.821.761.73
+ dense candidates0.700.851.731.64
API cost per query (¢)
Dense top-20 + GPT-51.401.201.901.40
DecisionRAG (Jev)0.220.291.732.33
+ dense candidates0.230.291.752.33

Table 5. Median latency (seconds) and API cost per query. DecisionRAG rows are live, one query at a time, without reused answers; GPT-5 covers reranking only. Dense top-100 + Qwen3-RR is served locally on one NVIDIA RTX A6000. Qwen3-RR: Qwen3-Reranker-8B; NQ: Natural Questions.

API costs range from 0.22¢ per query on MultiDoc2Dial to 2.33¢ on Natural Questions. Despite the much larger Natural Questions corpus, its measured latency is comparable to MTRAG (1.73 versus 1.76 seconds), illustrating how hierarchical structures can reduce retrieval latency (Equation (4)).

Understanding DecisionRAG

What makes decision-based retrieval effective? We examine how shared context sharpens relevance judgments, when multiple-choice ranking helps, and how navigation strategies and budgets trade off retrieval quality against efficiency.

Scoring Candidates in a Shared Context

A central design choice of DecisionRAG is to judge candidates against a shared context rather than independently. For Natural Questions design questions, we build pools of 5 to 1,000 candidates from the gold paragraph and hard negatives retrieved by BM25, SPLADE-v3, and e5 (Figure 5). Shared-context binary judgments consistently outperform scoring each candidate in a separate request, and multiple-choice ranking performs best within its 255-option limit. Beyond that limit, multiple-choice ranking of the binary top 255 retains much of this advantage: 53.7% Hit@1 with 1,000 candidates versus 47.6% for binary judgments alone.

Figure 5. Hit@1 on Natural Questions as the candidate pool grows, with all NN candidates scored in one request (150 questions).

The advantage carries over to the full pipeline: on MultiDoc2Dial, scoring the passages of all selected pages together rather than page by page raises R@1 from 36.7% to 44.4% (+7.7 points, 95% CI: [3.3, 12.2]; Appendix § 13.§ 1).

Request Size

In a shared context, each probability is formed in view of the alternatives: a candidate that would look relevant on its own receives a lower P(Yes)P(\text{Yes}) when a clearly better one is present. The shared context thus puts the individual probabilities into perspective, making them comparable across candidates. We analyze how this calibration improves as the number of candidates in the context increases.

To this end, we fix the pool at 200 candidates and split it into requests of varying sizes (Figure 6). Hit@1 increases from 47.8% when each candidate is scored separately to 54.3% with 50 candidates per request, with little further improvement for larger requests, so large pools can be split into parallel requests of about 50 candidates without loss. Multiple-choice ranking of all 200 candidates reaches 60.6%, another 6.8 points above binary judgments of all 200 in one request (95% CI: [2.5, 11.0]).

Figure 6. Hit@1 on Natural Questions when a fixed pool of 200 candidates is split into requests of mm candidates and ranked together (289 questions; band: 95% interval).

Decisions versus Generation

Prior work also navigates hierarchies with language models, but lets a generative LLM decide at each level via generating tokens [5, 44, 15]. To isolate the effect of deciding by parallel judgments instead of generation, we keep DecisionRAG’s hierarchy and budgets fixed and replace its decision model by GPT-5 (low reasoning effort), which reads the candidates of each level and writes out the numbers of those to keep, in the style of LATTICE [15]. On ConditionalQA it selects 3 of the 652 pages and ranks their paragraphs; on Natural Questions it selects 150 of the 1,000 clusters (when it lists fewer, we ask for the rest in the same conversation), then 30 pages from their titles, and ranks the paragraphs of the top pages as DecisionRAG does.

Generating the decisions token-by-token brings no consistent gain in accuracy (Table 6): GPT-5 leads DecisionRAG by 4.1 Hit@1 points on ConditionalQA, a gap that adding dense candidates closes, and trails it by 6.0 points on Natural Questions, while being 34 to 145 times slower and 21 to 25 times more expensive, because it must read and reason over every candidate of a level before naming them sequentially token-by-token.

Hit@1R@10s¢
ConditionalQA
GPT-5, generative73.873.628.26.1
DecisionRAG69.771.90.820.29
+ dense candidates73.476.30.850.29
Natural Questions
GPT-5, generative59.082.5250.857.2
DecisionRAG65.086.51.732.33
+ dense candidates64.088.51.642.33

Table 6. The same hierarchy, decided by generation versus decisions. Latency: median seconds per query; cost: cents per query. Natural Questions: the first 100 test questions of Table 2.

Does Multiple-Choice Ranking Improve Retrieval?

After the binary judgments, DecisionRAG optionally applies multiple-choice ranking to the top candidates to refine the final ranking. We examine whether this additional step improves retrieval beyond parallel binary judgments alone.

On MultiDoc2Dial and ConditionalQA design data, we compare configurations with and without multiple-choice ranking while keeping the candidate pool and preceding binary judgments fixed. Multiple-choice ranking improves MultiDoc2Dial R@1 from 50.1% to 51.3% and ConditionalQA Hit@1 from 71.9% to 76.5% (Table 7). Both configurations include additional dense retrieval candidates.

DatasetMetricBinary only+ Multiple-choice
MultiDoc2DialR@150.151.3
ConditionalQAHit@171.976.5

Table 7. Multiple-choice ranking on design data (%; Multi-Doc2Dial: 1,832 turns, ConditionalQA: 285 questions). Multiple-choice ranking reorders the top 20 passages, with only these passages in its context (Appendix § 11.§ 6).

The additional request takes 0.12–0.15 seconds and increases API cost by 4–33%, making multiple-choice ranking a cheap final step that sharpens the top rank.

Navigation versus Reranking

Do the gains come from navigating the hierarchy or from the decision model itself? We give Jey the top 100 passages of dense retrieval (Qwen3-Embedding-8B) and score them as DecisionRAG’s last stage does: binary judgments in one shared context, followed by multiple-choice ranking of the top 20 (top five on Natural Questions). We compare this reranker with DecisionRAG.

Navigating the hierarchy outperforms reranking the retriever’s candidates (Table 8): Hit@1 rises by 7.7 points on ConditionalQA (95% CI: [2.6, 13.3]) and by 7.0 points on Natural Questions ([3.4, 10.8]), while MultiDoc2Dial and MTRAG are level. Even without any retrieval index, DecisionRAG (Table 2) improves Hit@1 over the reranker by 6.0 points on Natural Questions ([2.4, 9.8]) and by 4.0 points on ConditionalQA. This holds although the dense top 100 contains the gold evidence more often than the navigated candidates (Natural Questions: 98.4% versus 94.0%; ConditionalQA: 98.5% versus 91.1%): when both contain it, navigation ranks it first more often (70.1% versus 61.9% and 76.7% versus 67.3%), plausibly because it presents whole pages in reading order rather than isolated passages.

Table 8. Jev reranking the dense top 100 versus DecisionRAG with dense candidates, both using the same dense index (%). MD2D: MultiDoc2Dial, seeded sample of 1,000 validation turns; CQA: ConditionalQA; MTRAG: official labels; NQ: Natural Questions.

Dense top-100 + Jev RerankDecisionRAG + dense candidates
MD2D R@149.951.2
R@1086.685.6
CQA Hit@165.773.4
R@1068.276.3
MTRAG R@538.338.7
nDCG@536.835.7
NQ Hit@159.466.4
nDCG@1072.277.6

Table 8.

Navigation Budget and Hierarchy

Finally, we examine how the navigation budget affects retrieval.

Pruning at the top level is what keeps navigation cheap on large corpora, but it is also where evidence can be lost: a page in a discarded cluster can never be retrieved. We investigate how many clusters must be kept to match the recall of scoring every page directly, varying the share of the 1,000 Natural Questions clusters that DecisionRAG keeps up to flat scoring of all 108,626 page titles (Figure 7). Recall@10 rises steeply up to about 10% of the clusters and then stays level: keeping 15% matches flat scoring (86.2% versus 84.5%) at a fifth of the cost (2.3¢ versus 12.1¢ per query) and a third of the latency (1.7 versus 4.5 seconds). Below 5%, the cluster holding the gold page is often discarded, and recall@10 falls to 63.7% at 1%.

Recall@10 and Jev cost per query as the top level keeps more clusters

Figure 7. Recall@10 (left axis) and Jev cost per query (right axis) on Natural Questions as the top level keeps more and more clusters. First 100 test questions of Table 2; band: 95% interval.

The same trade-off appears one level down, in the page budget. On MultiDoc2Dial, we increase the number of newly retained pages from three to ten, keeping the remaining configuration fixed. R@10 improves from 81.0% to 83.6%, while R@1 increases only from 46.9% to 47.4% (Table 9). The larger budget increases API cost by 83% and median latency by 28%. Exploring more pages therefore primarily improves evidence coverage, with diminishing returns for the top-ranked passage.

Table 9. Page budget on MultiDoc2Dial (validation, with multiple-choice ranking).

New pagesR@1 (%)R@10 (%)API cost (¢/query)Median time (s)
346.981.00.120.53
1047.483.60.220.68

Table 9.

Limitations

DecisionRAG relies on a corpus hierarchy whose structure and node descriptions allow the decision model to identify relevant branches. Poorly organized documents or uninformative descriptions can therefore hurt retrieval quality. Moreover, navigation prunes branches irreversibly, so relevant passages may be missed if their ancestors are discarded; we show that larger exploration budgets and complementary dense retrieval can mitigate this.

Conclusion

We introduced DecisionRAG, which brings the capabilities of large language models into retrieval by navigating corpus hierarchies with non-generative decisions. At each level, candidates are evaluated jointly in a shared context, allowing their relevance to be judged in view of one another. By progressively narrowing the search space, DecisionRAG retrieves relevant evidence with a number of sequential decisions that grows only logarithmically with corpus size under a balanced hierarchy. As these decisions improve, retrieval can inherit their capabilities directly.

Acknowledgements

This work was supported by the National Science Foundation (NSF) under grants OAC-2118310 and IIS-2530143, through the AI Research Institutes program (Award No. DMR-2433348), by New York-Presbyterian for the NYP-Cornell Cardiovascular AI Collaboration, and by arXiv.

Dataset and Corpus Details

MultiDoc2Dial

MultiDoc2Dial [9] contains goal-oriented dialogues grounded in 488 documents from four US government help centers. We build a help-center–page–passage hierarchy from the dataset’s metadata. Help centers are given by the domain field, and documents sharing a title after removing the #1/#2 part suffix form one page, giving 413 pages. We use the benchmark’s official structure-based segmentation into 4,110 passages.

Passages are represented by their full text and their page and parent-heading path, pages by their titles and section headings, and the four help centers by one-line descriptions written manually from their names. We evaluate on the official validation split (661 dialogues, 4,201 turns). Each turn has one gold passage containing the grounding span of the agent’s response. The released test split provides no usable evaluation examples.

ConditionalQA

ConditionalQA [40] contains scenario-based questions about UK government guidance whose answers may depend on several conditions and paragraphs. Its corpus contains 652 guides. We recover their structure from the HTML headings (h1, h2) and extract 58,483 paragraph-level units from p, li and tr elements. Pages are represented by their titles and top-level part headings, and paragraphs are grouped under their heading paths. Retrieval searches the full corpus rather than assuming the relevant guide is known. We evaluate on the 285 questions of the labeled development split, 271 of which have annotated evidence (about four paragraphs on average).

MTRAG–IBM Cloud

We use the IBM Cloud subset of MTRAG [21], with 8,578 documentation pages and 72,442 passages, keeping the benchmark’s document files and passage chunks. Pages are grouped into 172 products by the first path segment after /docs/ in their URLs. Because the corpus does not preserve HTML markup, we recover section headings from the text layout (short lines separated by blank lines). Pages are represented by their titles and recovered headings, and passages by their text and heading. Product descriptions are built automatically from the product name, the opening of its landing page and three page titles.

The held-out evaluation set contains 26 human-written conversations with 205 turns, 188 of which have annotated evidence (one to seven gold passages).

Expanded relevance judgments. MTRAG’s gold passages were selected from the annotators’ ELSER retrieval sessions, so relevant passages that ELSER did not surface cannot be gold. We therefore add LLM-judged relevant passages to the gold passages. For each turn, we pool the top five passages of all 18 systems evaluated on MTRAG, including released ELSER and the last-turn and rewritten-query baselines, together with the gold passages, representing identical texts once. GPT-5 receives the conversation, the reference reply and the shuffled candidates, without knowing which system retrieved them, and grades each passage as needed for the answer (2), related but insufficient (1) or unrelated (0). Grade-2 passages are added to the relevance set, and all original gold passages are kept. GPT-5 grades 87.9% of the gold passages as needed. The expanded labels reduce the dependence on ELSER. Table 12 compares all systems.

Natural Questions

We use the DAPR version [46] of Natural Questions [22], whose corpus splits 108,626 Wikipedia pages into 2,682,017 paragraphs. We evaluate on a fixed random sample of 500 of the 3,610 questions in DAPR’s test split, which is drawn from NQ’s development set. A question’s gold paragraphs (one to three) lie on a single page. Since Wikipedia has no structure above the page, we cluster the pages into 1,000 groups with kk-means over e5-base-v2 embeddings [47] of each page’s title and first sentence. Groups hold 15 to 350 pages (median 106). Each group is described by one sentence that Qwen3-8B writes from its page titles (Appendix § 11.§ 5). Pages are represented by their titles and paragraphs by their text, grouped under their page title.

Design Sets and Configuration Selection

Retrieval configurations (candidate representations, routing rules and retention budgets) are selected on design sets that are disjoint from the evaluation sets. Selection does not update the parameters of Jev. MultiDoc2Dial uses samples of its training split and ConditionalQA 300 training questions. MTRAG has no training split, so we use 50 synthetic IBM Cloud conversations (292 turns) and 131 IBM Cloud turns from MTRAG-UN [37], whose conversations are disjoint from the evaluation set.

Retrieval Configuration Details

Candidate Representations and Questions

At each stage, the judge receives a context containing the query and the numbered candidate descriptions, followed by one yes/no question per candidate. Candidates are listed in document order and grouped under their parent pages where applicable. Each stage builds a new context; probabilities from earlier stages are not included. Conversational queries contain the six most recent dialogue turns.

For conversational retrieval, the page and passage questions are “Is the information the agent needs for its next reply on Page nn?” and “Should the agent’s next reply be based on Passage nn?” For ConditionalQA and Natural Questions, they are “Is the information needed to answer the user’s question on Page nn?” and “Does Paragraph nn contain information needed to answer the user’s question?”

MultiDoc2Dial and the Conversational Notepad

The routing context describes the four help centers, with one question per center. We keep the highest-scoring center, the runner-up if the top probability is below 0.8, and the center of the most recently retrieved page. The page stage scores the pages of the kept centers and keeps ten. All passages of these pages, plus those of the two most recent notepad pages, are scored in one shared context.

The notepad, used on MultiDoc2Dial and MTRAG, lists distinct page identifiers by most recent retrieval. After each turn, the page of the top-ranked passage moves to the front. The page-stage context lists the notepad pages. The notepad is built only from the system’s own retrieval decisions, never from gold evidence, and previously retrieved passages are neither marked nor excluded.

ConditionalQA

The page stage scores all 652 guides in one request and keeps three, whose paragraphs are scored together under their heading paths.

MTRAG

The main configuration scores pages directly instead of routing through products. In a first pass, all 8,578 page titles are split into parallel requests under the input-size limit, keeping each product’s pages together, and the top 50 pages by pooled score are kept. Because these scores come from different contexts, a second pass rescores the 50 together with their product names and headings, and keeps ten.

Pages with more than 60 passages pass through an extra stage that scores their headings and keeps the top eight. At most 60 passages of these headings, in document order, continue to the passage stage.

Natural Questions

Each cluster is described by Qwen3-8B, given its page titles (most typical first, at most 300) and the prompt: “Write one sentence (at most 30 words) describing what the pages in this group are about. Name the main kinds of subjects it covers, including the less typical ones, so that a reader can tell whether a given Wikipedia page would belong to this group.” The judge scores the 1,000 cluster descriptions in eight parallel requests, asking for each whether “the page with the information needed to answer the user’s question [is] likely in Topic nn”, and keeps 150 clusters. Their page titles, about 17,500 per question, are scored in parallel requests of at most 20k tokens. The paragraph stage scores all paragraphs of the top three pages, the first ten of pages ranked 4–10 and the first three of pages ranked 11–30 in one shared context, about 230 paragraphs per question. Each paragraph’s probability is multiplied by its page’s.

Multiple-Choice Ranking

The multiple-choice question follows binary scoring. Its answer options are the top 20 candidates by P(yes)P(\text{yes}) (top five on Natural Questions), and its context lists only these candidates, in reading order and formatted as in binary scoring. On MultiDoc2Dial, ConditionalQA and Natural Questions, the choice probabilities reorder the shortlist, and all other candidates keep their binary order. On MTRAG, only the highest-probability choice moves to rank one. Compared with a choice request over the full binary context, the shorter context changes retrieval by at most one point on every dataset and shortens the MTRAG choice request from 0.30–0.37 to 0.14 seconds. Retrieval without the choice step is evaluated in Section 6.4.

Full Benchmark Results

The tables below extend Table 2 with secondary metrics, published reference results, task-trained systems and the full MTRAG comparison; cost and latency are in Table 5 and Appendix § 15. Rows named as in Table 2 are the same configurations. A dash denotes a result or measurement that is unavailable, never a zero. Open-weight judge results are in Table 17.

MultiDoc2Dial validation
Judge / trainingPagesR@1R@5R@10
1.7B / general320.538.344.2
1.7B / general1012.828.736.3
4B / general328.552.959.8
4B / general1021.947.256.5
8B / general340.466.071.9
8B / general1037.365.373.0
8B / MD2D only363.184.286.6
8B / MD2D only1064.387.691.3
8B / general →\to MD2D363.584.186.4
8B / general →\to MD2D1065.887.991.2
8B / general + MD2D + CQA (LoRA)360.782.785.8
8B / general + MD2D + CQA (LoRA)1060.886.190.0
ConditionalQA development (271 questions with evidence)
Judge / trainingPagesHit@1R@5R@10
8B / general335.436.644.5
8B / general538.437.345.5
8B / general + MD2D + CQA (LoRA)375.366.778.8
8B / general + MD2D + CQA (LoRA)577.167.780.2

Table 17. Open-weight judges (%). General: trained without MultiDoc2Dial or ConditionalQA data; + MD2D + CQA (LoRA): the DecisionRAG (8B trained) model of Table 4.

Cost and latency are in Table 5 and Table 15.

DatasetConfigurationScopennp50/p95 (s)API ¢/query
MultiDoc2DialDecisionRAG (Jev)W480.68/0.910.22
+ dense candidatesH480.70/1.160.23
Dense top-20 + GPT-5R1,00011.8/23.31.40
ConditionalQADecisionRAG (Jev)W500.82/1.020.29
+ dense candidatesH500.85/1.130.29
Dense top-20 + GPT-5R28514.4/31.41.20
MTRAGDecisionRAG (Jev)W241.76/2.261.73
+ dense candidatesH241.73/2.641.75
Dense top-20 + GPT-5R18812.2/20.81.90
Natural QuestionsDecisionRAG (Jev)W1001.73/2.542.33
+ dense candidatesH1001.64/2.232.33
Dense top-20 + GPT-5R50015.0/23.81.40

Table 15. Online timing per query. nn: timed queries; W: complete retrieval after initialization; H: retrieval and scoring with saved dense candidates; R: GPT-5 listwise reranking of the saved dense top 20, API only.

MTRAG. Table 12 compares all systems against the gold and judged-relevant passages of Appendix § 10.§ 3. DecisionRAG’s R@5 exceeds released ELSER by 7.6 points (95% interval [2.2, 13.1]), last-turn SPLADE-v3 by 6.8 [1.9, 12.3] and six-turn Qwen3-Reranker by 5.8 [1.9, 9.6]; it is also ahead of last-turn Qwen3-Reranker and GPT-5 reranking.

Retrieval-augmented generation

The generator is Qwen3.5-27B (FP8 weights, thinking disabled, greedy decoding), served with vLLM. Its input is a short task instruction, the passages numbered in each system’s rank order, and the query: the user’s situation and question on ConditionalQA, the question on Natural Questions, and the last six turns on MTRAG. It answers ConditionalQA with “Answer:” followed by yes, no, a short phrase or “not answerable” (optionally with conditions), Natural Questions with a phrase of at most five words, and MTRAG with a reply that says when the passages do not contain the answer. Exact match uses the standard answer normalization; the Natural Questions references are the short answers of NQ-open, available for 334 of the 500 questions. The judge is the same model, shown the query, the reference answer(s) and the generated answer, and asked whether the answer is correct (for MTRAG, consistent with the reference response, and a refusal on unanswerable turns). Part of the generations and verdicts were produced by the same model through the OpenRouter API instead of our local deployment; on 200 prompts answered by both, answer quality matched and the verdicts agreed on 96.5%.

Additional Analyses

Shared context and conversational memory

The configurations in Table 13 use 297 turns from 50 complete training dialogues, six-turn input, title-only page descriptions, ten newly selected pages, no help-center routing and no choice step, so their scores are not comparable to the main results. Separate scoring builds one context per page, containing the conversation, the page title and its passages with their heading paths; shared scoring lists the passages of all kept pages in one context, grouped by page. Both rank passages by P(yes)P(\mathrm{yes}) for “Should the agent’s next reply be based on Passage nn?”. Own notepad is the page memory of Appendix § 11.§ 2, built by each configuration from its own decisions. Paired R@1 differences, resampling whole dialogues, are +4.0 [0.7, 8.0] for adding the notepad and +7.7 [3.3, 12.2] for shared over separate scoring with notepads.

Passage contextNotepadR@1R@5R@10
SeparateNone32.754.563.6
SeparateOwn36.761.370.4
SharedOwn44.469.776.8

Table 13. MultiDoc2Dial passage context and page memory on 297 design turns (%).

Remaining retrieval errors

Given the gold page, passage scoring on MultiDoc2Dial design turns reaches 62.7 R@1 and 94.3 R@5. On the ConditionalQA development set, the three-page configuration without multiple-choice ranking misses the gold page for 8.5% of evidence-labeled questions, ranks a paragraph from another page first for 9.6%, and ranks another paragraph of the gold page first for 13.7%. Passage ranking thus remains a substantial source of error even when navigation reaches the relevant page.

Dense candidates and multiple-choice ranking

  • dense candidates adds the top 20 dense-retrieved passages (top 10 on ConditionalQA) to the navigated candidates, removes duplicates, and groups all candidates by page and section in reading order. Multiple-choice ranking either reorders the top 20 (order) or moves only its pick to rank one (top1). Table 14 reports the design-set comparisons, at three pages on MultiDoc2Dial and ConditionalQA and ten pages on MTRAG-UN. The design sets are 1,832 turns from 300 MultiDoc2Dial training dialogues, the 285 of the 300 ConditionalQA design questions that have paragraph evidence, and 86 labeled MTRAG-UN turns, one per conversation. Δ\Delta is the difference in the primary metric to the configuration with neither component, with an unadjusted paired 95% interval. With larger page budgets (ten pages on MultiDoc2Dial, five on ConditionalQA) the gains shrink: at most +0.6 R@1 on MultiDoc2Dial, with every interval containing zero, and at most +4.2 Hit@1 on ConditionalQA.

ConfigurationPrimaryR@10Δ primary [95% CI]
MultiDoc2Dial
3 pages48.282.5–
3 pages + dense 2050.186.1+1.9 [+0.9, +3.0]
3 pages + dense 20 + multiple-choice51.386.6+3.1 [+1.5, +4.7]
ConditionalQA
3 pages71.274.1–
3 pages + dense 1071.976.5+0.7 [−1.4, +2.8]
3 pages + dense 10 + multiple-choice76.578.0+5.3 [+1.1, +9.5]
MTRAG-UN
10 pages52.356.4–
10 pages + dense 2057.866.2+5.5 [+0.0, +11.4]
10 pages + dense 20 + multiple-choice (top1)57.566.2+5.2 [−0.3, +11.2]

Table 14. Last-stage design comparisons (%). Primary metric: R@1 (MultiDoc2Dial), Hit@1 (ConditionalQA), R@5 (MTRAG-UN).

Evaluation Sensitivity

We rescore the retrieval results under two alternative relevance definitions.

Identical passages under different IDs

A retrieved passage can repeat the gold text under a different ID. We group passages by their text after whitespace normalization (and lowercasing for MTRAG), map retrieved and gold IDs to these groups, and keep only the first occurrence of each group in a ranking, so duplicate copies cannot create multiple hits.

Text matching raises MultiDoc2Dial R@1 to 55.8 for DecisionRAG (57.0 with dense candidates) and to 40.3 for dense retrieval with Qwen3-Reranker; the ordering of systems is unchanged.

ConditionalQA under DAPR relevance judgments

DAPR provides alternative relevance judgments for ConditionalQA, evaluated with nDCG@10 (Wang et al., [46]) (last column of Table 11). On the same 271 questions, DecisionRAG’s nDCG@10 exceeds dense retrieval with Qwen3-Reranker by +5.3 points (95% interval [1.3, 9.2]) and GPT-5 by +4.0 [0.3, 7.8], and its DAPR R@10 exceeds GPT-5 by +8.2 [4.0, 12.3]. With dense candidates, nDCG@10 rises to 71.2.

Latency and Cost

DecisionRAG timing. Timing runs call the live API without reading cached answers, process one query at a time with up to eight concurrent requests within a query, and run from an AMD EPYC 9745 client. The timer covers complete retrieval, including retries and waiting, and starts after initialization (corpus loading); no warm-up query is discarded, and quantiles are computed from per-query totals. Each stage waits for the preceding one; the choice request follows the binary request over the top candidates only and adds 0.12 to 0.15 seconds. Rows with dense candidates (H) load saved dense candidates and exclude dense encoding and search, so they are not end-to-end latencies. MultiDoc2Dial timing uses 48 turns from eight complete training dialogues, each configuration with its own notepad; ConditionalQA uses 50 development questions, MTRAG 24 turns from three human conversations, and Natural Questions the first 100 of its 500 test questions. All rows were measured from the same client, the MultiDoc2Dial rows in a later session; small differences between live runs can reflect provider variation. The main MTRAG configuration sends the first page pass as requests of at most 10k tokens instead of 150k (about 40 parallel requests instead of three); on the 188 held-out turns this leaves R@5 and nDCG@5 unchanged (paired differences −0.7 and −0.2 points, intervals containing zero) and reduces median latency from 3.09 to 1.76 seconds.

Baseline timing. Locally served baselines (Table 16) are timed per query after model and index loading and warm-up, with GPU synchronization, on an RTX 3090 (BM25 on CPU). Dense top-100 + Qwen3-Reranker is timed on one RTX A6000 on seeded samples of 48, 50, 24 and 100 queries (MultiDoc2Dial, ConditionalQA, MTRAG, Natural Questions): query encoding with Qwen3-Embedding-8B, exact top-100 search over the corpus vectors (2.68M on Natural Questions), and reranking of the 100 candidates in batches of up to 32,768 padded tokens. Reranking accounts for nearly all of its latency; encoding and search take 0.05–0.08 seconds. GPT-5 (low reasoning effort) reranks 20 saved dense candidates with up to 16 API calls in flight; its timing and cost (rows R in Table 15) exclude dense retrieval and use a seeded sample of 1,000 validation turns on MultiDoc2Dial and the full evaluation sets elsewhere. Because hardware and samples differ across systems, latency ratios are not controlled speedups.

SystemMultiDoc2DialConditionalQAMTRAG
BM250.0010.0010.008
SPLADE-v3 (DistilBERT)–0.1170.298
SPLADE++–0.0900.177
Dense0.0660.0470.116
Dense top-100 + Qwen3-Reranker (A6000)7.513.3314.33

Table 16. Median latency of the locally served baselines (seconds per query, RTX 3090 unless noted; BM25 on CPU), including encoding and search. BM25 on ConditionalQA indexes headings with the paragraphs; MTRAG uses the six-turn input.

Offline preprocessing. Offline indexing is excluded from all online measurements. Encoding the corpus with Qwen3-Embedding-8B took 180 s for MultiDoc2Dial, 781 s for ConditionalQA and 9,215 s for MTRAG on an RTX 3090; SPLADE++ and SPLADE-v3 took 317 s and 243 s for MTRAG. DecisionRAG encodes no passages. Only on Natural Questions does building the hierarchy require an offline pass: embedding each page’s title and first sentence with e5-base-v2 for clustering and describing the 1,000 clusters with Qwen3-8B (Appendix § 10.§ 4).

Open-Weight Judge Training

Model. The open-weight judges are Qwen3-Base models (1.7B, 4B and 8B) whose language-model head is replaced by a scoring head. Each candidate is posed as its own question with the answer options “no” and “yes”, appended to the shared context. All questions start at the position right after the context and cannot attend to each other, so the judgments of different candidates depend only on the shared context, as in Equation (2). The head projects the hidden states of the question’s end token and of each option’s end token into a common space, scales their cosine similarity by a learned inverse temperature (capped at 100), and normalizes over the question’s options with a softmax; P(yes)P(\mathrm{yes}) is the probability of “yes”. Training minimizes the cross-entropy over the options. In the general and MultiDoc2Dial-only checkpoints, the end tokens of a question’s two options can also attend to each other; the LoRA checkpoint does not use this link.

General training. The general models are trained on a mixture of public multiple-choice and classification datasets, including reading comprehension (RACE, QuALITY), science and commonsense question answering (SciQ, ARC, CommonsenseQA, HellaSwag), natural language inference (MNLI), intent and topic classification, and dialogue, tool-use and web-navigation decisions (MultiWOZ, xLAM, Mind2Web). The label of each example is the dataset’s gold option; the alternatives are its own distractors or, for classification, a random subset of the class labels. The mixture contains no MultiDoc2Dial or ConditionalQA data.

Retrieval training data. MultiDoc2Dial training examples are generated from its training split with the routing, page and passage stages of the navigation; 3% of training dialogues are held out for monitoring. The generator simulates the notepad with a gold page with probability 0.65, includes the gold page among three candidates with probability 0.85, and adds a second help-center negative stage with probability 0.17; identical-text copies of the gold passage are never used as negatives. ConditionalQA examples are built by the navigation’s own page and paragraph stages, so their inputs match those at inference. A page-stage example lists all 652 pages with “yes” for the gold page; a paragraph-stage example lists every paragraph of five candidate pages with “yes” for the evidence paragraphs. The gold page’s rank among the five candidates is drawn from Jev’s page-stage ranks on the 300 ConditionalQA design questions, and the remaining candidates are the top BM25 pages; when the gold page is not drawn, all paragraphs are negatives. Paragraphs whose exact text appears as evidence on another page are not asked, and questions without evidence contribute only a page-stage example. Questions on 10% of the training pages are held out for monitoring.

Optimization. The fine-tuning runs on retrieval data use AdamW (β1=0.9\beta_{1} = 0.9, β2=0.95\beta_{2} = 0.95, no weight decay), gradient clipping at 1.0, 50 warm-up steps and cosine decay to 10% of the peak learning rate. The MD2D-only model trains the head alone for 200 steps (learning rate 10−310^{-3}) and then the full model for one epoch; general →\to MD2D continues from the general 8B checkpoint for one epoch without the head-only phase. Both use learning rates of 10−510^{-5} (backbone) and 10−410^{-4} (head), a maximum context of 32,768 tokens and packed rows of 40,960 tokens; overlong passage-stage records (5.5%) are dropped rather than truncated. The DecisionRAG (8B trained) model reported in Table 4 instead continues the general 8B model with LoRA (rank 64, α=128\alpha= 128, on the attention and MLP projections of every layer) and a fully trained head, both at learning rate 10−410^{-4}, for one epoch over 30% of the MultiDoc2Dial training dialogues (a seeded sample) and all ConditionalQA training examples, with rows of 65,536 tokens.

Results. The LoRA model is the only checkpoint evaluated on both datasets; the checkpoints trained on MultiDoc2Dial alone reach higher MultiDoc2Dial recall (Table 17). All models are evaluated with the retrieval configurations of Appendix § 11, ranking by their binary judgments.

References

  1. [1]Anthropic. Claude Opus 5.5. https://www.anthropic.com/claude, 2026.
  2. [2]BAAI. bge-reranker-v2-m3. Hugging Face model card, 2024. URL https://huggingface.co/BAAI/bge-reranker-v2-m3.
  3. [3]Barnett, S., Kurniawan, S., Thudumu, S., Brannelly, Z., and Abdelrazek, M. Seven failure points when engineering a retrieval augmented generation system. In CAIN, pp. 194–199. ACM, 2024.arxiv.org/abs/2401.05856
  4. [4]Bateni, M., Behnezhad, S., Derakhshan, M., Hajiaghayi, M., Kiveris, R., Lattanzi, S., and Mirrokni, V. S. Affinity clustering: Hierarchical clustering at scale. In NIPS, pp. 6864–6874, 2017.
  5. [5]Chen, H., Pasunuru, R., Weston, J., and Celikyilmaz, A. Walking down the memory maze: Beyond context limit through interactive reading. CoRR, abs/2310.05029, 2023.arxiv.org/abs/2310.05029
  6. [6]Chen, J., Xiao, S., Zhang, P., Luo, K., Lian, D., and Liu, Z. M3-embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. In ACL (Findings), volume ACL 2024 of Findings of ACL, pp. 2318–2335. Association for Computational Linguistics, 2024.arxiv.org/abs/2402.03216
  7. [7]Chen, S., Gutierrez, B. J., and Su, Y. Attention in large language models yields efficient zero-shot re-rankers. In ICLR. OpenReview.net, 2025.arxiv.org/abs/2410.02642
  8. [8]Dhulipala, L., Lacki, J., Lee, J., and Mirrokni, V. Terahac: Hierarchical agglomerative clustering of trillion-edge graphs. Proc. ACM Manag. Data, 1(3):221:1–221:27, 2023.arxiv.org/abs/2308.03578
  9. [9]Feng, S., Patel, S. S., Wan, H., and Joshi, S. Multidoc2dial: Modeling dialogues grounded in multiple documents. In EMNLP (1), pp. 6162–6176. Association for Computational Linguistics, 2021.
  10. [10]Formal, T., Lassance, C., Piwowarski, B., and Clinchant, S. From distillation to hard negative sampling: Making sparse neural IR models more effective. In SIGIR, pp. 2353–2359. ACM, 2022.
  11. [11]Gao, T., Yen, H., Yu, J., and Chen, D. Enabling large language models to generate text with citations. In EMNLP, pp. 6465–6488. Association for Computational Linguistics, 2023a.arxiv.org/abs/2305.14627
  12. [12]Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Guo, Q., Wang, M., and Wang, H. Retrieval-augmented generation for large language models: A survey. CoRR, abs/2312.10997, 2023b.arxiv.org/abs/2312.10997
  13. [13]Geva, M. and Berant, J. Learning to search in long documents using document structure. In COLING, pp. 161–176. Association for Computational Linguistics, 2018.arxiv.org/abs/1806.03529
  14. [14]Gupta, N., You, C., Bhojanapalli, S., Kumar, S., Dhillon, I. S., and Yu, F. X. Scalable in-context ranking with generative models. In NeurIPS, 2025.arxiv.org/abs/2510.05396
  15. [15]Gupta, N., Chang, W., Bui, N., Hsieh, C., and Dhillon, I. S. Llm-guided hierarchical search for end-to-end reasoning intensive retrieval. Trans. Mach. Learn. Res., 2026, 2026.arxiv.org/abs/2510.13217
  16. [16]Guu, K., Lee, K., Tung, Z., Pasupat, P., and Chang, M. Retrieval augmented language model pre-training. In ICML, volume 119 of Proceedings of Machine Learning Research, pp. 3929–3938. PMLR, 2020.
  17. [17]Izacard, G., Caron, M., Hosseini, L., Riedel, S., Bojanowski, P., Joulin, A., and Grave, E. Unsupervised dense information retrieval with contrastive learning. Trans. Mach. Learn. Res., 2022, 2022.arxiv.org/abs/2112.09118
  18. [18]Jeong, S., Baek, J., Cho, S., Hwang, S. J., and Park, J. C. Adaptive-rag: Learning to adapt retrieval-augmented large language models through question complexity. In NAACL-HLT, pp. 7036–7050. Association for Computational Linguistics, 2024.arxiv.org/abs/2403.14403
  19. [19]Jin, B., Zeng, H., Yue, Z., Wang, D., Zamani, H., and Han, J. Search-r1: Training llms to reason and leverage search engines with reinforcement learning. CoRR, abs/2503.09516, 2025.arxiv.org/abs/2503.09516
  20. [20]Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., and Yih, W. Dense passage retrieval for open-domain question answering. In EMNLP (1), pp. 6769–6781. Association for Computational Linguistics, 2020.arxiv.org/abs/2004.04906
  21. [21]Katsis, Y., Rosenthal, S., Fadnis, K., Gunasekara, C., Lee, Y., Popa, L., Shah, V., Zhu, H., Contractor, D., and Danilevsky, M. mtrag: A multi-turn conversational benchmark for evaluating retrieval-augmented generation systems. Trans. Assoc. Comput. Linguistics, 13:784–808, 2025.
  22. [22]Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A. P., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., Toutanova, K., Jones, L., Kelcey, M., Chang, M., Dai, A. M., Uszkoreit, J., Le, Q., and Petrov, S. Natural questions: a benchmark for question answering research. Trans. Assoc. Comput. Linguistics, 7:452–466, 2019.DOI
  23. [23]Lassance, C., Déjean, H., Formal, T., and Clinchant, S. Splade-v3: New baselines for SPLADE. CoRR, abs/2403.06789, 2024.
  24. [24]Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W., Rocktäschel, T., Riedel, S., and Kiela, D. Retrieval-augmented generation for knowledge-intensive NLP tasks. In NeurIPS, 2020.arxiv.org/abs/2005.11401
  25. [27]Liu, Y., Hashimoto, K., Zhou, Y., Yavuz, S., Xiong, C., and Yu, P. S. Dense hierarchical retrieval for open-domain question answering. In EMNLP (Findings), volume EMNLP 2021 of Findings of ACL, pp. 188–200. Association for Computational Linguistics, 2021.
  26. [28]Monath, N., Dubey, K. A., Guruganesh, G., Zaheer, M., Ahmed, A., McCallum, A., Mergen, G., Najork, M., Terzihan, M., Tjanaka, B., Wang, Y., and Wu, Y. Scalable hierarchical agglomerative clustering. In KDD, pp. 1245–1255. ACM, 2021.DOI
  27. [29]Nistér, D. and Stewénius, H. Scalable recognition with a vocabulary tree. In CVPR (2), pp. 2161–2168. IEEE Computer Society, 2006.DOI
  28. [30]Nogueira, R. and Cho, K. Passage re-ranking with BERT. CoRR, abs/1901.04085, 2019.arxiv.org/abs/1901.04085
  29. [31]Nogueira, R., Jiang, Z., Pradeep, R., and Lin, J. Document ranking with a pretrained sequence-to-sequence model. In EMNLP (Findings), volume EMNLP 2020 of Findings of ACL, pp. 708–718. Association for Computational Linguistics, 2020.arxiv.org/abs/2003.06713
  30. [33]OpenAI. Openai GPT-5 system card. CoRR, abs/2601.03267, 2026.
  31. [34]Pirolli, P. and Card, S. Information foraging. Psychological Review, 106(4):643–675, 1999.
  32. [35]Reddy, R. G., Doo, J., Xu, Y., Sultan, M. A., Swain, D., Sil, A., and Ji, H. FIRST: faster improved listwise reranking with single token decoding. In EMNLP, pp. 8642–8652. Association for Computational Linguistics, 2024.arxiv.org/abs/2406.15657
  33. [36]Robertson, S. E. and Zaragoza, H. The probabilistic relevance framework: BM25 and beyond. Found. Trends Inf. Retr., 3(4):333–389, 2009.
  34. [37]Rosenthal, S., Katsis, Y., Shah, V., He, L., Popa, L., and Danilevsky, M. MTRAG-UN: A benchmark for open challenges in multi-turn RAG conversations. In ACL (Findings), pp. 10363–10369. Association for Computational Linguistics, 2026.
  35. [38]Sarthi, P., Abdullah, S., Tuli, A., Khanna, S., Goldie, A., and Manning, C. D. RAPTOR: recursive abstractive processing for tree-organized retrieval. In ICLR. OpenReview.net, 2024.arxiv.org/abs/2401.18059
  36. [39]Shuster, K., Poff, S., Chen, M., Kiela, D., and Weston, J. Retrieval augmentation reduces hallucination in conversation. In EMNLP (Findings), volume EMNLP 2021 of Findings of ACL, pp. 3784–3803. Association for Computational Linguistics, 2021.
  37. [40]Sun, H., Cohen, W. W., and Salakhutdinov, R. Conditional-alqa: A complex reading comprehension dataset with conditional answers. In ACL (1), pp. 3627–3637. Association for Computational Linguistics, 2022.arxiv.org/abs/2110.06884
  38. [41]Sun, W., Yan, L., Ma, X., Wang, S., Ren, P., Chen, Z., Yin, D., and Ren, Z. Is chatgpt good at search? investigating large language models as re-ranking agents. In EMNLP, pp. 14918–14937. Association for Computational Linguistics, 2023.arxiv.org/abs/2304.09542
  39. [42]Team, Q. Qwen3 technical report. CoRR, abs/2505.09388, 2025.
  40. [43]TypeSafe AI. Jev 1.13: a decision model. https://openrouter.ai/typesafe/jev-1.13, 2026.
  41. [44]VectifyAI. PageIndex: Document index for reasoning-based RAG. Software repository, 2025. URL https://github.com/VectifyAI/PageIndex.
  42. [46]Wang, K., Reimers, N., and Gurevych, I. DAPR: A benchmark on document-aware passage retrieval. In ACL (1), pp. 4313–4330. Association for Computational Linguistics, 2024.arxiv.org/abs/2305.13915
  43. [47]Wang, L., Yang, N., Huang, X., Jiao, B., Yang, L., Jiang, D., Majumder, R., and Wei, F. Text embeddings by weakly-supervised contrastive pre-training. CoRR, abs/2212.03533, 2022.arxiv.org/abs/2212.03533
  44. [49]Yang, T., Yao, Z., Jin, B., Cui, L., Li, Y., Wang, G., Liu, X., and Neiswanger, W. Demystifying and enhancing the efficiency of large language model based search agents. In ICLR. proceedings.iclr.cc, 2026.arxiv.org/abs/2505.12065
  45. [50]Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K. R., and Cao, Y. React: Synergizing reasoning and acting in language models. In ICLR. OpenReview.net, 2023.arxiv.org/abs/2210.03629
  46. [51]Zhang, Y., Li, M., Long, D., Zhang, X., Lin, H., Yang, B., Xie, P., Yang, A., Liu, D., Lin, J., Huang, F., and Zhou, J. Qwen3 embedding: Advancing text embedding and reranking through foundation models. CoRR, abs/2506.05176, 2025.arxiv.org/abs/2506.05176
  47. [52]Zhang, Y., Fu, H., Zhang, X., and Nguyen, C. Structures facilitate retrieve, rerank, and generate. CoRR, abs/2606.03247, 2026.arxiv.org/abs/2606.03247
  48. [53]Zhuang, S., Zhuang, H., Koopman, B., and Zuccon, G. A setwise approach for effective and highly efficient zero-shot ranking with large language models. In SIGIR, pp. 38–47. ACM, 2024.DOI

Paper details

vlnc:W_rcj6zuhh
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 7

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents