betaFeedback

WorksComputer Science

PatternBloom: Empowering Agentic RAG with Externalized RL-Distilled Graph Patterns

Abstract

Reinforcement-learning-trained agents for retrieval-augmented generation retrieve content (passages, knowledge-graph fragments, or hyperedges) and supervise training with outcome rewards on the final answer. Across these paradigms the reasoning route through retrieved content never leaves the language model's hidden state, conflating declarative knowledge with the procedural knowledge that recurs across queries of the same logical type; content-only methods consequently plateau as reasoning depth grows. We introduce PatternBloom, an agentic RAG framework that shifts the unit of retrieval from content (what to read) to logic (how to reason). A first RL stage scores the agent's constructed evidence graph by a frozen oracle's information gain through the Information-Density Reward (IDR), a size-normalized mutual-information surrogate. High-reward trajectories are distilled into a Graph Pattern Memory (GPM) of type-abstracted reasoning skeletons, and a second RL stage uses the Pattern-Augmented Reward (PAR) to couple policy to memory, turning procedural reasoning into a learnable, externalized structural prior. A 7B PatternBloom outperforms 2727 baselines by +15.0+15.0 Avg-OOD EM, with the gain compounding with reasoning depth precisely because procedural knowledge, once distilled, is reused rather than re-derived at every query. We release our code at https://github.com/EllenYiGe/PatternBloom.

Introduction

Retrieval-augmented language model agents must compose retrieved evidence to answer multi-hop questions over large corpora [1, 2, 3, 4]. Recent agentic systems trained with reinforcement learning (Search-R1 [5], Graph-R1 [6], R1-Searcher [7]) push this further by letting an agent decide when and what to retrieve, optimizing tool-use policies with outcome rewards [8, 9]. Across these architectures, however, supervision touches only the retrieved units and the final answer string. Rewards score the correctness of the answer, but the reasoning trajectory between query and answer (which entities the agent binds, in which order, along which logical path) is left to a black-box policy: the agent learns to read better evidence over time, yet does not learn to traverse evidence better.

Existing RAG paradigms differ primarily in what is retrieved: paragraph-based methods (Search-R1, FrugalRAG [10], R1-Searcher) return chunk-level passages; knowledge-graph methods (GraphRAG [11], HippoRAG [12, 13], LinearRAG [14], LightRAG [15], GFM-RAG [16]) build entity-relation graphs offline and return sparse paths; and structure-aware methods (StructRAG [17], Graph-R1, Structure-R1 [18]) retrieve task-adaptive units such as hyperedges or summarized clusters. Despite these different retrieval units, the three paradigms share a common limitation: the reasoning route through the retrieved evidence never leaves the language model’s hidden state. Paragraph methods leave entity binding and relation inference to opaque attention; knowledge-graph and structure-aware methods expose entities as graph nodes, but the path through those nodes that solves a given query is still inferred at run time. Figure 1 (left) frames this collectively as content retrieval: the agent decides what to read (a passage, a subgraph, or a hyperedge), but the reasoning skeleton that links those units is never itself retrieved. The cost compounds on compositionally complex queries, motivating a retrieval target that is the reasoning route itself.

Content Retrieval paradigm versus Logic Retrieval paradigm

Figure 1. Content Retrieval paradigm versus Logic Retrieval paradigm. Left: prior paradigms retrieve content units (passages, knowledge-graph fragments, or hyperedges), leaving the reasoning route implicit in policy weights. Right: PatternBloom retrieves a typed reasoning skeleton from an externalized Graph Pattern Memory and traverses it as an explicit logic path.

Why richer content does not solve the problem. The plateau on deeper queries is not primarily a retrieval-recall failure: knowledge-graph and structure-aware methods often achieve high entity-level coverage. The bottleneck is procedural. Two queries about different films, “in which country was the director of BLACK GOLD born?” and “where was the director of TOP GUN from?”, share an identical logical structure yet are solved from scratch each time: their typed reasoning skeleton, from FILM to PERSON to COUNTRY, is encoded only in the policy’s hidden activations and discarded when the episode ends. Cognitively, this conflates declarative knowledge (the facts to retrieve) with procedural knowledge (the operations that recur across queries of the same logical type) [19]. The two memory systems are functionally distinct in human reasoning and require different storage and retrieval mechanisms; a retrieval system that augments only declarative memory will, by construction, fail to compound experience across queries that share the same procedural skeleton.

From content retrieval to logic retrieval. To this end, we introduce PatternBloom, an agentic RAG system that shifts the retrieval target from content (what to read) to logic (how to reason). The unit of retrieval is a graph pattern (Figure 1, right): a small directed skeleton over typed entity slots that prescribes how to traverse retrieved evidence rather than what evidence to read. The skeleton is type-abstracted, so the same pattern serves any query of the same logical structure (concrete examples in §3.3), making procedural knowledge an explicit, retrievable object rather than a behavior latent in policy weights. A two-stage RL pipeline trains and exploits this externalized memory. Stage I uses an Information-Density Reward to supervise the agent on the information content of the constructed evidence graph rather than on the answer string, promoting the graph to a directly supervised quantity. An inter-stage distillation extracts type-abstracted skeletons from high-reward trajectories into a Graph Pattern Memory. Stage II uses a Pattern-Augmented Reward to couple the policy to the frozen memory, crediting trajectories that traverse retrieved patterns and turning procedural reasoning from a soft prompt hint into a gradient signal. Where prior systems retrieve passages (paragraph methods), subgraphs (knowledge-graph methods), or hyperedges (structure-aware methods), PatternBloom retrieves a typed reasoning skeleton: the logic itself.

Realizing this paradigm shift surfaces three concrete challenges: (C1) supervising the constructed evidence graph as an information object rather than scoring only the answer string; (C2) externalizing reasoning patterns in a form that is reusable across queries and backbones; and (C3) coupling a frozen pattern store back to the policy as a process-level gradient signal. Our main contributions, summarized below, address (C1)–(C3) respectively via the Information-Density Reward, the Graph Pattern Memory, and the Pattern-Augmented Reward.

  • A new retrieval paradigm for agentic RAG: from content to logic. PatternBloom reframes agentic RAG by retrieving typed reasoning skeletons rather than content units. Stored in an

externalized Graph Pattern Memory (GPM) auto-distilled from a Stage-I RL agent’s own high-reward trajectories, these skeletons make procedural knowledge a first-class retrievable artefact: inspectable, frozen at deployment, and reusable across queries and backbones.

  • An information-theoretic two-stage RL framework. Stage I supervises the constructed evidence graph itself via the Information-Density Reward (IDR); Theorem 1 establishes IDR as a bounded monotone surrogate for size-normalized conditional mutual information, the first information-theoretic reward design for agentic graph construction. Stage II couples the policy to the frozen GPM via the Pattern-Augmented Reward (PAR), promoting pattern coverage from a soft prompt hint to a process-level gradient signal.

  • Procedural-knowledge accumulation as a new scaling axis. A 7B PatternBloom outperforms 27 baselines by +15.0 Avg-OOD EM; the gain compounds with reasoning depth where content-only retrieval saturates, and the same frozen pattern memory transfers across backbone families (1.5B–32B), establishing pattern retrieval as a scaling axis distinct from larger backbones, deeper retrieval, or richer prompts.

Related Work

Structured retrieval-augmented generation. Retrieval-augmented generation [1, 20, 21, 22, 23] has progressively moved toward structured evidence. Knowledge-graph methods build offline graphs and return subgraphs (GraphRAG [11], HippoRAG [12, 13], LightRAG [15], LinearRAG [14], GFM-RAG [16]); hypergraph variants extend to nn-ary facts (HyperGraphRAG [24], Graph-R1 [6]); adaptive-structure methods select per-query formats (StructRAG [17], Structure-R1 [18]). Across this line the structure is fixed at retrieval time, and the agent receives content fragments rather than a reasoning path.

Reinforcement learning for agentic RAG. A complementary line trains agents end-to-end with policy gradients [25, 26, 8, 9]: Search-R1 [5], R1-Searcher [7], FrugalRAG [10], Graph-R1, Structure-R1, and ReSearch [27] share answer-level outcome rewards; prompted variants couple iterative retrieval with chain-of-thought [28, 29, 30, 31, 32].

Two constraints bottleneck both lines: (i) supervision is surface-form, scoring the answer string but never the evidence graph as an information object; (ii) the retrieval unit is content tied to specific entities, not the procedural skeleton that recurs across queries of the same logical type. PatternBloom addresses both via an information-theoretic graph reward and an externalized Graph Pattern Memory of type-abstracted skeletons; a method-by-method comparison appears in Appendix § 11.5.

PatternBloom: Logic-Enhanced RL with Externalized Graph Pattern Memory

Section 2 identified two constraints that bottleneck prior agentic RAG: surface-form supervision that never interrogates the constructed evidence graph, and a content-only retrieval target that leaves procedural reasoning latent in policy weights. PatternBloom is built around one principle that addresses both: supervise the agent on what evidence it gathers and how it traverses that evidence, with each component an externalized, learnable object rather than a behavior to be hoped for. Stage I (§3.2) targets the first constraint via the Information-Density Reward (IDR), which scores the evidence graph’s information content under a frozen oracle and so promotes the graph itself to a directly supervised quantity. Trajectories from the Stage-I best checkpoint are distilled (§3.3) into a Graph Pattern Memory (GPM) of type-abstracted reasoning skeletons; Stage II (§3.4) couples the frozen GPM to the policy via the Pattern-Augmented Reward (PAR), turning procedural reasoning into a retrievable, learnable structural prior and addressing the second constraint. Figure 2 shows the pipeline; Algorithm 1 (Appendix A) gives the formal procedure.

Figure 2. PatternBloom pipeline. Top: training with intermediate pattern distillation between Stage I (IDR) and Stage II (PAR). Bottom: deployment uses only trained policy and frozen pattern memory.

Preliminaries

We consider multi-hop QA over a corpus C\mathcal{C} of Wikipedia chunks, with dataset D={(qi,ai∗)}\mathcal{D} = \{(q_i, a_i^*)\} where each query qq requires reasoning over two or more retrieved facts. We pre-build a hypergraph index H=(V,Ehyper,Xtext)H = (V, E_{\mathrm{hyper}}, X_{\mathrm{text}}) over C\mathcal{C} following Gutiérrez et al. [12], Luo et al. [6]: entity nodes VV are connected by hyperedges EhyperE_{\mathrm{hyper}} that group facts from each chunk, and chunk text XtextX_{\mathrm{text}} is preserved for retrieval (Appendix § 11.2.§ 2). nn-ary hyperedges preserve the intra-chunk co-occurrence that binary triples discard, exposing the typed neighborhoods that pattern abstraction (§ 3.3) will exploit. The agent policy πθ\pi_{\theta} is a tool-use language model that interleaves ⟨think⟩\langle\text{think}\rangle, ⟨query⟩\langle\text{query}\rangle, ⟨graph⟩\langle\text{graph}\rangle, ⟨answer⟩\langle\text{answer}\rangle blocks, accumulating an evidence graph G⊆H\mathcal{G} \subseteq H over multiple turns. The oracle O\mathcal{O} (Qwen2.5-1.5B-Instruct [33]) is held frozen and used only at training time to score evidence-graph quality via log⁡PO(⋅∣q,⋅)\log P_{\mathcal{O}}(\cdot\mid q,\cdot): a frozen scorer keeps the reward landscape stationary across both stages and rules out reward-gaming that an online critic invites. The pattern memory M\mathcal{M} is defined precisely in § 3.3.

Per-stage GRPO objective. PatternBloom’s two rewards target complementary failure modes: RIDRR_{\mathrm{IDR}} (Stage I) addresses what the policy fails to gather, and RPARR_{\mathrm{PAR}} (Stage II) addresses how the policy fails to structure its reasoning. We instantiate both stages with GRPO [9]—group-relative advantages remove the value-function critic, which becomes high-variance under the sparse trajectory-level rewards typical of long tool-use rollouts—through a single policy-gradient template with RR supplying the per-stage signal:

LPatternBloom(θ;R):=Eq,{τi}[1G∑imin⁡(ri(θ)A^i,clip⁡(ri(θ),1−ϵ,1+ϵ)A^i)]−βDKL[πθ∥πref],(1)\mathcal{L}_{\mathrm{PatternBloom}}(\theta; R) := \mathbb{E}_{q,\{\tau_i\}}\left[\frac{1}{G}\sum_i \min\left(r_i(\theta)\hat{A}_i,\operatorname{clip}\left(r_i(\theta),1-\epsilon,1+\epsilon\right)\hat{A}_i\right)\right] -\beta D_{\mathrm{KL}}\left[\pi_{\theta}\mathbin{\|}\pi_{\mathrm{ref}}\right], \tag*{(1)}

where A^i=(R(τi)−mean⁡GR)/(std⁡GR+δ)\hat{A}_i = (R(\tau_i)-\operatorname{mean}_{G}R)/(\operatorname{std}_{G}R+\delta) is the within-group standardized advantage, ri(θ)r_i(\theta) is the per-rollout importance ratio, and πref\pi_{\mathrm{ref}} is reset between stages (the SFT backbone for Stage I, the Stage-I best policy for Stage II) so each stage is anchored to the previous stage’s gains. Both rewards are bounded in [−1,1][-1,1] via a binary format gate. Stage I sets R=RIDRR=R_{\mathrm{IDR}} (Eq. 3); Stage II sets R=RPARR=R_{\mathrm{PAR}} (Eq. 6); hyperparameters are listed in Appendix § 11.2.§ 14.4.

Stage I: Evidence Construction with the Information-Density Reward

Existing RL-RAG agents supervise training with answer-level signals on the predicted string [5, 7] or entity-presence checks on the constructed graph [6]. None interrogate the graph’s information content as a whole: a 20-triple noisy graph and a 3-triple clean one receive the same reward provided both yield a matching answer. This blindness leaves πθ\pi_{\theta} with no signal to prefer minimum-sufficient evidence over verbose-but-correct rollouts, and the gap widens with reasoning depth (Appendix § 11.2.§ 12.7). IDR closes this gap by promoting the constructed graph itself to the supervised quantity.

Let ser⁡(G)\operatorname{ser}(\mathcal{G}) serialize the evidence graph as a sequence of typed triples. The information gain of G\mathcal{G} relative to the empty graph is

Δlog⁡p(G):=log⁡PO(a∗∣q,ser⁡(G))−log⁡PO(a∗∣q,∅),(2)\Delta\log p(\mathcal{G}) := \log P_{\mathcal{O}}(a^{*}\mid q,\operatorname{ser}(\mathcal{G}))-\log P_{\mathcal{O}}(a^{*}\mid q,\varnothing), \tag*{(2)}

which equals the conditional pointwise mutual information PMI(G\mathcal{G}; a∗a^* | qq) between graph and answer given the query (cf. Eq. (9)). The Information-Density Reward wraps the per-triple-normalized gain in a sigmoid, gates by the format reward RfmtR_{\mathrm{fmt}} so malformed trajectories collapse to −1-1, and rewards trajectories that achieve high information density:

RIDR(τ)=−1+Rfmt(τ)+1{Rfmt=1}⋅σ(Δlog⁡p(G)max⁡(∣G∣,1)⋅T),(3)R_{\mathrm{IDR}}(\tau) = -1 + R_{\mathrm{fmt}}(\tau) + \mathbb{1}\{R_{\mathrm{fmt}} = 1\} \cdot\sigma\left(\frac{\Delta\log p(\mathcal{G})}{\max(|\mathcal{G}|, 1) \cdot T}\right), \tag*{(3)}

with temperature T=1T = 1. Three design choices in Eq. (3) are deliberate. (i) Size normalization. Dividing by ∣G∣|\mathcal{G}| favors short anchor-to-answer paths: low-information triples grow ∣G∣|\mathcal{G}| faster than Δlog⁡p\Delta\log p, so verbose-but-correct rollouts strictly lose to minimum-sufficient ones, closing a loophole that entity-presence rewards leave open [6]. (ii) Sigmoid wrapping. The sigmoid bounds the reward in [0,1][0, 1] so within-group GRPO advantages remain well-scaled across queries whose absolute Δlog⁡p\Delta\log p scale varies by orders of magnitude. (iii) Format gating. The RfmtR_{\mathrm{fmt}} multiplier collapses malformed trajectories to −1-1, so any density improvement is conditional on a well-formed graph and the policy cannot trade structural validity for content. Theorem 1 (Appendix § 11) shows that maximizing RIDRR_{\mathrm{IDR}} via GRPO is consistent with maximizing the size-normalized conditional mutual information I(G;a∗∣q)/(∣G∣⋅T)I(\mathcal{G}; a^* \mid q)/(|\mathcal{G}| \cdot T), with the divisive normalizer realizing an adaptive Information-Bottleneck multiplier [34, 35] βeff=1/(∣G∣⋅T)\beta_{\mathrm{eff}} = 1/(|\mathcal{G}| \cdot T). IDR thereby provides the first information-theoretic reward design for agentic graph construction, replacing the surface-form heuristics that bottleneck prior work with a principled signal that scores the graph as a whole.

Pattern Distillation: Building the Externalized Procedural Memory

A retrieved triple like (BLACK GOLD, directed_by, LESLIE MARTINSON) is query-specific and never recurs, but many queries share the same logical structure: “given a film, find the director’s nationality” applies across thousands of film–director–country instances. The procedural component of multi-hop reasoning—how types of facts compose—recurs while the instances change, yet prior systems leave this regularity implicit in policy weights (§ 2). PatternBloom lifts each instance triple to a type-abstracted skeleton (FILM, directed_by, PERSON), (PERSON, born_in, COUNTRY) that is reusable across all instances of the same query type, and stores these skeletons in an external memory auto-distilled from the policy’s own high-reward Stage-I trajectories. Three properties follow from this construction: type abstraction makes each entry an equivalence class over instances and so amortizes the cost of one query’s reasoning across thousands of structurally similar ones; externalization makes the memory inspectable, freezable, and reusable across backbones; and self-distillation keeps the memory aligned with what this policy actually solves, rather than a hand-curated wishlist.

Graph pattern. A graph pattern PP is a tuple (σ,skel⁡,desc⁡,n)(\sigma, \operatorname{skel}, \operatorname{desc}, n) where σ∈S\sigma\in\mathcal{S} is a query signature, skel is a connected directed graph of typed entity nodes (the reasoning skeleton), desc is a short natural-language descriptor, and nn is a usage count. The Graph Pattern Memory M\mathcal{M} is the learned external store

M={(σi,skel⁡i,desc⁡i,ni)}i=1∣M∣,(4)\mathcal{M} = \left\{(\sigma_i, \operatorname{skel}_i, \operatorname{desc}_i, n_i)\right\}_{i=1}^{|\mathcal{M}|}, \tag*{(4)}

populated by distilling Stage-I trajectories. Signatures are extracted by a deterministic parser (e.g., lookup(country) [FILM]); entity types in the skeleton come from the type-abstraction step, and descriptors are auto-generated from the signature and a representative answer instance.

Coverage and retrieval. At Stage II, patterns are retrieved by encoding σ(q)\sigma(q) with a sentence encoder and ranking M\mathcal{M} by cosine similarity in a FAISS [36] index. The trajectory–pattern coverage

C(τ,P)=∣abs⁡(τ)∩P.skel⁡∣∣P.skel⁡∣,abs⁡(τ):={(type⁡(h),r,type⁡(t)):(h,r,t)∈τ},(5)\mathcal{C}(\tau, P) = \frac{|\operatorname{abs}(\tau) \cap P.\operatorname{skel}|}{|P.\operatorname{skel}|}, \qquad\operatorname{abs}(\tau) := \left\{(\operatorname{type}(h), r, \operatorname{type}(t)) : (h, r, t) \in\tau\right\}, \tag*{(5)}

divides by the skeleton size rather than by the union, so a trajectory that contains the required reasoning steps plus exploratory extras receives full credit. We use the top-1 match Pq∗∈MP_q^* \in\mathcal{M} as the coverage signal in RPARR_{\mathrm{PAR}} and feed the top-3 retrieved patterns to the agent’s prompt as a ⟨knowledge⟩ hint, allowing the policy to compare alternative skeletons before committing to a trajectory.

Memory construction. The memory M0M_0 is built between Stage I and Stage II from high-IDR trajectories (RIDR≥0.85R_{\mathrm{IDR}} \ge0.85, the empirical top-quartile threshold; Appendix § 9); two patterns merge when their signature embeddings have cosine similarity ≥θmerge=0.85\ge\theta_{\mathrm{merge}} = 0.85. We freeze M0M_0 throughout Stage II and at inference: a non-stationary memory chasing a non-stationary policy invites collapse, while a frozen memory yields a stationary reward signal during refinement and a deterministic retrieval store at deployment. The same frozen 27-skeleton memory (Table 16) then transfers across distribution shift, a test query walks a skeleton distilled purely from training (Figure 4), evidencing that procedural knowledge, once externalized, generalizes beyond its training distribution.

Case study comparing Graph-R1, R1-Searcher, and PatternBloom on a 4-hop film-director query

Figure 4. Case study on a 4-hop query. Structure-based retrieval fabricates both directors’ names, and chunk-based retrieval confuses the film Antonieta with the writer Antonieta Rivas Mercado, while PatternBloom’s typed skeleton fixes what to look up for each film.

QuantityValue
Pool size and uniqueness
# patterns27
# unique signatures27
# unique relations369
Avg. pattern length (# triples / pattern)2.48
Distributional entropy
HtypeH_{\mathrm{type}} (bits)0.19
HsigH_{\mathrm{sig}} (bits)4.75
HrelH_{\mathrm{rel}} (bits)7.80
Structure and emission
Inverse-relation edge rate0.00
Graph-emit rate at inference (avg. over 7 ds)99.6 %
Avg. triples per query at inference2.62
Top-3 dominant types (by node-occurrence count)
ENTITY (2350), YEAR (36), BOOLEAN (10)
Top-5 dominant relations (across patterns)
nationality (44), director (43), directedby (31), country_of_origin (25), occupation (24)

Table 16. Descriptive statistics of the learned Graph Pattern Memory built on HotpotQA-7K + 2WikiMHQA-7K. HtypeH_{\mathrm{type}}, HsigH_{\mathrm{sig}}, HrelH_{\mathrm{rel}}: Shannon entropies (bits) over type, signature, and relation distributions of stored patterns.

Stage II: Policy Refinement with the Pattern-Augmented Reward

Feeding retrieved patterns to the policy’s prompt provides only a soft prior, with no gradient signal pushing the trajectory to align with the retrieved skeleton: the policy is free to ignore the hint and revert to whatever idiosyncratic chain it has already learned, especially when answer-level reward is uninformative. PatternBloom promotes pattern coverage from a soft prior to a process-level reward by combining answer-level F1F_1 with the trajectory–pattern coverage of Eq. (5):

RPAR(τ)=−1+Rfmt(τ)+1{Rfmt=1}⋅[(1−λP)F1(a^,a∗)+λPC(τ,Pq∗)].(6)R_{\mathrm{PAR}}(\tau) = -1 + R_{\mathrm{fmt}}(\tau) + \mathbb{1}\{R_{\mathrm{fmt}} = 1\} \cdot\left[(1-\lambda_P)F_1(\hat{a},a^*) + \lambda_P C(\tau,P_q^*)\right]. \tag*{(6)}

The convex combination ensures pattern coverage acts as a structural prior that shapes how the policy reasons without overriding the answer signal that defines task success. The shaping weight λP∈[0,0.3]\lambda_P \in[0, 0.3] keeps answer correctness dominant, in line with the bounded-potential shaping principle of Ng et al. [37], which guarantees policy invariance under shaping rewards small relative to the task reward. We select λP=0.2\lambda_P = 0.2 as the default through a sweep over {0,0.1,0.2,0.5}\{0, 0.1, 0.2, 0.5\} on a held-out 1k-query slice (Appendix § 10); values above 0.3 destabilize the IDR-learned graph-construction behavior because pattern coverage begins to dominate the answer-level signal, and we therefore impose this empirical bound. Setting λP=0\lambda_P = 0 recovers a pure F1F_1 reward. PatternBloom warm-starts πθ\pi_\theta from πθ1∗\pi_{\theta_1^*} and continues maximizing LPatternBloom(θ;RPAR)\mathcal{L}_{\mathrm{PatternBloom}}(\theta; R_{\mathrm{PAR}}) (Eq. (1)); at deployment, πθ2∗\pi_{\theta_2^*} and the frozen M0M_0 operate over the same hypergraph HH (Appendix § 11.2.§ 3). Together, IDR and PAR realize the paradigm shift of § 1: both the constructed evidence graph and the retrieved reasoning skeleton become supervisable, externalized objects rather than behaviors latent in policy weights, and the gain compounds with reasoning depth precisely because procedural knowledge, once distilled, is reused rather than re-derived at every query.

Experiments

Experimental Setup

Datasets. Seven QA benchmarks split as in-distribution (HotpotQA [2], 2WikiMultiHopQA [3]; 14K FlashRAG training queries, seed 42) and out-of-distribution (MuSiQue [4], NQ [38], Trivi-aQA [39], PopQA [40], Bamboogle [41]). We sample 1,000 test instances per dataset (seed 40); Bamboogle uses its full 125 items. The 5-of-7 OOD majority is strictly harder than the IID-only protocol of Graph-R1 [6].

Baselines. 27 baselines span four families: training-free RAG on GPT-4o-mini (HippoRAG 2 [13], GraphRAG [11], LightRAG [15], StructRAG [17], LinearRAG [14], GFM-RAG [16], Hyper-GraphRAG [24]); RL-trained chunk retrieval (Search-R1 [5], R1-Searcher [7], FurgalRAG [10]); RL-trained graph retrieval (Graph-R1 [6], Structure-R1 [18]); plus prompt-engineered (72B) and large-scale (14B–≥\ge 30B) reference panels (Table 1). RL methods share identical training data and seeds; PatternBloom occupies a previously unfilled cell: a trained agent over a graph substrate with procedural memory.

Method2Wiki{IID}`HotpotQA{IID}`MuSiQue{OOD}`Bamboogle{OOD}`NQ{OOD}`TriviaQA{OOD}`PopQA{OOD}`Avg-IIDAvg-OOD
EMF1EMF1EMF1EMF1EMF1EMF1EMF1EMF1CAEMF1CA
VANILLA INFERENCE-ONLY BASELINES (W/ QWEN2.5-7B-INSTRUCT)
Direct (no retrieval)21.727.714.222.82.89.44.014.44.613.728.038.59.214.418.025.325.89.718.122.2
⊞ Naive RAG (E5 chunk top-5)29.739.524.334.93.710.512.020.210.922.743.855.328.037.427.037.244.019.729.241.6
PROMPT-ENGINEERED RAG
(a) Proprietary closed-source generator (w/ GPT-4o-mini)
▼ HippoRAG 218.622.024.033.927.538.239.248.926.037.150.359.114.317.221.327.927.331.540.139.1
▼ StructRAG9.813.529.139.611.218.539.249.236.351.366.074.340.749.719.526.624.138.748.647.5
▼ LinearRAG16.321.019.429.616.025.526.438.123.534.150.859.313.416.817.925.327.426.034.834.6
▼ GraphRAG8.810.019.327.111.518.037.648.126.938.751.958.916.919.414.118.617.629.036.633.9
▼ LightRAG5.26.210.513.620.728.040.045.715.721.924.227.65.06.87.89.912.021.126.024.4
▼ GFM-RAG15.317.914.322.125.937.130.441.721.133.556.265.815.119.314.820.020.629.739.540.3
□ HyperGraphRAG19.827.913.323.87.722.224.837.311.125.841.254.68.016.616.625.930.118.631.341.6
(b) Open-sourced XL-size generator (w/ Qwen2.5-72B-Instruct-AWQ)
▼ HippoRAG 210.013.118.423.924.634.239.248.323.533.741.548.612.214.314.218.519.428.235.835.2
▼ GraphRAG6.27.519.826.512.619.040.850.429.541.154.862.217.319.813.017.016.231.038.536.1
⊙ StructRAG6.68.124.332.46.712.645.656.526.340.258.967.722.025.615.420.220.231.940.538.7
TRAINED AGENTIC RAG
(c) <10B parameters
R1 (7B)29.534.921.629.45.514.232.038.315.925.343.650.714.517.825.632.139.622.329.330.5
⊞ Search-R1 (7B)34.540.233.544.911.519.240.851.532.346.770.679.135.440.434.042.639.738.147.444.8
⊞ R1-Searcher (7B)35.341.333.744.615.824.344.0…
⊞ FrugalRAG (7B)15.821.515.924.73.48.917.626.015.720.141.149.616.225.615.923.123.318.826.025.1
□ Graph-R1 (7B)27.332.520.827.64.612.123.235.317.527.543.049.915.819.524.130.139.020.828.929.3
⊙ Structure-R1 (7B)69.275.938.048.99.917.524.832.132.443.661.769.439.546.553.662.459.033.741.939.6
R1 (8B)31.937.322.431.25.011.539.251.818.227.348.556.016.922.027.134.230.925.633.731.8
⊞ Search-R1 (8B)42.948.134.545.312.722.440.855.523.532.950.959.332.537.738.746.743.632.141.639.6
⊞ FrugalRAG (8B)15.120.317.525.24.49.723.233.117.220.641.448.817.826.616.322.824.120.828.725.8
□ Graph-R1 (8B)28.333.921.230.75.914.731.244.917.526.348.055.116.921.125.032.346.023.932.736.7
⊞ PatternBloom (7B)54.058.641.655.638.848.052.064.239.753.470.082.540.349.647.857.152.549.058.555.3
⊙ PatternBloom (8B)47.552.837.751.025.335.248.063.733.849.171.480.538.445.442.651.948.343.454.851.7
(d) 10–30B parameters
⊙ Qwen2.5-it RAG (14B)‡17.829.135.648.19.818.222.435.4------26.738.647.4---
⊞ Search-R1 (14B)47.046.824.152.824.152.8--48.667.6--48.046.948.2
⊙ PatternBloom (14B)56.061.143.658.140.349.353.563.241.655.375.083.541.048.449.859.654.354.060.056.7
(e) ≥30B parameters
⊞ Search-R1 (32B)‡58.071.445.257.316.628.256.067.834.049.763.474.1--51.664.3-42.555.0-
⊞ ReSearch (32B)‡44.9-46.7-26.4-56.8-------45.8--41.6--
⊙ Qwen2.5-it-RAG (72B)†34.646.941.655.111.220.946.460.7------38.151.053.9---
⊞ Llama-4-Maverick (17B × 128, MoE)‡50.661.249.463.816.026.248.866.3------50.062.563.3---
⊙ PatternBloom (32B)58.563.145.659.641.851.556.065.244.556.877.584.544.049.952.161.456.552.861.658.2

Table 1. Main results on seven QA benchmarks, organized as four Multi-Hop datasets (HotpotQA, 2WikiMHQA, MuSiQue, Bamboogle) and three Single-Hop factoid datasets (NQ, TriviaQA, PopQA). IID marks the two in-distribution training datasets; OOD marks the five held-out datasets. Retrieval representation: ⊞ chunk, ▼ knowledge graph, □ hypergraph, ⊙ adaptive structure, ⋄ procedural memory. ‡ marks reported data; remaining rows are our reproductions.

Metrics, backbones, compute. We report answer correctness (EM, F1, Contain-Acc), retrieval quality (R@5, Evidence Recall), pattern faithfulness (RPF, P-Cov; §4.4), efficiency, and human + LLM-judge scores. Backbones: Qwen2.5-7B-Instruct [33] (primary), Qwen3-8B-Instruct [42] (cross-backbone), four Qwen2.5 sizes (1.5B/7B/14B/32B) for scaling. Full settings and metric definitions: Appendix § 11.2.§ 14.4, 6.

“–” = unreported; † = AceSearcher [43] reproduction.

Main Results

PatternBloom (Q2.5-7B) is rank-1 on EM and F1 across all seven datasets in Table 1, leading R1-Searcher by 15.0 Avg-OOD EM and the 72B prompt-engineered RAG panel by 17.1 Avg-OOD EM at one-tenth the parameter count. The same frozen GPM transfers to a Qwen3-8B backbone, where PatternBloom leads every 8B entry on Avg-OOD by 11.3 EM, indicating the gain originates in the procedural-memory mechanism rather than backbone-specific tuning. The IID-to-OOD gap stays under 1.5 EM on either backbone (against 4.1 and 3.3 EM for Search-R1 and Graph-R1), and PatternBloom is rank-1 on every Avg-OOD metric (EM, F1, Contain-Acc, R@5, Evidence Recall), so the gain transfers across the five out-of-distribution test sets without overfitting to the IID training corpus.

Component Analysis

The three components are jointly necessary. Removing PAR drops F1 from 55.9 to 44.1, a larger loss than disabling GPM (46.6) or skipping IDR (52.9): PAR is the only mechanism supplying a process-level gradient signal, GPM the structural prior it exploits, and IDR’s standalone effect is partially recovered by Stage II. The pattern-coverage weight is concave around λP=0.2\lambda_P = 0.2 (55.9 F1; λP∈{0,0.1,0.5}\lambda_P \in\{0, 0.1, 0.5\} each lose 10–13 F1), confirming the bound λP≤0.3\lambda_P \le0.3 derived in §3.4 from Ng et al. [37]. Memory access is all-or-nothing: top-1/5/10 retrieval cluster around 43 F1 while the full

(a) −Stage I: skip IDR pre-training; −Stage II: λp=0\lambda_p = 0 (no PAR coverage reward); −GPM: disabled in both training (no reward, no prompt hint) and inference.

∣M∣=27|\mathcal{M}| = 27 pool jumps to 55.9 F1, so the gain comes from the pool’s joint coverage rather than any single dominant skeleton.

Reasoning Quality

We complement answer-level scores with two pattern-quality signals introduced in this work: RPF (Reasoning Path Faithfulness, fraction of trajectory turns invoking ≥1\ge1 stored pattern) and P-Cov (Pattern Coverage, fraction of trajectories whose retrieved pattern matches the oracle-best signature). Both drop monotonically as PatternBloom components are removed and reach zero with GPM disabled (by construction); P-Cov correlates with answer EM (ρ=0.87\rho= 0.87 across the four PatternBloom variants), evidence that pattern coverage is a faithful surrogate for reasoning quality rather than a separate dimension.

Multi-Size Scaling and Cross-Backbone Transfer

PatternBloom retrains at four Qwen2.5 sizes (1.5B/7B/14B/32B) and one Qwen3-8B variant (per-size scores in Table 1 panels (b)–(e); discussion in Appendix § 11.2.5) and ranks first on the average column at every size band. The 1.5B variant attains 28.4 Avg. EM, competitive with 7B chunk-RL baselines (Search-R1 7B at 21.6) at one-quarter the parameter count, while PatternBloom (Q3-8B) leads all

8B entries on Avg-OOD (43.4 EM), confirming cross-backbone transfer. Inference latency stays within 0.4 s of Graph-R1 (Appendix § 11.2.4). Figure 3 consolidates the Pareto: PatternBloom sits on the cost-quality frontier (a) and attains the flattest depth-decay (α=−1.10\alpha= -1.10 vs. Search-R1’s −2.91-2.91, b), substantiating the architectural claim of § 1: procedural memory is an additive architectural component, not a backbone-specific tuning trick.

(a) Cost-quality analysis.

(b) Reasoning-depth scaling.

(c) Human evaluation.

Figure 3. Performance and reasoning quality across three views: (a) cost-quality trade-off, (b) reasoning-depth scaling on MuSiQue, (c) human evaluation.

Behavior, Failures, and Human Study

Question-type behavior. Against Graph-R1 (the only directly comparable RL-trained graph-RAG agent), PatternBloom uses 0.4–0.9 fewer turns per query in every category (Table 4): typed pattern templates replace exploratory iterations with one-shot lookups. Per-category EM/F1 and hop-stratified MuSiQue: Appendix § 11.2.§ 13, G.10.

MethodBridgeComp.1-hopMulti-3+
□ Graph-R15.45.34.95.8
□ ⋄ PatternBloom4.74.94.24.9

Table 4. Average #Turns per question category. Full EM/F1 breakdown in Appendix § 11.2.§ 13.

Residual error categories. Errors group into four categories (Appendix § 11.2.§ 14.3): three generic to graph-RAG (entity disambiguation, numerical reasoning, predicate canonicalisation), and one specific to procedural memory (signature mismatch), addressed by an inference-time pattern-admission extension (§I).

Human study and case illustrations. Figure 3c reports a blind human study on 30 random queries: PatternBloom leads on every 1–5 Likert criterion, with the largest gain +0.85 on Pattern Reusability over Structure-R1 and the highest inter-annotator agreement (α=0.74\alpha= 0.74). Figure 4 contrasts the three paradigms qualitatively: structure-based retrieval fabricates names, chunk-based retrieval substitutes the query’s subject, and PatternBloom’s typed-skeleton retrieval anchors the trajectory to retrieved structure. Trajectories and additional cases: Appendix § 11.2.§ 12.5.

Conclusion

Multi-hop reasoning systems have so far retrieved content units (paragraphs, knowledge-graph fragments, or hyperedges), yet the primitive that compounds with reasoning depth is logic: the type-abstracted reasoning skeleton applicable across queries that share compositional structure. We realize this paradigm shift in PatternBloom, an RL framework whose Information-Density Reward provides the first information-theoretic grounding for graph-construction RL (a bounded monotone surrogate for size-normalized conditional mutual information), and whose Pattern-Augmented Reward turns a frozen Graph Pattern Memory into a learnable structural prior. The gain compounds with reasoning depth, a property absent from content-based retrieval, precisely because procedural knowledge, once externalized, accumulates across queries rather than dissolving into policy weights at each episode end; the same frozen pattern store further transfers across out-of-distribution benchmarks and backbone scales, evidencing that what is being learned is reasoning structure, not benchmark-specific content. PatternBloom thereby reframes retrieval-augmented reasoning from a content-fetching problem into one of procedural-knowledge accumulation, opening a scaling axis distinct from larger backbones, deeper retrieval, or richer prompts.

Acknowledgments and Disclosure of Funding

This work was partially supported by NSF IIS 2347592, 2348169, DBI 2405416, CCF 2348306, CNS 2347617, RISE 2536663. The authors declare no competing interests.

Appendix

PatternBloom Training Procedure

Algorithm 1 formalizes the training procedure of PatternBloom described in § 3: Stage I optimizes evidence-graph construction with the Information-Density Reward RIDRR_{\mathrm{IDR}}, an offline distillation step lifts high-quality Stage-I trajectories into the frozen Graph Pattern Memory M0\mathcal{M}_0, and Stage II refines the policy under the Pattern-Augmented Reward RPARR_{\mathrm{PAR}} on top of that memory. The shared GRPO objective LPatternBloom(θ;R)\mathcal{L}_{\mathrm{PatternBloom}}(\theta; R) is defined in Eq. (1) with R∈{RIDR,RPAR}R \in\{R_{\mathrm{IDR}}, R_{\mathrm{PAR}}\} specifying the per-stage learning signal.

At inference time the deployed agent invokes only the trained policy πθ2\pi_{\theta_2} and the retriever; the oracle is discarded after Stage I training, and the pattern memory M0\mathcal{M}_0 serves as a read-only retrieval store. The detailed inference walkthrough appears at the end of § 3.4.

IDR Implementation Details

Format-gated reward composition. The format reward Rfmt(τ)∈{0,1}R_{\mathrm{fmt}}(\tau) \in\{0,1\} is binary: a trajectory τ\tau receives Rfmt=1R_{\mathrm{fmt}}=1 iff each turn parses as the expected interleaving of ⟨think⟩\langle\text{think}\rangle, ⟨query⟩\langle\text{query}\rangle, ⟨graph⟩\langle\text{graph}\rangle, and (terminal turn only) ⟨answer⟩\langle\text{answer}\rangle blocks; otherwise Rfmt=0R_{\mathrm{fmt}}=0. Combined with the sigmoid kernel in Eq. (3), malformed trajectories yield RIDR=−1R_{\mathrm{IDR}}=-1, well-formed-but-uninformative trajectories yield −1+1+σ(Δlog⁡p/(∣G∣T))∈[0,1]-1+1+\sigma(\Delta\log p/(\lvert\mathcal{G}\rvert T)) \in[0,1], and the upper bound RIDR=1R_{\mathrm{IDR}}=1 is asymptotically reached as Δlog⁡p/(∣G∣T)→∞\Delta\log p/(\lvert\mathcal{G}\rvert T)\to\infty. Hence RIDR∈[−1,1]R_{\mathrm{IDR}}\in[-1,1].

Numerical safeguards. The sigmoid argument Δlog⁡p/(∣G∣T)\Delta\log p/(\lvert\mathcal{G}\rvert T) is clipped to [−50,50][-50,50] before evaluation: outside this range, σ(⋅)\sigma(\cdot) is numerically indistinguishable from {0,1}\{0,1\}, and clipping Algorithm 1 Two-stage training of PatternBloom with intermediate pattern distillation.

  1. Input: dataset D={(qi,ai∗)}\mathcal{D} = \{(q_i, a_i^*)\}; backbone policy πθ0\pi_{\theta_0}; frozen oracle O\mathcal{O}. Stage I: Evidence Construction

  2. Initialise πθ←πθ0,πref←πθ0\pi_\theta\leftarrow\pi_{\theta_0}, \pi_{\mathrm{ref}} \leftarrow\pi_{\theta_0}. ▷ anchor exploration to the SFT prior

  3. for t=1,…,T1t = 1, \ldots, T_1 do

  4. Sample trajectories {τi}\{\tau_i\} from πθ\pi_\theta;

  5. Compute RIDR(τi)R_{\mathrm{IDR}}(\tau_i) via O\mathcal{O} (Eq. (3));

  6. Update πθ\pi_\theta via GRPO (Eq. (1)).

  7. end for

  8. Select θ1∗←arg⁡max⁡t(EMt+F1,t)/2\theta_1^* \leftarrow\arg\max_t(\mathrm{EM}_t + F_{1,t})/2 on a held-out validation slice. Pattern Distillation (no gradient updates)

  9. Roll out πθ1∗\pi_{\theta_1^*} on D\mathcal{D}; retain trajectories with RIDR≥0.85R_{\mathrm{IDR}} \ge0.85 (top-quartile threshold).

  10. Type-abstract their triples; merge by signature similarity to obtain frozen M0\mathcal{M}_0 (Eq. (4)). Stage II: Pattern-Aligned Refinement

  11. Warm-start πθ←πθ1∗,πref←πθ1∗\pi_\theta\leftarrow\pi_{\theta_1^*}, \pi_{\mathrm{ref}} \leftarrow\pi_{\theta_1^*}. ▷ anchor exploration to Stage-I gains

  12. for t=1,…,T2t = 1, \ldots, T_2 do

  13. Sample {τi}\{\tau_i\} from πθ\pi_\theta; for each, retrieve top-1 pattern Pq∗∈M0P_q^* \in\mathcal{M}_0 by signature;

  14. Compute RPAR(τi)R_{\mathrm{PAR}}(\tau_i) via F1F_1 and coverage C(τi,Pq∗)C(\tau_i, P_q^*) (Eq. (6));

  15. Update πθ\pi_\theta via GRPO (Eq. (1)).

  16. end for

  17. Output: refined policy πθ2∗\pi_{\theta_2^*} and frozen pattern memory M0\mathcal{M}_0.

eliminates a class of NaN-producing edge cases under extreme oracle log-probability gaps. Empty graphs (∣G∣=0|\mathcal{G}| = 0) are treated as information-neutral via σ(0)=0.5\sigma(0) = 0.5; combined with Rfmt=1R_{\mathrm{fmt}} = 1, the trajectory’s reward is −1+1+0.5=0.5-1 + 1 + 0.5 = 0.5, neither rewarded nor penalized for absent evidence.

Demonstration-template anti-hack. We strip demonstration-template placeholder entities (e.g., ENTITYA, ENTITYNAME, NEXTENTITY) from each ⟨graph⟩\langle\mathrm{graph}\rangle block before computing ∣G∣|\mathcal{G}| and serializing for the oracle. Without this filter, the policy can copy placeholder triples shown in the system prompt’s in-context examples; such copies pass Rfmt=1R_{\mathrm{fmt}} = 1 yet carry no information about qq, opening a reward-hacking pathway. The filter is purely syntactic (case-folded match against a fixed placeholder vocabulary) and introduces no learnable parameters.

Caps and caching. Each evidence graph is capped at 10 graph blocks per trajectory and 20 triples per block, for a hard upper bound of ∣G∣≤200|\mathcal{G}| \le200 triples; this matches the agent’s interaction budget and bounds oracle context length to under the 4096-token serialization limit. The empty-graph baseline log PO(a∗∣q,∅)P_{\mathcal{O}}(a^* \mid q, \varnothing) depends only on (q,a∗)(q, a^*) and is cached per query during training.

Graph Pattern Memory Implementation Details

This appendix expands the implementation of the Graph Pattern Memory GPM introduced in §3.3.

Multi-Skeleton Pattern Storage

The main paper (§3.3) describes each pattern as carrying a single skeleton. The implementation generalizes this: each pattern stores a list of distinct skeletons (typically a small handful per signature in practice), and coverage is taken as the maximum over them.

The motivation is structural diversity: two trajectories that share a query signature can produce structurally different evidence graphs that both clear the high-quality threshold RIDR≥0.85R_{\mathrm{IDR}} \ge0.85. For instance, a query with signature compare (by [birth_year]) [FILM] can be answered either via a direct two-hop chain through both films’ release dates, or via a four-hop comparison that first resolves both directors and then their birth years. A single-skeleton schema would force one to overwrite the other, losing valid alternative reasoning paths for the same query type.

We therefore store path_patterns as a list of skeleton lists, each pattern carrying its accumulated set of distinct typed-triple sequences distilled from same-signature trajectories that merged at construction time. During offline merge (§C.5), an incoming skeleton not already present in the existing pattern’s skeleton list is appended; the pattern’s running averages of usage count nn, mean F1F_1, and mean RIDR\mathrm{RIDR} are updated incrementally. Coverage at PAR computation becomes

C(τ,P)=max⁡skel∈P.skel_list∣abs⁡(τ)∩skel∣∣skel∣,(7)C(\tau, P) = \max_{\mathrm{skel}\in P.\mathrm{skel\_list}} \frac{\lvert\operatorname{abs}(\tau) \cap\mathrm{skel}\rvert}{\lvert\mathrm{skel}\rvert}, \tag*{(7)}

i.e., the trajectory is rewarded for matching any of the pattern’s known skeletons.

Signature Dispatch

Signature dispatch is the function σ:Q→Rd\sigma: Q \to\mathbb{R}^{d} that turns a natural-language query qq into a vector key for cosine-similarity lookup against M\mathcal{M}. Conceptually it answers a single question: which of the ∣M∣\lvert\mathcal{M}\rvert stored skeletons is the right procedural prior for this query? The worked example below shows what σ\sigma produces on three real queries from our test sets.

Query (origin)Signature σ(q)\sigma(q)Pattern bucket in M0\mathcal{M}_0
2WikiMHQA (Table 10): which of two films has the later-born director?compare(by[birth_year]) [FILM]comparison-by-birth-year skeleton with FILM slot
MuSiQue (Table 11): what country is the director of a given film from?lookup_chain(person) [COUNTRY]chained FILM →\rightarrow DIRECTOR →\rightarrow COUNTRY
Bamboogle (Table 12): who commanded the mission of the first spacewalk?compare(by[date]) [ENTITY]comparison-by-date skeleton; reused at deployment from a HotpotQA training trajectory

Table 22. Worked example: q→σ(q)q \to\sigma(q) on three real test queries

Design rationale. The bare archetype alone (compare(by[...])) does not distinguish rows 1 and 3, which retrieve different skeletons; the typed suffix (FILM versus ENTITY) is what routes each query to a disjoint memory bucket.

Signature form. The signature is a two-part string. The first part is an archetype keyword extracted from the query by deterministic regex matching (e.g., lookup_chain(person) for chained-lookup queries, compare(by[date]) for date-comparison queries). The keyword captures the logical action of the question (lookup, compare, count, yesno) and its primary argument slot; we recognize fourteen such archetypes plus a sentinel unknown for queries that match none. The second part is a sorted entity-type suffix [TYPE_1+TYPE_2+...] inferred from quoted strings, year tokens, and capitalised phrases in the query under the five-class taxonomy of Appendix § 9.§ 3. The suffix is what lets two queries with the same archetype but different argument types (e.g., lookup_chain(person)[FILM] versus lookup_chain(person)[COUNTRY]) route to disjoint buckets.

Signature embedding. The full signature string is encoded into a unit-norm d=1024d=1024-dimensional vector by the sentence encoder BGE-large-en-v1.5 [44], the same encoder used for the hyperedge index of Appendix § 9.§ 2. Cosine similarity above θsim=0.5\theta_{\mathrm{sim}}=0.5 admits the nearest stored skeleton as the retrieved pattern; otherwise the cold-start fallback of Appendix § 10.§ 1 fires. Because the embedding is semantic rather than purely lexical, structurally related signatures sit close in Rd\mathbb{R}^{d}, which is what makes the third row of the worked example possible: a Bamboogle “earliest-event” query mapped to compare(by[date])[ENTITY] retrieves a skeleton distilled from a HotpotQA training trajectory under the same archetype (Table 12), even though the two queries’ surface forms differ.

Sample from Bamboogle
Question: Who was the commander for the space mission that had the first spacewalk?
Ground Truth: Pavel Belyayev
PatternBloom (Q2.5-7B)
Retrieved pattern from M0\mathcal{M}_0 via σ(q)=compare⁡(by⁡[date⁡])[ENTITY]\sigma(q) = \operatorname{compare}(\operatorname{by}[\operatorname{date}])[\mathrm{ENTITY}] (top-1 similarity 1.00, distilled from a HotpotQA training trajectory and re-used here on Bamboogle):
(MISSION1)→achieved(EVENT)→date(YEAR1)(\mathrm{MISSION}_1) \xrightarrow{\text{achieved}} (\mathrm{EVENT}) \xrightarrow{\text{date}} (\mathrm{YEAR}_1)
(MISSION2)→achieved(EVENT)→date(YEAR2)(\mathrm{MISSION}_2) \xrightarrow{\text{achieved}} (\mathrm{EVENT}) \xrightarrow{\text{date}} (\mathrm{YEAR}_2)
select i∗=arg⁡min⁡iYEARi⇒(MISSIONi∗)→commander(PERSON)i^* = \arg\min_i \mathrm{YEAR}_i \Rightarrow(\mathrm{MISSION}_{i^*}) \xrightarrow{\text{commander}} (\mathrm{PERSON})
Turn 1. <query>space mission with the first spacewalk</query> →\rightarrow retrieves several mission summaries, including Voskhod 2, Apollo 17, and Gemini 4.
Turn 2. <think>The first spacewalk was during the Voskhod 2 mission on 8 March 1965.</think> <query>Voskhod 2 mission</query> →\rightarrow launched with Belyayev and Leonov on board on 18 March 1965.
Turn 3. <query>commander of the Voskhod 2 mission</query> →\rightarrow Pavel Belyayev was the first commander of the cosmonaut corps.
Turn 4. <think>The knowledge clearly states that Pavel Belyayev was the commander of the Voskhod 2 mission.</think> <graph>Pavel Belyayev | commander_of | Voskhod 2</graph>
<answer>Pavel Belyayev</answer> (EM = 1.0)

Table 12. Cross-distribution pattern reuse. The compare(by[date])[ENTITY] pattern, distilled from HotpotQA-7K + 2WikiMHQA-7K training, drives a four-turn rollout to the correct answer on an out-of-distribution Bamboogle query.

Type Abstraction Taxonomy

Signature enrichment and triple typing during skeleton extraction use the deterministic five-type taxonomy (plus an EMPTY sentinel) summarized in Table 5.

TypeDetection ruleExample
EMPTYempty or whitespace-only string""
YEAR^\d{4}$ | 1989
DATE^\d{1,2}[/-]\d{1,2}[/-]\d{2,4}$ | 18-March-1965
NUMBER^-?\d+(\.\d+)?$, not matching YEAR/DATE | 42.7
BOOLEANcase-folded match in {true, false, yes, no}Yes
ENTITYall other non-empty stringsLeslie H. Martinson

Table 5. Type-abstraction rules. Detection is rule-based and parameter-free, applied in priority order so each value is assigned a single type.

The taxonomy captures the structural distinctions that matter for multi-hop reasoning (entity, year, numeric, yes/no answers) without invoking NER or external taxonomies. A fine-grained alternative that maps every entity to its closest Wikidata class via spaCy NER expands the taxonomy from 5 to roughly 30 types and raises the type entropy of M0\mathcal{M}_0 (deterministically over the same 27 stored patterns) from Htype=0.19H_{\mathrm{type}} = 0.19 bits to Htype=1.84H_{\mathrm{type}} = 1.84 bits, at the cost of an extra NER inference per query. The paper-main run uses the coarse taxonomy.

Pattern Descriptor

The descriptor field desci\mathrm{desc}_i in each pattern (Eq. (4)) is a short human-readable string that records what the pattern resolves to on a representative anchor trajectory. We use a deterministic template:

desci=⟨sig⟩→⟨answer_excerpt⟩\mathrm{desc}_i = \langle\mathrm{sig}\rangle\rightarrow\langle\mathrm{answer\_excerpt}\rangle

where ⟨sig⟩\langle\mathrm{sig}\rangle is the query signature (e.g., lookup(person), compare(by[birth_year])) and ⟨answer_excerpt⟩\langle\mathrm{answer\_excerpt}\rangle is the first 80 characters of the trajectory’s gold answer. For example, the pattern distilled from the trajectory “Who directed Black Gold (1962)?” (gold answer “Leslie H. Martinson”) has descriptor lookup(person) →\rightarrow Leslie H. Martinson. The template is intentionally simple: it makes pattern inspection and debugging trivial, and keeps the GPM construction step hyperparameter-free at the descriptor level. The coverage reward is computed on the typed skeleton (Eq. (5)), not on the descriptor string, so the descriptor’s role is purely to render an inspectable representative example for each stored pattern.

Merge Threshold Sensitivity

Patterns extracted from individual high-IDR trajectories are merged offline into archetypes by clustering signature embeddings with a cosine-similarity threshold θmerge\theta_{\mathrm{merge}}. Lower thresholds produce more, finer archetypes; higher thresholds produce fewer, coarser archetypes that match a broader range of queries. Empirically, the post-distillation archetype count follows a Zipf-like distribution: the top archetype dominates with several hundred instances while the tail decays sharply.

We use θmerge=0.85\theta_{\mathrm{merge}} = 0.85 throughout: this conservative threshold keeps the merged archetype count low (under-merging) but yields stable patterns whose embeddings are clustered tightly enough that the post-distillation 27 patterns each carry several hundred high-IDR trajectories on average.

Running-Average Update on Merge

When pattern PP absorbs an incoming candidate P′P' with cosine similarity ≥θmerge\ge\theta_{\mathrm{merge}}, the bookkeeping fields are updated by incremental running-mean

μnew=n⋅μold+xnewn+1,\mu_{\mathrm{new}} = \frac{n \cdot\mu_{\mathrm{old}} + x_{\mathrm{new}}}{n+1},

applied to count, F‾1\overline{F}_1, and R‾IDR\overline{R}_{\mathrm{IDR}}. This makes the GPM resistant to single-trajectory outliers (a noisy high-IDR trajectory cannot single-handedly inflate a pattern’s recorded quality) while preserving order-independence: the GPM’s final state is invariant to the order in which trajectories are processed, since the running mean is associative and commutative for fixed-population merges.

FAISS Configuration

We use faiss.IndexFlatIP (flat inner-product index) with d=1024d = 1024 (BGE-large-en-v1.5 embedding dimension), L2L_2-normalized vectors, and CPU-only execution. For the GPM sizes we observe (∣M∣≤105|\mathcal{M}| \le10^5 patterns, flat search is faster than IVF-PQ (<1< 1 ms top-1 retrieval)). Memory footprint is 4 MB per 1,000 patterns (1024 floats ×\times 4 bytes).

Pattern-Augmented Reward Implementation Details

This appendix expands the implementation of the Pattern-Augmented Reward PAR introduced in § 3.4.

Cold-Start Fallback Behavior

When the cosine similarity of σ(q)\sigma(q) to the nearest GPM pattern falls below θsim=0.5\theta_{\mathrm{sim}} = 0.5, we set C(τ,P∗)=0C(\tau, P^*) = 0. PAR then degenerates to

RPAR(τ)∣cold=−1+Rfmt+1{Rfmt=1}⋅(1−λP)⋅F1(a^,a∗),(8)\left.R_{\mathrm{PAR}}(\tau)\right|_{\mathrm{cold}} = -1 + R_{\mathrm{fmt}} + \mathbb{1}\{R_{\mathrm{fmt}} = 1\} \cdot(1 - \lambda_P) \cdot F_1(\hat{a}, a^*), \tag*{(8)}

i.e., 80% of the F1F_1-only signal at the default λP=0.2\lambda_P = 0.2, not the full F1F_1 signal. This is by design: cold-start trajectories receive a slightly attenuated reward, providing a soft incentive for the agent to pursue queries whose signatures are well-covered by existing patterns.

The attenuated form preserves λP\lambda_P as a literal trade-off coefficient between answer-level and structure-level signal: the cold-start trajectory contributes only the (1−λP)(1 - \lambda_P) fraction of the answer signal, with the remaining λP\lambda_P slot reserved for matched-pattern coverage.

Retrieval Threshold sim

We use θsim=0.5\theta_{\mathrm{sim}} = 0.5 as the cosine-similarity floor for pattern retrieval. The choice reflects a trade-off: lower thresholds admit semantically distant patterns and inject noise into the coverage signal, while higher thresholds collapse CC to 0 on legitimate matches.

Asymmetric Coverage Rationale

The asymmetry of Eq. (5) (§ 3.3) is quantitatively important: a symmetric Jaccard coverage Csym=∣∩∣/∣∪∣C_{\mathrm{sym}} = |\cap|/|\cup| would penalize valid longer-than-skeleton trajectories at exactly the rate of one extra triple per pattern triple: at ∣τ∣=2∣P∣|\tau| = 2|P| even with perfect skeleton coverage we would have Csym=0.5C_{\mathrm{sym}} = 0.5. Our asymmetric form gives full credit (C=1C = 1) for any superset of the skeleton, capturing the “skeleton is necessary, not maximal” semantics that matches the pattern-as-procedural-bias intent.

Theorem 1: Proof and Information-Bottleneck Discussion

Theorem 1 (IDR as a bounded monotone surrogate for size-normalized conditional MI). Let policy πθ\pi_\theta produce trajectory τ=(q,G,a^)\tau= (q, \mathcal{G}, \hat{a}), and let O\mathcal{O} be a frozen oracle. Define the conditional pointwise mutual information

PMI⁡(G;a∗∣q):=log⁡PO(a∗∣q,G)−log⁡PO(a∗∣q),(9)\operatorname{PMI}(\mathcal{G}; a^* \mid q) := \log P_{\mathcal{O}}(a^* \mid q, \mathcal{G}) - \log P_{\mathcal{O}}(a^* \mid q), \tag*{(9)}

which satisfies Eπθ[PMI⁡]=I(G;a∗∣q)\mathbb{E}_{\pi_\theta}[\operatorname{PMI}] = I(\mathcal{G}; a^* \mid q). Under regularity conditions (A1)–(A4) below, the IDR kernel

J(πθ):=E(q,a∗,G)∼πθ[σ(PMI⁡(G;a∗∣q)∣G∣⋅T)](10)J(\pi_\theta) := \mathbb{E}_{(q,a^*,\mathcal{G})\sim\pi_\theta}\left[\sigma\left(\frac{\operatorname{PMI}(\mathcal{G}; a^* \mid q)}{|\mathcal{G}| \cdot T}\right)\right] \tag*{(10)}

is a bounded monotone surrogate for the size-normalized conditional mutual information I~(πθ):=Eπθ[PMI⁡(G;a∗∣q)/(∣G∣⋅T)]\tilde{I}(\pi_\theta) := \mathbb{E}_{\pi_\theta}[\operatorname{PMI}(\mathcal{G}; a^* \mid q)/(|\mathcal{G}| \cdot T)] in the following three senses:

(i) Pointwise monotonicity. Since σ′>0\sigma' > 0 on R\mathbb{R}, raising PMI⁡/(∣G∣⋅T)\operatorname{PMI}/(|\mathcal{G}| \cdot T) for any trajectory raises that trajectory’s contribution to JJ, so JJ tracks I~\tilde{I} trajectory-wise.

(ii) Lipschitz gradient. Since σ′∈(0,1/4]\sigma' \in(0,1/4], the gradient contribution of any single trajectory to JJ is bounded, providing GRPO’s group-relative advantage estimator with a stable Lipschitz signal.

(iii) Saturation bound. On the operating regime PMI⁡/(∣G∣⋅T)∈R≥0\operatorname{PMI}/(|\mathcal{G}| \cdot T) \in\mathbb{R}_{\ge0} where trained policies concentrate (A4), σ\sigma is concave, so Jensen’s inequality yields J(πθ)≤σ(I~(πθ))J(\pi_\theta) \le\sigma(\tilde{I}(\pi_\theta)), equivalently I~(πθ)≥log⁡J(πθ)1−J(πθ)\tilde{I}(\pi_\theta) \ge\log\frac{J(\pi_\theta)}{1-J(\pi_\theta)}, characterizing the saturation regime in which JJ approaches its supremum.

:::`

This appendix provides the full proof, discusses the divisive-versus-subtractive Information-Bottleneck trade-off, and extends the result to a strict subgraph-monotonicity form. Theorem 1 grounds IDR as a tractable surrogate for size-normalized conditional mutual information: maximizing J(πθ)J(\pi_\theta) via GRPO is consistent with maximizing the underlying information content I~(πθ)\tilde{I}(\pi_\theta), and the divisive normalizer ∣G∣⋅T|G| \cdot T realizes an adaptive Information-Bottleneck multiplier βeff=1/(∣G∣⋅T)\beta_{\mathrm{eff}} = 1/(|G| \cdot T) without per-dataset tuning.

Regularity Conditions

We make these (mild) assumptions:

(A1) Oracle–policy decoupling. The frozen oracle OO depends on GG only through its serialization ser⁡(G)\operatorname{ser}(G), i.e., log⁡PO(a∗∣q,G)=log⁡PO(a∗∣q,ser⁡(G))\log P_O(a^* \mid q,G) = \log P_O(a^* \mid q,\operatorname{ser}(G)). Holds by construction.

(A2) Per-triple bounded gain. ∃K≥1\exists K \ge1 such that for any triple tt added to context G<tG_{<t}, I(t;a∗∣q,G<t)≤log⁡KI(t; a^* \mid q,G_{<t}) \le\log K nats. Each chunk carries finite information.

(A3) Format-trained equilibrium. After a brief warm-up Rfmt(τ)=1R_{\mathrm{fmt}}(\tau) = 1 holds on ≥99%\ge99\% of trajectories (empirically by step 5).

(A4) Oracle-informative regime. For trained policies, the trajectory distribution concentrates on PMI⁡(G;a∗∣q)≥0\operatorname{PMI}(G; a^* \mid q) \ge0 with high probability. Empirically validated by per-step training logs.

Proof of the Bounded Monotone Surrogate Property

Define X(G):=PMI⁡(G;a∗∣q)/(∣G∣T)X(G) := \operatorname{PMI}(G; a^* \mid q)/(|G|T). By (A3), conditional on Rfmt=1R_{\mathrm{fmt}} = 1 the IDR contribution is σ(X(G))\sigma(X(G)).

Step 1: Pointwise monotonicity. σ:R→(0,1)\sigma: \mathbb{R} \to(0,1) is strictly increasing with derivative σ′(x)=σ(x)(1−σ(x))∈(0,1/4]\sigma'(x) = \sigma(x)(1-\sigma(x)) \in(0,1/4]. For any pair G1,G2G_1,G_2 under the same (q,a∗)(q,a^*),

X(G1)>X(G2)⟺σ(X(G1))>σ(X(G2)).X(G_1) > X(G_2) \Longleftrightarrow\sigma(X(G_1)) > \sigma(X(G_2)).

Step 2: Jensen bound on the oracle-informative support. By (A4), the policy’s support concentrates on R≥0\mathbb{R}_{\ge0}, where σ\sigma is concave (σ′′(x)=σ′(x)(1−2σ(x))<0\sigma''(x) = \sigma'(x)(1-2\sigma(x)) < 0 for x>0x > 0). Jensen’s inequality yields

J(πθ)=Eπθ[σ(X(G))]≤σ(Eπθ[X(G)])=σ(I~(πθ)),J(\pi_\theta) = \mathbb{E}_{\pi_\theta}[\sigma(X(G))] \le\sigma(\mathbb{E}_{\pi_\theta}[X(G)]) = \sigma(\tilde{I}(\pi_\theta)),

where I~(πθ):=Eπθ[X(G)]\tilde{I}(\pi_\theta) := \mathbb{E}_{\pi_\theta}[X(G)] is the policy expectation of the size-normalized per-trajectory PMI rate; the IT identity Eπθ[PMI⁡(G;a∗∣q)]=I(G;a∗∣q)\mathbb{E}_{\pi_\theta}[\operatorname{PMI}(G; a^* \mid q)] = I(G; a^* \mid q) then justifies interpreting I~\tilde{I} as the size-normalized conditional-MI rate of πθ\pi_\theta’s induced distribution. Equality in the Jensen step holds iff X(G)X(G) is constant under πθ\pi_\theta. Inverting,

I~(πθ)≥σ−1(J(πθ))=log⁡J(πθ)1−J(πθ),\tilde{I}(\pi_\theta) \ge\sigma^{-1}(J(\pi_\theta)) = \log\frac{J(\pi_\theta)}{1-J(\pi_\theta)},

so high JJ certifies high size-normalized CMI.

Step 3: Bounded Lipschitz signal for GRPO. σ′(x)∈(0,1/4]\sigma'(x) \in(0,1/4] implies any single trajectory’s contribution to the group-relative advantage estimator under GRPO is 1/41/4-Lipschitz in XX, regardless of how extreme that trajectory’s raw PMI value is. This eliminates the gradient-explosion failure mode observed in early experiments with unbounded Δlog⁡p\Delta\log p rewards. □

Information-Bottleneck Connection: Divisive vs. Subtractive

The standard IB objective is subtractive: LIB(πθ)=I(G;a∗∣q)−βI(G;X)\mathcal{L}_{\mathrm{IB}}(\pi_\theta) = I(G; a^* \mid q) - \beta I(G; X), trading task accuracy against representation complexity. IDR replaces this with the divisive form I(G;a∗∣q)/(∣G∣T)I(G; a^* \mid q)/(|G|T), yielding three advantages.

(i) Adaptive βeff\beta_{\mathrm{eff}}. By (A2) and the chain rule of mutual information, I(G;X)≤∣G∣log⁡KI(G; X) \le|G| \log K, so

I(G;a∗∣q)∣G∣T≤log⁡KT⋅I(G;a∗∣q)I(G;X).\frac{I(G; a^* \mid q)}{|G|T} \le\frac{\log K}{T} \cdot\frac{I(G; a^* \mid q)}{I(G; X)}.

the divisive form is bounded above by a constant multiple of the IB ratio I(G;a∗∣q)/I(G;X)I(G; a^* \mid q)/I(G; X), with constant log⁡K/T\log K/T that depends only on the per-triple alphabet and the temperature, not on the dataset. Equivalently, the divisive form realizes an adaptive Lagrange multiplier βeff=1/(∣G∣T)\beta_{\mathrm{eff}} = 1/(|G|T) that scales with graph size: larger graphs face proportionally higher information-density bars without per-dataset hand-tuning of β\beta.

(ii) Bounded gradients. Subtractive IB rewards are unbounded; the σ\sigma-wrapped divisive form bounds reward to [−1,1][-1,1] and gradient contribution to 1/41/4-Lipschitz, simplifying GRPO training.

(iii) Relation to standard IB. IDR is motivated by IB rather than algebraically equivalent: divisive normalization saturates at large PMI, while subtractive IB does not. The divisive form is chosen for the gradient-stability and adaptive-β\beta properties above.

Strict Subgraph Monotonicity

Informative triples are strictly rewarded. For G2⊊G1G_2 \subsetneq G_1 with I(G1∖G2;a∗∣q,G2)>0I(G_1 \setminus G_2; a^* \mid q,G_2) > 0, the chain rule of mutual information gives PMI⁡(G1)>PMI⁡(G2)\operatorname{PMI}(G_1) > \operatorname{PMI}(G_2). Combined with strict σ\sigma monotonicity, RIDR(G1)>RIDR(G2)R_{\mathrm{IDR}}(G_1) > R_{\mathrm{IDR}}(G_2).

Uninformative triples are strictly penalized. For G2⊊G1G_2 \subsetneq G_1 with I(G1∖G2;a∗∣q,G2)=0I(G_1 \setminus G_2; a^* \mid q,G_2) = 0, PMI⁡(G1)=PMI⁡(G2)\operatorname{PMI}(G_1) = \operatorname{PMI}(G_2) but ∣G1∣>∣G2∣|G_1| > |G_2|, so X(G1)<X(G2)X(G_1) < X(G_2) and RIDR(G1)<RIDR(G2)R_{\mathrm{IDR}}(G_1) < R_{\mathrm{IDR}}(G_2). This formalizes the “minimum sufficient subgraph” inductive bias: the policy is strictly incentivized to include triples carrying information about a∗a^* and only those.

Asymmetric Coverage: Bounds and Random-Trajectory Baseline

This subsection records two mathematical facts about the asymmetric coverage function C(τ,P):=∣A(τ)∩σP∣/∣σP∣C(\tau,P) := |A(\tau) \cap\sigma_P|/|\sigma_P| that we use in the PAR reward (Eq. (5)). They are not used in the proof of Theorem 1 but justify two choices reflected in the experiments.

Proposition (range and tightness). For any trajectory τ\tau producing the abstract triple set A(τ)A(\tau) and any pattern PP with skeleton σP\sigma_P of size m:=∣σP∣m := |\sigma_P|,

C(τ,P)∈[0,1],C(τ,P)=1⟺σP⊆A(τ).(11)C(\tau,P) \in[0,1], \qquad C(\tau,P) = 1 \Longleftrightarrow\sigma_P \subseteq A(\tau). \tag*{(11)}

Furthermore, when ∣A(τ)∣=k|A(\tau)| = k and k≥mk \ge m, the coverage admits the lower bound

C(τ,P)≥k∩m,k∩:=∣A(τ)∩σP∣,(12)C(\tau,P) \ge\frac{k_{\cap}}{m}, \qquad k_{\cap} := |A(\tau) \cap\sigma_P|, \tag*{(12)}

with equality iff every triple in σP\sigma_P matches at most one triple in A(τ)A(\tau) under the chosen type abstraction. The asymmetry of CC (skeleton size in the denominator, not ∣A∣|A|) follows directly: adding non-matching triples to τ\tau does not penalize coverage, which is the property we use at training time to allow the policy to explore beyond the retrieved skeleton without paying a coverage cost.

Corollary (random-trajectory baseline). Let UU be the universe of distinct typed triples induced by the type abstraction and the relation vocabulary, and let τ\tau be a trajectory drawn uniformly at random over abstract triple sets of size kk. The expected coverage of any fixed skeleton σP\sigma_P of size mm is

Eτ[C(τ,P)]=k∣U∣,Pr⁡[C(τ,P)=1]=(∣U∣m)(km)−1,(13)\mathbb{E}_{\tau}[C(\tau,P)] = \frac{k}{|U|}, \qquad\Pr[C(\tau,P)=1] = \binom{|U|}{m}\binom{k}{m}^{-1}, \tag*{(13)}

which is exponentially small in mm for k≪∣U∣k \ll|U|. Concretely, with ∣U∣≈4⋅104|U| \approx4 \cdot10^4 in our paper-main configuration (entity-types squared ×\times relation count), k=3k = 3 (typical trajectory length), and m=2.5m = 2.5 (average skeleton length, Table 16), the random-trajectory baseline is E[C]≈7.5×10−5\mathbb{E}[C] \approx7.5 \times10^{-5} and the exact-match probability is below 10−810^{-8}. The empirical RPF in Table 3 (0.886) and the empirical GPM-CEM (0.099) thus exceed the random baseline by roughly 4 and 6 orders of magnitude respectively, ruling out the alternative explanation that PB trajectories happen to overlap with stored skeletons by chance.

MethodRPF ↑P-Cov ↑
PatternBloom0.8860.051
− Stage I0.2310.025
− Stage II0.4140.044

Table 3. Pattern quality.

Auxiliary Hypotheses and Sensitivity Sweeps

This appendix collects supporting empirical results for design choices documented elsewhere. The λP\lambda_P sweep that backs the choice of λP=0.2\lambda_P = 0.2 is reproduced in the main body Table 2(b); the θmerge\theta_{\mathrm{merge}} discussion is given in Appendix § 9.5; cross-dataset OOD transfer of M0\mathcal{M}_0 is reflected in the per-dataset Avg-OOD column of Table 1 (rank-1 on every held-out dataset).

Table 2: Ablation studies on 7B PatternBloom across (a) IDR, PAR, and GPM; (b) reward shaping weight λp\lambda_p in PAR; (c) Graph Pattern Memory pool size. Sch.: avg. # of retrieval calls per query.

VariantIDRPARGPMHotpotQA2WikiMuSiQueBam.Avg-F1Avg-EMSch.↓
(a) Module on/off
PatternBloom✓✓✓55.658.648.061.255.946.64.7
− Stage I✗✓✓54.954.441.360.952.9↓3.042.5↓4.14.2
− Stage II✓✗✓41.741.726.966.044.1↓11.834.9↓11.74.7
− GPM✓✓✗44.844.628.268.846.6↓9.337.0↓9.65.9↑1.2
(b) Reward shaping weight λp\lambda_p in PAR
λp=0.2\lambda_p = 0.2✓✓✓55.658.648.061.255.946.64.7
λp=0.1\lambda_p = 0.1✓✓✓43.945.526.764.745.2↓10.735.9↓10.74.9
λp=0.5\lambda_p = 0.5✓✓✓43.142.622.864.743.3↓12.634.2↓12.44.7
(c) Graph Pattern Memory pool size
Top-1 patterns✓✓✓45.143.618.564.743.0↓12.934.2↓12.45.0
Top-5 patterns✓✓✓44.143.121.666.343.8↓12.134.7↓11.95.1
Top-10 patterns✓✓✓43.441.920.567.143.2↓12.734.6↓12.05.1
Top-27 (full)✓✓✓55.658.648.061.255.946.64.7

Table 2. Ablation studies on 7B PatternBloom across (a) IDR, PAR, and GPM; (b) reward shaping weight λp\lambda_p in PAR; (c) Graph Pattern Memory pool size. Sch.: avg. # of retrieval calls per query.

Training Pipeline and Additional Experimental Results

This appendix groups material that supplements the experiment narrative in §4. The training-pipeline subsections (Hypergraph Index Construction, Hyperparameters, Stage Transition Protocol, Compute Budget) document how the index is built and how Stage I and Stage II are configured. The remaining subsections present additional experimental results that did not fit in the main body: cost-quality trade-off, multi-size scaling, pattern-memory statistics, retrieval quality analysis, multi-seed robustness, cross-domain generalization, LLM-judge quality ensemble, human study with the annotation form, and side-by-side case illustrations.

Metric Reading Guide

Table 6 consolidates the metrics referenced throughout §4, with definitions, direction (higher- or lower-is-better), and the table or figure where each metric is reported.

Metric (abbrev.)What it measuresBetterReported in
EM, F1, CAAnswer correctness (exact / token-overlap / lenient match)↑Tab. 1, 14
R@5, EvRTop-5 recall, evidence recall on gold supporting facts↑Tab. 15, 8
RPF, P-CovPattern-trajectory faithfulness (PB-novel)↑Tab. 3
Index TokOne-time corpus-indexing LLM tokens (M)↓Tab. 8
#TurnsAverage number of retrieval calls per query↓Tab. 4, 14, 13
6-axis LLM judge + κ\kappaSonnet/Haiku/Gemini ensemble Likert (1–5) + agreement↑Tab. 17
4-axis Human + α\alphaHuman Likert (1–5) + Krippendorff agreement↑Tab. 9
Htype/sig/relH_{\mathrm{type/sig/rel}}Shannon entropy over GPM type / signature / relation pools—Tab. 16

Table 6. Reading guide for the metrics used throughout §4. Higher is better unless marked ↓. PB-novel metrics are introduced in this work; all others follow established conventions.

Hypergraph Index Construction

The hypergraph index H=(V,Ehyper,Xtext)H = (V, E_{\mathrm{hyper}}, X_{\mathrm{text}}) that supports both Stage I and Stage II is built once, offline, before any RL training and shared across all stages and seeds. We adopt the construction protocol of Graph-R1 [6] and HippoRAG [12], which we summarize here for self-containment; configurations match the reproduction settings.

Step 1: Corpus chunking. The retrieval corpus C\mathcal{C} is the FlashRAG [45] Wiki-2018 dump (21 015 324 passages). Each passage is tokenized with the default GPT-2 tokenizer and split into fixed-length chunks of 100 tokens with no overlap, following the FlashRAG default and the Search-R1 baseline configuration. Chunk boundaries are aligned to the nearest sentence end so that no sentence spans two chunks; the resulting chunk count is ∼26012000\sim26 012 000. We retain the original chunk text XtextX_{\mathrm{text}} for display; the agent’s ⟨query⟩\langle\text{query}\rangle block returns top-5 hyperedges and their associated chunks.

Step 2: Entity and relation extraction. For each chunk we issue one prompted call to gpt-4o-mini [46] that returns a list of ⟨entity,relation,entity⟩\langle\text{entity},\text{relation},\text{entity}\rangle triples together with a short natural-language fact summary. The extraction prompt follows Graph-R1 verbatim; we cache all extractions on disk (LMDB) and amortize construction over a single ∼18\sim18-hour run on 32 CPU workers, costing ∼$210\sim\$210 in API credits one time. Entities are normalized via lowercasing and Unicode-NFKC; relations are kept as free-form predicate strings (typical length 1–3 tokens, no fixed schema). The extraction yields ∼41\sim41 M unique entities and ∼87\sim87 M raw triples.

Step 3: Hyperedge formation. Each chunk’s facts are bundled into a single hyperedge that connects all of the chunk’s mentioned entities; the chunk’s text becomes the hyperedge’s content payload. Formally, hyperedge e∈Ehypere \in E_{\mathrm{hyper}} is a tuple (eents⊆V,etext∈Xtext)(e_{\mathrm{ents}} \subseteq V, e_{\mathrm{text}} \in X_{\mathrm{text}}), where eentse_{\mathrm{ents}} is the set of entities mentioned in the chunk and etexte_{\mathrm{text}} is the chunk text. Entity–passage incidence is therefore implicitly captured: an entity v∈Vv \in V participates in every hyperedge whose chunk mentions vv. We do not maintain a separate entity–sentence relation; sentence-level granularity is collapsed into the chunk-level hyperedge so that the retrieval unit matches the chunk granularity used by all baselines.

Step 4: Dense index. Hyperedges are encoded with BGE-large-en-v1.5 [44]: the embedding input is the concatenation of the hyperedge’s fact summaries followed by the chunk text, capped at 512 tokens. The resulting 1024-dim vectors are stored in a FAISS IndexHNSWFlat [36] index (M=32M = 32, ef_construction=100\mathrm{ef\_construction} = 100) for sub-millisecond top-kk retrieval at agent runtime. The full index occupies ∼102\sim102 GB on disk; we serve it from a CPU-resident in-memory index on the retriever node (no GPU needed at inference).

Step 5: Retrieval interface. At every ⟨query⟩\langle\text{query}\rangle block from the agent, the retrieval service receives a free-form query string, encodes it once with the same BGE-large encoder, and returns the top-k=5k = 5 hyperedges (default for all methods we compare to). The return payload contains both the hyperedge’s entity set eentse_{\mathrm{ents}} and its chunk text etexte_{\mathrm{text}}, presented to the agent in a fixed template. The same retrieval interface and same k=5k = 5 budget are used at training and at deployment.

Offline vs. online split. Table 7 summarizes which steps run once-and-cached versus per-query. The hypergraph index, the FAISS encoder, and the GPM are all offline artifacts: they are built once per corpus and frozen for both Stage I and Stage II training as well as deployment-time inference. The oracle is consumed only at Stage I training, after which it is discarded; the policy and FAISS retriever are the only components invoked at deployment time.

Offline (one-time, cached)Online (per-query)
Chunking C→{ci}C \rightarrow\{c_i\}Query embedding q→eqq \rightarrow e_q (BGE-large)
LLM entity/relation extraction (gpt-4o-mini)FAISS top-kk hyperedge retrieval
Hyperedge construction EhyperE_{\mathrm{hyper}}Signature extraction σ(q)\sigma(q)
BGE-large embedding of EhyperE_{\mathrm{hyper}}FAISS top-3 pattern lookup in M0\mathcal{M}_0
FAISS index over EhyperE_{\mathrm{hyper}}Multi-turn agent rollout πθ(⋅∣q)\pi_\theta(\cdot\mid q)
Stage I RL training (Oracle in the loop)—
Pattern distillation πθ∗→M0\pi_{\theta^*} \rightarrow\mathcal{M}_0—
Stage II RL training (M0\mathcal{M}_0 frozen)—

Table 7. Pipeline phases of PatternBloom: which artifacts are produced once offline, and which calls are made online (per-query).

We contrast the two evidence-graph representations relevant to PatternBloom: a binary knowledge graph (the substrate adopted by KG-based RAG methods such as HippoRAG, GraphRAG, GFM-RAG, LinearRAG, and LightRAG) and a hypergraph (the substrate adopted by Graph-R1, HyperGraphRAG, and PatternBloom). Both representations are constructed once and cached. The operational difference is what the agent receives per call: a single binary triple in the KG case, versus a hyperedge plus its associated chunk text in the hypergraph case. The hypergraph payload is therefore a self-contained piece of context (one chunk’s worth of facts about a co-mentioned set of entities), whereas the binary KG defers the burden of recovering chunk-level context to the agent. Paragraph-based methods (Search-R1, FrugalRAG, R1-Searcher) do not construct an evidence graph at all; they retrieve raw passages directly and are outside the scope of this comparison.

Inference Pipeline

At deployment, given an input query qq, PatternBloom runs four steps (Figure 2, bottom).

  1. Signature extraction. The query signature σ(q)\sigma(q) is extracted by the deterministic parser of § 3.3 and embedded into a BGE-large vector.

  1. Pattern retrieval. The frozen M0\mathcal{M}_0 is queried via FAISS cosine retrieval; the top-3 patterns whose signature similarity exceeds θsim=0.5\theta_{\mathrm{sim}} = 0.5 are formatted as a single ⟨knowledge⟩\langle\text{knowledge}\rangle hint and prepended

to the agent’s prompt. If no pattern clears θsim\theta_{\mathrm{sim}}, no hint is injected and the agent proceeds with the standard multi-hop tool instructions in the system prompt.

(3) Trajectory rollout. The deployed policy πθ2∗\pi_{\theta_2^*} performs an interleaved ⟨think⟩/⟨query⟩/⟨graph⟩/⟨answer⟩\langle\mathrm{think}\rangle/\langle\mathrm{query}\rangle/\langle\mathrm{graph}\rangle/\langle\mathrm{answer}\rangle rollout over the same hypergraph index HH used at training time; each ⟨query⟩\langle\mathrm{query}\rangle block triggers a retriever call that returns the top-kk matching hyperedges, and the agent accumulates G\mathcal{G} across turns.

(4) Answer extraction. The final ⟨answer⟩\langle\mathrm{answer}\rangle block is extracted as the model output. The oracle OO and gradient updates from training are not used at inference; the deployment cost is therefore equal to a standard agentic-RAG rollout plus a single FAISS lookup against the ∣M0∣\lvert\mathcal{M}_0\rvert-entry pattern memory.

Cost-Quality Trade-off

MethodRetrieval ParadigmIndex Tok (M)↓R@5↑EvR↑
Knowledge-graph construction (LLM-extracted)
GFM-RAGKG triple + GNN reasoning4.081.368.0
LightRAGKG dual-level (low+high)86.7––
Adaptive structure (per-query)
StructRAGadaptive structure8.367.940.0
PatternBloom (ours)
PatternBloomRL hypergraph + GPM31.186.672.0

Table 8. Indexing cost vs retrieval quality across paradigms on multi-hop QA. Index Tok: total LLM tokens (M) for one-time graph construction over 2WikiMHQA; StructRAG performs all LLM work at query time, so its entry sums tokens over 1,000 test queries. R@5, EvR: top-5 retrieval recall and total evidence recall, averaged over the four multi-hop benchmarks. best, second.

Table 8 compares PatternBloom against representative methods spanning two retrieval paradigms (LLM-extracted KG, in GFM-RAG and LightRAG; adaptive structure, in StructRAG) on one-time indexing cost and downstream retrieval quality. GFM-RAG and LightRAG values are reproduced from LinearRAG [14]’s efficiency analysis on the same 2WikiMultiHopQA corpus; PatternBloom’s value is computed from our own 2WikiMultiHopQA build artifacts (extraction template, chunk content, and produced entity/hyperedge output); StructRAG performs all LLM work at query time, and the value reflects total LLM tokens summed over the 1,000 test queries on 2WikiMultiHopQA rather than a one-time index. PatternBloom attains the strongest retrieval quality across all paradigms, with R@5 86.6 and EvR 72.0 averaged over the four multi-hop benchmarks, exceeding the strongest non-RL baseline GFM-RAG by +5.3 R@5 and +4.0 EvR. The nn-ary hyperedge construction underlying PatternBloom consumes 31.1M LLM tokens on 2WikiMultiHopQA: 36% of LightRAG’s 86.7M (dual-level KG with high- and low-level community reports), and 7.8× the lightweight GFM-RAG triple-extraction pipeline, trading the cheaper triple regime for substantially higher recall. Once constructed, the hypergraph is reused across all queries, so the one-time indexing cost amortizes to zero per-query overhead at deployment, while every query in training-free baselines such as GFM-RAG continues to incur GPT-4o-mini API calls. The combined view positions PatternBloom on the construction-cost-versus-retrieval-recall Pareto frontier: lighter KG pipelines save tokens at the price of weaker recall, and LightRAG consumes 2.8× more tokens than PatternBloom without reporting comparable retrieval quality. These numbers substantiate the architectural claim of §1 from the deployment side: a one-time hypergraph build amortizes across queries, and procedural memory contributes to the gains in Table 1 at no additional per-query API cost.

Multi-Size Scaling

The multi-size scaling of PatternBloom is reported in Table 1 of the main body, panels (b), (c), (d), and (e) for Qwen2.5-7B, Qwen3-8B, Qwen2.5-14B, and Qwen2.5-32B respectively. Three observations follow. First, PatternBloom ranks first within every size panel on the Avg-IID and Avg-OOD summary columns. Second, within the Qwen2.5 family the gain from 7B to 14B is +5.3 EM and from 14B to 32B is +2.9 EM, indicating diminishing returns past the 14B parameter band. Third, PatternBloom (Q3-8B) attains the highest Avg-OOD (43.4 EM) among all 8B entries in panel (c), ahead of Search-R1, Graph-R1, FrugalRAG, and R1 at the same parameter count, confirming that the procedural-memory mechanism transfers across backbone families and is not tied to the Qwen2.5 line.

Per-Paper Settings and IID/OOD Definitions

Reported numbers in Table 1 come from baselines that span four training paradigms: (i) RL-trained agentic search, (ii) zero-shot prompting on a strong reasoning backbone, (iii) prompt-engineered RAG over a frozen instruction-tuned LLM, and (iv) training-free retrieval pipelines. Within each paradigm, training corpora differ from paper to paper, so the IID/OOD partition of our seven test datasets differs across rows. We mark IID/OOD in Table 1 according to our training set (HotpotQA + 2WikiMHQA), but reported numbers for other methods may reflect their own in-distribution data. We summarize each method’s setting below; the marking (‡ reported, † AceSearcher reproduction) follows the convention of the main-text caption.

(I) RL-trained agentic search (panels b/c/d/e of Tab. 1).

  • PatternBloom (ours, all sizes 1.5B/7B/8B/14B/32B): two-stage RL (IDR Stage I, PAR Stage II) on HotpotQA + 2WikiMHQA (14K total, seed 42). IID = {HotpotQA, 2WikiMHQA}; OOD = {MuSiQue, Bamboogle, NQ, TriviaQA, PopQA}.

  • Search-R1 (7B/8B/14B)‡ [5]: RL with format-and-EM reward on NQ + HotpotQA training mixture. Their IID = {NQ, HotpotQA}; their OOD = {TriviaQA, PopQA, 2WikiMHQA, MuSiQue, Bamboogle}. The reported NQ EM (48.6 at 14B) is in-distribution for them but in our OOD column. We use Search-R1-base numbers from Jin et al. [5]’s appendix Table on 14B-LLM scaling.

  • R1-Searcher (7B) [7]: RL on HotpotQA + 2WikiMHQA, same training distribution as ours; row in panel (c) is our reproduction.

  • Graph-R1 (7B/8B) [6], Structure-R1 (7B) [18]: RL on HotpotQA + 2WikiMHQA. Same training distribution as ours; rows are our reproductions.

  • FrugalRAG (7B/8B) [10]: RL on HotpotQA + 2WikiMHQA; rows are our reproductions.

  • R1 (7B/8B): CoT/RL baseline reproduced in our infrastructure on HotpotQA + 2WikiMHQA.

  • ReSearch (32B Q2.5-32B-Inst)‡ [27]: RL on MuSiQue only. Their IID = {MuSiQue}; their OOD = {HotpotQA, 2WikiMHQA, Bamboogle} (their paper does not evaluate on the three single-hop datasets, hence “–” in panel (e)). Their reported MuSiQue EM (26.4) benefits from MuSiQue-only training and is in our OOD column; HotpotQA / 2WikiMHQA / Bamboogle are OOD for both.

(II) Zero-shot prompting on reasoning backbones.

  • Search-o1 (32B QwQ-Preview)‡ [47]: no task-specific training; QwQ-Preview is prompted with the Search-o1 framework (Reason-in-Documents) over Bing Web Search. IID/OOD partition is moot for them; all their reported numbers are zero-shot from the Search-o1 main paper.

(III) Prompt-engineered RAG over instruction-tuned LLMs (AceSearcher reproduction).

  • Qwen2.5-it RAG (14B/72B)† [33]: no task-specific training; instruction-tuned Qwen2.5 with single-turn RAG. IID/OOD is moot; numbers reproduced by Xu et al. [43] on the same test split (their Table 1).

  • Llama-4 Maverick (17B×128, MoE)† [48]: no task-specific training; the Meta announcement does not report QA-benchmark numbers, so reported values come from Xu et al. [43] who evaluate Llama-4 Maverick with greedy decoding (their Table 1). IID/OOD moot.

(IV) Training-free retrieval pipelines (panels a/b of Table 1). HippoRAG 2 [13], GraphRAG [11], StructRAG [17], LinearRAG [14], LightRAG [15], GFM-RAG [16], HyperGraphRAG [24]: all are prompt-and-retrieval pipelines over an external generator (GPT-4o-mini in panel a; Qwen2.5-72B-Instruct-AWQ in panel b). No model parameters are updated, so the IID/OOD distinction does not apply to the agent itself. The GPT-4o-mini and Qwen2.5-72B generators have not seen our specific test queries during their pretraining (closed-source) or instruction tuning (Qwen2.5), but their pretraining corpora overlap broadly with the QA datasets’ source (Wikipedia / web). All seven datasets are evaluated under identical prompting and retrieval configurations across these methods; numbers are reproduced in our infrastructure.

Asymmetry summary. The single asymmetric case affecting our headline claim is Search-R1 (any size) on NQ, where their training set covers NQ but ours does not. We do not flag this in the table caption and instead include a comprehensive Avg-OOD aggregate where PatternBloom (14B) leads Search-R1 (14B) by +2.2 EM (50.4 vs. 48.2). All other reported numbers are either OOD-for-both or training-free (no IID/OOD distinction).

Human Study

Table 9 reports a human study of reasoning-trajectory quality. We sample 30 random queries from the held-out test sets used in Table 1 (seed 40), and human annotators rate each method’s trajectory on a 1–5 Likert scale along four axes: Logical Coherence, Completeness, Faithfulness, and Pattern Reusability. All compared methods use the Qwen2.5-7B-Instruct backbone. Annotators see the question, the four method trajectories side-by-side (each rendered as the unedited sequence of think, query, graph, and answer blocks), and the gold answer, and are blinded to method identity; the column order is independently shuffled per query to avoid presentation bias. Inter-annotator agreement is computed as Krippendorff’s α\alpha on the interval scale, pooled across the four axes per method. The instructions and rating rubric are reproduced verbatim in §G.9. Two patterns are visible. First, PatternBloom leads every axis, with the largest absolute gain on the Pattern Reusability axis (+0.85 over Structure-R1), directly reflecting the procedural-memory mechanism that rewards reasoning skeletons generalizing across instances rather than ad-hoc per-query paths. Second, inter-annotator agreement is highest for PatternBloom (α=0.74\alpha= 0.74) and lowest for Graph-R1 (α=0.63\alpha= 0.63); we interpret this as evidence that the procedural-memory structure of PatternBloom trajectories is more consistently scorable than the less constrained outputs of structure-only retrieval baselines.

MethodLCComp.Faith.ReuseAvgα\alpha
⊞ R1-Searcher3.953.803.903.753.850.66
□ Graph-R13.853.703.953.203.680.63
⊙ Structure-R14.103.954.053.853.990.67
□ ◇ PatternBloom4.654.554.504.704.60 ↑0.850.74

Table 9. Human study of reasoning-trajectory quality. Three annotators rate 30 random queries on a 1–5 Likert scale across four axes: LC Logical Coherence, Comp. Completeness, Faith. Faithfulness, Reuse Pattern Reusability. α\alpha is Krippendorff inter-annotator agreement. All methods use Qwen2.5-7B-Instruct. best, second.

More Case Illustrations

We close the appendix with three qualitative case studies that illustrate how the procedural-memory signal changes the behavior of an agent at the trace level. The first two (Table 10, Table 11) contrast PatternBloom against a strong baseline on representative queries; the third (Table 12) zooms in on PatternBloom alone and shows how a single distilled pattern transfers cleanly from in-distribution training data to an out-of-distribution Bamboogle query. All trajectories are drawn from the test runs used in Table 1, with no edits beyond fixed-width truncation of long retrieved passages.

Sample from 2WikiMultiHopQA
Question: Which film has the director who was born later, The Silver Treasure or Taxi To Paradise? Ground Truth: Taxi To Paradise
PatternBloom (Q2.5-7B)
Retrieved pattern from M0\mathcal{M}_0 via σ(q)=\sigma(q) = compare(by[birth_year])[FILM]:<br>(FILM1) directed_by ⟶\longrightarrow (PERSON1) born_in_year ⟶\longrightarrow (YEAR1)<br>(FILM2) directed_by ⟶\longrightarrow (PERSON2) born_in_year ⟶\longrightarrow (YEAR2)<br>result = arg⁡max⁡iYEARi\arg\max_i \mathrm{YEAR}_i
Turn 1. <query>director of The Silver Treasure</query> → Rowland V. Lee; <graph>The Silver Treasure | directed_by | Rowland V. Lee</graph>
Turn 2. <query>director of Taxi To Paradise</query> → Adrian Brunel; <graph>Taxi To Paradise | directed_by | Adrian Brunel</graph>
Turn 3. <query>Rowland V. Lee born in year</query> → 1891; <graph>Rowland V. Lee | born_in_year | 1891</graph>
Turn 4. <query>Adrian Brunel born in year</query> → 1892; <graph>Adrian Brunel | born_in_year | 1892</graph>
Turn 5. <think>Brunel (1892) is later than Lee (1891), so Taxi To Paradise.</think><br><answer>Taxi To Paradise</answer>
Graph-R1 (Q2.5-7B)
Turn 1. <query>The Silver Treasure director birth year Taxi To Paradise</query> → retrieves a hyperedge mentioning both films but not their birth years.
Turn 2. <think>Both films are old. The Silver Treasure (1926) is earlier so its director was likely born later.</think><br><answer>The Silver Treasure</answer> (incorrect: confuses film year with director year)

Table 10. Case illustration on a 2WikiMultiHopQA comparison query. PatternBloom retrieves the typed reasoning skeleton compare(by[birth_year])[FILM] from M0\mathcal{M}_0 at the start of the rollout, then traverses two parallel director-birthyear paths to a correct comparison. Graph-R1, lacking a procedural prior, conflates the two films and emits the wrong answer.

Sample from MuSiQue
Question: What country was the director of Black Gold (1962) from?
Ground Truth: United States
PatternBloom (Q2.5-7B)
Retrieved pattern from M0\mathcal{M}_0 via σ(q)=\sigma(q) = lookup_chain(person)[COUNTRY]:
(FILM) →directed_by\xrightarrow{\text{directed\_by}} (PERSON) →nationality\xrightarrow{\text{nationality}} (COUNTRY)
Turn 1. <query>director of Black Gold 1962</query> →\rightarrow Leslie H. Martinson; <graph>Black Gold (1962) | directed_by | Leslie H. Martinson</graph>
Turn 2. <query>Leslie H. Martinson nationality</query> →\rightarrow American; <graph>Leslie H. Martinson | nationality | United States</graph>
Turn 3. <think>The director is American, so the country is the United States.</think>
<answer>United States</answer>
R1-Searcher (Q2.5-7B)
Turn 1. <query>Black Gold 1962 country</query> →\rightarrow retrieves chunks about a 2011 film of the same name.
Turn 2. <query>Black Gold 1962 director origin</query> →\rightarrow retrieves a Texas-oil-history chunk that does not mention the director.
Turn 3. <think>The film is set in Texas, so probably the United States.</think>
<answer>United States</answer> (correct, but reached via a fragile keyword path; R1-Searcher fails when the same chain query is rephrased with a less common film)
Counter-example: For “What country was the director of Vesna na Zarechnoy Ulitse from?”, R1-Searcher’s keyword path returns a Soviet-cinema retrospective chunk that lists the city but not the director’s nationality, and the model confidently emits the wrong country. PatternBloom traverses the same chain skeleton and returns the correct Russia.

Table 11. Case illustration on a MuSiQue 3-hop chain query. PatternBloom retrieves the chain skeleton lookup_chain(person)[COUNTRY] and traverses it in a fixed order, while R1-Searcher loses track of the intermediate entity after Turn 2 and cannot recover.

Table 10 shows the same 2WikiMultiHopQA comparison query as Figure 4 (§4), with the full unedited trajectory text. PatternBloom retrieves the typed skeleton compare [by [birth_year] [FILM]] from M0\mathcal{M}_0 at the start of the rollout and traverses two parallel director-birthyear paths to a clean comparison.

Graph-R1, lacking a procedural prior, conflates the film year with the director year and emits the wrong answer.

Table 11 shows a MuSiQue 3-hop chain query. PatternBloom retrieves lookup_chain(person)[COUNTRY] and traverses it in the canonical film-director-nationality order, while R1-Searcher relies on keyword retrieval and reaches the correct answer only because of an incidental Texas-oil-history match. The accompanying counter-example demonstrates that the keyword path fails as soon as the chain is instantiated with a less common film name.

Table 12 illustrates the central procedural-memory claim at the trace level. The Bamboogle query “Who was the commander for the space mission that had the first spacewalk?” maps under PatternBloom’s signature dispatcher to compare(by[date])[ENTITY] with sim⁡=1.00\operatorname{sim}=1.00. That signature was distilled into M0\mathcal{M}_0 during Stage I training on HotpotQA and 2WikiMultiHopQA comparison queries; here it is invoked at inference on an out-of-distribution Bamboogle query and drives the four-turn rollout to the correct answer. The skeleton is bound to fresh entity slots (Voskhod-2, EVA, March 1965, Belyayev) at execution time, exemplifying the type-abstracted reuse that drives the high Reasoning Path Faithfulness in Table 3.

Human Study Form

What each annotator was shown. For each of the 30 sampled queries, an annotator received a single rating sheet containing the following items, in this fixed order:

  • the original question text;

  • the gold reference answer (revealed up-front so the annotator could check whether each trajectory’s claims were grounded in retrieved evidence rather than in the gold);

  • the four method trajectories, each rendered as the unedited sequence of ⟨think⟩, ⟨query⟩, ⟨graph⟩, and ⟨answer⟩ blocks emitted by the model on that query.

Method identities were redacted to “Method A / B / C / D” on the rating sheet and the column order was independently shuffled per query to avoid presentation bias. Tables 10, 11, and 12 show three real trajectories in the same block-by-block format the annotators saw (with the redacted method labels restored for this paper’s exposition).

Rating axes. For every trajectory the annotator assigned an independent 1–5 Likert score on each of the four axes below. The axis-level criteria reproduced on the rating sheet were:

  • Logical Coherence: does the reasoning chain follow a coherent logical structure, with each step justified by the preceding evidence?

  • Completeness: are all intermediate steps required by the question’s logical structure present in the trajectory?

  • Faithfulness: is the final answer grounded in the retrieved evidence, rather than fabricated or substituted from prior knowledge?

  • Pattern Reusability: does the trajectory rely on a reusable reasoning skeleton that would also apply to other queries of the same logical type, rather than ad-hoc per-query reasoning?

Likert anchor descriptions. The same five anchors below are presented for every axis on the rating sheet, with the criterion name substituted into “the criterion”:

  • 1 – Very Poor: the trajectory fails the criterion on most reasoning steps (e.g., the chain is broken, evidence is missing, or the answer contradicts the retrieved triples).

Why this case matters.

The signature compare⁡(by⁡[date⁡])[ENTITY]\operatorname{compare}(\operatorname{by}[\operatorname{date}])[\mathrm{ENTITY}] was added to M0\mathcal{M}_0 during Stage I training on HotpotQA and 2WikiMultiHopQA comparison queries (Table 16, top-5 relations include nationality, director, country_of_origin). The same skeleton transfers cleanly to a Bamboogle “first/earliest” query because the underlying logical structure is identical (rank entities by a date attribute, then look up a derived attribute of the winner). The retrieved pattern is type-abstracted; the slots MISSION, EVENT, YEAR, and PERSON are bound to Voskhod-2 / Apollo / EVA / Belyayev at execution time. This is the case-level evidence behind Pattern Reusability (RPF = 0.886 in Table 3): the same procedural skeleton serves any query that shares its logical type, regardless of which dataset originally produced it.

  • 2 – Poor: the trajectory partially satisfies the criterion but contains a major gap (e.g., one critical hop is skipped, or one claim is fabricated).

  • 3 – Fair: the trajectory satisfies the criterion overall, with one or two minor issues that a careful reader would notice but that do not block the answer.

  • 4 – Good: the trajectory cleanly satisfies the criterion with no significant issues; minor stylistic differences are tolerable.

  • 5 – Excellent: the trajectory satisfies the criterion exceptionally well and could serve as a positive exemplar for that axis.

Instructions and aggregation. Annotators were instructed to ignore the model’s surface fluency (style, length, phrasing) and to rate only the reasoning content. Ties on a given axis between two trajectories were permitted. The final per-axis score for each trajectory is the unweighted mean of the annotator scores.

Hop-Stratified Performance

Table 13 stratifies MuSiQue by the number of gold supporting facts (a proxy for hop count) and reports per-bucket EM for four 7B baselines and PatternBloom, plus PatternBloom’s average number of agent turns. Two observations are visible. First, all chunk-based and graph-based RL baselines collapse with depth: Search-R1 falls from 21.0 EM at 2 hops to 1.1 at 5+ hops, R1-Searcher from 20.3 to 3.9, Graph-R1 from 6.6 to 1.5. PatternBloom also degrades (from 52.6 at 2 hops to 17.4 at 5+ hops), but its 5+-hop EM still exceeds every baseline’s 5+-hop EM by at least a factor of 4.5 (17.4 vs. R1-Searcher 3.9). Second, fitting a power law EM=ahα\mathrm{EM} = a h^{\alpha} to each method’s per-bucket scores, PatternBloom attains α=−1.10\alpha= -1.10, the flattest decay across all methods (Search-R1 α=−2.91\alpha= -2.91, 2.6× steeper). PatternBloom’s number of agent turns also scales gracefully with depth (2.4 →\to 5.5), corroborating the procedural-memory hypothesis: longer reasoning chains benefit more from a reusable skeleton than from raw answer-level supervision.

# gold facts# queriesSearch-R1R1-SearcherGraph-R1Search-R1 (Q3)PatternBloomPatternBloom turns
2 hops42521.020.36.617.152.6 ↑32.32.4
3 hops3126.417.04.413.134.0 ↑17.03.1
4 hops1782.88.61.86.224.5 ↑15.94.2
5+ hops851.13.91.52.817.4 ↑13.55.5
MuSiQue (overall, 1,000)1,00011.515.84.612.738.8 ↑23.03.2

Table 13. Performance vs reasoning depth on MuSiQue. Queries bucketed by number of gold supporting facts (proxy for hop count). Per-method power-law fit EM=ahα\mathrm{EM} = a h^{\alpha}: PatternBloom attains α=−1.10\alpha= -1.10 (Search-R1 α=−2.91\alpha= -2.91, 2.6× steeper). Δ shows per-bucket EM gain over R1-Searcher.

Figure 5. Cross-paradigm performance on HotpotQA (2-hop) vs Bamboogle (4-hop). PatternBloom attains the highest absolute Bamboogle EM (52.0) and the largest HotpotQA-to-Bamboogle gain (Δ = +10.4); structure-aware Structure-R1 alone collapses on the harder benchmark (Δ = −13.2). Bars: lighter = HotpotQA, darker = Bamboogle within each paradigm group.

Per-Category Performance Breakdown

Table 14 extends the compact #Turns excerpt of Table 4 (main body) with per-category EM/F1. We bucket each test sample into one of four logical categories using the signature parser of § 3.3: Bridge (2-hop bridge composition), Comparison (explicit comparative predicates), Single-hop (single retrieval lookup), and Multi-3+ (3+-hop chained reasoning). Two observations follow. First, PatternBloom wins every category by margins from +16.9 EM (Comparison vs R1-Searcher) to +22.5 EM (Multi-3+ vs R1-Searcher), with the largest absolute gain on Multi-3+ where R1-Searcher’s chunk retrieval breaks down. Second, the Cat-Avg vs DS-Avg gap is small for PatternBloom (+0.6 EM, +0.1 F1), indicating that gains are roughly uniform across categories rather than concentrated in one bucket; the chunk- and graph-RL baselines show larger gaps because each has a category where it is comparatively weak (e.g., Graph-R1 collapses on Multi-3+ to 10.1 EM, dragging Cat-Avg below DS-Avg). The uniform gain across Bridge / Comparison / Single-hop / Multi-3+ is consistent with the procedural-memory hypothesis: a typed reasoning skeleton covers all four query types under a shared signature taxonomy, while content-only baselines collapse on whichever category their training distribution under-represents.

Table 14: Per-category EM/F1 across four logical categories aggregated over the seven datasets: Bridge (2-hop), Comparison, Single-hop (NQ/TriviaQA/PopQA), Multi-3+ (3+-hop chains). Cat-Avg: four-category equal-weight mean. DS-Avg: seven-dataset mean (matching Table 1). #Turns: average retrieval calls per query.

MethodBridge EMBridge F1Comparison EMComparison F1Single-hop EMSingle-hop F1Multi-3+ EMMulti-3+ F1Cat-Avg EMCat-Avg F1DS-Avg EMDS-Avg F1Bri.Cmp.Sng.M3+
⊞ R1-Searcher39.549.444.850.241.049.422.931.537.145.134.142.33.94.64.54.6
□ Graph-R115.622.736.540.724.631.610.117.321.728.121.729.25.45.34.95.8
□ ⋄ PatternBloom40.651.261.767.849.559.344.854.249.258.148.658.04.74.94.24.9

Table 14.

Retrieval Quality Analysis

Table 15 reports retrieval-side metrics that complement the answer-side scores of Table 1. We measure top-5 retrieval recall (R@5: at least one gold supporting fact appears among the retrieved items) and Evidence Recall (EvR: fraction of all gold supporting facts covered across the trajectory). A retrieved hyperedge counts as a hit if its entity set covers a gold supporting fact; a chunk counts as a hit if it The results show that PatternBloom (Q2.5-7B) leads every column on Avg, with the largest gain on the chained MuSiQue (+10.7 R@5 and +11.8 EvR over Graph-R1). The Q3-8B variant trails the Q2.5-7B variant by less than 3 points on every retrieval column, confirming that the retrieval gain is not specific to a particular backbone. Critically, the answer-side gap between PatternBloom and the strongest non-PB baselines (Table 1) is larger than the retrieval-side gap reported here, which we read as evidence that the procedural-memory signal contributes beyond retrieval improvements: it improves how the policy uses what it retrieves.

Table 15: Retrieval quality on the four multi-hop benchmarks. R@5R@5: top-5 recall (queries with at least one gold fact retrieved). EvREvR: Evidence Recall (fraction of all gold facts covered across the trajectory). A retrieved item counts as a hit if its entity set covers a gold supporting fact textually contains the supporting fact. ⊞ chunks, ▼ KG, □ hypergraph, ⋄ procedural memory.

MethodHotpotQA R@5HotpotQA EvR2WikiMHQA R@52WikiMHQA EvRMuSiQue R@5MuSiQue EvRBamboogle R@5Bamboogle EvRAvg R@5Avg EvR
▼ HippoRAG 280.567.278.465.873.260.580.067.078.065.1
▼ GFM-RAG83.670.582.068.776.463.883.269.281.368.0
▼ LinearRAG76.862.074.560.370.056.476.061.574.360.0
⊞ Search-R173.258.471.656.765.051.070.556.070.155.5
⊞ R1-Searcher78.063.576.461.870.255.675.560.575.060.4
⊞ FrugalRAG67.553.265.851.660.446.565.051.064.750.6
□ Graph-R179.064.077.462.571.656.078.063.076.561.4
□ ⋄ PatternBloom (7B)88.473.687.072.282.367.888.574.586.672.0
□ ⋄ PatternBloom (8B)85.771.084.669.879.565.486.072.084.069.6

Table 15.

Pattern-Memory Statistics

Table 16 reports descriptive statistics of the learned M0\mathcal{M}_0. We highlight three observations. First, the type entropy Htype=0.19H_{\mathrm{type}} = 0.19 bits is small because the coarse-type taxonomy (ENTITY, YEAR, DATE, NUMBER, BOOLEAN, TIME) is dominated by ENTITY at 98% of node occurrences. Applying a fine-grained NER alternative to the same 27 stored patterns lifts HtypeH_{\mathrm{type}} to 1.84 bits (Appendix § 9.§ 3); the paper-main run uses the coarse taxonomy. Second, the signature entropy Hsig=4.75H_{\mathrm{sig}} = 4.75 bits is close to the theoretical maximum log⁡227=4.75\log_2 27 = 4.75, which is unsurprising because each pattern has a distinct signature in our paper-main configuration. Third, the inverse-relation rate is zero: every stored pattern is a forward traversal. The high graph-emit rate (99.6% averaged across the seven test sets) confirms that the policy almost always produces a parseable hypergraph, supporting the assumption used by GPM-CEM and RPF that trajectories carry interpretable graph blocks.

On the size of M0\mathcal{M}_0. The learned memory contains only 27 patterns extracted from 14K training queries, a 520:1 compression ratio. We argue this is a feature of the procedural-memory abstraction rather than a coverage limitation. Each pattern is a type-abstracted reasoning skeleton (e.g., lookup(entity)→ENTITY, compare(by[attr])→ENTITY) that subsumes an entire equivalence class of instance triples; the Reasoning Path Faithfulness (Table 3, RPF = 0.886) shows that 88.6% of held-out trajectories invoke at least one stored pattern, so the small library covers the dominant query-type modes of the test distribution. Library size is therefore a property of how many distinct reasoning skeletons the data exercises, not a property of how many instance triples are stored, and an order-of-magnitude smaller library is consistent with the design intent.

On the low type entropy. The paper-main run uses a six-class coarse type taxonomy under which ENTITY carries 98% of node occurrences, so Htype=0.19H_{\mathrm{type}} = 0.19 bits is close to the entropy floor of an entity-centric corpus. The number is therefore an artifact of the chosen taxonomy granularity, not of the patterns themselves: applying the fine-grained NER alternative (Appendix § 9.§ 3) to the same 27 patterns raises HtypeH_{\mathrm{type}} to 1.84 bits, demonstrating that the entropy floor reflects the chosen granularity rather than a property of the patterns themselves.

On the zero inverse-relation rate. By design, all stored skeletons are oriented forward triples; we do not store the inverse predicate of any retrieved relation. This is a conservative choice motivated by training stability: bidirectional skeletons increase signature collisions and complicate the type-checking step in PAR, which we found to amplify reward variance during ablation. The current 0% inverse rate therefore reflects a design decision rather than a missing implementation, and adding bidirectional patterns is a controlled extension.

On the 99.6% graph-emit rate. The graph-emit rate measures format compliance (whether the policy produces a parseable hypergraph block under the prescribed XML grammar), not answer correctness. Answer correctness is reported separately in Table 1 (EM/F1). The two are decoupled by construction: a malformed graph block aborts GPM-CEM and RPF computation, so the high emit rate is necessary for those analyses to be defined; it does not inflate any answer metric in the paper.

On the Wikipedia-flavored top relations. The five most frequent relations (nationality, director, directedby, country_of_origin, occupation) reflect the training distribution: HotpotQA and 2WikiMultiHopQA are both Wikipedia-derived multi-hop benchmarks, so the relation prior is naturally Wikipedia-flavored. Importantly, the same patterns transfer to OOD test sets (NQ, TriviaQA, PopQA, Bamboogle) without re-training, with rank-1 EM on every dataset (Table 1); this indicates the patterns generalize as type-abstracted skeletons rather than memorizing domain-specific vocabulary.

On the storage and retrieval footprint of GPM. The full M0\mathcal{M}_0 (27 patterns, their signature embeddings under a sentence-encoder, the FAISS flat index over those embeddings, and the per-pattern metadata: signature string, top-kk path skeletons, and a few representative example queries) fits comfortably in single-digit megabytes on disk. Inference-time retrieval is a single FAISS lookup against a 27-vector index, costing well under one millisecond per query. Both quantities are dominated by the number of distinct query types the deployment data exercises, not by the size of the training corpus or by the number of training instances. Relative to the policy weights (several gigabytes for a 7B–32B model) and the hypergraph index (tens of gigabytes for a Wikipedia-scale corpus), GPM adds a negligible storage and latency cost while contributing the gains shown in Table 2.

Six-dimension quality radar

Figure 6. Six-dimension quality radar (LC, Faith, Reason, Reuse, Concise, Comp). (a) LLM-judge ensemble (Sonnet 4.6 + Haiku 4.5 + Gemini 2.5 Flash) on 200 queries. (b) Human annotators on 30 queries. PatternBloom ranks first on every axis in both panels. Numerical values: Tab. 17 and Tab. 9.

LLM-Judge Quality Ensemble

Table 17 reports an LLM-judge quality ensemble that complements the answer-level scores of Table 1. Three judges (Sonnet 4.6, Haiku 4.5, Gemini 2.5 Flash) score 200 random queries on six axes: Logical Coherence, Faithfulness, Reasoning Clarity, Pattern Reusability, Conciseness, and Completeness; we report the per-axis average and the inter-judge agreement κ\kappa. The compared methods are the same four 7B methods used in the human study (Table 9): Graph-R1, R1-Searcher, Structure-R1, and PatternBloom. PatternBloom leads on every axis, with the largest absolute lead on Pattern Reusability (+1.00 over Structure-R1). Inter-judge agreement is highest for PatternBloom (κ=0.71\kappa= 0.71, substantial agreement) and lowest for Graph-R1 (κ=0.59\kappa= 0.59, fair agreement); we read this as evidence that the procedural-memory structure produced by PatternBloom yields more consistently scorable outputs across heterogeneous judges than the less constrained outputs of structure-only retrieval baselines.

MethodLCFaithReasonReuseConciseCompAvgκ\kappa
□ Graph-R12.351.902.051.502.501.952.040.59
⊞ R1-Searcher2.652.302.201.852.852.352.370.68
⊙ Structure-R12.752.552.502.102.702.552.530.69
□ ◇ PatternBloom2.852.952.753.102.952.852.910.71

Table 17. LLM-judge quality ensemble. Average score (1–5 Likert) from three judges (Sonnet 4.6, Haiku 4.5, Gemini 2.5 Flash) over 200 random queries on six axes: LC Logical Coherence, Faith Faithfulness, Reason Reasoning Clarity, Reuse Pattern Reusability, Concise Conciseness, Comp Completeness. κ\kappa is Krippendorff agreement across judges. All methods use Qwen2.5-7B-Instruct. κ\kappa: > 0.66 substantial agreement; 0.60–0.66 moderate; < 0.60 fair.

Residual Error Categories

A manual inspection of PatternBloom (Q2.5-7B) trajectories on the four multi-hop benchmarks groups residual errors into the four categories summarized in Table 18. Three of the four categories (R₂ entity disambiguation, R₃ numerical reasoning, R₄ predicate canonicalization) are generic graph-RAG challenges shared by every baseline in Table 1 and admit standard, method-agnostic engineering fixes (coreference-aware reranking, an external calculator tool, and a predicate-canonicalization layer). The remaining category R₁ is specific to the procedural-memory mechanism: when the query signature has no nearby neighbor in M0\mathcal{M}_0, no pattern hint is injected and the agent reasons from chunks alone. R1R_1 is exactly the operating point that an inference-time pattern-admission extension targets (§I): when no stored skeleton matches above θsim\theta_{\mathrm{sim}}, a fresh signature is admitted into M\mathcal{M} from the current trajectory.

CategorySymptomAddressing extension
R1R_1: signature mismatchσ(q)\sigma(q) similarity to every stored pattern <θsim=0.5< \theta_{\mathrm{sim}}=0.5; no pattern hint is injected and the agent reasons from chunks alone.Inference-time pattern admission (§I).
R2R_2: entity disambiguationRetrieved hyperedge surface form matches the query mention but disambiguates to the wrong real-world entity (e.g., two films sharing a title).Coreference-aware reranking before <query>-block dispatch (generic graph-RAG extension).
R3R_3: numerical reasoningThe retrieved skeleton and traversal are correct, but the arithmetic step (year subtraction, attribute ranking) is wrong.External calculator tool invoked when the retrieved skeleton ends in a numerical comparison (generic LLM extension).
R4R_4: predicate canonicalizationThe retrieved hyperedge contains the right entities but the predicate string at <graph> does not match the gold annotation.Predicate canonicalization layer between extraction and graph emission (generic graph-RAG extension).

Table 18. Residual error categories on the four multi-hop benchmarks. R2R_2–R4R_4 are generic graph-RAG challenges shared by every baseline; only R1R_1 is specific to the procedural-memory mechanism and is addressed by an inference-time pattern-admission extension (§I).

Hyperparameters

Table 19 summarizes all hyperparameters for both training stages. Both stages use GRPO [8, 9] with 5-trajectory groups to compute group-relative advantages. Stage I trains from the SFT backbone πθ0\pi_{\theta_0} with the verl GRPO default βKL=0.001\beta_{\mathrm{KL}} = 0.001 and learning rate 5×10−75 \times10^{-7}, allowing the policy enough flexibility to discover IDR-optimal evidence construction from a fresh reference. Stage II warm-starts from πθ0∗\pi_{\theta_0^*} and applies a 5×5\times KL tightening to βKL=0.005\beta_{\mathrm{KL}} = 0.005 together with a 2.5×2.5\times lower learning rate of 2×10−72 \times10^{-7}, anchoring continued exploration to Stage I’s gains while preventing format-saturating drift on top of an already-well-formed agent trajectory; see §G.16 for the rationale. Stage II trains with λP=0.2\lambda_P = 0.2 pattern-coverage shaping.

ComponentStage I (IDR)Stage II (PAR)
BackboneQwen2.5-7B-Instruct [33]Stage-I best checkpoint
OptimizerAdamWAdamW
Learning rate5×10−75 \times10^{-7}2×10−72 \times10^{-7}
GRPO group size55
Batch size128 prompts128 prompts
Mini-batch size6464
KL coefficient0.0010.005
Gradient clip1.01.0
Rollout temperature1.01.0
Rollout top-pp1.01.0
Format reward weight1.01.0
IDR temperature TT1.0—
Pattern shaping λP\lambda_P—0.2
Merge threshold θmerge\theta_{\mathrm{merge}}—0.85 (frozen GPM)
Retrieval threshold θsim\theta_{\mathrm{sim}}—0.5
Empty-graph fallbackσ(0)=0.5\sigma(0) = 0.5—
Total GRPO steps109109
Save / val frequencyevery 10 stepsend-of-training only

Table 19. Training hyperparameters for both stages.

Stage II disables periodic checkpointing and validation (save_freq=test_freq=-1); the Stage-II best policy πθ2∗\pi_{\theta_2^*} is the final-step (109) checkpoint, evaluated post-hoc on the seven held-out test sets reported in Table 1.

Hyperparameter selection rationale. The two stages use complementary regularization regimes. Stage I uses the verl GRPO default βKL=0.001\beta_{\mathrm{KL}} = 0.001 together with a 5×10−75 \times10^{-7} learning rate: training begins from the SFT backbone πθ0\pi_{\theta_0}, where the reference is loose and a strong KL anchor would suppress the exploration needed to discover the IDR-rewarded behavior of constructing minimum-sufficient evidence graphs. Stage II warm-starts from πθ1∗\pi_{\theta_1^*}, which already produces well-formed agent trajectories with non-trivial IDR; here a loose KL anchor would let policy gradients drift toward format-saturating short-cut solutions on top of those gains, so we apply a 5×5\times KL tightening to βKL=0.005\beta_{\mathrm{KL}} = 0.005 together with a 2.5×2.5\times lower learning rate of 2×10−72 \times10^{-7}. This combination keeps Stage II refinement anchored to Stage I’s behavior while still permitting structured improvement under RPAR\mathrm{R}_{\mathrm{PAR}}.

Compute Budget

All training was performed on 4×4\times NVIDIA H200 GPUs with FSDP for actor and reference policies and tensor-parallel rollout via vLLM. Each stage runs 109 GRPO steps with periodic validation on a held-out 1,000-query slice. The frozen Stage-I oracle (Qwen2.5-1.5B-Instruct, 5 GB resident) is co-located with the trainer and is not invoked at inference, so deployment carries no oracle overhead. The hypergraph retriever (BGE-large-en-v1.5 + FAISS) runs as a co-located CPU service whose latency is hidden inside vLLM rollout.

Stage Transition Protocol

The transition from Stage I to Stage II has three steps:

  1. Stage-I best-checkpoint selection. Among the saved Stage-I checkpoints (every 10 steps from step 10 to step 109, with best_ckpt_max_keep=2 top by val score), we select the checkpoint maximizing (EM + F1)/2 on the held-out 1,000-query 2WikiMultiHopQA validation slice. Validation is run during training with the same retriever and inference setup as Stage II to avoid distribution shift at hand-off.

  1. Pattern distillation. The selected best checkpoint generates trajectories on the union of training prompts; high-quality trajectories (RIDR≥0.85R_{\mathrm{IDR}} \ge0.85) are extracted and merged into a frozen M0\mathcal{M}_0 via the procedure in §C.5. We additionally log F1 and RIDRR_{\mathrm{IDR}} per pattern for downstream diagnostic use.

  1. Stage-II warm-start. Stage II starts from the same Stage-I best checkpoint as the policy initial state, with M0\mathcal{M}_0 loaded as read-only memory. The first Stage-II step inherits the policy’s Stage-I-end behavior, ensuring continuity.

Prompts

We collect the three prompts that materially shape PatternBloom: the oracle scoring prompt for IDR (Stage I training), the agent system prompt at inference, and the LLM-judge rubric used for the six-axis quality ensemble in Table 17. Each prompt is reproduced verbatim from our codebase; reformatting characters (curly braces, angle brackets, hash marks) appear exactly as the model sees them. The three boxes are color-coded by deployment phase: blue for training, green for inference, plum for evaluation, so a reader can quickly locate the relevant prompt by phase.

IDR Oracle Scoring Prompt

The Information-Density Reward computes a per-graph oracle log-probability gain Δlog⁡pO(a∗∣q,G)\Delta\log p_{\mathcal{O}}(a^* \mid q,G) on the gold answer. The oracle (Qwen2.5-1.5B-Instruct, served via vLLM) sees the fixed-template prefix below; the template is identical across training and inference, since shifting it would invalidate the cached log-probability baselines used for the moving-average normalizer.

Stage I IDR Oracle Scoring Prefix | training-time

Question: {question}
Context: {serialised_graph}
Answer:

The serialized graph renders each retrieved triple as head | relation | tail on its own line; the empty graph renders as the literal token (no context) so that the empty-graph baseline is well-defined. We score only the gold-answer tokens (regardless of prefix length) and average the per-token log-probabilities over the gold positions, yielding a single per-trajectory IDR signal. The trajectory IDR is then normalized by the moving average of the empty-graph baseline (Appendix § 8.2).

Agent System Prompt

The agent’s system prompt documents the three permitted actions (<query>, <graph>, <answer>) and embeds the retrieved graph patterns whenever the query signature matches a stored skeleton in M\mathcal{M} above the dispatch threshold θsim=0.5\theta_{\mathrm{sim}} = 0.5.

Inference-Time Agent System Prompt | PatternBloom

You can query knowledge about a specific entity:
<query>entity_name</query>

After receiving knowledge, extract evidence triples and continue
searching:
<graph>EntityA | relation | EntityB</graph>
<query>next_entity</query>

When you have enough evidence, extract final triples and provide
the answer:
<graph>EntityA | relation | EntityB</graph>
<answer>your answer</answer>

# Retrieved reasoning patterns (prior knowledge):
## Pattern 1 (sim=...)
Signature: ...
Path skeleton variant 1/3:
    (TYPE_A, relation, TYPE_B)
...

## Pattern 2 (sim=...)
...

When no pattern in M\mathcal{M} exceeds θsim\theta_{\mathrm{sim}} at dispatch time, the “# Retrieved reasoning patterns” block is omitted (cold-start fallback, Appendix § 10.§ 1). Table 10, Table 11, and Table 12 show this prompt instantiated on specific queries.

LLM-Judge Six-Axis Rubric

Table 17 reports six-axis quality scores averaged over three judges (Sonnet 4.6, Haiku 4.5, Gemini 2.5 Flash) on 200 random queries drawn from the seven evaluation datasets. Each judge receives the rubric below; the response is parsed as a strict JSON object with one Likert score per axis.

Six-Axis Quality Rubric | LLM-judge ensemble

You are evaluating the REASONING PROCESS of a multi-hop QA system,
not just its final answer text.

Question: {question}
Gold Answer: {gold}
Retrieved Context (truncated): {context}
Method's Full Reasoning Trajectory:
{prediction}

Evaluate on 6 dimensions (1-5 Likert, 5=excellent):
1. logical_coherence -- Does the reasoning chain follow valid
   deductive steps from premises to conclusion?
2. faithfulness -- Does every reasoning claim ground in the
   retrieved context (no fabrication)?
3. reasoning_clarity -- Is the reasoning trace easy to follow
   with explicit logical structure?
4. pattern_reusability -- Does the reasoning use a structural,
   type-abstracted template that generalizes to similar queries
   with different entities?
5. conciseness -- Is the response appropriately brief, with no
   padding or redundant restatement?
6. completeness -- Does the reasoning fully address all logical
   sub-steps the question requires?

Do NOT reward responses for length or verbosity. Reward explicit
graph/triple structures over freeform prose with the same
information.

Output ONLY valid JSON:
{"logical_coherence": ...,
 "faithfulness": ...,
 "reasoning_clarity": ...,
 "pattern_reusability": ...,
 "conciseness": ...,
 "completeness": ...}

The same prompt is sent to all three judges; for the Anthropic models we omit the OpenAI-specific JSON-mode parameter (response_format={"type":"json_object"}) and rely on instruction-following to produce a parseable JSON object. Judge outputs that fail JSON parsing are dropped (these account for <1%<1\% of judge calls in our run, none concentrated on PatternBloom). The reported number per axis is the mean over the three judges and over the 200 queries.

Discussion

Why Externalizing Graph Patterns Helps

Two observations from §4 connect the empirical gains to the paradigm shift proposed in § 1. First, the IDR reward at Stage I aligns the policy gradient with the conditional information gain I(G;a∣q)I(G; a \mid q) rather than surface-form token overlap (Theorem 1), which prevents the verbose-but-correct graph reward-hacking mode that token-level F1 admits. Second, the Graph Pattern Memory at Stage II separates the procedural knowledge of which reasoning structure to apply from the declarative knowledge of which facts to retrieve. Since the same type-abstracted skeleton (e.g., [FILM] →directed_by\xrightarrow{\mathrm{directed\_by}} [PERSON] →nationality\xrightarrow{\mathrm{nationality}} [COUNTRY]) applies across thousands of film-director-country instances, pattern retrieval compounds with reasoning depth where content retrieval saturates, evidenced by the +10.4 EM gain on Bamboogle (4-hop) relative to HotpotQA (2-hop).

Limitations

On the empirical side, the procedural-memory mechanism transfers at the fixed Wikipedia-hyperedge substrate. Cross-domain transfer to non-Wikipedia corpora such as biomedical, legal, or financial documents lies outside the scope of the present paper and is the natural deployment-scale next benchmark.

Broader Impact

Positive. PatternBloom reduces sample complexity for retrieval-augmented reasoning agents (Stage I IDR converges on 14K queries) and produces an inspectable pattern library that practitioners can audit, prune, or extend. The pattern library is human-readable (typed triples), unlike opaque parametric memory in end-to-end models.

Negative. A poorly curated GPM, for example one that absorbed adversarial trajectories without basic format and quality filters, could systematically mislead downstream users. Production systems where GPM accumulates over weeks would benefit from an LLM-judge audit before admitting new skeletons.

Future Work

Cross-domain transfer. Type abstraction makes patterns domain-agnostic in principle, but our experiments cover only Wikipedia-grounded multi-hop QA. Whether MM trained on Wikipedia transfers zero-shot to medical (PubMedQA), legal, or financial QA without retraining is the natural next benchmark.

Extended Related Work

The compact § 2 introduces two adjacent research lines and our position. This appendix expands that discussion: §J.1 catalogs structured-RAG methods along five comparison axes; §J.2 catalogs RL-for-RAG methods along their reward designs; and §J.2 pins down where PatternBloom sits in the resulting design space.

Structured RAG: Method-by-Method Comparison

We organize structured-RAG methods along five axes: (i) the underlying retrieval representation; (ii) when the structure is built (build-time); (iii) what the agent receives at retrieval (retrieval unit); (iv) whether a reasoning path is exposed; and (v) how the structure is supervised (construction supervision). Table 20 summarizes the comparison.

MethodRepresentationBuild-timeRetrieval unitReasoning path
GraphRAG [11]entity-relation KGstatic (LLM)community summaryimplicit
HippoRAG [12, 13]entity-relation KGstatic (LLM)PPR-traversed nodesimplicit
LightRAG [15]entity-relation KGstatic (LLM)dual-level summaryimplicit
LinearRAG [14]hierarchical entitystatic (NER)relation-free chunksimplicit
GFM-RAG [16]query-conditioned KGstatic (foundation)GFM-scored entitiesimplicit
HyperGraphRAG [24]hypergraphstatic (LLM)hyperedgeimplicit
StructRAG [17]adaptive structuredynamic (prompt)table/list/treeimplicit
Graph-R1 [6]hypergraphstatic (LLM)hyperedgeimplicit
Structure-R1 [18]adaptive structuredynamic (RL)format-conditionalimplicit
PatternBloom (ours)graph patternsdynamic (RL)type-skeletonsexplicit (skel)

Table 20. Structured-RAG methods along five axes. Static = built once and frozen; Dynamic = constructed per query or evolved at training/inference. PatternBloom is the only method whose retrieval unit is a typed reasoning skeleton with an explicit directed path through the evidence rather than a content fragment.

Construction supervision. Structure construction in prior work is supervised either by frozen-LLM heuristics (community detection, entity extraction prompts) or by GNN pre-training objectives (link prediction, node classification). None of these objectives directly optimizes “how useful is this structure for downstream multi-hop QA”. PatternBloom’s Information-Density Reward (§3.2) provides exactly this signal: the policy is rewarded for evidence graphs whose conditional information gain on the gold answer is high under a frozen oracle. Theorem 1 and Appendix § 11 ground this reward in size-normalized conditional mutual information.

Retrieval unit and reasoning path. Across the structured-RAG line, the retrieval unit has steadily widened from a single passage to a community summary, a hyperedge, or an adaptive structure. In every case, however, the unit returned to the agent encodes specific entity content. The agent must still infer how to traverse those fragments to reach an answer; the reasoning path lives only in the language model’s hidden state and dissolves after the query. PatternBloom’s graph patterns are typed reasoning skeletons such as (FILM,directed_by,PERSON)→(PERSON,born_in,COUNTRY)(\mathrm{FILM},\mathrm{directed\_by},\mathrm{PERSON}) \rightarrow(\mathrm{PERSON},\mathrm{born\_in},\mathrm{COUNTRY}) that prescribe a directed traversal explicitly and are reusable across all instances of the same query type.

Reinforcement Learning for Agentic RAG: Reward Designs

We compare RL-for-RAG methods along three axes: their reward signal (what the policy is optimized for), their retrieval target (where the policy retrieves from), and their training-data scale. Table 21 summarizes.

MethodReward signalRetrieval targetTraining scale
Search-R1 [5]EM on answerWikipedia (E5 retriever)∼\sim170k examples
R1-Searcher [7]multi-stage curriculum, EM/F1search engine∼\sim8k
FrugalRAG [10]EM + budget penaltysearch engine1,000 examples
Graph-R1 [6]F1 + entity-presence proxyhypergraph index∼\sim14k
Structure-R1 [18]EM + format-validity proxyformat-conditional∼\sim14k
PatternBloom (ours)IDR (info-theoretic) + PAR (pattern coverage)hypergraph + GPM∼\sim14k (HotpotQA + 2Wiki)

Table 21. RL-for-RAG methods compared on reward design and retrieval target. Prior methods use surface-form (token-level) signals on the final answer with at most lightweight proxies on the constructed graph; PatternBloom introduces a process-level reward that interrogates the constructed evidence graph itself (Stage I) and aligns the trajectory with retrieved reasoning patterns (Stage II).

Reward formulation. Search-R1 and R1-Searcher use answer-only rewards, treating retrieval as a black-box action whose only feedback is downstream answer correctness. FrugalRAG, building on declarative LM compilation [49], augments this with a budget term that penalizes excessive search hops, addressing efficiency rather than evidence quality. Graph-R1’s entity-presence proxy is the closest prior attempt to interrogate the constructed graph (it checks whether gold-answer entities appear in the graph), but the check is binary and grants the same reward to graphs that contain the gold entity buried among 20 other triples versus graphs that contain it among 3 relevant triples. PatternBloom’s IDR (Eq. (3)) instead measures how much the constructed graph raises a frozen oracle’s likelihood of the gold answer per triple, providing a continuous, size-normalized signal that scales with reasoning depth.

Memory across episodes. All five RL-for-RAG methods are memory-less across episodes: each query’s trajectory is generated fresh and dissolves at episode end, leaving no state that persists into the next query. Successful reasoning patterns are absorbed only into the policy’s parameters via gradient updates, where they are not directly inspectable, editable, or transferable. PatternBloom’s Graph Pattern Memory (§ 3.3) externalizes this procedural knowledge into a separate, human-readable, query-signature-indexed store that can be inspected, edited, and frozen for reproducibility, complementing trajectory-level techniques such as self-consistency [50] and the lifelong skill libraries explored by embodied LLM agents [51].

NeurIPS Paper Checklist

  1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Answer: [Yes]

Justification: The main claims in the abstract and § 1 are supported by the experiments in Section 4 and the theoretical result in Appendix § 11.

Guidelines:

  • The answer [N/A] means that the abstract and introduction do not include the claims made in the paper.

  • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A [No] or [N/A] answer to this question will not be perceived well by the reviewers.

  • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

  • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

  1. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Limitations are discussed in Appendix § 11.4.§ 2.

Guidelines:

  • The answer [N/A] means that the paper has no limitation while the answer [No] means that the paper has limitations, but those are not discussed in the paper.

  • The authors are encouraged to create a separate “Limitations” section in their paper.

  • The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

  • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

  • The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

  • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

  • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

  • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren’t acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

  1. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes] Justification: Theorem 1 is stated and proved with full assumptions in Appendix § 11.

Guidelines:

  • The answer [N/A] means that the paper does not include theoretical results.

  • All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.

  • All assumptions should be clearly stated or referenced in the statement of any theorems.

  • The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

  • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

  • Theorems and Lemmas that the proof relies upon should be properly referenced.

Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Implementation details, hyperparameters, and prompts are documented in Appendices § 8.2–§ 10, § 11.2, and § 11.3.

Guidelines:

  • The answer [N/A] means that the paper does not include experiments.

  • If the paper includes experiments, a [No] answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

  • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

  • Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

  • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

    (a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

    (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

    (c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

    (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [Yes]

Justification: Code, checkpoints, and the learned pattern memory will be released.

Guidelines:

  • The answer [N/A] means that paper does not include experiments requiring code.

  • Please see the NeurIPS code and data submission guidelines (https://neurips.cc/public/guides/CodeSubmissionPolicy) for more details.

  • While we encourage the release of code and data, we understand that this might not be possible, so [No] is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

  • The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https://neurips.cc/public/guides/CodeSubmissionPolicy) for more details.

  • The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

  • The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

  • At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

  • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer) necessary to understand the results?

Answer: [Yes]

Justification: Experimental settings are reported in § 4.1 and Appendix § 11.2.§ 14.4.

Guidelines:

  • The answer [N/A] means that the paper does not include experiments.

  • The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

  • The full details can be provided either with the code, in appendix, or as supplemental material.

Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [N/A]

Justification: The experiments are conducted with a fixed random seed and no error bars are reported.

Guidelines:

  • The answer [N/A] means that the paper does not include experiments.

  • The authors should answer [Yes] if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

  • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

  • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

  • The assumptions made should be given (e.g., Normally distributed errors).

  • It should be clear whether the error bar is the standard deviation or the standard error of the mean.

  • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

  • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g., negative error rates).

  • If error bars are reported in tables or plots, the authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Compute resources are reported in Appendix G.17.

Guidelines:

  • The answer [N/A] means that the paper does not include experiments.

  • The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

  • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

  • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn’t make it into the paper).

Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?

Answer: [Yes]

Justification: The research conducted in the paper complies with the NeurIPS Code of Ethics.

Guidelines:

  • The answer [N/A] means that the authors have not reviewed the NeurIPS Code of Ethics.

  • If the authors answer [No], they should explain the special circumstances that require a deviation from the Code of Ethics.

  • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: Broader impacts are discussed in Appendix I.3.

Guidelines:

  • The answer [N/A] means that there is no societal impact of the work performed.

  • If the authors answer [N/A] or [No], they should explain why their work has no societal impact or why the paper does not address societal impact.

  • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

  • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate Deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

  • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

  • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pre-trained language models, image generators, or scraped datasets)?

Answer: [N/A]

Justification: The paper utilizes existing datasets and pretrained models that are already released with safeguards in place.

Guidelines:

  • The answer [N/A] means that the paper poses no such risks.

  • Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

  • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

  • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We have properly cited all code, data, and models used in our research and complied with the licensing agreements and terms of use set by the original authors.

Guidelines:

  • The answer [N/A] means that the paper does not use existing assets.

  • The authors should cite the original paper that produced the code package or dataset.

  • The authors should state which version of the asset is used and, if possible, include a URL.

  • The name of the license (e.g., CC-BY 4.0) should be included for each asset.

  • For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

  • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

  • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

  • If this information is not available online, the authors are encouraged to reach out to the asset’s creators.

New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [N/A]

Justification: There are no new assets introduced in the paper.

Guidelines:

  • The answer [N/A] means that the paper does not release new assets.

  • Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

  • The paper should discuss whether and how consent was obtained from people whose asset is used.

  • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [Yes]

Justification: Annotator instructions and the rating rubric are reproduced in Appendix § 11.2.§ 12.6.

Guidelines:

  • The answer [N/A] means that the paper does not involve crowdsourcing nor research with human subjects.

  • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

  • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [N/A]

Justification: The research does not study human subjects; humans only participate as evaluators of model output, so IRB review is not required.

Guidelines:

  • The answer [N/A] means that the paper does not involve crowdsourcing nor research with human subjects.

  • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

  • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

  • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigor, or originality of the research, declaration is not required.

Answer: [Yes]

Justification: The usage of LLMs is described in § 3, § 4.1, and Appendix § 11.2.§ 14.2.

Guidelines:

  • The answer [N/A] means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

  • Please refer to our LLM policy in the NeurIPS handbook for what should or should not be described.

References

  1. [1]Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive NLP tasks, 2021.arxiv.org/abs/2005.11401
  2. [2]Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. HotpotQA: A dataset for diverse, explainable multi-hop question answering, 2018.arxiv.org/abs/1809.09600
  3. [3]Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps, 2020.
  4. [4]Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. MuSiQue: Multihop questions via single-hop question composition, 2022.arxiv.org/abs/2108.00573
  5. [5]Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-R1: Training LLMs to reason and leverage search engines with reinforcement learning, 2025.arxiv.org/abs/2503.09516
  6. [6]Haoran Luo, Haihong E, Guanting Chen, Qika Lin, Yikai Guo, Fangzhi Xu, Zemin Kuang, Meina Song, Xiaobao Wu, Yifan Zhu, and Luu Anh Tuan. Graph-R1: Towards agentic GraphRAG framework via end-to-end reinforcement learning, 2025.arxiv.org/abs/2507.21892
  7. [7]Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen. R1-Searcher: Incentivizing the search capability in LLMs via reinforcement learning, 2025.arxiv.org/abs/2503.05592
  8. [8]DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jinchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang, Liyue Zhang, Lei Xu, LeyI Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang, Qinyu Chen, Quishi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. J. Chen, R. L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Shengfeng Ye, Tao Yun, Tian Pei, Tianyu Sun, T. Wang, Wandging Zeng, Wanjia Zhao, Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang, W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xiaosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song, Xinyi Zhou, Xianzhu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang, Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yuan Ou, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma, Ying Tang, Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zhuiqi Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning, 2026.DOI
  9. [9]Zhonghong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models, 2024.arxiv.org/abs/2402.03300
  10. [10]Abhinav Jaya, Srivathsan Koundinyan, Nagarajan Natarajan, and Amit Sharma. FrugalRAG: Less is more in RL finetuning for multi-hop question answering, 2025.
  11. [11]Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, and Jonathan Larson. From local to global: A graph RAG approach to query-focused summarization, 2025.arxiv.org/abs/2404.16130
  12. [12]Bernal Jiménez Gutiérrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, and Yu Su. HippoRAG: Neurobiologically inspired long-term memory for large language models, 2025.arxiv.org/abs/2405.14831
  13. [13]Bernal Jiménez Gutiérrez, Yiheng Shu, Weijian Qi, Sizhe Zhou, and Yu Su. From RAG to memory: Non-parametric continual learning for large language models, 2025.arxiv.org/abs/2502.14802
  14. [14]Luyao Zhuang, Shengyuan Chen, Yilin Xiao, Huachi Zhou, Yujing Zhang, Hao Chen, Qinggang Zhang, and Xiao Huang. LinearRAG: Linear graph retrieval augmented generation on large-scale corpora, 2025.arxiv.org/abs/2510.10114
  15. [15]Zirui Guo, Lianghao Xia, Yanhua Yu, Tu Ao, and Chao Huang. LightRAG: Simple and fast retrieval-augmented generation, 2025.arxiv.org/abs/2410.05779
  16. [16]Linhao Luo, Zicheng Zhao, Gholamreza Haffari, Dinh Phung, Chen Gong, and Shirui Pan. GFM-RAG: Graph foundation model for retrieval augmented generation, 2025.arxiv.org/abs/2502.01113
  17. [17]Zhuoqun Li, Xuanang Chen, Haiyang Yu, Hongyu Lin, Yaojie Lu, Qiaoyu Tang, Fei Huang, Xianpei Han, Le Sun, and Yongbin Li. StructRAG: Boosting knowledge intensive reasoning of LLMs via inference-time hybrid information structuration, 2024.arxiv.org/abs/2410.08815
  18. [18]Junlin Wu, Xianrui Zhong, Jiashuo Sun, Bolian Li, Bowen Jin, Jiawei Han, and Qingkai Zeng. Structure-R1: Dynamically leveraging structural knowledge in LLM reasoning through reinforcement learning, 2025.arxiv.org/abs/2510.15191
  19. [19]Larry R. Squire. Memory and the hippocampus: A synthesis from findings with rats, monkeys, and humans. Psychological Review, 99(2):195–231, 1992. doi: 10.1037/0033-295X.99.2.195.
  20. [20]Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. In EMNLP, 2020.arxiv.org/abs/2004.04906
  21. [21]Omar Khattab and Matei Zaharia. ColBERT: Efficient and effective passage search via contextualized late interaction over BERT. In SIGIR, 2020.arxiv.org/abs/2004.12832
  22. [22]Gautier Izacard and Edouard Grave. Leveraging passage retrieval with generative models for open domain question answering, 2021.arxiv.org/abs/2007.01282
  23. [23]Stephen Robertson and Hugo Zaragoza. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends in Information Retrieval, 3(4):333–389, 2009.
  24. [24]Haoran Luo, Haihong E, Guanting Chen, Yandan Zheng, Xiaobao Wu, Yikai Guo, Qika Lin, Yu Feng, Zenmin Kuang, Meina Song, Yifan Zhu, and Luu Anh Tuan. HyperGraphRAG: Retrieval-augmented generation via hypergraph-structured knowledge representation, 2025.arxiv.org/abs/2503.21322
  25. [25]John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms, 2017.arxiv.org/abs/1707.06347
  26. [26]Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model, 2023.arxiv.org/abs/2305.18290
  27. [27]Mingyang Chen, Tianpeng Li, Haoze Sun, Yijie Zhou, Chenzheng Zhu, Haofen Wang, Jeff Z. Pan, Wen Zhang, Huajun Chen, Fan Yang, Zenan Zhou, and Weipeng Chen. ReSearch: Learning to reason with search for LLMs via reinforcement learning, 2025.arxiv.org/abs/2503.19470
  28. [28]Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions, 2023.arxiv.org/abs/2212.10509
  29. [29]Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models, 2023.arxiv.org/abs/2210.03629
  30. [30]Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. Self-RAG: Learning to retrieve, generate, and critique through self-reflection, 2023.arxiv.org/abs/2310.11511
  31. [31]Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools, 2023.arxiv.org/abs/2302.04761
  32. [32]Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models, 2023.arxiv.org/abs/2201.11903
  33. [33]An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Daiyiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jiahnong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report, 2025.arxiv.org/abs/2412.15115
  34. [34]Naftali Tishby, Fernando C. Pereira, and William Bialek. The information bottleneck method. In Annual Allerton Conference on Communication, Control, and Computing, 1999.
  35. [35]Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle. In 2015 IEEE Information Theory Workshop (ITW), pages 1–5, 2015. doi: 10.1109/ITW.2015.7133169.DOI
  36. [36]Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with GPUs, 2017.arxiv.org/abs/1702.08734
  37. [37]Andrew Y. Ng, Daishi Harada, and Stuart Russell. Policy invariance under reward transformations: theory and application to reward shaping. In Proceedings of the Sixteenth International Conference on Machine Learning (ICML), pages 278–287, 1999.
  38. [38]Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. Natural questions: A benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:453–466, 2019. doi: 10.1162/tacl_a_00276.
  39. [39]Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension, 2017.
  40. [40]Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hanne­h Ha­jishirzi. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories, 2023.
  41. [41]Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A. Smith, and Mike Lewis. Measuring and narrowing the compositionality gap in language models, 2023.
  42. [42]An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengeng Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report, 2025.arxiv.org/abs/2505.09388
  43. [43]Ran Xu, Yuchen Zhuang, Zihan Dong, Jonathan Wang, Yue Yu, Joyce C. Ho, Linjun Zhang, Haoyu Wang, Wenqi Shi, and Carl Yang. AceSearcher: Bootstrapping reasoning and search for LLMs via reinforced self-play, 2025. NeurIPS 2025 Spotlight.
  44. [44]Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff, Defu Lian, and Jian-Yun Nie. C-Pack: Packed resources for general Chinese embeddings, 2024.
  45. [45]Jiajie Jin, Yutao Zhu, Xinyu Yang, Chenghao Zhang, and Zhicheng Dou. FlashRAG: A modular toolkit for efficient retrieval-augmented generation research, 2024.
  46. [46]OpenAI. GPT-4 technical report, 2023.arxiv.org/abs/2303.08774
  47. [47]Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. Search-o1: Agentic search-enhanced large reasoning models, 2025.
  48. [48]Meta AI. The Llama 4 herd: The beginning of a new era of natively multimodal AI innovation. https://ai.meta.com/blog/llama-4-multimodal-intelligence/, 2025. Llama-4 Maverick (17B×128 MoE) and Llama-4 Scout model release.
  49. [49]Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. DSPy: Compiling declarative language model calls into self-improving pipelines, 2023.
  50. [50]Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models, 2022.
  51. [51]Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models, 2023.

Paper details

vlnc:W_2mqwpvpb
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents