betaFeedback

WorksComputer Science

LIBRA: Programmatic Induction of Executable Tool Libraries

Abstract

Large language model agents increasingly retrieve tools from large catalogs, yet most retrievers rank each tool independently by request similarity. For multi tool tasks, this recovers the goal matching tool while omitting prerequisite tools required by the workflow. We introduce LIBRA, a framework for Library Induction by Runtime Assembly that turns a flat tool catalog into an executable tool library. An LLM proposes per tool contracts of candidate invocations, reusable outputs and state changes, and success conditions; LIBRA then tests these compositions in a sandbox and retains the relations supported at runtime. Constructed before user queries, the library requires no dependency annotations, tool use trajectories, or task specific training, and supports both one-shot and agentic retrieval. On AppWorld, BFCL, and SciToolEval, LIBRA improves one-shot Recall and NDCG over the same dense retriever by 7.9 and 3.7 percentage points on average. Within ToolReAGt, it yields corresponding gains of 4.5 and 2.9 points after ten retrieval iterations.

Introduction

Large language models (LLMs) increasingly act as agents that accomplish tasks by calling external tools and APIs [4] (Wang et al. 2025). As tool catalogs grow, exposing every available tool to the LLM becomes costly and unreliable: the catalog may exceed the context window, while irrelevant tools increase inference cost and degrade tool selection. Practical systems therefore retrieve a small candidate set for each user request [3] (Zheng et al. 2024).

Most tool retrievers, however, represent and rank tools independently. This abstraction is insufficient for tasks that require multiple tools to execute in sequence. A user typically states the desired outcome rather than the complete operational procedure, leaving prerequisites such as authentication, identifier lookup, state initialization, and format conversion absent from the request. Consequently, a retriever may identify the tool that directly matches the stated goal while omitting upstream tools whose outputs or state are required for execution. Semantic relevance alone therefore does not guarantee a retrieval set that contains the prerequisite tools required by a valid workflow.

Existing dependency aware approaches obtain tool relations from supplied hierarchies, metadata based inference, trained dependency models, or historical tool use trajectories [13, 9, 15, 4] (Zheng et al. 2024). Although these approaches can improve retrieval, they do not fully address a new executable catalog for which dependency annotations, tool-use demonstrations, and usage trajectories are unavailable. As summarized in Table 1, methods that learn tool structure either depend on existing tool use trajectories or training, or accept metadata inferred relations without testing whether the tools actually compose. This distinction matters because a relation can appear plausible from tool metadata yet fail in the target runtime. We therefore investigate the following question: How can we construct reusable knowledge of tool composition before user queries or usage histories become available?

MethodTrajectory freeTraining freeExecution validation
ToolNet ([13])✗✓✗
TGR ([9])✓✗✗
GTool ([4])✗✗✗
Ego-Trip ([1])✓✓✗
LIBRA✓✓✓

Table 1. Comparison of methods for learning tool dependencies. Prior methods either depend on existing tool use trajectories or training, or retain metadata inferred relations without runtime validation. LIBRA performs trajectory free, training free dependency induction and validates candidate tool compositions through execution in the target environment.

We propose LIBRA, a framework for Library Induction by Runtime Assembly.1 We distinguish a tool catalog, which contains callable interfaces and public metadata, from an executable tool library, which augments the catalog with per tool executable contracts and runtime supported composition relations. Each contract specifies candidate invocations, reusable outputs and state changes, and observable success conditions. LIBRA executes type compatible compositions in a sandbox and can retain input provenance relations, compatibility relations, and state dependencies supported by constructing counterfactual tests. Each test checks not merely that a composition succeeds, but that it fails without the candidate dependency; this yields relations that are more precise than metadata inferred structure (Appendix § 15). The resulting query independent library augments tool retrieval with reusable composition knowledge, but does not generate or execute task specific plans. We therefore evaluate its ability to recover tools required under each benchmark’s evaluation protocol.

The executable tool library is constructed ahead of user queries and requires neither existing tool use trajectories nor task specific training. At retrieval time, a dense base retriever identifies semantically relevant seed tools, while the library promotes their prerequisite tools, which need not be mentioned in the request. Because the library is query independent, the same artifact can augment both one-shot retrievers and agentic retrievers that search iteratively. LIBRA thus learns how tools compose once and exposes that knowledge to subsequent retrieval workflows.

We evaluate LIBRA on AppWorld, BFCL, and SciToolEval against lexical, dense, representation augmented, graph based, and agentic retrieval baselines. Relative to the same dense retriever, LIBRA improves one-shot Recall and NDCG by 7.9 and 3.7 percentage points on average across the three benchmarks. When integrated into ToolReAGt, LIBRA improves Recall and NDCG by 4.5 and 2.9 percentage points, averaged across three benchmarks, three agent LLMs, and three random seeds. These results demonstrate that an execution supported tool library provides reusable retrieval guidance beyond semantic similarity alone.

Our contributions are as follows:

  • We introduce LIBRA, which uses LLM proposed executable contracts and a counterfactual sandbox execution test, retaining a relation only when the provider value succeeds while an independently constructed control fails, to construct a reusable tool library of runtime supported composition relations without dependency annotations, existing tool use trajectories, or task specific training.

  • We demonstrate that the induced tool library improves both one-shot and agentic retrieval across three tool use benchmarks and three agent LLMs.

  • We present case studies of prerequisite tools that LIBRA recovers despite their low semantic similarity to the request, together with a breakdown of the remaining causes of failed dependency recovery.

Related Work

Tool representations and structural retrieval. The dominant approach to large tool catalogs represents and ranks tools independently. Sparse retrievers instantiate this approach with lexical matching (Robertson and Zaragoza, 2009; [19]), while dense retrievers use embedding similarity. Representation augmentation methods enrich either the query or each tool’s description: PLUTO decomposes requests and edits tool descriptions ([11]); ReInvoke generates synthetic indexing queries and rewrites invocation intent at inference time ([5]); and ToolREx constructs structured tool profiles ([14]). Parametric methods instead internalize tool identity: ToolGen assigns a learned token to each tool (Wang et al., 2025), whereas Tool-Weaver learns hierarchical relationships among tools from semantics and existing tool use trajectories ([7]).

Figure 1. Overview of LIBRA. Stage 1: Before user queries arrive, an LLM proposes an executable contract of each tool. LIBRA tests each candidate composition in the target runtime. Successful compositions are retained to construct the executable tool library, while unsuccessful ones are removed. For example, although card and trans_id share the same integer type, a payment card identifier cannot be used as a transaction identifier. Stage 2: Given the request “Find Alice in my contact and I will make the transfer,” dense retrieval returns Venmo.Search_users. LIBRA performs one-hop propagation and recovers its prerequisite tool Venmo.Login.

Structure aware approaches differ along two axes: whether they support retrieval without target domain usage traces and whether structure induction requires training. ToolRerank relies on ToolBench’s existing API hierarchy and trains a retriever, reranker, and query type classifier (Zheng et al., 2024). TGR constructs a new catalog graph from tool metadata using a dependency discriminator trained on synthetic and manually labeled tool pairs ([9]). GTool derives edges from historical tool use trajectories and trains a GNN for missing-edge prediction and tool planning ([4]). Ego-Trip supports deployment without target domain usage traces or parameter training, but infers relations solely from tool metadata through LLM prompting ([1]).

LIBRA requires neither prior trajectories nor task specific training, as it induces composition relations from successful executions in the target runtime, retaining a relation only when a counter-factual test confirms that the composition succeeds with the candidate dependency and fails without it. The resulting reusable executable library improves both one-shot and agentic retrieval. Table 1 summarizes these differences.

Agentic retrieval. A second family integrates tool retrieval into agentic workflows. Iterative Feedback repeatedly asks the tool using LLM to refine the retriever’s view of the query and candidates (Xu et al., 2024). ToolReaGt uses ReAct prompting to decompose requests and retrieve relevant tools over multiple rounds ([3]; Yao et al., 2023). ToolQP trains a query planner on synthetic trajectories using supervised learning and reinforcement learning with verifiable rewards to decompose requests and perform retrieval ([8]). DTDR estimates next tool structure from existing trajectories and demonstrations to condition retrieval on both the request and tool call history ([15]). These methods can recover tools omitted by one-shot retrieval, but perform adaptive reasoning separately for each request; ToolQP and DTDR additionally depend on training or prior tool use histories. LIBRA instead performs training free, ahead-of-time structural augmentation, converting a flat catalog into a reusable executable tool library containing compositional relations before queries arrive. We further show that the resulting library improves agentic retrieval.

Method

Given a tool catalog T={t1,…,tn}\mathcal{T} = \{t_1,\ldots,t_n\} and its runtime, LIBRA constructs an executable tool library

L=(T,C,E),(1)\mathcal{L} = (\mathcal{T},\mathcal{C},\mathcal{E}), \tag*{(1)}

where C={ci}i=1n\mathcal{C} = \{c_i\}_{i=1}^{n} contains an executable contract for each tool, and E\mathcal{E} contains the relations retained by the composition tests below. Here, runtime-validated means that a relation passes its specified test under the tested contract and sandbox state; it is not a universal guarantee over all possible inputs or states. For an edge tp→tct_p \rightarrow t_c, tpt_p is the provider, whose execution produces a value or establishes a state, and tct_c is the consumer, whose execution uses that value or state. At retrieval time, only original tool metadata will be embedded to provide representations, while the relations admitted to the benchmark specific retrieval graph support propagation. Algorithms 1 and 2 in Appendix H provide pseudocode for library construction and one-hop graph propagation, respectively.

Ahead-of-Time Library Construction

Executable Contracts Generation. LIBRA processes each tool independently using only its public description, signature, and input and output schema. An LLM produces a contract

ci=(Πi,Xi,Si+,Si−),(2)c_i = (\Pi_i, \mathcal{X}_i, S_i^{+}, S_i^{-}), \tag*{(2)}

where Πi\Pi_i is a set of candidate calls, Xi\mathcal{X}_i specifies reusable values from the call arguments, returned response, or runtime state changes, and Si+S_i^{+} and Si−S_i^{-} describe observable success and failure outcomes. Each candidate call π∈Πi\pi\in\Pi_i specifies how every input parameter is filled and whether the call is expected to succeed or fail. A parameter may receive a proposed literal, obtain a value from an earlier tool call, or be omitted when it is optional.

LIBRA executes the candidate calls in each contract in isolated sandbox states. If an observed result matches neither outcome specification, the LLM receives the execution feedback and revises the contract. For each tool, LIBRA makes one initial contract-generation call and at most two feedback-based revision calls. Thus, for a catalog of nn tools, the total number of LLM calls is linear in nn.

Runtime Relation Induction. LIBRA tests whether an observed value produced by a provider tool tpt_p can be used as an input to a consumer tool tct_c. Each relation is admitted by a counterfactual test rather than a single success check, which separates genuine dependencies from merely schema compatible pairs. LIBRA first executes tpt_p, extracts an output declared in the contract, and supplies that value to a compatible input of tct_c. If the invocation succeeds, LIBRA invokes tct_c again from the same saved sandbox state, replacing that input with an independently constructed literal or omitting it when the input is optional. If the call with the observed provider value succeeds while the literal-or-omission control fails, LIBRA retains an input-provenance relation tp→tct_p \rightarrow t_c: the upstream value is necessary, relative to the tested control, for that consumer invocation to succeed under the fixed sandbox state. State-mediated prerequisites are assessed separately by the before–after state control. If both invocations succeed, the connection is retained as a compatibility relation only when the provider output and consumer input are also semantically aligned.

Some tools communicate through environment state rather than explicit values. LIBRA tests these cases by invoking the consumer tct_c with identical arguments before and after executing a candidate provider tpt_p. A state dependency tp→tct_p \rightarrow t_c is retained only when the consumer fails before tpt_p is executed and succeeds afterward. Observed values and states produced by successful calls are reused in later invocations, allowing LIBRA to discover multi-hop compositions.

Retrieval with Graph Propagation

Given a query qq, the base retriever assigns each tool a normalized dense relevance score

sq(t)=1+cos⁡(h(q),h(t))2,(3)s_q(t) = \frac{1 + \cos(h(q), h(t))}{2}, \tag*{(3)}

where hh is the dense encoder applied to the original tool metadata and the executable contracts are not embedded as retrieval documents. The retriever selects the top KsK_s tools as consumer seeds. Retrieval propagates over every relation admitted to the benchmark-specific retrieval graph, including compatibility relations when they are retained.

Starting from each seeded consumer, LIBRA follows incoming graph edges for one hop to its providers. For each input of a seeded consumer, LIBRA retains at most two providers: the provider with the highest output-to-input field similarity and the provider with the highest dense relevance sqs_q. If the same provider satisfies both criteria, it is retained only once. This pruning recovers plausible prerequisites without adding every provider that passes the schema check to the candidate set.

Let Cq(u)C_q(u) denote the set of distinct seeded consumers connected to provider uu. Each consumer propagates its relevance score sq(t)s_q(t) to uu. LIBRA averages the provider’s original relevance with these consumer scores:

gq(u)=sq(u)+∑t∈Cq(u)sq(t)1+∣Cq(u)∣.(4)g_q(u) = \frac{s_q(u) + \sum_{t \in C_q(u)} s_q(t)}{1 + \lvert C_q(u) \rvert}. \tag*{(4)}

LIBRA then sets

sq′(u)=max⁡(sq(u),gq(u)).(5)s'_q(u) = \max\left(s_q(u), g_q(u)\right). \tag*{(5)}

A consumer contributes at most once to a provider, even when several of its inputs select the same provider. Averaging prevents a tool from receiving a higher score solely because it has more graph neighbors. Tools outside the retrieval graph retain their original dense scores. LIBRA selects the top KcK_c tools under sq′s'_q as the candidate set.

Evaluation

We evaluate LIBRA as a tool-retrieval method. Specifically, we ask whether the induced library improves the recovery and ranking of tools in one-shot retrieval and whether the same library remains useful for agentic retrieval. Retrieval metrics measure candidate-set quality rather than downstream task completion.

Experimental Setup

Benchmarks. We evaluate on three benchmarks spanning different catalog sizes, domains, and composition patterns: AppWorld ((Trivedi et al., 2024)) covers everyday app workflows; SciToolEval ([6]) targets scientific computing pipelines; and the multi turn base split of BFCL ([16]) contains function calling tasks over filesystem and utility APIs. We convert each benchmark into a common retrieval format comprising a query, a benchmark wide tool catalog, and a gold set of required tools. For each benchmark, LIBRA constructs one library using only public tool metadata and the executable runtime; it does not access task instructions, gold tool sets, or split labels during construction. Required tool annotations are used only for post retrieval evaluation, and no task level annotations from these benchmarks are used to select retrieval settings. Further details are provided in Appendix § 17.

Baselines. We compare against following sparse and dense retrieval methods. BM25 ((Robertson and Zaragoza, 2009)) and Query Tool Similarity (QTS) represent lexical and dense retrieval without structural augmentation. ToolREX ([14]) augments each tool with an LLM generated profile before embedding. TGR-Adapt ([9]) and Ego-Trip ([1]) both construct tool graphs from metadata via LLM based dependency classification and relation extraction, respectively. We additionally include SciToolKG, a knowledge graph retriever, as a baseline ([6]). We do not compare against methods that require existing tool use trajectories or task specific training as they are inapplicable to our setting. For agentic retrieval, we integrate each method into ToolReAGt ([3]) with three agent LLMs: Gemma-4-12B, Nemotron-9B, and Qwen3.5-9B ([18, 2, 18]). All prompts are in Appendix § 8.

Metrics and protocol. We report Precision, Recall, and NDCG at K=15K = 15 for AppWorld and BFCL and K=10K = 10 for SciToolEval. All embedding based methods use Qwen-3 Embedding 0.6B (Zhang et al., 2025), return at most 20 candidates, and are evaluated without query time reranking.

Figure 2. Agentic retrieval after 10 iterations. Each panel reports Recall and NDCG for one agent LLM and benchmark. Points show the mean over three random seeds, and error bars show standard error. LIBRA attains the highest Recall in 8 of the 9 settings and the highest NDCG on SciToolEval and BFCL for all three agent LLMs.

Library construction. LIBRA constructs the executable tool library once per catalog using DeepSeek-v4-Flash (Xu et al., 2026) for contract generation. The library is constructed once per benchmark and reused across all LIBRA queries and all agent LLMs. We report additional statistics such as catalog size in Appendix § 16.

One-Shot Retrieval

Table 2 compares LIBRA with one-shot retrieval baselines. Relative to QTS, which uses the same embedding model without the induced retrieval graph, LIBRA improves Recall and NDCG by 7.9 and 3.7 percentage points on average across the three benchmarks.

MethodAppWorld@15 PAppWorld@15 RAppWorld@15 NDCGSciToolEval@10 PSciToolEval@10 RSciToolEval@10 NDCGBFCL@15 PBFCL@15 RBFCL@15 NDCG
BM25 ((Robertson and Zaragoza, 2009))10.920.018.812.086.454.126.275.569.5
QTS15.528.732.811.384.183.928.681.876.2
ToolREX ([14])14.526.830.711.183.183.827.177.771.9
TGR-Adapt ([9])19.736.339.910.276.275.128.581.473.6
Ego-Trip ([1])15.829.233.211.384.183.930.485.778.4
SciToolKG ([6])–––12.689.782.5–––
LIBRA21.840.739.112.791.186.630.686.478.4

Table 2. One-shot retrieval results without query time reranking. Dataset@K denotes the evaluation cutoff. Values are percentages. The best result in each column is highlighted, and the second best is underlined.

On SciToolEval, LIBRA outperforms the official knowledge graph retriever SciToolKG by 1.4 and 4.1 points for Recall and NDCG. The largest gains occur on AppWorld, whose tasks require 8.18 tools on average: LIBRA improves Recall by 12 points and NDCG by 6.3 points over QTS.

Agentic Retrieval

Compared with QTS, which uses the same dense retrieval backend without the executable tool library, LIBRA improves Recall and NDCG by 4.5 and 2.9 percentage points, averaged across three benchmarks, three agent LLMs, and three random seeds. Averaged across LLMs and seeds, the gains are 8.5 Recall and 4.7 NDCG points on AppWorld, 1.3 and 1.8 points on SciToolEval, and 3.6 and 2.3 points on BFCL.

LIBRA achieves the highest Recall for every agent LLM on AppWorld and BFCL. On SciToolEval, it achieves the highest Recall with Qwen3.5 and Nemotron and trails Ego-Trip by only 0.1 points with Gemma-4. It also achieves the highest NDCG for every agent LLM on SciToolEval and BFCL. Although TGR-Adapt obtains higher NDCG on AppWorld, LIBRA retrieves a larger fraction of the required tools. These results show that the same query independent executable library remains effective as agents reformulate their searches over multiple rounds, without modifying the ReAct policy or training the agent LLM.

Analysis of Dependency Recovery

Dependencies Bridge Semantic Gaps

The experimental results show that LIBRA retrieves more required tools. They leave two questions open: why a dependency library helps and why recovery still fails in some cases. Following the evaluation setup, we examine the frozen oneshot rankings and induced libraries using K=15K = 15 for AppWorld and BFCL and K=10K = 10 for SciToolEval.

Figure 3 shows three representative dependencies recovered by LIBRA despite low semantic similarity between each provider and the request. The AppWorld request asks to move an alarm, placing phone.update_alarm at Dense rank 1. The authentication prerequisite is implicit, leaving phone.login at rank 28. In BFCL, a request to read the tail of a newly created CSV directly matches GorillaFileSystem.tail. Its state creating prerequisite GorillaFileSystem.touch is ranked 16. The rank gap is largest on SciToolEval: a request for the NPR2 descriptor retrieves GetNPR2 at rank 1, whereas the required format converter SELFIEStoSMILES is ranked 169. In all three cases, LIBRA moves the provider into the top KK.

Figure 3. We projected tool embeddings separately for each benchmark using t-SNE, with background colors indicating different clusters. In each example, Dense retrieval ranks the consumer (blue) within the top KK but misses its provider (orange): (a) phone.update_alarm (#1) depends on phone.login (#28); (b) tail (#3) depends on touch (#16); and (c) GetNPR2 (#1) depends on SELFIEStoSMILES (#169). LIBRA follows the retained relation and recovers the provider. For SciToolEval and AppWorld, we observed runtime dependencies connecting tools across different clusters.

These examples represent three forms of composition: authentication, environment state, and data format conversion. The semantic embeddings of the provider and consumer tools need not be similar because the tools implement distinct operations. Dense retrieval therefore behaves as intended when it finds the consumer that directly matches the request; the missing signal is the dependency that specifies which prerequisite must be executed first. The induced library supplies this dependency signal without forcing prerequisite tools to resemble the query or their consumers in embedding space.

Dependency Recovery Breakdown

We analyze gold tool occurrences ranked outside the top KK by query tool similarity. We assign each occurrence to one of five mutually exclusive outcomes. If LIBRA moves the provider into the top KK, the occurrence is marked as recovered. Otherwise, the occurrence is assigned to one of four failure causes: the provider has no outgoing edge in the retrieval graph; none of its connected consumers is retrieved as a seed; it is not selected among competing providers; or its propagated score remains below the cutoff.

Table 3 reveals different causes of failed recovery across benchmarks. SciToolEval has the highest recovery rate: LIBRA recovers 58.8% of gold tool occurrences ranked outside the top KK by query tool similarity, and almost every remaining miss has no edge in the retrieval graph. On BFCL, most failures occur later in the recovery pipeline. Only 2.7% of the Dense misses have no outgoing graph edge; for 30.4%, no connected consumer is retrieved as a seed; 12.1% are not selected among competing providers; and 21.4% are promoted but remain below the cutoff. Seed retrieval and ranking are therefore the main causes of failed dependency recovery on BFCL.

BenchmarkDense misses (nn)No graph edge (%)Consumer not seeded (%)Provider not selected (%)Below cutoff (%)Recovered (%)
AppWorld86065.113.03.40.218.3
BFCL2242.730.412.121.433.5
SciToolEval8538.80.02.40.058.8

Table 3. Recovery outcomes for query specific gold tool occurrences outside the top KK under Dense retrieval. Each occurrence is assigned to exactly one outcome; percentages sum to 100 within each benchmark.

Failure Case Studies

We examine one representative example for each failure category in Table 3. These examples show how graph coverage, seed retrieval, provider selection, and score propagation can each limit dependency recovery.

No graph edge. The corresponding relation may be absent because ahead-of-time construction cannot execute a prerequisite that requires a query specific input. For example, phone.search_contacts must retrieve a recipient’s phone_number for phone.send_text_message. At execution time, the environment builds the active task database from shared base data, applies task specific changes, and sets the requesting user. The query then specifies which contact to retrieve. During LIBRA’s ahead-of-time construction, however, contracts are generated from public tool metadata without access to the task instruction. As a result, executing phone.search_contacts produces no phone_number for the downstream consumer. Without this output, LIBRA cannot retain the corresponding input provenance relation between phone.search_contacts and phone.send_text_message.

Consumer not seeded. A retained relation cannot contribute to retrieval when its consumer is not retrieved as a seed. For example, the induced graph contains an edge from the gold prerequisite login in the filesystem API to the gold consumer create_file. However, the provider login and consumer create_file have Dense ranks 99 and 387, respectively. Because create_file is not retrieved as a seed, the graph edge is never activated and cannot improve retrieval.

Provider not selected. A valid provider is excluded when competing providers score higher under both per input selection criteria. For example, a query asks to follow every artist appearing in the user’s liked songs and albums. The gold plan uses show_album to obtain the artist_id required by follow_artist, and the induced graph contains this edge. However, show_following_artists also connects to the same input and has higher output input field similarity (0.928 versus 0.783) and higher query relevance (Dense rank 2 versus rank 31). Consequently, show_album satisfies neither per input selection criterion, is not selected, and receives no propagated score.

Below cutoff. A selected provider can remain unrecovered when score propagation does not lift it above the retrieval cutoff. For example, the vehicle API’s startEngine tool is a Dense seed at rank 10, and the library contains a state dependency from lockDoors to startEngine. Propagation promotes lockDoors from rank 55 to rank 35, which remains below the cutoff.

Conclusion

We introduced LIBRA to transform a flat executable tool catalog into a reusable library of runtime-supported composition relations. From LLM-proposed contracts, LIBRA admits a relation only when a counterfactual sandbox test confirms it, and propagates the validated relations to recover workflow prerequisites omitted from user requests. Construction uses CPU resources but requires no GPU for model training, with a number of LLM calls linear in the catalog size. Once constructed, the library can be deployed across tasks: it improves one-shot retrieval over dense baselines and remains effective within ToolReaGT across benchmarks and agent LLMs, showing that runtime-supported composition complements semantic similarity. By separating catalog-level composition knowledge from query-time reasoning, LIBRA enables reuse without query-specific reconstruction.

Limitations

LIBRA requires a benchmark whose tools are actually executable in an isolable, resettable runtime; many synthetic tool use benchmarks provide only tool signatures or mocked responses, to which LIBRA does not apply. Construction also incurs an upfront execution cost, though it amortizes across queries.

The counterfactual test governs the precision of retained relations, not the coverage of relations proposed in the first place. Because ahead-of-time construction cannot observe the query specific environment, some prerequisites cannot be exercised during construction, leaving their relations absent before any test applies. This gap is benchmark dependent: it dominates on AppWorld but is negligible on BFCL (Table 3). A retained relation also cannot help if its consumer is not retrieved as a seed.

Prompts

We report the LLM prompts used in all experiments, including LIBRA contract generation, ToolREX profile construction, TGR and Ego-Trip graph construction, and ToolReAGt agentic retrieval.

LIBRA uses the following prompt for initial contract generation and runtime feedback revisions. For each tool, one initial generation call and at most two revisions share a three-call budget. Braced terms denote values serialized directly from the tool metadata, runtime feedback, or the contract schema.

# System
Write an executable call contract for one tool from its
public metadata and supplied same-tool feedback only.

# User
{
  "contract_version": "tool_call_contract_v2",
  "revision": {revision},
  "feedback_policy": {feedback_policy},
  "public_tool_metadata": {public_tool_metadata},
  "invocation_record_schema": {invocation_record_schema},
  "required_output_schema": {tool_call_contract_schema},
  "rules": {contract_rules},
  "previous_contract": {previous_contract},
  "instruction": "Return one complete replacement contract
  ...",
  "same_tool_invocation_feedback": {same_tool_feedback},
  "feedback_note": {feedback_note}
}
# System
Write an executable call contract for one tool from its
public metadata and supplied same-tool feedback only.

# User
{
  "contract_version": "tool_call_contract_v2",
  "revision": {revision},
  "feedback_policy": {feedback_policy},
  "public_tool_metadata": {public_tool_metadata},
  "invocation_record_schema": {invocation_record_schema},
  "required_output_schema": {tool_call_contract_schema},
  "rules": {contract_rules},
  "previous_contract": {previous_contract},
  "instruction": "Return one complete replacement contract
  ...",
  "same_tool_invocation_feedback": {same_tool_feedback},
  "feedback_note": {feedback_note}
}

Figure 4. LIBRA executable contract generation prompt. Braced fields are populated from the tool metadata, contract schema, or same-tool runtime feedback.

The last four fields are included only during revision or when same-tool execution feedback is available. The complete shared rule list used in contract_rules is shown below.

1. Describe exactly one supplied tool and return only the complete JSON contract.
2. Represent every required public parameter in every call template.
3. Use literal when a caller can construct the value from public information alone.
4. Use observation when successful use needs an exact value learned at runtime.
5. Dependency-required is independent of whether the public signature marks a parameter required: use observation exactly when this call must refer to a particular pre-existing value that a caller cannot choose or derive from public information.
6. A schema-valid placeholder or guessed identifier, token, handle, or reference is not a constructible literal when an arbitrary same-type value would fail; never use a dummy value merely because it has the right JSON type.
7. Every literal in an expected=success template must be a plausible value that can work as written using public information alone; otherwise use observation.
8. A success template may and should contain multiple observation arguments when successful use needs multiple exact runtime values.
9. Test constructibility against success: if replacing an input with another arbitrary schema-valid value would require different pre-existing setup or make the call fail, that input is not independently constructible and must use observation.
10. Inputs that a call freely chooses for a new value ---such as initial attributes, a newly chosen password, user-authored content, amounts, quantities, dates, flags, and search or filter criteria---are constructible literals unless the current tool metadata explicitly requires a pre-existing reference.

Table 7.

  1. For a call that authenticates, reads, changes, deletes, or sends to, or relates a pre-existing record, use observation for every identity-bearing account value, credential, recipient/contact, identifier, handle, code, or path that must match that record. A caller typing the value does not make it independently constructible.

  1. In an expected-success authentication or sign-in template, both the existing account identifier and its existing secret are observations; never invent either. Account creation instead constructs its new identifier and secret. Credential recovery observes the existing account and issued recovery code but constructs the replacement secret.

  1. Do not use observation merely because another tool could output a schema-compatible value; the value must be tied to a particular pre-existing entity, credential, file, or state needed by this call.

  1. Use omit only for an optional parameter that the template intentionally omits.

  1. Omit is orthogonal to dependency: when an optional parameter is supplied and must reference a particular pre-existing value, use observation.

  1. Across the proposed success templates, give every optional public parameter that a caller can construct at least one literal example; another success template may still omit it.

  1. Literals are frozen proposals; ordinary propagation will not replace them.

  1. Include at least one proposed success template.

  1. Declare every reusable value with an explicit JSONPath export.

  1. Caller-supplied arguments remain observable after the call; their mechanical $.parameter_name exports are normalized from the public signature.

  1. Export every scalar leaf shown in the public success response; use [*] for array elements.

  1. For every response or state_changes export, set propagate=true when its exact runtime value identifies, locates, describes, authenticates, or carries specific entity/content/state information that a later tool call could meaningfully reuse.

  1. Every identifier or foreign key, opaque handle/token/code, address/contact, path/URL, timestamp/date, user-authored content, entity attribute, and monetary value must propagate, even when echoed or easy to replace with some other schema-valid value.

  1. Set propagate=false only for generic acknowledgement/status/error text, boolean flags, aggregate counts/ratings/ranks, pagination/control metadata, or display-only summaries that neither identify nor describe a particular entity or content item.

  1. Judge propagation from this tool alone and never from the runtime scalar’s magnitude; a small integer or short string can still be an opaque resource reference.

  1. Exports may select arguments, response, or state_changes and require a JSON Schema.

  1. When same-tool feedback supports semantic names and exact paths for scalar state fields, export every such field; never return only a representative subset.

  1. Same-tool JIT feedback is rendered exactly according to feedback_policy; use only the supplied representation to map meaningful field names to exact paths.

  1. Every state export must name exactly one semantic field and use one explicit JSONPath.

  1. A state_changes export must select reusable scalar leaves, not only a whole array or object container.

  1. Never use recursive catch-all exports such as $.[*]; omit a hidden value when its field identity cannot be supported by the supplied context.

  1. Never copy a concrete runtime scalar from invocation.result into a template, export name, path, or schema.

  1. Do not export arbitrary descendants whose semantic identity is supported by neither public metadata nor supplied same-tool feedback.

  1. Use standard JSON Schema Draft 2020-12 for observable

Table 8.

  1. Outcome schemas are over the normalized invocation record, never the raw response alone.

  2. A success outcome schema must match top-level status success and put response constraints under output.

  3. A failure outcome schema must match top-level status rejected or error and put response constraints under output.

  4. The success and failure schemas must describe disjoint observable outcomes.

  5. Use only this tool’s public metadata and supplied same-tool feedback.

  6. Do not infer another tool, graph, task, benchmark, fixture, or application rule.

Shared rules supplied through contract_rules during LIBRA contract generation
success and failure outcomes.
35. Outcome schemas are over the normalized invocation
    record, never the raw response alone.
36. A success outcome schema must match top-level status
    success and put response constraints under output.
37. A failure outcome schema must match top-level status
    rejected or error and put response constraints under
    output.
38. The success and failure schemas must describe disjoint
    observable outcomes.
39. Use only this tool’s public metadata and supplied same
    -tool feedback.
40. Do not infer another tool, graph, task, benchmark,
    fixture, or application rule.

Figure 5. Shared rules supplied through contract_rules during LIBRA contract generation.

Catalog-specific calling conventions. AppWorld uses the shared prompt unchanged. SciToolEval and BFCL insert the following rules after Rule 1 and replace Rules 14, 15, 20, and 21 with their registry-specific versions.

Registry-specific calling conventions used for SciToolEval and BFCL
# Inserted registry rules
- Return only fields defined by required_output_schema;
  metadata and calling-convention fields are input
  context, not contract output.
- {calling_convention_rule}
- Copy template argument keys exactly from
  public_tool_metadata.parameters[*].name; never
  invent an alias or use a parameter name found only
  in prose.
- If public_tool_metadata.parameters is empty, every
  template arguments object must be empty.
- Every literal must validate against that parameter’s
  exact JSON Schema using native JSON types; numbers
  and booleans are not quoted strings.
- Never emit a long payload as a literal. For a sequence,
  document, file, blob, encoded value, or any input
  whose plausible example would be long, use
  observation instead; every literal must remain short
  enough to inspect on one line.
- Use the shortest literal that can plausibly succeed;
  when metadata states a minimum length, meet it
  without padding beyond that minimum.

# Registry replacements
- Use omit when a parameter is intentionally absent from
  this template; omit is a call choice, not a model-
  filled value.
- Omit is orthogonal to dependency: a field may be absent
  in one template and still require an upstream
  observation whenever supplied.
- Do not declare caller-argument exports; they are added
  mechanically from the public interface.
- For a structured public success response, do not repeat
  its scalar exports; exact public field labels and
  paths are added mechanically. For a scalar success
  response, declare exactly one response export at
  path \$ and name it with the shortest snake_case
  semantic value name explicitly supported by this
  tool’s public function, description, or return
  documentation. Use response_value only when that
  metadata gives the scalar no semantic identity.

# Appended registry rules
- Keep outcome schemas minimal: constrain normalized
  status and output shape without copying the full
  public response object.
- Before returning, check every template key against
  public_tool_metadata.parameters[*].name, every
  required parameter against its template, and every
  literal against its listed schema.

# System-message suffix
Return compact JSON. Never write or repeat a long payload
literal; represent that input as observation.

Figure 6. Registry-specific calling conventions used for SciToolEval and BFCL.

For BFCL, the seventh inserted rule and the failure-outcome rule are replaced, and the final three rules below are appended.

Additional contract-generation rules used for BFCL stateful tools
- For a state-changing action, put first a success
  template that completes the operation described by
  the public metadata rather than a partial, no-op, or
  opposite transition. When the metadata describes a
  bounded degree, use its complementing boundary; when a
  collection selects independently named components
  and lists the complete allowed set, include that
  full set. Prefer this one effect-complete template
  over extra reset or diversity templates.
- A failure outcome schema normally matches top-level
  status rejected or error. It may instead match top-
  level status success only when public metadata
  explicitly defines an output value as a business-
  level failure.
- A transport-success response can still report a business
  -level failure. Use an output const or enum as a
  success/failure discriminator only when this tool’s
  public metadata explicitly defines those values.
- For a read, query, or status operation, a returned
  boolean is successful result data unless public
  metadata explicitly says the operation failed; otherwise
  both true and false belong to the success outcome.
- For an action, authentication, or mutation whose public
  metadata explicitly defines success and failure flag
  values, constrain those values in disjoint success
  and failure schemas.

Figure 7. Additional contract-generation rules used for BFCL stateful tools.

When feedback is provided, feedback_note states that it is restricted to the current tool and that concrete runtime values may be used only to interpret field structure, never copied into persisted literals. For the complete-same-tool policy used in all three benchmarks, the exact rule is:

Feedback instruction used when revising a contract after same-tool execution
Feedback is restricted to this tool. invocation_result is
the complete return record for the call that
triggered JIT repair; other feedback scalars remain
structural. Use runtime values only to understand
field structure and never copy a concrete feedback
value into any persisted contract literal. Named
state columns or keys are structural evidence:
enumerate every scalar field whose semantic name and
exact path the record supports.

Figure 8. Feedback instruction used when revising a contract after same-tool execution.

The calling-convention placeholder is filled with exactly one of the following rules, according to the public interface:

Calling-convention text selected according to the benchmark tool interface
This runtime receives one JSON object: template.arguments
are exactly that object’s public keys, with their
original spelling.

This runtime uses the declared callable signature:
template.arguments are exactly its public parameter
names.

Figure 9. Calling-convention text selected according to the benchmark tool interface.

Tool-REX initial documentation-generation prompt

Figure 10. Tool-REX initial documentation-generation prompt.

Tool-REX profile-refinement prompt

Figure 11. Tool-REX profile-refinement prompt. Gemma-4-31B applies it to profiles rejected by the consistency judge.

Tool-REX consistency-judgement prompt

Figure 12. Tool-REX consistency-judgement prompt. It returns a binary decision before a generated profile is accepted.

Image not extracted

Figure 13.

TGR dependency judgement prompt

Figure 13 (PDF p. 14). TGR dependency judgement prompt. It assigns a directed dependency label or no_dependency to every unordered tool pair.

If neither tool’s output is a required input of the other, the answer is {"label":"no_dependency"}.

Now classify this pair:

Tool A:

Tool ID: {tool_a_id}

Signature: {tool_a_signature}

Docstring:

{tool_a_docstring}

Tool B:

Tool ID: {tool_b_id}

Signature: {tool_b_signature}

Docstring:

{tool_b_docstring}

Return exactly one JSON object such as {"label":"no_dependency"}.

Ego-Trip graph-extraction prompt

Figure 14. Ego-Trip graph-extraction prompt adapted to each benchmark catalog.

System

Extract a compact knowledge graph from one tool description. Return only JSON with a ’relationships’ array. Every item must contain the string fields ’head’, ’relationship’, and ’tail’. Use concise, consistent entity names; split compound claims; do not duplicate triples or create self-relations.

User

The exact tool title is {tool_name}. Extract head-relation-tail triples from the metadata below. Use only these relations when applicable: has_line_of_business, has_entity, has_parameter, contains, related_to, used_for, has_attribute, associated_with, managed_by, part_of, required_for, depends_on, produces, receives_from, involved_in, reports_to, responsible_for, affects, includes. Link the title to its category with has_line_of_business, to each input parameter with has_parameter, and to other salient concepts with has_entity. The exact tool title may occur only in those three relations.

Scenario title: {tool_name}

Line of business: {category}

Function: {function}

Description: {description}

Input parameters: {parameter_names}

Output schema: {"relationships":[{"head":"...","relationship":"...","tail":"..."}]}

All agentic retrieval experiments use the plain_fulltask condition below.

ToolReaGt agent prompt

Figure 15. ToolReaGt agent prompt used in all agentic retrieval experiments.

the following format:

Thought: The current language of the user is: (user’s language). I need to use a tool to help me answer the question.

Action: tool name (one of {tool_names}) if using a tool.

Action Input: the input to the tool, in a JSON format representing the kwargs (e.g. {"input": "hello world", "num_beams": 5}).

Please ALWAYS start with a Thought. NEVER surround your response with markdown code markers. You may use code markers within your response if you need to. Please use a valid JSON format for the Action Input. Do NOT do this {’input’: ’hello world’, ’num_beams’: 5}.

If this format is used, the tool will respond in the following format:

Observation: tool response

You should keep repeating the above format till you have enough information to answer the question without using any more tools. At that point, you MUST respond in one of the following two formats:

Thought: I can answer without using any more tools. I’ll use the user’s language to answer

Answer: [your answer here, in the same language as the user’s question]

Thought: I cannot answer the question with the provided tools.

Answer: [your answer here, in the same language as the user’s question]

Current Conversation

Below is the current conversation consisting of interleaving human and assistant messages.

User

Given the following complex task: "{task}" and the list of tools in the context, you must select the best tools to solve the complex task. You must obtain that context by calling ”tool_retriever” with a suitable query before selecting tools.

The retriever is exposed as tool_retriever with one string argument, input. The tool_desc placeholder uses the corresponding description below. At later turns, LlamaIndex appends the preceding Thought, Action, and JSON Observation records to the same conversation.

Retriever descriptions supplied to Tool-ReaGt

Figure 16. Retriever descriptions supplied to Tool-ReaGt for the compared retrieval backends.

QTS, Tool-REX, and TGR

Search the complete tool registry with a focused natural-language query and return the top ranked tool descriptions.

Ego-Trip

Search the tool knowledge graph with a focused natural-language query and return tools from the matched ego graphs.

LIBRA

Search an execution-validated dependency library with a focused natural-language query and return a ranked tool list.

Execution Validation Ablation

We examine the effect of execution-based filtering by fixing the dense retriever and varying only the retrieval graph used for score propagation.

All three variants use Qwen-3 Embedding-0.6B over the original tool metadata, as in QTS, and no reranking is applied. The first variant (QTS) performs no propagation. The second propagates scores over all schema-compatible edges, i.e., the candidate graph before execution filtering. LIBRA propagates only over edges retained through successful execution in the target runtime.

Table 4 (PDF p. 15) shows that schema-compatible edges alone are insufficient. Propagation over the execution-filtered graph consistently outperforms both QTS and propagation over the unfiltered schema-compatible graph.

QTSSchema-OnlyLIBRA
AppWorld @15 R / NDCG28.7 / 32.829.2 / 33.140.7 / 39.1
SciToolEval @10 R / NDCG84.1 / 83.984.1 / 83.991.1 / 86.6
BFCL @15 R / NDCG81.8 / 76.281.4 / 76.086.4 / 78.4

Table 4 (PDF p. 15). Ablation of graph quality with fixed dense retrieval. All variants use the same encoder over original tool metadata, differing only in the retrieval graph.

Library Construction Statistics

Table 5 reports the scale and runtime cost of LIBRA’s ahead-of-time library construction. For each tool, DeepSeek-V4-Flash generates an initial executable contract with temperature zero, and LIBRA permits at most two feedback based revisions. For AppWorld, SciToolEval, and BFCL, respectively, 218, 48, and 37 contracts required at least one revision.

AppWorldSciToolEvalBFCL
Catalog tools473477128
Candidate compositions581,690163,60812,762
Retained relations (value based / state)1,946 / 0732 / 0595 / 77
Wall clock (min:s)260:4020:454:28
Seconds per 1,000 candidates26.97.621.0

Table 5. Library construction statistics. Candidate compositions are schema compatible combinations of provider exports and consumer inputs before runtime filtering. A single tool pair may contribute multiple candidate compositions. Relation counts report value based relations and state dependencies included in the benchmark specific retrieval graph. Normalized time is calculated from the recorded wall clock time and candidate composition count.

The resulting contracts define candidate compositions between schema compatible provider exports and consumer inputs. A single provider consumer tool pair can contribute multiple candidate compositions when different exports or inputs are involved. Across the three benchmarks, LIBRA generates between 12,762 and 581,690 candidate compositions before runtime filtering and retains between 672 and 1,946 execution supported relations.

Each library construction job was configured with 96 CPU workers. The AppWorld job requested 384 GB of memory, whereas the SciToolEval and BFCL jobs each requested 128 GB. Each job additionally used one L40S GPU to serve field embeddings. The recorded wall clock times were 260 minutes 40 seconds for AppWorld, 20 minutes 45 seconds for SciToolEval, and 4 minutes 28 seconds for BFCL.

The total construction time depends on two distinct factors. The first is the number of candidate compositions that must be tested. This quantity depends not only on catalog size, but also on the numbers of exported provider values, consumer inputs, and schema compatible matches between them. For example, AppWorld and SciToolEval contain similar numbers of tools, 473 and 477, but produce 581,690 and 163,608 candidate compositions, respectively.

The second factor is the cost of validating each candidate composition. A candidate composition is not equivalent to a single sandbox execution: its validation can invoke the provider, the consumer, and one or more control calls. The invoked tools can also differ in their internal execution cost, including both constant overhead and asymptotic complexity with respect to their own input sizes. In addition, the three benchmarks use different executable backends. Consequently, candidate count alone does not determine wall clock time. When normalized by candidate count, the recorded times correspond to approximately 26.9, 7.6, and 21.0 seconds per 1,000 candidate compositions for AppWorld, SciToolEval, and BFCL, respectively. AppWorld takes approximately 12.6 times as long as SciToolEval despite generating approximately 3.6 times as many candidate compositions.

In the worst case, the number of candidate compositions is quadratic in the catalog size, with additional multiplicative factors from the numbers of provider exports and consumer inputs. Schema compatibility restricts the number of compositions considered in practice.

None of the three benchmarks provides complete provider consumer dependency annotations. We therefore do not construct surrogate edge level ground truth or claim precision against a canonical dependency graph. Here, runtime validated means that a retained relation passed the corresponding composition test in the benchmark’s executable environment: AppWorld’s local stateful runtime, SciToolEval’s service based tool runtime, or BFCL’s released executable backend. Runtime validation provides execution evidence under the tested contract and environment state rather than a universal guarantee over all possible inputs and states.

Dataset Processing and Baseline Adaptation

Benchmark conversion and curation. We convert each benchmark into a common retrieval format containing a query, a shared tool catalog, and a gold required-tool annotation. When the released annotation contains an ordered call sequence, we preserve it as tool_path; retrieval evaluation is defined over the set of distinct tools in that sequence. For AppWorld, we evaluate on the 90 training and 57 development tasks with public required-API annotations; hidden test annotations are unavailable. For SciToolEval, the released files contain 530 tasks: 150 level-1 and 380 level-2. We evaluate a frozen subset of 270 tasks and exclude the remaining 260. Each retained task keeps its released split name and row index, and we verify that its question and ordered gold tool path match the corresponding released record exactly. The same subset is used for all retrieval methods.

Tasks are excluded when they involve tools that cannot be executed in the runtime. Inspecting the released implementation, we identify one important source of incompatibility: the IsValidInChIKey, InChIKeyToSMILES, InChIKeyToInChI, and InChIKeyToMOL wrappers all invoke the legacy ChemSpider InChI web service. On such non-executable tools LIBRA induces no composition relations and reduces to the dense baseline, so the exclusion confers no advantage specific to LIBRA over the non-execution baselines. For BFCL, we use the 200 tasks from the BFCL v4 multi_turn_base split and the 128 functions in its released multi-turn function registry. Each task is matched by identifier to one record in possible_answer/BFCL_v4_multi_turn_base.json. We read the record’s ground_truth field as an ordered list of turns, where each turn contains the official function calls for that turn. We flatten these calls in chronological order and map every raw function name to its unique fully qualified API name.

The stored tool_path preserves both call order and repeated calls. Retrieval relevance, however, is defined over the set of distinct fully qualified functions in this sequence; repeated invocations of the same function therefore receive no additional relevance credit. We do not union or intersect alternative workflows, nor do we select among multiple candidate answers: the converter consumes the single ground_truth object associated with each task.

The retrieval query is formed by collecting every non-empty user message from all turns, preserving the original turn order, and joining the messages with newline separators. Assistant messages, tool outputs, function arguments, and the released ground-truth calls are not included in the retrieval query.

Retrieval metrics. Let GiG_i denote the set of distinct gold tools for query ii, and let

RiK=(ri1,…,riK)R_i^K = (r_{i1}, \ldots, r_{iK})

denote the ranked top-KK retrieval result. We compute

Precision⁡@Ki=1K∑j=1K1[rij∈Gi],\operatorname{Precision}@K_i = \frac{1}{K} \sum_{j=1}^{K} \mathbb{1}[r_{ij} \in G_i],

and

Recall⁡@Ki=1∣Gi∣∑j=1K1[rij∈Gi].\operatorname{Recall}@K_i = \frac{1}{|G_i|} \sum_{j=1}^{K} \mathbb{1}[r_{ij} \in G_i].

For NDCG, tool relevance is binary:

rel⁡ij=1[rij∈Gi].\operatorname{rel}_{ij} = \mathbb{1}[r_{ij} \in G_i].

We compute

DCG⁡@Ki=∑j=1Krel⁡ijlog⁡2(j+1)\operatorname{DCG}@K_i = \sum_{j=1}^{K} \frac{\operatorname{rel}_{ij}}{\log_2(j+1)}

and

IDCG⁡@Ki=∑j=1min⁡(K,∣Gi∣)1log⁡2(j+1).\operatorname{IDCG}@K_i = \sum_{j=1}^{\min(K,|G_i|)} \frac{1}{\log_2(j+1)}.

Then

NDCG@Ki=DCG@KiIDCG@Ki.\mathrm{NDCG}@K_i = \frac{\mathrm{DCG}@K_i}{\mathrm{IDCG}@K_i}.

All reported Precision, Recall, and NDCG values are unweighted averages; therefore, tasks with larger gold sets do not receive greater weight. Repeated calls to the same gold function, including repeated calls in BFCL workflows, are collapsed into one relevant tool before metric computation.

LIBRA is training-free and does not access task instructions, gold tool sets, or split labels during library construction. Required-tool annotations are used only after retrieval for evaluation. We perform no hyperparameter selection using task-level annotations from AppWorld, SciToolEval, or BFCL.

All embedding-based methods use the same Qwen-3 Embedding-0.6B encoder. For the standard dense representation, we embed the original serialized tool metadata. For Tool-REX, DeepSeek-V4-Flash generates a structured profile for each tool, and Gemma-4-31B performs one consistency check against the source metadata before the profile is embedded using the same encoder.

The original TGR implementation and trained dependency discriminator are not publicly available; we therefore implement a TGR-style adaptation in which DeepSeek-V4-Flash labels dependencies for tool pairs from their metadata, after which we apply TGR’s parameter-free graph propagation to the frozen tool embeddings. For Ego-Trip, we adapt its metadata based prompting procedure to each benchmark, use the dense retrieval tool in our experiments, and keep the remaining prompts the same. All prompts are available in Appendix § 8.

Retrieval Sensitivity

We vary three retrieval-time settings while holding the executable library, embedding model, and evaluation cutoff fixed (Table 6).

SettingVariantAppWorld@15SciToolEval@10BFCL@15
Provider selectionField-best only40.3 / 38.992.0 / 87.086.0 / 78.2
Dense-best only39.7 / 38.588.0 / 85.384.8 / 77.7
Both (default)40.7 / 39.191.1 / 86.686.4 / 78.4
All retained38.6 / 38.576.4 / 75.085.3 / 78.0
Seed KsK_s1040.8 / 39.691.9 / 86.986.2 / 78.5
15 (default)40.7 / 39.191.1 / 86.686.4 / 78.4
2040.6 / 39.091.3 / 86.786.2 / 78.3
Hop hh1 (default)40.7 / 39.191.1 / 86.686.4 / 78.4
240.4 / 39.188.8 / 85.586.4 / 78.5

Table 6. Sensitivity to retrieval-time settings. The induced library is fixed throughout. Values are R / NDCG (%).

Provider selection. Retaining all providers connected by the retained relations degrades Recall sharply on SciToolEval (76.4 vs. 91.1), showing that unpruned retained relations can introduce retrieval noise. Field-best and dense-best selection each recover most of the gain independently; combining them (default) yields the most consistent results across benchmarks.

Seed size. Performance is stable across Ks∈{10,15,20}K_s \in\{10, 15, 20\}, with fluctuations below one percentage point on all benchmarks.

Hop depth. Extending propagation to two hops slightly reduces Recall on SciToolEval (−2.3) while leaving AppWorld and BFCL essentially unchanged.

Implementation Details

BM25 was implemented directly rather than through an external retrieval package. We lowercase text and tokenize it using the regular expression [A-Za-z0-9_]+, with k1=1.5k_1 = 1.5 and b=0.75b = 0.75. Precision, Recall, and NDCG were also implemented directly according to the definitions in Appendix D.

Dense rankings use the released Qwen/Qwen3-Embedding-0.6B representations. Tool and query embeddings are ℓ2\ell_2-normalized, cosine similarity is computed by NumPy matrix multiplication, and rankings are obtained with argsort; no nearest-neighbor package is used ([10]).

For Figure 3, we project each benchmark’s tool embeddings independently using sklearn.manifold.TSNE with two components, cosine distance, perplexity 30, automatic learning rate, random initialization, random seed 17, and 1,500 iterations ([17] (van der Maaten and Hinton 2008)). Model inference uses vLLM ([12]).

Agentic Retrieval across Iterations

Figure 17. Agentic retrieval across ToolReAGt iterations. Each subfigure reports Recall and NDCG for three agent LLMs on AppWorld, SciToolEval, and BFCL at their benchmark-specific evaluation depths. Results show the mean over three random seeds, and error bars show standard error.

Figure 17 shows additional results for agentic retrieval across different iterations. Each subfigure reports Recall and NDCG for three agent LLMs on AppWorld, SciToolEval, and BFCL at their benchmark-specific evaluation depths. Results show the mean over three random seeds, and error bars show standard error. LIBRA achieves the highest Recall in 30 settings and the highest NDCG in 24 across all 36 settings.

Figure 17 (PDF p. 19). Agentic retrieval across ToolReAGt iterations (continued).

Relative to standard dense retriever QTS, LIBRA improves Recall and NDCG by 4.5 and 3.0 points on average.

Algorithms

Algorithms 1–2 summarize the key steps in library construction and graph propagation. Here, mim_i is tool tit_i’s public metadata, and Run⁡σ\operatorname{Run}_{\sigma} starts from the state σ\sigma. The contract is used by Extract and succ / fail. Let foutf_{\mathrm{out}} and finf_{\mathrm{in}} denote output- and input-field names. A work item (σ,ti,πi,β)(\sigma, t_i, \pi_i, \beta) runs candidate call πi∈Πi\pi_i \in\Pi_i of tool tit_i from state σ\sigma. The binding map β\beta is empty for an initial call; otherwise, β(fin)=(tp,fout,v)\beta(f_{\mathrm{in}}) = (t_p, f_{\mathrm{out}}, v) fills finf_{\mathrm{in}} with value vv from provider tpt_p’s output field foutf_{\mathrm{out}}. Field alignment passes when cos⁡(hf(fout),hf(fin))>τ\cos(h_f(f_{\mathrm{out}}), h_f(f_{\mathrm{in}})) > \tau, where hfh_f is the dense encoder for field names. We write Eprov\mathcal{E}_{\mathrm{prov}}, Ecomp\mathcal{E}_{\mathrm{comp}}, and Estate\mathcal{E}_{\mathrm{state}} for input-provenance relations, compatibility relations, and state dependencies, respectively.

Algorithm 1: Library construction.

Input: Tool catalog T\mathcal{T}, public metadata {mi}i=1n\{m_i\}_{i=1}^{n}, runtime R\mathcal{R}, field encoder hfh_f, and threshold τ\tau Output: Executable library L=(T,C,E)\mathcal{L} = (\mathcal{T}, \mathcal{C}, \mathcal{E})

  1. C,W←∅,∅\mathcal{C}, \mathcal{W} \leftarrow\varnothing, \varnothing

  2. Eprov,Ecomp,Estate←∅,∅,∅\mathcal{E}_{\mathrm{prov}}, \mathcal{E}_{\mathrm{comp}}, \mathcal{E}_{\mathrm{state}} \leftarrow\varnothing, \varnothing, \varnothing

  3. foreach ti∈Tt_i \in\mathcal{T} do

  4. ci=(Πi,Xi,Si+,Si−)←LLMGenerate⁡(mi)\quad c_i = (\Pi_i, \mathcal{X}_i, S_i^{+}, S_i^{-}) \leftarrow\operatorname{LLMGenerate}(m_i)

  5. bi←2, C←C∪{ci}\quad b_i \leftarrow2,\ \mathcal{C} \leftarrow\mathcal{C} \cup\{c_i\}

  6. σ0←Fresh⁡(R)\sigma_0 \leftarrow\operatorname{Fresh}(\mathcal{R})

  7. foreach ti∈T, π∈Πit_i \in\mathcal{T},\ \pi\in\Pi_i do

  8. \quadAdd (σ0,ti,π,∅)(\sigma_0, t_i, \pi, \varnothing) to W\mathcal{W}

  9. while W≠∅\mathcal{W} \ne\varnothing do

  10. (σ,ti,πi,β)←Pop⁡(W)\quad(\sigma, t_i, \pi_i, \beta) \leftarrow\operatorname{Pop}(\mathcal{W})

  11. π^i←Fill⁡(πi,β)\quad\widehat{\pi}_i \leftarrow\operatorname{Fill}(\pi_i, \beta)

  12. (ruse,σ′)←Run⁡σ(ti,π^i)\quad(r_{\mathrm{use}}, \sigma') \leftarrow\operatorname{Run}_{\sigma}(t_i, \widehat{\pi}_i)

  13. \quadwhile ¬[succ⁡(ruse)∨fail⁡(ruse)]∧bi>0\neg[\operatorname{succ}(r_{\mathrm{use}}) \lor\operatorname{fail}(r_{\mathrm{use}})] \land b_i > 0 do

  14. ci←LLMReviseContract⁡(ti,mi,ruse)\qquad c_i \leftarrow\operatorname{LLMReviseContract}(t_i, m_i, r_{\mathrm{use}})

  15. bi←bi−1\qquad b_i \leftarrow b_i - 1

  16. \quadif succ⁡(ruse)\operatorname{succ}(r_{\mathrm{use}}) then

  17. \qquadforeach β(fin)=(tp,fout,v)\beta(f_{\mathrm{in}}) = (t_p, f_{\mathrm{out}}, v) do

  18. v~←\qquad\qquad\widetilde{v} \leftarrow a test value not produced by tpt_p

  19. (ralt,_)←Run⁡σ(ti,π^i[fin←v~])\qquad\qquad(r_{\mathrm{alt}}, \_) \leftarrow\operatorname{Run}_{\sigma}(t_i, \widehat{\pi}_i[f_{\mathrm{in}} \leftarrow\widetilde{v}])

  20. \qquad\qquadif fail⁡(ralt)\operatorname{fail}(r_{\mathrm{alt}}) then

  21. Eprov←Eprov∪{(tp,fout,ti,fin)}\qquad\qquad\qquad\mathcal{E}_{\mathrm{prov}} \leftarrow\mathcal{E}_{\mathrm{prov}} \cup\{(t_p, f_{\mathrm{out}}, t_i, f_{\mathrm{in}})\}

  22. \qquad\qquadelse if succ⁡(ralt)\operatorname{succ}(r_{\mathrm{alt}}) and cos⁡(hf(fout),hf(fin))>τ\cos(h_f(f_{\mathrm{out}}), h_f(f_{\mathrm{in}})) > \tau then

  23. Ecomp←Ecomp∪{(tp,fout,ti,fin)}\qquad\qquad\qquad\mathcal{E}_{\mathrm{comp}} \leftarrow\mathcal{E}_{\mathrm{comp}} \cup\{(t_p, f_{\mathrm{out}}, t_i, f_{\mathrm{in}})\}

  24. \qquadforeach (fout,v)∈Extract⁡(Xi,π^i,ruse)(f_{\mathrm{out}}, v) \in\operatorname{Extract}(\mathcal{X}_i, \widehat{\pi}_i, r_{\mathrm{use}}) do

  25. \qquad\qquadAdd candidate calls using vv as an input to W\mathcal{W}

  26. \quadif σ′≠σ\sigma' \ne\sigma then

  27. \qquadforeach tc∈T, πc∈Πct_c \in\mathcal{T},\ \pi_c \in\Pi_c do

  28. (rbefore,_)←Run⁡σ(tc,πc)\qquad\qquad(r_{\mathrm{before}}, \_) \leftarrow\operatorname{Run}_{\sigma}(t_c, \pi_c)

  29. (rafter,_)←Run⁡σ′(tc,πc)\qquad\qquad(r_{\mathrm{after}}, \_) \leftarrow\operatorname{Run}_{\sigma'}(t_c, \pi_c)

  30. \qquad\qquadif fail⁡(rbefore)\operatorname{fail}(r_{\mathrm{before}}) and succ⁡(rafter)\operatorname{succ}(r_{\mathrm{after}}) then

  31. Estate←Estate∪{(ti,tc)}\qquad\qquad\qquad\mathcal{E}_{\mathrm{state}} \leftarrow\mathcal{E}_{\mathrm{state}} \cup\{(t_i, t_c)\}

  32. \qquad\qquadAdd (σ′,tc,πc,∅)(\sigma', t_c, \pi_c, \varnothing) to W\mathcal{W}

  33. E←Eprov∪Ecomp∪Estate\mathcal{E} \leftarrow\mathcal{E}_{\mathrm{prov}} \cup\mathcal{E}_{\mathrm{comp}} \cup\mathcal{E}_{\mathrm{state}}

  34. return (T,C,E)(\mathcal{T}, \mathcal{C}, \mathcal{E})

Algorithm 2: One-hop retrieval with graph propagation.

Input: Query qq, library L\mathcal{L}, encoders hh, hfh_f, seed size KsK_s, and candidate size KcK_c

Output: Retrieved candidate set TqT_q

  1. foreach t∈Tt \in\mathcal{T} do

    1. sq(t)←[1+cos⁡(h(q),h(t))]/2s_q(t) \leftarrow[1 + \cos(h(q), h(t))]/2

    2. sq′(t)←sq(t)s'_q(t) \leftarrow s_q(t)

  2. Sq←TopK⁡(T,sq,Ks)S_q \leftarrow\operatorname{TopK}(\mathcal{T}, s_q, K_s)

  3. foreach seeded consumer t∈Sqt \in S_q do

    1. P^(t)←{u:(u,t)∈Estate}\widehat{P}(t) \leftarrow\{u : (u,t) \in\mathcal{E}_{\mathrm{state}}\}

    2. foreach input-field name finf_{\mathrm{in}} of tt do

      1. F(t,fin)←{(u,fout):(u,fout,t,fin)∈Eprov∪Ecomp}F(t,f_{\mathrm{in}}) \leftarrow\{(u,f_{\mathrm{out}}) : (u,f_{\mathrm{out}},t,f_{\mathrm{in}}) \in\mathcal{E}_{\mathrm{prov}} \cup\mathcal{E}_{\mathrm{comp}}\}

      2. P(t,fin)←{u:∃fout,(u,fout)∈F(t,fin)}P(t,f_{\mathrm{in}}) \leftarrow\{u : \exists f_{\mathrm{out}}, (u,f_{\mathrm{out}}) \in F(t,f_{\mathrm{in}})\}

      3. if P(t,fin)≠∅P(t,f_{\mathrm{in}}) \ne\emptyset then

        1. (ufield,fout∗)←arg max⁡(u,fout)∈F(t,fin)cos⁡(hf(fout),hf(fin))(u_{\mathrm{field}}, f^*_{\mathrm{out}}) \leftarrow\underset{(u,f_{\mathrm{out}}) \in F(t,f_{\mathrm{in}})}{\operatorname{arg\,max}} \cos(h_f(f_{\mathrm{out}}), h_f(f_{\mathrm{in}}))

        2. udense←arg max⁡u∈P(t,fin)sq(u)u_{\mathrm{dense}} \leftarrow\underset{u \in P(t,f_{\mathrm{in}})}{\operatorname{arg\,max}} s_q(u)

        3. P^(t)←P^(t)∪{ufield,udense}\widehat{P}(t) \leftarrow\widehat{P}(t) \cup\{u_{\mathrm{field}}, u_{\mathrm{dense}}\}

  4. foreach provider u∈Tu \in\mathcal{T} do

    1. Cq(u)←{t∈Sq:u∈P^(t)}C_q(u) \leftarrow\{t \in S_q : u \in\widehat{P}(t)\}

    2. if Cq(u)≠∅C_q(u) \ne\emptyset then

      1. gq(u)←(sq(u)+∑t∈Cq(u)sq(t))/(1+∣Cq(u)∣)g_q(u) \leftarrow\left(s_q(u) + \sum_{t \in C_q(u)} s_q(t)\right)/(1 + |C_q(u)|)

      2. sq′(u)←max⁡{sq(u),gq(u)}s'_q(u) \leftarrow\max\{s_q(u), g_q(u)\}

  5. Tq←TopK⁡(T,sq′,Kc)T_q \leftarrow\operatorname{TopK}(\mathcal{T}, s'_q, K_c)

  6. return TqT_q

Acknowledgments

Zekai Li and Zhenke Liu contributed equally so they can prioritize their name in their resume.

References

  1. [1]Sahil Bansal, Sai Shruthi Sistla, Aarti Arikatala, and Sebastian Schreiber. 2025. Planning agents on ego-trip: Leveraging hybrid ego-graph entities for improved tool retrieval in enterprise task planning. In Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics, pages 903–918, Mumbai, India. The Asian Federation of Natural Language Processing and The Association for Computational Linguistics.
  2. [2]Aaron Blakeman, Aaron Grottafiori, Aarti Basant, Abhiha Gupta, Abhinav Khattar, Adi Reduchintala, Aditya Vavre, Akanksha Shukla, Akhiad Bercovich, Aleksander Ficek, and 1 others. 2025. Nvidia nemotron 3: Efficient and open intelligence. arXiv preprint arXiv:2512.20856.arxiv.org/abs/2512.20856
  3. [3]Norbert Braunschweiler, Rama Doddipatla, and Tudor-catalin Zorila. 2025. ToolReAGt: Tool retrieval for LLM-based complex task solution via retrieval augmented generation. In Proceedings of the 3rd Workshop on Towards Knowledgeable Foundation Models (KnowFM), pages 75–83, Vienna, Austria. Association for Computational Linguistics.
  4. [4]Wenjie Chen, Di Yao, Wenbin Li, Xuying Meng, Chang Gong, and Jingping Bi. 2026. GTool: Graph enhanced tool planning with large language model. In The Fourteenth International Conference on Learning Representations.arxiv.org/abs/2508.12725
  5. [5]Yanfei Chen, Jinsung Yoon, Devendra Singh Sachan, Qingze Wang, Vincent Cohen-Addad, Mohammad-hossein Bateni, Chen-Yu Lee, and Tomas Pfister. 2024. Re-invoke: Tool invocation rewriting for zero-shot tool retrieval. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 4705–4726, Miami, Florida, USA. Association for Computational Linguistics.arxiv.org/abs/2408.01875
  6. [6]Keyan Ding, Jing Yu, Junjie Huang, Yuchen Yang, Qiang Zhang, and Huajun Chen. 2025. Scitoolagent: a knowledge-graph-driven scientific agent for tool integration. Nature Computational Science, 5(10):962–972.arxiv.org/abs/2507.20280
  7. [7]Bowen Fang, Wen Ye, Yunyue Su, Jinghao Zhang, Qiang Liu, Yesheng Liu, Xin Sun, Shu Wu, Jiabing Yang, Baole Wei, and Liang Wang. 2026. Tool-Weaver: Weaving collaborative semantics for scalable tool use in large language models. In The Fourteenth International Conference on Learning Representations.arxiv.org/abs/2601.21947
  8. [8]Wei Fang and James R. Glass. 2026. Beyond single-shot: Multi-step tool retrieval via query planning. In Findings of the Association for Computational Linguistics: ACL 2026, pages 42119–42144, San Diego, California, United States. Association for Computational Linguistics.
  9. [9]Linfeng Gao, Yaoxiang Wang, Minlong Peng, Jialong Tang, Yuzhe Shang, Mingming Sun, and Jinsong Su. 2025. Tool graph retriever: Exploring dependency graph-based tool retrieval for large language models. arXiv preprint arXiv:2508.05152.arxiv.org/abs/2508.05152
  10. [10]Charles R. Harris, K. Jarrod Millman, Stéfan J. van der Walt, and 1 others. 2020. Array programming with NumPy. Nature, 585(7825):357–362.
  11. [11]Tenghao Huang, Dongwon Jung, Vaibhav Kumar, Mohammad Kachuee, Xiang Li, Puyang Xu, and Muhao Chen. 2024. Planning and editing what you retrieve for enhanced tool learning. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 975–988, Mexico City, Mexico. Association for Computational Linguistics.arxiv.org/abs/2404.00450
  12. [12]Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th symposium on operating systems principles, pages 611–626.arxiv.org/abs/2309.06180
  13. [13]Xukun Liu, Zhiyuan Peng, Xiaoyuan Yi, Xing Xie, Lirong Xiang, Yuchen Liu, and Dongkuan Xu. 2024. Toolnet: Connecting large language models with massive tools via tool graph. arXiv preprint arXiv:2403.00839.arxiv.org/abs/2403.00839
  14. [14]Xuan Lu, Haohang Huang, Rui Meng, Yaohui Jin, Wenjun Zeng, and Xiaoyu Shen. 2026. Tools are under-documented: Simple document expansion boosts tool retrieval. In The Fourteenth International Conference on Learning Representations.arxiv.org/abs/2510.22670
  15. [15]Bhrij Patel, Davide Belli, Amir Jalalirad, Maximilian Arnold, Aleksandr Ermolov, and Bence Major. 2026. Dynamic tool dependency retrieval for lightweight function calling. In Findings of the Association for Computational Linguistics: ACL 2026, pages 33649–33672, San Diego, California, United States. Association for Computational Linguistics.
  16. [16]Shishir G Patil, Huanzhi Mao, Fanjia Yan, Charlie Cheng-Jie Ji, Vishnu Suresh, Ion Stoica, and Joseph E. Gonzalez. 2025. The berkeley function calling leaderboard (BFCL): From tool use to agentic evaluation of large language models. In Forty-second International Conference on Machine Learning.
  17. [17]F. Pedregosa and 1 others. 2011. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12:2825–2830.arxiv.org/abs/1201.0490
  18. [18]Qwen Team. 2026. Qwen3.5: Towards native multi-modal agents.
  19. [19]Juan Ramos. 2003. Using TF-IDF to determine word relevance in document queries. In Proceedings of the First Instructional Conference on Machine Learning.

Paper details

vlnc:W_akve5thz
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 12

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents