WorksComputer Science
Behavior Quotient Learning for Low-Rank Adaptation of LLM Agents
Abstract
LLM-based agents rely on heterogeneous interaction capabilities to accomplish complex tasks. Existing approaches often distribute these capabilities across multiple LoRA adapters, which increases adapter storage requirements and introduces routing overhead during inference. A single LoRA avoids this overhead, but learning from diverse agent trajectories under a fixed rank budget presents two challenges. First, trajectories with different interaction traces and parameter gradients can induce equivalent changes in decision distributions, causing repeated updates to overemphasize redundant behavioral changes. Second, an aggregated update may exceed the rank budget of the adapter, and approximating it in weight space can distort the decision changes that it is intended to produce. We propose BQ-LoRA, a low-rank adaptation framework that organizes trajectory updates through a local behavior quotient manifold. It contains two modules, i.e., behavior quotient balancing (BQB) and decision preserving compression (DPC). BQB constructs the quotient manifold from decision distributions and reweights trajectory update directions according to their local density in the quotient tangent space. DPC projects the balanced gradient onto the intrinsic fixed rank tangent space and refactorizes the resulting target by jointly controlling effective weight error and distortion of decision distributions. Experiments on AppWorld and BrowseComp-Plus compare BQ-LoRA with standard LoRA and recent low-rank adaptation methods, while separate ablations evaluate the complementary contributions of both components.
Introduction
Agents built on large language models (LLMs) interact with external environments to accomplish specified goals ([48]; [?]). Despite broad capabilities, LLM-based agents often require task-specific training to operate effectively in new interaction settings ([51]; [13]). Low-Rank Adaptation (LoRA) ([?]) offers an efficient alternative to full-model fine-tuning by learning factorized updates to frozen backbone weights.
LoRA-based agent adaptation broadly follows two designs. The first assigns different agent capabilities to multiple adapters. In multi-agent workflows, separate LoRA groups support reasoning, execution, and summarization ([16]). Within tool-use workflows, adapter specialization separates tool selection from argument generation ([21]) and reasoning from tool use ([28]). More recent work trains a dedicated adapter for each identified capability gap, with inference-time routing selecting the relevant adapter ([22]). Managing multiple specialized adapters requires coordination during inference. Adapter-dependent states can also increase serving costs in multi-agent workflows ([20]). The second avoids adapter-management overhead by learning a shared low-rank update with a single adapter. For agent adaptation, the shared update is learned directly from interaction trajectories ([6]; [38]; [24]). This design incorporates supervision for reasoning and action generation into a common set of low-rank parameters, allowing the same adapter to support the agent across successive interaction steps. More generally, single-adapter LoRA variants improve the parameterization and optimization of low-rank updates to make more effective use of a limited trainable parameter budget ([59]; [26]; [52]).
However, behavioral redundancy among trajectory updates can limit how effectively a single fixed-rank adapter learns from diverse supervision. Behavioral redundancy arises when different trajectory updates induce similar changes in decision representations at the same reference histories. Trajectory synthesis broadens interaction coverage [7], while trajectory curation improves supervision quality [40]. Training utility is also assessed through model-dependent signals [12, 44, 23] and critical-step analysis [9, 25]. These methods identify useful supervision but do not compare the behavioral effects of different trajectory updates. Behavioral equivalence can occur despite substantial differences in trajectory text and parameter gradients. Under a fixed rank budget, repeated equivalent updates can disproportionately shape the learned adapter and leave updates with distinct effects on these reference decisions underrepresented. This gap raises a central question. How can a single fixed-rank LoRA learn diverse agent behaviors?
Addressing this question presents two challenges. Firstly, how to identify behaviorally equivalent trajectory updates? (CH1) Recorded trajectories alone do not reveal whether their training updates are behaviorally equivalent. Different interaction traces and parameter gradients may still induce similar changes in decision representations. TopoCurate [47] estimates semantic equivalence from similarities between recorded tool actions and environment observations. However, this criterion does not compare how trajectory updates change decision representations. Identifying behavioral equivalence therefore requires comparing these changes at the same reference histories. Secondly, how to preserve changes in decision representations under a fixed LoRA rank constraint? (CH2) Diverse trajectory updates can produce a desired effective update whose rank exceeds this constraint. Mapping such an update to a single fixed-rank adapter necessarily introduces approximation error. Weight-space error alone does not capture how compression distorts these changes. A suitable fixed-rank representative must preserve the effects of the target on decision representations and remain invariant to equivalent LoRA factorizations.
To address these challenges, we propose BQ-LoRA, a framework that learns diverse agent behaviors through a single fixed-rank LoRA adapter. Our key idea is to use a shared geometry of decision behavior to guide both trajectory weighting and low-rank compression. For CH1, we develop behavior quotient balancing (BQB), which compares trajectory update directions through their first-order effects on normalized decision representations at fixed interaction histories. This comparison identifies locally equivalent behavioral changes while disregarding parameter differences that are invisible to the reference decisions. BQB then reweights the directions according to their local concentration, reducing repeated contributions and giving greater relative influence to distinct behavioral changes. For CH2, we develop decision preserving compression (DPC), which projects the balanced direction onto the tangent space of the current fixed-rank adapter. The resulting target depends on the effective weights and remains invariant to equivalent LoRA factorizations. DPC compresses this target by jointly controlling weight approximation error and first-order decision distortion in the same behavior geometry. Together, the two modules connect behavioral coverage during training with preservation under the rank constraint, yielding one adapter that can be deployed directly.
We summarize our contributions as follows: (1) We formalize behavioral equivalence among updates through a local quotient geometry and introduce BQB to balance trajectory contributions according to their effects on agent decisions. (2) We develop DPC, which combines a factorization-invariant tangent projection with compression that accounts for both weight error and first-order decision distortion under the original LoRA rank budget. (3) We evaluate BQ-LoRA on AppWorld and BrowseComp-Plus with two backbone sizes, comparing performance and interaction counts against standard adaptation, single-adapter optimization, adapter specialization, and trajectory curation.
Related Work
Low-Rank Adaptation
Low-rank adaptation (LoRA) freezes pretrained weights and parameterizes task-specific updates using trainable low-rank factors [?]. Existing LoRA variants improve adaptation through informative initialization [35, 41, 56, 14, ?], adaptive rank allocation [54, 18, 37, 10, 29, 49], modified training procedures [17, 42, 7], and input-conditioned utilization of the low-rank subspace ([3, ?, 46, 61]). Geometry-aware methods further improve LoRA optimization through Riemannian or transformation-invariant preconditioning ([52, ?, ?]zhang2025reforarefactoredlow,castin2026balancedloraremoving), and manifold-based formulations ({cite:p}zheng2026lorasanefficient,bogachev2026lorameetsriemannian,li2025stella_subspacelearning,lion2025polarpolardecomposed). Multi-LoRA approaches use multiple low-rank parameter sets. General-purpose research studies the routing and composition of LoRA experts for multi-task adaptation and adapter reuse ({cite:p}huang2024lorahubefficientcrosstask,wu2024mixtureoflora,dou2024lormaoealleviatingworld,liu2024adamolefinetuninglarge,liao2025hmoramakingllms,xu2025meteoramultipletasksembedded,li2026loramixercoordinatemodular,zhuang2026ldmolelearnable). Recent work on agentic systems uses multiple specialized adapters to model heterogeneous capabilities and coordinate their activation across agent trajectories ({cite:p}han2025moragentparameterefficient,liu2026flexoraentropyguidedflexible,kadekodi2026dualtunedecoupledfinetuning,li2026reasoningtoolusecompete,kang2026tracecapabilitytargeted`).
Trajectory Redundancy
Agent trajectories with different interaction traces may provide overlapping training signals, making trajectory redundancy an important consideration in agent post-training. Existing work has primarily addressed this issue as a data-quality problem, using diversity-aware synthesis ([7]) and failure-driven curation ([40]) to improve the composition of training data before optimization. Moving from data composition to training effects, prior work estimates sample utility from optimization signals ([12, 44, 23]) and localizes useful supervision through critical-step selection ([9, 25]). These criteria identify useful supervision at different levels, but they do not determine whether distinct trajectory-induced updates produce equivalent changes in agent behavior. TopoCurate ([47]) examines this equivalence through a semantic quotient topology based on embedding similarities between tool actions and environment observations. Because this relation is defined over recorded interaction content, TopoCurate does not determine whether different training updates induce the same behavioral change. Interface perturbation studies further show that successful task execution does not necessarily reflect semantic tool-use competence ([15]). Our work defines trajectory redundancy through update-induced behavioral changes, using the resulting equivalence to balance supervision before representing the learned update with a single fixed-rank LoRA.
Preliminaries
A quotient manifold is obtained by identifying equivalent points in a smooth total space ([4]). Let be a smooth manifold equipped with an equivalence relation . For , its equivalence class and the resulting quotient set are defined as:
The canonical projection is given by . Equipped with a smooth structure, is a quotient manifold if is a smooth submersion, i.e., for every , where denotes the differential of at . The fiber through is , which is an embedded submanifold of . Its tangent space at defines the vertical space . Consequently, two tangent vectors represent the same tangent vector in if and only if their difference is vertical:
Method
Problem Statement
We consider an LLM-based agent with LoRA applied to weight matrices. For each adapted weight matrix indexed by , LoRA parameterizes the adapted weight as:
The matrices and are the frozen pretrained and adapted weights, respectively, and is the effective LoRA update. The update is parameterized by trainable factors and , with scaling coefficient . Their inner dimension defines the fixed rank budget and ensures .
An agent trajectory is denoted by , where is the task, is the history before step , is the agent output, and is the environment feedback. Each output consists of intermediate reasoning and a decision . The decision either advances the interaction or serves as the final task output. Given a collection of trajectories , we write for the effective updates and let denote the task loss associated with . Standard LoRA training minimizes the empirical task loss:
Framework Overview
The overall framework of BQ-LoRA is illustrated in Figure 1, showing how trajectory updates are balanced and incorporated into a single LoRA adapter. It consists of two modules, i.e., behavior quotient balancing (BQB) and decision preserving compression (DPC). First, BQB constructs normalized decision representations at fixed interaction histories and identifies effective updates that produce the same representations, forming a local behavior quotient geometry. Within this geometry, trajectory update directions are compared through their first-order effects on the reference decision representations. Their local concentration determines their relative weights, reducing the dominance of repeatedly represented behavioral changes and increasing the contribution of less represented ones. The weighted trajectory directions are then aggregated into a shared effective update direction. Second, DPC projects the balanced direction onto the fixed rank tangent space induced by the current LoRA factors and takes a step along it to form an effective-weight target. It then compresses this target under the original rank budget by jointly minimizing weight approximation error and the distortion of first-order decision responses measured using the same behavior geometry. The compressed target is refactorized into updated LoRA factors for the next training step. These two stages are repeated during training to balance trajectory contributions while controlling the behavioral distortion introduced by the rank constraint.
Figure 1. Overview of BQ-LoRA. BQB constructs a behavior quotient manifold and balances trajectory updates using quotient distances. DPC projects the balanced direction onto the fixed rank LoRA tangent space and refactorizes the resulting target while preserving decision changes.
Behavior Quotient Balancing
Behavior Quotient Geometry. Trajectory updates can differ substantially in parameter space while inducing the same change in agent decisions. Their differences may include components that are invisible at the decision histories of interest, causing parameter-space distances to overstate behavioral diversity. We therefore construct a behavior map from decision representations and identify effective updates that agree under this map. Let be the effective-update space with the product Frobenius inner product, and let denote the final representation passed to the frozen output head at history . For each reference decision , let count its generation positions and let extend with the portion of preceding position , for . These prefixes include the recorded reasoning and preceding decision tokens, so each representation describes the model state from which a decision token is predicted. We keep the prefixes fixed across effective updates and collect the resulting representations in the behavior map:
The normalization averages squared representation responses over generation positions, decisions, and trajectories. Using this map, we define behavioral equivalence on the reference collection:
On a sufficiently small open neighborhood of the current iterate where is smooth with constant differential rank, these equivalence classes form the behavior quotient manifold . The quotient map assigns each effective update to its behavior class, and the Euclidean geometry of induces a Riemannian metric on . At the current iterate, the quotient tangent space identifies directions whose difference lies in . Such directions have the same first-order response on the reference decisions even when their effective-weight components differ. This construction makes behavioral overlap measurable at the current model: invisible components contribute no quotient distance, while distinguishable decision responses remain separated.
Trajectory Update Balancing. The quotient geometry reveals overlap between trajectory updates, but uniform weighting still counts each trajectory as an independent contribution. When many trajectories induce similar decision responses, their repeated contributions can dominate the aggregate direction. Behavioral changes represented by fewer trajectories then receive less influence, even when they provide distinct responses on the reference collection. We use the local concentration of quotient tangent directions to adjust these relative contributions. For each trajectory, we take its task direction and map it to the quotient tangent . Its behavior response describes how training on trajectory would change all reference decision representations to first order. We compare two such directions under the quotient metric :
The right-hand expression evaluates the quotient tangent distance directly from the two behavior responses. It can be computed using Jacobian-vector products without explicitly constructing quotient coordinates. We then use Gaussian affinities [36] to accumulate the overlap surrounding each direction and set its relative weight:
Here is a data-dependent bandwidth that determines the scale of the neighborhood comparison. A larger means that more of the trajectory collection produces responses close to that of . The exponent controls how strongly this overlap reduces the relative weight of a trajectory. The normalization keeps , preserving the total weight of the empirical objective, and recovers uniform weighting. For groups of similar responses with sufficiently small overlap between groups, approximately equalizes their total contributions. Holding the computed weights fixed during the task update, we combine the original trajectory directions:
By linearity of , the aggregate induces the corresponding weighted mean of the trajectory quotient tangents. For , this reduces the relative contribution of densely represented behavioral changes and increases that of less represented changes. Each task loss continues to determine the sign and direction of its own update, while the positive weights control its contribution to the shared adapter. BQB therefore adjusts behavioral coverage while retaining the original task supervision, without requiring predefined capability partitions.
Decision Preserving Compression
Intrinsic Update Projection. BQB aggregates trajectory directions in the effective-update space, whereas the stored LoRA adapter changes its weights through low-rank factors. The balanced direction may consequently contain components that these factors cannot realize to first order at the current iterate. To obtain a locally attainable update, we project it onto the space of effective directions induced by perturbing the factors. This projection should depend on the current effective weights, so that choosing an equivalent factorization does not change the target. For each layer , let denote the manifold of matrices in with rank . At an iterate with , differentiating yields the tangent space of effective directions attainable to first order through the LoRA factors [42]:
The two terms describe the effective changes obtained by varying either factor, and their sum contains all first-order directions attainable at . Let and have orthonormal columns spanning and , respectively. We denote the Frobenius orthogonal projection onto by . The projected layerwise direction is [1]:
The first two terms retain the components accessible through the current column or row space, while the last term subtracts their shared component. With a learning rate , the projected direction defines the layerwise affine target:
We collect these layerwise targets as . The Frobenius projection gives the closest locally attainable direction to , minimizing the alteration of the balanced direction within the tangent constraint. Because the projectors and depend only on the column and row spaces of , the projected direction and the resulting target are invariant to equivalent LoRA factorizations. This gives the compression stage a target defined by the effective weights of the adapter and its local ability to change them. The first-order decision change induced by is the reachable target effect that the subsequent refactorization aims to preserve.
Fixed-Rank Refactorization. The tangent projection ensures that the update direction is attainable to first order, but a finite step along that direction need not remain on the fixed-rank manifold. In particular, the affine target can be expressed as the sum of two matrices of rank at most , giving . When , the target cannot be represented under the original LoRA rank budget and must be compressed. Equal-sized weight errors can cause different changes in the reference decisions because the behavior map is more sensitive to some residual directions than others. Weight-space approximation alone therefore does not distinguish which parts of the target matter most for its decision effect. We use the behavior geometry from BQB to guide the selection of a rank-constrained representative. Let contain candidate updates satisfying for every layer. Each candidate is further constrained by and . These constraints restrict the refactorization to the column and row spaces already present in the projected target. Within this candidate set, we select the next effective update by jointly measuring weight error and behavioral distortion:
The first term measures the effective-weight error introduced by rank reduction. For the second term, linearity gives . It thus compares the first-order decision responses induced by the candidate step and the target step at the same current iterate. The reference histories and the differential remain fixed during refactorization, so this term measures the distortion caused by replacing with . The coefficient controls the tradeoff between effective-weight approximation and decision preservation. Because the behavior response is evaluated across all adapted layers jointly, the criterion accounts for their combined effect on the reference decisions. The restricted target spaces permit optimization through cores of size at most . This objective favors representatives that approximate the target weights while retaining the decision changes identified by the same geometry used for trajectory balancing. The rank constraints guarantee factors and satisfying . The compressed target can be stored as a single LoRA adapter under the original rank budgets, with decision distortion included in the compression criterion.
| Model | Method | AppWorld Test-N TGC | AppWorld Test-N SGC | AppWorld Test-C TGC | AppWorld Test-C SGC | BrowseComp-Plus BM25 Accuracy | BrowseComp-Plus BM25 Recall | BrowseComp-Plus Embedding Accuracy | BrowseComp-Plus Embedding Recall |
| Qwen3.5-4B | Base Agent | 27.98 | 10.71 | 11.99 | 2.16 | 20.00 | 22.43 | 36.67 | 44.77 |
| Qwen3.5-4B | Full | 58.33 | 39.29 | 40.29 | 21.58 | 26.00 | 26.60 | 44.00 | 48.10 |
| Qwen3.5-4B | LoRA | 52.98 | 32.14 | 35.49 | 17.99 | 20.67 | 22.28 | 40.00 | 45.77 |
| Qwen3.5-4B | LoRA-S | 58.93 | 41.07 | 39.33 | 19.42 | 26.67 | 27.07 | 42.00 | 47.30 |
| Qwen3.5-4B | MoRAgent | 60.12 | 42.86 | 40.05 | 21.58 | 25.33 | 26.40 | 43.33 | 48.30 |
| Qwen3.5-4B | DART | 56.55 | 37.50 | 39.09 | 20.14 | 28.00 | 28.90 | 45.33 | 49.10 |
| Qwen3.5-4B | TopoCurate | 55.95 | 37.50 | 41.01 | 23.02 | 26.00 | 27.60 | 43.33 | 48.80 |
| Qwen3.5-4B | BQ-LoRA | 58.93 | 44.64 | 43.65 | 24.46 | 28.67 | 29.30 | 45.33 | 50.10 |
| Qwen3.5-9B | Base Agent | 36.31 | 19.64 | 23.02 | 10.79 | 24.00 | 24.08 | 44.67 | 44.20 |
| Qwen3.5-9B | Full | 69.64 | 53.57 | 49.64 | 27.34 | 30.00 | 28.40 | 49.33 | 48.20 |
| Qwen3.5-9B | LoRA | 60.71 | 42.86 | 45.08 | 20.86 | 26.00 | 24.83 | 46.00 | 45.20 |
| Qwen3.5-9B | LoRA-S | 66.67 | 51.79 | 47.72 | 25.18 | 29.33 | 27.60 | 48.00 | 47.50 |
| Qwen3.5-9B | MoRAgent | 69.05 | 55.36 | 49.16 | 28.06 | 29.33 | 28.10 | 49.33 | 49.10 |
| Qwen3.5-9B | DART | 66.67 | 48.21 | 48.20 | 25.90 | 31.33 | 30.20 | 50.67 | 49.70 |
| Qwen3.5-9B | TopoCurate | 64.88 | 48.21 | 50.12 | 29.50 | 29.33 | 29.10 | 49.33 | 49.30 |
| Qwen3.5-9B | BQ-LoRA | 68.45 | 55.36 | 53.00 | 31.65 | 32.67 | 31.10 | 51.33 | 50.90 |
Table 1. Task performance on AppWorld and BrowseComp-Plus (%, ↑).
Experiments
Experimental Setups
Datasets. We conduct experiments on AppWorld [39] and BrowseComp-Plus [7]. AppWorld evaluates agents on multi-step tasks in a stateful world of simulated applications. Given a natural language request, the agent invokes application APIs and responds to environment feedback to complete the task. Programmatic tests verify task completion and the resulting application states. BrowseComp-Plus evaluates deep search agents on difficult questions that require iterative retrieval and evidence synthesis. Agents search a fixed document corpus containing human-verified supporting evidence. For both benchmarks, we perform supervised fine-tuning on collected successful trajectories.
Baselines. We compare BQ-LoRA with seven baselines. Standard adaptation baselines include: (1) Base, which evaluates the pretrained model without adaptation, (2) Full FT, which updates all model parameters, and (3) LoRA [?], which introduces low-rank updates into each adapted layer. We then compare two multi-LoRA methods designed for agent tuning: (4) MoRAgent [16], which assigns separate LoRA groups to the reasoner, executor, and summarizer roles, and (5) DART [28], which separates reasoning and tool use updates across distinct low-rank modules. For single-LoRA optimization, we include (6) LoRA-S [59], which optimizes low-rank factors on a quotient manifold through a Sylvester equation. For trajectory curation, we include (7) TopoCurate [47], which uses a semantic quotient topology of multi-trial interactions to select informative trajectories.
| Model | Method | AppWorld Test-N Avg. Steps | AppWorld Test-C Avg. Steps | BrowseComp-Plus BM25 Search Calls | BrowseComp-Plus Embedding Search Calls |
| Qwen3.5-4B | Base Agent | 31.28 | 36.64 | 16.95 | 15.54 |
| Qwen3.5-4B | Full | 17.75 | 26.10 | 16.70 | 11.65 |
| Qwen3.5-4B | LoRA | 18.26 | 27.12 | 17.73 | 14.35 |
| Qwen3.5-4B | LoRA-S | 18.65 | 27.20 | 17.20 | 13.25 |
| Qwen3.5-4B | MoRAgent | 17.10 | 25.35 | 16.85 | 12.35 |
| Qwen3.5-4B | DART | 18.00 | 26.25 | 16.15 | 11.90 |
| Qwen3.5-4B | TopoCurate | 17.35 | 24.90 | 15.75 | 12.05 |
| Qwen3.5-4B | BQ-LoRA | 16.70 | 24.10 | 15.35 | 11.35 |
| Qwen3.5-9B | Base Agent | 26.20 | 29.92 | 15.65 | 13.16 |
| Qwen3.5-9B | Full | 17.45 | 24.25 | 15.50 | 11.10 |
| Qwen3.5-9B | LoRA | 17.16 | 24.96 | 17.92 | 13.48 |
| Qwen3.5-9B | LoRA-S | 17.65 | 25.75 | 16.95 | 12.65 |
| Qwen3.5-9B | MoRAgent | 16.10 | 23.60 | 16.30 | 11.80 |
| Qwen3.5-9B | DART | 16.90 | 24.20 | 15.75 | 11.55 |
| Qwen3.5-9B | TopoCurate | 16.65 | 23.15 | 15.30 | 11.65 |
| Qwen3.5-9B | BQ-LoRA | 15.95 | 22.70 | 15.05 | 10.85 |
Table 2. Mean interaction counts on AppWorld (steps) and BrowseComp-Plus (search calls) (↓).
Performance Evaluation
Main Results. Tables 1 and 2 summarize task performance and interaction counts, respectively. We make the following observations: (1) Standard LoRA improves on the base agent, but its gains are limited on evidence retrieval and it remains well below the stronger baselines. This gap suggests that standard low-rank adaptation leaves room for better allocation of adapter capacity across agent behaviors. (2) LoRA-S consistently outperforms standard LoRA, supporting the value of eliminating redundant degrees of freedom in the low-rank factorization. However, its equivalence relation identifies factorizations of the same effective weights and does not address redundancy between trajectory-induced behavioral changes. (3) Multiple adapters provide additional benefits through capability specialization. MoRAgent performs best on Test-N among the low-rank baselines, while DART leads these baselines on BrowseComp-Plus. However, these gains rely on separate adapters and predefined role or capability partitions, and their relative strengths vary across tasks. (4) TopoCurate improves trajectory selection by reducing semantic repetition and retaining informative interactions, yielding the strongest baseline performance on Test-C. However, similarity between recorded actions and observations does not determine whether optimizing two trajectories will induce the same behavioral change. Its curation criterion therefore leaves redundancy in the resulting behavioral changes unresolved. (5) Averaged over the eight reported performance metrics, BQ-LoRA outperforms the strongest baseline, MoRAgent, by 2.14 and 2.12 percentage points for the 4B and 9B backbones, respectively. It also achieves the lowest interaction counts in all evaluated settings. Balancing trajectory updates and preserving their decision effects during compression enable strong performance and efficient interaction in a single fixed-rank adapter.
Ablation Study. Table 3 evaluates variants with each module removed. (1) Removing BQB reduces Test-C performance by 3.57 and 3.83 percentage points for the 4B and 9B backbones, respectively, and lowers BrowseComp-Plus accuracy with both retrievers. This supports balancing trajectory updates by their effects on decision representations. The reduction is larger on Test-C than on Test-N for both backbones, indicating that the benefit of balancing varies across evaluation settings. (2) Removing DPC reduces performance in all eight settings, with average drops across the four metrics of 3.64 and 3.57 percentage points for the 4B and 9B backbones, respectively. This supports preserving decision changes when compressing the balanced update under the fixed rank budget. The consistent drops across AppWorld and both BrowseComp-Plus retrievers suggest that decision preservation contributes across different forms of agent interaction. (3) The full method leads in seven of eight settings. Without BQB, the 4B model scores 0.20 percentage points higher on Test-N. The two modules provide complementary benefits, although balancing does not improve every setting. Both partial variants also outperform standard LoRA in every setting, indicating that each component retains useful gains when applied without the other. The ranking of the partial variants is consistent across backbones: retaining DPC gives higher mean performance than retaining BQB alone. Relative to standard LoRA, the full method improves the mean of the four ablation metrics by 6.86 and 6.92 percentage points for the 4B and 9B backbones, respectively.
| Model | Method | Test-N | Test-C | BM25 | Embedding |
| Qwen3.5-4B | LoRA | 52.98 ± 1.10 | 35.49 ± 1.45 | 20.67 ± 1.30 | 40.00 ± 1.00 |
| Qwen3.5-4B | BQ-LoRA w/o BQB | 59.13 ± 1.00 | 40.08 ± 1.60 | 27.07 ± 1.40 | 43.10 ± 1.15 |
| Qwen3.5-4B | BQ-LoRA w/o DPC | 55.38 ± 1.35 | 39.59 ± 1.75 | 24.67 ± 1.60 | 42.40 ± 1.35 |
| Qwen3.5-4B | BQ-LoRA | 58.93 ± 1.05 | 43.65 ± 1.40 | 28.67 ± 1.25 | 45.33 ± 1.05 |
| Qwen3.5-9B | LoRA | 60.71 ± 1.00 | 45.08 ± 1.30 | 26.00 ± 1.20 | 46.00 ± 0.95 |
| Qwen3.5-9B | BQ-LoRA w/o BQB | 67.67 ± 0.90 | 49.17 ± 1.35 | 30.13 ± 1.25 | 49.10 ± 1.00 |
| Qwen3.5-9B | BQ-LoRA w/o DPC | 63.51 ± 1.20 | 49.78 ± 1.55 | 29.20 ± 1.45 | 48.70 ± 1.15 |
| Qwen3.5-9B | BQ-LoRA | 68.45 ± 1.00 | 53.00 ± 1.25 | 32.67 ± 1.15 | 51.33 ± 0.95 |
Table 3. Ablation results on AppWorld (TGC) and BrowseComp-Plus (accuracy) (%, ↑).
Hyperparameter Sensitivity. We study the balancing exponent and compression coefficient . We test and , varying one coefficient at a time while fixing the other at 1. Figure 2 reports averages over the eight performance metrics for both backbones. (1) Performance peaks at for 4B and for 9B. At , all trajectories receive equal weight, providing a reference without density-based balancing. Increasing reduces the relative contribution of densely represented responses. Both backbones improve over uniform weighting, with the 4B curve flattening near the upper end and the 9B curve continuing to rise. (2) Both backbones perform best at , with nearby values remaining competitive. A small gives greater relative emphasis to weight approximation, while a large increases the penalty on local decision distortion. The effective-weight error remains part of the objective at every tested value, so this sweep varies the balance between the two compression criteria. Performance decreases toward both ends of the tested range, supporting an intermediate tradeoff between weight fidelity and decision preservation. This pattern holds across a sweep spanning two orders of magnitude in and is shared by both backbone sizes.

Figure 2. Hyperparameter sensitivity.
Conclusion
We presented BQ-LoRA for adapting LLM agents with a single fixed-rank LoRA. BQB balances trajectory updates in a local behavior quotient geometry, reducing repeated contributions from updates with equivalent effects on decision representations. DPC projects the balanced update and compresses the resulting target while controlling decision distortion. Together, they preserve the original rank budget and require no capability-specific routing. Experiments on AppWorld and BrowseComp-Plus show strong task performance and reduced interaction counts.
References
- [1]Fengmiao Bian, Jinyang ZHENG, Ziyun Liu, Jianzhuo Luo, and Jian-Feng CAI. Finding low-rank matrix weights in dnns via riemannian optimization: Radagrad and radamw. In D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen (eds.), Advances in Neural Information Processing Systems, volume 38, Main Conference, pp. 59916–59950. Curran Associates, Inc., 2025. doi: 10.52202/085713-2004. URL https://proceedings.neurips.cc/paper_files/paper/2025/file/5679173c400b332796426e443ab5ea0d-Paper-Conference.pdf.DOI
- [3]Shubhankar Borse, Shreya Kadambi, Nilesh Prasad Pandey, Kartikeya Bhardwaj, Viswanath Ganapathy, Sweta Priyadarshi, Risheek Garrepalli, Rafael Esteves, Munawar Hayat, and Fatih Porikli. Fourier: Fourier low-rank adaptation. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (eds.), Advances in Neural Information Processing Systems, volume 37, pp. 71504–71539. Curran Associates, Inc., 2024. doi: 10.52202/079017-2284. URL https://proceedings.neurips.cc/paper_files/paper/2024/file/83960718b4d12f7999852f61bf1cf00f-Paper-Conference.pdf.DOI
- [4]Nicolas Boumal. An Introduction to Optimization on Smooth Manifolds. Cambridge University Press, 2023. doi: 10.1017/9781009166164.DOI
- [6]Baian Chen, Chang Shu, Ehsan Shareghi, Nigel Collier, Karthik Narasimhan, and Shunyu Yao. Fireact: Toward language agent fine-tuning, 2023. URL https://arxiv.org/abs/2310.05915.
- [7]Guhong Chen, Chenghao Sun, Cheng Fu, Qiyao Wang, Zhihong Huang, ChaoPeng Wei, Guangxu Chen, Feitong Fang, Ahmadreza Argha, Bing Zhao, Xander Xu, Qi Han, Hamid Alinejad-Rokny, Qiang Qu, Binhua Li, Shiwen Ni, Min Yang, HU Wei, and Yongbin Li. Beyond quantity: Trajectory diversity scaling for code agents. In Maria Liakata, Vivian P. Morreira, Jiajun Zhang, and David Jurgens (eds.), Findings of the Association for Computational Linguistics: ACL 2026, pp. 15676–15691, San Diego, California, United States, July 2026a. Association for Computational Linguistics. ISBN 979-8-89176-395-1. doi: 10.18653/v1/2026.findings-acl.768. URL https://aclanthology.org/2026.findings-acl.768/.DOI
- [9]Zijian Chen, Xueguang Ma, Shengyao Zhuang, Ping Nie, Kai Zou, Sahel Sharifymoghaddam, Andrew Liu, Joshua Green, Kshama Patel, Ruoxi Meng, Mingyi Su, Yanxi Li, Haoran Hong, Xinyu Shi, Xuye Liu, Hosna Oyarhosseini, Nandan Thakur, Crystina Zhang, Luyu Gao, Wenhu Chen, and Jimmy Lin. BrowseComp-plus: A fair and disentangled evaluation benchmark for deep search agents. In Maria Liakata, Viviane P. Moreira, Jiajun Zhang, and David Jurgens (eds.), Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 22349–22370, San Diego, California, United States, July 2026c. Association for Computational Linguistics. ISBN 979-8-89176-390-6. doi: 10.18653/v1/2026.acl-long.1023. URL https://aclanthology.org/2026.acl-long.1023/.
- [10]Xuan Cui, Hui Yue Li, Run Zeng, Yunfei Zhao, Jinrui Qian, Wei Duan, Bo Liu, and Zhappeng Zhou. IGU-LoRA: Adaptive rank allocation via integrated gradients and uncertainty-aware scoring. In The Fourteenth International Conference on Learning Representations, 2026. URL https://openreview.net/forum?id=MnToYQx9My.arxiv.org/abs/2603.13792
- [12]Ruishan Fang, Siyuan Lu, Chenyi Zhuang, and Tao Lin. Rods: Reward-driven online data synthesis for multi-turn tool-use agents, 2026a. URL https://arxiv.org/abs/2606.19047.
- [13]Runnan Fang, Shihao Cai, Baixuan Li, Jialong Wu, Guangyu Li, Wenbiao Yin, Xinyu Wang, Xiaobin Wang, Liangcai Su, Zhen Zhang, Shibin Wu, Zhengwei Tao, Yong Jiang, Pengjun Xie, Ningyu Zhang, Fei Huang, Wentao Zhang, and Jingren Zhou. Towards general agentic intelligence via environment scaling. In Maria Liakata, Viviane P. Moreira, Jiajun Zhang, and David Jurgens (eds.), Findings of the Association for Computational Linguistics: ACL 2026, pp. 17610–17621, San Diego, California, United States, July 2026b. Association for Computational Linguistics. ISBN 979-8-89176-395-1. doi: 10.18653/v1/2026.findings-acl.872. URL https://aclanthology.org/2026.findings-acl.872/.DOI
- [14]Zhi-Quan Feng, Ying-Jia Lin, and Hung-Yu Kao. Learning in the fisher subspace: A guided initialization for LoRA fine-tuning. In Forty-third International Conference on Machine Learning, 2026. URL https://openreview.net/forum?id=P76Q8Pi1E9.arxiv.org/abs/2605.01046
- [15]Weizheng Gu, Chengze Li, Zhuohao Yu, Mengyuan Sun, Zhibang Yang, Wei Wang, Hongrui Jia, Shikun Zhang, and Wei Ye. What do agents learn from trajectory-SFT: Semantics or interfaces? In Forty-third International Conference on Machine Learning, 2026. URL https://openreview.net/forum?id=uli3XoSXQG.
- [16]Jing Han, Binwei Yan, Tianyu Guo, Zheyuan Bai, Mengyu Zheng, Hanting Chen, and Ying Nie. MoRAgent: Parameter efficient agent tuning with mixture-of-roles. In Aarti Singh, Maryam Fazel, Daniel Hsu, Simon Lacoste-Julien, Felix Berkenkamp, Tegan Maharaj, Kiri Wagstaff, and Jerry Zhu (eds.), Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pp. 21916–21931. PMLR, 13–19 Jul 2025. URL https://proceedings.mlr.press/v267/han25j.html.arxiv.org/abs/2512.21708
- [17]Soufiane Hayou, Nikhil Ghosh, and Bin Yu. LoRA+: Efficient low rank adaptation of large models. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 17783–17806. PMLR, 21–27 Jul 2024. URL https://proceedings.mlr.press/v235/hayou24a.html.
- [18]haonan he, Peng Ye, Yuchen Ren, yuan yuan, LuyangZhou, ShucunJu, and lei chen. Gora: Gradient-driven adaptive low rank adaptation. In D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen (eds.), Advances in Neural Information Processing Systems, volume 38, pp. 114074–114102. Curran Associates, Inc., 2025. doi: 10.52202/085713-3805. URL https://proceedings.neurips.cc/paper_files/paper/2025/file/a5e4907a40c0d0cb8433a35c714ba9d79-paper_Conference.pdf. Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=nZeVKeeFYf9.
- [20]Hyesung Jeon, Hyeongju Ha, and Jae-Joon Kim. LRAgent: Efficient KV cache sharing for multi-loRA LLM agents. In Forty-third International Conference on Machine Learning, 2026. URL https://openreview.net/forum?id=ooRukY3zjQ.arxiv.org/abs/2602.01053
- [21]Rohan Kadekodi, Zhan Jin, Keisuke Kamahori, Yile Gu, Sean Khatiri, Noah H. Bavindirli, Sergey Gorbunov, and Baris Kasikci. Dualtune: Decoupled fine-tuning for on-device agentic systems, 2026. URL https://openreview.net/forum?id=YwPx0y0RE.DOI
- [22]Hangoo Kang, Tarun Suresh, Jon Saad-Falcon, and Azalia Mirhoseini. TRACE: Capability-targeted agentic training. In Second Workshop on Agents in the Wild: Safety, Security, and Beyond, 2026. URL https://openreview.net/forum?id=p37UqCmcxG.arxiv.org/abs/2604.05336
- [23]Yongchan Kwon, Eric Wu, Kevin Wu, and James Y Zou. Datainf: Efficiently estimating data influence in lora-tuned llms and diffusion models. In B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun (eds.), International Conference on Learning Representations, volume 2024, pp. 21921–21942, 2024. URL https://proceedings.iclr.cc/paper_files/paper/2024/file/5e84a0f233611a1dc8fb794dc52415a3-Paper-Conference.pdf.arxiv.org/abs/2310.00902
- [24]Hanyu Lai, Junjie Gao, Xiao Liu, Yifan Xu, Shudan Zhang, Yuxiao Dong, and Jie Tang. AndroidGen: Building an android language agent under data scarcity. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2727–2749, Vienna, Austria, July 2025. Association for Computational Linguistics. ISBN 979-8-89176-251-0. doi: 10.18653/v1/2025.acl-long.138. URL https://aclanthology.org/2025.acl-long.138/.DOI
- [25]Mukai Li, Qingcheng Zeng, Tianqing Fang, Zhenwen Liang, Linfeng Song, Qi Liu, Haitao Mi, and Dong Yu. Verified critical step optimization for LLM agents. In Maria Liakata, Viviane P. Moreira, Jiajun Zhang, and David Jurgens (eds.), Findings of the Association for Computational Linguistics: ACL 2026, pp. 39627–39639, San Diego, California, United States, July 2026. Association for Computational Linguistics. ISBN 979-8-89176-395-1. doi: 10.18653/v1/2026.findings-acl.1974. URL https://aclanthology.org/2026.findings-acl.1974/.DOI
- [26]Shiwei Li, Xiandi Luo, Haozhao Wang, Xing Tang, Ziqiang Cui, Dugang Liu, Yuhua Li, Xiuqiang He, and Ruixuan Li. Beyond higher rank: Token-wise input-output projections for efficient low-rank adaptation. In D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen (eds.), Advances in Neural Information Processing Systems, volume 38, Main Conference, pp. 136414–136435. Curran Associates, Inc., 2025a. doi: 10.5555/085713-4552. URL https://proceedings.neurips.cc/paper_files/paper/2025/file/c74429a930838b5bac03c681f0a0ce11-Paper-Conference.pdf.arxiv.org/abs/2510.23123
- [28]Yu Li, Mingyang Yi, Xiuyu Li, Ju Fan, Fuxin Jiang, Binbin Chen, Peng Li, Jie Song, and Tieying Zhang. Reasoning and tool-use compete in agentic rl:from quantifying interference to disentangled tuning, 2026c. URL https://arxiv.org/abs/2602.00994.
- [29]Zhizhong Li, Sina Sajjadmanesh, Jingtao Li, and Lingjuan Lyu. Stella: Subspace learning in low-rank adaptation using stiefel manifold. In D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen (eds.), Advances in Neural Information Processing Systems, volume 38, Main Conference, pp. 75066–75092. Curran Associates, Inc., 2025b. doi: 10.52202/085713-2521. URL https://proceedings.neurips.cc/paper_files/paper/2025/file/6cb0c6e7d50d5d65613f0456ca85e2db-Paper-Conference.pdf.DOI
- [35]Fanxu Meng, Zhaohui Wang, and Muhan Zhang. Pissa: Principal singular values and singular vectors adaptation of large language models. In A. Glorbeson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (eds.), Advances in Neural Information Processing Systems, volume 37, pp. 121038–121072. Curran Associates, Inc., 2024. doi: 10.52202/079017-3846. URL https://proceedings.neurips.cc/paper_files/paper/2024/file/db36f46d03cc9e3a2a5e10b93e6428f2-Paper-Conference.pdf.DOI
- [36]Boaz Nadler, Stephane Lafon, Ioannis Kevrekidis, and Ronald Coifman. Diffusion maps, spectral clustering and eigenfunctions of fokker-planck operators. In Y. Weiss, B. Schölkopf, and J. Platt (eds.), Advances in Neural Information Processing Systems, volume 18. MIT Press, 2005. URL https://proceedings.neurips.cc/paper_files/paper/2005/file/2a0f97f81755e2878b264adf39cba68e-Paper.pdf.arxiv.org/abs/math/0506090
- [37]Fabian Paischer, Lukas Hauzenberger, Thomas Schmied, Benedikt Alkin, Marc Deisenroth, and Sepp Hochreiter. Parameter efficient fine-tuning via explained variance adaptation. In D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen (eds.), Advances in Neural Information Processing Systems, volume 38, pp. 46215–46267. Curran Associates, Inc., 2025. doi: 10.52202/085713-1539. URL https://proceedings.neurips.cc/paper_files/paper/2025/file/41d33bd41fd44bd9dba0e092047cf213-Paper-Conference.pdf.DOI
- [38]Cheng Qian, Emre Can Acikgoz, Hongru Wang, Xiusi Chen, Ayirup Sil, Dilek Hakkani-Tür, Gokhan Tur, and Heng Ji. SMART: Self-aware agent for tool overuse mitigation. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Findings of the Association for Computational Linguistics: ACL 2025, pp. 4604–4621, Vienna, Austria, July 2025. Association for Computational Linguistics. ISBN 979-8-89176-256-5. doi: 10.18653/ v1/2025.findings-acl.239. URL https://aclanthology.org/2025.findings-acl.239/.arxiv.org/abs/2502.11435
- [39]Harsh Trivedi, Tushar Khot, Mareike Hartmann, Ruskin Manku, Vinty Dong, Edward Li, Shashank Gupta, Ashish Sabharwal, and Niranjan Balasubramanian. AppWorld: A controllable world of apps and people for benchmarking interactive coding agents. In ACL, 2024.arxiv.org/abs/2407.18901
- [40]Dingzirui Wang, Xuanliang Zhang, Keyan Xu, Qingfu Zhu, and Wanxiang Che. Curateevo: Data-curation evolving for agentic post-training, 2026. URL https://arxiv.org/abs/2607.06140.
- [41]Shaowen Wang, Linxi Yu, and Jian Li. Lora-ga: Low-rank adaptation with gradient approximation. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (eds.), Advances in Neural Information Processing Systems, volume 37, pp. 54905–54931. Curran Associates, Inc., 2024. doi: 10.52202/079017-1741. URL https://proceedings.neurips.cc/paper_files/paper/2024/file/62c4718cc34f6a0a62fb81c4a2095a1-Paper-Conference.pdf.DOI
- [42]Zhengbo Wang, Jian Liang, Ran He, Zilei Wang, and Tieniu Tan. LoRA-pro: Are low-rank adapters properly optimized? In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=gTwRMU3lJ5.arxiv.org/abs/2407.18242
- [44]Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. LESS: Selecting influential data for targeted instruction tuning. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 54104–54132. PMLR, 21–27 Jul 2024. URL https://proceedings.mlr.press/v235/xia24c.html.arxiv.org/abs/2402.04333
- [46]Guang Yang, Changhao Guan, Chao Huang, Yufeng Chen, and Kaiyu Huang. Adaptive utilization of low-rank adaptation via conditioned gating. In Forty-third International Conference on Machine Learning, 2026a. URL https://openreview.net/forum?id=cA7SvKSTdH.
- [47]Jinluan Yang, Yuxin Liu, Zhengyu Chen, Chengcheng Han, Yueqing Sun, Qi Gu, Hui Su, Xunliang Cai, Fei Wu, and Kun Kuang. Topocurate:modeling interaction topology for tool-use agent training, 2026b. URL https://arxiv.org/abs/2603.01714.
- [48]Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=WE_vluYUL-X.arxiv.org/abs/2210.03629
- [49]Jingqi Ye, haonan he, Minglei Li, Fujun Han, Tao Chen, and Peng Ye. Gradient intrinsic dimensionality alignment: Narrowing the gap between low-rank adaptation and full fine-tuning. In The Fourteenth International Conference on Learning Representations, 2026. URL https://openreview.net/forum?id=koBvnQ6pUx.
- [51]Aohan Zeng, Mingdao Liu, Rui Lu, Bowen Wang, Xiao Liu, Yuxiao Dong, and Jie Tang. Agent-Tuning: Enabling generalized agent abilities for LLMs. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguistics: ACL 2024, pp. 3053–3077, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.findings-acl.181. URL https://aclanthology.org/2024.findings-acl.181/.DOI
- [52]Fangzhao Zhang and Mert Pilanci. Riemannian preconditioned LoRA for fine-tuning foundation models. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 59641–59669. PMLR, 21–27 Jul 2024. URL https://proceedings.mlr.press/v235/zhang24ax.html.arxiv.org/abs/2402.02347
- [54]Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. Adaptive budget allocation for parameter-efficient fine-tuning. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=1q62uWRJjiY.arxiv.org/abs/2303.10512
- [56]Yilang Zhang, Bingcong Li, and Georgios Giannakis. Refora: Refactored low-rank adaptation for efficient fine-tuning of large models. In D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen (eds.), Advances in Neural Information Processing Systems, volume 38, Main Conference, pp. 92847–92878. Curran Associates, Inc., 2025a. doi: 10.52202/085713-3106. URL https://proceedings.neurips.cc/paper_files/paper/2025/file/85c1ee0b3cdf49b84f14b631b01f2711-Paper-Conference.pdf.DOI
- [59]Jinyang Zheng and Tong Wu. Lora-s: An efficient low rank adaptation scheme via sylvester equation. In C. Vondrick, B. Hariharan, C. Raffel, L. Pinto, D. Yang, and A. Faust (eds.), International Conference on Learning Representations, volume 2026, pp. 101788–101811, 2026. URL https://proceedings.iclr.cc/paper_files/paper/2026/file/a6a90bcc2aa470c3871b2d39a67d26e8-Paper-Conference.pdf.
- [61]Heming Zou, Yunliang Zang, Wutong Xu, Yao Zhu, and Xiangyang Ji. Flylora: Boosting task decoupling and parameter efficiency via implicit rank-wise mixture-of-experts. In D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen (eds.), Advances in Neural Information Processing Systems, volume 38, Main Conference, pp. 10386–10419. Curran Associates, Inc., 2025. doi: 10.52202/085713-0349. URL https://proceedings.neurips.cc/paper_files/paper/2025/file/0f234479d4be0102b6a9ec5ebf75e881-Paper-Conference.pdf.DOI
