LoRASpace: A Pool-Wide Shared Substrate for Static and Dynamic Multi-LoRA Composition

Introduction

Low-Rank Adaptation (LoRA) [8] has made the specialization of existing language and diffusion image generation models feasible by training small low-rank adapters that attach to a frozen backbone on demand, rather than updating the full set of model weights. The simplicity of this construction has produced a second-order effect, where open repositories such as CivitAI [3] and Hugging Face [10] now host LoRA pools containing many trained LoRA adapters per backbone, each capturing a distinct visual concept, character, style, or downstream language task. As these libraries grow into the thousands, attention has shifted from training a single LoRA adapter (LoRAs for short) to using several at once. A user generating a portrait may want one LoRA for the subject’s identity, a second for clothing, and a third for the background, all active during a single image.

Two ambient strategies have emerged for this multi-LoRA problem. The first, static merge [11, 32, 33], folds the chosen LoRAs into a single weight update (delta for short) before inference, so only one forward pass is required. Static merge is cheap to deploy but tends to lose individual LoRA identity, since unrelated low-rank directions are summed in the ambient parameter space. The second strategy, often called composite [Zhong et al., 2024, Zou et al., 2025], runs one forward pass per active LoRA and aggregates the outputs. Composite preserves quality more faithfully but pays linearly in compute and memory with the number of active LoRAs. On modern diffusion backbones, this can mean several minutes per image at high LoRA counts, with proportional cost growth on language models. The two strategies sit at opposite corners of a quality-cost trade-off because both treat each LoRA as an independent low-rank addition in ambient space.

LoRASpace. We observe that both the merge and composite views ignore a structural property of real LoRA pools. When the per-LoRA factors of a pool are stacked and decomposed jointly, the resulting subspace is much smaller than the sum of individual ranks. On language LoRAs trained on related tasks this reflects alignment between adapters, while on diffusion pools it comes largely from layer-width saturation and each adapter’s own spectral tail (§ 4.2). We capture this with the rank-fill ratio $\rho= D / \sum_i r_i$, where $D$ is the rank-truncated subspace dimension obtained by an energy-truncated SVD of the stacked factors, and $\sum_i r_i$ is the no-overlap upper bound. Based on this overlap, we introduce LORASPACE, a training-free composition operator whose goal is to recover composite-level quality at near-merge inference cost while supporting input-dependent routing, built from a rank-truncated pool-wide substrate $(U, V)$ computed once at pool-creation time and reused for any active subset of LoRAs chosen at inference. The substrate supports two deployment modes from one cache. A static mode folds time-invariant routing into a single combined delta at ambient-merge inference cost. A dynamic mode accepts context-dependent routing through a router function $\{w_i\}$ that maps the current generation context (token, step, pixel, or prompt) to per-LoRA weights, at substantially lower cost than composite. On diffusion pools, LoRASPACE matches or exceeds composite quality at merge cost (+0.74 CLIP-T on Pool A) and reaches 6.1× speedup over composite at high LoRA counts, mostly by running one backbone pass instead of one per LoRA. On the near-orthogonal LoRA-Retriever LLM benchmark, the framework predicts no compression speedup, and accuracy gains over ambient mixture come from pairing the substrate with external retrieval (+0.96 Macro Task).

Contributions.

• Cross-modality rank-fill measurement. Among the first measurements, to our knowledge, of the pool-wide rank-fill ratio $\rho= D / \sum_i r_i$ at energy threshold $\tau= 0.99$ on real LoRA pools: $\rho\le0.57$ on diffusion pools with sub-linear scaling and $\rho\approx0.62$ on a related-task LLM pool (LoRA-Hub at $K=48$), versus $\rho\approx0.96$ on the diversity-curated LoRA-Retriever benchmark, with pool composition rather than modality predicting where compression is large.

• Dual-mode (static + dynamic) pool-wide substrate. LORASPACE is a rank-truncated pool-wide operator supporting two deployment modes from one precompute. Static-mode folds time-invariant routing into a single combined delta at ambient-merge cost. Dynamic-mode evaluates context-conditioned routing (per-token, per-step, per-pixel, per-prompt) through the same shared bases. The dynamic-mode interface is router-pluggable, and we evaluate four routers (§ 4.5).

• Mechanism ablation. The quality lift over ambient merge is rank-$D$ truncation acting as regularisation. The pool-wide basis is approximately combo-optimal at that rank, so one SVD amortises across all combinations and supports dynamic routing without per-combo SVDs (§ 4.3).

Related Work

Where LoRASPACE sits. Existing training-free multi-LoRA methods trade off cost against compositionality. Static merges and shared-basis merges collapse the active subset into one delta at

Table 1: Where LORASPACE sits among training-free multi-LoRA methods. Columns: pool-wide basis ($U$, $V$) (defined in § 3.4), active LoRAs per step, deployment mode (static vs. dynamic), and per-image cost normalised to ambient merge (1×). To our knowledge, no other reviewed family combines a pool-wide basis with simultaneous $N \ge2$ composition and both deployment modes from one precompute (Eq. (5)).

Method family (representative work)	Pool-wide basis	Active per step	Mode	Per-image cost
Static merge (Task Arith. [11], TIES [32], DARE [33])	X	$N$	static	1×
Shared-basis merge (Knots [26], Core-Space [18])	per merge	$N$	static	1×
Composite & step/mask routing (ComposLoRA [Zhong et al., 2024], CMLoRA [Zou et al., 2025], FreeFuse [12], MultLFG [23])	X	$N$	dynamic	$\sim N\times$
Per-input router-bound composition (SpectR [5], K-LoRA [17], FreeLoRA [36], LoRAHub [9], LoRARetriever [35])	X	1 to $N$	dynamic	$\sim N\times$
Compress-then-Serve [6]	✓	1	static	1×
Trained MoE-LoRA (X-LoRA [1], MoLE [30], Arrow [16], PHATGOOSE [14])†	X	$N$	dynamic (trained)	$\sim N\times$
LORASPACE (ours)	✓	$N$	static + dynamic	$\sim1\times$ static, sub-composite dynamic‡

ambient cost (1×), but freeze the composition before inference. Composite, step/mask routing, and per-input router methods recover input-dependent routing at $\sim N\times$ cost, because each active LoRA runs as its own forward pass. The closest neighbour, Compress-then-Serve [6], introduces a pool-wide shared basis but targets single-LoRA serving ($N=1$) rather than $N \ge2$ composition. Tab. 1 arranges these methods along four axes (pool-wide basis, active LoRAs per step, static vs. dynamic, per-image cost) and identifies the empty slot. No reviewed method combines a pool-wide basis with simultaneous $N \ge2$ composition and both deployment modes (defined in § 3.4) from one precompute. LORASPACE fills it. The decomposition itself is standard. What is new is applying it once to a whole pool, so that every adapter keeps its own coordinates in one shared space and any combination is served in one pass from one compact object. Two further capabilities separate it from the closest pool-level methods. $\rho$ predicts before deployment whether a pool compresses (§ 4.2), and a newly published adapter can be admitted by extending the bases without recomputing the pool (App. § 15). App. § 16 compares the methods on these requirements. We exercise three adaptive routers (spatial [5], temporal [Zou et al., 2025], semantic [12]) plus a uniform-merge baseline [11], all adapted from existing designs.

Static merge. Static merge methods fold a chosen subset of LoRAs into a single ambient delta before inference. Task-arithmetic [11] averages deltas, while TIES-Merging [32] resolves sign conflicts and DARE [33] adds drop-and-rescale delta sparsification. The shared property is that all active LoRAs become one low-rank update, so inference matches a single-LoRA model in cost. Quality drops on multi-LoRA tasks [Zhong et al., 2024], which is usually attributed to interference between LoRAs trained for different concepts when their deltas are summed.

Composite and route-then-apply. Composite methods avoid this interference by keeping LoRAs separate at inference. ComposLoRA [Zhong et al., 2024] runs one full forward pass per active LoRA, paying linearly in compute and memory with $N$. Adaptive selection reduces cost through dynamic routing: CMLoRA [Zou et al., 2025] and MultLFG [23] schedule dominant LoRAs across diffusion steps via frequency-domain signals; FreeFuse [12] routes subject tokens to spatial regions at test time; ZipLoRA [24], K-LoRA [17], and FreeLoRA [36] target narrower 2-LoRA / autoregressive personalization settings. None of these expose a pool-wide precomputed substrate handling arbitrary multi-LoRA composition with both static and dynamic deployment. Orthogonality-based merge analyses such as OMCD [34] are complementary: they study inter-LoRA orthogonality as an explanatory variable for merge quality, where our framework treats pool-wide rank-fill as the structural variable.

Subspace methods. A separate line of work compresses LoRA representations through linear projection. Knots [26] column-stacks per-LoRA deltas $\Delta W_i = B_iA_i$ and applies one SVD as an alignment tool before TIES-style merging on small LoRA sets. CoreSpace [18] stacks per-LoRA factors and aligns them in a fixed $\sum_i r_i$ basis without truncation. Reported evaluations are limited to small-pool settings, so we do not include its numbers in our pool-scale comparison. Compress-then-Serve [6] learns a pool-wide shared basis $(U,V)$ together with per-LoRA middle matrices $\Sigma_i$ to compress the joint product $B_iA_i$ (the per-LoRA delta), for efficient serving of many single-LoRA requests on LLM backbones. Systems such as Punica [2] and SLoRA [25] optimise serving many LoRA-specialised requests, but do not address simultaneous semantic composition of multiple LoRAs in one generation. CtS notes after its Proposition 1 that training noise makes the joint rank generically full ($\tilde{r} = \sum_i r_i$); our energy-truncated measurement of $D$ at $\tau=0.99$ is consistent with this high-effective-rank regime on the LLM pool ($\rho=0.96$) but substantially smaller on diffusion pools ($\rho\in[0.46, 0.57]$, Table 2). LORASPACE differs from CtS in three respects: (i) the inference setting is simultaneous $N\geq2$ composition rather than single-LoRA serving; (ii) we use paired marginal SVDs to factorise the energy-normalised $B$ and $A$ stacks separately rather than the joint product; (iii) the substrate exposes a router-compatible operator $U(\sum_i w_i\hat{B}_i\hat{A}_i)V^\top x$ (defined in § 3.4) supporting both static and dynamic deployment from one precompute.

Pool	$K$	$\sum_i r_i$	$D$ (mean)	$\rho$	$\beta$
Pool A (SD 1.5)	11	784	446	0.57	0.43
Pool B (SD 1.5)	17	1112	521	0.47	0.49
Pool C (SDXL)	17	1792	819	0.46	0.55
LoRA-Hub (FLAN-T5-Large)	48	768	474	0.62	0.88
LoRA-Retriever (LLaMA-2-7B)	48	384	368	0.96	0.94

Single-active routing. Switch-style methods activate exactly one LoRA at any token or step (SpectR [5], X-LoRA [1], PHATGOOSE [14], MoLE [30], Arrow [16]). They pick one LoRA at each location, so the multi-LoRA combination question never arises and they are not directly comparable to composition methods. We report a switch baseline in App. § 25 for completeness.

LLM-side composition. LoRA-Retriever [35] composes task-specialised LoRAs for LLMs by selecting a top-$K$ subset via INSTRUCTOR [27] embeddings, then either fusing factors ($B_pA_q$ cross-terms) or running a weighted ambient mixture. LoRAHub [9] optimises LoRA weights with CMA-ES [7] on a held-out validation set. Both treat composition as ambient-space averaging; neither exploits pool-level rank structure. LORASPACE’s operator $\sum_i w_i\hat{B}_i\hat{A}_i$ contains only per-LoRA products by construction, so the cross-product approximation error that degrades fusion-style factor combination (App. § 27) does not arise. We use the LoRA-Retriever pool to characterise the high-effective-rank regime as a counterpoint to the diffusion regime.

Method

A LoRA $i$ (e.g., an image style LoRA) comprises two matrices $A_i$ and $B_i$ for each layer. In general, as indicated later in § 3.1, a scaled linear combination of $A_iB_i$ pairs produces the desired delta. In LORASPACE, we produce these deltas by pre-computing a decomposition (construction § 3.2) of the stacked $A_i$s and $B_i$s and using their sub-space representations (low-dimension) $\hat{A}_i$ and $\hat{B}_i$ appropriately projected. As we control the rank of this decomposition, and hence the number of actual deltas, the computational complexity drastically reduces, as we show in the following sections.

Setup and notation

Consider a single LoRA-modified linear layer with frozen base weight $W_0 \in\mathbb{R}^{d_{\mathrm{out}}\times d_{\mathrm{in}}}$. A LoRA [8] augments this layer with a low-rank update $\Delta W_i = s_iB_iA_i$, where $B_i \in\mathbb{R}^{d_{\mathrm{out}}\times r_i}$ and $A_i \in\mathbb{R}^{r_i\times d_{\mathrm{in}}}$ are the LoRA’s left and right factors, $r_i \ll\min(d_{\mathrm{in}},d_{\mathrm{out}})$ is the LoRA rank, and $s_i = \alpha_i/r_i$ is the per-LoRA scaling factor with $\alpha_i$ the LoRA scaling hyperparameter (set at training time). We write $\bar{r} = (1/K)\sum_i r_i$ for the average pool rank. Given a pool $\mathcal{P} = \{1,\ldots,K\}$ of trained LoRAs, the multi-LoRA composition target on input $x \in\mathbb{R}^{d_{\mathrm{in}}}$ is

where $\{w_i\}_{i=1}^{N}$ are per-LoRA weights for an active subset of size $N \le K$. Static merge precomputes a fixed sum once and discards routing. Composite evaluates Eq. (1) by running $N$ forward passes per token and aggregating outputs. LORASPACE evaluates a router-weighted form of Eq. (1) in one base-model pass with LoRA-side work linear in $N$ inside a shared $D$-dimensional subspace ($D \ll d_{\mathrm{out}}$), exploiting structural rank overlap.

Energy-normalised shared subspace

The motivating observation is empirical. When per-LoRA factors from a real diffusion pool are stacked and decomposed by paired marginal SVDs, the resulting subspace dimension is much smaller than $\sum_i r_i$. We characterise this empirically in § 4.2, which separates alignment between adapters from layer-width saturation and each adapter’s own spectral tail. Here we use it to construct an energy-truncated shared subspace.

Frobenius normalisation. Raw factor stacking is dominated by high-norm LoRAs. Frobenius normalisation puts every LoRA on a common scale before decomposition:

We then form pool stacks $\bar{B}_{\mathrm{stack}} = [\bar{B}_1 \cdots\bar{B}_K] \in\mathbb{R}^{d_{\mathrm{out}}\times\sum_i r_i}$ and $\bar{A}_{\mathrm{stack}} = [\bar{A}_1;\cdots;\bar{A}_K] \in\mathbb{R}^{\sum_i r_i\times d_{\mathrm{in}}}$.

Paired marginal SVDs and adaptive truncation. We compute two thin SVDs $\bar{B}_{\mathrm{stack}} = U_B\Sigma_B V_B^{\mathsf{T}}$ and $\bar{A}_{\mathrm{stack}} = U_A\Sigma_A V_A^{\mathsf{T}}$, then truncate each at energy fraction $\tau\in(0,1)$:

The shared subspace dimension is $D = \min(\max(D_B,D_A), \sum_i r_i, d_{\mathrm{in}}, d_{\mathrm{out}})$. The shared bases are $U = U_B[:, :D] \in\mathbb{R}^{d_{\mathrm{out}}\times D}$ and $V = V_A[:, :D] \in\mathbb{R}^{d_{\mathrm{in}}\times D}$ (with $V_A$ the right-singular-vector matrix in $\bar{A}_{\mathrm{stack}} = U_A\Sigma_A V_A^{\mathsf{T}}$), the leading left and right singular vectors of $\bar{B}_{\mathrm{stack}}$ and $\bar{A}_{\mathrm{stack}}$, both with orthonormal columns. By the Eckart-Young theorem [4], $U$ and $V$ are the optimal rank-$D$ subspaces for the marginal $B$-stack and $A$-stack reconstructions separately. This is optimal for each marginal factor projection but not a joint optimum for the product $\sum_i s_i B_i A_i$. A joint SVD of the scaled deltas is closer to product-optimal (per-LoRA error $0.35\%$ against $0.88\%$ on Pool C at equal $D$), but the difference does not reach the generated images, and the marginal bases are built from thin factors on CPU and can be extended when the pool changes (App. § 13). We find empirically that retaining the low-energy directions does not improve composition metrics within the practical $\tau$ range (§ 4.4).

Per-LoRA projected factors

Each LoRA is re-parameterised by projecting its raw factors onto the shared subspace:

The LoRA scaling $s_i$ is split symmetrically as $\sqrt{s_i}$ on each side. The per-layer cache $\mathcal{C}_{\ell} = \{U,V,\{(\hat{B}_i,\hat{A}_i)\}_{i=1}^{K}\}$ is computed once per pool, stored on CPU memory, and streamed to the GPU as needed. Cache size is $O(D(d_{\mathrm{out}} + d_{\mathrm{in}}) + K D\bar{r})$ versus $O(K\bar{r}(d_{\mathrm{out}} + d_{\mathrm{in}}))$ for ambient storage, which gives a meaningful reduction whenever $D \ll d_{\mathrm{out}}$.

Composition operator and deployment modes

At inference, the LORASPACE operator processes input $x$ in three steps. Project once into the subspace $x_{\mathrm{proj}} = V^{\mathsf{T}}x \in\mathbb{R}^{D}$. Compute a per-LoRA subspace contribution $y_i = \hat{B}_i(\hat{A}_i x_{\mathrm{proj}}) \in\mathbb{R}^{D}$ for each active $i$. Mix and back-project to give $\Delta y^{\mathrm{LS}}(x) = U \sum_{i=1}^{N} w_i y_i$. Compactly,

The input projection $V^{\mathsf{T}}x$ and output back-projection $U(\cdot)$ are shared across all active LoRAs. The per-LoRA work is entirely inside the $D$-dimensional subspace. The shared bases do not orthogonalise adapters or split directions among them. $(U,V)$ spans the union of the pool’s directions and each adapter keeps dense coordinates inside it. Across 4,800 audited axes (12 SDXL layers, both sides, top 200 per side) the median adapter-ownership Herfindahl index is 0.12, so a typical axis carries about eight adapters. At full rank the operator equals the ambient weighted sum (Prop. 1), so any interaction the ambient sum produces, such as a lighting LoRA changing how a character LoRA renders, is kept. Truncation only removes a low-energy tail (composed $N=5$ error 0.84% at $\tau= 0.99$, App. § 13).

Static and dynamic deployment from one substrate. When $\{w_i\}$ are time-invariant (uniform or per-prompt/pre-defined routing), the inner sum $M = \sum_i w_i\hat{B}_i\hat{A}_i \in\mathbb{R}^{D \times D}$ folds into a single ambient delta $\Delta= UMV^{\mathsf{T}}$ at ambient-merge cost (static-mode). Time-varying routers (per-token, per-step, per-pixel) require dynamic-mode. Eq. (5) is then evaluated in the subspace at every UNet pass, with shared $V^{\mathsf{T}}x$ and $U(\cdot)$ paid once per pass independent of $N$. The same $(U,V)$ serves both modes.

Static-mode is not ambient merge. Substituting Eq. (4) gives $\Delta y^{\mathrm{LS}}(x) = P_U \Delta y^{\star}(P_Vx)$ with $P_U = UU^{\mathsf{T}}$, $P_V = VV^{\mathsf{T}}$ the projectors onto the truncated subspaces. Since $D < \min(d_{\mathrm{out}},d_{\mathrm{in}})$ on most layers (682 of 722 on Pool C), $P_U,P_V \ne I$ there and the two operators differ from the unprojected target $\Delta y^{\star}$. § 4.3 reports the empirical effect alongside a mechanism ablation.

Reconstruction guarantee

Proofs are in Appendix B. At $\tau= 0.99$ the per-side discarded singular value is bounded by $\sqrt{(1-\tau)\bar{K}} \le0.41$ for $K \le17$; § 4.2 confirms this approximation does not degrade outputs in practice.

Router interface

The composition operator is router-pluggable. Any function producing per-LoRA weights $\{w_i\}$ plugs in unchanged. We adopt uniform weights $(w_i = 1/N)$ as the default and characterise three adaptive routers as plug-in extensions: spatial (per-token activation energy, in the spirit of SpectR [5]), temporal (per-step subspace-band, following CMLoRA [Zou et al., 2025]), and semantic (per-concept token-to-region routing, in the spirit of FreeFuse [12]). Full specifications are in App. § 9. For LLM pools, LoRASPACE pairs with an INSTRUCTOR retriever [27] following the LoRA-Retriever protocol [35], showing that internal and external routing policies share a single composition substrate.

Complexity

We count the incremental LoRA-side linear algebra for one layer processing one token; Appendix § 10 reports the full per-step cost including base-model forwards. For $N$ active LoRAs of representative rank $\bar{r}$, ambient composition adds $O(N\bar{r}(d_{\mathrm{in}} + d_{\mathrm{out}}))$ per layer, linear in both $N$ and ambient width. LoRASPACE costs

The shared term is paid once regardless of $N$. As pool size $K$ grows, the projection size scales as $D \propto K^{\beta}$ with measured $\beta\in[0.43, 0.55]$ on diffusion pools (§ 4.2), so the projection term grows sub-linearly with the pool. The per-LoRA term is linear in $N$ but scaled by $D$, which is much smaller than the ambient $d_{\mathrm{out}}$. The adapter-side crossover with an optimised single-pass ambient mixture is $D < \frac{2N\bar{r}d}{2d + N\bar{r}}$. This bound is always below $N\bar{r}$, so it is not met at the default $\tau= 0.99$ (Pool C, mean $D = 819$ against $N\bar{r} \approx527$ at $N=5$). It is met at the compressed operating points validated in App. § 13 ($\bar{D} \approx107$ at $\tau= 0.70$ and $\approx49$ at $\tau= 0.50$), so the subspace path saves adapter-side arithmetic only in that reduced-storage regime. Appendix § 10 gives the full per-method comparison including memory.

Experiments

We organise the evaluation around four findings: rank-fill regime characterisation across diffusion and LLM pools (§ 4.2); a quality + mechanism comparison showing static-mode LoRASPACE delivers composite-level quality at ambient-merge cost and that the lift is rank-$D$ truncation in any basis (§ 4.3); robustness to the energy truncation threshold (§ 4.4); and a routing ablation showing router quality varies on Pool A (temporal helps, semantic is flat, spatial regresses on ImageReward; § 4.5). A predictive-theory check on the near rank-orthogonal LLM regime, including external retrieval coupling, is in § 4.6 (App. § 27 for the full scoreboard).

Setup

Pools and backbones. Pool A (11 reality-style LoRAs, SD 1.5 [22]) and Pool B (17 LoRAs, SD 1.5) are drawn from CivitAI [3]. Pool C (17 reality-style LoRAs, SDXL [20]) is curated from CivitAI and Hugging Face [10]. The LLM pool is the 48-LoRA LLaMA-2-7B [28] task-LoRA set from LoRA-Retriever [35].

Metrics. For diffusion, we report CLIP-T [21] for prompt alignment, ImageReward [31] for human-preference alignment, and a GPT-4o [15] composition rubric (Composition Quality + Image Quality, 0 to 10 each, position-bias averaged across two orderings) following ComposLoRA [Zhong et al., 2024]. For LLMs, we use the LoRA-Retriever Macro Task accuracy.

Software and hardware. All diffusion experiments run on a single RTX 3090 (24 GB) using PyTorch [19] and the diffusers library [29], with DPM-Solver++ [13] as the sampler. Pool A: 100 denoising steps, $512 \times512$. Pool C: 50 denoising steps, $1024 \times1024$. We use 240 stratified combinations per seed for Pool A, distributed across active sizes $N \in\{2, 3, 4, 5\}$, and 200 for Pool C, across $N \in\{2, \ldots, 6\}$. Pool B is reported in the appendix at $N \in\{7, 8\}$ (Appendix § 18) to probe the high-cardinality regime where the rank-fill ratio is lowest ($\rho= 0.47$ at $K = 17$); Pool A and Pool C cover the full active-LoRA range.

Subspace dimension grows sub-linearly with pool size

We measure the rank-fill ratio $\rho= D/\sum_i r_i$ at energy threshold $\tau= 0.99$ (the value used throughout) and the growth exponent $\beta$ obtained by fitting $D \propto K^\beta$ with category-aware random subsampling at three random seeds. Table 2 summarises the result. The three diffusion pools have $\rho$ between 0.46 and 0.57 and $\beta$ between 0.43 and 0.55, both far below the $\rho= 1$, $\beta= 1$ no-overlap baseline. The two LLM pools span the full range: a 48-LoRA sub-sample of LoRA-Hub over related FLAN-T5 NLP tasks lands at $\rho= 0.62$, $\beta= 0.88$, comparable to diffusion overlap, while the LoRA-Retriever benchmark, curated for retrieval-task diversity, sits near the no-overlap baseline at $\rho= 0.96$, $\beta= 0.94$.

The source of the low $\rho$ differs between regimes. We split $\rho= \rho_{\mathrm{intra}} \times\rho_{\mathrm{cross}}$ into within-LoRA rank slack and a cross-LoRA term, and compare $D$ with a spectrum-matched random null (App. F). Within-LoRA slack is nowhere dominant ($\rho_{\mathrm{intra}} \ge0.84$ on every pool). The related-task LoRA-Hub pool [9] needs about 30% fewer dimensions than its matched null, an alignment associated with shared task structure, while the diversity-curated LoRA-Retriever sits at its null. On the diffusion pools the stacked ranks exceed the layer width on most layers, and on the SDXL pools $D$ sits at its null (0.98 on Pool C), so their low $\rho$ is real compressibility that is largely explained by width saturation and per-adapter spectra, and we leave its link to training data open. Pool composition, not modality, predicts where compression is large and where it is not. The framework also predicts a saturation regime when $\sum_i r_i \ge d$: the full LoRA-Hub pool ($K=196$) crosses the threshold at $K^\star= d/r = 64$, with each new LoRA adding $\sim11$ dimensions to $\bar{D}$ below threshold and $\sim1$ above (App. T).

Quality, speed, and memory

We compare three composition strategies on Pool A: ambient static merge with weight 1 per LoRA, so the deltas are summed, ambient composite with $N$ forward passes per step, and LORASPACE with uniform $1/N$ routing weights. Table 3 reports CLIP-T and ImageReward averaged across three random seeds (240 combinations per seed, $N \in\{2,3,4,5\}$), and the GPT-4o rubric from one seed (the position-bias-averaged GPT-4o evaluation is expensive; three-seed consistency on CLIP-T and ImageReward suffices to verify robustness).

Method	CLIP-T ↑	ImageReward ↑	GPT-4o Total ↑	GPT-4o Win% ↑
merge	33.60 ± 0.19	0.760 ± 0.008	14.32	21.9
composite	35.27 ± 0.19	0.953 ± 0.011	15.24	29.5
LORASPACE (uniform)	36.01 ± 0.16	1.041 ± 0.035	15.61	48.6

LORASPACE with uniform routing exceeds composite by +0.74 CLIP-T, +0.09 ImageReward, +0.37 GPT-4o total, and almost twenty percentage points of GPT-4o win rate (48.6% vs. 29.5%), with 33.3% being the no-preference baseline for three competitors. It also exceeds ambient merge by larger margins. The inter-method gap exceeds the across-seed standard deviation on every metric.

Mechanism ablation. We test whether the quality lift depends on the pool-wide basis with two static-delta variants at the same rank $D$. truncated-merge is the combo-specific best-fit SVD of the ambient merge; subspace-merge bakes the pool-wide $(U, V)$ readout into a fixed delta per combo. Both run at ambient-merge per-image cost. On Pool A (seed 111, 240 combos, $N=2$ to 5), both match per-step LoRASPACE-uniform within fp16 noise (Tab. 16). At equal rank $D$ the basis choice does not affect quality. The lift over ambient merge (+2.40 CLIP-T, +0.32 ImgRew) is therefore not a property of the pool-wide basis. Both variants average the LoRAs with weights $1/N$ where merge sums them, and at $\tau= 0.99$ the truncation changes the composed delta by under 1% on Pool C (App. G).

Why the pool-wide basis. Amortisation: truncated-merge requires 192 layer SVDs per combo (∼1.4 h across Pool A’s 1011 combos) vs. one 21.8 s pool-wide precompute (App. P). Dynamic-mode: when $\{w_i\}$ depend on context, no single static delta exists for truncated-merge, while $(U, V)$ supports both modes from one precompute. At about 5 s of SVDs per combination, the precompute pays off beyond about five distinct combinations, so per-combination truncated merge remains the better choice for a static deployment with a few fixed combinations. Serving 30 combinations from the 37-adapter union of App. H averages 35.1 s per combination including the one-time load, against 43.9 s for repeated PEFT load and merge. Keeping all adapters resident instead needs per-adapter format fixes, and even then 36 of the 37 load at 19.0 GiB and one $1024^2$ generation peaks at 23.1 of the card’s 24 GiB.

Pool C (speed, memory, quality). Figure 2 summarises per-image cost. Dynamic-mode stays near-flat in $N$ (6.1× faster than composite at $N=6$). Most of this gain comes from running one backbone pass instead of $N$. At $N=5$ a single-pass ambient mixture already accounts for 4.9× of the 5.3×, and the substrate path adds 1.08× (Tab. 20). Static-mode inherits ambient merge cost (23 to 39 s, Tab. 18). Across 20 combos uniform LoRASPACE scores 32.89 CLIP-T / 1.040 ImgRew vs. composite (31.91 / 0.973) and merge (29.41 / 0.721), with the CLIP-T gap over composite growing from +0.17 at $N=2$ to +1.37 at $N=6$ (Tab. 17, additional qualitative examples in App. Q).

Energy threshold ablation

Sweeping $\tau\in\{0.95, 0.97, 0.99, 0.999\}$ on Pool A grows $\bar{D}$ by 43% while quality varies non-monotonically within seed-level noise ($|\Delta| < 0.05$ CLIP-T, $|\Delta| < 0.008$ ImageReward). We adopt $\tau= 0.99$ as the default (Table 5).

$\tau$	$D_{\min}$	$D$ (mean)	$D_{\max}$	CLIP-T	ImageReward
0.95	224	356	568	35.76	1.071
0.97	248	392	611	35.81	1.072
0.99	283	446	663	35.77	1.064
0.999	314	508	741	35.80	1.067

Routing ablation: substrate is router-tolerant

On Pool A across three seeds, the four routers from § 3.6 (uniform, spatial, temporal, semantic) all exceed the merge baseline by large margins (Table 15, App. § 19). Temporal gives the largest ImageReward gain over uniform, semantic is flat, and spatial regresses. The substrate carries the quality, with routers contributing small modulations.

Router	CLIP-T ↑	ImageReward ↑
uniform	35.82 ± 0.10	1.050 ± 0.029
spatial	35.43 ± 0.34	0.857 ± 0.062
temporal	35.92 ± 0.10	1.086 ± 0.030
semantic	35.83 ± 0.09	1.050 ± 0.026

Cross-modality predictive check on the LLM regime

On the near rank-orthogonal LoRA-Retriever LLaMA-2-7B benchmark ($K=48$, $\rho= 0.96$), the substrate alone (loraspace-fullpool) scores 50.49 Macro Task vs. the ambient mixture’s 53.55. INSTRUCTOR-retrieval coupling (loraspace-hybrid) recovers 54.51 at 1.52 vs 1.47 samp/s, matching the framework’s prediction that compression helps only with retrieval coupling (full scoreboard in App. § 27).

Conclusion

We empirically characterise pool-wide rank overlap in real LoRA collections, finding $\rho\in[0.46, 0.62]$ across three diffusion pools and a 48-LoRA LoRA-Hub sub-pool, with LoRA-Retriever as the near-orthogonal exception ($\rho\approx0.96$). LORASPACE turns this overlap into a pool-wide substrate from one SVD, matching composite quality at merge cost in static-mode and reaching 2.3 to 6.1$\times$ speedup in dynamic-mode, mostly from avoiding per-LoRA backbone passes. A mechanism ablation shows that the lift does not depend on the pool-wide basis, making the basis valuable for amortisation and dynamic capability (limitations and broader impact in App. § 28).

When to use LORASPACE. LORASPACE targets deployments that repeatedly compose several LoRAs under a memory budget. With one active adapter or unconstrained memory, ambient serving is the right choice. Within that setting, $\rho$ is a cheap screen (CPU minutes, no base model). Below about 0.6 we deploy the substrate, as on the three diffusion pools, with LoRA-Hub (0.62) at the boundary. Above about 0.9, as on LoRA-Retriever, the substrate is worth using for routing or retrieval coupling rather than for compression at the default $\tau$. In between, a $\tau$ sweep on held-out prompts decides (CLIP-T drop under 0.5 and structural similarity at least 0.95). These cut points separate our pools empirically and are not universal constants, and $\rho$ falls with pool size (0.95 at $K=2$ to 0.57 at $K=11$, App. § 11), so very small pools are better served ambiently. Uncurated and mixed pools follow the same rule (App. § 14).

Acknowledgments and Disclosure of Funding

The computational resources for this research were supported by the Accelerating Higher Education Expansion and Development (AHEAD) Operation Grant No. 6026-LK/8743-LK from the Ministry of Higher Education, Sri Lanka, funded by the World Bank. R. Rodrigo acknowledges the University of Moratuwa Senate Research Committee Grant No. SRC/LT/2021/20. We also thank Jaseci Labs for its support. The authors declare no competing interests.

Pool-wide precompute illustration

Proof of Proposition 1

Proof. Under the span conditions, $UU^\top B_i = B_i$ (because $UU^\top$ is the identity on $\operatorname{col}(U) \supseteq\operatorname{col}(B_i)$) and $A_iVV^\top= A_i$ (analogously). Substituting Eq. (4) into Eq. (5):

Proof of Corollary 2. Let $P = UU^\top$ and $Q = VV^\top$ be the projectors onto $\operatorname{col}(U)$ and $\operatorname{col}(V)$. For each LoRA $i$, decompose $B_i = PB_i + E_i^B$ with $E_i^B = (I-P)B_i$, and $A_i = A_iQ + E_i^A$ with $E_i^A = A_i(I-Q)$. Then

By the Eckart-Young theorem [4], $\lVert(I-P)\bar{B}_{\mathrm{stack}}\rVert_2 = \sigma_{D+1}^B$. Since $\bar{B}_i = B_i/\lVert B_i\rVert_F$ is a column-block of $\bar{B}_{\mathrm{stack}}$, we have $\lVert E_i^B\rVert_2 \le\lVert B_i\rVert_F \cdot\sigma_{D+1}^B$, and analogously $\lVert E_i^A\rVert_2 \le\lVert A_i\rVert_F \cdot\sigma_{D+1}^A$. Applying $\lVert MNx\rVert_2 \le\lVert M\rVert_2\lVert N\rVert_2\lVert x\rVert_2$, bounding $\lVert A_i\rVert_2 \le\lVert A_i\rVert_F \le\mu^A$ and $\lVert B_i\rVert_2 \le\lVert B_i\rVert_F \le\mu^B$, summing the three error terms, and collecting over $i$ yields Eq. (6).

Empirical anchor (SDXL Pool C, $\tau= 0.99$). At $\tau= 0.99$ with $K = 17$ pool LoRAs, the energy-fraction definition gives $\sigma_{D+1}^B \le\sqrt{(1-\tau)K} \approx0.41$. The corollary applies a Frobenius-norm chain on $\lVert A_i\rVert$, $\lVert B_i\rVert$ that loses orders of magnitude in tightness; we therefore treat Cor. 2 as a worst-case sanity statement rather than a predictive bound. Empirical robustness of $\tau= 0.99$ is established by the energy-threshold sweep in § 4.4, where quality varies non-monotonically within seed-level noise across $\tau\in[0.95, 0.999]$.

Router specifications

The four router variants summarised in § 3.6 are detailed below. All share the same operator interface: a function producing per-LoRA weights $\{w_i\}$ that plug into Eq. (5).

Spatial router (per-token activation energy). Per-position normalised energy $e_i(p) = \lVert y_i(p)\rVert_2^2/(\lVert x_{\mathrm{proj}}(p)\rVert_2^2 + \epsilon)$ scored across active LoRAs, top-$k$ selection, temperature softmax. All runs use $k=1$, so each token is routed to its highest-energy LoRA. Spirit of SpectR [5] but applied over per-LoRA contributions inside the pool subspace.

Temporal router (per-step subspace-band). Following CMLoRA’s dominant/non-dominant scheme [Zou et al., 2025] but with the routing signal computed as the late-band energy fraction inside the shared subspace, $m = \sum_{d \ge D/2}\lVert x_{\mathrm{proj}}[\cdot,\cdot,d]\rVert^2 / \sum_d\lVert x_{\mathrm{proj}}[\cdot,\cdot,d]\rVert^2$. Signal and operator share the same coordinate system.

Semantic router (per-concept spatial mask). A short warmup pass collects cross-attention maps for each LoRA’s trigger tokens, following the token-to-region routing motivation of FreeFuse [12]. Resulting per-LoRA masks $M_i \in[0,1]^{H \times W}$ are bilinearly resampled and applied at image-token axes, with activation-energy fallback at text-token axes.

External retrieval coupling. For LLM pools, LoRASPACE pairs with an INSTRUCTOR retriever [27] following the LoRA-Retriever protocol [35]. The retrieved top-$K$ subset is composed through the same shared-subspace operator. This demonstrates that internal (activation- or band-derived) and external (retrieval-derived) routing policies share a single composition substrate.

Per-method complexity comparison

Table 4 consolidates per-step compute and peak GPU memory across all evaluated methods. The key comparison is the Scaling in N column: output-space methods (composite, cmlora) scale linearly in compute and memory; PEFT-resident single-active methods (switch) hold per-step compute roughly constant but materialize all $K_{\mathrm{pool}}$ LoRAs on GPU regardless of how many are active per step; static-merge methods (merge, knots) achieve $O(1)$ inference cost but lose routing capability; LoRASPACE scales with the subspace dimension $D \le\min(\sum_i r_i,d)$, which saturates at the layer width when the pool grows large enough.

Method	Per-step compute	Peak GPU memory	Scaling in $N$
merge	$O(Ld^2)$	$O(Ld^2)$	$\Theta(1)$
switch	$O(L(d^2+\bar r d))$	$O(L(d^2+K_{\mathrm{pool}}\bar r d))$	compute $\Theta(1)$, mem $\Theta(K_{\mathrm{pool}})$
composite	$O(NLd^2S)$	$O(L(d^2+K_{\mathrm{pool}}\bar r d))$	both $\Theta(N)$
cmlora	$O(NLd^2S)$	$O(L(d^2+K_{\mathrm{pool}}\bar r d))$	both $\Theta(N)$
knots	$O(Ld^2)$	$O(Ld^2)$	$\Theta(1)$
LoRASPACE	$O(L(dN+\bar rN)D)$, $D \le\min(\sum_i r_i,d)$	$O(L(d^2+dD))$ GPU $+$ $O(LK_{\mathrm{pool}}\bar rD)$ CPU	linear in $N$ inside subspace, $D$ caps with $K$

Saturation thresholds. Saturation ($D = d$) requires $\sum_i r_i \ge d$, where $d = \min(d_{\mathrm{in}},d_{\mathrm{out}})$ is the width of each layer. SD 1.5 LoRA layers have widths from 320 to 1280, and SDXL layers 640 or 1280. Counted per layer, the stacked ranks of Pool C reach the layer width on all 722 layers, and those of Pools A and B on 132 of 192 layers (every layer narrower than 1280). The diffusion pools therefore sit in the width-saturated regime on most layers, consistent with $D$ matching the spectrum-matched null on the SDXL pools (App. F). The LLM pool has $\sum_i r_i = 384$ against $d = 4096$ (9% of threshold), deep in the rank-bound regime.

Rank overlap ablation: $D$ vs. pool size and energy threshold

This appendix documents the per-pool subspace dimension ablation that underlies Table 2.

Energy threshold sweep (Pool A, SD 1.5, $K = 11$).

$K$	$\sum_i r_i$ (mean)	$D$ (mean $\pm$ std)	$\rho$
2	224	$213 \pm50$	0.95
4	304	$275 \pm22$	0.91
6	456	$353 \pm35$	0.77
8	584	$397 \pm38$	0.68
11	784	$446 \pm0$	0.57

Pool size sweep (Pool A, $\tau= 0.99$).

SDXL Pool C vs. LLM comparison. Pool C ($K = 17$, SDXL) reaches $\rho= 0.46$ at full pool size, with the rank-fill already at 0.69 to 0.72 for $K=4$ subsets. This heavier overlap relative to Pool A is consistent with Pool C’s higher proportion of same-type LoRAs (lighting, material, background) that share scene-element structure. The LLM pool ($K = 48$, LLaMA-2-7B [28]) maintains $\rho\ge0.96$ at all tested pool sizes with growth exponent $\beta= 0.94$, which is effectively linear and confirms near-zero rank overlap across disjoint NLP-task LoRAs.

Rank-fill decomposition and matched null

The rank-fill ratio $\rho= D / \sum_i r_i$ has nominal ranks in the denominator, so it mixes each adapter’s internal rank slack with structure between adapters. With per-adapter effective ranks $r_i^{\mathrm{eff}}$ computed at the same $\tau$ on the same Frobenius-normalised factor stacks as $D$,

where $\rho_{\mathrm{intra}}$ isolates within-LoRA redundancy. Both cross-LoRA structure and stacked ranks approaching the layer width lower $\rho_{\mathrm{cross}}$, so we also compare $D$ with a spectrum-matched random null $D_{\mathrm{null}}$, built from random per-adapter subspaces that keep each block’s own singular spectrum, with the same layer shapes and widths (one seeded draw per layer, averaged over all layers). Table 7 reports every pool at $\tau= 0.99$.

Pool	$\rho$	$\rho_{\mathrm{intra}}$	$\rho_{\mathrm{cross}}$	$D/D_{\mathrm{null}}$
Pool C (SDXL, $K=17$)	0.46	0.84	0.55	0.98
Six stylized SDXL LoRAs ($K=6$)	0.97	1.00	0.97	1.00
Community SDXL, uncurated ($K=20$)	0.87	0.99	0.88	1.00
Pool C + community union ($K=37$)	0.43	0.89	0.48	0.99
LoRA-Hub (FLAN-T5, $K=48$)	0.60	1.00	0.60	0.70
Wild FLAN-T5 LoRAs, uncurated ($K=12$)	0.84	1.00	0.85	0.92
LoRA-Hub + wild union ($K=60$)	0.52	1.00	0.52	0.67
LoRA-Retriever (LLaMA-2, $K=48$)	0.98	1.00	0.98	1.00

Within-LoRA slack is material only on the two pools that contain the Pool C adapters ($\rho_{\mathrm{intra}} = 0.84$ and 0.89) and is 0.99 to 1.00 elsewhere. On the LLM pools, far from width saturation, the related-task pools need about 30% fewer dimensions than their matched nulls (0.70 for LoRA-Hub and 0.67 for its union with an uncurated harvest), while LoRA-Retriever sits at its null. This alignment is consistent with shared task structure, but the control does not isolate training data as the cause. On the SDXL pools, whose stacked ranks exceed the layer widths, $D$ sits at the null (0.98 to 1.00). Their low $\rho$ is real compressibility, largely explained by width saturation and per-adapter spectra rather than by measured alignment, and it does not depend on a curated or related pool. Repeating the decomposition at the compressed operating points of App. § 13 instead of $\tau= 0.99$ attributes 75 to 83% of the log-compression to within-adapter tails on both pool types tested.

Joint versus marginal bases and compressed thresholds

Joint versus marginal SVD. We compare the paired marginal bases of § 3.2 with bases from a joint SVD of the stacked scaled deltas $s_iB_iA_i$, at the same rank $D$ (the $D$ the marginal method selects at $\tau= 0.99$), so capacity is equal. Table 8 reports relative Frobenius reconstruction error of the per-LoRA deltas and of composed uniform $N=5$ deltas on Pool C, on 24 layers spanning nine layer types across the down, mid and up blocks, then on all 722 layers.

Setting	$D$	Per-LoRA error marginal	Per-LoRA error joint	Composed error ($N=5$) marginal	Composed error ($N=5$) joint
Full pool, $K=17$, 24 layers	822	0.88%	0.35%	0.84%	0.43%
Small pool, $K=6$, 24 layers	374	0.52%	0.22%	0.56%	0.32%
Full pool, $K=17$, all 722 layers	per layer	1.01%	0.43%	0.95%	0.53%

The joint construction is closer to product-optimal in weight space, with worst cases of a few percent on low-energy layers (8.7% marginal against 7.3% joint). The difference does not reach generation. A cache built from the joint bases at the same per-layer $D$ regenerates all 30 benchmark combinations at DINO similarity 0.999 to the marginal-cache images (CLIP-T 25.87 against 25.92). The joint spectrum is also more compact ($D \approx148$ on these layers against 822). A joint core $U^{\mathsf{T}}(s_iB_iA_i)V$ has rank at most $r_i$ and refactors exactly into thin factors, so at matched $D$ both caches store the same number of parameters, while at its own $\tau= 0.99$ the joint cache is about $5\times$ smaller (1.77 against 8.63 GB in fp16) and builds 14% faster.

We keep the marginal bases as the default because they can be built and grown from native thin factors in a small memory budget. The marginal build streams thin factor stacks and never forms a dense $d_{\mathrm{out}} \times d_{\mathrm{in}}$ delta, so its peak transient memory per layer is 83 MB against 1.3 GB for our joint build, which keeps the precompute on CPU (21.8 s for Pool A). The marginal coordinates also support the append-only extension of App. § 15 and the exact recovery of Prop. 1. The joint substrate is the more compact choice when the dense build is affordable, and the marginal one suits pools that change.

Compressed thresholds. Table 9 lowers $\tau$ below the default after submission (105 images, 15 combinations at $N=5$, an 18-adapter manifest with mean $D = 785$ at $\tau= 0.99$, and our own prompt set, so CLIP-T is not comparable to the main tables). These compressed caches, and those of the next paragraph, are made by truncating the $\tau= 0.99$ cache to $\min(D_B,D_A)$ per layer rather than the $\max(D_B,D_A)$ of § 3.2, so at a given $\tau$ they are no larger than a fresh build. CLIP-T and ImageReward stay flat within run-to-run variation down to $D \approx50$, the same order as the joint spectrum’s $D$.

$\tau$	mean $D$	CLIP-T	ImageReward	DINO
0.99	785	25.33	1.137	1.000
0.70	107	25.67	1.158	0.981
0.50	49	25.61	1.148	0.967
0.30	22	25.27	1.117	0.949

The basis matters at compressed sizes. At $\tau= 0.99$ the choice of basis does not change quality (§ 4.3). At compressed sizes it does. Regenerating 30 fixed-seed combinations at identical per-layer $D$ with four bases at $\tau= 0.50$ ($D \approx49$, Pool C cache 0.63 GB against 9.5 GB), the pool basis keeps DINO similarity 0.967 to the full-cache images (CLIP-T 25.93 against 25.92), while a random basis and a basis built from a different pool fall to 0.877 and 0.901 because the adapters no longer come through. A per-combination Eckart-Young basis also holds quality (0.986) but costs minutes of dense SVDs per combination and leaves no reusable object. The pool basis keeps 0.956 and 0.945 at $\tau= 0.40$ and 0.30 (caches of 0.40 and 0.26 GB), while both controls stay at 0.88 to 0.90.

Heterogeneous and uncurated pools

Most of our pools are earlier benchmarks adopted unchanged. To test a pool nobody on our team chose, we harvested the 20 most-downloaded valid community SDXL LoRAs by a fixed rule (Hugging Face SDXL tree, downloads descending, 16 creators, ranks 4 to 128) and measured them on the same 722 layers as Pool C. We also added six strongly stylized SDXL LoRAs. Table 10 reports $\rho$ at $\tau= 0.99$.

Pool	$K$	$\sum_i r_i$	$D$	$\rho$
Pool C (control)	17	1792	819	0.46
Six stylized LoRAs alone	6	192	186	0.97
Pool C + six stylized	23	1984	848	0.43
Community LoRAs, uncurated	20	673	580	0.87
Pool C + community	37	2337	921	0.43

Unrelated adapters among themselves stay nearly incompressible ($\rho= 0.97$ for the stylized set and 0.87 for the community set), as on the LLM side (an uncurated harvest of 12 FLAN-T5 task LoRAs at 0.84, LoRA-Retriever at 0.96). A mixed pool still deploys. Adding the six stylized adapters grows the pooled $D$ by only 3.5% (819 to 848, against 186 for the stylized set alone), and in a mixed 17-adapter pool of Pool C and stylized adapters every adapter keeps at least 99.9% of its delta energy (12 layers). New adapters are detectable before use. Every community adapter lies outside the Pool C bases (captured energy 0.15 to 0.64, mean 0.35, against 0.999 or more for in-pool adapters), and every one is fully represented after the extension of App. I (captured energy 1.00 with about 27 extra directions per layer). The union also absorbed engineering differences, such as varying per-layer ranks, unsupported layer types and file-format differences, which the merge path had to normalise case by case.

On the LLM side, the 12 harvested FLAN-T5 adapters (car-rental dialogue, SQuAD question answering, meeting summarisation and Open-Orca instruction following, from different creators) need $D = 155$ alone but add only 35 dimensions to the 48-task LoRA-Hub substrate, and the union sits further below its matched null than LoRA-Hub itself (0.67 against 0.70, App. § 12).

End-to-end check on the union. One substrate over the 37-adapter union reproduces the equal-weight ambient composition on 30 combinations (DINO similarity 0.999 against per-combination PEFT merge). A no-adapter base arm on the same prompts and seed scores 0.90 and 0.85 at $N=3$ and $N=5$ against the same merges, so the metric discriminates. This control bounds metric headroom only. The 20 combinations at $N=3$ use community adapters without trigger words, so their prompts and base image are identical.

Changing pools: basis extension and tolerance

We measure what happens when a LoRA arrives after the substrate is built, on Pool C (24 layers, energy-weighted, leave-one-out over all 17 adapters).

Projection onto the unchanged bases. A held-out adapter’s delta projected onto the substrate built from the other 16 keeps on average only 0.30 of its energy (0.21 to 0.45), and composed $N=5$ error averages 0.34 when that adapter is used immediately. A larger existing pool does not help (captured energy 0.10, 0.18, 0.26 and 0.30 at $K=4, 8, 12, 16$), and twenty uncurated community adapters capture only 0.15 to 0.64 on the Pool C bases (App. § 14). Each adapter brings its own few effective directions, so a new LoRA lies largely outside the existing span.

Basis extension. For each layer we take the new LoRA’s residual against the existing bases, $R_B = (I - UU^{\mathsf{T}})B_{\mathrm{new}}$ and $R_A = A_{\mathrm{new}}(I - VV^{\mathsf{T}})$, and append its top-$q$ singular directions to $(U,V)$. Existing coordinates are untouched. Admission takes 33 s on CPU over all 722 layers including the cache write, against 7 min 41 s on GPU for a full rebuild at $K=37$. Table 11 reports captured energy. Reaching 0.99 needs on average only 12 extra directions per layer (5 to 28) against $D \approx820$.

Update path	Captured energy	Extra directions per layer
Projection, unchanged bases	0.30	0
Extension, $q=4$	0.92	4
Extension, $q=8$	0.96	8
Extension, $q=16$	0.98	16
Extension, $q=32$	1.00	32

End-to-end check. We rebuilt both caches and regenerated 30 fixed-seed $N=5$ combinations. The extension cache reproduces the full-recompute images at DINO similarity 0.999 (worst 0.995) on the 10 combinations that contain the new LoRA, with CLIP-T unchanged. The projection cache drifts on exactly those combinations (0.982, worst 0.936) and nowhere else (0.999 on the other 20).

Sequential arrivals. Starting from a 6-adapter core and admitting the other 11 adapters one at a time, extension keeps $D$ smaller than a fresh recompute (final mean $D$ of 480 against 822). Composed error grows from 3.3% after the first arrival to 6.4% after the eleventh, against 0.84% for a fresh build. This motivates the update rule in App. V, which is to extend on arrival, monitor composed error, and rebuild periodically off the serving path.

Capability comparison

Tab. 12 compares LoRASPACE with the closest training-free methods on four requirements of the deployment we target, many LoRAs composed under a memory budget. The last two rows are measured rather than asserted (§4.2, App. I).

Requirement	CtS	KnOTS	Core-Space	Composite	Per-combo truncated merge	LoRASPACE
Serve any active subset, static or routed, from one precompute	no	no	no	n/a	no	yes
Context-dependent weights without $N$ backbone passes	no	no	no	no	no	yes
Predict before deployment whether a pool compresses	partial$^{*}$	no	no	no	no	yes ($\rho$)
Admit a newly published adapter without recomputing the pool	no	no	no	n/a	no	yes (extension)

High-cardinality prompt truncation mitigation

CLIP-L’s 77-token hard limit is reached in all 15 sampled Pool C combos at $N=7$ (maximum prompt length 92 tokens), silently dropping the last-listed LoRA’s trigger from the cross-attention context. A deterministic 20-line trigger-deduplication rule (compact) fits every combo under 74 tokens and recovers +0.36 GPT-4o composition points on average across the three LORASPACE variants (Table 13). The mitigation is orthogonal to the composition operator and applies equally to all multi-LoRA methods. Pool C headline numbers use native triggers (identical truncation applied to every method); Pool B high-cardinality cells ($N = 7, 8$) use compact.

Method	Composition (native)	Composition (compact)	$\Delta$ composition	$\Delta$ image
spatial-LoRASPACE	4.15	4.27	+0.12	−0.19
temporal-LoRASPACE	3.82	4.38	+0.56	−0.01
semantic-LoRASPACE	3.78	4.20	+0.42	−0.13
Mean	3.92	4.28	+0.36	−0.11

Pool B (SD1.5, broader pool): full high-cardinality results

Method	$N=7$ CLIP-T	$N=7$ ImgReward	$N=7$ DINO (vs. comp.)	$N=8$ CLIP-T	$N=8$ ImgReward	$N=8$ DINO (vs. comp.)
merge	32.16	0.06	0.508	32.49	0.01	0.434
switch	36.43	1.23	0.689	36.10	1.10	0.650
composite	35.65	0.61	n/a	34.47	0.39	n/a
cmlora	34.59	0.60	0.833	34.60	0.38	0.809
spatial-LoRASPACE	34.79	0.70	0.512	34.03	0.59	0.399
temporal-LoRASPACE	34.58	1.07	0.698	33.81	1.19	0.625
semantic-LoRASPACE	34.81	1.04	0.679	33.76	1.14	0.615

Routing ablation: full table

Tab. 15 reports the full per-router quality for the routing ablation summarised in § 4.5.

Mechanism ablation: per-$N$ breakdown

The mechanism comparison in § 4.3 reports means over $N = 2$ to 5. Tab. 16 gives the per-$N$ breakdown for both rank-$D$ static-delta variants on Pool A (single seed 111, 240 combos total).

subspace-merge (pool-wide $(U, V)$ baked into a static delta) and truncated-merge (per-combo SVD of the ambient merge) track each other within $\le0.12$ CLIP-T and $\le0.04$ ImageReward at every $N$, supporting the body conclusion that the basis choice does not affect quality at equal rank $D$. Both also track per-step uniform LoRASPACE (Table 3) within fp16 noise.

$N$	truncated-merge CLIP-T ↑	truncated-merge ImgRew ↑	subspace-merge CLIP-T ↑	subspace-merge ImgRew ↑
2	34.57	1.053	34.45	1.022
3	35.31	1.037	35.39	1.028
4	36.13	1.078	36.11	1.081
5	37.00	1.126	37.10	1.115
Mean	35.75	1.073	35.76	1.062

Pool C quality across active-LoRA sizes

Pool C uses 17 SDXL LoRAs across six categories (character, background, lighting, material, expression, accessory). Quality is measured at $N \in\{2, 3, 4, 5, 6\}$ across 40 stratified combinations per $N$, seed 111, 50 DPM-Solver++ steps, 1024 × 1024.

$N$	merge CLIP-T ↑	merge ImgRew ↑	composite CLIP-T ↑	composite ImgRew ↑	LoRASPACE (uniform) CLIP-T ↑	LoRASPACE (uniform) ImgRew ↑
2	29.33	1.300	31.50	1.292	31.67	1.289
3	31.32	0.894	32.95	1.108	33.60	1.089
4	28.82	0.799	31.79	1.058	32.95	1.106
5	28.53	0.426	30.48	0.829	32.01	1.028
6	29.06	0.184	32.85	0.577	34.22	0.691
Mean	29.41	0.721	31.91	0.973	32.89	1.040

Speed and precompute cost

Tables 18 and 19 report per-image wall-clock and GPU memory across all evaluated methods on Pool C (SDXL) and Pool A (SD 1.5) respectively. Precompute cost is discussed below.

Method	Time @ $N=2$ (s)	Time @ $N=5$ (s)	Peak VRAM	Load VRAM
merge	18.9	32.7	3.49 GB	2.88 GB
switch	17.5	22.0	3.49 GB	2.88 GB
composite	68.2	142.6	3.49 GB	2.89 GB
cmlora	59.5	138.5	3.49 GB	2.89 GB
spatial-LoRASPACE	37.6	30.7	4.09 GB	2.88 GB
temporal-LoRASPACE	28.7	44.3	4.09 GB	2.88 GB
semantic-LoRASPACE	31.7	34.0	4.09 GB	2.88 GB

Precompute cost. Precompute is a one-time pool-level cost: load every LoRA’s factor pair, build the energy-normalised $B$- and $A$-stacks per layer, run two thin SVDs, and project per-LoRA factors into the truncated bases. For Pool A (SD 1.5, $K=11$, $\sum_i r_i=784$, 192 LoRA-bearing layers) the dominant per-layer SVD is on a $d_{\mathrm{out}} \times\sum r_i$ matrix with $d_{\mathrm{out}} \in\{320, 640, 1280\}$; the streaming script runs end-to-end in 21.8 s SVD time + 0.7 s pickle write on CPU (single-thread Intel Xeon) and produces an 866 MB cache. For Pool C (SDXL, $K=17$, $\sum_i r_i=1792$, 722 LoRA-bearing layers) the same pipeline produces a 9.54 GB cache; the lazy per-layer streaming precompute peaks at 10.1 GiB of host memory (maximum resident set size). The cache is loaded once at pipeline initialisation (Pool A: 0.6 s; Pool C: 5.4 s on a SATA SSD) and is shared across all subsequent generations: precompute is amortised across the entire experiment and never enters the per-image latency budget.

Where the dynamic-mode speedup comes from. To separate backbone-pass avoidance from the subspace path, we add a single-pass ambient mixture that keeps all $N$ adapters active as PEFT modules and runs one backbone pass (Table 20, measured on Pool C with 15 combinations at $N=5$, seed 111, 50 steps, $1024^2$, idle GPU, medians over the 14 inter-image intervals of one sequential run per condition). Of the 5.3× gain of LORASPACE over composite at $N=5$, 4.9× comes from running one backbone pass instead of five and 1.08× from the subspace path. In isolation, the subspace path at $\tau=0.99$ is 6 to 7% slower than the ambient factor path (47.6 against 44.4 ms at 4096 tokens and 29.9 against 28.0 ms at 1024 tokens, per-layer microbenchmark summed over 48 sampled layers with the real cache factors), as Eq. (7) predicts when $D > N\bar r$. The 1.08× therefore reflects the overhead of keeping $N$ PEFT adapters active rather than an arithmetic advantage of the subspace. The ambient condition supports per-step weight changes, but per-token routing would need further implementation.

Condition	Backbone passes	Adapter path	Median s/image
composite	5	none (per-pass guidance)	181.9
Ambient single-pass	1	$N$ thin factor pairs	37.3
LoRASPACE	1	shared subspace (Eq. (5))	34.5

Additional qualitative examples

This appendix supplements Figure 3 with paired generations for further Pool C combos. Within each row the prompt and seed are held fixed across all three methods, so visual differences reflect the composition operator only. We do not show Pool A or Pool B generations, because their character LoRAs depict real people (App. § 28). For reference, Pool A per-image cost on RTX 3090 (SD 1.5, $512^{2}$, 100 DPM-Solver++ steps) is $\sim16$ s (merge), $\sim58$ s at $N=4$ or $\sim71$ s at $N=5$ (composite), and $\sim17$ s at $N=4$ or $\sim15$ s at $N=5$ (LORASPACE), with peak VRAM 3.5 GB for merge and composite and 4.1 GB for LORASPACE.

Pool C qualitative examples (SDXL, $1024^{2}$, 50 DPM-Solver++ steps). Per-image cost on RTX 3090: merge $\sim31$ s ($N=4$) / $\sim36$ s ($N=5$) / $\sim39$ s ($N=6$); composite $\sim151$ s ($N=4$) / $\sim187$ s ($N=5$) / $\sim225$ s ($N=6$); LORASPACE $\sim40$ s ($N=4$) / $\sim42$ s ($N=5$) / $\sim45$ s ($N=6$). Peak VRAM: merge 22.3 GB, composite 22.8 GB, LORASPACE 15.6 to 16.8 GB.

Synthetic pool scaling stress test

To probe the asymptotic regime beyond real LoRA pools, we run $K \in\{4, 8, 16, 32, 64\}$ random rank-8 LoRAs on SD 1.5 ($N=4$ active per call, 5 DPM-Solver++ [13] steps, $512^2$). composite’s per-step time grows from 0.119 to 0.681 s over the $16\times$ pool growth at fixed $N$, while LORASPACE stays within 0.132 to 0.164 s and is 4.8× faster at $K=64$. Load-time GPU VRAM is constant at 2.16 to 2.18 GB for LORASPACE and grows linearly for composite. The crossover from composite-faster to LoRASpace-faster occurs at $K \approx8$; the gap widens monotonically thereafter.

Note on composite scaling at fixed $N$. The per-step asymptotic in Table 4 is $O(NLd^2S)$, independent of $K$. The $\sim K$ growth measured here at $N=4$ reflects implementation-level overhead in the diffusers PEFT-loading path, which iterates over loaded LoRAs per step at small step counts. At the 50 denoising steps used in Pool C body measurements this overhead amortises and the per-step cost is dominated by the $O(NLd^2S)$ term; the body Pool C numbers (Table 18) are consistent with that asymptotic.

Method	Time @ $N=2$ (s)	Time @ $N=7$ (s)	Peak VRAM	Load VRAM
merge	23.0	43.5†	22.34 GB	18.19 GB
switch	23.3	23.4	22.83 GB	19.68 GB
composite	76.3	262.2	22.83 GB	19.69 GB
cmlora	77.1	261.5	22.83 GB	19.69 GB
spatial-LoRASPACE	34.6	48.0	15.60 to 16.80 GB	6.46 GB
temporal-LoRASPACE	32.9	42.7	15.60 to 16.80 GB	6.46 GB
semantic-LoRASPACE	32.9	42.5	15.60 to 16.80 GB	6.46 GB

Switch baseline scoreboard

We exclude single-active routing methods such as Switch [5] from the main composition comparison because they activate one LoRA at a time and never combine multiple LoRAs in a single forward pass. They are also order-sensitive by construction: rotating which LoRA fires at which step changes the output. For completeness, Table 21 reports Switch on the same pools as the main quality results.

Pool	Setting	CLIP-T ↑	ImageReward ↑	DINO (vs. comp.)
Pool A SD 1.5	mean $N = 2$ to 5, 240 combos	35.81	1.14	0.73
Pool C SDXL	mean $N = 2$ to 6, 200 combos	32.10	1.15	0.79
Pool B SD 1.5	$N = 7$, 48 combos	36.43	1.23	0.69
Pool B SD 1.5	$N = 8$, 48 combos	36.10	1.10	0.65
LoRA-Retriever	Macro Task / samp.s$^{-1}$	51.54	Macro Task at 1.82 samp.s$^{-1}$

The order-sensitivity manifests on Pool C at $N=7$ as a mean DINO self-similarity of 0.71 between canonical and reversed LoRA orderings, with 10/10 tested combos falling below the 0.95 order-invariant threshold. Composition methods (merge, composite, LORASPACE) are order-invariant by design.

LLM D ablation: saturation and rank-orthogonality

The rank-overlap framework predicts two qualitatively different regimes for $D$ as $K_{\mathrm{pool}}$ grows. In the rank-bound regime ($\sum_i r_i < d$), $D$ grows sub-linearly with $K$ when LoRAs share structure. In the saturation regime ($\sum_i r_i \ge d$), $D$ is capped at the layer width and each new LoRA adds at most one new direction.

We measure both regimes empirically.

LoRA-Retriever near-orthogonality (rank-bound regime). The LLaMA-2-7B LoRA-Retriever pool ($K=48$) [35] has uniform $r_i = 8$, $\sum_i r_i = 384$, against $d = 4096$. Across pool size sweeps with $K \in\{2, 4, 8, 16, 32, 48\}$, the rank-fill ratio stays at $\rho\ge0.94$ throughout, with $\beta= 0.94$. LoRAs trained on disjoint NLP tasks (sentiment, machine translation, reading comprehension) contribute nearly orthogonal directions, consistent with tasks that share little structure (the pool sits at its spectrum-matched null, App. F).

LoRA-Hub saturation crossing (saturation regime). The LoRA-Hub FLAN-T5-Large pool [9] has $K = 196$ LoRAs with uniform $r_i = 16$ and layer width $d = 1024$. Saturation occurs at $K^* = d/r = 64$, where $K \cdot r = d$. We measure $\bar{D}$ across $K \in\{4, 8, 16, 32, 64, 128, 196\}$. Below $K^*$, each new LoRA adds approximately 11 new dimensions to $\bar{D}$. Above $K^*$, each new LoRA adds approximately 1 new dimension, consistent with the saturation prediction. At the full pool $K = 196$, $\bar{D} = 810$, which is 79% of layer width: the subspace never fully fills even at $3 \times$ saturation $K$, leaving residual capacity for additional LoRAs.

This is the first measurement we are aware of that crosses the saturation boundary in a real LoRA pool. It validates the framework’s prediction of a regime change at $K = d/r$ and provides an empirical anchor for what to expect at very large pool sizes. Our diffusion pools are already width-saturated on most layers (App. § 10), but none was swept across the boundary, so LoRA-Hub is the only pool where we observe the crossing itself.

LLM cross-modality: full scoreboard and extended discussion

Table 22: Full LoRA-Retriever benchmark [35] on LLaMA-2-7B [28] (48 LoRAs, 2,395 samples, 14 methods). Bold = best non-Oracle; _underline_ = best non-LoRASPACE non-Oracle.

Method	Macro Task	Macro Domain	samp/s
base (no LoRA)	4.24	4.17	0.60
oracle (task-matched)	57.54	59.55	1.91
merge (Task Arithmetic)	45.72	46.96	2.18
ties	48.71	51.39	1.93
dare	40.86	41.22	2.04
knots	45.44	47.01	0.31
spectr	41.90	42.74	0.97
lorahub	46.19	46.68	0.12
switch	51.54	52.69	1.82
loraretriever-mixture	_53.55_	_56.43_	1.47
loraretriever-fusion	43.56	44.14	1.77
loraspace-fullpool (no retrieval)	50.49	53.00	1.48
loraspace-instructor	54.01	56.79	1.44
loraspace-hybrid	54.51	57.20	1.52

Why loraretriever-fusion underperforms loraretriever-mixture. Fusion mode forms a fused left factor $\bar{A} = \sum_p w_p A_p$ and fused right factor $\bar{B} = \sum_p w_p B_p$, then applies $\bar{B}\bar{A}$: this introduces cross-products $B_p A_q$ for $p \ne q$, which are not part of the target delta sum $\sum_p w_p B_p A_p$. These cross-terms are approximation errors that degrade composition quality relative to the clean mixture mode, explaining the substantial score difference (43.56 vs. 53.55 Macro Task).

Ablation decomposition. The loraspace-hybrid score (54.51) decomposes as: loraspace-fullpool (50.49) + INSTRUCTOR retrieval gate (+3.52) + calibrated spectral routing over retrieved subset (+0.50). The substrate alone (50.49) is above all static merges but below the retrieval-augmented baselines. The retrieval gate is the dominant quality driver, as expected for a pool of 48 highly diverse task LoRAs.

Per-domain breakdown. loraspace-hybrid beats Oracle on five of ten domains (commonsense, closed-book QA, sentiment, coreference, translation) where blending complementary LoRAs helps. Losses concentrate on format-sensitive domains (reading comprehension, NLI, structured generation) where blending dilutes the precise output format required by the task-matched LoRA.

Limitations

This appendix collects the empirical scope and assumptions of the work referenced from § 5.

Pool coverage and post-saturation behaviour. The empirical characterisation in § 4.2 covers three diffusion pools (Pool A, B, C) and two LLM pools (LoRA-Retriever and LoRA-Hub). The diffusion pools already exceed the layer width on most layers (App. § 10), but only the LoRA-Hub pool is swept across the saturation boundary (Appendix § 26), so how $D$ grows past it on diffusion pools is extrapolated rather than measured.

Changing pools. The substrate is computed for a given pool, and in deployment pools change. An unchanged substrate has no tolerance budget for a new LoRA. Projected onto the existing bases, a held-out Pool C adapter keeps only 0.30 of its energy on average, and a larger existing pool does not fix this (App. § 15). Extending the bases with the new adapter’s residual directions admits it in 33 s on CPU without recomputing the pool, and reproduces the full-recompute images, but the composed error grows slowly as arrivals accumulate (3.3% after the first to 6.4% after the eleventh, against 0.84% for a fresh build). The update rule we use is to extend on arrival, monitor composed error, and rebuild periodically off the serving path. Removing an adapter needs no recomputation, because the bases still span the remaining adapters.

Interpreting the rank-fill ratio. The rank-fill ratio may combine within-LoRA redundancy with cross-LoRA overlap. App. § 12 quantifies the two with the decomposition $\rho= \rho_{\mathrm{intra}} \times\rho_{\mathrm{cross}}$ and a spectrum-matched null.

Efficiency baselines. Efficiency conclusions depend on the baseline implementation. Our merge, composite and single-pass ambient baselines use the PEFT runtime. Against a single-pass ambient mixture, the dynamic-mode gain at $\tau= 0.99$ comes from backbone-pass avoidance rather than from arithmetic inside the subspace (App. § 22).

Seed coverage of routing and mechanism ablations. The routing ablation (§ 4.5) is reported as mean $\pm$ across-seed standard deviation over three random seeds (240 stratified combinations per seed); the mechanism ablation (§ 4.3) is reported at a single seed. The main quality table (§ 4.3) reports CLIP-T and ImageReward across three random seeds with small across-seed standard deviations; GPT-4o evaluation is from one seed due to API cost. These are precision limitations rather than soundness ones: the across-seed standard deviations do not cross the inter-method gaps on ImageReward (where the routing pattern is established), and the pattern is consistent across $N$.

Layer scope. The operator is currently formulated for $1 \times1$ linear projections, which covers all attention and most MLP modules in the diffusion and LLM backbones we test. Extensions to convolutional layers with kernel size larger than one, and to non-linear LoRA formats (DoRA, AdaLoRA), are out of scope here.

Router scope. The four routers we evaluate (uniform, spatial, temporal, semantic, plus the external retrieval coupling) are sufficient to demonstrate that the operator interface admits arbitrary $\{w_i\}$, but they are simple hand-designed examples and not exhaustive. Learned routers, in particular, may exploit the shared subspace structure in ways not captured by hand-designed signals. We see this as a natural extension rather than a limitation of the framework itself.

Asset and identity considerations. The three Pool A character LoRAs (also used in Pool B) are public CivitAI LoRAs that reproduce the likeness of real people, and CivitAI has since removed them. We use them only as inputs to the composition benchmark. We do not show their generations, name the people, or redistribute the weights, and our released pool manifests list them by SHA-256 hash only. We do not propose new likeness models. Downstream use should follow source platforms’ policies and applicable law.

LoRA pools and prompts

This appendix lists every LoRA used in our diffusion and LLM pools, with category groupings, trigger words, and example combo prompts. SHA-256 hashes and stable source URLs are in the per-pool JSON manifests in the pools/ directory of the released code.

Pool A (SD1.5, 11 LoRAs, reality style)

Five categories: 3 character + 2 clothing + 2 style + 2 background + 2 object. All LoRAs at $r=32$, $\alpha=16$ unless noted.

Table 23: Pool A LoRAs with trigger words. Source: CivitAI (URLs and SHA-256 hashes in the pool manifests).

ID	Name	Trigger words
character
character_1	Real-person likeness LoRA	identity trigger and appearance descriptors (omitted)
character_2	Real-person likeness LoRA	identity trigger and appearance descriptors (omitted)
character_3	Real-person likeness LoRA	identity trigger and appearance descriptors (omitted)
clothing
clothing_1	Thai University Uniform	mahalaiuniform, white shirt short sleeves, black pencil skirt
clothing_2	School Dress	school uniform, white shirt, red tie, blue pleated microskirt
style
style_1	Japanese Film Color Style	film overlay, film grain
style_2	Bright Style	bright lighting
background
background_1	Library Bookshelf	lib_bg, library bookshelf
background_2	Forest	slg, river, forest
object
object_1	Umbrella	transparent umbrella
object_2	Bubble Gum	bubble gum

Pool A prompt template. Identical across all combos in the pool; combo selection only changes which trigger lists get concatenated.

RAW photo, subject, 8k uhd, dslr, high quality, Fujifilm XT3, half-length portrait from knees up, ⟨trigger_1⟩, ⟨trigger_2⟩, ...

For the negative prompt we use the 44-token ComposLoRA reality negative (artifact + style-contamination + anatomy-failure tokens).

Example Pool A combos.

• $N=2$, [character_1, background_2]: “RAW photo, subject, …, ⟨character_1 triggers⟩, slg, river, forest”.

• $N=5$, [character_1, clothing_1, style_1, background_1, object_1]: “RAW photo, subject, …, ⟨character_1 triggers⟩, mahalaiuniform, white shirt short sleeves, black pencil skirt, film overlay, film grain, lib_bg, library bookshelf, transparent umbrella”.

Pool B (SD1.5, 17 LoRAs, Pool A + 6 high-cardinality)

Pool A’s 11 LoRAs plus 6 new ones across 3 new categories: detail (2), pose (2), expression (2). Probes the high-cardinality regime ($N \in\{7,8\}$) where rank-fill is lowest ($\rho=0.47$ at $K=17$).

Pool B trigger deduplication. At $N \ge7$, the body of the prompt approaches CLIP-L’s 77-token limit. The Pool B body experiments use a compact trigger-deduplication rule (App. § 17) that drops repeated tokens across active LoRAs before assembly.

Pool C (SDXL, 17 LoRAs)

Six categories: 3 character ($r=32$, $\alpha=16$) + 3 background + 4 lighting + 4 material + 2 expression + 1 accessory (last five at $r=128$, $\alpha=1$). SDXL backbone, $1024^{2}$ generation.

ID	Name	Trigger words
detail (quality enhancers, no explicit trigger)
detail_1	Detail Tweaker	(none)
detail_2	Add More Details	(none)
pose
pose_1	Shirt Tug Pose	shirt tug
pose_2	Covering Eyes Pose	covering eyes
expression
expression_1	Screaming / Open Mouth	open_mouth, screaming
expression_2	Sharp Eyes / Kubrick Stare	sharp_eyes

ID	Name	Trigger words
character
character_1	SL Nobodies CamelliaXL	CamelliaXL, a young woman, detailed face
character_2	SL Nobodies HelenaXL	HelenaXL, a young woman, detailed face
character_3	SL Nobodies VioletXL	VioletXL, a young woman, detailed face
background
background_1	Polyhedron Catwalk	catwalk, runway, fashion show
background_3	Polyhedron Power Grid	energy, high voltage, power pole, transformer station, tangled cables
background_4	Polyhedron Dandelions	dandelion, dandelion seed, yellow dandelion
lighting
lighting_2	Polyhedron God Rays	god rays, sunbeams, sunbeam
lighting_3	Polyhedron Chiaroscuro	chiaroscuro style, dim light
lighting_4	Polyhedron Fog	fog, haze, foggy, mist
lighting_5	Polyhedron Golden Hour	golden hour, sunset, sunrise
material
material_1	Polyhedron Corrosion	corrosion, rust
material_2	Polyhedron Old Wood	old wood, wooden planks, wooden wall, wooden floor
material_3	Polyhedron Old Gold	gold, old gold
material_4	Polyhedron Old Copper	old copper, copper, patina, copper pot
expression
expression_1	Polyhedron Anger	anger, screaming, angry
expression_3	Polyhedron Fear	fear, fearful, anxious
accessory
accessory_3	Sunglasses on Top of Head	with sunglasses on top of head

Example Pool C Combos.

• $N=4$, [character_1, background_1, lighting_2, material_1]: “RAW photo, subject, . . . , CamelliaXL, a young woman, detailed face, catwalk, runway, fashion show, god rays, sunbeams, sunbeam, corrosion, rust”.

• $N=6$, [character_2, background_3, lighting_4, material_2, expression_1, accessory_3]: “RAW photo, subject, . . . , HelenaXL, . . . , energy, high voltage, power pole, transformer station, tangled cables, fog, haze, foggy, mist, old wood, wooden planks, wooden floor, anger, screaming, angry, with sunglasses on top of head”.

LLM pool (LLaMA-2-7B, 48 task LoRAs, LoRA-Retriever benchmark)

48 task-specialised LoRAs from the LoRA-Retriever benchmark [35], all $r=8$. We adopt the authors’ 10-domain grouping verbatim.

Domain (count)	Tasks
closed-book QA (4)	arc_challenge, arc_easy, natural_questions, trivia_qa
commonsense (4)	copa, hellaswag, piqa, story_cloze
coreference (2)	definite_pronoun_resolution, wsc
NLI (10)	anli_r1, anli_r2, anli_r3, cb, mnli_matched, mnli_mismatched, qnli, rte, snli, wnli
paraphrase (4)	glue_mrpc, glue_qqp, paws_wiki, stsb
RC w/ commonsense (2)	cosmos_qa, record
reading-comprehension (6)	bool_q, drop, multirc, openbookqa, squad_v1, squad_v2
sentiment (4)	imdb_reviews, sentiment140, sst2, yelp_polarity_reviews
structured-to-text (4)	common_gen, dart, e2e_nlg, web_nlg_en
translation (8)	para_crawl_enes, wmt14_enfr, wmt16_translate_csen, wmt16_translate_deen, wmt16_translate_fien, wmt16_translate_roen, wmt16_translate_ruen, wmt16_translate_tren

LLM evaluation uses task-native prompts directly from the LoRA-Retriever benchmark (no LoRA trigger tokens; each LoRA is task-specialised, not concept-keyed). Multi-task composition replaces “compose multiple LoRAs for one image” with “compose multiple LoRAs for one prompt”: at inference, we activate either the full 48-pool substrate (loraspace-fullpool) or a top-$K$ retrieved subset (loraspace-instructor / loraspace-hybrid).

Combo selection protocol

For each pool we use stratified random sampling over $\binom{K}{N}$ combos with category-balance constraints (each combo activates at most one LoRA per category where pool design allows). Pool A and Pool B use 240 combos per seed across $N \in\{2, 3, 4, 5\}$. Pool C uses 200 combos per method (40 per $N$ across $N \in\{2, \ldots, 6\}$). Identical combo lists are shared across all methods within a pool to enable per-combo paired comparison.