betaFeedback

WorksComputer Science

In-context learning from self-generated trajectories for adaptive model reduction

Abstract

High-fidelity simulations of complex physical systems are often too expensive for repeated prediction, design, and control. Reduced-order models address this computational cost by compressing high-dimensional dynamics into low-dimensional representations that can be evaluated rapidly, but they often lose accuracy when online dynamics drift beyond the offline training data. Adaptive methods address this limitation by updating the reduced subspace online using external, out-of-span information, such as full-order corrections or sensor snapshots. We discovered that a complementary and previously unexploited in-span adaptation channel exists within the current reduced subspace. To exploit this channel, we continually update the reduced representation using the model's own predictions through an incremental singular-value decomposition with a forgetting factor. This produces a trajectory-informed spectral preconditioner in which the reduced subspace remains unchanged, while the basis is reweighted and realigned according to the directions visited by the evolving dynamics. This internal reorganization prepares the basis to absorb future out-of-span corrections more effectively, improving adaptation without requiring additional external information. We expose the mechanism through a three-dimensional spiral example and demonstrate its benefits on nonlinear partial differential equations, including viscous Burgers and Fisher–KPP dynamics. We also discuss how in-span learning can be interpreted as a dynamical-systems analogue of in-context learning. More broadly, in-span learning suggests a new principle for computational science, revealing that model-generated trajectories contain more usable information than previously recognized.

Introduction

Introduction

Across science and engineering, applications such as aircraft design [1], weather forecasting [2, 3, 4], combustion predictions [5], power-grid control [6], uncertainty quantification [7, 8], and digital twins [9, 10] all rely on numerical models that require expensive computations. There is also a need to run these computations repeatedly across parameters, controls, uncertain inputs, or real-time decisions. This setting calls for simulators that are accurate enough to trust and inexpensive enough to use freely.

Reduced-order models (ROMs) address this need by compressing high-dimensional simulations into low-dimensional representations [11, 12, 13, 14]. Although a discretized physical model may contain millions of degrees of freedom, its relevant trajectories often occupy a much smaller region of state space. Projection-based ROMs exploit this structure by constructing a low-dimensional subspace, commonly from proper orthogonal decomposition (POD) of full-order snapshots [15, 16, 17], and evolving only the coordinates of the solution in that subspace [18]. When the online dynamics remain close to the training data, this compression can deliver orders-of-magnitude speed-ups [21]. The same compression also creates the central failure mode of ROMs. A static ROM is trained offline and deployed with a fixed basis. It can interpolate accurately within the regime represented by its snapshots, but it loses predictive power when the online trajectory drifts beyond that regime. This is a form of distribution shift that exists whether the reduced representation is a linear subspace, a nonlinear manifold [23], or a learned surrogate [22]. For projection-based ROMs considered in this work, this limitation is especially severe for systems with slowly decaying Kolmogorov NN-width, where no fixed low-dimensional linear subspace can represent the evolving solution uniformly well over long times [24, 25]. Transport-dominated flows, shocks, moving fronts, combustion, and detonation dynamics are typical examples [26, 27, 28, 29, 30]. They are also among the settings where fast predictive simulation would be most valuable.

Adaptive ROMs attempt to overcome this rigidity by allowing the reduced representation to evolve during the online simulation. Early efforts included interpolation between locally trained bases [31, 32, 33, 34]. Pioneered by Peherstorfer & Willcox [35, 36], incremental basis approaches have gained attention, where the basis is updated online with newly acquired information [37, 38, 39, 40, 41]. A related line of work draws on subspace-tracking ideas [42, 43, 44, 45, 46, 47, 48, 49], which have motivated geometric basis updates [50] and history-aware adaptation [51]. These incremental basis-updating approaches differ in how they obtain new information and how they update the basis, but they share a common premise. The basis is adapted using information with a nonzero component outside the current reduced subspace. This external information may come from a full-order state, a sensor snapshot, or another correction oracle. We call such data out-of-span. The premise is natural because only out-of-span information can move the subspace. A snapshot that lies exactly inside the current subspace has zero residual and therefore appears, from the usual subspace-update viewpoint, to add no value.

In this paper, we show that this reasoning misses a complementary adaptation channel. Between external corrections, a ROM continuously produces predictions of its own state. By construction, these predictions lie inside the current reduced subspace, and we therefore call them in-span. They cannot expand the subspace, but they can still reorganize the representation inside it. We show that when the ROM’s own predictions are streamed through an incremental singular-value decomposition (iSVD) [43, 44, 45] with forgetting, they rotate the orthonormal basis within the same subspace and reweight its singular spectrum. The result is a trajectory-informed spectral preconditioner. It does not change what the ROM can represent at that instant, but it changes how the basis is prepared to absorb the next external correction. We refer to the resulting adaptive ROM as SPIN, for Spectral Preconditioning via IN-span learning (Figure 1). Throughout the paper, we use baseline adaptive ROM for the iSVD-based adaptive model that only uses external out-of-span corrections and performs no in-span updates between correction events. It is notable that the baseline adaptive ROM is itself representative of the state-of-the-art [51].

Figure 1. In-span learning adapts a reduced-order model from its own predictions. Three reduced-order modeling strategies are shown through their offline/online timelines and the corresponding evolution of the reduced representation. (a) Three-dimensional spiral prediction comparing the exact full-order model (FOM) trajectory, static ROM, baseline adaptive ROM, and SPIN ROM. (b) A static ROM is trained offline and then deployed online with no adaptation. Its reduced trajectory remains confined to the original static subspace and can lose track of the full-order trajectory as the dynamics leave the training regime. (c) A baseline adaptive ROM queries external out-of-span information during the online phase, such as a correction snapshot qcorrq_{\mathrm{corr}}, and uses it to move the subspace toward the full-order dynamics. (d) The proposed SPIN ROM uses the same external correction events, but also streams its own intermediate predictions as in-span information. These in-span updates do not move the subspace by themselves, but they reweight and reorient the basis before the next correction arrives, preparing the reduced representation to absorb out-of-span information more effectively.

We develop in-span learning first at the level of the update mechanism, then in a closed-form spiral example, and finally in nonlinear PDEs where the same mechanism improves long time-horizon prediction. Across these examples, the same conclusion emerges. Reduced models can learn not only from new data outside their span, but also from the trajectory they have already produced inside it.

Results

A hidden adaptation channel inside the current subspace

We start with a brief overview of the in-span learning mechanism, showing how a model’s own predictions can contribute to its adaptation. A summary of the notation used throughout the paper is provided in Section A. Consider a ROM with orthonormal basis Φ∈RN×r\Phi\in\mathbb{R}^{N \times r} and singular values collected in Σ:=diag⁡(σ1,…,σr)\Sigma:= \operatorname{diag}(\sigma_1,\ldots,\sigma_r). At any time step, the ROM prediction has the form qROM:=Φaq_{\mathrm{ROM}} := \Phi a, where a∈Rra \in\mathbb{R}^{r} is the reduced coordinate vector. This prediction lies exactly in the current reduced subspace, so an iSVD update sees zero orthogonal residual and cannot add a new direction. The point of in-span learning is that this zero-residual snapshot can still change the basis state. Let C:=Σ2C := \Sigma^2 denote the current covariance represented in reduced coordinates. With forgetting factor γin∈(0,1]\gamma_{\mathrm{in}} \in(0,1], the in-span iSVD update reduces to a rank-one update of this reduced covariance,

C+=γin2C+aaT.(1)C^{+} = \gamma_{\mathrm{in}}^{2} C + aa^{\mathsf{T}} . \tag*{(1)}

Diagonalizing this updated small r×rr \times r matrix, C+=UΛUTC^+ = U\Lambda U^{\mathsf{T}}, gives the new squared singular values through Λ\Lambda and an orthogonal rotation U∈Rr×rU \in\mathbb{R}^{r \times r} of the basis inside the same subspace. Equivalently,

Σ+=Λ1/2,Φ+=ΦU,span⁡(Φ+)=span⁡(Φ).(2)\Sigma^+ = \Lambda^{1/2}, \qquad\Phi^+ = \Phi U, \qquad\operatorname{span}(\Phi^+) = \operatorname{span}(\Phi). \tag*{(2)}

Thus, an in-span update preserves the geometric object that determines what states the ROM can represent, but changes the spectral object that determines the modal weights. The formal iSVD update, the derivation of Eq. (1), and the proof of Eq. (2) are given in § 5.4 and § 5.5.

(1) gives the mechanism a simple interpretation. The vector aa records which directions of the current basis the ROM trajectory is using. The update discounts the previous spectral state by γin2\gamma_{\mathrm{in}}^2 and adds the rank-one contribution aaTaa^{\mathsf{T}} from the current ROM prediction. A direction that is strongly represented in the recent trajectory can be reinforced, whereas a direction that receives too little trajectory energy can decay. The outcome is trajectory-dependent and controlled by the forgetting factor, which sets the memory horizon of in-span learning. With γin\gamma_{\mathrm{in}} close to one, the ROM accumulates many recent predictions and reorganizes the basis around sustained dynamics. With smaller γin\gamma_{\mathrm{in}}, it emphasizes the most recent trajectory and can rapidly suppress inactive directions. The reinforcement–suppression threshold is derived in § 5.5.

This in-span update matters because the adaptive ROMs studied here operate at fixed rank. When the next external correction arrives, the new out-of-span information must be merged with the existing basis and then truncated back to rank rr. The outcome of that truncation depends not only on the incoming correction, but also on the spectral state and orientation of the basis that receives it. In-span learning therefore acts as a trajectory-informed spectral preconditioner for the next out-of-span update. This mechanism uses no additional full-order information. Between external corrections, the ROM already produces the states used in the in-span update. The only extra operation is a small SVD in the reduced space, followed by a rotation and reweighting of the existing basis and the usual refresh of basis-dependent quantities in the hyper-reduced model when needed. The cost and implementation details are described in § 5.9 and § 5.7.

A toy example to expose a key mechanism

To illustrate in-span learning in a simple setting, we begin with a three-dimensional spiral whose full-order dynamics are known exactly as q(t):=[cos⁡tsin⁡tαt]Tq(t) := [\cos t \quad\sin t \quad\alpha t]^{\mathsf{T}} with α=0.4\alpha= 0.4. This trajectory solves an affine linear system in R3\mathbb{R}^3, which we advance with its exact discrete flow. The full setup is given in § 5.1.

The initial ROM is deliberately minimal. We form a rank-2 POD basis Φ0\Phi_0 from only two full-order snapshots, q1q^1 and q2q^2, and project the exact discrete dynamics onto the resulting plane S0:=span⁡(Φ0)=span⁡(q1,q2)S_0 := \operatorname{span}(\Phi_0) = \operatorname{span}(q^1,q^2). The static ROM trajectory remains confined to S0S_0 and drifts as the true spiral leaves this plane. This small example already contains the essential failure mode of a static ROM, in which the model can evolve only inside the space it was given.

The in-span updates use only the ROM trajectory. At an intermediate online time step kk, before the next external correction, the ROM produces qROMk:=Φ0ak∈S0q_{\mathrm{ROM}}^k := \Phi_0 a^k \in S_0, where ak∈R2a^k \in\mathbb{R}^2 is the reduced coordinate vector in the initial basis Φ0\Phi_0. Let n1,…,nsn_1,\ldots,n_s denote the time indices of the ss in-span ROM predictions streamed since the previous external correction. After these ss updates, the reduced covariance is

Cs=γin2sΣ02+∑j=1sγin2(s−j)anj(anj)T,(3)C_s = \gamma_{\mathrm{in}}^{2s}\Sigma_0^2 + \sum_{j=1}^{s}\gamma_{\mathrm{in}}^{2(s-j)}a^{n_j}(a^{n_j})^{\mathsf{T}}, \tag*{(3)}

where Σ0\Sigma_0 contains the initial singular values associated with Φ0\Phi_0. We use a small value γin=0.1\gamma_{\mathrm{in}} = 0.1 to make the spectral contraction visible in the spiral illustration; in the PDE experiments, the inspan forgetting factors are chosen ≈1\approx1. Writing Cs=UsΛsUs⊤C_s = U_s \Lambda_s U_s^{\top} gives the preconditioned singular values through Λs1/2\Lambda_s^{1/2} and the in-plane basis rotation through Φs=Φ0Us\Phi_s = \Phi_0 U_s, with span⁡(Φs)=S0\operatorname{span}(\Phi_s) = \mathcal{S}_0.

Figure 2 shows this effect geometrically. The full-order spiral continues in three dimensions, while the ROM trajectory evolves inside the initial rank-2 plane. In this particular example, the reduced trajectory repeatedly visits one dominant in-plane direction while leaving the other nearly inactive. The covariance ellipse therefore rotates toward the visited direction and contracts along the direction that receives little trajectory energy. The initial singular values Σ0=diag⁡(2.92,0.25)\Sigma_0 = \operatorname{diag}(2.92, 0.25) are transformed by four in-span updates into ΣSPIN=diag⁡(2.48,0.03)\Sigma_{\mathrm{SPIN}} = \operatorname{diag}(2.48, 0.03). The second singular value is reduced by almost an order of magnitude, while the subspace itself is unchanged.

Figure 2. A closed-form spiral reveals in-span spectral preconditioning. (a) The exact full-order spiral trajectory (FOM), the rank-2 ROM trajectory advanced inside its fixed initial subspace S0\mathcal{S}_0, and the growing discrepancy (error) between them; the two snapshots q1q^1, q2q^2 used to build the initial basis are highlighted, and the grey plane marks S0\mathcal{S}_0. (b)–(e) Covariance ellipses inside S0\mathcal{S}_0 after s=1,2,3,4s = 1, 2, 3, 4 in-span updates, drawn in the fixed S0\mathcal{S}_0 coordinate frame with axes along ϕ0,1\phi_{0,1} and ϕ0,2\phi_{0,2}. Each ellipse is set by the eigenvectors and eigenvalues of the reduced covariance CsC_s. Although the in-span updates use only ROM-predicted states lying in S0\mathcal{S}_0 and therefore add no new direction, the spectrum rotates and contracts. The active direction is reinforced while the unvisited direction is progressively suppressed. The subspace is unchanged; only the basis orientation and singular spectrum evolve.

Here, we separate two objects that are often conflated. The subspace determines which states the ROM can represent, while the spectrum assigns importance to the directions already present within that subspace. The in-span recursion leaves the first object unchanged but modifies the second. On its own, this cannot correct the ROM, because no new direction has entered the span. Its effect becomes visible when an out-of-span correction later arrives.

In-span preconditioning changes how the next correction is absorbed

To receive an out-of-span correction without having access to the exact full-order trajectory, we perform a full-order operator query. In the spiral example, the exact one-step flow is the affine map qk+1=Gqk+cq^{k+1} = Gq^k + c, where GG and cc are obtained from the exact spiral dynamics in § 5.1. Starting from the ROM state qROM5q^5_{\mathrm{ROM}}, this gives qcorr:=GqROM5+cq_{\mathrm{corr}} := Gq^5_{\mathrm{ROM}} + c. At the first out-of-span correction, this snapshot is identical for the baseline adaptive ROM and the SPIN ROM. Its residual relative to the initial ROM plane has norm ρ:=∥qcorr−Φ0Φ0Tqcorr∥2=0.42\rho:= \lVert q_{\mathrm{corr}} - \Phi_0\Phi_0^{\mathsf{T}}q_{\mathrm{corr}}\rVert_2 = 0.42, so both methods receive the same out-of-span information. The difference is the basis state that receives it. In the baseline adaptive ROM, the correction is applied to the original spectrum; in the SPIN ROM, it is applied after the ROM trajectory has preconditioned the spectrum.

We now show that, in this example, in-span preconditioning acts as a weak-mode suppressor. The ROM trajectory has reduced the strength of a nearly inactive in-plane direction, so the incoming residual faces a smaller modal competitor when the update is truncated back to rank rr. To quantify this local rank competition, we compare the residual strength ρ2\rho^2 with the weakest retained in-plane direction before the residual is added. For a correction with in-plane coefficients α=ΦpreTqcorr\alpha= \Phi_{\mathrm{pre}}^{\mathsf{T}}q_{\mathrm{corr}}, define Ccorr:=γout2Σpre2+ααTC_{\mathrm{corr}} := \gamma_{\mathrm{out}}^2\Sigma_{\mathrm{pre}}^2 + \alpha\alpha^{\mathsf{T}}, which is the in-plane block of the small iSVD core covariance. We then define the residual-to-weakest-mode ratio η:=ρ2λr(Ccorr)\eta:= \frac{\rho^2}{\lambda_r(C_{\mathrm{corr}})}. Here, η≪1\eta\ll1 indicates that the residual is weak relative to the existing in-plane core content, while η≫1\eta\gg1 indicates that it can compete for one of the retained rank-rr slots. This ratio is used only as a diagnostic of the local rank competition; the actual truncated update is determined by the full core SVD, which is examined in Section B.

For the same correction snapshot, and with γout=1.0\gamma_{\mathrm{out}} = 1.0 for simplicity, the baseline adaptive update gives λr(Ccorr,baseline)=1.45\lambda_r(C_{\mathrm{corr, baseline}}) = 1.45 and ηbaseline≈0.12\eta_{\mathrm{baseline}} \approx0.12, whereas the SPIN update gives λr(Ccorr,SPIN)=0.05\lambda_r(C_{\mathrm{corr, SPIN}}) = 0.05 and ηSPIN≈3.67\eta_{\mathrm{SPIN}} \approx3.67. The residual norm has not changed. The increase in η\eta comes from the denominator, meaning that the same residual now competes against a much smaller in-plane core direction.

The geometric outcome is shown in Figure 3. With out-of-span adaptation alone, the updated plane changes only modestly and the new residual is mostly discarded. With in-span preconditioning, the updated plane swings toward the new local direction of the spiral. Quantitatively, enabling in-span learning increases the residual capture from 0.26 to 0.80 and the plane-change angle from 15.06∘15.06^\circ to 53.29∘53.29^\circ, while decreasing the correction error from 1.02×10−21.02 \times10^{-2} to 8.84×10−48.84 \times10^{-4}. These diagnostics are defined in § 5.8. Full numerical details for the spiral example are provided in Section B, and a comparison with several alternative fixed-rank basis updates is available in Section C.

Figure 3. The same correction is better absorbed after in-span preconditioning. Both adaptive ROMs receive the same correction qcorrq_{\mathrm{corr}} and use the same truncated iSVD update; only the incoming basis state differs. (a,b) Baseline adaptive and SPIN updates. In-span preconditioning rotates and reweights the basis before the correction arrives, so the post-update plane swings more strongly toward the local dynamics. (c) Modal slot competition, including correction energy, λr(C)\lambda_r(C), and η=ρ2/λr(C)\eta= \rho^2/\lambda_r(C). (d) Residual capture, plane-change angle, and correction error. In this spiral example, in-span preconditioning suppresses inactive modal content so the same residual can occupy the weak slot of the fixed-rank basis.

SPIN improves nonlinear PDE prediction

The spiral example isolates the mechanism, but it does not test whether the same effect survives the numerical ingredients of a practical ROM. We therefore consider two nonlinear PDEs, namely viscous Burgers and Fisher–KPP, solved with implicit time integration and hyper-reduced (via QDEIM [52, 53]) least-squares Petrov–Galerkin (LSPG) [54, 19] ROMs. This setting tests in-span learning within a broadly used framework for nonlinear scientific simulation. Viscous Burgers dynamics tests transport of a coherent structure, while Fisher–KPP reaction–diffusion tests front propagation and saturation driven by a nonlinear reaction term. In both cases, all reduced models are initialized from a negligible offline training phase, so the comparison is deliberately extrapolative.

For both PDEs we compare the full-order model, a static ROM, the baseline adaptive ROM, and the proposed SPIN ROM. The two adaptive ROMs use the same rank, QDEIM sample count, correction interval, and full-order operator query mechanism to receive the correction snapshot. The only algorithmic difference is that SPIN also updates its basis and singular values between external corrections using the ROM’s own intermediate predictions. The full discretizations, LSPG–QDEIM formulation, correction procedure, and hyperparameters are given in § 5.2 and § 5.6.

For each adaptive model, the forgetting factors reported here are selected from an ablation sweep that minimizes the mean relative L2L_2 error over the online trajectory. Thus, the comparison is between optimized memory configurations of the two adaptive strategies. For Burgers, the best baseline adaptive ROM uses γout=0.01\gamma_{\mathrm{out}} = 0.01, while the best SPIN ROM uses (γin,γout)=(1.0,0.25)(\gamma_{\mathrm{in}}, \gamma_{\mathrm{out}}) = (1.0, 0.25). For Fisher–KPP, the corresponding values are γout=0.1\gamma_{\mathrm{out}} = 0.1 and (γin,γout)=(0.9,0.25)(\gamma_{\mathrm{in}}, \gamma_{\mathrm{out}}) = (0.9, 0.25). The full forgetting-factor ablations, correction-interval sweeps, and additional PDE robustness tests are reported in Section D, with the benchmark parameters summarized in Section E for reproduction

purposes.

Spectral diagnostics confirm the in-span mechanism

Figure 4 shows Burgers predictions, where we use a rank-4 ROM with m=4m = 4 QDEIM sample points, trained from only the first four full-order snapshots from t=0t = 0 to t=0.004t = 0.004, and evaluated over a 500-step prediction horizon from t=0t = 0 to t=0.50t = 0.50, with out-of-span corrections every zs=10z_s = 10 time steps. The static ROM fails quickly because its basis represents only the early offline regime. The baseline adaptive ROM performs substantially better and follows the advecting pulse reasonably well, showing that the external corrections supply useful missing directions. Adding the in-span channel further reduces amplitude and shape errors as the pulse moves across the domain, and the SPIN error remains consistently below the baseline adaptive error without adding full-order correction steps.

Figure 4. SPIN ROM improves viscous Burgers prediction. Viscous Burgers dynamics with rank r=4r = 4 and m=4m = 4 QDEIM sample points. The offline basis is trained on the initial 4 time steps from t=0t = 0 to t=0.004t = 0.004, while prediction is evaluated over 500 time steps from t=0t = 0 to t=0.50t = 0.50, with external corrections every zs=10z_s = 10 steps. (a) Solution profiles u(x,t)u(x,t) at four reported times. The static ROM remains tied to the early training regime and drifts away from the transported pulse. The baseline adaptive ROM follows the FOM more closely, while the SPIN ROM further reduces the amplitude and shape errors. (b) Relative L2L_2 error over time. Faint vertical marks indicate the out-of-span correction events. The SPIN ROM uses the same rank, sample count, correction interval, and number of full-order correction steps as the baseline adaptive ROM, but maintains a lower error throughout the trajectory.

Fisher–KPP in Figure 5 provides a more demanding test. A localized initial pulse grows through the reaction term, develops two traveling fronts, and eventually saturates over much of the domain. We again use a rank-4 LSPG–QDEIM ROM with m=4m = 4 sample points, trained from only the first four full-order snapshots from t=0t = 0 to t=0.004t = 0.004, and evaluated over a 500-step prediction horizon from t=0t = 0 to t=0.50t = 0.50, with out-of-span corrections every zs=25z_s = 25 time steps. The static ROM misses the expanding front after the solution leaves the training window. The baseline ROM improves on the static model, but develops visible oscillations in the saturated region and front-location errors. In contrast, the SPIN ROM follows both the plateau and the moving fronts more closely, with a larger separation between the adaptive error curves than in Burgers.

Figure 5. SPIN ROM improves Fisher–KPP front propagation. Fisher–KPP reaction–diffusion dynamics with rank r=4r = 4 and m=4m = 4 QDEIM sample points. The offline basis is trained on the initial 4 time steps from t=0t = 0 to t=0.004t = 0.004, while prediction is evaluated over 500 time steps from t=0t = 0 to t=0.50t = 0.50, with external corrections every zs=25z_s = 25 steps. (a) Solution profiles u(x,t)u(x,t) at four reported times. The static ROM misses the expanding fronts after the solution leaves the training window. The baseline adaptive ROM improves on the static model but develops oscillations in the saturated region, while the SPIN ROM tracks the plateau and moving fronts more closely. (b) Space–time fields and error. The FOM, baseline adaptive ROM, and SPIN ROM are shown as fields over (x,t)(x,t), together with the relative L2L_2 error history and pointwise absolute error fields. The SPIN ROM produces a visibly cleaner space–time structure and substantially smaller error over most of the trajectory.

Spectral diagnostics confirm the in-span mechanism

The PDE results show that SPIN adaptation improves prediction. We next examine whether the same spectral mechanism identified in the spiral also appears in these nonlinear systems. We therefore measure three diagnostics that separate modal reweighting, in-plane basis rotation, and genuine subspace motion, computed along the online trajectory for the baseline adaptive and SPIN ROMs. These results are shown in Figure 6 for both PDEs.

Figure 6. The algebraic fingerprints of in-span learning in nonlinear ROMs. Diagnostics for (a) Burgers and (b) Fisher–KPP, comparing the baseline adaptive and SPIN ROMs. Panels (i), (ii), (iv) and (v) show the retained singular values σ1,…,σ4\sigma_1,\ldots,\sigma_4. Panel (iii) shows basis reorientation, ∥I−Φk+1TΦk∥F\lVert I-\Phi_{k+1}^{\mathsf{T}}\Phi_k\rVert_{\mathrm{F}}. Panel (vi) shows genuine subspace motion, max⁡isin⁡θi\max_i\sin\theta_i. In-span updates continuously reshape and rotate the basis inside an essentially fixed subspace, while out-of-span corrections supply the geometric motion.

The first diagnostic is the time history of the singular values carried by the adaptive basis. In a baseline adaptive ROM, these values change only at external correction events. In a SPIN ROM, they are updated at every intermediate step by the ROM trajectory itself, directly reflecting the reduced covariance update in Eq. (1). Unlike the first spiral correction, where one nearly inactive direction is strongly suppressed, the PDE spectra show richer trajectory-dependent behavior, where several retained modes are repeatedly reinforced between correction events. This is consistent with γin≈1.0\gamma_{\mathrm{in}} \approx1.0 for Burgers and Fisher–KPP, which imply little or no in-span forgetting. The diagnostic message is therefore that successful in-span learning makes the spectral state trajectory-informed.

The second diagnostic measures basis reorientation through ∥I−ΦkTΦk+1∥F\lVert I-\Phi_k^{\mathsf{T}}\Phi_{k+1}\rVert_F. This quantity is sensitive to changes in the orthonormal coordinate frame. It can be nonzero even when the two bases span essentially the same subspace. For the baseline adaptive ROM, the diagnostic is zero between correction events and spikes when an external correction updates the basis. For the SPIN ROM, it remains nonzero throughout the intervals between corrections. This confirms that the ROM predictions are not merely rescaling singular values, and they continuously rotate the basis inside the current subspace.

The third diagnostic isolates genuine subspace change. We compute the principal angles θi\theta_i between consecutive subspaces from the singular values of ΦkTΦk+1\Phi_k^{\mathsf{T}}\Phi_{k+1} and report max⁡isin⁡θi\max_i \sin\theta_i. This quantity is insensitive to a pure rotation of the coordinate frame inside the same subspace. Between external corrections, it remains close to numerical zero for the SPIN ROM, confirming that in-span updates preserve the subspace up to numerical precision. At correction events, it jumps for both adaptive models, because only then does out-of-span information enter. The jump is not identical for the two methods, showing that the same type of external event acts on different preconditioned basis states.

Discussion

Projection-based ROMs are usually described by their subspace. That description is correct, but incomplete. A reduced model also carries a basis spectrum, and in fixed-rank adaptive ROMs this spectrum shapes how future updates allocate limited modal capacity. We showed that these two objects can be adapted by different information streams. Out-of-span corrections move the subspace because they contain directions the ROM cannot currently represent, while in-span predictions reorganize the spectral state inside the existing subspace. This separation reveals a hidden degree of freedom in adaptive model reduction. We demonstrated in-span learning first on a closed-form spiral, where in-span update acts as a weak-mode suppressor, and then on Burgers and Fisher–KPP dynamics, where the same in-span channel improves hyper-reduced nonlinear ROM predictions. Across these examples, the common theme is trajectory awareness. The ROM’s own predictions are not merely outputs; they are information about how the current representation is being used.

Several insights follow from this view:

  • In-span predictions carry adaptive information. Although they cannot move the trial subspace, they can rotate and reweight the directions inside it.

  • Spectral preconditioning is memory-controlled. In a low-memory regime, it can suppress inactive directions and open modal capacity, as in the spiral. In high-memory regimes, it can instead reinforce several retained modes while still making the basis trajectory-informed.

  • In-span and out-of-span learning are complementary. External corrections supply missing directions, while in-span updates prepare the spectral state on which those corrections act.

  • The gains are obtained without additional full-order queries. The SPIN ROM uses the same external correction schedule as the baseline adaptive model, but changes how the available information is organized and used.

These results also clarify where the present framework can be improved. External information remains necessary, because in-span updates cannot create directions absent from the current reduced subspace and cannot rescue a ROM whose trajectory is already a poor guide to the local dynamics. The correction interval and forgetting factors are also decided before the simulation and remain fixed during the online prediction, whereas future methods could choose when to request out-of-span information and how strongly to forget based on online indicators such as residual growth or spectral drift. Higher-dimensional flows, multiphysics systems, and noisy or partial observations remain important research directions to investigate.

We can also draw a connection between in-span learning and in-context learning in large language models [55, 56, 57]. In the linear-attention and fast-weight view of sequence models [58, 59], this correspondence can be made mathematically explicit. Specifically, attention layers maintain a state matrix updated by rank-one outer products of internally generated representations, often discounted by a gate, and use that state to adapt their outputs without changing trained weights. The in-span update has the same basic skeleton. It accumulates a state CC from outer products akakTa_k a_k^{\mathsf{T}} of the model’s own reduced coordinates, discounts that state by γin2\gamma_{\mathrm{in}}^2, and uses it to reorganize a fixed representation. The shared structure is test-time adaptation through a decaying outer-product accumulator over self-generated representations. Adaptive ROMs may therefore provide a low-rank dynamical-systems setting in which to study how a fixed representation is reorganized by its own context. A detailed mathematical analysis linking in-span learning to in-context learning is provided in Section F.

More broadly, in-span learning extends beyond linear ROMs. It suggests that linear ROMs, nonlinear manifold models, and scientific machine-learning surrogates in general may all carry useful self-generated information in their evolving trajectories, creating an unexploited route toward more adaptive computational models.

Methods

Full-order models and time discretization

We consider three dynamical systems. The first is a closed-form spiral in R3\mathbb{R}^{3}, providing a detailed and transparent view of the mechanism. The second and third are nonlinear partial differential equations, namely viscous Burgers dynamics and Fisher–KPP reaction–diffusion, used to test the same mechanism in more practical settings.

At the fully discrete level, the full-order model is written as a nonlinear time-stepping problem for a state qn∈RNq^{n} \in\mathbb{R}^{N} at time tnt_{n},

Rn(qn;qn−1)=0,(4)R^{n}\left(q^{n};q^{n-1}\right) = 0, \tag*{(4)}

where Rn:RN→RNR^{n} : \mathbb{R}^{N} \to\mathbb{R}^{N} denotes the residual of the chosen time discretization. For the PDE benchmarks, we use implicit backward Euler,

Rn(qn;qn−1):=qn−qn−1−Δt f(qn),(5)R^{n}\left(q^{n};q^{n-1}\right) := q^{n} - q^{n-1} - \Delta t\, f\left(q^{n}\right), \tag*{(5)}

where ff is the semi-discrete spatial operator. The nonlinear system (5) is solved by Newton iterations in the full-order model (FOM) and by a sampled least-squares problem in the ROM.

The spiral test case is defined by the exact trajectory

q(t):=[cos⁡tsin⁡tαt],α=0.4.(6)q(t) := \begin{bmatrix} \cos t \\ \sin t \\ \alpha t \end{bmatrix}, \qquad\alpha= 0.4. \tag*{(6)}

This trajectory solves the affine linear system

q˙=Aq+b,A:=[0−10100000],b:=[00α].(7)\dot{q} = A q + b, \qquad A := \begin{bmatrix} 0 & -1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix}, \qquad b := \begin{bmatrix} 0 \\ 0 \\ \alpha\end{bmatrix}. \tag*{(7)}

For time step Δt=0.35\Delta t = 0.35, the exact discrete map is

qk+1=Gqk+c,G:=exp⁡(AΔt),c:=[00αΔt].(8)q^{k+1} = G q^{k} + c, \qquad G := \exp(A\Delta t), \qquad c := \begin{bmatrix} 0 \\ 0 \\ \alpha\Delta t \end{bmatrix}. \tag*{(8)}

The simulation begins at t0=4.0t_{0} = 4.0. Because Eq. (8) is exact, the spiral experiment contains no time-integration error. All discrepancies between the FOM and ROM are therefore caused by the reduced representation and its updates.

The Burgers benchmark follows the one-dimensional viscous Burgers equation

∂tu+u ∂xu=ν ∂xxu,x∈[0,1],(9)\partial_{t}u + u\,\partial_{x}u = \nu\,\partial_{xx}u, \qquad x \in[0,1], \tag*{(9)}

with periodic boundary conditions and viscosity ν=10−2\nu= 10^{-2}. The spatial domain is discretized with Nx=1000N_x = 1000 uniform grid points. The convective term is approximated by a first-order upwind finite-difference scheme and the diffusive term by a centred second-order method. Time integration uses backward Euler with Δt=10−3\Delta t = 10^{-3} over a horizon of 500 time steps to propagate a Gaussian initial condition.

The Fisher–KPP benchmark is

∂tu=D ∂xxu+βu(1−u),x∈[0,1],(10)\partial_t u = D\,\partial_{xx}u + \beta u(1-u), \qquad x \in[0,1], \tag*{(10)}

with periodic boundary conditions, diffusion coefficient D=10−4D = 10^{-4}, and reaction rate β=20\beta= 20. The spatial domain is discretized with Nx=256N_x = 256 uniform grid points using centred finite differences. Time integration again uses backward Euler with Δt=10−3\Delta t = 10^{-3} over 500 time steps. Starting from a localized pulse, the solution develops travelling fronts and then saturates toward u≈1u \approx1 over much of the domain.

Reduced-order representation

All ROMs use an affine, scaled trial representation of the full-order state,

qn≈qref+D−1Φan,(11)q^n \approx q_{\mathrm{ref}} + D^{-1}\Phi a^n, \tag*{(11)}

where qref∈RNq_{\mathrm{ref}} \in\mathbb{R}^{N} is a reference state, D∈RN×ND \in\mathbb{R}^{N\times N} is a nonsingular diagonal scaling matrix, Φ∈RN×r\Phi\in\mathbb{R}^{N\times r} is an orthonormal basis, and an∈Rra^n \in\mathbb{R}^{r} is the reduced coordinate vector. The rank satisfies r≪Nr \ll N.

The initial basis is constructed from a short training snapshot matrix

Xtrain:=[q1−qrefq2−qref⋯qNs−qref],(12)X_{\mathrm{train}} := \left[q^1-q_{\mathrm{ref}}\quad q^2-q_{\mathrm{ref}}\quad\cdots\quad q^{N_s}-q_{\mathrm{ref}}\right], \tag*{(12)}

where NsN_s is the number of training snapshots. After scaling, we compute

DXtrain=UΣVT,(13)DX_{\mathrm{train}} = U\Sigma V^{\mathsf{T}}, \tag*{(13)}

and set Φ\Phi equal to the first rr left singular vectors. In all PDE experiments reported in the main text, the offline window contains only four FOM snapshots and the ROM rank is r=4r = 4. The static ROM therefore represents a deliberately minimal offline model, while the adaptive ROMs are tested in a genuinely extrapolative setting.

For the spiral, the dynamics are linear and the discrete flow is known exactly, so we use a simple Galerkin projection. The initial rank-2 basis Φ0\Phi_0 is obtained from the two snapshots q1q^1 and q2q^2. The reduced state satisfies

an+1=Aran+cr,Ar:=Φ0TGΦ0,cr:=Φ0Tc,(14)a^{n+1} = A_r a^n + c_r,\qquad A_r := \Phi_0^{\mathsf{T}}G\Phi_0,\qquad c_r := \Phi_0^{\mathsf{T}}c, \tag*{(14)}

with initial coordinate a2=Φ0TD(q2−qref)a^2 = \Phi_0^{\mathsf{T}}D(q^2-q_{\mathrm{ref}}). The corresponding ROM state is

qROMn:=qref+D−1Φ0an.(15)q_{\mathrm{ROM}}^n := q_{\mathrm{ref}} + D^{-1}\Phi_0a^n. \tag*{(15)}

Note that in the spiral results, we take a zero reference state qrefq_{\mathrm{ref}} and an identity scaling matrix DD for simplicity. Since the basis is rank two, the ROM trajectory remains in the initial plane

S0:=span⁡(Φ0).(16)\mathcal{S}_0 := \operatorname{span}(\Phi_0). \tag*{(16)}

For Burgers and Fisher–KPP, since the full-order models utilize implicit temporal integration, we use a least-squares Petrov–Galerkin (LSPG) [54, 19] formulation at the fully discrete level.

Substituting the reduced ansatz (11) into the FOM residual gives the reduced least-squares problem

an=arg min⁡a~∈Rr∥DRn(qref+D−1Φa~;qROMn−1)∥22.(17)a^n = \underset{\tilde{a}\in\mathbb{R}^r}{\operatorname{arg\,min}}\left\|D R^n\left(q_{\mathrm{ref}} + D^{-1}\Phi\tilde{a};q_{\mathrm{ROM}}^{n-1}\right)\right\|_2^2. \tag*{(17)}

Here qROMn−1:=qref+D−1Φan−1q_{\mathrm{ROM}}^{n-1} := q_{\mathrm{ref}} + D^{-1}\Phi a^{n-1} is the previous ROM state. This LSPG formulation minimizes the fully discrete residual and is well-suited to implicit time stepping.

To reduce the online cost, we use QDEIM hyper-reduction [52, 53]. Let S∈RN×mS \in\mathbb{R}^{N\times m} be a sampling matrix that selects mm residual entries, with m≪Nm \ll N. The sampled LSPG problem is

an=arg min⁡a~∈Rr∥STDRn(qref+D−1Φa~;qROMn−1)∥22.(18)a^n = \underset{\tilde{a}\in\mathbb{R}^r}{\operatorname{arg\,min}}\left\|S^{\mathsf{T}}D R^n\left(q_{\mathrm{ref}} + D^{-1}\Phi\tilde{a};q_{\mathrm{ROM}}^{n-1}\right)\right\|_2^2. \tag*{(18)}

The sample indices are chosen by QDEIM through a column-pivoted QR factorization of ΦT\Phi^{\mathsf{T}}. In the PDE experiments, we use m=4m = 4 sample points.

Information channels in an adaptive ROM

The adaptive ROM receives two kinds of snapshots, distinguished by their relationship to the current reduced subspace.

Definition 1 (Out-of-span and in-span snapshots). Let Φ∈RN×r\Phi\in\mathbb{R}^{N\times r} be the current orthonormal basis, and let y∈RNy \in\mathbb{R}^N be a preprocessed snapshot passed to a basis update. Define the projection coefficient, residual, and residual norm by

w:=ΦTy,ry:=y−Φw,ρ:=∥ry∥2.(19)w := \Phi^{\mathsf{T}}y,\qquad r_y := y-\Phi w,\qquad\rho:= \|r_y\|_2. \tag*{(19)}

The snapshot is in-span if ρ=0\rho= 0, equivalently y∈span⁡(Φ)y \in\operatorname{span}(\Phi). It is out-of-span if ρ>0\rho> 0.

In the algorithms studied here, out-of-span snapshots are generated by occasional full-order operator queries. These snapshots may contain components not represented by the current basis and can therefore move the trial subspace. In-span snapshots are the ROM’s own predicted states. They satisfy y=Φay = \Phi a by construction and therefore cannot introduce a new direction. The next Methods subsections show that, despite this zero residual, in-span snapshots still update the spectral state of the basis through the singular values and in-plane rotations of an iSVD update.

iSVD update with forgetting

The online basis update used throughout the paper is an iSVD with forgetting. The update maintains a rank-rr approximation of an exponentially weighted snapshot history.

Definition 2 (Truncated iSVD update with forgetting [43, 44, 45]). Let Φ∈RN×r\Phi\in\mathbb{R}^{N\times r} be an orthonormal basis, let Σ=diag⁡(σ1,…,σr)\Sigma= \operatorname{diag}(\sigma_1,\ldots,\sigma_r) contain the retained singular values associated with Φ\Phi, and let y∈RNy \in\mathbb{R}^N be a preprocessed incoming snapshot. Define the projection coefficient, residual, and residual norm as in Eq. (19). If ρ>0\rho> 0, set r^:=ry/ρ\hat{r} := r_y/\rho. For a forgetting factor γ∈(0,1]\gamma\in(0,1], form the core matrix

K:=[γΣw0Tρ]∈R(r+1)×(r+1).(20)K := \begin{bmatrix}\gamma\Sigma& w\\ 0^{\mathsf{T}} & \rho\end{bmatrix} \in\mathbb{R}^{(r+1)\times(r+1)}. \tag*{(20)}

Let

K=UKΣKVKT(21)K = U_K\Sigma_K V_K^{\mathsf{T}} \tag*{(21)}

be its singular-value decomposition, with singular values in non-increasing order, obtained from the eigendecomposition of the full core covariance

KKT=[γ2Σ2+wwTρwρwTρ2]=UKΣK2UKT.(22)KK^{\mathsf{T}} = \begin{bmatrix} \gamma^2\Sigma^2 + ww^{\mathsf{T}} & \rho w\\ \rho w^{\mathsf{T}} & \rho^2 \end{bmatrix} = U_K\Sigma_K^2U_K^{\mathsf{T}}. \tag*{(22)}

The rank-rr update is

Φ+=[Φ r~]UK(:,1:r),Σ+=ΣK(1:r,1:r).(23)\Phi^{+} = \left[\Phi\ \tilde{r}\right]U_K(:,1:r), \qquad\Sigma^{+} = \Sigma_K(1:r,1:r). \tag*{(23)}

If ρ=0\rho=0, the residual direction is not defined and the update is interpreted through the reduced core described later in Statement 3.

The update in Statement 2 is equivalent to appending the new snapshot to a weighted history matrix. Let YkY_k denote the exponentially weighted data matrix containing the snapshots assimilated up to update step kk,

Yk:=[γk−1y1  γk−2y2  ⋯  γyk−1  yk].Y_k := \left[\gamma^{k-1}y_1\ \ \gamma^{k-2}y_2\ \ \cdots\ \ \gamma y_{k-1}\ \ y_k\right].

The current rank-rr representation approximates this weighted data matrix by Yk≈ΦΣVTY_k \approx\Phi\Sigma V^{\mathsf{T}}. Receiving a new snapshot yy gives the augmented weighted history

Yk+1=[γYk  y].Y_{k+1} = \left[\gamma Y_k\ \ y\right].

The small matrix KK in Eq. (20) contains all information needed to update the left singular vectors of this augmented matrix. The forgetting factor γ\gamma discounts previous information before the new snapshot is added, and acts as a memory valve in this sense.

Equivalently, the update tracks the dominant eigenspace of an exponentially weighted second-moment matrix. If Mk:=YkYkTM_k := Y_kY_k^{\mathsf{T}} denotes the weighted second-moment matrix in the ambient coordinates, then the weighted-history update gives

Mk+1=γ2Mk+yyT.(24)M_{k+1} = \gamma^2 M_k + yy^{\mathsf{T}}. \tag*{(24)}

In-span learning preserves the subspace and reorganizes the spectrum

We now specialize Statement 2 to the case in which the incoming snapshot is a ROM prediction. In scaled coordinates, such a snapshot has the form

y=Φa,a∈Rr.(25)y = \Phi a, \qquad a \in\mathbb{R}^{r}. \tag*{(25)}

Hence, according to Eq. (19), w=aw=a, ry=0r_y=0, and ρ=0\rho=0. No new ambient direction can be added. Nevertheless, the singular values and the coordinate frame inside the subspace can change.

Theorem 3 (Subspace-preserving spectral preconditioning by in-span learning). Let Φ∈RN×r\Phi\in\mathbb{R}^{N\times r} be orthonormal, let Σ=diag⁡(σ1,…,σr)\Sigma= \operatorname{diag}(\sigma_1,\ldots,\sigma_r), and let y=Φay=\Phi a be an in-span snapshot. Applying the iSVD update with forgetting factor γin∈(0,1]\gamma_{\mathrm{in}}\in(0,1] gives

Φ+=ΦU,Σ+=Λ1/2,(26)\Phi^{+} = \Phi U, \qquad\Sigma^{+} = \Lambda^{1/2}, \tag*{(26)}

where U∈Rr×rU\in\mathbb{R}^{r\times r} is orthogonal and

UΛUT=γin2Σ2+aaT.(27)U\Lambda U^{\mathsf{T}} = \gamma_{\mathrm{in}}^2\Sigma^2 + aa^{\mathsf{T}}. \tag*{(27)}

Consequently,

span⁡(Φ+)=span⁡(Φ).(28)\operatorname{span}(\Phi^{+}) = \operatorname{span}(\Phi). \tag*{(28)}

The singular values obey the rank-one interlacing inequalities

σ1+≥γinσ1≥σ2+≥⋯≥γinσr−1≥σr+≥γinσr.(29)\sigma_1^{+} \ge\gamma_{\mathrm{in}}\sigma_1 \ge\sigma_2^{+} \ge\cdots\ge\gamma_{\mathrm{in}}\sigma_{r-1} \ge\sigma_r^{+} \ge\gamma_{\mathrm{in}}\sigma_r. \tag*{(29)}

Proof. For an in-span snapshot, y=Φay = \Phi a. Therefore w=ΦTy=aw = \Phi^{\mathsf{T}}y = a and ρ=0\rho= 0. The core matrix in Eq. (20) has zero last row,

K=[γinΣa0T0].(30)K = \begin{bmatrix} \gamma_{\mathrm{in}}\Sigma& a \\ 0^{\mathsf{T}} & 0 \end{bmatrix}. \tag*{(30)}

Its nonzero singular values are the singular values of the reduced matrix

Kin=[γinΣa]∈Rr×(r+1).(31)K_{\mathrm{in}} = \begin{bmatrix} \gamma_{\mathrm{in}}\Sigma& a \end{bmatrix} \in\mathbb{R}^{r \times(r+1)}. \tag*{(31)}

Since

KinKinT=γin2Σ2+aaT,(32)K_{\mathrm{in}}K_{\mathrm{in}}^{\mathsf{T}} = \gamma_{\mathrm{in}}^2\Sigma^2 + aa^{\mathsf{T}}, \tag*{(32)}

the left singular vectors of KinK_{\mathrm{in}} are the eigenvectors of the reduced covariance in Eq. (27). The updated basis is therefore Φ+=ΦU\Phi^+ = \Phi U for an orthogonal matrix UU, which proves subspace preservation. The interlacing inequalities follow from Cauchy’s interlacing theorem for a positive semidefinite rank-one perturbation of the diagonal matrix γin2Σ2\gamma_{\mathrm{in}}^2\Sigma^2.

After ss consecutive in-span updates with reduced coordinates an1,…,ansa^{n_1},\ldots,a^{n_s}, the reduced covariance is

Cs=γin2sΣ02+∑j=1sγin2(s−j)anj(anj)T.(33)C_s = \gamma_{\mathrm{in}}^{2s}\Sigma_0^2 + \sum_{j=1}^{s}\gamma_{\mathrm{in}}^{2(s-j)}a^{n_j}(a^{n_j})^{\mathsf{T}}. \tag*{(33)}

This is the covariance recursion used in the spiral example. It shows explicitly how the ROM trajectory reshapes the spectrum. Directions aligned with recent coefficient vectors can receive repeated rank-one reinforcement, while directions receiving little trajectory energy can be suppressed by forgetting. Here, we say that a direction is reinforced if its post-update directional energy exceeds its pre-update value, and suppressed if its post-update directional energy is smaller.

Definition 4 (Directional reinforcement and suppression). Let ξ∈Rr\xi\in\mathbb{R}^{r} be a unit direction in the current, pre-update reduced coordinates. For an in-span update with reduced coordinate aa, define the pre-update and post-update directional energies by

Eξ:=ξTΣ2ξ,Eξ+:=ξT(γin2Σ2+aaT)ξ.(34)E_{\xi} := \xi^{\mathsf{T}}\Sigma^2\xi,\qquad E_{\xi}^{+} := \xi^{\mathsf{T}}\left(\gamma_{\mathrm{in}}^2\Sigma^2 + aa^{\mathsf{T}}\right)\xi. \tag*{(34)}

We say that the fixed pre-update direction ξ\xi is reinforced if Eξ+>EξE_{\xi}^{+} > E_{\xi} and suppressed if Eξ+<EξE_{\xi}^{+} < E_{\xi}.

Using Eq. (34),

Eξ+−Eξ=(ξTa)2−(1−γin2)Eξ.(35)E_{\xi}^{+} - E_{\xi} = (\xi^{\mathsf{T}}a)^2 - (1-\gamma_{\mathrm{in}}^2)E_{\xi}. \tag*{(35)}

Therefore the fixed pre-update direction ξ\xi is reinforced if

∣ξTa∣>1−γin2Eξ,(36)\lvert\xi^{\mathsf{T}}a\rvert> \sqrt{1-\gamma_{\mathrm{in}}^2}\sqrt{E_{\xi}}, \tag*{(36)}

and suppressed if the inequality is reversed. In particular, for a pre-update singular-vector direction eie_i,

Eei+−Eei=ai2−(1−γin2)σi2,(37)E_{e_i}^{+} - E_{e_i} = a_i^2 - (1-\gamma_{\mathrm{in}}^2)\sigma_i^2, \tag*{(37)}

so that this pre-update direction is reinforced when

∣ai∣>1−γin2 σi,(38)\lvert a_i\rvert> \sqrt{1-\gamma_{\mathrm{in}}^2}\,\sigma_i, \tag*{(38)}

and suppressed when ∣ai∣\lvert a_i\rvert falls below this threshold.

This analysis also shows that in-span spectral preconditioning does not always appear as suppression of the weakest mode. If the ROM trajectory has enough projected energy in a direction to exceed the threshold, that direction is reinforced even under forgetting. If the projected energy falls below the threshold, the direction decays. The spiral example with aggressive forgetting realizes the latter case for one nearly inactive direction, whereas the PDE examples show richer regimes with less forgetting in which several retained directions are reinforced while the basis still rotates and reweights.

Full-order operator query

The out-of-span channel requires occasional information that is not constrained to the current reduced subspace. In this work, that information is supplied by a full-order operator query. The idea is to let the ROM advance the system most of the time, and to call the full-order model only at selected correction events.

Suppose an out-of-span event is triggered at time step n+1n+1. Before the correction, the ROM has produced the state

qROMn:=qref+D−1Φan.(39)q_{\mathrm{ROM}}^{n} := q_{\mathrm{ref}} + D^{-1}\Phi a^{n}. \tag*{(39)}

We then initialize one full-order step from this ROM state and solve

Rn+1(qcorrn+1;qROMn)=0.(40)R^{n+1}\left(q_{\mathrm{corr}}^{n+1};q_{\mathrm{ROM}}^{n}\right)=0. \tag*{(40)}

The resulting state qcorrn+1q_{\mathrm{corr}}^{n+1} is used as the external correction snapshot. In scaled coordinates,

ycorr:=D(qcorrn+1−qref).(41)y_{\mathrm{corr}} := D\left(q_{\mathrm{corr}}^{n+1}-q_{\mathrm{ref}}\right). \tag*{(41)}

This snapshot is then passed to the out-of-span iSVD update with forgetting factor γout\gamma_{\mathrm{out}}.

This correction is different from a coarse full-order lookahead trajectory previously used in [51]. A coarse-FOM oracle advances an auxiliary full-order state on a larger time step and uses that state as the correction signal. The full-order operator query used here instead takes a single fine-grid full-order step from the current ROM state. We use it because it gives a simple hybrid model, where the reduced model carries the trajectory most of the time and occasionally interacts with the full-order operator to receive a local out-of-span correction snapshot.

For the spiral example, Eq. (40) reduces to one exact full-order map applied to the ROM state,

qcorr:=GqROMn+c.(42)q_{\mathrm{corr}} := Gq_{\mathrm{ROM}}^{n}+c. \tag*{(42)}

For the PDE benchmarks, Eq. (40) is solved with the same backward-Euler residual used by the FOM, but only at the correction times.

SPIN algorithm

The complete adaptive procedure (Algorithm 1) alternates between three operations. First, the ROM advances one time step using the current basis and sample set. Second, if the step is not an external correction time, the ROM-predicted state is used for an in-span update with forgetting factor γin\gamma_{\mathrm{in}}. Third, every zsz_s steps, a full-order operator query is performed, and the produced snapshot is used for an out-of-span update with forgetting factor γout\gamma_{\mathrm{out}}.

After any basis update, basis-dependent quantities must be refreshed. For the LSPG–QDEIM ROMs, this includes the sampled basis and its pseudoinverse. After an out-of-span update, the QDEIM sample indices are recomputed. After an in-span update, the subspace is unchanged but the basis vectors have rotated inside it, so the sampled operators depending on the basis are refreshed. The reduced coordinates are also transferred to the updated basis by projecting the current reconstructed ROM state onto the new basis.

Algorithm 1 SPIN ROM algorithm

Require: Initial basis Φ\Phi, singular values σ\sigma, reference state qrefq_{\mathrm{ref}}, scaling matrix DD, sample matrix SS, reduced state a0a^0, update interval zsz_s, forgetting factors γin\gamma_{\mathrm{in}} and γout\gamma_{\mathrm{out}}, final step NtN_t

  1. for n=0,1,…,Nt−1n = 0, 1, \ldots, N_t - 1 do

  2. Advance the sampled ROM one step:

an+1←ROM_Step⁡(Φ,S,an)a^{n+1} \leftarrow\operatorname{ROM\_Step}(\Phi, S, a^n)
  1. Reconstruct the ROM state:

qROMn+1←qref+D−1Φan+1q_{\mathrm{ROM}}^{n+1} \leftarrow q_{\mathrm{ref}} + D^{-1}\Phi a^{n+1}
  1. if (n+1) mod zs=0(n + 1) \bmod z_s = 0 then — out-of-span adaptation

  2. Perform one full-order operator query:

Rn+1(qcorrn+1,qROMn)=0R^{n+1}\left(q_{\mathrm{corr}}^{n+1},q_{\mathrm{ROM}}^{n}\right) = 0
  1. Preprocess the correction:

y←D(qcorrn+1−qref)y \leftarrow D\left(q_{\mathrm{corr}}^{n+1} - q_{\mathrm{ref}}\right)
  1. Update the basis and singular values by out-of-span iSVD:

(Φ,σ)←iSVD⁡(Φ,σ,y,γout)(\Phi,\sigma) \leftarrow\operatorname{iSVD}(\Phi,\sigma,y,\gamma_{\mathrm{out}})
  1. Recompute QDEIM sample matrix SS and sampled operators.

  2. else — in-span adaptation

  3. Preprocess the ROM prediction:

y←D(qROMn+1−qref)y \leftarrow D\left(q_{\mathrm{ROM}}^{n+1} - q_{\mathrm{ref}}\right)
  1. Update the basis and singular values by in-span iSVD:

(Φ,σ)←iSVD⁡(Φ,σ,y,γin)(\Phi,\sigma) \leftarrow\operatorname{iSVD}(\Phi,\sigma,y,\gamma_{\mathrm{in}})
  1. Refresh sampled operators.

  2. end if

  3. Transfer the current state to the updated basis:

an+1←ΦTD(qROMn+1−qref)a^{n+1} \leftarrow\Phi^{\mathsf{T}}D\left(q_{\mathrm{ROM}}^{n+1} - q_{\mathrm{ref}}\right)
  1. end for

The baseline adaptive ROM used for comparison can be obtained from Algorithm 1 by omitting the in-span update in the else branch. The static ROM is obtained by omitting both in-span and out-of-span updates. The implementation used in this work is available at https://github.com/APHedayat/SPIN.

Metrics and diagnostics

We use two groups of diagnostics. The first group measures how a single out-of-span correction is absorbed by a fixed-rank update. Let Φpre\Phi_{\mathrm{pre}} and Φpost\Phi_{\mathrm{post}} denote the orthonormal bases immediately before and after the correction update, and let qcorrq_{\mathrm{corr}} be the correction snapshot. The out-of-span residual relative to the pre-update subspace is

r:=qcorr−ΦpreΦpreTqcorr,r^:=r/∥r∥2.(43)r := q_{\mathrm{corr}} - \Phi_{\mathrm{pre}}\Phi_{\mathrm{pre}}^{\mathsf{T}}q_{\mathrm{corr}}, \qquad\hat{r} := r/\lVert r\rVert_2. \tag*{(43)}

The residual capture is

χres:=∥ΦpostTr^∥2(44)\chi_{\mathrm{res}} := \lVert\Phi_{\mathrm{post}}^{\mathsf{T}}\hat{r}\rVert_2 \tag*{(44)}

which measures how much of the new residual direction lies in the post-update subspace. The plane-change angle is the largest principal angle between the pre- and post-update subspaces,

θplane:=max⁡iarccos⁡(ζi(ΦpreTΦpost)),(45)\theta_{\mathrm{plane}} := \max_i \arccos\left(\zeta_i\left(\Phi_{\mathrm{pre}}^{\mathsf{T}}\Phi_{\mathrm{post}}\right)\right), \tag*{(45)}

where ζi(⋅)\zeta_i(\cdot) are singular values. The correction error is

ecorr:=∥qcorr−ΦpostΦpostTqcorr∥2.(46)e_{\mathrm{corr}} := \left\lVert q_{\mathrm{corr}} - \Phi_{\mathrm{post}}\Phi_{\mathrm{post}}^{\mathsf{T}}q_{\mathrm{corr}}\right\rVert_2. \tag*{(46)}

This measures the part of the correction snapshot not represented by the updated subspace. The second group of diagnostics measures how the adaptive basis, spectrum, and subspace evolve over the online trajectory. The first trajectory diagnostic is the singular-value history,

σ1n,…,σrn.(47)\sigma_1^n,\ldots,\sigma_r^n. \tag*{(47)}

These values measure how the adaptive algorithm weights the retained modes. In the baseline adaptive ROM, they change only at correction events. In the SPIN ROM, they may change at every time step. The second trajectory diagnostic measures basis reorientation between consecutive steps,

dbasisn:=∥I−(Φn+1)TΦn∥F.(48)d_{\mathrm{basis}}^n := \left\lVert I - \left(\Phi^{n+1}\right)^{\mathsf{T}}\Phi^n\right\rVert_F. \tag*{(48)}

This quantity is sensitive to changes in the orthonormal frame. It can be nonzero even when the two bases span the same subspace. The third trajectory diagnostic measures genuine subspace change. Let the singular values of

(Φn)TΦn+1(49)\left(\Phi^n\right)^{\mathsf{T}}\Phi^{n+1} \tag*{(49)}

be s1,…,srs_1,\ldots,s_r, with si=cos⁡θis_i = \cos\theta_i, where θi\theta_i are the principal angles between the two subspaces. We report

dsubspacen:=max⁡isin⁡θi=1−min⁡isi2.(50)d_{\mathrm{subspace}}^n := \max_i \sin\theta_i = \sqrt{1-\min_i s_i^2}. \tag*{(50)}

This diagnostic is zero when the subspace is unchanged, even if the basis has rotated inside that subspace. The combination of nonzero dbasisnd_{\mathrm{basis}}^n and zero dsubspacend_{\mathrm{subspace}}^n is therefore the numerical signature of a pure in-span update.

Computational complexity

The additional cost of in-span learning is small compared with the cost of obtaining out-of-span information from the full-order operator. Let NN be the full-order dimension, rr the ROM rank, and mm the number of sampled residual entries, with rr, m≪Nm \ll N.

We first define the cost of one hyper-reduced LSPG time step. A sampled LSPG step solves a nonlinear least-squares problem in rr unknowns using only mm sampled residual entries. Let Ceval(m)\mathcal{C}_{\mathrm{eval}}^{(m)} denote the cost of evaluating the sampled residual and sampled Jacobian contributions required

Appendices

by one Newton iteration. Once these sampled quantities are available, the dense reduced linear algebra scales as

CLSPG:=Ceval(m)+O(mr2+r3)(51)C_{\mathrm{LSPG}} := C_{\mathrm{eval}}^{(m)} + O(mr^2 + r^3) \tag*{(51)}

per Newton iteration. Thus, between correction events, the online simulation remains reduced and sampled.

The cost of an iSVD basis update is roughly similar for in-span and out-of-span snapshots. For a general out-of-span snapshot, the update projects the snapshot onto the current basis, forms the orthogonal residual, computes the SVD of a small (r+1)×(r+1)(r + 1) \times(r + 1) core matrix, and rotates the ambient basis. The dominant scaling is

CiSVDout:=O(Nr2+r3).(52)C_{\mathrm{iSVD}}^{\mathrm{out}} := O(Nr^2 + r^3). \tag*{(52)}

For an in-span snapshot, the orthogonal residual is zero and the update can be performed through the reduced covariance in Eq. (27). This avoids the residual construction, but the small eigendecomposition and the ambient basis rotation remain. Therefore, the in-span update costs as

CiSVDin:=O(Nr2+r3).(53)C_{\mathrm{iSVD}}^{\mathrm{in}} := O(Nr^2 + r^3). \tag*{(53)}

The in-span update can be cheaper in implementations that delay the ambient rotation and accumulate the transformation in reduced coordinates, but in the implementation considered for PDE cases the leading order is the same as for an out-of-span iSVD update.

Basis changes may also require refreshing hyper-reduction quantities. We denote this cost by CrefreshC_{\mathrm{refresh}}. If QDEIM sampling points are recomputed from a pivoted QR factorization of Φ⊤\Phi^\top, then

CQDEIM:=O(Nr2).(54)C_{\mathrm{QDEIM}} := O(Nr^2). \tag*{(54)}

In-span updates preserve the trial subspace, so one may keep the same sampling set between out-of-span corrections and update only the sampled basis-dependent quantities. If the sampling points are recomputed after every basis rotation, then the same CQDEIMC_{\mathrm{QDEIM}} cost applies.

The bottleneck in the adaptive procedure is the full-order operator query used to obtain an out-of-span correction. For implicit PDE solvers, this query requires nonlinear iterations, residual and Jacobian evaluations, and large linear solves. We write its cost as

CFOM:=O(Nβ),β>1,(55)C_{\mathrm{FOM}} := O(N^\beta), \qquad\beta> 1, \tag*{(55)}

where β\beta depends on the discretization, nonlinear solver, linear solver, preconditioner, and sparsity structure. This term is typically much larger than the reduced dense linear algebra when NN is large and r,m≪Nr,m \ll N.

With these ingredients, the cost of one in-span update is

Cin:=CiSVDin+Crefreshin,(56)C_{\mathrm{in}} := C_{\mathrm{iSVD}}^{\mathrm{in}} + C_{\mathrm{refresh}}^{\mathrm{in}}, \tag*{(56)}

where CrefreshinC_{\mathrm{refresh}}^{\mathrm{in}} denotes any update of sampled basis-dependent quantities performed after the in-span rotation. The cost of one out-of-span correction event is

Cout:=CFOM+CiSVDout+CQDEIM+Crefreshout,(57)C_{\mathrm{out}} := C_{\mathrm{FOM}} + C_{\mathrm{iSVD}}^{\mathrm{out}} + C_{\mathrm{QDEIM}} + C_{\mathrm{refresh}}^{\mathrm{out}}, \tag*{(57)}

where CrefreshoutC_{\mathrm{refresh}}^{\mathrm{out}} denotes the remaining basis-dependent reduced-operator or sampled-operator updates.

The key comparison is that the SPIN ROM uses the same out-of-span correction schedule as the baseline adaptive ROM. It therefore does not add any CFOMC_{\mathrm{FOM}} terms. If the correction interval is zsz_s, then between two out-of-span corrections the SPIN model adds

(zs−1)Cin(58)(z_s - 1)C_{\mathrm{in}} \tag*{(58)}

relative to the baseline adaptive model, but it does not change the number of full-order operator queries. In the large-scale implicit PDE regime, CFOM=O(Nβ)\mathcal{C}_{\mathrm{FOM}} = O(N^{\beta}) is the dominant cost, whereas the in-span operations are reduced or low-rank dense linear algebra.

Data availability

The data supporting the findings of this work are available from the corresponding author upon request.

Code availability

The code used to generate the numerical results in this work is available at https://github.com/APHedayat/SPIN.

Acknowledgements

A.H. and K.D. were supported by the Office of Under Secretary of Defense for Research and Engineering (OUSD(RE)) grant N00014-21-1-295. L.B. acknowledges NSF award CCF-2331590 and ARO award W911NF-26-1-A219.

Author contributions

A.H.: Methodology, Software, Investigation, Data Curation, Writing - Original Draft, Visualization. L.B.: Conceptualization, Methodology, Writing - Review & Editing, Supervision. K.D.: Conceptualization, Methodology, Investigation, Writing - Review & Editing, Supervision, Funding acquisition.

Competing interests

The authors declare no competing interests.

Appendices

The following appendices contain the Supplementary Information accompanying the manuscript. They provide notation, full numerical details for the spiral example, comparisons with alternative fixed-rank updates, hyperparameter sensitivity studies for the nonlinear PDE benchmarks, reproducibility information, and a mathematical connection between in-span learning and in-context learning supporting the main text.

Notation

Table A1 summarizes the main symbols used in the paper.

SymbolMeaning
NNFull-order dimension
rrROM rank
mmNumber of hyper-reduction sample points
NtN_tNumber of online prediction time steps
qnq^nFull-order state at time step nn
qROMnq_{\mathrm{ROM}}^nROM-reconstructed state at time step nn
qcorrq_{\mathrm{corr}}Correction snapshot obtained from a full-order operator query
qrefq_{\mathrm{ref}}Reference state used in the affine ROM representation
DDDiagonal scaling matrix
Φ∈RN×r\Phi\in\mathbb{R}^{N \times r}Orthonormal reduced basis
Φ0\Phi_0Initial reduced basis
Φpre,Φpost\Phi_{\mathrm{pre}}, \Phi_{\mathrm{post}}Reduced bases immediately before and after a correction update
Σ=diag⁡(σ1,…,σr)\Sigma= \operatorname{diag}(\sigma_1,\ldots,\sigma_r)Retained singular values associated with Φ\Phi
an∈Rra^n \in\mathbb{R}^rReduced coordinate vector at time step nn
SSHyper-reduction sampling matrix
zsz_sNumber of ROM time steps between out-of-span corrections
γin\gamma_{\mathrm{in}}Forgetting factor for in-span updates
γout\gamma_{\mathrm{out}}Forgetting factor for out-of-span updates
yyPreprocessed snapshot passed to iSVD
w=ΦTyw = \Phi^{\mathsf{T}} yIn-plane coefficients of yy
ry=y−Φwr_y = y - \Phi wOrthogonal residual of yy with respect to span⁡(Φ)\operatorname{span}(\Phi)
ρ=∥ry∥2\rho= \lVert r_y \rVert_2Residual norm
KKSmall iSVD core matrix
YkY_kExponentially weighted snapshot-history matrix after kk updates
Mk=YkYkTM_k = Y_k Y_k^{\mathsf{T}}Exponentially weighted second-moment matrix in ambient coordinates
CCReduced covariance represented in reduced coordinates
CsC_sReduced covariance after ss consecutive in-span updates
CcorrC_{\mathrm{corr}}In-plane block of the iSVD core covariance used to diagnose local rank competition
η=ρ2/λr(Ccorr)\eta= \rho^2/\lambda_r(C_{\mathrm{corr}})Residual-to-weakest-mode ratio
χres\chi_{\mathrm{res}}Residual capture by the post-update basis
θplane\theta_{\mathrm{plane}}Largest principal angle between the pre- and post-update subspaces
ecorre_{\mathrm{corr}}Correction error after projection onto the post-update subspace
dbasisnd_{\mathrm{basis}}^nBasis-reorientation diagnostic between steps nn and n+1n+1
dsubspacend_{\mathrm{subspace}}^nGenuine subspace-motion diagnostic between steps nn and n+1n+1
θi\theta_iPrincipal angles between two subspaces
ζi(⋅)\zeta_i(\cdot)Singular values of a matrix argument, used in principal-angle diagnostics
CFOM\mathcal{C}_{\mathrm{FOM}}Cost of one full-order operator query
Cin,Cout\mathcal{C}_{\mathrm{in}}, \mathcal{C}_{\mathrm{out}}Costs of one in-span update and one out-of-span correction event

Table A1. Notation used in the main text and appendices.

Full numerical details for the spiral example

The spiral trajectory is

q(t)=[cos⁡tsin⁡tαt],α=0.4.(59)q(t) = \begin{bmatrix} \cos t \\ \sin t \\ \alpha t \end{bmatrix}, \qquad\alpha= 0.4. \tag*{(59)}

The time step is Δt=0.35\Delta t = 0.35 and the initial time is t0=4.0t_0 = 4.0. The exact discrete dynamics are

qk+1=Gqk+c,(60)q^{k+1} = Gq^k + c, \tag*{(60)}

with

G=[0.93937271−0.3428978100.342897810.939372710001],c=[000.14].(61)G = \begin{bmatrix} 0.93937271 & -0.34289781 & 0 \\ 0.34289781 & 0.93937271 & 0 \\ 0 & 0 & 1 \end{bmatrix}, \qquad c = \begin{bmatrix} 0 \\ 0 \\ 0.14 \end{bmatrix}. \tag*{(61)}

The initial rank-2 basis is obtained from the two snapshots q1q^1 and q2q^2:

Φ0=[−0.086456250.99594990−0.46958378−0.018889890.878644630.08790323].(62)\Phi_0 = \begin{bmatrix} -0.08645625 & 0.99594990 \\ -0.46958378 & -0.01888989 \\ 0.87864463 & 0.08790323 \end{bmatrix}. \tag*{(62)}

The associated singular values are

Σ0=diag⁡(2.91533718, 0.25061752).(63)\Sigma_0 = \operatorname{diag}(2.91533718,\ 0.25061752). \tag*{(63)}

With γin=0.1\gamma_{\mathrm{in}} = 0.1, four in-span updates give the preconditioned basis

Φin,pre=[−0.40529412−0.913853140.42031170−0.21024705−0.811834190.34737389],(64)\Phi_{\mathrm{in,pre}} = \begin{bmatrix} -0.40529412 & -0.91385314 \\ 0.42031170 & -0.21024705 \\ -0.81183419 & 0.34737389 \end{bmatrix}, \tag*{(64)}

and singular values

Σin,pre=diag⁡(2.47602980, 0.03093119).(65)\Sigma_{\mathrm{in,pre}} = \operatorname{diag}(2.47602980,\ 0.03093119). \tag*{(65)}

The subspace is unchanged:

span⁡(Φin,pre)=span⁡(Φ0).(66)\operatorname{span}(\Phi_{\mathrm{in,pre}}) = \operatorname{span}(\Phi_0). \tag*{(66)}

Full-order operator query

The out-of-span correction is generated from the ROM state,

qcorr=GqROM5+c=[1.29594031−0.629134672.13998855].(67)q_{\mathrm{corr}} = Gq_{\mathrm{ROM}}^5 + c = \begin{bmatrix} 1.29594031 \\ -0.62913467 \\ 2.13998855 \end{bmatrix}. \tag*{(67)}

The residual relative to the initial ROM plane is

r=qcorr−Φ0Φ0Tqcorr=[−0.010292140.368094320.19571200],(68)r = q_{\mathrm{corr}} - \Phi_0 \Phi_0^{\mathsf{T}} q_{\mathrm{corr}} = \begin{bmatrix} -0.01029214 \\ 0.36809432 \\ 0.19571200 \end{bmatrix}, \tag*{(68)}

with

ρ=∥r∥2=0.41701624,ρ2=0.17390255.(69)\rho= \lVert r\rVert_2 = 0.41701624,\qquad\rho^2 = 0.17390255. \tag*{(69)}

The correction coefficients in the two pre-update coordinate systems are

Φ0Tqcorr=[2.063678751.49068781],Φin,preTqcorr=[−2.52698551−0.30864925].(70)\Phi_0^{\mathsf{T}}q_{\mathrm{corr}} = \begin{bmatrix} 2.06367875\\ 1.49068781 \end{bmatrix}, \qquad \Phi_{\mathrm{in,pre}}^{\mathsf{T}}q_{\mathrm{corr}} = \begin{bmatrix} -2.52698551\\ -0.30864925 \end{bmatrix}. \tag*{(70)}

Rank-2 competition

The competition matrices are

Ccorr,out=Σ02+(Φ0Tqcorr)(Φ0Tqcorr)T,(71)C_{\mathrm{corr,out}} = \Sigma_0^2 + (\Phi_0^{\mathsf{T}}q_{\mathrm{corr}})(\Phi_0^{\mathsf{T}}q_{\mathrm{corr}})^{\mathsf{T}}, \tag*{(71)}

and

Ccorr,in=Σin,pre2+(Φin,preTqcorr)(Φin,preTqcorr)T.(72)C_{\mathrm{corr,in}} = \Sigma_{\mathrm{in,pre}}^2 + (\Phi_{\mathrm{in,pre}}^{\mathsf{T}}q_{\mathrm{corr}})(\Phi_{\mathrm{in,pre}}^{\mathsf{T}}q_{\mathrm{corr}})^{\mathsf{T}}. \tag*{(72)}

Their weakest eigenvalues are

λmin⁡(Ccorr,out)=1.44819350,λmin⁡(Ccorr,in)=0.04743386.(73)\lambda_{\min}(C_{\mathrm{corr,out}}) = 1.44819350,\qquad \lambda_{\min}(C_{\mathrm{corr,in}}) = 0.04743386. \tag*{(73)}

Thus

ηout=ρ2λmin⁡(Ccorr,out)≈0.12,ηin=ρ2λmin⁡(Ccorr,in)≈3.67.(74)\eta_{\mathrm{out}} = \frac{\rho^2}{\lambda_{\min}(C_{\mathrm{corr,out}})} \approx0.12,\qquad \eta_{\mathrm{in}} = \frac{\rho^2}{\lambda_{\min}(C_{\mathrm{corr,in}})} \approx3.67. \tag*{(74)}

Post-update bases

Applying the same truncated iSVD update (with γout=1.0\gamma_{\mathrm{out}} = 1.0 for simplicity) to both pre-update states gives

Φout,post=[0.17789643−0.94264402−0.39173043−0.331138170.902718190.04206862],(75)\Phi_{\mathrm{out,post}} = \begin{bmatrix} 0.17789643 & -0.94264402\\ -0.39173043 & -0.33113817\\ 0.90271819 & 0.04206862 \end{bmatrix}, \tag*{(75)}

and

Φin,post=[−0.458184820.480870610.330128480.87292840−0.825276840.08221594].(76)\Phi_{\mathrm{in,post}} = \begin{bmatrix} -0.45818482 & 0.48087061\\ 0.33012848 & 0.87292840\\ -0.82527684 & 0.08221594 \end{bmatrix}. \tag*{(76)}

We define the residual capture as ∥ΦpostTr^∥2\lVert\Phi_{\mathrm{post}}^{\mathsf{T}}\hat{r}\rVert_2, where r^:=r/∥r∥2\hat{r} := r/\lVert r\rVert_2, the plane angle as the largest principal angle max⁡iarccos⁡(ζi(ΦpreTΦpost))\max_i \arccos(\zeta_i(\Phi_{\mathrm{pre}}^{\mathsf{T}}\Phi_{\mathrm{post}})), where ζi(⋅)\zeta_i(\cdot) are singular values, the retained alignment as max⁡j∣(ϕjpost)Tr^∣\max_j \lvert(\phi_j^{\mathrm{post}})^{\mathsf{T}}\hat{r}\rvert, where ϕjpost\phi_j^{\mathrm{post}} is the jjth column of Φpost\Phi_{\mathrm{post}}, and the correction error as ∥qcorr−ΦpostΦpostTqcorr∥2\lVert q_{\mathrm{corr}}-\Phi_{\mathrm{post}}\Phi_{\mathrm{post}}^{\mathsf{T}}q_{\mathrm{corr}}\rVert_2. The measured outcomes are listed in Table B1.

MethodResidual capturePlane angleRetained alignmentCorrection error
Baseline adaptive0.26015.06∘15.06^\circ0.2491.02×10−21.02 \times10^{-2}
SPIN0.80253.29∘53.29^\circ0.7978.84×10−48.84 \times10^{-4}

Table B1. Main spiral diagnostics for baseline adaptive and SPIN updates.

Core-matrix interpretation

The same effect can be read directly from the small iSVD core matrix. For a pre-update basis Φpre\Phi_{\mathrm{pre}} and singular values Σpre\Sigma_{\mathrm{pre}}, write

α=ΦpreTqcorr,r=qcorr−Φpreα,ρ=∥r∥2,r^=r/ρ.\alpha= \Phi_{\mathrm{pre}}^{\mathsf{T}}q_{\mathrm{corr}}, \qquad r = q_{\mathrm{corr}}-\Phi_{\mathrm{pre}}\alpha, \qquad\rho= \lVert r\rVert_2, \qquad\hat{r}=r/\rho.

The out-of-span iSVD update forms the augmented basis [Φpre r^][\Phi_{\mathrm{pre}}\ \hat{r}] and the core matrix

K=[γoutΣpreα0Tρ](77)K = \begin{bmatrix} \gamma_{\mathrm{out}}\Sigma_{\mathrm{pre}} & \alpha\\ 0^{\mathsf{T}} & \rho \end{bmatrix} \tag*{(77)}

Here we use γout=1.0\gamma_{\mathrm{out}}=1.0. The left singular vectors of KK determine how the retained post-update basis vectors mix the old pre-update modes with the new residual direction. Because the update is truncated back to rank two, the first retained vector corresponds to the dominant slot, while the second retained vector is the weak slot where the old weak mode competes with the incoming residual.

For the baseline adaptive model, Φpre=Φ0\Phi_{\mathrm{pre}}=\Phi_0 and Σpre=Σ0\Sigma_{\mathrm{pre}}=\Sigma_0, giving

Kout=[2.9153371802.0636787500.250617521.49068781000.41701624](78)K_{\mathrm{out}}= \begin{bmatrix} 2.91533718 & 0 & 2.06367875 \\ 0 & 0.25061752 & 1.49068781 \\ 0 & 0 & 0.41701624 \end{bmatrix} \tag*{(78)}

The second retained left singular vector is

uK,out(2)=[0.27395794−0.92887308−0.24928267](79)u^{(2)}_{K,\mathrm{out}}= \begin{bmatrix} 0.27395794 \\ -0.92887308 \\ -0.24928267 \end{bmatrix} \tag*{(79)}

The entries correspond, respectively, to the old first mode, the old second mode, and the new residual direction. Thus the weak retained slot is still dominated by the old second mode, with magnitude 0.929, while only 0.249 of the slot points along the residual direction.

For the SPIN model, Φpre=Φin,pre\Phi_{\mathrm{pre}}=\Phi_{\mathrm{in,pre}} and Σpre=Σin,pre\Sigma_{\mathrm{pre}}=\Sigma_{\mathrm{in,pre}}, giving

Kin=[2.476029800−2.5269855100.03093119−0.30864925000.41701624](80)K_{\mathrm{in}}= \begin{bmatrix} 2.47602980 & 0 & -2.52698551 \\ 0 & 0.03093119 & -0.30864925 \\ 0 & 0 & 0.41701624 \end{bmatrix} \tag*{(80)}

The corresponding retained core vector is

uK,in(2)=[0.10526228−0.594416070.79723860](81)u^{(2)}_{K,\mathrm{in}}= \begin{bmatrix} 0.10526228 \\ -0.59441607 \\ 0.79723860 \end{bmatrix} \tag*{(81)}

Now the same weak slot is much more strongly aligned with the residual direction: the residual contribution has magnitude 0.797, while the old second-mode contribution has dropped to 0.594. The signs are arbitrary because singular vectors are sign-indeterminate; the magnitudes show how the fixed rank-two budget is allocated. In this sense, in-span preconditioning does not add a third basis vector. It repurposes the weak retained slot so that the same out-of-span correction survives the truncation more effectively.

Alternative fixed-rank updates in the spiral example

The spiral also allows comparison with several alternative fixed-rank update rules. All methods receive the same correction qcorrq_{\mathrm{corr}} and use the same rank budget.

Rank-one interpolation

The rank-one interpolation update of Ref. [38] takes the form

Φraw+=Φ0+rαTαTα,α=Φ0Tqcorr.(82)\Phi_{\mathrm{raw}}^{+} = \Phi_0 + \frac{r\alpha^{\mathsf{T}}}{\alpha^{\mathsf{T}}\alpha}, \qquad\alpha= \Phi_0^{\mathsf{T}}q_{\mathrm{corr}}. \tag*{(82)}

This update is constructed so that

Φraw+α=Φ0α+r=qcorr.(83)\Phi_{\mathrm{raw}}^{+}\alpha= \Phi_0\alpha+ r = q_{\mathrm{corr}}. \tag*{(83)}

Therefore the correction lies exactly in the updated span. However, exact interpolation of one state does not imply that the updated basis is well aligned with the new residual direction.

Greedy residual replacement

A greedy residual replacement keeps the dominant old mode and replaces the weak mode by the normalized residual:

ϕ1,greedy+=ϕ10,ϕ2,greedy+=r^.(84)\phi_{1,\mathrm{greedy}}^{+} = \phi_1^0, \qquad\phi_{2,\mathrm{greedy}}^{+} = \hat{r}. \tag*{(84)}

This achieves perfect residual alignment, but discards the in-plane contribution associated with the second old mode. In the spiral, the correction error is therefore

∥qcorr−qROM,greedy6∥2=∣α2∣=1.49068781.(85)\left\lVert q_{\mathrm{corr}} - q_{\mathrm{ROM,greedy}}^{6} \right\rVert_2 = \lvert\alpha_2 \rvert= 1.49068781. \tag*{(85)}

Optimal one-slot rotation

If the dominant first mode is fixed and only the weak second slot is allowed to rotate in the plane spanned by ϕ20\phi_2^0 and r^\hat{r}, the exact-correction update is

ϕ2,opt+=α2ϕ20+rα22+ρ2.(86)\phi_{2,\mathrm{opt}}^{+} = \frac{\alpha_2\phi_2^0 + r}{\sqrt{\alpha_2^2 + \rho^2}}. \tag*{(86)}

This update makes the correction exact while keeping ϕ10\phi_1^0 fixed. Its residual alignment is

ρα22+ρ2=0.26940447.(87)\frac{\rho}{\sqrt{\alpha_2^2 + \rho^2}} = 0.26940447. \tag*{(87)}

The comparison clarifies the role of in-span learning. It does not optimize correction interpolation alone, nor residual alignment alone. Instead, it brings a trajectory-preconditioned basis into the same truncated update, which allows the weak modal slot to be repurposed without discarding essential in-plane content. To compare the alternatives, we use the same residual capture, plane angle, retained alignment, and correction-error diagnostics defined above. We also report the slot angle

θslot=arccos⁡(∣(ϕ2pre)Tϕ2post∣),\theta_{\mathrm{slot}} = \arccos\left(\left\lvert(\phi_2^{\mathrm{pre}})^{\mathsf{T}}\phi_2^{\mathrm{post}} \right\rvert\right),

where ϕ2pre\phi_2^{\mathrm{pre}} is the pre-update weak mode and ϕ2post\phi_2^{\mathrm{post}} is the post-update vector assigned to that weak modal slot. The absolute value removes the arbitrary sign of basis vectors. A small slot angle means that the weak slot remains close to its old direction, whereas a large slot angle indicates that the slot has been substantially repurposed. The resulting comparison is given in Table C1 and visualized in Figure C1.

MethodCapturePlane angleSlot angleRetained alignmentCorrection error
Baseline adaptive0.26015.06∘^\circ21.74∘^\circ0.2491.02×10−21.02 \times10^{-2}
SPIN0.80253.29∘^\circ53.53∘^\circ0.7978.84×10−48.84 \times10^{-4}
Rank-one interpolation0.1629.30∘^\circ5.43∘^\circ0.1320
Greedy replacement1.00090.00∘^\circ90.00∘^\circ1.0001.491
Optimal slot rotation0.26915.63∘^\circ15.63∘^\circ0.2690

Table C1. Comparison of fixed-rank updates on the spiral correction.

Figure C1. Alternative fixed-rank updates expose the tradeoff between fitting the correction and retaining the residual. Rank-one interpolation and optimal one-slot rotation can fit the correction exactly, but do not strongly align the basis with the new residual direction. Greedy residual replacement captures the residual perfectly, but discards important in-plane content. The SPIN update occupies a different part of the tradeoff because the basis has already been organized by the ROM trajectory before the correction arrives.

Hyperparameter sensitivity and additional PDE results

The SPIN framework is governed by three main parameters: the in-span forgetting factor γin\gamma_{\mathrm{in}}, the out-of-span forgetting factor γout\gamma_{\mathrm{out}}, and the out-of-span correction interval zsz_{\mathrm{s}}. For each benchmark, we tune the forgetting factors by minimizing the mean relative L2L_2 error over the online trajectory. The baseline adaptive is tuned over its own γout\gamma_{\mathrm{out}} values, and the SPIN model is tuned over the two-dimensional (γin,γout)(\gamma_{\mathrm{in}}, \gamma_{\mathrm{out}}) grid (Figure D1 and Figure D2). The main-text comparisons therefore use the best-performing version of each adaptive strategy.

Figure D1. Sensitivity to in-span forgetting and correction interval. When γin\gamma_{\mathrm{in}} is close to one, in-span forgetting is weak and the ROM retains a longer memory of its own trajectory; when γin\gamma_{\mathrm{in}} is small, the in-span update emphasizes recent states more aggressively. The most useful regime balances trajectory memory, spectral reweighting and stability between correction events. In the γin×γout\gamma_{\mathrm{in}} \times\gamma_{\mathrm{out}} maps (panels c and f), the reference value reported in the title is the best baseline adaptive error obtained by sweeping its out-of-span forgetting factor (Figure D2).

Figure D2. Out-of-span forgetting for the baseline adaptive model. The baseline adaptive model’s mean relative L2L_2 error as a function of its out-of-span forgetting factor γout\gamma_{\mathrm{out}} (swept on a logarithmic axis), with the SPIN error shown as a dashed reference line. The optimal out-of-span forgetting is problem-dependent—near γout≈0.01\gamma_{\mathrm{out}} \approx0.01 for Burgers and γout≈0.1\gamma_{\mathrm{out}} \approx0.1 for Fisher–KPP—but in both cases the best-tuned baseline adaptive ROM remains substantially less accurate than the SPIN ROM.

For Burgers, the best baseline adaptive model uses γout=0.01\gamma_{\mathrm{out}} = 0.01, while the best SPIN model uses (γin,γout)=(1.0,0.25)(\gamma_{\mathrm{in}}, \gamma_{\mathrm{out}}) = (1.0, 0.25). For Fisher–KPP, the best baseline adaptive model uses γout=0.1\gamma_{\mathrm{out}} = 0.1, while the best SPIN model uses (γin,γout)=(0.9,0.25)(\gamma_{\mathrm{in}}, \gamma_{\mathrm{out}}) = (0.9, 0.25). These values are reported in the main text and repeated in Table E1.

BenchmarkParameterValueNotes
Spiralα\alpha0.4Spiral vertical drift
SpiralΔt\Delta t0.35Exact discrete map
Spiralt0t_04.0Initial time
Spiralrr2Rank-2 ROM
Spiralγin\gamma_{\mathrm{in}}0.1In-span forgetting
BurgersNxN_x1000Periodic finite-difference grid
Burgersν\nu10−210^{-2}Viscosity
BurgersΔt\Delta t10−310^{-3}Backward Euler
BurgersNtN_t500Prediction steps
Burgersrr4ROM rank
Burgersmm4QDEIM samples
Burgerszsz_s10Correction interval
Burgersγin,γout\gamma_{\mathrm{in}}, \gamma_{\mathrm{out}}1.0, 0.25Forgetting factors (SPIN)
Burgersγout\gamma_{\mathrm{out}}0.01Forgetting factor (baseline adaptive)
Fisher–KPPNxN_x256Periodic finite-difference grid
Fisher–KPPDD10−410^{-4}Diffusion coefficient
Fisher–KPPβ\beta20Reaction rate
Fisher–KPPΔt\Delta t10−310^{-3}Backward Euler
Fisher–KPPNtN_t500Prediction steps
Fisher–KPPrr4ROM rank
Fisher–KPPmm4QDEIM samples
Fisher–KPPzsz_s25Correction interval
Fisher–KPPγin,γout\gamma_{\mathrm{in}}, \gamma_{\mathrm{out}}0.9, 0.25Forgetting factors (SPIN)
Fisher–KPPγout\gamma_{\mathrm{out}}0.1Forgetting factor (baseline adaptive)

Table E1. Main benchmark parameters.

The sensitivity maps also clarify the role of in-span forgetting. The best SPIN models occur at large γin\gamma_{\mathrm{in}}, meaning that the PDE benchmarks do not benefit from aggressive in-span forgetting. This is consistent with the spectral diagnostics in the main text: unlike the first spiral correction, the PDE trajectories carry enough energy in the retained modes that the singular values are reinforced rather than suppressed. The role of in-span learning in these cases is therefore not to erase a stale mode, but to continuously condition the basis spectrum and orientation using the ROM trajectory.

The maps show a useful practical pattern. When γin\gamma_{\mathrm{in}} is large, the model can become sensitive to the out-of-span forgetting factor γout\gamma_{\mathrm{out}}, and poor choices of γout\gamma_{\mathrm{out}} may lead to much larger errors. When γin\gamma_{\mathrm{in}} is smaller, the performance is often more robust across γout\gamma_{\mathrm{out}}, although not always optimal. Thus, aggressive in-span memory can produce the best models when paired with a suitable out-of-span forgetting factor, while smaller γin\gamma_{\mathrm{in}} may provide a safer but more conservative operating regime.

The correction interval zsz_s controls how long the ROM evolves between external corrections. Larger zsz_s reduces the number of full-order operator queries, but gives the ROM more time to drift; smaller zsz_s provides more frequent out-of-span information but increases the hybrid cost. Figure D1 shows that adding the in-span channel shifts this tradeoff. For a fixed target error, the SPIN ROM can use a substantially larger zsz_s than the baseline adaptive ROM, meaning that it can evolve for longer on its own before requiring new external information. This supports the central interpretation of in-span learning as trajectory-informed basis preparation: between correction events, the ROM uses its own predictions to keep the basis better organized, so external corrections can be queried less frequently for the same accuracy level. The interval cannot be made arbitrarily large, however, because if the ROM trajectory drifts too far, its in-span predictions no longer provide a reliable guide to the local dynamics.

Figure D3 reports additional Burgers and Fisher–KPP diagnostics, including robustness across initial conditions and final-time profiles for representative cases.

Additional PDE results for Burgers and Fisher–KPP initial-condition robustness and final-time profiles

Figure D3. Additional PDE results support the main-text comparisons. Initial-condition robustness tests for Burgers and Fisher–KPP. All ROMs are trained with rank r=4r = 4 and m=4m = 4 QDEIM sample points, with offline basis trained on the initial 4 time steps from t=0t = 0 to t=0.004t = 0.004, while prediction is evaluated over 500 time steps from t=0t = 0 to t=0.50t = 0.50, with external corrections every zs=10z_s = 10 steps. Across these additional cases, the SPIN ROM uses the same out-of-span correction mechanism as the baseline adaptive ROM, but produces smaller errors and more accurate final-time profiles.

Reproducibility information

Reproducibility information

The main numerical settings used to generate the spiral, Burgers, and Fisher–KPP results are collected in Table E1. Together with the model definitions, update rules, and diagnostics described in the Methods and appendices above, as well as the source code available at https://github.com/APHedayat/SPIN, these parameters specify the benchmark configurations used for all results reported in the main text and appendices.

Mathematical connection between in-span and in-context learning

In-context learning (ICL) broadly refers to the ability of a model to modify its inference-time behavior using information contained in the current context, without retraining the underlying offline model [55, 56, 57]. Although most commonly associated with large language models, several analyses have shown that the mechanism can be understood more generally as an inference-time computation performed over a sequence of contextual representations. In particular, linearized attention admits a recurrent fast-weight interpretation in which context is accumulated through rank-one outer-product updates [58, 59]. We show here that the in-span update underlying SPIN has the same mathematical structure.

Outer-product memory in linear attention

Consider a sequence of representations x1,x2,…x^{1}, x^{2}, \ldots. In a linear-attention layer, each representation generates a key, value, and query,

kj=φ(Wkxj),vj=Wvxj,qj=φ(Wqxj),(88)k^{j} = \varphi(W_{k}x^{j}), \qquad v^{j} = W_{v}x^{j}, \qquad q^{j} = \varphi(W_{q}x^{j}), \tag*{(88)}

where φ\varphi denotes the feature map used to linearize attention. Ignoring normalization, the causal attention computation can be written in terms of a recurrent context matrix

Fj=Fj−1+vj(kj)T,(89)F_{j} = F_{j-1} + v^{j}(k^{j})^{\mathsf{T}}, \tag*{(89)}

with the corresponding output obtained by querying this accumulated state,

y~j=Fjqj.(90)\widetilde{y}^{j} = F_{j}q^{j}. \tag*{(90)}

Thus, rather than explicitly retaining the entire preceding sequence, the model can carry forward a finite-dimensional state FjF_{j} containing a sum of outer products generated from the preceding context.

For comparison with the forgetting mechanism used by SPIN, consider the exponentially weighted form

Fj=λFj−1+vj(kj)T,0<λ≤1,(91)F_{j} = \lambda F_{j-1} + v^{j}(k^{j})^{\mathsf{T}}, \qquad0 < \lambda\leq1, \tag*{(91)}

which gives

Fs=λsF0+∑j=1sλs−jvj(kj)T(92)F_{s} = \lambda^{s}F_{0} + \sum_{j=1}^{s}\lambda^{s-j}v^{j}(k^{j})^{\mathsf{T}} \tag*{(92)}

after applying ss representations starting from F0F_{0}. The choice λ=1\lambda= 1 recovers the purely additive fast-weight accumulation in Eq. (F.2); introducing λ<1\lambda< 1 simply gives greater weight to recent context.

The in-span state is a symmetric fast-weight memory

Between external corrections, SPIN receives only states generated by the ROM itself. Let aj∈Rra^{j} \in\mathbb{R}^{r} denote the reduced coordinates of the jjth in-span prediction. As derived in the main text, the reduced covariance evolves according to

Cj=γin2Cj−1+aj(aj)T,(93)C_{j} = \gamma_{\mathrm{in}}^{2}C_{j-1} + a^{j}(a^{j})^{\mathsf{T}}, \tag*{(93)}

and therefore

Cs=γin2sC0+∑j=1sγin2(s−j)aj(aj)T.(94)C_{s} = \gamma_{\mathrm{in}}^{2s}C_{0} + \sum_{j=1}^{s}\gamma_{\mathrm{in}}^{2(s-j)}a^{j}(a^{j})^{\mathsf{T}}. \tag*{(94)}

Comparing Eq. (F.4) and Eq. (F.6) shows an exact correspondence under

λ=γin2,vj=kj=aj,Fj=Cj.(95)\lambda= \gamma_{\mathrm{in}}^{2}, \qquad v^{j} = k^{j} = a^{j}, \qquad F_{j} = C_{j}. \tag*{(95)}

Thus, the SPIN covariance is a symmetric fast-weight memory whose context consists of the model’s own reduced trajectory. The initial state C0=Σ02C_0 = \Sigma_0^2 plays the role of a prior spectral memory inherited from the offline model, while each new prediction contributes the rank-one contextual update aj(aj)⊤a^j(a^j)^\top. Forgetting determines how rapidly earlier context is discounted.

In this sense, in-span learning can be viewed as a form of self-generated in-context learning for dynamical systems, where the surrogate uses the context created by its own evolving trajectory to reorganize its internal representation and thereby change how it responds to future information.

References

  1. [1]Lyu, Z., Kenway, G. K. W. & Martins, J. R. R. A. Aerodynamic shape optimization investigations of the Common Research Model wing benchmark. AIAA J. 53, 968–985 (2015).DOI
  2. [2]Lam, R. et al. Learning skillful medium-range global weather forecasting. Science 382, 1416–1421 (2023).
  3. [3]Kochkov, D. et al. Neural general circulation models for weather and climate. Nature 632, 1060–1066 (2024).DOI
  4. [4]Bodnar, C. et al. A foundation model for the Earth system. Nature 641, 1180–1187 (2025).
  5. [5]Swischuk, R., Kramer, B., Huang, C. & Willcox, K. Learning physics-based reduced-order models for a single-injector combustion process. AIAA J. 58, 2658–2672 (2020).
  6. [6]Khaloie, H. et al. Review of machine learning techniques for optimal power flow. Appl. Energy 388, 125637 (2025).
  7. [7]Xiu, D. & Karniadakis, G. E. Modeling uncertainty in flow simulations via generalized polynomial chaos. J. Comput. Phys. 187, 137–167 (2003).
  8. [8]Degen, D., Cacace, M. & Wellmann, F. 3D multi-physics uncertainty quantification using physics-based machine learning. Sci. Rep. 12, 17491 (2022).
  9. [9]Niederer, S. A., Sacks, M. S., Girolami, M. & Willcox, K. Scaling digital twins from the artisanal to the industrial. Nat. Comput. Sci. 1, 313–320 (2021).
  10. [10]Kapteyn, M. G., Pretorius, J. V. R. & Willcox, K. E. A probabilistic graphical model foundation for enabling predictive digital twins at scale. Nat. Comput. Sci. 1, 337–347 (2021).
  11. [11]Antoulas, A. C. Approximation of Large-Scale Dynamical Systems (SIAM, Philadelphia, 2005).
  12. [12]Rowley, C. W. & Dawson, S. T. M. Model reduction for flow analysis and control. Annu. Rev. Fluid Mech. 49, 387–417 (2017).
  13. [13]Antoulas, A. C., Beattie, C. A. & Gügercin, S. Interpolatory Methods for Model Reduction (SIAM, Philadelphia, 2020).
  14. [14]Kramer, B., Peherstorfer, B. & Willcox, K. Learning nonlinear reduced models from data with operator inference. Annu. Rev. Fluid Mech. 56, 521–548 (2024).
  15. [15]Lumley, J. L. The structure of inhomogeneous turbulent flows. In Yaglom, A. M. & Tatarski, V. I. (eds.) Atmospheric Turbulence and Radio Wave Propagation, 166–178 (Nauka, Moscow, 1967).
  16. [16]Sirovich, L. Turbulence and the dynamics of coherent structures. I–III. Quart. Appl. Math. 45, 561–590 (1987).
  17. [17]Holmes, P., Lumley, J. L. & Berkooz, G. Turbulence, Coherent Structures, Dynamical Systems and Symmetry (Cambridge Univ. Press, Cambridge, 1996).
  18. [18]Benner, P., Gugercin, S. & Willcox, K. A survey of projection-based model reduction methods for parametric dynamical systems. SIAM Review 57, 483–531 (2015).
  19. [19]Carlberg, K., Barone, M. & Antil, H. Galerkin v. least-squares Petrov–Galerkin projection in nonlinear model reduction. J. Comput. Phys. 330, 693–734 (2017).
  20. [21]Huang, C., Duraisamy, K. & Merkle, C. L. Component-based reduced order modeling of large-scale complex systems. Front. Phys. 10, 900064 (2022).
  21. [22]Lee, K. & Carlberg, K. T. Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders. J. Comput. Phys. 404, 108973 (2020).
  22. [23]Geelen, R., Wright, S. & Willcox, K. Operator inference for non-intrusive model reduction with quadratic manifolds. Comput. Methods Appl. Mech. Eng. 403, 115717 (2023).
  23. [24]Peherstorfer, B. Breaking the Kolmogorov barrier with nonlinear model reduction. Not. Am. Math. Soc. 69, 725–733 (2022).
  24. [25]Barnett, J. & Farhat, C. Quadratic approximation manifold for mitigating the Kolmogorov barrier in nonlinear projection-based model order reduction. J. Comput. Phys. 464, 111348 (2022).
  25. [26]Reiss, J., Schulze, P., Sesterhenn, J. & Mehrmann, V. The shifted proper orthogonal decomposition: a mode decomposition for multiple transport phenomena. SIAM J. Sci. Comput. 40, A1322–A1344 (2018).
  26. [27]Nair, N. J. & Balajewicz, M. Transported snapshot model order reduction approach for parametric, steady-state fluid flows containing parameter-dependent shocks. Int. J. Numer. Methods Eng. 117, 1234–1262 (2019).
  27. [28]Rim, D., Peherstorfer, B. & Mandli, K. T. Manifold approximations via transported subspaces: model reduction for transport-dominated problems. SIAM J. Sci. Comput. 45, A170–A199 (2023).
  28. [29]Mirhoseini, M. A. & Zahr, M. J. Model reduction of convection-dominated partial differential equations via optimization-based implicit feature tracking. J. Comput. Phys. 473, 111739 (2023).
  29. [30]Arnold-Medabalimi, N., Huang, C. & Duraisamy, K. Large-eddy simulation and challenges for projection-based reduced-order modeling of a gas turbine model combustor. Int. J. Spray Combust. Dyn. 14, 153–175 (2022).
  30. [31]Amsallem, D. & Farhat, C. Interpolation method for adapting reduced-order models and application to aeroelasticity. AIAA J. 46, 1803–1813 (2008).
  31. [32]Amsallem, D. & Farhat, C. An online method for interpolating linear parametric reduced-order models. SIAM J. Sci. Comput. 33, 2169–2198 (2011).
  32. [33]Amsallem, D., Zahr, M. J. & Farhat, C. Nonlinear model order reduction based on local reduced-order bases. Int. J. Numer. Methods Eng. 92, 891–916 (2012).
  33. [34]Peherstorfer, B., Butnaru, D., Willcox, K. & Bungartz, H. J. Localized discrete empirical interpolation method. SIAM J. Sci. Comput. 36, A168–A192 (2014).
  34. [35]Peherstorfer, B. & Willcox, K. Dynamic data-driven reduced-order models. Comput. Methods Appl. Mech. Eng. 291, 21–41 (2015).
  35. [36]Peherstorfer, B. & Willcox, K. Online adaptive model reduction for nonlinear systems via low-rank updates. SIAM J. Sci. Comput. 37, A2123–A2150 (2015).
  36. [37]Peherstorfer, B. Model reduction for transport-dominated problems via online adaptive bases and adaptive sampling. SIAM J. Sci. Comput. 42, A2803–A2836 (2020).
  37. [38]Huang, C. & Duraisamy, K. Predictive reduced order modeling of chaotic multi-scale problems using adaptively sampled projections. J. Comput. Phys. 491, 112356 (2023).
  38. [39]Singh, R., Uy, W. I. T. & Peherstorfer, B. Lookahead data-gathering strategies for online adaptive model reduction of transport-dominated problems. Chaos 33, 113112 (2023).
  39. [40]Mohaghegh, A. & Huang, C. Feature-guided sampling strategy for adaptive model order reduction of convection-dominated problems. J. Comput. Phys. 545, 114468 (2026).
  40. [41]Hedayat, A., Padovan, A. & Duraisamy, K. Toward adaptive non-intrusive reduced-order models: design and challenges. Struct. Multidiscip. Optim. 69, 151 (2026).
  41. [42]Balzano, L., Chi, Y. & Lu, Y. M. Streaming PCA and subspace tracking: the missing data case. Proc. IEEE 106, 1293–1310 (2018).
  42. [43]Bunch, J. R. & Nielsen, C. P. Updating the singular value decomposition. Numer. Math. 31, 111–129 (1978).
  43. [44]Brand, M. Incremental singular value decomposition of uncertain data with missing values. In European Conference on Computer Vision (ECCV), 707–720 (Springer, 2002).
  44. [45]Brand, M. Fast low-rank modifications of the thin singular value decomposition. Linear Algebra Appl. 415, 20–30 (2006).
  45. [46]Oja, E. Simplified neuron model as a principal component analyzer. J. Math. Biol. 15, 267–273 (1982).
  46. [47]Balzano, L., Nowak, R. & Recht, B. Online identification and tracking of subspaces from highly incomplete information. In Proc. 48th Annu. Allerton Conf., 704–711 (2010).
  47. [48]Yang, B. Projection approximation subspace tracking. IEEE Trans. Signal Process. 43, 95–107 (1995).
  48. [49]Chi, Y., Eldar, Y. C. & Calderbank, R. PETRELS: parallel subspace estimation and tracking by recursive least squares from partial observations. IEEE Trans. Signal Process. 61, 5947–5959 (2013).
  49. [50]Zimmermann, R., Peherstorfer, B. & Willcox, K. Geometric subspace updates with applications to online adaptive nonlinear model reduction. SIAM J. Matrix Anal. Appl. 39, 234–261 (2018).
  50. [51]Hedayat, A., Mohaghegh, A., Balzano, L., Huang, C. & Duraisamy, K. History-aware adaptive reduced-order models via incremental singular value decomposition. Preprint at arXiv:2605.28684 (2026).arxiv.org/abs/2605.28684
  51. [52]Chaturantabut, S. & Sorensen, D. C. Nonlinear model reduction via discrete empirical interpolation. SIAM J. Sci. Comput. 32, 2737–2764 (2010).
  52. [53]Drmač, Z. & Gugercin, S. A new selection operator for the discrete empirical interpolation method. SIAM J. Sci. Comput. 38, A631–A648 (2016).
  53. [54]Carlberg, K., Bou-Mosleh, C. & Farhat, C. Efficient non-linear model reduction via a least-squares Petrov–Galerkin projection and compressive tensor approximations. Int. J. Numer. Methods Eng. 86, 155–181 (2011).
  54. [55]Brown, T. B. et al. Language models are few-shot learners. In Adv. Neural Inf. Process. Syst. (NeurIPS) (2020).
  55. [56]Xie, S. M., Raghunathan, A., Liang, P. & Ma, T. An explanation of in-context learning as implicit Bayesian inference. In Int. Conf. Learn. Represent. (ICLR) (2022).
  56. [57]Akyürek, E., Schuurmans, D., Andreas, J., Ma, T. & Zhou, D. What learning algorithm is in-context learning? Investigations with linear models. In Int. Conf. Learn. Represent. (ICLR) (2023).
  57. [58]A. Katharopoulos, A. Vyas, N. Pappas, F. Fleuret. Transformers are RNNs: fast autoregressive transformers with linear attention. ICML, (2020).
  58. [59]I. Schlag, K. Irie, J. Schmidhuber. Linear transformers are secretly fast weight programmers. ICML, (2021).

Paper details

vlnc:W_6mvan5wa
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 9

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents