WorksComputer Science
In-context learning from self-generated trajectories for adaptive model reduction
Abstract
High-fidelity simulations of complex physical systems are often too expensive for repeated prediction, design, and control. Reduced-order models address this computational cost by compressing high-dimensional dynamics into low-dimensional representations that can be evaluated rapidly, but they often lose accuracy when online dynamics drift beyond the offline training data. Adaptive methods address this limitation by updating the reduced subspace online using external, out-of-span information, such as full-order corrections or sensor snapshots. We discovered that a complementary and previously unexploited in-span adaptation channel exists within the current reduced subspace. To exploit this channel, we continually update the reduced representation using the model's own predictions through an incremental singular-value decomposition with a forgetting factor. This produces a trajectory-informed spectral preconditioner in which the reduced subspace remains unchanged, while the basis is reweighted and realigned according to the directions visited by the evolving dynamics. This internal reorganization prepares the basis to absorb future out-of-span corrections more effectively, improving adaptation without requiring additional external information. We expose the mechanism through a three-dimensional spiral example and demonstrate its benefits on nonlinear partial differential equations, including viscous Burgers and Fisher–KPP dynamics. We also discuss how in-span learning can be interpreted as a dynamical-systems analogue of in-context learning. More broadly, in-span learning suggests a new principle for computational science, revealing that model-generated trajectories contain more usable information than previously recognized.
Introduction
Introduction
Across science and engineering, applications such as aircraft design [1], weather forecasting [2, 3, 4], combustion predictions [5], power-grid control [6], uncertainty quantification [7, 8], and digital twins [9, 10] all rely on numerical models that require expensive computations. There is also a need to run these computations repeatedly across parameters, controls, uncertain inputs, or real-time decisions. This setting calls for simulators that are accurate enough to trust and inexpensive enough to use freely.
Reduced-order models (ROMs) address this need by compressing high-dimensional simulations into low-dimensional representations [11, 12, 13, 14]. Although a discretized physical model may contain millions of degrees of freedom, its relevant trajectories often occupy a much smaller region of state space. Projection-based ROMs exploit this structure by constructing a low-dimensional subspace, commonly from proper orthogonal decomposition (POD) of full-order snapshots [15, 16, 17], and evolving only the coordinates of the solution in that subspace [18]. When the online dynamics remain close to the training data, this compression can deliver orders-of-magnitude speed-ups [21]. The same compression also creates the central failure mode of ROMs. A static ROM is trained offline and deployed with a fixed basis. It can interpolate accurately within the regime represented by its snapshots, but it loses predictive power when the online trajectory drifts beyond that regime. This is a form of distribution shift that exists whether the reduced representation is a linear subspace, a nonlinear manifold [23], or a learned surrogate [22]. For projection-based ROMs considered in this work, this limitation is especially severe for systems with slowly decaying Kolmogorov -width, where no fixed low-dimensional linear subspace can represent the evolving solution uniformly well over long times [24, 25]. Transport-dominated flows, shocks, moving fronts, combustion, and detonation dynamics are typical examples [26, 27, 28, 29, 30]. They are also among the settings where fast predictive simulation would be most valuable.
Adaptive ROMs attempt to overcome this rigidity by allowing the reduced representation to evolve during the online simulation. Early efforts included interpolation between locally trained bases [31, 32, 33, 34]. Pioneered by Peherstorfer & Willcox [35, 36], incremental basis approaches have gained attention, where the basis is updated online with newly acquired information [37, 38, 39, 40, 41]. A related line of work draws on subspace-tracking ideas [42, 43, 44, 45, 46, 47, 48, 49], which have motivated geometric basis updates [50] and history-aware adaptation [51]. These incremental basis-updating approaches differ in how they obtain new information and how they update the basis, but they share a common premise. The basis is adapted using information with a nonzero component outside the current reduced subspace. This external information may come from a full-order state, a sensor snapshot, or another correction oracle. We call such data out-of-span. The premise is natural because only out-of-span information can move the subspace. A snapshot that lies exactly inside the current subspace has zero residual and therefore appears, from the usual subspace-update viewpoint, to add no value.
In this paper, we show that this reasoning misses a complementary adaptation channel. Between external corrections, a ROM continuously produces predictions of its own state. By construction, these predictions lie inside the current reduced subspace, and we therefore call them in-span. They cannot expand the subspace, but they can still reorganize the representation inside it. We show that when the ROM’s own predictions are streamed through an incremental singular-value decomposition (iSVD) [43, 44, 45] with forgetting, they rotate the orthonormal basis within the same subspace and reweight its singular spectrum. The result is a trajectory-informed spectral preconditioner. It does not change what the ROM can represent at that instant, but it changes how the basis is prepared to absorb the next external correction. We refer to the resulting adaptive ROM as SPIN, for Spectral Preconditioning via IN-span learning (Figure 1). Throughout the paper, we use baseline adaptive ROM for the iSVD-based adaptive model that only uses external out-of-span corrections and performs no in-span updates between correction events. It is notable that the baseline adaptive ROM is itself representative of the state-of-the-art [51].
Figure 1. In-span learning adapts a reduced-order model from its own predictions. Three reduced-order modeling strategies are shown through their offline/online timelines and the corresponding evolution of the reduced representation. (a) Three-dimensional spiral prediction comparing the exact full-order model (FOM) trajectory, static ROM, baseline adaptive ROM, and SPIN ROM. (b) A static ROM is trained offline and then deployed online with no adaptation. Its reduced trajectory remains confined to the original static subspace and can lose track of the full-order trajectory as the dynamics leave the training regime. (c) A baseline adaptive ROM queries external out-of-span information during the online phase, such as a correction snapshot , and uses it to move the subspace toward the full-order dynamics. (d) The proposed SPIN ROM uses the same external correction events, but also streams its own intermediate predictions as in-span information. These in-span updates do not move the subspace by themselves, but they reweight and reorient the basis before the next correction arrives, preparing the reduced representation to absorb out-of-span information more effectively.
We develop in-span learning first at the level of the update mechanism, then in a closed-form spiral example, and finally in nonlinear PDEs where the same mechanism improves long time-horizon prediction. Across these examples, the same conclusion emerges. Reduced models can learn not only from new data outside their span, but also from the trajectory they have already produced inside it.
Results
A hidden adaptation channel inside the current subspace
We start with a brief overview of the in-span learning mechanism, showing how a model’s own predictions can contribute to its adaptation. A summary of the notation used throughout the paper is provided in Section A. Consider a ROM with orthonormal basis and singular values collected in . At any time step, the ROM prediction has the form , where is the reduced coordinate vector. This prediction lies exactly in the current reduced subspace, so an iSVD update sees zero orthogonal residual and cannot add a new direction. The point of in-span learning is that this zero-residual snapshot can still change the basis state. Let denote the current covariance represented in reduced coordinates. With forgetting factor , the in-span iSVD update reduces to a rank-one update of this reduced covariance,
Diagonalizing this updated small matrix, , gives the new squared singular values through and an orthogonal rotation of the basis inside the same subspace. Equivalently,
Thus, an in-span update preserves the geometric object that determines what states the ROM can represent, but changes the spectral object that determines the modal weights. The formal iSVD update, the derivation of Eq. (1), and the proof of Eq. (2) are given in § 5.4 and § 5.5.
(1) gives the mechanism a simple interpretation. The vector records which directions of the current basis the ROM trajectory is using. The update discounts the previous spectral state by and adds the rank-one contribution from the current ROM prediction. A direction that is strongly represented in the recent trajectory can be reinforced, whereas a direction that receives too little trajectory energy can decay. The outcome is trajectory-dependent and controlled by the forgetting factor, which sets the memory horizon of in-span learning. With close to one, the ROM accumulates many recent predictions and reorganizes the basis around sustained dynamics. With smaller , it emphasizes the most recent trajectory and can rapidly suppress inactive directions. The reinforcement–suppression threshold is derived in § 5.5.
This in-span update matters because the adaptive ROMs studied here operate at fixed rank. When the next external correction arrives, the new out-of-span information must be merged with the existing basis and then truncated back to rank . The outcome of that truncation depends not only on the incoming correction, but also on the spectral state and orientation of the basis that receives it. In-span learning therefore acts as a trajectory-informed spectral preconditioner for the next out-of-span update. This mechanism uses no additional full-order information. Between external corrections, the ROM already produces the states used in the in-span update. The only extra operation is a small SVD in the reduced space, followed by a rotation and reweighting of the existing basis and the usual refresh of basis-dependent quantities in the hyper-reduced model when needed. The cost and implementation details are described in § 5.9 and § 5.7.
A toy example to expose a key mechanism
To illustrate in-span learning in a simple setting, we begin with a three-dimensional spiral whose full-order dynamics are known exactly as with . This trajectory solves an affine linear system in , which we advance with its exact discrete flow. The full setup is given in § 5.1.
The initial ROM is deliberately minimal. We form a rank-2 POD basis from only two full-order snapshots, and , and project the exact discrete dynamics onto the resulting plane . The static ROM trajectory remains confined to and drifts as the true spiral leaves this plane. This small example already contains the essential failure mode of a static ROM, in which the model can evolve only inside the space it was given.
The in-span updates use only the ROM trajectory. At an intermediate online time step , before the next external correction, the ROM produces , where is the reduced coordinate vector in the initial basis . Let denote the time indices of the in-span ROM predictions streamed since the previous external correction. After these updates, the reduced covariance is
where contains the initial singular values associated with . We use a small value to make the spectral contraction visible in the spiral illustration; in the PDE experiments, the inspan forgetting factors are chosen . Writing gives the preconditioned singular values through and the in-plane basis rotation through , with .
Figure 2 shows this effect geometrically. The full-order spiral continues in three dimensions, while the ROM trajectory evolves inside the initial rank-2 plane. In this particular example, the reduced trajectory repeatedly visits one dominant in-plane direction while leaving the other nearly inactive. The covariance ellipse therefore rotates toward the visited direction and contracts along the direction that receives little trajectory energy. The initial singular values are transformed by four in-span updates into . The second singular value is reduced by almost an order of magnitude, while the subspace itself is unchanged.
Figure 2. A closed-form spiral reveals in-span spectral preconditioning. (a) The exact full-order spiral trajectory (FOM), the rank-2 ROM trajectory advanced inside its fixed initial subspace , and the growing discrepancy (error) between them; the two snapshots , used to build the initial basis are highlighted, and the grey plane marks . (b)–(e) Covariance ellipses inside after in-span updates, drawn in the fixed coordinate frame with axes along and . Each ellipse is set by the eigenvectors and eigenvalues of the reduced covariance . Although the in-span updates use only ROM-predicted states lying in and therefore add no new direction, the spectrum rotates and contracts. The active direction is reinforced while the unvisited direction is progressively suppressed. The subspace is unchanged; only the basis orientation and singular spectrum evolve.
Here, we separate two objects that are often conflated. The subspace determines which states the ROM can represent, while the spectrum assigns importance to the directions already present within that subspace. The in-span recursion leaves the first object unchanged but modifies the second. On its own, this cannot correct the ROM, because no new direction has entered the span. Its effect becomes visible when an out-of-span correction later arrives.
In-span preconditioning changes how the next correction is absorbed
To receive an out-of-span correction without having access to the exact full-order trajectory, we perform a full-order operator query. In the spiral example, the exact one-step flow is the affine map , where and are obtained from the exact spiral dynamics in § 5.1. Starting from the ROM state , this gives . At the first out-of-span correction, this snapshot is identical for the baseline adaptive ROM and the SPIN ROM. Its residual relative to the initial ROM plane has norm , so both methods receive the same out-of-span information. The difference is the basis state that receives it. In the baseline adaptive ROM, the correction is applied to the original spectrum; in the SPIN ROM, it is applied after the ROM trajectory has preconditioned the spectrum.
We now show that, in this example, in-span preconditioning acts as a weak-mode suppressor. The ROM trajectory has reduced the strength of a nearly inactive in-plane direction, so the incoming residual faces a smaller modal competitor when the update is truncated back to rank . To quantify this local rank competition, we compare the residual strength with the weakest retained in-plane direction before the residual is added. For a correction with in-plane coefficients , define , which is the in-plane block of the small iSVD core covariance. We then define the residual-to-weakest-mode ratio . Here, indicates that the residual is weak relative to the existing in-plane core content, while indicates that it can compete for one of the retained rank- slots. This ratio is used only as a diagnostic of the local rank competition; the actual truncated update is determined by the full core SVD, which is examined in Section B.
For the same correction snapshot, and with for simplicity, the baseline adaptive update gives and , whereas the SPIN update gives and . The residual norm has not changed. The increase in comes from the denominator, meaning that the same residual now competes against a much smaller in-plane core direction.
The geometric outcome is shown in Figure 3. With out-of-span adaptation alone, the updated plane changes only modestly and the new residual is mostly discarded. With in-span preconditioning, the updated plane swings toward the new local direction of the spiral. Quantitatively, enabling in-span learning increases the residual capture from 0.26 to 0.80 and the plane-change angle from to , while decreasing the correction error from to . These diagnostics are defined in § 5.8. Full numerical details for the spiral example are provided in Section B, and a comparison with several alternative fixed-rank basis updates is available in Section C.
Figure 3. The same correction is better absorbed after in-span preconditioning. Both adaptive ROMs receive the same correction and use the same truncated iSVD update; only the incoming basis state differs. (a,b) Baseline adaptive and SPIN updates. In-span preconditioning rotates and reweights the basis before the correction arrives, so the post-update plane swings more strongly toward the local dynamics. (c) Modal slot competition, including correction energy, , and . (d) Residual capture, plane-change angle, and correction error. In this spiral example, in-span preconditioning suppresses inactive modal content so the same residual can occupy the weak slot of the fixed-rank basis.
SPIN improves nonlinear PDE prediction
The spiral example isolates the mechanism, but it does not test whether the same effect survives the numerical ingredients of a practical ROM. We therefore consider two nonlinear PDEs, namely viscous Burgers and Fisher–KPP, solved with implicit time integration and hyper-reduced (via QDEIM [52, 53]) least-squares Petrov–Galerkin (LSPG) [54, 19] ROMs. This setting tests in-span learning within a broadly used framework for nonlinear scientific simulation. Viscous Burgers dynamics tests transport of a coherent structure, while Fisher–KPP reaction–diffusion tests front propagation and saturation driven by a nonlinear reaction term. In both cases, all reduced models are initialized from a negligible offline training phase, so the comparison is deliberately extrapolative.
For both PDEs we compare the full-order model, a static ROM, the baseline adaptive ROM, and the proposed SPIN ROM. The two adaptive ROMs use the same rank, QDEIM sample count, correction interval, and full-order operator query mechanism to receive the correction snapshot. The only algorithmic difference is that SPIN also updates its basis and singular values between external corrections using the ROM’s own intermediate predictions. The full discretizations, LSPG–QDEIM formulation, correction procedure, and hyperparameters are given in § 5.2 and § 5.6.
For each adaptive model, the forgetting factors reported here are selected from an ablation sweep that minimizes the mean relative error over the online trajectory. Thus, the comparison is between optimized memory configurations of the two adaptive strategies. For Burgers, the best baseline adaptive ROM uses , while the best SPIN ROM uses . For Fisher–KPP, the corresponding values are and . The full forgetting-factor ablations, correction-interval sweeps, and additional PDE robustness tests are reported in Section D, with the benchmark parameters summarized in Section E for reproduction
purposes.
Spectral diagnostics confirm the in-span mechanism
Figure 4 shows Burgers predictions, where we use a rank-4 ROM with QDEIM sample points, trained from only the first four full-order snapshots from to , and evaluated over a 500-step prediction horizon from to , with out-of-span corrections every time steps. The static ROM fails quickly because its basis represents only the early offline regime. The baseline adaptive ROM performs substantially better and follows the advecting pulse reasonably well, showing that the external corrections supply useful missing directions. Adding the in-span channel further reduces amplitude and shape errors as the pulse moves across the domain, and the SPIN error remains consistently below the baseline adaptive error without adding full-order correction steps.
Figure 4. SPIN ROM improves viscous Burgers prediction. Viscous Burgers dynamics with rank and QDEIM sample points. The offline basis is trained on the initial 4 time steps from to , while prediction is evaluated over 500 time steps from to , with external corrections every steps. (a) Solution profiles at four reported times. The static ROM remains tied to the early training regime and drifts away from the transported pulse. The baseline adaptive ROM follows the FOM more closely, while the SPIN ROM further reduces the amplitude and shape errors. (b) Relative error over time. Faint vertical marks indicate the out-of-span correction events. The SPIN ROM uses the same rank, sample count, correction interval, and number of full-order correction steps as the baseline adaptive ROM, but maintains a lower error throughout the trajectory.
Fisher–KPP in Figure 5 provides a more demanding test. A localized initial pulse grows through the reaction term, develops two traveling fronts, and eventually saturates over much of the domain. We again use a rank-4 LSPG–QDEIM ROM with sample points, trained from only the first four full-order snapshots from to , and evaluated over a 500-step prediction horizon from to , with out-of-span corrections every time steps. The static ROM misses the expanding front after the solution leaves the training window. The baseline ROM improves on the static model, but develops visible oscillations in the saturated region and front-location errors. In contrast, the SPIN ROM follows both the plateau and the moving fronts more closely, with a larger separation between the adaptive error curves than in Burgers.
Figure 5. SPIN ROM improves Fisher–KPP front propagation. Fisher–KPP reaction–diffusion dynamics with rank and QDEIM sample points. The offline basis is trained on the initial 4 time steps from to , while prediction is evaluated over 500 time steps from to , with external corrections every steps. (a) Solution profiles at four reported times. The static ROM misses the expanding fronts after the solution leaves the training window. The baseline adaptive ROM improves on the static model but develops oscillations in the saturated region, while the SPIN ROM tracks the plateau and moving fronts more closely. (b) Space–time fields and error. The FOM, baseline adaptive ROM, and SPIN ROM are shown as fields over , together with the relative error history and pointwise absolute error fields. The SPIN ROM produces a visibly cleaner space–time structure and substantially smaller error over most of the trajectory.
Spectral diagnostics confirm the in-span mechanism
The PDE results show that SPIN adaptation improves prediction. We next examine whether the same spectral mechanism identified in the spiral also appears in these nonlinear systems. We therefore measure three diagnostics that separate modal reweighting, in-plane basis rotation, and genuine subspace motion, computed along the online trajectory for the baseline adaptive and SPIN ROMs. These results are shown in Figure 6 for both PDEs.
Figure 6. The algebraic fingerprints of in-span learning in nonlinear ROMs. Diagnostics for (a) Burgers and (b) Fisher–KPP, comparing the baseline adaptive and SPIN ROMs. Panels (i), (ii), (iv) and (v) show the retained singular values . Panel (iii) shows basis reorientation, . Panel (vi) shows genuine subspace motion, . In-span updates continuously reshape and rotate the basis inside an essentially fixed subspace, while out-of-span corrections supply the geometric motion.
The first diagnostic is the time history of the singular values carried by the adaptive basis. In a baseline adaptive ROM, these values change only at external correction events. In a SPIN ROM, they are updated at every intermediate step by the ROM trajectory itself, directly reflecting the reduced covariance update in Eq. (1). Unlike the first spiral correction, where one nearly inactive direction is strongly suppressed, the PDE spectra show richer trajectory-dependent behavior, where several retained modes are repeatedly reinforced between correction events. This is consistent with for Burgers and Fisher–KPP, which imply little or no in-span forgetting. The diagnostic message is therefore that successful in-span learning makes the spectral state trajectory-informed.
The second diagnostic measures basis reorientation through . This quantity is sensitive to changes in the orthonormal coordinate frame. It can be nonzero even when the two bases span essentially the same subspace. For the baseline adaptive ROM, the diagnostic is zero between correction events and spikes when an external correction updates the basis. For the SPIN ROM, it remains nonzero throughout the intervals between corrections. This confirms that the ROM predictions are not merely rescaling singular values, and they continuously rotate the basis inside the current subspace.
The third diagnostic isolates genuine subspace change. We compute the principal angles between consecutive subspaces from the singular values of and report . This quantity is insensitive to a pure rotation of the coordinate frame inside the same subspace. Between external corrections, it remains close to numerical zero for the SPIN ROM, confirming that in-span updates preserve the subspace up to numerical precision. At correction events, it jumps for both adaptive models, because only then does out-of-span information enter. The jump is not identical for the two methods, showing that the same type of external event acts on different preconditioned basis states.
Discussion
Projection-based ROMs are usually described by their subspace. That description is correct, but incomplete. A reduced model also carries a basis spectrum, and in fixed-rank adaptive ROMs this spectrum shapes how future updates allocate limited modal capacity. We showed that these two objects can be adapted by different information streams. Out-of-span corrections move the subspace because they contain directions the ROM cannot currently represent, while in-span predictions reorganize the spectral state inside the existing subspace. This separation reveals a hidden degree of freedom in adaptive model reduction. We demonstrated in-span learning first on a closed-form spiral, where in-span update acts as a weak-mode suppressor, and then on Burgers and Fisher–KPP dynamics, where the same in-span channel improves hyper-reduced nonlinear ROM predictions. Across these examples, the common theme is trajectory awareness. The ROM’s own predictions are not merely outputs; they are information about how the current representation is being used.
Several insights follow from this view:
In-span predictions carry adaptive information. Although they cannot move the trial subspace, they can rotate and reweight the directions inside it.
Spectral preconditioning is memory-controlled. In a low-memory regime, it can suppress inactive directions and open modal capacity, as in the spiral. In high-memory regimes, it can instead reinforce several retained modes while still making the basis trajectory-informed.
In-span and out-of-span learning are complementary. External corrections supply missing directions, while in-span updates prepare the spectral state on which those corrections act.
The gains are obtained without additional full-order queries. The SPIN ROM uses the same external correction schedule as the baseline adaptive model, but changes how the available information is organized and used.
These results also clarify where the present framework can be improved. External information remains necessary, because in-span updates cannot create directions absent from the current reduced subspace and cannot rescue a ROM whose trajectory is already a poor guide to the local dynamics. The correction interval and forgetting factors are also decided before the simulation and remain fixed during the online prediction, whereas future methods could choose when to request out-of-span information and how strongly to forget based on online indicators such as residual growth or spectral drift. Higher-dimensional flows, multiphysics systems, and noisy or partial observations remain important research directions to investigate.
We can also draw a connection between in-span learning and in-context learning in large language models [55, 56, 57]. In the linear-attention and fast-weight view of sequence models [58, 59], this correspondence can be made mathematically explicit. Specifically, attention layers maintain a state matrix updated by rank-one outer products of internally generated representations, often discounted by a gate, and use that state to adapt their outputs without changing trained weights. The in-span update has the same basic skeleton. It accumulates a state from outer products of the model’s own reduced coordinates, discounts that state by , and uses it to reorganize a fixed representation. The shared structure is test-time adaptation through a decaying outer-product accumulator over self-generated representations. Adaptive ROMs may therefore provide a low-rank dynamical-systems setting in which to study how a fixed representation is reorganized by its own context. A detailed mathematical analysis linking in-span learning to in-context learning is provided in Section F.
More broadly, in-span learning extends beyond linear ROMs. It suggests that linear ROMs, nonlinear manifold models, and scientific machine-learning surrogates in general may all carry useful self-generated information in their evolving trajectories, creating an unexploited route toward more adaptive computational models.
Methods
Full-order models and time discretization
We consider three dynamical systems. The first is a closed-form spiral in , providing a detailed and transparent view of the mechanism. The second and third are nonlinear partial differential equations, namely viscous Burgers dynamics and Fisher–KPP reaction–diffusion, used to test the same mechanism in more practical settings.
At the fully discrete level, the full-order model is written as a nonlinear time-stepping problem for a state at time ,
where denotes the residual of the chosen time discretization. For the PDE benchmarks, we use implicit backward Euler,
where is the semi-discrete spatial operator. The nonlinear system (5) is solved by Newton iterations in the full-order model (FOM) and by a sampled least-squares problem in the ROM.
The spiral test case is defined by the exact trajectory
This trajectory solves the affine linear system
For time step , the exact discrete map is
The simulation begins at . Because Eq. (8) is exact, the spiral experiment contains no time-integration error. All discrepancies between the FOM and ROM are therefore caused by the reduced representation and its updates.
The Burgers benchmark follows the one-dimensional viscous Burgers equation
with periodic boundary conditions and viscosity . The spatial domain is discretized with uniform grid points. The convective term is approximated by a first-order upwind finite-difference scheme and the diffusive term by a centred second-order method. Time integration uses backward Euler with over a horizon of 500 time steps to propagate a Gaussian initial condition.
The Fisher–KPP benchmark is
with periodic boundary conditions, diffusion coefficient , and reaction rate . The spatial domain is discretized with uniform grid points using centred finite differences. Time integration again uses backward Euler with over 500 time steps. Starting from a localized pulse, the solution develops travelling fronts and then saturates toward over much of the domain.
Reduced-order representation
All ROMs use an affine, scaled trial representation of the full-order state,
where is a reference state, is a nonsingular diagonal scaling matrix, is an orthonormal basis, and is the reduced coordinate vector. The rank satisfies .
The initial basis is constructed from a short training snapshot matrix
where is the number of training snapshots. After scaling, we compute
and set equal to the first left singular vectors. In all PDE experiments reported in the main text, the offline window contains only four FOM snapshots and the ROM rank is . The static ROM therefore represents a deliberately minimal offline model, while the adaptive ROMs are tested in a genuinely extrapolative setting.
For the spiral, the dynamics are linear and the discrete flow is known exactly, so we use a simple Galerkin projection. The initial rank-2 basis is obtained from the two snapshots and . The reduced state satisfies
with initial coordinate . The corresponding ROM state is
Note that in the spiral results, we take a zero reference state and an identity scaling matrix for simplicity. Since the basis is rank two, the ROM trajectory remains in the initial plane
For Burgers and Fisher–KPP, since the full-order models utilize implicit temporal integration, we use a least-squares Petrov–Galerkin (LSPG) [54, 19] formulation at the fully discrete level.
Substituting the reduced ansatz (11) into the FOM residual gives the reduced least-squares problem
Here is the previous ROM state. This LSPG formulation minimizes the fully discrete residual and is well-suited to implicit time stepping.
To reduce the online cost, we use QDEIM hyper-reduction [52, 53]. Let be a sampling matrix that selects residual entries, with . The sampled LSPG problem is
The sample indices are chosen by QDEIM through a column-pivoted QR factorization of . In the PDE experiments, we use sample points.
Information channels in an adaptive ROM
The adaptive ROM receives two kinds of snapshots, distinguished by their relationship to the current reduced subspace.
Definition 1 (Out-of-span and in-span snapshots). Let be the current orthonormal basis, and let be a preprocessed snapshot passed to a basis update. Define the projection coefficient, residual, and residual norm by
The snapshot is in-span if , equivalently . It is out-of-span if .
In the algorithms studied here, out-of-span snapshots are generated by occasional full-order operator queries. These snapshots may contain components not represented by the current basis and can therefore move the trial subspace. In-span snapshots are the ROM’s own predicted states. They satisfy by construction and therefore cannot introduce a new direction. The next Methods subsections show that, despite this zero residual, in-span snapshots still update the spectral state of the basis through the singular values and in-plane rotations of an iSVD update.
iSVD update with forgetting
The online basis update used throughout the paper is an iSVD with forgetting. The update maintains a rank- approximation of an exponentially weighted snapshot history.
Definition 2 (Truncated iSVD update with forgetting [43, 44, 45]). Let be an orthonormal basis, let contain the retained singular values associated with , and let be a preprocessed incoming snapshot. Define the projection coefficient, residual, and residual norm as in Eq. (19). If , set . For a forgetting factor , form the core matrix
Let
be its singular-value decomposition, with singular values in non-increasing order, obtained from the eigendecomposition of the full core covariance
The rank- update is
If , the residual direction is not defined and the update is interpreted through the reduced core described later in Statement 3.
The update in Statement 2 is equivalent to appending the new snapshot to a weighted history matrix. Let denote the exponentially weighted data matrix containing the snapshots assimilated up to update step ,
The current rank- representation approximates this weighted data matrix by . Receiving a new snapshot gives the augmented weighted history
The small matrix in Eq. (20) contains all information needed to update the left singular vectors of this augmented matrix. The forgetting factor discounts previous information before the new snapshot is added, and acts as a memory valve in this sense.
Equivalently, the update tracks the dominant eigenspace of an exponentially weighted second-moment matrix. If denotes the weighted second-moment matrix in the ambient coordinates, then the weighted-history update gives
In-span learning preserves the subspace and reorganizes the spectrum
We now specialize Statement 2 to the case in which the incoming snapshot is a ROM prediction. In scaled coordinates, such a snapshot has the form
Hence, according to Eq. (19), , , and . No new ambient direction can be added. Nevertheless, the singular values and the coordinate frame inside the subspace can change.
Theorem 3 (Subspace-preserving spectral preconditioning by in-span learning). Let be orthonormal, let , and let be an in-span snapshot. Applying the iSVD update with forgetting factor gives
where is orthogonal and
Consequently,
The singular values obey the rank-one interlacing inequalities
Proof. For an in-span snapshot, . Therefore and . The core matrix in Eq. (20) has zero last row,
Its nonzero singular values are the singular values of the reduced matrix
Since
the left singular vectors of are the eigenvectors of the reduced covariance in Eq. (27). The updated basis is therefore for an orthogonal matrix , which proves subspace preservation. The interlacing inequalities follow from Cauchy’s interlacing theorem for a positive semidefinite rank-one perturbation of the diagonal matrix .
After consecutive in-span updates with reduced coordinates , the reduced covariance is
This is the covariance recursion used in the spiral example. It shows explicitly how the ROM trajectory reshapes the spectrum. Directions aligned with recent coefficient vectors can receive repeated rank-one reinforcement, while directions receiving little trajectory energy can be suppressed by forgetting. Here, we say that a direction is reinforced if its post-update directional energy exceeds its pre-update value, and suppressed if its post-update directional energy is smaller.
Definition 4 (Directional reinforcement and suppression). Let be a unit direction in the current, pre-update reduced coordinates. For an in-span update with reduced coordinate , define the pre-update and post-update directional energies by
We say that the fixed pre-update direction is reinforced if and suppressed if .
Using Eq. (34),
Therefore the fixed pre-update direction is reinforced if
and suppressed if the inequality is reversed. In particular, for a pre-update singular-vector direction ,
so that this pre-update direction is reinforced when
and suppressed when falls below this threshold.
This analysis also shows that in-span spectral preconditioning does not always appear as suppression of the weakest mode. If the ROM trajectory has enough projected energy in a direction to exceed the threshold, that direction is reinforced even under forgetting. If the projected energy falls below the threshold, the direction decays. The spiral example with aggressive forgetting realizes the latter case for one nearly inactive direction, whereas the PDE examples show richer regimes with less forgetting in which several retained directions are reinforced while the basis still rotates and reweights.
Full-order operator query
The out-of-span channel requires occasional information that is not constrained to the current reduced subspace. In this work, that information is supplied by a full-order operator query. The idea is to let the ROM advance the system most of the time, and to call the full-order model only at selected correction events.
Suppose an out-of-span event is triggered at time step . Before the correction, the ROM has produced the state
We then initialize one full-order step from this ROM state and solve
The resulting state is used as the external correction snapshot. In scaled coordinates,
This snapshot is then passed to the out-of-span iSVD update with forgetting factor .
This correction is different from a coarse full-order lookahead trajectory previously used in [51]. A coarse-FOM oracle advances an auxiliary full-order state on a larger time step and uses that state as the correction signal. The full-order operator query used here instead takes a single fine-grid full-order step from the current ROM state. We use it because it gives a simple hybrid model, where the reduced model carries the trajectory most of the time and occasionally interacts with the full-order operator to receive a local out-of-span correction snapshot.
For the spiral example, Eq. (40) reduces to one exact full-order map applied to the ROM state,
For the PDE benchmarks, Eq. (40) is solved with the same backward-Euler residual used by the FOM, but only at the correction times.
SPIN algorithm
The complete adaptive procedure (Algorithm 1) alternates between three operations. First, the ROM advances one time step using the current basis and sample set. Second, if the step is not an external correction time, the ROM-predicted state is used for an in-span update with forgetting factor . Third, every steps, a full-order operator query is performed, and the produced snapshot is used for an out-of-span update with forgetting factor .
After any basis update, basis-dependent quantities must be refreshed. For the LSPG–QDEIM ROMs, this includes the sampled basis and its pseudoinverse. After an out-of-span update, the QDEIM sample indices are recomputed. After an in-span update, the subspace is unchanged but the basis vectors have rotated inside it, so the sampled operators depending on the basis are refreshed. The reduced coordinates are also transferred to the updated basis by projecting the current reconstructed ROM state onto the new basis.
Algorithm 1 SPIN ROM algorithm
Require: Initial basis , singular values , reference state , scaling matrix , sample matrix , reduced state , update interval , forgetting factors and , final step
for do
Advance the sampled ROM one step:
Reconstruct the ROM state:
if then — out-of-span adaptation
Perform one full-order operator query:
Preprocess the correction:
Update the basis and singular values by out-of-span iSVD:
Recompute QDEIM sample matrix and sampled operators.
else — in-span adaptation
Preprocess the ROM prediction:
Update the basis and singular values by in-span iSVD:
Refresh sampled operators.
end if
Transfer the current state to the updated basis:
end for
The baseline adaptive ROM used for comparison can be obtained from Algorithm 1 by omitting the in-span update in the else branch. The static ROM is obtained by omitting both in-span and out-of-span updates. The implementation used in this work is available at https://github.com/APHedayat/SPIN.
Metrics and diagnostics
We use two groups of diagnostics. The first group measures how a single out-of-span correction is absorbed by a fixed-rank update. Let and denote the orthonormal bases immediately before and after the correction update, and let be the correction snapshot. The out-of-span residual relative to the pre-update subspace is
The residual capture is
which measures how much of the new residual direction lies in the post-update subspace. The plane-change angle is the largest principal angle between the pre- and post-update subspaces,
where are singular values. The correction error is
This measures the part of the correction snapshot not represented by the updated subspace. The second group of diagnostics measures how the adaptive basis, spectrum, and subspace evolve over the online trajectory. The first trajectory diagnostic is the singular-value history,
These values measure how the adaptive algorithm weights the retained modes. In the baseline adaptive ROM, they change only at correction events. In the SPIN ROM, they may change at every time step. The second trajectory diagnostic measures basis reorientation between consecutive steps,
This quantity is sensitive to changes in the orthonormal frame. It can be nonzero even when the two bases span the same subspace. The third trajectory diagnostic measures genuine subspace change. Let the singular values of
be , with , where are the principal angles between the two subspaces. We report
This diagnostic is zero when the subspace is unchanged, even if the basis has rotated inside that subspace. The combination of nonzero and zero is therefore the numerical signature of a pure in-span update.
Computational complexity
The additional cost of in-span learning is small compared with the cost of obtaining out-of-span information from the full-order operator. Let be the full-order dimension, the ROM rank, and the number of sampled residual entries, with , .
We first define the cost of one hyper-reduced LSPG time step. A sampled LSPG step solves a nonlinear least-squares problem in unknowns using only sampled residual entries. Let denote the cost of evaluating the sampled residual and sampled Jacobian contributions required
Appendices
by one Newton iteration. Once these sampled quantities are available, the dense reduced linear algebra scales as
per Newton iteration. Thus, between correction events, the online simulation remains reduced and sampled.
The cost of an iSVD basis update is roughly similar for in-span and out-of-span snapshots. For a general out-of-span snapshot, the update projects the snapshot onto the current basis, forms the orthogonal residual, computes the SVD of a small core matrix, and rotates the ambient basis. The dominant scaling is
For an in-span snapshot, the orthogonal residual is zero and the update can be performed through the reduced covariance in Eq. (27). This avoids the residual construction, but the small eigendecomposition and the ambient basis rotation remain. Therefore, the in-span update costs as
The in-span update can be cheaper in implementations that delay the ambient rotation and accumulate the transformation in reduced coordinates, but in the implementation considered for PDE cases the leading order is the same as for an out-of-span iSVD update.
Basis changes may also require refreshing hyper-reduction quantities. We denote this cost by . If QDEIM sampling points are recomputed from a pivoted QR factorization of , then
In-span updates preserve the trial subspace, so one may keep the same sampling set between out-of-span corrections and update only the sampled basis-dependent quantities. If the sampling points are recomputed after every basis rotation, then the same cost applies.
The bottleneck in the adaptive procedure is the full-order operator query used to obtain an out-of-span correction. For implicit PDE solvers, this query requires nonlinear iterations, residual and Jacobian evaluations, and large linear solves. We write its cost as
where depends on the discretization, nonlinear solver, linear solver, preconditioner, and sparsity structure. This term is typically much larger than the reduced dense linear algebra when is large and .
With these ingredients, the cost of one in-span update is
where denotes any update of sampled basis-dependent quantities performed after the in-span rotation. The cost of one out-of-span correction event is
where denotes the remaining basis-dependent reduced-operator or sampled-operator updates.
The key comparison is that the SPIN ROM uses the same out-of-span correction schedule as the baseline adaptive ROM. It therefore does not add any terms. If the correction interval is , then between two out-of-span corrections the SPIN model adds
relative to the baseline adaptive model, but it does not change the number of full-order operator queries. In the large-scale implicit PDE regime, is the dominant cost, whereas the in-span operations are reduced or low-rank dense linear algebra.
Data availability
The data supporting the findings of this work are available from the corresponding author upon request.
Code availability
The code used to generate the numerical results in this work is available at https://github.com/APHedayat/SPIN.
Acknowledgements
A.H. and K.D. were supported by the Office of Under Secretary of Defense for Research and Engineering (OUSD(RE)) grant N00014-21-1-295. L.B. acknowledges NSF award CCF-2331590 and ARO award W911NF-26-1-A219.
Author contributions
A.H.: Methodology, Software, Investigation, Data Curation, Writing - Original Draft, Visualization. L.B.: Conceptualization, Methodology, Writing - Review & Editing, Supervision. K.D.: Conceptualization, Methodology, Investigation, Writing - Review & Editing, Supervision, Funding acquisition.
Competing interests
The authors declare no competing interests.
Appendices
The following appendices contain the Supplementary Information accompanying the manuscript. They provide notation, full numerical details for the spiral example, comparisons with alternative fixed-rank updates, hyperparameter sensitivity studies for the nonlinear PDE benchmarks, reproducibility information, and a mathematical connection between in-span learning and in-context learning supporting the main text.
Notation
Table A1 summarizes the main symbols used in the paper.
| Symbol | Meaning |
| Full-order dimension | |
| ROM rank | |
| Number of hyper-reduction sample points | |
| Number of online prediction time steps | |
| Full-order state at time step | |
| ROM-reconstructed state at time step | |
| Correction snapshot obtained from a full-order operator query | |
| Reference state used in the affine ROM representation | |
| Diagonal scaling matrix | |
| Orthonormal reduced basis | |
| Initial reduced basis | |
| Reduced bases immediately before and after a correction update | |
| Retained singular values associated with | |
| Reduced coordinate vector at time step | |
| Hyper-reduction sampling matrix | |
| Number of ROM time steps between out-of-span corrections | |
| Forgetting factor for in-span updates | |
| Forgetting factor for out-of-span updates | |
| Preprocessed snapshot passed to iSVD | |
| In-plane coefficients of | |
| Orthogonal residual of with respect to | |
| Residual norm | |
| Small iSVD core matrix | |
| Exponentially weighted snapshot-history matrix after updates | |
| Exponentially weighted second-moment matrix in ambient coordinates | |
| Reduced covariance represented in reduced coordinates | |
| Reduced covariance after consecutive in-span updates | |
| In-plane block of the iSVD core covariance used to diagnose local rank competition | |
| Residual-to-weakest-mode ratio | |
| Residual capture by the post-update basis | |
| Largest principal angle between the pre- and post-update subspaces | |
| Correction error after projection onto the post-update subspace | |
| Basis-reorientation diagnostic between steps and | |
| Genuine subspace-motion diagnostic between steps and | |
| Principal angles between two subspaces | |
| Singular values of a matrix argument, used in principal-angle diagnostics | |
| Cost of one full-order operator query | |
| Costs of one in-span update and one out-of-span correction event |
Table A1. Notation used in the main text and appendices.
Full numerical details for the spiral example
The spiral trajectory is
The time step is and the initial time is . The exact discrete dynamics are
with
The initial rank-2 basis is obtained from the two snapshots and :
The associated singular values are
With , four in-span updates give the preconditioned basis
and singular values
The subspace is unchanged:
Full-order operator query
The out-of-span correction is generated from the ROM state,
The residual relative to the initial ROM plane is
with
The correction coefficients in the two pre-update coordinate systems are
Rank-2 competition
The competition matrices are
and
Their weakest eigenvalues are
Thus
Post-update bases
Applying the same truncated iSVD update (with for simplicity) to both pre-update states gives
and
We define the residual capture as , where , the plane angle as the largest principal angle , where are singular values, the retained alignment as , where is the th column of , and the correction error as . The measured outcomes are listed in Table B1.
| Method | Residual capture | Plane angle | Retained alignment | Correction error |
| Baseline adaptive | 0.260 | 0.249 | ||
| SPIN | 0.802 | 0.797 |
Table B1. Main spiral diagnostics for baseline adaptive and SPIN updates.
Core-matrix interpretation
The same effect can be read directly from the small iSVD core matrix. For a pre-update basis and singular values , write
The out-of-span iSVD update forms the augmented basis and the core matrix
Here we use . The left singular vectors of determine how the retained post-update basis vectors mix the old pre-update modes with the new residual direction. Because the update is truncated back to rank two, the first retained vector corresponds to the dominant slot, while the second retained vector is the weak slot where the old weak mode competes with the incoming residual.
For the baseline adaptive model, and , giving
The second retained left singular vector is
The entries correspond, respectively, to the old first mode, the old second mode, and the new residual direction. Thus the weak retained slot is still dominated by the old second mode, with magnitude 0.929, while only 0.249 of the slot points along the residual direction.
For the SPIN model, and , giving
The corresponding retained core vector is
Now the same weak slot is much more strongly aligned with the residual direction: the residual contribution has magnitude 0.797, while the old second-mode contribution has dropped to 0.594. The signs are arbitrary because singular vectors are sign-indeterminate; the magnitudes show how the fixed rank-two budget is allocated. In this sense, in-span preconditioning does not add a third basis vector. It repurposes the weak retained slot so that the same out-of-span correction survives the truncation more effectively.
Alternative fixed-rank updates in the spiral example
The spiral also allows comparison with several alternative fixed-rank update rules. All methods receive the same correction and use the same rank budget.
Rank-one interpolation
The rank-one interpolation update of Ref. [38] takes the form
This update is constructed so that
Therefore the correction lies exactly in the updated span. However, exact interpolation of one state does not imply that the updated basis is well aligned with the new residual direction.
Greedy residual replacement
A greedy residual replacement keeps the dominant old mode and replaces the weak mode by the normalized residual:
This achieves perfect residual alignment, but discards the in-plane contribution associated with the second old mode. In the spiral, the correction error is therefore
Optimal one-slot rotation
If the dominant first mode is fixed and only the weak second slot is allowed to rotate in the plane spanned by and , the exact-correction update is
This update makes the correction exact while keeping fixed. Its residual alignment is
The comparison clarifies the role of in-span learning. It does not optimize correction interpolation alone, nor residual alignment alone. Instead, it brings a trajectory-preconditioned basis into the same truncated update, which allows the weak modal slot to be repurposed without discarding essential in-plane content. To compare the alternatives, we use the same residual capture, plane angle, retained alignment, and correction-error diagnostics defined above. We also report the slot angle
where is the pre-update weak mode and is the post-update vector assigned to that weak modal slot. The absolute value removes the arbitrary sign of basis vectors. A small slot angle means that the weak slot remains close to its old direction, whereas a large slot angle indicates that the slot has been substantially repurposed. The resulting comparison is given in Table C1 and visualized in Figure C1.
| Method | Capture | Plane angle | Slot angle | Retained alignment | Correction error |
| Baseline adaptive | 0.260 | 15.06 | 21.74 | 0.249 | |
| SPIN | 0.802 | 53.29 | 53.53 | 0.797 | |
| Rank-one interpolation | 0.162 | 9.30 | 5.43 | 0.132 | 0 |
| Greedy replacement | 1.000 | 90.00 | 90.00 | 1.000 | 1.491 |
| Optimal slot rotation | 0.269 | 15.63 | 15.63 | 0.269 | 0 |
Table C1. Comparison of fixed-rank updates on the spiral correction.
Figure C1. Alternative fixed-rank updates expose the tradeoff between fitting the correction and retaining the residual. Rank-one interpolation and optimal one-slot rotation can fit the correction exactly, but do not strongly align the basis with the new residual direction. Greedy residual replacement captures the residual perfectly, but discards important in-plane content. The SPIN update occupies a different part of the tradeoff because the basis has already been organized by the ROM trajectory before the correction arrives.
Hyperparameter sensitivity and additional PDE results
The SPIN framework is governed by three main parameters: the in-span forgetting factor , the out-of-span forgetting factor , and the out-of-span correction interval . For each benchmark, we tune the forgetting factors by minimizing the mean relative error over the online trajectory. The baseline adaptive is tuned over its own values, and the SPIN model is tuned over the two-dimensional grid (Figure D1 and Figure D2). The main-text comparisons therefore use the best-performing version of each adaptive strategy.
Figure D1. Sensitivity to in-span forgetting and correction interval. When is close to one, in-span forgetting is weak and the ROM retains a longer memory of its own trajectory; when is small, the in-span update emphasizes recent states more aggressively. The most useful regime balances trajectory memory, spectral reweighting and stability between correction events. In the maps (panels c and f), the reference value reported in the title is the best baseline adaptive error obtained by sweeping its out-of-span forgetting factor (Figure D2).
(a)
(b)
Figure D2. Out-of-span forgetting for the baseline adaptive model. The baseline adaptive model’s mean relative error as a function of its out-of-span forgetting factor (swept on a logarithmic axis), with the SPIN error shown as a dashed reference line. The optimal out-of-span forgetting is problem-dependent—near for Burgers and for Fisher–KPP—but in both cases the best-tuned baseline adaptive ROM remains substantially less accurate than the SPIN ROM.
For Burgers, the best baseline adaptive model uses , while the best SPIN model uses . For Fisher–KPP, the best baseline adaptive model uses , while the best SPIN model uses . These values are reported in the main text and repeated in Table E1.
| Benchmark | Parameter | Value | Notes |
| Spiral | 0.4 | Spiral vertical drift | |
| Spiral | 0.35 | Exact discrete map | |
| Spiral | 4.0 | Initial time | |
| Spiral | 2 | Rank-2 ROM | |
| Spiral | 0.1 | In-span forgetting | |
| Burgers | 1000 | Periodic finite-difference grid | |
| Burgers | Viscosity | ||
| Burgers | Backward Euler | ||
| Burgers | 500 | Prediction steps | |
| Burgers | 4 | ROM rank | |
| Burgers | 4 | QDEIM samples | |
| Burgers | 10 | Correction interval | |
| Burgers | 1.0, 0.25 | Forgetting factors (SPIN) | |
| Burgers | 0.01 | Forgetting factor (baseline adaptive) | |
| Fisher–KPP | 256 | Periodic finite-difference grid | |
| Fisher–KPP | Diffusion coefficient | ||
| Fisher–KPP | 20 | Reaction rate | |
| Fisher–KPP | Backward Euler | ||
| Fisher–KPP | 500 | Prediction steps | |
| Fisher–KPP | 4 | ROM rank | |
| Fisher–KPP | 4 | QDEIM samples | |
| Fisher–KPP | 25 | Correction interval | |
| Fisher–KPP | 0.9, 0.25 | Forgetting factors (SPIN) | |
| Fisher–KPP | 0.1 | Forgetting factor (baseline adaptive) |
Table E1. Main benchmark parameters.
The sensitivity maps also clarify the role of in-span forgetting. The best SPIN models occur at large , meaning that the PDE benchmarks do not benefit from aggressive in-span forgetting. This is consistent with the spectral diagnostics in the main text: unlike the first spiral correction, the PDE trajectories carry enough energy in the retained modes that the singular values are reinforced rather than suppressed. The role of in-span learning in these cases is therefore not to erase a stale mode, but to continuously condition the basis spectrum and orientation using the ROM trajectory.
The maps show a useful practical pattern. When is large, the model can become sensitive to the out-of-span forgetting factor , and poor choices of may lead to much larger errors. When is smaller, the performance is often more robust across , although not always optimal. Thus, aggressive in-span memory can produce the best models when paired with a suitable out-of-span forgetting factor, while smaller may provide a safer but more conservative operating regime.
The correction interval controls how long the ROM evolves between external corrections. Larger reduces the number of full-order operator queries, but gives the ROM more time to drift; smaller provides more frequent out-of-span information but increases the hybrid cost. Figure D1 shows that adding the in-span channel shifts this tradeoff. For a fixed target error, the SPIN ROM can use a substantially larger than the baseline adaptive ROM, meaning that it can evolve for longer on its own before requiring new external information. This supports the central interpretation of in-span learning as trajectory-informed basis preparation: between correction events, the ROM uses its own predictions to keep the basis better organized, so external corrections can be queried less frequently for the same accuracy level. The interval cannot be made arbitrarily large, however, because if the ROM trajectory drifts too far, its in-span predictions no longer provide a reliable guide to the local dynamics.
Figure D3 reports additional Burgers and Fisher–KPP diagnostics, including robustness across initial conditions and final-time profiles for representative cases.

Figure D3. Additional PDE results support the main-text comparisons. Initial-condition robustness tests for Burgers and Fisher–KPP. All ROMs are trained with rank and QDEIM sample points, with offline basis trained on the initial 4 time steps from to , while prediction is evaluated over 500 time steps from to , with external corrections every steps. Across these additional cases, the SPIN ROM uses the same out-of-span correction mechanism as the baseline adaptive ROM, but produces smaller errors and more accurate final-time profiles.
Reproducibility information
Reproducibility information
The main numerical settings used to generate the spiral, Burgers, and Fisher–KPP results are collected in Table E1. Together with the model definitions, update rules, and diagnostics described in the Methods and appendices above, as well as the source code available at https://github.com/APHedayat/SPIN, these parameters specify the benchmark configurations used for all results reported in the main text and appendices.
Mathematical connection between in-span and in-context learning
In-context learning (ICL) broadly refers to the ability of a model to modify its inference-time behavior using information contained in the current context, without retraining the underlying offline model [55, 56, 57]. Although most commonly associated with large language models, several analyses have shown that the mechanism can be understood more generally as an inference-time computation performed over a sequence of contextual representations. In particular, linearized attention admits a recurrent fast-weight interpretation in which context is accumulated through rank-one outer-product updates [58, 59]. We show here that the in-span update underlying SPIN has the same mathematical structure.
Outer-product memory in linear attention
Consider a sequence of representations . In a linear-attention layer, each representation generates a key, value, and query,
where denotes the feature map used to linearize attention. Ignoring normalization, the causal attention computation can be written in terms of a recurrent context matrix
with the corresponding output obtained by querying this accumulated state,
Thus, rather than explicitly retaining the entire preceding sequence, the model can carry forward a finite-dimensional state containing a sum of outer products generated from the preceding context.
For comparison with the forgetting mechanism used by SPIN, consider the exponentially weighted form
which gives
after applying representations starting from . The choice recovers the purely additive fast-weight accumulation in Eq. (F.2); introducing simply gives greater weight to recent context.
The in-span state is a symmetric fast-weight memory
Between external corrections, SPIN receives only states generated by the ROM itself. Let denote the reduced coordinates of the th in-span prediction. As derived in the main text, the reduced covariance evolves according to
and therefore
Comparing Eq. (F.4) and Eq. (F.6) shows an exact correspondence under
Thus, the SPIN covariance is a symmetric fast-weight memory whose context consists of the model’s own reduced trajectory. The initial state plays the role of a prior spectral memory inherited from the offline model, while each new prediction contributes the rank-one contextual update . Forgetting determines how rapidly earlier context is discounted.
In this sense, in-span learning can be viewed as a form of self-generated in-context learning for dynamical systems, where the surrogate uses the context created by its own evolving trajectory to reorganize its internal representation and thereby change how it responds to future information.
References
- [1]Lyu, Z., Kenway, G. K. W. & Martins, J. R. R. A. Aerodynamic shape optimization investigations of the Common Research Model wing benchmark. AIAA J. 53, 968–985 (2015).DOI
- [2]Lam, R. et al. Learning skillful medium-range global weather forecasting. Science 382, 1416–1421 (2023).
- [3]Kochkov, D. et al. Neural general circulation models for weather and climate. Nature 632, 1060–1066 (2024).DOI
- [4]Bodnar, C. et al. A foundation model for the Earth system. Nature 641, 1180–1187 (2025).
- [5]Swischuk, R., Kramer, B., Huang, C. & Willcox, K. Learning physics-based reduced-order models for a single-injector combustion process. AIAA J. 58, 2658–2672 (2020).
- [6]Khaloie, H. et al. Review of machine learning techniques for optimal power flow. Appl. Energy 388, 125637 (2025).
- [7]Xiu, D. & Karniadakis, G. E. Modeling uncertainty in flow simulations via generalized polynomial chaos. J. Comput. Phys. 187, 137–167 (2003).
- [8]Degen, D., Cacace, M. & Wellmann, F. 3D multi-physics uncertainty quantification using physics-based machine learning. Sci. Rep. 12, 17491 (2022).
- [9]Niederer, S. A., Sacks, M. S., Girolami, M. & Willcox, K. Scaling digital twins from the artisanal to the industrial. Nat. Comput. Sci. 1, 313–320 (2021).
- [10]Kapteyn, M. G., Pretorius, J. V. R. & Willcox, K. E. A probabilistic graphical model foundation for enabling predictive digital twins at scale. Nat. Comput. Sci. 1, 337–347 (2021).
- [11]Antoulas, A. C. Approximation of Large-Scale Dynamical Systems (SIAM, Philadelphia, 2005).
- [12]Rowley, C. W. & Dawson, S. T. M. Model reduction for flow analysis and control. Annu. Rev. Fluid Mech. 49, 387–417 (2017).
- [13]Antoulas, A. C., Beattie, C. A. & Gügercin, S. Interpolatory Methods for Model Reduction (SIAM, Philadelphia, 2020).
- [14]Kramer, B., Peherstorfer, B. & Willcox, K. Learning nonlinear reduced models from data with operator inference. Annu. Rev. Fluid Mech. 56, 521–548 (2024).
- [15]Lumley, J. L. The structure of inhomogeneous turbulent flows. In Yaglom, A. M. & Tatarski, V. I. (eds.) Atmospheric Turbulence and Radio Wave Propagation, 166–178 (Nauka, Moscow, 1967).
- [16]Sirovich, L. Turbulence and the dynamics of coherent structures. I–III. Quart. Appl. Math. 45, 561–590 (1987).
- [17]Holmes, P., Lumley, J. L. & Berkooz, G. Turbulence, Coherent Structures, Dynamical Systems and Symmetry (Cambridge Univ. Press, Cambridge, 1996).
- [18]Benner, P., Gugercin, S. & Willcox, K. A survey of projection-based model reduction methods for parametric dynamical systems. SIAM Review 57, 483–531 (2015).
- [19]Carlberg, K., Barone, M. & Antil, H. Galerkin v. least-squares Petrov–Galerkin projection in nonlinear model reduction. J. Comput. Phys. 330, 693–734 (2017).
- [21]Huang, C., Duraisamy, K. & Merkle, C. L. Component-based reduced order modeling of large-scale complex systems. Front. Phys. 10, 900064 (2022).
- [22]Lee, K. & Carlberg, K. T. Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders. J. Comput. Phys. 404, 108973 (2020).
- [23]Geelen, R., Wright, S. & Willcox, K. Operator inference for non-intrusive model reduction with quadratic manifolds. Comput. Methods Appl. Mech. Eng. 403, 115717 (2023).
- [24]Peherstorfer, B. Breaking the Kolmogorov barrier with nonlinear model reduction. Not. Am. Math. Soc. 69, 725–733 (2022).
- [25]Barnett, J. & Farhat, C. Quadratic approximation manifold for mitigating the Kolmogorov barrier in nonlinear projection-based model order reduction. J. Comput. Phys. 464, 111348 (2022).
- [26]Reiss, J., Schulze, P., Sesterhenn, J. & Mehrmann, V. The shifted proper orthogonal decomposition: a mode decomposition for multiple transport phenomena. SIAM J. Sci. Comput. 40, A1322–A1344 (2018).
- [27]Nair, N. J. & Balajewicz, M. Transported snapshot model order reduction approach for parametric, steady-state fluid flows containing parameter-dependent shocks. Int. J. Numer. Methods Eng. 117, 1234–1262 (2019).
- [28]Rim, D., Peherstorfer, B. & Mandli, K. T. Manifold approximations via transported subspaces: model reduction for transport-dominated problems. SIAM J. Sci. Comput. 45, A170–A199 (2023).
- [29]Mirhoseini, M. A. & Zahr, M. J. Model reduction of convection-dominated partial differential equations via optimization-based implicit feature tracking. J. Comput. Phys. 473, 111739 (2023).
- [30]Arnold-Medabalimi, N., Huang, C. & Duraisamy, K. Large-eddy simulation and challenges for projection-based reduced-order modeling of a gas turbine model combustor. Int. J. Spray Combust. Dyn. 14, 153–175 (2022).
- [31]Amsallem, D. & Farhat, C. Interpolation method for adapting reduced-order models and application to aeroelasticity. AIAA J. 46, 1803–1813 (2008).
- [32]Amsallem, D. & Farhat, C. An online method for interpolating linear parametric reduced-order models. SIAM J. Sci. Comput. 33, 2169–2198 (2011).
- [33]Amsallem, D., Zahr, M. J. & Farhat, C. Nonlinear model order reduction based on local reduced-order bases. Int. J. Numer. Methods Eng. 92, 891–916 (2012).
- [34]Peherstorfer, B., Butnaru, D., Willcox, K. & Bungartz, H. J. Localized discrete empirical interpolation method. SIAM J. Sci. Comput. 36, A168–A192 (2014).
- [35]Peherstorfer, B. & Willcox, K. Dynamic data-driven reduced-order models. Comput. Methods Appl. Mech. Eng. 291, 21–41 (2015).
- [36]Peherstorfer, B. & Willcox, K. Online adaptive model reduction for nonlinear systems via low-rank updates. SIAM J. Sci. Comput. 37, A2123–A2150 (2015).
- [37]Peherstorfer, B. Model reduction for transport-dominated problems via online adaptive bases and adaptive sampling. SIAM J. Sci. Comput. 42, A2803–A2836 (2020).
- [38]Huang, C. & Duraisamy, K. Predictive reduced order modeling of chaotic multi-scale problems using adaptively sampled projections. J. Comput. Phys. 491, 112356 (2023).
- [39]Singh, R., Uy, W. I. T. & Peherstorfer, B. Lookahead data-gathering strategies for online adaptive model reduction of transport-dominated problems. Chaos 33, 113112 (2023).
- [40]Mohaghegh, A. & Huang, C. Feature-guided sampling strategy for adaptive model order reduction of convection-dominated problems. J. Comput. Phys. 545, 114468 (2026).
- [41]Hedayat, A., Padovan, A. & Duraisamy, K. Toward adaptive non-intrusive reduced-order models: design and challenges. Struct. Multidiscip. Optim. 69, 151 (2026).
- [42]Balzano, L., Chi, Y. & Lu, Y. M. Streaming PCA and subspace tracking: the missing data case. Proc. IEEE 106, 1293–1310 (2018).
- [43]Bunch, J. R. & Nielsen, C. P. Updating the singular value decomposition. Numer. Math. 31, 111–129 (1978).
- [44]Brand, M. Incremental singular value decomposition of uncertain data with missing values. In European Conference on Computer Vision (ECCV), 707–720 (Springer, 2002).
- [45]Brand, M. Fast low-rank modifications of the thin singular value decomposition. Linear Algebra Appl. 415, 20–30 (2006).
- [46]Oja, E. Simplified neuron model as a principal component analyzer. J. Math. Biol. 15, 267–273 (1982).
- [47]Balzano, L., Nowak, R. & Recht, B. Online identification and tracking of subspaces from highly incomplete information. In Proc. 48th Annu. Allerton Conf., 704–711 (2010).
- [48]Yang, B. Projection approximation subspace tracking. IEEE Trans. Signal Process. 43, 95–107 (1995).
- [49]Chi, Y., Eldar, Y. C. & Calderbank, R. PETRELS: parallel subspace estimation and tracking by recursive least squares from partial observations. IEEE Trans. Signal Process. 61, 5947–5959 (2013).
- [50]Zimmermann, R., Peherstorfer, B. & Willcox, K. Geometric subspace updates with applications to online adaptive nonlinear model reduction. SIAM J. Matrix Anal. Appl. 39, 234–261 (2018).
- [51]Hedayat, A., Mohaghegh, A., Balzano, L., Huang, C. & Duraisamy, K. History-aware adaptive reduced-order models via incremental singular value decomposition. Preprint at arXiv:2605.28684 (2026).arxiv.org/abs/2605.28684
- [52]Chaturantabut, S. & Sorensen, D. C. Nonlinear model reduction via discrete empirical interpolation. SIAM J. Sci. Comput. 32, 2737–2764 (2010).
- [53]Drmač, Z. & Gugercin, S. A new selection operator for the discrete empirical interpolation method. SIAM J. Sci. Comput. 38, A631–A648 (2016).
- [54]Carlberg, K., Bou-Mosleh, C. & Farhat, C. Efficient non-linear model reduction via a least-squares Petrov–Galerkin projection and compressive tensor approximations. Int. J. Numer. Methods Eng. 86, 155–181 (2011).
- [55]Brown, T. B. et al. Language models are few-shot learners. In Adv. Neural Inf. Process. Syst. (NeurIPS) (2020).
- [56]Xie, S. M., Raghunathan, A., Liang, P. & Ma, T. An explanation of in-context learning as implicit Bayesian inference. In Int. Conf. Learn. Represent. (ICLR) (2022).
- [57]Akyürek, E., Schuurmans, D., Andreas, J., Ma, T. & Zhou, D. What learning algorithm is in-context learning? Investigations with linear models. In Int. Conf. Learn. Represent. (ICLR) (2023).
- [58]A. Katharopoulos, A. Vyas, N. Pappas, F. Fleuret. Transformers are RNNs: fast autoregressive transformers with linear attention. ICML, (2020).
- [59]I. Schlag, K. Irie, J. Schmidhuber. Linear transformers are secretly fast weight programmers. ICML, (2021).









