WorksComputer Science
Structured Extrema Errors in Classical Surrogates for Viscous Burgers: A Physics-Consistent Interpretation
Abstract
We study the local errors of classical machine-learning surrogate models, which approximate the time evolution of the one-dimensional viscous Burgers equation. Four models are compared on the same prediction task, using the spatial grid values directly: radial basis function (RBF) kernel ridge regression (KRR), linear Ridge, ExtraTrees, and Random Forests. Across all four models, the one-step residual, defined here as the true value minus the predicted value at each grid point, forms clear curved branches near predicted maxima and minima. A more detailed analysis of KRR shows that these errors are much more strongly related to the second spatial derivative, which measures local curvature, than to the first spatial derivative. Near a smooth extremum, predicted value and curvature form a local two-branch fold. Under our local curvature-based model of the residual, this fold predicts a leading-order near-parabolic relation between predicted value and residual. This geometric result motivates a direct test of the Burgers advection (transport) and diffusion (smoothing) terms. For KRR and Ridge, regression tests on held-out trajectories, a control that breaks the spatial alignment of the diffusion term, and a spectral test of high-frequency content are consistent with insufficient viscous smoothing at moderate and high viscosity. In this case, the surrogate retains more small-scale structure than the true future state. The same physical explanation is much weaker for the tree models. Finally, a correction that uses only predicted quantities reduces both one-step error and error during recursive rollout, where each prediction is used as the next input.
Introduction
Machine-learning methods are widely used to approximate solutions and solution operators of partial differential equations (PDEs). A solution operator maps an input field or parameter to a solution field [1, 3, 4, 5]. Their accuracy is commonly reported with global measures such as mean squared error or relative error [4, 5, 6]. These measures summarize the prediction over the full domain, but they do not describe where the largest errors occur or whether those errors have a repeatable local structure. Scientific interpretability work therefore emphasizes quantitative tests and checks based on domain knowledge [13, 14, 15].
There are also well-established links between numerical error and differential operators. Modified-equation analysis describes discretization errors through additional differential terms [16, 22]. In learned PDE models, governing-equation residuals have also been used to measure and correct prediction errors [18, 19]. Here, a governing-equation residual means the mismatch obtained when a predicted solution is inserted into the PDE. These ideas lead to the question studied here: can the prediction error of a learned surrogate have a simple local structure, and can that structure be linked to terms in the governing PDE?
We study this question for the one-dimensional viscous Burgers equation, a standard benchmark in scientific machine learning [4, 6]. The comparison uses four classical models: radial basis function (RBF) kernel ridge regression (KRR) [7], linear Ridge [8], ExtraTrees [9], and Random Forests [10]. All models use the same discretized spatial grid values and viscosity as input. This gives a controlled setting for comparing their error structure.
The first observation is that the pointwise residual is strongly structured, where is the true future value and is the predicted value. When is plotted against , clear curved branches appear near predicted maxima and minima. The same general pattern is present for all four model families, although its size and orientation vary between models. A qualitative KRR example is shown in Figure 1: the main solution shape is reproduced, while some of the largest errors occur around peaks and troughs.
Figure 1. Qualitative KRR rollout example. Here, rollout means that the model is applied repeatedly and each prediction is used as the next input. The prediction follows the main shape of the reference solution, while some of the largest visible errors occur near peaks and troughs. This figure is used only as motivation. The quantitative results use the fixed protocol described in § 3 and Appendix § 7.
To understand why these branches appear, we first model the local discrepancy using three simple components: amplitude change, spatial shift, and smoothing. The model uses a translation operator, which shifts a field in space, and a heat operator, which represents diffusion and smoothing [20, 21]. These components are also consistent with standard descriptions of spatial-shift and smoothing errors in numerical analysis [16, 22]. To first order, meaning that we keep only terms linear in the small changes, we show that the residual is a combination of the predicted value, its first derivative, and its second derivative. The derivation is given in Appendix § 8.
This first-order model points directly to extrema. At a smooth interior extremum, the first spatial derivative is zero by the standard first-order condition for differentiable functions [28]. Our experiments show that the second-derivative term, which measures local curvature, explains much more of the KRR residual than the first-derivative term. This motivates a closer study of the local geometry. Using Taylor’s theorem [28], we show that predicted value and curvature form a local fold: the two sides of a smooth extremum form two branches in the value–curvature relation. When this fold is combined with the local curvature-based residual model, it predicts a leading-order near-parabolic relation between predicted value and residual. The full derivation is given in Appendix § 8.
Curvature also has a direct physical role in the Burgers equation because the viscous diffusion term is proportional to the second spatial derivative [22]. This connection motivates the next test. We compare the observed residual with the Burgers advection and diffusion terms using regression evaluated on held-out trajectories. We also use a spatial negative control: the diffusion field is shifted so that its values are no longer aligned with the residual at the same grid points. For KRR and Ridge, the correctly aligned diffusion term explains a large part of the residual near extrema, while the advection contribution is much smaller. The same diffusion attribution is much weaker for ExtraTrees and Random Forests.
We test the diffusion interpretation in a second way. First, we define an effective-diffusion diagnostic: under a local heat model, it measures the amount of diffusion that would reproduce the surrogate’s smoothing. Second, we use a spectral measure of small-scale content, based on Fourier coefficients. The heat-model relation follows from the standard heat semigroup [21], while the spectral measure follows from Fourier differentiation and Parseval’s identity [23]. At moderate and high viscosity, both tests are consistent with insufficient viscous smoothing for KRR and Ridge. In this case, the surrogate retains more small-scale structure than the true future state. The weaker tree-model attribution shows that similar residual geometry can arise from different error mechanisms.
The final question is whether the identified structure contains useful information. An oracle experiment, meaning a test that is allowed to use the true residual profile, first measures how much local residual energy can be removed. We then train a correction that uses only quantities computed from the predicted state and does not use the true residual at test time. The correction is tested on held-out one-step predictions and during recursive rollout, where each prediction is used as the next input. The main contributions are:
We show that large residuals are concentrated near predicted extrema and that structured extrema-local branches occur across four classical surrogate families.
For KRR, we show that curvature is the main first-order quantity associated with these residuals. A local fold analysis then explains why a near-parabolic structure can appear between predicted value and residual under the curvature-based residual model.
For KRR and Ridge, regression against the Burgers terms on held-out trajectories, a spatial negative control, effective-diffusion estimates, and spectral measurements are consistent with insufficient viscous smoothing at moderate and high viscosity. The same explanation is much weaker for the tree models.
A correction learned only from predicted quantities reduces KRR error in both one-step prediction and recursive rollout.
Related Work
Machine learning has been widely used to approximate solutions and solution operators of partial differential equations (PDEs), through physics-informed neural networks [1, 2] and operator-learning methods such as DeepONet [3] and Fourier neural operators [4, 5]. The viscous Burgers equation is a common benchmark in this literature [4, 6]. Kernel-based methods have also been studied for learning PDE solution maps and operators [11, 12]. We use classical raw-grid surrogates as controlled models for studying structured prediction errors.
Our work is also related to scientific interpretability and model-error analysis. Scientific explanations should be tested quantitatively and checked against domain knowledge [13, 14, 15]. Modified-equation analysis connects numerical errors to differential operators [16], while data-driven methods aim to identify simple dynamical models from observations [17]. More directly related to our work, Cao et al. [18] use the governing PDE residual to correct neural-operator predictions, and Jha [19] develops a residual-based correction operator for nonlinear PDE surrogates. These works use physical structure to improve surrogate predictions and focus mainly on estimating or correcting the error. We start from the prediction residual , study its local structure, and ask which terms of the governing PDE explain that structure. We then test this interpretation on held-out trajectories, with negative controls, across different regimes, and through error correction.
Problem Setting and Method
Burgers equation and prediction task
We study the standard one-dimensional viscous Burgers equation [6, 22] on the periodic domain :
where is the solution field and is the viscosity.
For a fixed prediction horizon , we define the numerical flow map by
where is the reference solution sampled on the spatial grid at the current stored time.
We define a raw-grid surrogate as a learned function
where is the predicted next state. In one-step evaluation, every prediction starts from a true input state .
Data and surrogate models
We generate the initial conditions using the following five-mode family:
Equation (4) is an experimental choice made in this work to generate smooth periodic initial states with different shapes.
For each trajectory, we sample one viscosity from
This is an experimental choice in our data-generation protocol.
The spatial grid has points. We generate the reference trajectories with a fourth-order Runge–Kutta (RK4) time integrator, a standard explicit time-integration method [22]. We use time step for 5000 RK4 steps and store a state every 250 solver steps. We therefore define the canonical one-step horizon as
The canonical data set contains 30 trajectories. We split them into 24 training and 6 validation trajectories using seed 42. Full numerical and model settings are listed in Appendix § 7.
We compare RBF KRR [7], linear Ridge regression [8], ExtraTrees [9], and Random Forests [10]. KRR is the strongest model in the canonical comparison and gives the clearest local branches, so we use it as the reference model for the detailed geometric, correction, and rollout analyses.
Prediction residual and predicted extrema
For a true one-step target and its surrogate prediction , we define the pointwise residual by
We approximate the first and second spatial derivatives with standard centered periodic finite differences [22]:
We define a predicted local maximum as a grid point satisfying
and define a predicted local minimum by reversing both inequalities.
We define the near-extrema mask as all grid points within periodic radius 3 of a predicted extremum.
For local geometry, we define a radius-5 window around each predicted extremum, giving 11 grid points per window. We define active windows using the 90th-percentile rule stated precisely in Appendix § 7.
Local geometry
We use standard principal-component analysis (PCA) to summarize the local shape and orientation of each residual branch [26]. For each active local window, we define the centered vectors
Following the standard covariance formulation of PCA [26], we define
Let be the eigenvalues of and let be the principal eigenvector. Following the standard PCA interpretation of the first principal direction and eigenvalue spread [26], we define the branch angle and elongation ratio as
Thus corresponds to a branch aligned with the predicted-value axis and to a branch aligned with the residual axis . The angle and ratio in Eq. (9) are summary measures defined in this work from the standard PCA quantities.
For descriptive conic fits, we use the standard general quadratic form for a planar conic [25]:
where we standardize and inside each radius-5 window.
We estimate the coefficients algebraically using the right singular vector associated with the smallest singular value of the corresponding design matrix, following standard algebraic conic-fitting methods [25].
The quantity is the standard quadratic-part discriminant used in conic classification [25]. We use it only to construct descriptive labels for our fitted windows. For numerical stability, we define the normalized discriminant
We also define the numerical tolerance
We call a fit ellipse-like if , hyperbola-like if , and parabola-like if . The normalization and the value of are choices made in this work for numerical classification.
Evaluation protocol
For evaluated states on an -point grid, we define the pointwise mean squared error by
We define the relative error over the same set of states by
Both equations are definitions of the evaluation measures used in this work.
For the physics attribution in § 5.4, we use five-fold cross-fitting by trajectory over all 30 trajectories. Therefore every residual used in that analysis comes from a surrogate that was not trained on the same trajectory. Bootstrap intervals are computed by resampling whole trajectories rather than individual grid points.
Geometric Analysis
A first-order residual model
We introduce a local model that separates three simple differences between the reference solution and the surrogate prediction: amplitude, spatial shift, and smoothing.
For a periodic field , we use the standard translation operator [20], written as
We also use the standard heat semigroup [21]. Motivated by these two operators and by the standard separation of phase and dissipative numerical errors [16, 22], we introduce the local error model
Equation (14) is a model introduced in this work. It is used only as a local description of possible amplitude, shift, and smoothing errors.
Using the standard first-order expansions of the translation and heat operators, we show in Appendix § 8 that Eq. (14) gives
to first order.
For a differentiable continuous field, the first derivative is zero at an interior local extremum by the standard first-order condition [28]. Our extrema are detected on a discrete grid, so we use
as a local approximation near the detected extrema. Using this approximation in Eq. (15), we obtain
This relation motivates testing curvature near predicted extrema.
Fold geometry near an extremum
For the local analysis, we consider a sufficiently smooth local representation of the predicted field. Let be a non-degenerate continuous extremum. We define
For a nearby point , we define
Proposition 1 (Local fold relation). Under the smoothness and non-degeneracy assumptions above, we show in Appendix § 8 that, to leading order,
Equation (17) is locally parabolic in the plane.
Combining Eq. (17) with the first-order extremum relation Eq. (16), we show in Appendix § 8 that, for ,
Equation (18) gives a leading-order quadratic relation in the plane under the local curvature-based residual model. Its leading-order class is parabolic. Finite-window empirical fits can also appear ellipse-like or hyperbola-like because they include higher-order terms, finite-window effects, discretization, and fitting error.
The leading-order result describes the local geometric structure; it does not imply that a conic fitted to a finite window will always recover the correct local shape. Figure 8 shows representative empirical windows and their fitted conics. Some fits follow the observed branch well, while others select a different conic type from the shape suggested by the data. We study these finite-window fitting limitations in Appendix § 9.
Figure 8. Representative KRR windows and their fitted implicit conics in the (, ) and (, ) planes. Some fitted conics follow the observed local branch, while others select a different conic type from the shape suggested by the data. These examples illustrate both the local geometric structure and the limits of finite-window conic fitting.
Experiments and Interpretation
Structured residuals appear across model families
Figure 2 shows the main empirical observation on the same canonical validation set and with common axes. In all four model families, forms coherent off-axis branches. KRR and Ridge remain comparatively concentrated around . ExtraTrees and Random Forests show larger residual magnitudes, with branches oriented more strongly along the -axis.
Figure 2. Residual geometry across four classical surrogate families on the same canonical validation set, shown with common axes. Structured off-axis branches are visible across all models. KRR and Ridge remain comparatively concentrated around , while ExtraTrees and Random Forests show larger residual magnitudes and branches oriented more strongly along the -axis.
Table 1 reports the error and the local geometry measures defined in Sec. 3. The tree branches are close to the -axis, while KRR and Ridge have smaller orientation angles. The number of active branches is also stable for the stochastic tree models across three seeds: for ExtraTrees and for Random Forests.
| Model | MSE | Rel. | Active | Median | Median |
| RBF KRR | 0.220 | 93 | 2.51 | ||
| Linear Ridge | 0.241 | 103 | 2.58 | ||
| ExtraTrees | 0.598 | 68 | 8.29 | ||
| Random Forest | 0.686 | 67 | 7.25 |
Table 1. Canonical cross-model error and active-window geometry.
The common result is that classical raw-grid surrogates can develop a structured extrema-local error. Its scale and orientation depend strongly on the model family.
The largest residuals are concentrated near extrema
We next focus on KRR. Figure 3 highlights points inside the radius-3 neighborhood of predicted extrema. These points trace the off-axis branches and show directly that the largest residual structures are tied to peaks and troughs.
Figure 3. Canonical one-step KRR residuals. Points within the radius-3 neighborhood of predicted extrema trace the main off-axis residual branches.
The near-extrema mask contains 23.9% of all grid points, but it contains 39.6% of the largest 5% of residuals and 40.9% of the largest 1%. Table 2 reports the corresponding residual statistics.
| Subset | Mean | 95th pct. | Max | |
| All points | 15,360 | 0.027 | 0.110 | 0.907 |
| Exact extrema | 552 | 0.035 | 0.155 | 0.886 |
| Near extrema | 3,668 | 0.036 | 0.158 | 0.907 |
| Far from extrema | 11,692 | 0.024 | 0.094 | 0.686 |
Table 2. KRR residual concentration in the canonical protocol.
These results show that extrema occupy a minority of grid points but contain a disproportionate share of the largest residuals.
Testing the first-order residual model
To test the three quantities predicted by Eq. (15), we use a separate high-sample diagnostic protocol. This experiment is distinct from the canonical protocol used in § 5.1 and § 5.2. It trains KRR on 5,000 subsampled training states and evaluates 20,000 validation states, giving 2,560,000 pointwise residual samples. In this diagnostic setting, the selected KRR parameters are and , and the one-step MSE is . The saved-time horizon and sampling differ from the canonical experiment, so we do not compare its absolute MSE with the canonical MSE. The full protocol is summarized in Appendix § 7.
On this diagnostic set, we define the linear regression
Equation (19) is introduced in this work only to compare how strongly the first-order quantities are related to the observed residual.
Table 3 shows three clear patterns. First, explains almost none of the residual variance. Second, curvature is the dominant single term. Third, the full model explains more variance at extrema than away from them, especially at maxima.
| Subset | ||||
| All points | 0.091 | 0.035 | 0.001 | 0.088 |
| Maxima | 0.274 | 0.057 | 0.003 | 0.242 |
| Minima | 0.173 | 0.046 | 0.002 | 0.156 |
| All extrema | 0.225 | 0.052 | 0.002 | 0.185 |
| Non-extrema | 0.087 | 0.034 | 0.001 | 0.084 |
Table 3. KRR diagnostic regression on the large diagnostic protocol. The viscosity-only contribution is negligible and is omitted from the table.
The first-derivative term has very little explanatory power in this diagnostic experiment. The curvature term has the largest explanatory power among the three first-order quantities, consistent with the reduced extremum relation in Eq. (16).
We now return to the canonical protocol for the geometric plots and the remaining KRR analyses. Figure 4 shows the relation between predicted value and predicted curvature near extrema. The observed fold is consistent with the relation that we show in Eq. (17).
Figure 4. Predicted value versus predicted curvature near KRR extrema. The structured value–curvature relation is consistent with the fold geometry shown in § 4.
Detailed finite-window conic counts and held-out conic fits are reported in Appendix § 9.
Connecting the residual to the Burgers equation
The Burgers equation in Eq. (1) contains separate advection and diffusion terms. Evaluating these terms on the predicted field, we define
To test how strongly these two terms are related to the observed residual, we define the diagnostic regression
Equation (21) is an attribution model introduced in this work. In Appendix B, we show from the fundamental theorem of calculus and Eq. (1) why the residual rate can be compared with the time-averaged Burgers terms.
For a feature , we define its held-out contribution by
We compute all values with trajectory-level cross-fitting.
As a spatial negative control, we periodically shift the diffusion field inside each state. This keeps the same diffusion values but breaks their pointwise alignment with the residual. We define the shift gap as the difference between the held-out obtained with the correctly aligned diffusion field and the held-out obtained with the shifted field.
Table 4 shows a large diffusion contribution and a large spatial shift gap for KRR and Ridge, while the advection contribution is much smaller. ExtraTrees and Random Forests show much smaller diffusion contributions and shift gaps.
| Model | Full | Shift gap | ||
| RBF KRR | 0.420 | 0.386 | 0.032 | 0.392 |
| Linear Ridge | 0.534 | 0.407 | 0.051 | 0.438 |
| ExtraTrees | 0.101 | 0.010 | 0.001 | 0.010 |
| Random Forest | 0.018 | 0.034 | 0.035 |
Table 4. Held-out attribution near predicted extrema. “Shift gap” is the loss of held-out after spatially shifting the diffusion field.
For Random Forest, is larger than the full-model . The reduced held-out model obtained after removing diffusion has negative . By the definition , this happens when the held-out squared error is larger than the total held-out variance.
Because the diffusion feature defined in Eq. (20) is proportional to curvature, this result gives a physical interpretation of the curvature result. The spatial negative control and the spectral test below provide additional evidence.
Effective diffusion and spectral roughness
To interpret the sign and scale of the fitted diffusion coefficient , we introduce the following short-horizon local comparison:
Equation (23) uses the standard heat semigroup [21]. It is a diagnostic model introduced in this work. The parameter is used only as a local diagnostic quantity.
Using the standard first-order expansion of the heat semigroup, we show in Appendix B that, under this local model,
We focus the interpretation on the moderate- and high-viscosity regimes and . Table 5 reports the resulting effective-diffusion ratios at short and long horizons.
We also define a spectral diagnostic that does not use the residual regression. Let denote the Fourier coefficients of . Using Fourier differentiation and Parseval’s identity [23], we show in Appendix B that second-derivative energy weights frequency proportionally to . Based on this result, we define
By definition, means that the prediction has more high-frequency content relative to its total spectral energy than the corresponding true future state.
| Model | \multicolumn{2}{c}{} | \multicolumn{2}{c}{} | |||
| KRR | 0.05 | 0.728 | 2.40 | 0.920 | 5.66 |
| KRR | 0.10 | 0.750 | 2.90 | 0.965 | 6.97 |
| Ridge | 0.05 | 0.681 | 3.24 | 0.911 | 6.64 |
| Ridge | 0.10 | 0.712 | 4.81 | 0.958 | 7.24 |
Table 5. Effective-diffusion and spectral-roughness diagnostics in the moderate/high-viscosity regimes.
At , all four moderate/high-viscosity values satisfy , and all four roughness ratios satisfy . Together, these results are consistent with insufficient viscous smoothing: the surrogate retains more small-scale structure than the true future state.
The two diagnostics have different dependence on the prediction horizon. In our experiments, moves toward one as increases, while the roughness ratio grows. We do not assign a specific cause to this difference. The complete viscosity–horizon sweep is given in Appendix § 10.
Can the identified structure help reduce the error?
A useful interpretation should explain a substantial part of the error and should provide information that can be used on unseen trajectories. We test both levels.
Oracle geometric upper bound. We first define an oracle test in which a local amplitude–curvature residual profile is fitted using the true residual from the same active KRR window. Because the residual being corrected is used during fitting, this test measures an upper bound on how much of the local structured error can be removed.
The oracle fit removes a median 85% of active-window residual energy, and 97% of active windows have more than half of their residual energy removed.
Figure 5. Oracle geometric upper bound on active KRR windows. The local correction is fitted using each window’s true residual, so this experiment measures achievable local error removal when the residual profile is known.
Fully non-oracle correction. We next define a correction that uses only quantities available from the prediction. The correction uses radius-5 profiles of , , and , local scalar summaries, and , with separate models for maxima and minima. The main profile predictor uses gradient boosting [24] together with a frequency-band calibration. All correction strengths are selected using training trajectories only; no validation residual is used before evaluation. The full procedure is defined in Appendix § 11.
The profile–spectral correction reduces all-point MSE by 31.24%, near-extrema MSE by 49.54%, and active-window MSE by 45.22%. Relative decreases from 0.220 to 0.183.
We define the fraction of active-window residual energy removed, denoted by , in Appendix § 11. For the main correction, its median value is
| Method | All red. | Near red. | Active red. | Rel. | Median |
| Profile + spectral filter | 31.24% | 49.54% | 45.22% | 0.183 | 0.534 |
Table 6. Main non-oracle one-step KRR correction result, canonical protocol. Reductions are relative to plain KRR.
The residual plot confirms that the correction acts directly on the structured branches. Small and moderate branches move toward , while the largest branches remain only partly corrected.
Figure 6. Canonical one-step KRR residuals before and after the fully non-oracle profile–spectral correction. Small and moderate extrema branches move toward ; the largest branches remain only partly corrected.
Recursive rollout. We define a rollout as recursive use of the surrogate: after the first step, each predicted state becomes the input to the next prediction. This differs from one-step evaluation, where each prediction starts from a true state.
Starting from the true initial state, we define the plain rollout by
To study how the error accumulates, we define the state error
and the one-step surrogate residual evaluated at the predicted state
We define the Jacobian of the reference flow with respect to its state input as
For , we define the propagation matrix
where is the identity matrix.
Assuming that is twice continuously differentiable in a neighborhood of the rollout states, we show in Appendix § 12, using the standard multivariable Taylor theorem [28], that the accumulated -step error satisfies, to first order,
Equation (29) shows explicitly how one-step errors accumulate recursively. A residual produced at step is propagated through the later flow sensitivities before contributing to the error at step . Earlier one-step errors can therefore influence all later states.
We apply the same non-oracle correction inside a 20-step KRR rollout. The final relative error decreases from 0.935 to 0.616. Pooled all-point MSE decreases from 0.0270 to 0.0159, and pooled near-extrema MSE decreases from 0.0353 to 0.0165. On the fixed plain-rollout active-window set, the median residual energy removed is , with 79% of windows improved.
| Metric | Plain rollout | Corrected rollout |
| Final relative | 0.935 | 0.616 |
| Pooled all-point MSE | 0.0270 | 0.0159 |
| Pooled near-extrema MSE | 0.0353 | 0.0165 |
| Pooled median | 0.090 | 0.057 |
| Pooled | 33.2% | 43.8% |
Table 7. Recursive KRR rollout over the six validation trajectories.
Figure 7. Relative error during recursive KRR rollout, with and without the non-oracle correction applied in the loop.
The correction is used as a test of whether the diagnosed local structure contains usable information. The one-step and rollout reductions show that it does, although the largest peak errors remain the most difficult cases.
Discussion and limitations
The experiments support a two-level interpretation. First, extrema create a folded value–curvature geometry that can organize residuals into curved local branches across different estimators. Second, the physical attribution of those branches depends on the model family. For KRR and Ridge, the clearest moderate/high-viscosity interpretation is insufficient viscous smoothing. The tree ensembles share the geometric pattern but show much weaker endpoint diffusion attribution.
Several limitations are important. The study concerns one one-dimensional PDE and one raw-grid representation, so the geometric pattern should not yet be claimed as universal. The quantity is a horizon-dependent diagnostic and should not be interpreted as a learned physical viscosity. The diffusion regression is closely related to the earlier curvature regression, so the negative control and spectral roughness provide important additional evidence. Low-viscosity results are less uniform. Finally, local conic fits generalize poorly when transferred directly to held-out windows. The main geometric result therefore concerns the fold/extrema mechanism, while the finite-window conics remain descriptive diagnostics.
Conclusion
Classical raw-grid surrogates for viscous Burgers can have structured local errors that are hidden by aggregate metrics. Across KRR, Ridge, ExtraTrees, and Random Forests, one-step residuals form coherent branches near predicted extrema. For the KRR reference, curvature dominates transport, and combining the local extremum fold with the curvature-based residual model predicts a leading-order near-parabolic relation in the plane. Physics-based tests then separate geometry from mechanism: KRR and Ridge show evidence consistent with insufficient viscous smoothing at moderate and high viscosity, while the tree ensembles show much weaker diffusion attribution. Non-oracle one-step and rollout corrections further show that the diagnosed structure carries predictive information. These results illustrate a practical route from an observed surrogate failure to a testable physical interpretation.
Full Experimental Setup
We use three experimental protocols. They serve different purposes, so their absolute errors should not be mixed.
| Setting | Large diagnostic | Canonical | Replication |
| Purpose | first-order regression | main results | held-out conic test |
| Data | 5,000 train states, 20,000 val. states | 30 trajectories | 30 fresh trajectories |
| Train/validation | subsampled states | 24/6, seed 42 | 24/6 |
| Validation points | 2,560,000 | 15,360 | 15,360 |
| KRR | 0.01 | 0.01 | |
| RBF | |||
| KRR one-step MSE | |||
| Relative | — | 0.2201 | 0.2157 |
Table 8. Summary of the three experimental protocols used in the paper.
The large diagnostic protocol uses a one-frame saved-time prediction horizon from the original diagnostic data set. It is used only for the regression in § 5.3. Its sampling and saved-time horizon differ from the canonical protocol, so its absolute MSE is not compared with the canonical MSE.
The canonical protocol uses the periodic domain , , and . Reference trajectories are generated with RK4 [22], using solver time step for 5000 RK4 steps. States are stored every 250 steps, which gives the one-step horizon . Initial conditions use the five-mode family in Eq. (4). One viscosity from is sampled once per trajectory. The 30 trajectories are split into 24 training and 6 validation trajectories with seed 42.
For the canonical cross-model comparison, RBF KRR uses regularization 0.01 and RBF parameter . Linear Ridge uses penalty 1. ExtraTrees uses 300 trees, feature fraction 0.7, minimum leaf size 1, and seed 42. Random Forest uses 200 trees with the same feature fraction, minimum leaf size, and seed.
The replication protocol repeats the canonical settings with a fresh draw of 30 trajectories and a fresh 24/6 split. It is used only for the held-out conic tests in Appendix § 9. Its KRR validation MSE is and its relative error is 0.2157.
Active-window definition. Within each extremum type separately, we define as the 90th percentile of over the radius-3 near-extrema points. We define a radius-5 window as active if its peak exceeds the corresponding . The active-window MSE is computed over the union of active-window grid points.
Local conic fit. Within each radius-5 window, we standardize and to zero mean and unit variance. Following the standard algebraic representation of conics [25], we form the design matrix with rows
We define the fitted coefficient vector as the right singular vector associated with the smallest singular value. Coefficients are normalized before computing Eq. (11).
Derivations and Supporting Calculations
First-order operator perturbation
For a sufficiently smooth periodic field, standard translation-group theory gives [20]
For a sufficiently smooth field , the standard heat-semigroup expansion gives [21]
We now show Eq. (15). Substituting Eq. (30) into Eq. (14), applying Eq. (31), and multiplying by gives
Keeping only first-order terms gives Eq. (15).
Fold relation and residual relation
We now show Eq. (17). We define
Because is a differentiable interior local extremum, the standard first-order condition gives
[28]. Taylor’s theorem [28] gives
and
Using the definitions of and , these two expansions give
and
Squaring the second relation gives
Multiplying the first relation by gives
Therefore,
Keeping the leading term gives Eq. (17).
We next show Eq. (18) under the first-order local residual model. Near the extremum, the first-derivative shift contribution is small under the approximation used in § 4, and Eq. (16) gives
For ,
Substituting this relation into Eq. (17) gives
Multiplying by gives Eq. (18). Thus the relation follows from the fold relation together with the stated first-order residual model.
If , the displayed fold is not the leading asymmetric relation. In that case, one must continue the Taylor expansion to the first nonzero odd derivative.
Why the residual rate is compared with Burgers terms
We now justify the use of in Eq. (21).
From the definition of the residual, subtracting the same input state from the two future states gives the exact identity
For the reference solution, the fundamental theorem of calculus [28] gives
Using the Burgers equation Eq. (1), we obtain
Equations (36) and (38) show that compares the reference and surrogate average rates of change, while the reference rate is governed by the Burgers advection and diffusion terms. This provides the stated motivation for the endpoint features and in the diagnostic regression.
Effective-diffusion diagnostic
We now show Eq. (24) under the local diffusion model of Eq. (23). In this diagnostic, denotes the surrogate endpoint .
For sufficiently regular , the standard heat-semigroup expansion gives [21]
and
Subtracting gives
Dividing by gives
To compare this expression with the regression feature , we make one additional short-horizon regularity assumption: is smooth enough that the heat-semigroup expansion remains valid after applying two spatial derivatives. Since , the same expansion then gives
Hence
and Eq. (39) becomes
The diffusion part of Eq. (21) is . Matching the leading coefficient of gives
Since ,
which is Eq. (24). This is a leading-order short-horizon diagnostic relation under the assumptions stated above.
Why the roughness diagnostic uses fourth-power frequency weighting
We now justify the weighting in Eq. (25).
For a periodic Fourier series,
Standard Fourier differentiation gives [23]
Applying Parseval’s identity gives
Equation (40) shows that second-derivative energy weights Fourier mode by , up to the common constant . Normalizing by the total Fourier energy gives the roughness measure defined in Eq. (25).
Additional Geometric Results
Finite-window conic classes
Using the conic classification defined in § 3, we obtain the counts in Table 9.
| Type | Ellipse-like | Hyperbola-like | Parabola-like | Total |
| Maxima | 24 | 29 | 0 | 53 |
| Minima | 26 | 13 | 1 | 40 |
Table 9. Canonical active-window conic classifications.
These empirical classes contain finite-window and higher-order effects. The leading-order relation shown in Eq. (18) is parabolic.
Figure 9. Distributions of normalized finite-window conic discriminants for active KRR windows.
Held-out conic transfer
To test whether a fitted local conic is a reusable prediction rule, we use the independent replication protocol defined in Appendix A. For each active 11-point window, the within-window test fits the conic on 6 points and evaluates it on the remaining 5 points. This is our held-out conic test; it is separate from the canonical experiment.
Full Physical Sweep
For completeness, we also fit a single physics regression pooled across all viscosities. This gives
and
These are coefficients from one regression fitted jointly across all viscosities. They are not averages of the coefficients, or of , obtained from the separate per-viscosity regressions. For this reason, the pooled effective-diffusion value can lie outside the range of the per-viscosity values. We use the separate moderate- and high-viscosity results for the physical interpretation in the main text.
At and , the fitted values are
and
At and , the fitted values are
and
Using Eq. (24), we obtain the full effective-diffusion values reported in Table 11.
| Model | |||||
| KRR | 0.01 | 1.267 | 0.689 | 0.476 | 0.570 |
| KRR | 0.02 | 1.094 | 0.938 | 0.788 | 0.731 |
| KRR | 0.05 | 0.728 | 0.765 | 0.848 | 0.920 |
| KRR | 0.10 | 0.750 | 0.860 | 0.932 | 0.965 |
| Ridge | 0.01 | 0.459 | 0.415 | 0.590 | 0.763 |
| Ridge | 0.02 | 0.817 | 0.791 | 0.714 | 0.746 |
| Ridge | 0.05 | 0.681 | 0.734 | 0.813 | 0.911 |
| Ridge | 0.10 | 0.712 | 0.854 | 0.933 | 0.958 |
Table 11. Full endpoint effective-diffusion diagnostic. Entries are from separate per-viscosity regressions.
The low-viscosity rows are less coherent. In particular, KRR gives at and . We therefore restrict the insufficient-smoothing interpretation to the moderate- and high-viscosity regimes.
For reference, the corresponding curvature-energy ratios (prediction divided by true target) at are
| KRR | 2.3494 | 3.3894 |
| Ridge | 3.1304 | 5.8239 |
and at are
| KRR | 6.3002 | 17.1209 |
| Ridge | 9.6401 | 19.9307 |
Correction Details and Ablations
For an active local window with true residual and predicted residual correction , we define the fraction of residual energy removed as
By this definition, means that all residual energy in the window is removed, while means that the squared residual energy is unchanged.
The fully non-oracle correction uses sign-canonicalized radius-5 profiles of , , and , meaning that the two extremum types are represented in comparable coordinates before fitting. The features also include local scalar summaries and . Maxima and minima are modeled separately. The main profile predictor uses gradient boosting [24]. Overlapping predicted windows are combined with center-weighted triangular aggregation, which gives larger weight to points closer to the window center. The spectral calibration learns frequency-band gains using training data only. The PCA-template baseline uses standard low-dimensional principal-component ideas [26, 27].
| Method | All red. | Near red. | Active red. | Rel. | Median |
| Profile correction | 24.29% | 43.99% | 39.80% | 0.192 | 0.448 |
| Profile + spectral filter | 31.24% | 49.54% | 45.22% | 0.183 | 0.534 |
| PCA-template | 12.36% | 20.69% | 22.26% | 0.206 | 0.353 |
| RF local-window squeeze | 8.48% | 11.22% | 29.51% | 0.211 | 0.410 |
| Tail-aware ensemble | 31.08% | 49.51% | 45.42% | 0.183 | 0.531 |
Table 12. Full non-oracle one-step KRR correction ablation. Reductions are relative to plain KRR.
The profile–spectral method is the main correction because the larger tail-aware ensemble gives almost the same result.
Qualitative corrected-state snapshot. Figure 10 gives a direct view of the one-step prediction before and after correction. It is kept in the appendix because the main quantitative evidence is already given by the residual plot and error tables.
Figure 10. One-step true target, plain KRR prediction, and corrected prediction for one validation state. The correction brings several smaller extrema closer to the reference, while some of the largest extrema remain difficult and can be moved too strongly. This figure is qualitative; the correction metrics are computed over the full canonical validation set.
Recursive Rollout Details
We now show Eq. (29).
By the definitions of the reference and surrogate rollouts,
Using the definition
and the one-step residual defined in Eq. (27), we obtain the exact identity
Because
we assume that is twice continuously differentiable in a neighborhood of the rollout states. The standard multivariable Taylor theorem [28] then gives
where, by definition,
Substituting Eq. (43) into Eq. (42) gives
Keeping only the first-order terms in the rollout error gives
Because the rollout starts from the true initial state,
Repeated substitution gives
and
The formulas below keep only these first-order terms; the propagated second-order remainders are omitted. We now show the general first-order form by induction. Assume that
Using the first-order recursion gives
Substituting the induction hypothesis gives
By the definition of the propagation matrices,
and
Therefore,
This shows by induction that
which is Eq. (29).
| One-step | Plain rollout | Corrected rollout | |
| Final rel. | — | 0.935 | 0.616 |
| Pooled all-point MSE | — | 0.0270 | 0.0159 |
| Pooled near-extrema MSE | — | 0.0353 | 0.0165 |
| Pooled median | 0.011 | 0.090 | 0.057 |
| Pooled | 87.7% | 33.2% | 43.8% |
| Active-window median | — | 0.65 (79% improved) |
Table 13. Additional recursive KRR rollout statistics.
Figure 11. Matched-sample comparison of one-step and recursive KRR residuals. The near-zero cloud becomes less dominant during rollout, while the structured off-axis branches remain visible.
Figure 12. Plain versus corrected residuals on the fixed active-window set during recursive rollout.
Additional Robustness Checks
The cross-model conclusion is not driven by one random tree seed. Across three seeds, the measured number of active windows is
for ExtraTrees and
for Random Forests, where the uncertainty is the standard deviation across the three seeds. For completeness, Figure 13 shows the plain KRR residual diagnostic without any rollout-calibrated gain.
Figure 13. Plain KRR residual diagnostic without rollout gain. The structured off-axis branches remain visible.
Coverage-preserving trimmed conic refits change the measured finite-window conic class for only about 6–8% of active windows. The held-out transfer results in Table 10 show that the conic fits should remain descriptive local diagnostics.
| Extremum | Class | All | Train | Held-out |
| Max | ellipse-like | 0.038 | 0.006 | 0.145 |
| Max | hyperbola-like | 0.036 | 0.007 | 0.152 |
| Min | ellipse-like | 0.039 | 0.009 | 0.154 |
| Min | hyperbola-like | 0.017 | 0.005 | 0.107 |
Table 10. Within-window held-out conic RMSE on the independent replication protocol. The conic is fit on 6 of the 11 points and evaluated on the remaining 5. The large held-out gap shows that local conics are useful as descriptive diagnostics but transfer poorly as prediction rules.
References
- [1]M. Raissi, P. Perdikaris, and G. E. Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707, 2019.DOI
- [2]G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, and L. Yang. Physics-informed machine learning. Nature Reviews Physics, 3:422–440, 2021.DOI
- [3]L. Lu, P. Jin, G. Pang, Z. Zhang, and G. E. Karniadakis. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3:218–229, 2021.
- [4]Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and A. Anandkumar. Fourier neural operator for parametric partial differential equations. In International Conference on Learning Representation, 2021.
- [5]N. Kovachki, Z. Li, B. Liu, K. Azizzadenesheli, K. Bhattacharya, A. Stuart, and A. Anandkumar. Neural operator: Learning maps between function spaces with applications to PDEs. Journal of Machine Learning Research, 24(89):1–97, 2023.
- [6]M. Takamoto, T. Praditia, R. Leiteritz, D. MacKinlay, F. Alesiani, D. Pflüger, and M. Niepert. PDEBench: An extensive benchmark for scientific machine learning. In Advances in Neural Information Processing Systems, Datasets and Benchmarks Track, 2022.arxiv.org/abs/2210.07182
- [7]B. Schölkopf and A. J. Smola. Learning with Kernels. MIT Press, 2002.DOI
- [8]A. E. Hoerl and R. W. Kennard. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1):55–67, 1970.
- [9]P. Geurts, D. Ernst, and L. Wehenkel. Extremely randomized trees. Machine Learning, 63:3–42, 2006.DOI
- [10]L. Breiman. Random forests. Machine Learning, 45:5–32, 2001.
- [11]N. H. Nelsen and A. M. Stuart. Operator learning using random features: A tool for scientific computing. SIAM Review, 66(3):535–571, 2024.DOI
- [12]G. Stepaniants. Learning partial differential equations in reproducing kernel Hilbert spaces. Journal of Machine Learning Research, 24(86):1–72, 2023.
- [13]F. Doshi-Velez and B. Kim. Towards a rigorous science of interpretable machine learning. arXiv:1702.08608, 2017.
- [14]J. Adebayo, J. Gilmer, M. Muelly, I. Goodfellow, M. Hardt, and B. Kim. Sanity checks for saliency maps. In Advances in Neural Information Processing Systems, 2018.arxiv.org/abs/1810.03292
- [15]R. Roscher, B. Bohn, M. F. Duarte, and J. Garcke. Explainable machine learning for scientific insights and discoveries. IEEE Access, 8:42200–42216, 2020.DOI
- [16]R. F. Warming and B. J. Hyett. The modified equation approach to the stability and accuracy analysis of finite-difference methods. Journal of Computational Physics, 14(2):159–179, 1974.DOI
- [17]S. L. Brunton, J. L. Proctor, and J. N. Kutz. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the National Academy of Sciences, 113(15):3932–3937, 2016.DOI
- [18]L. Cao, T. O’Leary-Roseberry, P. K. Jha, J. T. Oden, and O. Ghattas. Residual-based error correction for neural operator accelerated infinite-dimensional Bayesian inverse problems. Journal of Computational Physics, 486:112104, 2023.DOI
- [19]P. K. Jha. Residual-based error corrector operator to enhance accuracy and reliability of neural operator surrogates of nonlinear variational boundary-value problems. Computer Methods in Applied Mechanics and Engineering, 419:116595, 2024.DOI
- [20]K.-J. Engel and R. Nagel. One-Parameter Semigroups for Linear Evolution Equations. Springer, 2000.
- [21]A. Pazy. Semigroups of Linear Operators and Applications to Partial Differential Equations. Springer, 1983.DOI
- [22]D. R. Durran. Numerical Methods for Fluid Dynamics: With Applications to Geophysics. Springer, second edition, 2010.
- [23]L. N. Trefethen. Spectral Methods in MATLAB. SIAM, 2000.DOI
- [24]J. H. Friedman. Greedy function approximation: A gradient boosting machine. The Annals of Statistics, 29(5):1189–1232, 2001.DOI
- [25]A. Fitzgibbon, M. Pilu, and R. B. Fisher. Direct least square fitting of ellipses. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(5):476–480, 1999.DOI
- [26]I. T. Jolliffe. Principal Component Analysis. Springer, second edition, 2002.
- [27]G. Berkooz, P. Holmes, and J. L. Lumley. The proper orthogonal decomposition in the analysis of turbulent flows. Annual Review of Fluid Mechanics, 25:539–575, 1993.
- [28]W. Rudin. Principles of Mathematical Analysis. McGraw–Hill, third edition, 1976.