Introduction and related works
Solving partial differential equations (PDEs) is fundamental across diverse fields, including fluid dynamics, climate modeling, and biophysics [24 , 19 ] . Traditional numerical solvers for PDEs often incur prohibitive computational costs, particularly when repeated evaluations are required in applications. Recently, scientific machine learning has emerged as a powerful paradigm, leveraging advances in deep neural networks to deliver fast and accurate approximations of PDE solutions, thereby enabling high-fidelity simulations and real-time predictive capabilities that were previously unattainable [4 , 11 , 22 ] . Deep learning for PDEs can be broadly categorized into: data-driven operator learning, physics-informed deep learning, and meta-learning physics-informed neural networks (PINNs).
Data-driven operator learning: Data-driven operator learning focuses on training neural networks to directly map input functions, such as PDE parameters or initial conditions, to their corresponding PDE solutions. Well-known examples of this approach include DeepONet [Lu et al., 2021], Derivative-
Preprint. enhanced-DeepONet [35 ] , Transformer-DeepONet [47 ] , Fourier neural operators (FNO) [29 ] , Factorized-FNO [40 ] , Decomposed-FNO [28 ] , and Tucker-FNO [54 ] . Once trained, these models can generalize to new parameter settings (e.g., new initial conditions), enabling fast prediction of complex dynamical behaviors. However, they lack explicit physics constraints and interpretability, and their generalization depends on adequacy of labeled training data, which are often expensive to obtain [30 ] . For example, Transformer-DeepONet typically requires at least 1,000 different PDE solutions for adequate performance [47 ] .
Physics-informed deep learning: An alternative paradigm, physics-informed deep learning, incorporates governing PDEs and boundary conditions into the training loss function [49 , 53 , 46 ] . By penalizing violation of physical laws, these models can be trained effectively even in data-sparse regimes, often requiring only initial or boundary conditions. Representative approaches include physics-informed variants of operator learning, such as physics-informed DeepONet [41 ] , physics-informed neural operators (PINO) [30 ] , and HyPINO [1 ] . Another prominent method is PINN [37 ] , which learns to map the spatiotemporal coordinates to the PDE solution for a given system. While these methods alleviate the need for large labeled datasets, their reliance on physics-informed objectives often results in prohibitively long training time, due to highly nonconvex loss landscapes caused by the stiff PDE constraints [7 , 43 , 13 ] , especially for nonlinear PDEs. For example, training physics-informed DeepONet on a family of Burgers’ equations can take more than 9 hours [41 ] , while PINNs may require over 24 hours to solve a single Kuramoto-Sivashinsky equation [45 ] .
Meta-learning PINNs: Adapting PINN to new PDEs or parameter settings typically requires costly retraining. Transfer learning strategies accelerate convergence by reusing knowledge from related tasks [48 , 42 ] . Parameterized PINNs (P 2 P^{2} P 2 INNs) further incorporate problem parameters as network inputs, enabling a single model to represent families of PDEs without labeled data while preserving physical consistency [9 ] . However, P 2 P^{2} P 2 INNs still suffer from the optimization challenges of physics-based loss functions, and their adaptation to new tasks remains heavily reliant on gradient-based fine-tuning which is often slow to converge, especially for nonlinear PDEs. Meta-learning methods aim to enable rapid adaptation to unseen PDEs with limited data by learning parameter initializations or update rules [33 , 50 ] . MAML-style approaches learn transferable parameter priors [31 , 51 ] , while other methods leverage latent codes or meta-optimizers for task-specific adaptation [23 , 8 ] . To avoid gradient-based adaptation, Baldwinian-PINN introduces a gradient-free meta-learning strategy based on neuroevolution, evolving shared representations and fine-tuning the output layer via least squares [52 ] . While highly efficient for linear PDEs, its single-layer architecture limits nonlinear expressiveness. Moreover, its Picard (lagging-of-coefficients) iteration exhibits only linear convergence, leading to slow solution refinement for nonlinear problems [6 , 34 ] .
See Appendix A for an extended literature review. In summary, existing deep learning approaches for solving nonlinear PDEs remain constrained by substantial data requirements and/or slow, optimization-heavy adaptation. For this reason, we propose a novel meta-learning PINN framework (Figure 1 ), termed Newton-PINet . The main contributions are as follows:
Figure 1. Newton-PINet model framework. Stage 1 (pre-training) meta-learns the task-agnostic parameters Θ = { w ~ , λ p d e , λ r e g } \Theta= \{\tilde{w}, \lambda_{\mathrm{pde}}, \lambda_{\mathrm{reg}}\} Θ = { w ~ , λ pde , λ reg } by gradient descent (outer loop), while the output-layer weights w w w of each training task are updated by gradient-free Tikhonov-regularized least squares (inner loop); Stage 2 (inference) freezes Θ \Theta Θ and adapts only w w w to new tasks. The bottom panel compares Picard and Newton linearization; the latter accelerates the convergence of nonlinear iterations for solving nonlinear PDEs.
We introduce a physics-informed multilayer network with skip connections from early hidden layers to the output, where the final-layer weights are computed using least-squares approach (Tikhonov regularization). This skip-connected Tikhonov regularization PINN architecture improves the model’s nonlinear representation capabilities.
We employ a two-stage learning strategy, where the first stage uses a gradient-based method to meta-learn the nonlinear hidden layers’ network weights and essential hyperparameters for task-specific adaptation in an unsupervised or few-shot manner. At inference (test time), task-specific adaptation is confined to the output layer, which can be rapidly updated in a gradient-free manner using Tikhonov regularization.
We integrate Newton linearization into the Tikhonov-regularized PINN and establish its equivalence to the classical regularized Gauss-Newton method through mathematical derivation, with experiments validating its near-quadratic convergence. The Newton linearization accelerates nonlinear least-squares convergence, providing a more efficient alternative to the
traditional Picard approach during both meta-learning and inference. We therefore refer to the proposed physics-informed network with Newton linearization as Newton-PINet.
Results on diverse nonlinear PDE benchmarks show that, compared with recent neural solvers, Newton-PINet significantly reduces the reliance on training data while achieving fast and accurate generalization to new PDE tasks through physics-consistent fine-tuning.
Preliminaries
For simplicity, consider a nonlinear PDE with spatial variable x x x , time t t t , and solution u u u defined on Ω × [ 0 , T ] \Omega\times[0,T] Ω × [ 0 , T ] :
P D E : N θ [ u ( x , t ) ] = q ( x , t ) x ∈ Ω , t ∈ [ 0 , T ] , I C : u ( x , t = 0 ) = u 0 ( x ) x ∈ Ω , B C : B [ u ( x , t ) ] = g ( x , t ) x ∈ ∂ Ω , t ∈ [ 0 , T ] . (1) \begin{aligned}
\mathrm{PDE}: \quad\mathcal{N}_{\theta}[u(x,t)] &= q(x,t) \quad x \in\Omega,\ t \in[0,T],\\
\mathrm{IC}: \quad u(x,t=0) &= u_0(x) \quad x \in\Omega,\\
\mathrm{BC}: \quad\mathcal{B}[u(x,t)] &= g(x,t) \quad x \in\partial\Omega,\ t \in[0,T]. \tag*{(1)}
\end{aligned} PDE : N θ [ u ( x , t )] IC : u ( x , t = 0 ) BC : B [ u ( x , t )] = q ( x , t ) x ∈ Ω , t ∈ [ 0 , T ] , = u 0 ( x ) x ∈ Ω , = g ( x , t ) x ∈ ∂ Ω , t ∈ [ 0 , T ] . (1)
where the general differential operator N θ \mathcal{N}_{\theta} N θ can include both linear and nonlinear combinations of the temporal and spatial derivatives and PDE parameters θ \theta θ , q ( x , t ) q(x,t) q ( x , t ) is the source term, u 0 ( x ) u_0(x) u 0 ( x ) is the initial condition (IC), and g ( x , t ) g(x,t) g ( x , t ) is the boundary condition (BC).
Single PDE problem: Standard PINNs aim to solve a single PDE instance (a single task ) defined by specific PDE parameters, IC, and BC. PINNs are trained by minimizing the discrepancy between Eq. (1) and the model’s prediction.
Towards generalizable PINNs: There is growing interest in models capable of generalizing across a set of tasks belonging to some underlying task-distribution p ( T ) p(\mathcal{T}) p ( T ) , e.g., a family of PDEs spanning different PDE parameters, ICs, and BCs. For meta-learning PINNs, the goal is to learn network initializations using training tasks sampled from p ( T ) p(\mathcal{T}) p ( T ) that enable fast, accurate, and physics-aware predictions on unseen scenarios, i.e., any new task from the distribution T i ∼ p ( T ) \mathcal{T}_i \sim p(\mathcal{T}) T i ∼ p ( T ) through fine-tuning [50 ] .
Methodology
Skip-connected Tikhonov regularization PINN: We introduce skip connections from all hidden layers to the output layer, with the output-layer weights computed using Tikhonov regularization [17 ] . The skip-connected neural architecture improves expressivity, ensuring stable and accurate least-squares computation by increasing the output-layer width through stacking additional hidden layers, while still maintaining a moderate number of nodes per layer. As shown in Figure 1 , we employ the neural networks with L + 1 L+1 L + 1 layers, where the input is x = ( x , t ) \mathbf{x}=(x,t) x = ( x , t ) (layer 0) and the output is u u u (layer L L L ). Each hidden layer l = 1 , … , L − 1 l=1,\ldots,L-1 l = 1 , … , L − 1 contains the same number of neurons (N n N_n N n ). Sinusoidal feature embeddings are applied at the first layer [49 ] , and all hidden layers use sin ( ⋅ ) \sin(\cdot) sin ( ⋅ ) activation functions. The pre-final output f \mathbf{f} f is constructed as the concatenation of all hidden activations (skip connections). All trainable network parameters up to the pre-final layer are denoted w ~ \widetilde{\mathbf{w}} w . The output u u u can be computed by u ( x ) = f ( x ; w ~ ) T w u(\mathbf{x})=\mathbf{f}(\mathbf{x};\widetilde{\mathbf{w}})^T\mathbf{w} u ( x ) = f ( x ; w ) T w , where w \mathbf{w} w is the output-layer weights. See Appendix § 10 .§ 1 for a detailed description of the skip-connected model architecture.
The objective of task-specific learning is to determine the best set of w \mathbf{w} w such that u ( x ) = f ( x ; w ~ ) T w u(\mathbf{x})=\mathbf{f}(\mathbf{x};\widetilde{\mathbf{w}})^T\mathbf{w} u ( x ) = f ( x ; w ) T w satisfies the PDE, IC, and BC defined in Eq. (1) for a task T i ∼ p ( T ) T_i\sim p(T) T i ∼ p ( T ) . This can be formulated as a physics-based least-squares problem. Given collocation points for PDE residual, points at the initial time, and points on the boundary, together with the loss importance hyperparameter λ p d e \lambda_{\mathrm{pde}} λ pde , the following system is obtained:
\begin{bmatrix}
\cdots& \lambda_{\mathrm{pde}}\cdot\mathcal{N}_{\theta}[f_j(\mathbf{x}^{\mathrm{pde}}_1,t^{\mathrm{pde}}_1;\widetilde{\mathbf{w}})] & \cdots\\
\cdots& f_j^i(\mathbf{x}^{\mathrm{ic}}_1,0;\widetilde{\mathbf{w}}) & \cdots\\
& \vdots& \\
\cdots& \mathcal{B}[f_j(\mathbf{x}^{\mathrm{bc}}_1,t^{\mathrm{bc}}_1;\widetilde{\mathbf{w}})] & \cdots\\
& \vdots&
\end{bmatrix}
\begin{bmatrix}
\vdots\\
w_j\\
\vdots
\end{bmatrix}
=
\begin{bmatrix}
\lambda_{\mathrm{pde}}\cdot q(\mathbf{x}^{\mathrm{pde}}_1,t^{\mathrm{pde}}_1)\\
\vdots\\
u_0(\mathbf{x}^{\mathrm{ic}}_1)\\
g(\mathbf{x}^{\mathrm{bc}}_1,t^{\mathrm{bc}}_1)\\
\vdots
\end{bmatrix}, \tag*{(2)}\end{bmatrix}
which can be compactly written as
A w = b . A\mathbf{w}=\mathbf{b}. A w = b .
Here A w A\mathbf{w} A w represents the left-hand side of Eq. (1) at a given set of collocation points, and b \mathbf{b} b denotes the corresponding right-hand side of Eq. (1). The number of rows in A A A and b \mathbf{b} b equals the total number of collocation points. The number of columns in A A A corresponds to the product of the network’s pre-final feature dimension and the output dimension. The final output weights w \mathbf{w} w are obtained via Tikhonov regularization: arg min w ∥ A w − b ∥ 2 2 + λ r e g w T w \mathop{\operatorname{arg\,min}}_{\mathbf{w}}\lVert A\mathbf{w}-\mathbf{b}\rVert_2^2+\lambda_{\mathrm{reg}}\mathbf{w}^T\mathbf{w} arg min w ∥ A w − b ∥ 2 2 + λ reg w T w . λ r e g \lambda_{\mathrm{reg}} λ reg is the Tikhonov regularization parameter whose proper setting improves the conditioning of the problem. The system has a closed-form solution, either w = ( λ r e g I + A T A ) − 1 A T b \mathbf{w}=(\lambda_{\mathrm{reg}}I+A^TA)^{-1}A^T\mathbf{b} w = ( λ reg I + A T A ) − 1 A T b for over-determined system or A T ( λ r e g I + A A T ) − 1 b A^T(\lambda_{\mathrm{reg}}I+AA^T)^{-1}\mathbf{b} A T ( λ reg I + A A T ) − 1 b for under-determined one, enabling fast, gradient-free updates. See Appendix § 10 .§ 2 for the details.
Note that the Tikhonov-regularized formulation provides a closed-form result in a single computation step for any linear PDE. For nonlinear PDEs, the nonlinear terms must be linearized so that the system can be cast into a linear form. An initial guess is then required, followed by a few nonlinear iterations (a process that can also be interpreted as gradient-free fine-tuning) to update w \mathbf{w} w toward the optimal solution. Hereafter, we use N N N to denote the number of nonlinear iterations.
The proposed Newton-PINet model framework consists of two stages (Figure 1 ): (i) meta-learning on training tasks and (ii) gradient-free fine-tuning on new tasks.
Meta-learning on training tasks: The meta-learning objective is to optimize the network weights w ~ \widetilde{\mathbf{w}} w , importance hyperparameter λ p d e \lambda_{\mathrm{pde}} λ pde of PDE loss relative to IC/BC, and the regularization parameter λ r e g \lambda_{\mathrm{reg}} λ reg , collectively denoted Θ \Theta Θ , to learn task-agnostic representations spanning a family of PDEs and enable fast generalization to unseen tasks via fine-tuning only the output-layer weights using the Tikhonov regularization update. In our framework, the outer loop of meta-learning updates the learnable parameters via gradient descent with learning rate η \eta η , while task-specific adaptation in the inner loop, or generalization, is performed through gradient-free Tikhonov regularization applied to the output-layer weights. The meta-learning objective (outer-loop loss) can be either the physics-based least-squares error l L S E ( w ∗ ) = ∥ A w ∗ − b ∥ 2 2 l_{\mathrm{LSE}}(\mathbf{w}^*)=\lVert A\mathbf{w}^*-\mathbf{b}\rVert_2^2 l LSE ( w ∗ ) = ∥ A w ∗ − b ∥ 2 2 , or the data-driven mean squared error l M S E ( w ∗ ) = 1 n ∑ s ( u s l a b e l − f ( x ; w ~ ) T w ∗ ) 2 l_{\mathrm{MSE}}(w^*) = \frac{1}{n}\sum_s (u_s^{\mathrm{label}} - f(x;\widetilde{w})^T w^*)^2 l MSE ( w ∗ ) = n 1 ∑ s ( u s label − f ( x ; w ) T w ∗ ) 2 given labeled data { u s l a b e l } s = 1 n \{u_s^{\mathrm{label}}\}_{s=1}^n { u s label } s = 1 n where n n n is the total number of collocation points (PDE residual, IC, and BC). Here, Tikhonov regularization is used to compute the optimal task-specific output-layer weights w ∗ w^* w ∗ , enabling the model to specialize to any realization of the task. l L S E l_{\mathrm{LSE}} l LSE -based learning is termed unsupervised learning . l M S E l_{\mathrm{MSE}} l MSE -based learning is termed hybrid learning , since it couples physics-based Tikhonov updates in the inner loop with data-driven minimization in the outer loop. See Algorithm 1 for the pseudocode and Appendix B.3 for a detailed mathematical description.
Gradient-free fine-tuning to new tasks: After meta-learning, Θ \Theta Θ is fixed. For a new task T i ∼ p ( T ) \mathcal{T}_i \sim p(\mathcal{T}) T i ∼ p ( T ) with different PDE parameters or IC/BC, Tikhonov regularization is used to compute the task-specific output-layer weights, enabling fast gradient-free adaptation to the new PDE instance. Since this update is independent of gradient-based backpropagation optimizers such as stochastic gradient descent (SGD), the resulting adaptation is extremely fast while remaining physics-compliant. See Algorithm 2 for the pseudocode.
Algorithm 1 : Meta-learning on training tasks
Require: Initialize Θ; p(T ); η
Require: C, constructor of least-squares system (A, b); see Eq. (2)
while not done do
Sample training tasks {Ti} ∼ p(T )
for each task Ti do
Get initial guess u(1)
for k = 1 to N do
(A, b) = C(Ti, Θ, uk)
w = { (λregI + AᵀA)−1Aᵀb, or
Aᵀ(λregI + AAᵀ)−1b
u(k+1) = fᵀw
end for
Compute loss lTi
end for
Θ ← Θ − η∇Θ ∑i lTi
end while
Algorithm 2 : Gradient-free fine-tuning to new tasks
Require: Freeze Θ
for each test task Ti do
Get initial guess u(1)
for k = 1 to N do
(A, b) = C(Ti, Θ, uk)
w = { (λregI + AᵀA)−1Aᵀb, or
Aᵀ(λregI + AAᵀ)−1b
u(k+1) = fᵀw
end for
Store solution uTi = u(N+1)
end for
return all task solution {uTi}
Newton linearization: Note that the Tikhonov regularization applies to linear PDEs, where a linear matrix system can be constructed and solved in a single step. For nonlinear PDEs, the nonlinear terms must first be linearized so that the system can be cast into a linear form. The previous approaches typically relied on Picard method (lagging-of-coefficients), which linearizes the nonlinear terms using the solution from the previous iteration and then performs least-squares solves for multiple nonlinear iterations. Although simple, it is only linearly convergent [14 ] . To address this limitation, we use the Newton linearization approach to approximate the nonlinear term by Taylor expansion around the current iterate. For a function F F F of one state variable u u u , this gives F ( u k + 1 ) = F ( u k ) + d F d u ∣ k ( u k + 1 − u k ) + H.O.T. F(u^{k+1}) = F(u^k) + \left.\frac{dF}{du}\right|_k (u^{k+1} - u^k) + \text{H.O.T.} F ( u k + 1 ) = F ( u k ) + d u d F k ( u k + 1 − u k ) + H.O.T. , where k k k is the nonlinear iteration step, and H.O.T denotes the higher-order terms of Taylor expansion. For a function of two variables u u u and v v v , the expansion becomes F ( u k + 1 , v k + 1 ) = F ( u k , v k ) + ∂ F ∂ u ∣ k ( u k + 1 − u k ) + ∂ F ∂ v ∣ k ( v k + 1 − v k ) + H.O.T. F(u^{k+1},v^{k+1}) = F(u^k,v^k) + \left.\frac{\partial F}{\partial u}\right|_k (u^{k+1} - u^k) + \left.\frac{\partial F}{\partial v}\right|_k (v^{k+1} - v^k) + \text{H.O.T.} F ( u k + 1 , v k + 1 ) = F ( u k , v k ) + ∂ u ∂ F k ( u k + 1 − u k ) + ∂ v ∂ F k ( v k + 1 − v k ) + H.O.T. Based on this principle, the nonlinear terms commonly arising in PDEs can be expressed in a Newton-linearized form amenable to Tikhonov regularization. Table 1 summarizes several representative nonlinear terms derived by our work.
Table 1: Newton-linearized expressions of several nonlinear terms derived by our work. m m m denotes the exponent. k k k and k + 1 k+1 k + 1 refer to the current and next nonlinear iterations, respectively.
Table 1.
A detailed mathematical derivation is provided in Appendix B.5. In Appendix C, we further establish that the Newton linearization used in the Tikhonov-regularized PINN is essentially equivalent to the classical regularized Gauss-Newton method, with experiments validating its near-quadratic convergence.
Experiment results
We compare the performance of the proposed Newton-PINet (Newton linearization), PINet (Picard linearization) and purely data-driven deep neural network (DNN) on several representative classes of nonlinear PDEs:
(i) Nonlinear convection-type PDEs: ∂ u ∂ t + β u m ∂ u ∂ x − γ ∂ 2 u ∂ x 2 + δ ∂ 3 u ∂ x 3 + σ ∂ 4 u ∂ x 4 = 0 \frac{\partial u}{\partial t}+\beta u^m\frac{\partial u}{\partial x}-\gamma\frac{\partial^2u}{\partial x^2}+\delta\frac{\partial^3u}{\partial x^3}+\sigma\frac{\partial^4u}{\partial x^4}=0 ∂ t ∂ u + β u m ∂ x ∂ u − γ ∂ x 2 ∂ 2 u + δ ∂ x 3 ∂ 3 u + σ ∂ x 4 ∂ 4 u = 0 , where u ( x , t ) u(x,t) u ( x , t ) is the state variable, β , γ , δ , σ \beta,\gamma,\delta,\sigma β , γ , δ , σ are PDE parameters, and m m m denotes the nonlinearity order. We test three 1D time-dependent problems: Burgers, generalized Korteweg-de Vries (KdV), and Kuramoto-Sivashinsky (K-S) equations. In addition, we consider a 2D lid-driven cavity (LDC) flow governed by the Navier-Stokes equations. These problems span nonlinear convection, high-order dispersion/dissipation, and viscous flow with pressure-velocity coupling, providing canonical benchmarks for assessing generalization across different nonlinearities and dimensions.
(ii) Nonlinear forcing-type PDEs: ∂ u ∂ t + α ∇ u − γ Δ u + R ( u ) = q \frac{\partial u}{\partial t}+\alpha\nabla u-\gamma\Delta u+R(u)=q ∂ t ∂ u + α ∇ u − γ Δ u + R ( u ) = q , where R ( u ) R(u) R ( u ) denotes a nonlinear reaction operator, which may take polynomial, hyperbolic, exponential, or logarithmic forms, and q q q represents an external source term. We consider four 1D time-dependent problems: convection-diffusion-reaction (CDR), Klein-Gordon (K-G), hyperbolic heat, and logarithmic heat equations; two 2D problems: Helmholtz and parametric diffusion-reaction equations; and one 3D problem: parametric diffusion-reaction equation. These equations represent systems where diffusion and wave propagation interact with nonlinear reaction or source terms, making them central to the modeling of chemical kinetics, heat transfer, quantum fields, and complex reaction-diffusion phenomena in physics and engineering.
See Appendix § 12 .§ 1 for detailed descriptions of all PDE problems and meta-learning configurations (summarized in Table 8 ). See Appendix § 12 .§ 2 for details on data generation, error metrics, and computational setup. Note that, unless otherwise specified, the meta-learning outer-loop loss function for the Newton-PINet is l M S E l_{\mathrm{MSE}} l MSE (hybrid learning). We also adopt a temporal domain decomposition strategy within our meta-learning framework to leverage temporal causality [42 ] as a means of improving accuracy for complex time-dependent PDEs. Each time block is trained to model only short-term dynamics; however, iterative composition across blocks during inference ensures seamless integration over the full temporal horizon. Table 2 below summarizes the performance of the models on the nonlinear PDEs considered in this study.
Table 2. Summary of model performance on nonlinear PDEs in this study. The reported meta-learning and fine-tuning time correspond to the Newton-PINet. “NaN” entries under PINet indicate numerical overflow during computation. See Appendix § 12 .§ 1 for detailed descriptions of all PDE problems and meta-learning configurations (summarized in Table 8 ). See Appendix § 12 .§ 2 for details on data generation, error metrics, and computational setup.
Table 8. Summary of meta-learning parameter configurations on nonlinear PDEs in this study. For time-dependent PDEs, the mesh size is reported as t × x t\times x t × x , whereas for time-independent PDEs it is reported as x × y x\times y x × y (× z \times z × z ). “Mesh size” and “No. A rows” refer to settings in the meta-learning stage unless otherwise marked with “(test)”, which denotes the inference stage. “Dim.” denotes the dimension, and “No.” stands for “number of”. “–” indicates that the time-blocking strategy is not applicable for time-independent problems.
Learning to solve PDEs with nonlinear convection term
Burgers’ problems: ∂ u ∂ t + u ∂ u ∂ x − γ ∂ 2 u ∂ x 2 = 0 \frac{\partial u}{\partial t}+u\frac{\partial u}{\partial x}-\gamma\frac{\partial^2u}{\partial x^2}=0 ∂ t ∂ u + u ∂ x ∂ u − γ ∂ x 2 ∂ 2 u = 0 . We first fix the initial condition as u ( x , 0 ) = − sin ( π x ) u(x,0)=-\sin(\pi x) u ( x , 0 ) = − sin ( π x ) , and vary the viscosity parameter γ \gamma γ in the range [ 0.001 , 0.05 ] [0.001,0.05] [ 0.001 , 0.05 ] with an increment of 0.001 to generate 50 tasks. Among them, 16 are randomly selected for training, and the remaining 34 are used as test tasks. Under each condition, we keep the number of nonlinear iterations (N) for inner-loop fine-tuning in meta-learning stage consistent with those in generalization stage. We then compare Newton-PINet and PINet across different conditions, with the number of nonlinear iterations (N) ranging from 4 to 12. As shown in Figure 2 (a), with an increase of N, Newton-PINet achieves significantly faster convergence during meta-learning compared to PINet. Meanwhile, Newton-PINet achieves lower test errors than PINet. For example, when N = 6, the average MSE across all test tasks of PINet is 1.10 × 10 − 4 1.10\times10^{-4} 1.10 × 1 0 − 4 , whereas that of Newton-PINet is 1.79 × 10 − 6 1.79\times10^{-6} 1.79 × 1 0 − 6 (see Figure 2 (b) for the prediction plots). These results demonstrate that Newton linearization significantly improves the convergence of meta-learning, which in turn enhances generalization accuracy on test tasks.
Figure 2. Burgers’ problem. (a) Meta-learning convergence of Newton-PINet vs. PINet under different nonlinear iterations (N) (lines and shaded areas: the median convergence path and interquartile ranges of 5 random-seed runs). (b) Boxplot of error distributions across all test tasks and predictions by Newton-PINet (N=6).
Model comparison under varying initial conditions: We evaluate the model under the setting γ = 0.01 \gamma= 0.01 γ = 0.01 in Burgers’ problem, where the initial conditions are sampled from a Gaussian random field N ( 0 , 625 ( − Δ + 25 I ) − 4 ) \mathcal{N}(0,625(-\Delta+ 25I)^{-4}) N ( 0 , 625 ( − Δ + 25 I ) − 4 ) to construct both training and test tasks. We compare (i) unsupervised Newton-PINet (1 time block) and (ii) hybrid Newton-PINet (4 time blocks) against popular neural operator baselines in terms of training time and test error. The unsupervised operator considered is Physics-Only DeepONet (PO-DeepONet) [41 ] . The supervised operators include DeepONet [Lu et al., 2021] and Transformer-DeepONet with Trunk net enhanced by Fourier coefficients (T-DeepONet-TF) [47 ] , while the hybrid operator is PINO (a physicsinformed FNO) [30 ] . Note that PINO uses a training loss that combines both data-driven and PDE-based physics losses. The results are summarized in Table 3 . Compared to PO-DeepONet, unsupervised Newton-PINet reduces training time from 9.25h to 291s while improving test accuracy (in a data-absent scenario). Compared to supervised and hybrid neural operators, hybrid Newton-PINet consistently achieves superior test accuracy, with significantly reduced training cost. More results under different γ \gamma γ configurations are reported in Appendix Table 12.
Table 3. Comparison of training cost and performance at γ = 0.01 \gamma= 0.01 γ = 0.01 on Burgers’ problem.
Generalized KdV, K-S, and LDC problems: We further evaluate Newton-PINet on more complex nonlinear PDEs, including the generalized KdV equations with higher-order nonlinear terms u ∂ u ∂ x u \frac{\partial u}{\partial x} u ∂ x ∂ u , u 2 ∂ u ∂ x u^2 \frac{\partial u}{\partial x} u 2 ∂ x ∂ u , u 3 ∂ u ∂ x u^3 \frac{\partial u}{\partial x} u 3 ∂ x ∂ u , the K-S equations with nonlinear term u ∂ u ∂ x u \frac{\partial u}{\partial x} u ∂ x ∂ u , and the LDC problem governed by the Navier-Stokes equations. These PDEs pose highly challenging benchmarks for PINNs. The state-of-the-art PirateNets+SOAP [45 ] requires over 24 hours of training to solve a single K-S problem and must be retrained for each new problem. In contrast, our Newton-PINet completes the meta-learning stage in under half an hour , while single-task adaptation takes less than one second , demonstrating a clear advantage in computational efficiency. The results show that Newton-PINet achieves test errors of MSE = 1.94 × 10 − 3 = 1.94 \times10^{-3} = 1.94 × 1 0 − 3 on the generalized KdV, MSE = 3.45 × 10 − 2 = 3.45 \times10^{-2} = 3.45 × 1 0 − 2 on the K-S, and MSE = 1.67 × 10 − 4 = 1.67 \times10^{-4} = 1.67 × 1 0 − 4 on the LDC problem, outperforming PINet and DNN under the same configuration. Figure 5 , Figure 6 , Figure 7 show the prediction results of these problems. This demonstrates that our Newton linearization can improve the model performance across a variety of nonlinear convective PDEs. Note that the regularization parameter (λ r e g \lambda_{\mathrm{reg}} λ reg ) is meta-learned rather than manually tuned, and is shown to be robust across diverse test tasks (Appendix § 14 .§ 1 ). The initial guess (u g u e s s u_{\mathrm{guess}} u guess ) may affect convergence and Appendix § 14 .§ 2 provides simple strategies to mitigate this sensitivity. In addition, since the Tikhonov solution depends on A A A , Appendix § 14 .§ 3 analyzes its impact on inference time, showing that larger A A A increases the computational cost.
Figure 5. Prediction results of Newton-PINet for the generalized KdV problem.
Figure 6. Prediction results of Newton-PINet for the K-S problem.
Figure 7. Prediction result of Newton-PINet for the 2D lid-driven cavity problem. Velocity magnitude is computed as V = u 2 + v 2 V = \sqrt{u^2 + v^2} V = u 2 + v 2 .
Learning to solve PDEs with nonlinear forcing term
Convection-diffusion-reaction (CDR) problem: ∂ u ∂ t + α ∂ u ∂ x − γ ∂ 2 u ∂ x 2 − ρ 1 u + ρ 2 u 2 + ρ 3 u 3 = 0. \frac{\partial u}{\partial t} + \alpha\frac{\partial u}{\partial x} - \gamma\frac{\partial^2 u}{\partial x^2} - \rho_1 u + \rho_2 u^2 + \rho_3 u^3 = 0. ∂ t ∂ u + α ∂ x ∂ u − γ ∂ x 2 ∂ 2 u − ρ 1 u + ρ 2 u 2 + ρ 3 u 3 = 0. We generate 40 training and 160 test tasks by varying PDE parameters and initial conditions: α = 1 \alpha= 1 α = 1 , γ ∈ { 0.005 , 0.01 , 0.05 } \gamma\in\{0.005, 0.01, 0.05\} γ ∈ { 0.005 , 0.01 , 0.05 } , ρ 1 , ρ 2 ∈ { 0 , 1 , 3 , 5 } \rho_1, \rho_2 \in\{0, 1, 3, 5\} ρ 1 , ρ 2 ∈ { 0 , 1 , 3 , 5 } , ρ 3 = 5 \rho_3 = 5 ρ 3 = 5 , and u ( x , t = 0 ) = ∑ j = 1 J A j sin ( l j x + ϕ j ) u(x,t = 0) = \sum_{j=1}^{J} A_j \sin(l_j x + \phi_j) u ( x , t = 0 ) = ∑ j = 1 J A j sin ( l j x + ϕ j ) , where J = 5 J = 5 J = 5 , A j ∈ [ 0.1 , 0.5 ] A_j \in[0.1, 0.5] A j ∈ [ 0.1 , 0.5 ] , l j ∈ { 1 , 2 , 3 , 4 } l_j \in\{1, 2, 3, 4\} l j ∈ { 1 , 2 , 3 , 4 } , and ϕ j ∈ [ − π , π ] \phi_j \in[-\pi, \pi] ϕ j ∈ [ − π , π ] . We adopt a 4-time-block decomposition during meta-training and inference to improve accuracy. Figure 3 (a) and (b) show the meta-learning process and test results. It can be seen that PINet converges well during meta-learning, and its test error decreases as nonlinear iteration (N) increases. In contrast, Newton-PINet converges much faster and achieves lower test error. As shown in Figure 3 (c) and (d), for a comparable error level, Newton-PINet (N = 4) requires less than half the meta-training and fine-tuning time of PINet (N = 12). These results indicate that Newton linearization can substantially accelerate nonlinear iterations without significantly introducing additional time cost in either meta-learning or inference. More prediction results under various test conditions are provided in Appendix Figure 8. Note that different time-blocking strategies can affect model inference performance, as discussed in Appendix § 14 .§ 4 . As shown in Figure 3 , the predictions exhibit drift at temporal block boundaries. We can mitigate this drift by increasing the initial condition weight, as detailed in Appendix § 14 .§ 5 .
Figure 3. Convection-diffusion-reaction (CDR) problem. (a) Meta-learning convergence of Newton-PINet vs. PINet under different nonlinear iterations (N). (b,c) Test task MSE and runtime aggregated from 5 runs. In (b), the two circled regions correspond to Newton-PINet (N=4) and PINet (N=12), respectively. (d) Error distribution boxplot across all test tasks for different models; the prediction fields correspond to Newton-PINet (N=4).
Additional nonlinear forcing-type PDEs: We evaluate Newton-PINet on several other challenging nonlinear forcing-type PDEs: Klein-Gordon, hyperbolic/logarithmic heat, Helmholtz, and parametric diffusion-reaction equations. The results show that Newton-PINet outperforms the DNN and PINet baselines on nearly all problems. Notably, for problems with hyperbolic or logarithmic nonlinearities, the Picard approach used by PINet often fails to converge or leads to unstable training, whereas Newton-PINet remains stable and accurate. Additional results for these problems are provided in Appendix § 12 .§ 3 and Figure 9 . We further demonstrate the model’s applicability to different boundary condition types in Appendix § 14 .§ 6 .
Figure 9. Prediction results of Newton-PINet for the other 5 nonlinear PDE problems.
Model generalization to unseen boundary conditions and irregular geometries: We verify that the model, pretrained only on samples with Neumann boundary conditions, can effectively generalize to unseen tasks with Dirichlet and mixed boundary conditions (see Appendix § 12 .§ 3 and Table 9 ). Furthermore, the model, pretrained only on samples with regular geometry, can effectively generalize to test cases with complex and varying irregular geometries, as detailed in Appendix § 12 .§ 3 , Table 10 and Table 11 , Figure 11 and Figure 12 .
Table 9. Model generalization to unseen boundary condition (BC) for the 1D+time hyperbolic heat problem.
Table 10. Generalization to unseen irregular geometries and parameters for the 1D+time hyperbolic heat problem.
Table 11. Generalization to unseen irregular geometries and parameters for the 3D parametric diffusion-reaction problem.
Figure 11. Prediction results of Newton-PINet for the hyperbolic heat problem across varying domain shapes and PDE parameters, with meta-training performed on regular geometry.
Figure 12. Prediction results of Newton-PINet for the 3D parametric diffusion-reaction equation across varying domain shapes and PDE parameters, with meta-training performed on regular geometry.
Model comparison on a nonlinear reaction-diffusion problem: We further evaluate our method on a challenging 1D nonlinear reaction-diffusion problem ∂ u ∂ t − γ ∂ 2 u ∂ x 2 − ρ u ( 1 − u ) = 0 \frac{\partial u}{\partial t}-\gamma\frac{\partial^2u}{\partial x^2}-\rho u(1-u)=0 ∂ t ∂ u − γ ∂ x 2 ∂ 2 u − ρ u ( 1 − u ) = 0 [26 ] . In this benchmark, the initial condition is fixed, while the PDE parameters are sampled over γ ∈ [ 1 , 5 ] \gamma\in[1,5] γ ∈ [ 1 , 5 ] and ρ ∈ [ − 5 , 5 ] \rho\in[-5,5] ρ ∈ [ − 5 , 5 ] . The benchmark setup and baseline results are taken from the physics-informed neural solver (PI-neural-solver) [2 ] , which is trained on 800 labeled tasks and evaluated on 200 unseen tasks. In contrast, our Newton-PINet, under a hybrid training scheme, uses only 50 labeled tasks, i.e., 16 × 16\times 16 × fewer tasks, and is evaluated on the same test set. For a fair comparison, both models are trained and evaluated on an NVIDIA A6000 GPU. As shown in Table 4 , Newton-PINet achieves a test MSE that is approximately three orders of magnitude lower than that of the PI-neural-solver, while requiring only 301 seconds (5 minutes) of total meta-training wall-clock time, compared to several hours for the baseline. These results highlight the high data efficiency and computational efficiency of Newton-PINet. More detailed comparisons with additional baselines, as well as model performance under different GPU configurations, are provided in Appendix § 13 .§ 2 and E.3. Model prediction results are presented in Appendix Figure 13.
Table 4. Model comparison on a 1D nonlinear reaction-diffusion problem.
Comparison with other meta-learning PINN and neural operator methods: We further compare the performance of Newton-PINet with state-of-the-art gradient-based meta-learning PINNs [33 ] and gradient-free Baldwin-PINN [52 ] . Detailed results are provided in Appendix § 13 .§ 4 and Table 15 . On a parametric diffusion-reaction problem, Newton-PINet achieves approximately a 179 × 179\times 179 × improvement in generalization accuracy while requiring 7000 × 7000\times 7000 × less fine-tuning time compared with gradient-based meta-learning PINNs. Moreover, relative to Baldwinian-PINN, Newton-PINet reduces test MSE by 2∼5 orders of magnitude across benchmark problems and shortens the task-specific fine-tuning time by roughly one order of magnitude. These results highlight Newton-PINet’s superior generalization performance and computational speed. We next evaluate the performance of our method and state-of-the-art neural operators on the newly introduced PDE benchmarks, including the generalized KdV, K-S, CDR, and LDC problems. The results show that Newton-PINet achieves higher generalization accuracy than neural operators under identical training configurations. Detailed results are provided in Appendix § 13 .§ 5 and Table 17 . Furthermore, Appendix § 13 .§ 6 compares the inference time of Newton-PINet with the runtime of classical numerical solvers, demonstrating substantial speedups achieved by our model while maintaining comparable accuracy.
Table 15. Model comparison on nonlinear benchmark problems. Baselines are reported from Penwarden et al. [ [33 ] , Wong et al. [ [52 ] . “–” denotes results not reported in the references.
Table 17. Model comparison on the newly introduced benchmarks. Best and second-best results are highlighted in bold and underlined, respectively.
In Appendix § 14 .7, we demonstrate that Newton-PINet benefits significantly from skip connections, which enhance robustness to depth, width, and mesh resolution, thereby reducing the need for extensive hyperparameter tuning. In Appendix § 15 , we discuss Newton-PINet’s scalability. Briefly, since the features are generated by a neural representation, each row of A is dense rather than sparse in the numerical sense. The computational cost primarily depends on the expressiveness of pre-final layer features. By meta-learning a compact yet informative representation, the least-squares solve remains efficient even for large collocation sets.
Limitations & Outlook
Current experiments focus on 1D+time, 2D, and 3D cases. For more complex 2D/3D+time or multiphysics systems, larger, denser, and more ill-conditioned A matrix, combined with GPU memory limitations, hinders efficient solution. Future work will focus on developing low-rank strategies and scalable solvers for efficiently solving dense neural feature-based least-squares problems, thereby enhancing overall model performance. Additionally, for long-time prediction of chaotic systems such as K-S, we currently require 15 temporal blocks for accurate inference. Enabling single-shot prediction or reducing dependence on temporal decomposition remains an open challenge.
Conclusions
Newton-PINet has been validated on 1D, 2D, and 3D PDEs, as well as across diverse boundary conditions and domain geometries, covering a broad range of challenging nonlinear PDE operators. Newton-PINet demonstrates superior computational efficiency compared with state-of-the-art neural operators and meta-learning PINN methods. The main strengths of our model can be summarized as follows. (1) Computational efficiency: task adaptation in both meta-learning and inference is performed via least-squares updates of only the output layer. This gradient-free fine-tuning is faster than typical methods such as SGD, requiring just one step for linear PDEs and a few iterations for nonlinear PDEs. The Newton linearization further accelerates nonlinear solves. (2) Data efficiency: because the least-squares fine-tuning for new-task generalization is physics-informed, the meta-learning stage of our model requires very few labeled training samples and can converge in only a few epochs. (3) Accuracy: Tikhonov regularization stabilizes the least-squares solve, reducing ill-conditioning and improving generalization accuracy. Altogether, Newton-PINet achieves high generalization accuracy while requiring an order of magnitude fewer training tasks than state-of-the-art baselines. Task-specific inference on new tasks is also orders of magnitude faster compared to gradient-based meta-learning PINNs. These advantages make Newton-PINet a practical and scalable framework for learning large families of nonlinear PDEs in few-shot and real-time scenarios.
Acknowledgments and Disclosure of Funding
This research was in part supported by the National Research Foundation, Singapore under its AI-for-Science (AI4S) Catalytic Grant (Award No.: NRF-AI4SCT_2025-0007), and in part supported by the National Research Foundation, Singapore through the AI Singapore Programme, under the project “AI-based urban cooling technology development” (Award No.: AISG3-TC-2024-014-SGKR). The authors declare no competing interests.
Appendix
In this Appendix, we provide comprehensive materials to facilitate a deeper understanding of our study.
Appendix A provides an extended review of the literature.
Appendix B presents a detailed description of the Newton-PINet model, including the meta-learning and gradient-free fine-tuning framework, and the Newton linearization method.
Appendix C provides theoretical and experimental evidence showing that Newton linearization used in the Tikhonov-regularized PINN achieves near-quadratic convergence.
Appendix D describes the data generation, computational configurations, and additional experimental results.
Appendix E provides a comprehensive comparison of model performance on public datasets and newly introduced nonlinear benchmark problems.
Appendix F provides ablation studies covering meta-learning hyperparameters, factors influencing inference cost, model architecture design, and related aspects.
Appendix G presents an analysis of the computational complexity of the model and discusses its limitations and failure modes.
Extended review of related work
In this section, we provide an extended review of the literature and clarify its relation to our work.
Existing PINN architectures, such as DPM and PirateNets [25 , 44 ] , employ standard residual connections (e.g., x + f ( x ) x + f(x) x + f ( x ) ) to improve gradient flow and stabilize training. In contrast, our approach adopts a skip-connected design that concatenates features from all hidden layers and feeds them into the output layer, followed by a least-squares solve. This design aims to construct a richer basis space, where each hidden layer progressively contributes additional nonlinear features, thereby enhancing representation capacity and improving the conditioning and accuracy of the physics-informed Tikhonov solution, rather than merely stabilizing training.
Least-squares Gradient Descent (LSGD) [12 ] combines least-squares formulations with gradient-based optimization, focusing on data-driven single-task learning with linear structures. In contrast, our method extends least-squares formulations to meta-learning for nonlinear PDEs through a physics-informed Tikhonov framework. Specifically, the system is constructed from physical constraints (PDE residuals, ICs, and BCs), and Newton linearization transforms nonlinear residuals into local linear least-squares problems. This enables efficient gradient-free inner-loop adaptation with inherent enforcement of physical laws, which is absent in LSGD.
Methods such as Almost No Inner Loop (ANIL) [36 ] improve efficiency by restricting adaptation to the last layer. While sharing this idea, these approaches rely on gradient-based updates. In contrast, our method performs closed-form, physics-constrained least-squares updates via Tikhonov regularization. By eliminating inner-loop gradient descent and exploiting the algebraic structure of PDEs, our approach enables fast and stable adaptation to new tasks, fundamentally differing from conventional data-driven meta-learning methods.
Solving complex nonlinear PDEs via meta-learning presents fundamental challenges that require entirely different algorithmic considerations from traditional data-driven methods like LSGD or ANIL. By synergistically integrating basis-enriching skip-connections, Newton linearization, and physics-constrained Tikhonov least-squares, our work proposes a distinct physics-informed meta-learning framework tailored to the algebraic structure of physical systems, which is firmly backed by both theoretical and empirical analyses.
Newton-PINet model
This study presents a skip-connected Tikhonov regularization PINN model for meta-learning nonlinear PDEs, enhanced with Newton linearization, to enable efficient few-shot learning and fast generalization, hereafter referred to as Newton-PINet.
Skip-connected PINN architecture
Building upon the conventional feed-forward MLP architecture of PINNs, we introduce skip connections from all hidden layers to the output layer, with the output-layer weights computed using Tikhonov regularization.
The skip-connected neural architecture enables us to provide better expressivity to ensure stable and accurate least-squares computation by increasing the width of the output layer through the stacking of additional hidden layers, even while maintaining a moderate number of nodes per layer. In addition, the concatenation mechanism can be interpreted as expanding a richer basis space: each added hidden layer introduces new nonlinear features, analogous to incorporating higher-order terms in a polynomial or Chebyshev-type basis, leading to a more expressive representation with reduced truncation effects. A complementary viewpoint is that skip connections also help stabilize optimization—much like in ResNets—by mitigating vanishing-gradient issues in deeper architectures.
This architecture resembles extreme learning machines (ELMs) in its stacked structure [15 ] . However, in traditional ELMs, hidden-layer weights are randomly initialized, which makes training deeper architectures increasingly difficult and can compromise the accuracy of the solution. In contrast, our framework employs a meta-learning strategy to update the hidden-layer weights instead of relying on random initialization. As a result, the network depth can be increased appropriately to improve nonlinear representation capacity while maintaining stable and accurate generalization.
We employ the Tikhonov-regularized PINN with L + 1 L+1 L + 1 layers, where the input is x ∈ R D i n \mathbf{x} \in\mathbb{R}^{D_{in}} x ∈ R D in (layer 0) and the output is u ∈ R D o \mathbf{u} \in\mathbb{R}^{D_o} u ∈ R D o (layer L L L ). Here, D i n D_{in} D in and D o D_o D o denote the dimensionality of the input and output variables, respectively. Each hidden layer l = 1 , … , L − 1 l=1,\ldots,L-1 l = 1 , … , L − 1 contains the same number of neurons (N n N_n N n ). Sinusoidal feature embeddings are applied at the first layer [49 ] , and all hidden layers use sin ( ⋅ ) \sin(\cdot) sin ( ⋅ ) activation functions. Skip connections from all hidden layers to the output layer are employed to allow each hidden layer to contribute nonlinearly to the final representation without additional weights in the concatenation.
The transformation and activation in the first hidden layer are defined as
z j 1 = ∑ d = 1 D i n W j , d 1 x d + b j 1 , f j 1 ( x ) = sin ( 2 π z j 1 ) , j = 1 , … , N n , (3) z_j^1=\sum_{d=1}^{D_{in}}W_{j,d}^1x_d+b_j^1,\qquad f_j^1(\mathbf{x})=\sin(2\pi z_j^1),\qquad j=1,\ldots,N_n, \tag*{(3)} z j 1 = d = 1 ∑ D in W j , d 1 x d + b j 1 , f j 1 ( x ) = sin ( 2 π z j 1 ) , j = 1 , … , N n , (3)
where j j j and d d d denote neurons in the current and previous layers respectively, W 1 ∈ R N n × D i n W^1 \in\mathbb{R}^{N_n\times D_{in}} W 1 ∈ R N n × D in , and b 1 ∈ R N n b^1 \in\mathbb{R}^{N_n} b 1 ∈ R N n . We incorporate Sinusoidal feature embeddings in the first layer to enhance the representation of high-frequency components.
For subsequent hidden layers l = 2 , … , L − 1 l=2,\ldots,L-1 l = 2 , … , L − 1 , we have
z j l = ∑ d = 1 N n W j , d l f d l − 1 + b j l , f j l ( x ) = sin ( z j l ) , j = 1 , … , N n , (4) z_j^l=\sum_{d=1}^{N_n}W_{j,d}^l f_d^{l-1}+b_j^l,\qquad f_j^l(\mathbf{x})=\sin(z_j^l),\qquad j=1,\ldots,N_n, \tag*{(4)} z j l = d = 1 ∑ N n W j , d l f d l − 1 + b j l , f j l ( x ) = sin ( z j l ) , j = 1 , … , N n , (4)
where W l ∈ R N n × N n W^l \in\mathbb{R}^{N_n\times N_n} W l ∈ R N n × N n and b l ∈ R N n b^l \in\mathbb{R}^{N_n} b l ∈ R N n .
All trainable network parameters up to the pre-final layer are denoted as w ~ = [ W l , b l ] \tilde{\mathbf{w}}=[W^l,b^l] w ~ = [ W l , b l ] , l = 1 , … , L − 1 l=1,\ldots,L-1 l = 1 , … , L − 1 .
The pre-final output is constructed as the concatenation of all hidden activations (skip connections):
f ( x ; w ~ ) T = [ f 1 1 , … , f N n 1 , f 1 2 , … , f N n 2 , … , f 1 L − 1 , … , f N n L − 1 ] , (5) f(\mathbf{x};\tilde{\mathbf{w}})^{T}=\left[f_1^1,\ldots,f_{N_n}^1,\ f_1^2,\ldots,f_{N_n}^2,\ \ldots,\ f_1^{L-1},\ldots,f_{N_n}^{L-1}\right], \tag*{(5)} f ( x ; w ~ ) T = [ f 1 1 , … , f N n 1 , f 1 2 , … , f N n 2 , … , f 1 L − 1 , … , f N n L − 1 ] , (5)
which is then connected to the output layer with D o D_o D o dimension, so that f ( x ; w ~ ) T ∈ R D o × N n ( L − 1 ) f(\mathbf{x};\tilde{\mathbf{w}})^T \in\mathbb{R}^{D_o\times N_n(L-1)} f ( x ; w ~ ) T ∈ R D o × N n ( L − 1 ) . Then, we need to flatten it, resulting in f ( x ; w ~ ) T ∈ R 1 × D o N n ( L − 1 ) f(\mathbf{x};\tilde{\mathbf{w}})^T \in\mathbb{R}^{1\times D_oN_n(L-1)} f ( x ; w ~ ) T ∈ R 1 × D o N n ( L − 1 ) .
The output u u u can be computed by
u ( x ) = ∑ j w j f j = f ( x ; w ~ ) T w , (6) u(\mathbf{x}) = \sum_j w_j f_j = \mathbf{f}(\mathbf{x};\widetilde{\mathbf{w}})^T\mathbf{w}, \tag*{(6)} u ( x ) = j ∑ w j f j = f ( x ; w ) T w , (6)
where w T = [ … w j … ] ∈ R 1 × D o N n ( L − 1 ) \mathbf{w}^T = [\ldots w_j \ldots] \in\mathbb{R}^{1 \times D_oN_n(L-1)} w T = [ … w j … ] ∈ R 1 × D o N n ( L − 1 ) denotes the output-layer weights.
The objective of task-specific learning is to determine the best set of w \mathbf{w} w such that u ( x ) = f ( x ; w ~ ) T w u(\mathbf{x}) = \mathbf{f}(\mathbf{x};\widetilde{\mathbf{w}})^T\mathbf{w} u ( x ) = f ( x ; w ) T w satisfies the PDE, IC, and BC, for a task (T i \mathcal{T}_i T i ):
P D E : N θ [ u ( x , t ) ] = q ( x , t ) x ∈ Ω , t ∈ [ 0 , T ] , I C : u ( x , t = 0 ) = u 0 ( x ) x ∈ Ω , B C : B [ u ( x , t ) ] = g ( x , t ) x ∈ ∂ Ω , t ∈ [ 0 , T ] , (7) \begin{aligned}
\mathrm{PDE}: \quad&\mathcal{N}_{\theta}[u(\mathbf{x},t)] = q(\mathbf{x},t) && \mathbf{x} \in\Omega,\ t \in[0,T], \\
\mathrm{IC}: \quad&u(\mathbf{x},t=0) = u_0(\mathbf{x}) && \mathbf{x} \in\Omega, \\
\mathrm{BC}: \quad&\mathcal{B}[u(\mathbf{x},t)] = g(\mathbf{x},t) && \mathbf{x} \in\partial\Omega,\ t \in[0,T], \tag*{(7)}
\end{aligned} PDE : IC : BC : N θ [ u ( x , t )] = q ( x , t ) u ( x , t = 0 ) = u 0 ( x ) B [ u ( x , t )] = g ( x , t ) x ∈ Ω , t ∈ [ 0 , T ] , x ∈ Ω , x ∈ ∂ Ω , t ∈ [ 0 , T ] , (7)
where N θ \mathcal{N}_{\theta} N θ , q q q , u 0 u_0 u 0 , g g g denote the differential operator with PDE parameters θ \theta θ , the source term, IC, and BC, respectively.
This leads to a physics-based least-squares formulation. Let x = ( x , t ) \mathbf{x}=(x,t) x = ( x , t ) . Given collocation points for the PDE residual, ( x n p d e , t n p d e ) (x_n^{\mathrm{pde}},t_n^{\mathrm{pde}}) ( x n pde , t n pde ) , n = 1 , … , n p d e n=1,\ldots,n_{\mathrm{pde}} n = 1 , … , n pde ; points at the initial time, ( x n i c , 0 ) (x_n^{\mathrm{ic}},0) ( x n ic , 0 ) , n = 1 , … , n i c n=1,\ldots,n_{\mathrm{ic}} n = 1 , … , n ic ; and points on the boundary, ( x n b c , t n b c ) (x_n^{\mathrm{bc}},t_n^{\mathrm{bc}}) ( x n bc , t n bc ) , n = 1 , … , n b c n=1,\ldots,n_{\mathrm{bc}} n = 1 , … , n bc , where n p d e n_{\mathrm{pde}} n pde , n i c n_{\mathrm{ic}} n ic , and n b c n_{\mathrm{bc}} n bc denote the numbers of the corresponding points, respectively. The loss importance hyperparameter λ p d e \lambda_{\mathrm{pde}} λ pde is further introduced, and the following system is obtained:
\begin{bmatrix}
\ldots& \lambda_{\mathrm{pde}}\cdot\mathcal{N}_{\theta}[f_j(x_1^{\mathrm{pde}},t_1^{\mathrm{pde}};\widetilde{\mathbf{w}})] & \ldots\\
& \vdots& \\
\ldots& \lambda_{\mathrm{pde}}\cdot\mathcal{N}_{\theta}[f_j(x_{n_{\mathrm{pde}}}^{\mathrm{pde}},t_{n_{\mathrm{pde}}}^{\mathrm{pde}};\widetilde{\mathbf{w}})] & \ldots\\
\ldots& f_j(x_1^{\mathrm{ic}},0;\widetilde{\mathbf{w}}) & \ldots\\
& \vdots& \\
\ldots& f_j(x_{n_{\mathrm{ic}}}^{\mathrm{ic}},0;\widetilde{\mathbf{w}}) & \ldots\\
\ldots& \mathcal{B}[f_j(x_1^{\mathrm{bc}},t_1^{\mathrm{bc}};\widetilde{\mathbf{w}})] & \ldots\\
& \vdots& \\
\ldots& \mathcal{B}[f_j(x_{n_{\mathrm{bc}}}^{\mathrm{bc}},t_{n_{\mathrm{bc}}}^{\mathrm{bc}};\widetilde{\mathbf{w}})] & \ldots
\end{bmatrix}
\begin{bmatrix}
\vdots\\
w_j \\
\vdots
\end{bmatrix}
=
\begin{bmatrix}
\lambda_{\mathrm{pde}}\cdot q(x_1^{\mathrm{pde}},t_1^{\mathrm{pde}}) \\
\vdots\\
\lambda_{\mathrm{pde}}\cdot q(x_{n_{\mathrm{pde}}}^{\mathrm{pde}},t_{n_{\mathrm{pde}}}^{\mathrm{pde}}) \\
u_0(x_1^{\mathrm{ic}}) \\
\vdots\\
u_0(x_{n_{\mathrm{ic}}}^{\mathrm{ic}}) \\
g(x_1^{\mathrm{bc}},t_1^{\mathrm{bc}}) \\
\vdots\\
g(x_{n_{\mathrm{bc}}}^{\mathrm{bc}},t_{n_{\mathrm{bc}}}^{\mathrm{bc}})
\end{bmatrix}, \tag*{(8)}\end{bmatrix}
which can be compactly written as
A w = b . A\mathbf{w}=\mathbf{b}. A w = b .
Here A w A\mathbf{w} A w represents the left-hand side of Eq. (7) at a given set of collocation points, b \mathbf{b} b denotes the corresponding right-hand side of Eq. (7) .
Therefore, using the PDE, IC, and BC, the physics-informed matrix A A A and vector b \mathbf{b} b can be assembled. Concretely, the pre-final representation f \mathbf{f} f is obtained via forward propagation, and its derivatives with respect to ( x , t ) (x,t) ( x , t ) in N θ \mathcal{N}_{\theta} N θ are computed through automatic differentiation (AD).
For a single task, i.e., a single PDE instance, the number of rows in A A A and b \mathbf{b} b equals the total number of collocation points, including those for the PDE residual, IC, and BC. The number of columns in A A A corresponds to the product of the pre-final feature dimension N n ( L − 1 ) N_n(L-1) N n ( L − 1 ) and the output dimension D o D_o D o . This results in A ∈ R ( n p d e + n i c + n b c ) × D o N n ( L − 1 ) A \in\mathbb{R}^{(n_{\mathrm{pde}}+n_{\mathrm{ic}}+n_{\mathrm{bc}})\times D_oN_n(L-1)} A ∈ R ( n pde + n ic + n bc ) × D o N n ( L − 1 ) and b ∈ R ( n p d e + n i c + n b c ) × 1 \mathbf{b} \in\mathbb{R}^{(n_{\mathrm{pde}}+n_{\mathrm{ic}}+n_{\mathrm{bc}})\times1} b ∈ R ( n pde + n ic + n bc ) × 1 .
The final output weights w \mathbf{w} w are obtained via Tikhonov regularization:
arg min w ∥ A w − b ∥ 2 2 + λ r e g w T w , (9) \underset{\mathbf{w}}{\operatorname{arg\,min}}\ \|A\mathbf{w}-\mathbf{b}\|_2^2+\lambda_{\mathrm{reg}}\mathbf{w}^T\mathbf{w}, \tag*{(9)} w arg min ∥ A w − b ∥ 2 2 + λ reg w T w , (9)
where λ r e g ≥ 0 \lambda_{\mathrm{reg}} \ge0 λ reg ≥ 0 denotes the regularization parameter. Compared with ordinary least squares (OLS), which minimizes only the residual norm ∥ A w − b ∥ 2 2 \|A\mathbf{w}-\mathbf{b}\|_2^2 ∥ A w − b ∥ 2 2 , Tikhonov regularization introduces an additional penalty term λ r e g w T w \lambda_{\mathrm{reg}}\mathbf{w}^T\mathbf{w} λ reg w T w . This term suppresses large weights, stabilizes the solution, and improves the conditioning of the problem.
Note that for linear PDE, the closed-form Tikhonov-regularized solution of Eq. (8) can be obtained in a single step:
w = { ( λ r e g I + A T A ) − 1 A T b , if xref‘Eq. (8)‘ is over-determined , A T ( λ r e g I + A A T ) − 1 b , if xref‘Eq. (8)‘ is under-determined . (10) w =
\begin{cases}
(\lambda_{\mathrm{reg}}I + A^{T}A)^{-1}A^{T}b, & \text{if {xref}`Eq. (8)` is over-determined}, \\
A^{T}(\lambda_{\mathrm{reg}}I + AA^{T})^{-1}b, & \text{if {xref}`Eq. (8)` is under-determined}.
\end{cases} \tag*{(10)} w = { ( λ reg I + A T A ) − 1 A T b , A T ( λ reg I + A A T ) − 1 b , if xref ‘Eq. (8)‘ is over-determined , if xref ‘Eq. (8)‘ is under-determined . (10)
For nonlinear PDEs, the nonlinear terms must first be linearized so that the system can be cast in the form of Eq. (8) . For example, in the Burgers’ equation: u t + u u x − γ u x x = 0 u_t + uu_x - \gamma u_{xx} = 0 u t + u u x − γ u xx = 0 , we can compute
u = f T w , u t = ∂ u ∂ t = f t T w , u x = ∂ u ∂ x = f x T w , u x x = ∂ 2 u ∂ x 2 = f x x T w , u = f^{T}w,\quad u_t = \frac{\partial u}{\partial t} = f_t^{T}w,\quad u_x = \frac{\partial u}{\partial x} = f_x^{T}w,\quad u_{xx} = \frac{\partial^2 u}{\partial x^2} = f_{xx}^{T}w, u = f T w , u t = ∂ t ∂ u = f t T w , u x = ∂ x ∂ u = f x T w , u xx = ∂ x 2 ∂ 2 u = f xx T w ,
where the pre-final representation f f f and its derivatives can be obtained via forward propagation and AD. However, the nonlinear term u u x uu_x u u x cannot be directly expressed as a combination of f f f (or f x f_x f x ) and w w w , which motivates the need for linearization. Under Newton linearization, the nonlinear term can be approximated as
( u u x ) k + 1 ≈ u k + 1 u x k + u k u x k + 1 − u k u x k , (uu_x)^{k+1} \approx u^{k+1}u_x^k + u^k u_x^{k+1} - u^k u_x^k, ( u u x ) k + 1 ≈ u k + 1 u x k + u k u x k + 1 − u k u x k ,
where k k k and k + 1 k+1 k + 1 denote the current and next nonlinear iteration step, respectively. Given initial guesses for u u u and u x u_x u x (e.g., setting u u u and u x u_x u x at all time steps equal to the initial field), the corresponding entries in A A A and b b b are updated as:
N θ [ f j k + 1 ( x n p d e ; w ~ ) ] = ∂ f j k + 1 ( x n p d e ; w ~ ) ∂ t + u k ( x n p d e ) ∂ f j k + 1 ( x n p d e ; w ~ ) ∂ x + f j k + 1 ( x n p d e ; w ~ ) ∂ u k ( x n p d e ) ∂ x − γ ∂ 2 f j k + 1 ( x n p d e ; w ~ ) ∂ x 2 , (11) \mathcal{N}_{\theta}\left[f_j^{k+1}(x_n^{\mathrm{pde}};\widetilde{w})\right] =
\frac{\partial f_j^{k+1}(x_n^{\mathrm{pde}};\widetilde{w})}{\partial t}
+ u^k(x_n^{\mathrm{pde}})\frac{\partial f_j^{k+1}(x_n^{\mathrm{pde}};\widetilde{w})}{\partial x}
+ f_j^{k+1}(x_n^{\mathrm{pde}};\widetilde{w})\frac{\partial u^k(x_n^{\mathrm{pde}})}{\partial x}
- \gamma\frac{\partial^2 f_j^{k+1}(x_n^{\mathrm{pde}};\widetilde{w})}{\partial x^2}, \tag*{(11)} N θ [ f j k + 1 ( x n pde ; w ) ] = ∂ t ∂ f j k + 1 ( x n pde ; w ) + u k ( x n pde ) ∂ x ∂ f j k + 1 ( x n pde ; w ) + f j k + 1 ( x n pde ; w ) ∂ x ∂ u k ( x n pde ) − γ ∂ x 2 ∂ 2 f j k + 1 ( x n pde ; w ) , (11)
q ( x n p d e ) = u k ( x n p d e ) ∂ u k ( x n p d e ) ∂ x . (12) q(x_n^{\mathrm{pde}}) = u^k(x_n^{\mathrm{pde}})\frac{\partial u^k(x_n^{\mathrm{pde}})}{\partial x}. \tag*{(12)} q ( x n pde ) = u k ( x n pde ) ∂ x ∂ u k ( x n pde ) . (12)
This leads to an iterative nonlinear solve, where Eq. (10) is repeatedly solved to update w w w toward the optimal solution.
How to make Tikhonov solver tractable? The task adaptation is formulated as a Tikhonov-regularized least-squares problem, whose tractability mainly depends on the size of A A A , the Tikhonov regularization parameter (λ r e g \lambda_{\mathrm{reg}} λ reg ), the PDE-loss importance hyperparameter (λ p d e \lambda_{\mathrm{pde}} λ pde ), and the choice of the initial field during nonlinear iterations.
The matrix A A A is constructed from the network features at each sampling point of the PDE residuals/ICs/BCs, which are generated by a multilayer neural representation with skip connections. The distribution of these features directly determines the conditioning of the solve. In addition, when the system requires a large number of sampling points, the number of rows in A A A increases, making it more difficult to solve. Since the Tikhonov update only involves A T A ∈ R D o N n ( L − 1 ) × D o N n ( L − 1 ) A^{T}A \in\mathbb{R}^{D_oN_n(L-1)\times D_oN_n(L-1)} A T A ∈ R D o N n ( L − 1 ) × D o N n ( L − 1 ) , the computational cost mainly depends on the network feature dimension. Thus, the practical scaling for large collocation sets may derive more from the fact that one can still control the cost of the solve through meta-learning a better (yet minimal) set of pre-final layer features.
For nonlinear problems, an inappropriate initial field may slow convergence. Therefore, we use the initial condition as u g u e s s u_{\mathrm{guess}} u guess (setting u g u e s s u_{\mathrm{guess}} u guess at all time steps equal to the initial field) for time-dependent problems and employ temporal domain decomposition for more complex dynamics, ensuring that each block starts closer to the true solution, which improves the stability of the least-squares updates.
The specific discussion on the scalability of the model is provided in Appendix § 15 .
Meta-learning PINN on training tasks
The trainable hyperparameters during meta-learning include network weights (w ~ \widetilde{w} w ), PDE loss importance hyperparameter (λ p d e \lambda_{\mathrm{pde}} λ pde ), and regularization parameter (λ r e g \lambda_{\mathrm{reg}} λ reg ), collectively denoted as Θ \Theta Θ . These components are crucial for achieving optimal performance in task-specific physics-informed learning.
The meta-learning objective is to optimize Θ \Theta Θ to learn task-agnostic representations spanning a family of PDEs, enabling fast generalization to unseen tasks via fine-tuning only the output-layer weights using the Tikhonov regularization update. In our framework, the outer loop of meta-learning updates the learnable parameters via gradient descent, while task-specific adaptation in the inner loop, or generalization, is performed through gradient-free Tikhonov regularization applied to the output-layer weights. See Algorithm 1 for the pseudocode workflow.
We consider a distribution over tasks p ( T ) p(\mathcal{T}) p ( T ) that we want our model to be able to adapt to. The meta-objective (loss) may consist solely of physics-based least-squares error (l L S E l_{\mathrm{LSE}} l LSE , unsupervised), data-driven mean squared error (l M S E l_{\mathrm{MSE}} l MSE , few-shot training), or a combination of both:
min Θ E T i ∼ p ( T ) E w ~ ∼ p θ ( w ~ ) [ l L S E ( w ∗ ) + l M S E ( w ∗ ) ] . (13) \min_{\Theta}\mathbb{E}_{\mathcal{T}_i\sim p(\mathcal{T})}\mathbb{E}_{\widetilde{w}\sim p_{\theta}(\widetilde{w})}\left[l_{\mathrm{LSE}}(w^*)+l_{\mathrm{MSE}}(w^*)\right]. \tag*{(13)} Θ min E T i ∼ p ( T ) E w ∼ p θ ( w ) [ l LSE ( w ∗ ) + l MSE ( w ∗ ) ] . (13)
Here, Tikhonov regularization is used to obtain the optimal output-layer weights w ∗ w^* w ∗ that allow the model to specialize to any realization of task T i ∼ p ( T ) \mathcal{T}_i\sim p(\mathcal{T}) T i ∼ p ( T ) for the given network’s w ~ ∼ p θ ( w ~ ) \widetilde{w}\sim p_{\theta}(\widetilde{w}) w ∼ p θ ( w ) .
l L S E l_{\mathrm{LSE}} l LSE and l M S E l_{\mathrm{MSE}} l MSE are
l L S E ( w ∗ ) = ( A w ∗ − b ) T ( A w ∗ − b ) , (14) l_{\mathrm{LSE}}(w^*)=(Aw^*-b)^{\mathsf{T}}(Aw^*-b), \tag*{(14)} l LSE ( w ∗ ) = ( A w ∗ − b ) T ( A w ∗ − b ) , (14)
l M S E ( w ∗ ) = 1 n ∑ s = 1 n ( u s l a b e l − ∑ j w j ∗ f j ( x s , t s ; w ~ ) ) 2 , (15) l_{\mathrm{MSE}}(w^*)=\frac{1}{n}\sum_{s=1}^{n}\left(u_s^{\mathrm{label}}-\sum_j w_j^* f_j(x_s,t_s;\widetilde{w})\right)^2, \tag*{(15)} l MSE ( w ∗ ) = n 1 s = 1 ∑ n ( u s label − j ∑ w j ∗ f j ( x s , t s ; w ) ) 2 , (15)
given labeled data { u s l a b e l } s = 1 n \{u_s^{\mathrm{label}}\}_{s=1}^{n} { u s label } s = 1 n (n n n denotes the total number of collocation points including PDE residual, IC, and BC constraints for a task).
We refer to l L S E l_{\mathrm{LSE}} l LSE -based learning as unsupervised learning , and to l M S E l_{\mathrm{MSE}} l MSE -based learning as hybrid learning (a combination of physics-based Tikhonov updates and data-driven loss). Unless otherwise stated, for meta-learning loss (outer-loop), our model does not consider the combined loss formulation l L S E + l M S E l_{\mathrm{LSE}}+l_{\mathrm{MSE}} l LSE + l MSE , as in some cases the combined objective exhibits poorer generalization performance compared with using l M S E l_{\mathrm{MSE}} l MSE alone.
In practice, we observe that for simple nonlinear PDEs (e.g., Burgers’ equation), training with l L S E l_{\mathrm{LSE}} l LSE alone already leads to strong generalization. However, for more complex PDEs, incorporating a small amount of labeled data through the l M S E l_{\mathrm{MSE}} l MSE term becomes essential for both stability and accuracy.
Gradient-free fine-tuning to new tasks
After meta-learning, the network weights (w ~ \widetilde{w} w ), the PDE-loss importance hyperparameter (λ p d e \lambda_{\mathrm{pde}} λ pde ), and the regularization parameter (λ r e g \lambda_{\mathrm{reg}} λ reg ) are fixed. For a new task with different PDE parameters or IC/BC conditions, Tikhonov regularization is used to compute the output-layer weights (w w w ), enabling fast gradient-free adaptation to the new PDE instance. See Algorithm 2 for the pseudocode workflow. Since this update is independent of gradient-based backpropagation optimizers such as stochastic gradient descent (SGD), the resulting adaptation is extremely fast while remaining physics-compliant.
Linearization for nonlinear PDEs
It should be noted that the Tikhonov regularization is directly applicable to linear PDEs, where a linear matrix system can be constructed and solved in a single step. For nonlinear PDEs, the nonlinear terms must first be linearized so that the system can be cast into a linear form. The previous approaches typically relied on Picard method (lagging-of-coefficients), which linearizes the nonlinear terms using the solution from the previous iteration and then performs least-squares solves for multiple nonlinear iterations.
Picard iterations. Take the Burgers’ equation as an example: u t + u u x − ν u x x = 0 u_t+uu_x-\nu u_{xx}=0 u t + u u x − ν u xx = 0 . The Picard method approximately linearizes the nonlinear term as: ( u u x ) k + 1 = u k u x k + 1 (uu_x)^{k+1}=u^k u_x^{k+1} ( u u x ) k + 1 = u k u x k + 1 , where k k k and k + 1 k+1 k + 1 denote the current and next iteration steps, respectively. An initial guess for u u u is provided, e.g., setting u u u at all time steps equal to the initial field. Under this formulation, the entries of the least-squares matrix A A A and vector b b b in Eq. (8) corresponding to the ( k + 1 ) (k+1) ( k + 1 ) -th iteration, n n n -th PDE collocation point, and the j j j -th output-layer neuron are given by:
N θ [ f j k + 1 ( x n p d e ; w ~ ) ] = ∂ f j k + 1 ( x n p d e ; w ~ ) ∂ t + u k ( x n p d e ) ∂ f j k + 1 ( x n p d e ; w ~ ) ∂ x − γ ∂ 2 f j k + 1 ( x n p d e ; w ~ ) ∂ x 2 (16) \mathcal{N}_{\theta}\left[f_j^{k+1}\left(\mathbf{x}_n^{\mathrm{pde}};\widetilde{\mathbf{w}}\right)\right] = \frac{\partial f_j^{k+1}\left(\mathbf{x}_n^{\mathrm{pde}};\widetilde{\mathbf{w}}\right)}{\partial t} + u^k\left(\mathbf{x}_n^{\mathrm{pde}}\right)\frac{\partial f_j^{k+1}\left(\mathbf{x}_n^{\mathrm{pde}};\widetilde{\mathbf{w}}\right)}{\partial x} - \gamma\frac{\partial^2 f_j^{k+1}\left(\mathbf{x}_n^{\mathrm{pde}};\widetilde{\mathbf{w}}\right)}{\partial x^2} \tag*{(16)} N θ [ f j k + 1 ( x n pde ; w ) ] = ∂ t ∂ f j k + 1 ( x n pde ; w ) + u k ( x n pde ) ∂ x ∂ f j k + 1 ( x n pde ; w ) − γ ∂ x 2 ∂ 2 f j k + 1 ( x n pde ; w ) (16)
q ( x n p d e ) = 0 , (17) q\left(\mathbf{x}_n^{\mathrm{pde}}\right)=0, \tag*{(17)} q ( x n pde ) = 0 , (17)
where f j f_j f j denotes the pre-final output of the neural network. Note that u k u^k u k here is the known solution, either the initial guess or the solution at the k k k -th iteration.
The final output weights w T = [ … , w j , … ] \mathbf{w}^{T}=[\ldots,w_j,\ldots] w T = [ … , w j , … ] are obtained via Tikhonov regularization-based nonlinear iterations. At each iteration, the solution is updated by u ( x ) = ∑ j w j f j u(x)=\sum_j w_jf_j u ( x ) = ∑ j w j f j , which then serves as the new guess for the next iteration. This procedure is repeated until w \mathbf{w} w satisfies a prescribed convergence criterion or the maximum number of iterations is reached.
Newton linearization iterations. The Picard approach suffers from slow convergence of nonlinear PDE solutions due to its linear convergence speed. To overcome these limitations, we use the Newton linearization approach.
Let F ( u k + 1 ) F(u^{k+1}) F ( u k + 1 ) denote the nonlinear term as a function of the state variable u u u . In Newton linearization, a first-order Taylor expansion is performed around the current iterate u k u^k u k , yielding
F ( u k + 1 ) = F ( u k ) + d F d u ∣ k ( u k + 1 − u k ) + H.O.T. , (18) F(u^{k+1}) = F(u^k) + \left.\frac{dF}{du}\right|^k (u^{k+1}-u^k) + \text{H.O.T.}, \tag*{(18)} F ( u k + 1 ) = F ( u k ) + d u d F k ( u k + 1 − u k ) + H.O.T. , (18)
where “H.O.T” denotes the higher-order truncated terms.
According to Eq. (18) , we can derive a general expression for the nonlinear terms u m u^m u m with m m m denoting the exponent that commonly appear in PDEs, as follows:
( u m ) k + 1 ≈ ( u m ) k + m ( u m − 1 ) k ( u k + 1 − u k ) = m ( u k ) m − 1 u k + 1 + ( 1 − m ) ( u k ) m (19) \begin{aligned}
(u^m)^{k+1} &\approx(u^m)^k + m(u^{m-1})^k(u^{k+1}-u^k) \\
&= m(u^k)^{m-1}u^{k+1} + (1-m)(u^k)^m \tag*{(19)}
\end{aligned} ( u m ) k + 1 ≈ ( u m ) k + m ( u m − 1 ) k ( u k + 1 − u k ) = m ( u k ) m − 1 u k + 1 + ( 1 − m ) ( u k ) m (19)
The corresponding expressions for m = 2 m=2 m = 2 and m = 3 m=3 m = 3 are as follows:
( u 2 ) k + 1 ≈ 2 u k + 1 u k − u k u k (20) (u^2)^{k+1} \approx2u^{k+1}u^k-u^ku^k \tag*{(20)} ( u 2 ) k + 1 ≈ 2 u k + 1 u k − u k u k (20)
( u 3 ) k + 1 ≈ 3 u k + 1 u k u k − 2 u k u k u k (21) (u^3)^{k+1} \approx3u^{k+1}u^ku^k-2u^ku^ku^k \tag*{(21)} ( u 3 ) k + 1 ≈ 3 u k + 1 u k u k − 2 u k u k u k (21)
For nonlinear terms involving multiple state variables, i.e., F ( u k + 1 , v k + 1 ) F(u^{k+1},v^{k+1}) F ( u k + 1 , v k + 1 ) , the corresponding Taylor expansion is given by:
F ( u k + 1 , v k + 1 ) = F ( u k , v k ) + ∂ F ∂ u ∣ k ( u k + 1 − u k ) + ∂ F ∂ v ∣ k ( v k + 1 − v k ) + H.O.T (22) F(u^{k+1},v^{k+1})=F(u^k,v^k)+\left.\frac{\partial F}{\partial u}\right|^k(u^{k+1}-u^k)+\left.\frac{\partial F}{\partial v}\right|^k(v^{k+1}-v^k)+\text{H.O.T} \tag*{(22)} F ( u k + 1 , v k + 1 ) = F ( u k , v k ) + ∂ u ∂ F k ( u k + 1 − u k ) + ∂ v ∂ F k ( v k + 1 − v k ) + H.O.T (22)
Based on Eq. (22) , the general Newton linearization form for nonlinear convective terms of arbitrary order can be expressed as:
( u m u x ) k + 1 ≈ ( u m ) k u x k + u x k [ ( u m ) k + 1 − ( u m ) k ] + ( u m ) k ( u x k + 1 − u x k ) = ( u m ) k + 1 u x k + ( u m ) k u x k + 1 − ( u m ) k u x k = [ m ( u k ) m − 1 u k + 1 + ( 1 − m ) ( u k ) m ] u x k + ( u k ) m u x k + 1 − ( u k ) m u x k = m ( u k ) m − 1 u x k u k + 1 + ( 1 − m ) ( u k ) m u x k + ( u k ) m u x k + 1 − ( u k ) m u x k = m ( u k ) m − 1 u x k u k + 1 + ( u k ) m u x k + 1 − m ( u k ) m u x k (23) \begin{aligned}
(u^m u_x)^{k+1} &\approx(u^m)^k u_x^k + u_x^k\left[(u^m)^{k+1}-(u^m)^k\right]+(u^m)^k(u_x^{k+1}-u_x^k) \\
&= (u^m)^{k+1}u_x^k+(u^m)^k u_x^{k+1}-(u^m)^k u_x^k \\
&= \left[m(u^k)^{m-1}u^{k+1}+(1-m)(u^k)^m\right]u_x^k+(u^k)^m u_x^{k+1}-(u^k)^m u_x^k \\
&= m(u^k)^{m-1}u_x^k u^{k+1}+(1-m)(u^k)^m u_x^k+(u^k)^m u_x^{k+1}-(u^k)^m u_x^k \\
&= m(u^k)^{m-1}u_x^k u^{k+1}+(u^k)^m u_x^{k+1}-m(u^k)^m u_x^k \tag*{(23)}
\end{aligned} ( u m u x ) k + 1 ≈ ( u m ) k u x k + u x k [ ( u m ) k + 1 − ( u m ) k ] + ( u m ) k ( u x k + 1 − u x k ) = ( u m ) k + 1 u x k + ( u m ) k u x k + 1 − ( u m ) k u x k = [ m ( u k ) m − 1 u k + 1 + ( 1 − m ) ( u k ) m ] u x k + ( u k ) m u x k + 1 − ( u k ) m u x k = m ( u k ) m − 1 u x k u k + 1 + ( 1 − m ) ( u k ) m u x k + ( u k ) m u x k + 1 − ( u k ) m u x k = m ( u k ) m − 1 u x k u k + 1 + ( u k ) m u x k + 1 − m ( u k ) m u x k (23)
Accordingly, the corresponding expressions for the nonlinear convective terms with m = 1 m=1 m = 1 , m = 2 m=2 m = 2 , and m = 3 m=3 m = 3 are given by:
( u u x ) k + 1 ≈ u k + 1 u x k + u k u x k + 1 − u k u x k (24) (u u_x)^{k+1}\approx u^{k+1}u_x^k+u^ku_x^{k+1}-u^ku_x^k \tag*{(24)} ( u u x ) k + 1 ≈ u k + 1 u x k + u k u x k + 1 − u k u x k (24)
( u 2 u x ) k + 1 ≈ 2 u k u x k u k + 1 + ( u k ) 2 u x k + 1 − 2 ( u k ) 2 u x k (25) (u^2u_x)^{k+1} \approx2u^ku_x^ku^{k+1}+(u^k)^2u_x^{k+1}-2(u^k)^2u_x^k \tag*{(25)} ( u 2 u x ) k + 1 ≈ 2 u k u x k u k + 1 + ( u k ) 2 u x k + 1 − 2 ( u k ) 2 u x k (25)
( u 3 u x ) k + 1 ≈ 3 ( u k ) 2 u x k u k + 1 + ( u k ) 3 u x k + 1 − 3 ( u k ) 3 u x k (26) (u^3u_x)^{k+1} \approx3(u^k)^2u_x^ku^{k+1}+(u^k)^3u_x^{k+1}-3(u^k)^3u_x^k \tag*{(26)} ( u 3 u x ) k + 1 ≈ 3 ( u k ) 2 u x k u k + 1 + ( u k ) 3 u x k + 1 − 3 ( u k ) 3 u x k (26)
Therefore, the nonlinear term in the Burgers’ equation under Newton linearization can be expressed as ( u u x ) k + 1 ≈ u k + 1 u x k + u k u x k + 1 − u k u x k (uu_x)^{k+1}\approx u^{k+1}u_x^k+u^ku_x^{k+1}-u^ku_x^k ( u u x ) k + 1 ≈ u k + 1 u x k + u k u x k + 1 − u k u x k , where the underlined terms originate from higher-order corrections that refine the classical Picard linearization: ( u u x ) k + 1 = u k u x k + 1 (uu_x)^{k+1}=u^ku_x^{k+1} ( u u x ) k + 1 = u k u x k + 1 . Given initial guesses for u u u and u x u_x u x (e.g., setting u u u and u x u_x u x at all time steps equal to the initial field), the corresponding entries in A \mathbf{A} A and b \mathbf{b} b are updated as:
N θ [ f j k + 1 ( x n p d e ; w ~ ) ] = ∂ f j k + 1 ( x n p d e ; w ~ ) ∂ t + u k ( x n p d e ) ∂ f j k + 1 ( x n p d e ; w ~ ) ∂ x + f j k + 1 ( x n p d e ; w ~ ) ∂ u k ( x n p d e ) ∂ x − γ ∂ 2 f j k + 1 ( x n p d e ; w ~ ) ∂ x 2 (27) \mathcal{N}_{\theta}\left[f_j^{k+1}(\mathbf{x}_n^{\mathrm{pde}};\widetilde{\mathbf{w}})\right]=\frac{\partial f_j^{k+1}(\mathbf{x}_n^{\mathrm{pde}};\widetilde{\mathbf{w}})}{\partial t}+u^k(\mathbf{x}_n^{\mathrm{pde}})\frac{\partial f_j^{k+1}(\mathbf{x}_n^{\mathrm{pde}};\widetilde{\mathbf{w}})}{\partial x}+f_j^{k+1}(\mathbf{x}_n^{\mathrm{pde}};\widetilde{\mathbf{w}})\frac{\partial u^k(\mathbf{x}_n^{\mathrm{pde}})}{\partial x}-\gamma\frac{\partial^2 f_j^{k+1}(\mathbf{x}_n^{\mathrm{pde}};\widetilde{\mathbf{w}})}{\partial x^2} \tag*{(27)} N θ [ f j k + 1 ( x n pde ; w ) ] = ∂ t ∂ f j k + 1 ( x n pde ; w ) + u k ( x n pde ) ∂ x ∂ f j k + 1 ( x n pde ; w ) + f j k + 1 ( x n pde ; w ) ∂ x ∂ u k ( x n pde ) − γ ∂ x 2 ∂ 2 f j k + 1 ( x n pde ; w ) (27)
q ( x n p d e ) = u k ( x n p d e ) ∂ u k ( x n p d e ) ∂ x . (28) q(\mathbf{x}_n^{\mathrm{pde}})=u^k(\mathbf{x}_n^{\mathrm{pde}})\frac{\partial u^k(\mathbf{x}_n^{\mathrm{pde}})}{\partial x}. \tag*{(28)} q ( x n pde ) = u k ( x n pde ) ∂ x ∂ u k ( x n pde ) . (28)
The Newton linearization method substantially accelerates the convergence of Tikhonov regularization-based nonlinear iterations due to its near-quadratic convergence property (see Appendix § 11 for a proof). Consequently, it reduces the time complexity of meta-learning and enhances the speed of fine-tuning and generalization on new tasks.
Similarly, the Newton-linearized expressions for other nonlinear terms, such as hyperbolic, exponential, and logarithmic forms, are given as follows:
[ sinh ( u ) ] k + 1 ≈ sinh ( u k ) + cosh ( u k ) ( u k + 1 − u k ) = cosh ( u k ) u k + 1 + sinh ( u k ) − cosh ( u k ) u k (29) \begin{aligned}
[\sinh(u)]^{k+1} &\approx\sinh(u^k)+\cosh(u^k)(u^{k+1}-u^k)\\
&=\cosh(u^k)u^{k+1}+\sinh(u^k)-\cosh(u^k)u^k \tag*{(29)}
\end{aligned} [ sinh ( u ) ] k + 1 ≈ sinh ( u k ) + cosh ( u k ) ( u k + 1 − u k ) = cosh ( u k ) u k + 1 + sinh ( u k ) − cosh ( u k ) u k (29)
[ cosh ( u ) ] k + 1 ≈ cosh ( u k ) + sinh ( u k ) ( u k + 1 − u k ) = sinh ( u k ) u k + 1 + cosh ( u k ) − sinh ( u k ) u k (30) \begin{aligned}
[\cosh(u)]^{k+1} &\approx\cosh(u^k)+\sinh(u^k)(u^{k+1}-u^k)\\
&=\sinh(u^k)u^{k+1}+\cosh(u^k)-\sinh(u^k)u^k \tag*{(30)}
\end{aligned} [ cosh ( u ) ] k + 1 ≈ cosh ( u k ) + sinh ( u k ) ( u k + 1 − u k ) = sinh ( u k ) u k + 1 + cosh ( u k ) − sinh ( u k ) u k (30)
[ tanh ( u ) ] k + 1 ≈ tanh ( u k ) + ( 1 − tanh 2 ( u k ) ) ( u k + 1 − u k ) = ( 1 − tanh 2 ( u k ) ) u k + 1 + tanh ( u k ) − ( 1 − tanh 2 ( u k ) ) u k (31) \begin{aligned}
[\tanh(u)]^{k+1} &\approx\tanh(u^k)+(1-\tanh^2(u^k))(u^{k+1}-u^k)\\
&=(1-\tanh^2(u^k))u^{k+1}+\tanh(u^k)\\
&\quad-(1-\tanh^2(u^k))u^k \tag*{(31)}
\end{aligned} [ tanh ( u ) ] k + 1 ≈ tanh ( u k ) + ( 1 − tanh 2 ( u k )) ( u k + 1 − u k ) = ( 1 − tanh 2 ( u k )) u k + 1 + tanh ( u k ) − ( 1 − tanh 2 ( u k )) u k (31)
[ exp ( u ) ] k + 1 ≈ exp ( u k ) + exp ( u k ) ( u k + 1 − u k ) = exp ( u k ) u k + 1 + exp ( u k ) − exp ( u k ) u k (32) \begin{aligned}
[\exp(u)]^{k+1} &\approx\exp(u^k)+\exp(u^k)(u^{k+1}-u^k)\\
&=\exp(u^k)u^{k+1}+\exp(u^k)-\exp(u^k)u^k \tag*{(32)}
\end{aligned} [ exp ( u ) ] k + 1 ≈ exp ( u k ) + exp ( u k ) ( u k + 1 − u k ) = exp ( u k ) u k + 1 + exp ( u k ) − exp ( u k ) u k (32)
[ u ln ( u ) ] k + 1 ≈ u k ln ( u k ) + ( ln ( u k ) + 1 ) ( u k + 1 − u k ) = ( ln ( u k ) + 1 ) u k + 1 + u k ln ( u k ) − ( ln ( u k ) + 1 ) u k (33) \begin{aligned}
[u\ln(u)]^{k+1} &\approx u^k\ln(u^k)+(\ln(u^k)+1)(u^{k+1}-u^k)\\
&=(\ln(u^k)+1)u^{k+1}+u^k\ln(u^k)-(\ln(u^k)+1)u^k \tag*{(33)}
\end{aligned} [ u ln ( u ) ] k + 1 ≈ u k ln ( u k ) + ( ln ( u k ) + 1 ) ( u k + 1 − u k ) = ( ln ( u k ) + 1 ) u k + 1 + u k ln ( u k ) − ( ln ( u k ) + 1 ) u k (33)
Convergence analysis of the Newton-PINet
Newton linearization has long been employed in numerical PDE solvers [38 , 39 , 6 ] , e.g., to linearize the convective term in the Navier-Stokes equations. We integrate Newton linearization into the Tikhonov-regularized PINN, resulting in Newton-PINet, which significantly improves the performance of meta-learning nonlinear PDEs. This appendix provides theoretical analysis and experimental validation showing that the Newton-PINet optimization procedure is essentially equivalent to the classical regularized Gauss-Newton method and achieves near-quadratic convergence.
The output u u u and its spatial derivative u x u_x u x at the n n n -th collocation point in the Tikhonov-regularized PINN are given by
u ( x n ) = ∑ j f j ( x n ) w j , u x ( x n ) = ∑ j f x , j ( x n ) w j ⇒ u = Φ w , u x = Φ x w , (34) u(\mathbf{x}_n)=\sum_j f_j(\mathbf{x}_n)w_j,\quad u_x(\mathbf{x}_n)=\sum_j f_{x,j}(\mathbf{x}_n)w_j\quad\Rightarrow\quad u=\mathbf{\Phi}\mathbf{w},\quad u_x=\mathbf{\Phi}_x\mathbf{w}, \tag*{(34)} u ( x n ) = j ∑ f j ( x n ) w j , u x ( x n ) = j ∑ f x , j ( x n ) w j ⇒ u = Φw , u x = Φ x w , (34)
where the pre-final features { f j } \{f_j\} { f j } are generated by hidden-layer transformations followed by smooth activation functions, and are thus inherently bounded and continuously differentiable. Here, w T = [ … , w j , … ] \mathbf{w}^T=[\ldots,w_j,\ldots] w T = [ … , w j , … ] denotes the output-layer weights, Φ n j = f j ( x n ) \Phi_{nj}=f_j(\mathbf{x}_n) Φ nj = f j ( x n ) , and ( Φ x ) n j = f x , j ( x n ) (\Phi_x)_{nj}=f_{x,j}(\mathbf{x}_n) ( Φ x ) nj = f x , j ( x n ) .
To provide a simple illustration, we consider a nonlinear equation F = u u x F=uu_x F = u u x defined on the domain x ∈ [ 0 , 1 ] x\in[0,1] x ∈ [ 0 , 1 ] , subject to the boundary conditions u ( 0 ) = 0 u(0)=0 u ( 0 ) = 0 and u ( 1 ) = 1 u(1)=1 u ( 1 ) = 1 . The PDE residuals evaluated at the collocation points { x n } \{x_n\} { x n } are then given by:
r n ( w ) = u ( x n ) u x ( x n ) = ( Φ w ) n ( Φ x w ) n . (35) r_n(\mathbf{w})=u(x_n)u_x(x_n)=(\mathbf{\Phi}\mathbf{w})_n(\mathbf{\Phi}_x\mathbf{w})_n. \tag*{(35)} r n ( w ) = u ( x n ) u x ( x n ) = ( Φw ) n ( Φ x w ) n . (35)
The residuals corresponding to the boundary conditions are
r 0 ( w ) = u ( 0 ) − 0 = Φ ( 0 , : ) w , r 1 ( w ) = u ( 1 ) − 1 = Φ ( 1 , : ) w − 1. (36) r_0(\mathbf{w})=u(0)-0=\mathbf{\Phi}(0,:)\mathbf{w},\quad r_1(\mathbf{w})=u(1)-1=\mathbf{\Phi}(1,:)\mathbf{w}-1. \tag*{(36)} r 0 ( w ) = u ( 0 ) − 0 = Φ ( 0 , : ) w , r 1 ( w ) = u ( 1 ) − 1 = Φ ( 1 , : ) w − 1. (36)
We consider the nonlinear system of equations in a general form: A ( w ) w = b \mathbf{A}(\mathbf{w})\mathbf{w}=\mathbf{b} A ( w ) w = b .
By stacking all the residuals, we obtain the unified nonlinear system:
F ( w ) = A ( w ) w − b = [ u ⊙ u x Φ ( 0 , : ) w Φ ( 1 , : ) w − 1 ] = [ ( Φ w ) ⊙ ( Φ x w ) Φ ( 0 , : ) w Φ ( 1 , : ) w − 1 ] = 0 , (37) F(\mathbf{w})=\mathbf{A}(\mathbf{w})\mathbf{w}-\mathbf{b}
=
\begin{bmatrix}
u\odot u_x\\
\mathbf{\Phi}(0,:)\mathbf{w}\\
\mathbf{\Phi}(1,:)\mathbf{w}-1
\end{bmatrix}
=
\begin{bmatrix}
(\mathbf{\Phi}\mathbf{w})\odot(\mathbf{\Phi}_x\mathbf{w})\\
\mathbf{\Phi}(0,:)\mathbf{w}\\
\mathbf{\Phi}(1,:)\mathbf{w}-1
\end{bmatrix}
=0, \tag*{(37)} F ( w ) = A ( w ) w − b = u ⊙ u x Φ ( 0 , : ) w Φ ( 1 , : ) w − 1 = ( Φw ) ⊙ ( Φ x w ) Φ ( 0 , : ) w Φ ( 1 , : ) w − 1 = 0 , (37)
where ⊙ \odot ⊙ denotes the elementwise product.
Classical Gauss-Newton method
In our experiments, the matrix A \mathbf{A} A is predominantly overdetermined. For example, in the Burgers’ equation, A ∈ R 6810 × 512 \mathbf{A}\in\mathbb{R}^{6810\times512} A ∈ R 6810 × 512 . In this setting, the Newton-PINet optimization can be characterized as the Gauss-Newton method [14 ] . Specifically, the goal of the Gauss-Newton method is to minimize a nonlinear least-squares objective function:
L ( w ) = 1 2 ∥ F ( w ) ∥ 2 2 = 1 2 ∑ i = 1 m F i ( w ) 2 . (38) \mathcal{L}(\mathbf{w})=\frac{1}{2}\lVert F(\mathbf{w})\rVert_2^2=\frac{1}{2}\sum_{i=1}^{m}F_i(\mathbf{w})^2. \tag*{(38)} L ( w ) = 2 1 ∥ F ( w ) ∥ 2 2 = 2 1 i = 1 ∑ m F i ( w ) 2 . (38)
The Gauss-Newton method provides an iterative scheme to minimize this objective by approximating the true Hessian matrix ∇ 2 L ( w ) \nabla^2\mathcal{L}(\mathbf{w}) ∇ 2 L ( w ) . While the exact Hessian is given by ∇ 2 L ( w ) = J F ( w ) T J F ( w ) + ∑ i F i ( w ) ∇ 2 F i ( w ) \nabla^2\mathcal{L}(\mathbf{w})=J_F(\mathbf{w})^T J_F(\mathbf{w})+\sum_i F_i(\mathbf{w})\nabla^2F_i(\mathbf{w}) ∇ 2 L ( w ) = J F ( w ) T J F ( w ) + ∑ i F i ( w ) ∇ 2 F i ( w ) , the Gauss-Newton method neglects the second-order term ∑ i F i ∇ 2 F i \sum_i F_i\nabla^2F_i ∑ i F i ∇ 2 F i , assuming it is small. This yields the approximated Hessian H G N ≈ J F ( w ) T J F ( w ) H_{\mathrm{GN}}\approx J_F(\mathbf{w})^T J_F(\mathbf{w}) H GN ≈ J F ( w ) T J F ( w ) .
Setting the gradient to zero yields the Gauss-Newton update step Δ w = w k + 1 − w k \Delta\mathbf{w}=\mathbf{w}^{k+1}-\mathbf{w}^k Δ w = w k + 1 − w k , which is determined by solving the normal equations:
( J F ( w k ) T J F ( w k ) ) ( w k + 1 − w k ) = − J F ( w k ) T F ( w k ) . (39) \left(J_F(\mathbf{w}^k)^T J_F(\mathbf{w}^k)\right)(\mathbf{w}^{k+1}-\mathbf{w}^k)=-J_F(\mathbf{w}^k)^T F(\mathbf{w}^k). \tag*{(39)} ( J F ( w k ) T J F ( w k ) ) ( w k + 1 − w k ) = − J F ( w k ) T F ( w k ) . (39)
This update is mathematically equivalent to solving the following linearized overdetermined system in the least-squares sense:
J F ( w k ) w k + 1 = J F ( w k ) w k − F ( w k ) . (40) J_F(\mathbf{w}^k)\mathbf{w}^{k+1}=J_F(\mathbf{w}^k)\mathbf{w}^k-F(\mathbf{w}^k). \tag*{(40)} J F ( w k ) w k + 1 = J F ( w k ) w k − F ( w k ) . (40)
Our Newton-PINet: Newton linearization used in Tikhonov-regularized PINN
In our Newton-PINet framework, the nonlinear PDE operators are first linearized at the continuous level (using Newton expansion), and the resulting discretized system is written in the linear form A ( w k ) w k + 1 = b k \mathbf{A}(\mathbf{w}^k)\mathbf{w}^{k+1}=\mathbf{b}^k A ( w k ) w k + 1 = b k . This linear system is then solved iteratively using linear least squares.
Using the Newton expansion for u u x uu_x u u x (as shown in Eq. (24) ) as an illustrative example, the linearization of ( u u x ) k + 1 (uu_x)^{k+1} ( u u x ) k + 1 is:
( u u x ) k + 1 ≈ u k + 1 u x k + u k u x k + 1 − u k u x k , (41) (uu_x)^{k+1} \approx u^{k+1}u_x^k+u^ku_x^{k+1}-u^ku_x^k, \tag*{(41)} ( u u x ) k + 1 ≈ u k + 1 u x k + u k u x k + 1 − u k u x k , (41)
where k k k and k + 1 k+1 k + 1 denote the current and next iteration steps, respectively. Defining u k = Φ w k u^k=\Phi w^k u k = Φ w k and u x k = Φ x w k u_x^k=\Phi_xw^k u x k = Φ x w k , we have:
( u u x ) k + 1 = ( Φ w ) ( k + 1 ) ⊙ ( Φ x w ) ( k + 1 ) ≈ ( Φ x w k ) ⊙ ( Φ w ( k + 1 ) ) + ( Φ w k ) ⊙ ( Φ x w ( k + 1 ) ) − ( Φ w k ) ⊙ ( Φ x w k ) . (42) \begin{aligned}
(uu_x)^{k+1}=(\Phi w)^{(k+1)}\odot(\Phi_xw)^{(k+1)} \\
&\approx(\Phi_xw^k)\odot(\Phi w^{(k+1)})+(\Phi w^k)\odot(\Phi_xw^{(k+1)})-(\Phi w^k)\odot(\Phi_xw^k). \tag*{(42)}
\end{aligned} ( u u x ) k + 1 = ( Φ w ) ( k + 1 ) ⊙ ( Φ x w ) ( k + 1 ) ≈ ( Φ x w k ) ⊙ ( Φ w ( k + 1 ) ) + ( Φ w k ) ⊙ ( Φ x w ( k + 1 ) ) − ( Φ w k ) ⊙ ( Φ x w k ) . (42)
Through matrix manipulations, this becomes:
( Φ x w k ) ⊙ ( Φ w ( k + 1 ) ) = diag ( Φ x w k ) ( Φ w ( k + 1 ) ) = ( diag ( Φ x w k ) Φ ) w ( k + 1 ) , (\Phi_xw^k)\odot(\Phi w^{(k+1)})=\operatorname{diag}(\Phi_xw^k)(\Phi w^{(k+1)})=\left(\operatorname{diag}(\Phi_xw^k)\Phi\right)w^{(k+1)}, ( Φ x w k ) ⊙ ( Φ w ( k + 1 ) ) = diag ( Φ x w k ) ( Φ w ( k + 1 ) ) = ( diag ( Φ x w k ) Φ ) w ( k + 1 ) ,
( Φ w k ) ⊙ ( Φ x w ( k + 1 ) ) = diag ( Φ w k ) ( Φ x w ( k + 1 ) ) = ( diag ( Φ w k ) Φ x ) w ( k + 1 ) . (\Phi w^k)\odot(\Phi_xw^{(k+1)})=\operatorname{diag}(\Phi w^k)(\Phi_xw^{(k+1)})=\left(\operatorname{diag}(\Phi w^k)\Phi_x\right)w^{(k+1)}. ( Φ w k ) ⊙ ( Φ x w ( k + 1 ) ) = diag ( Φ w k ) ( Φ x w ( k + 1 ) ) = ( diag ( Φ w k ) Φ x ) w ( k + 1 ) .
Consequently, the constructed matrix A ( w k ) A(w^k) A ( w k ) and the right-hand side vector b k b^k b k are explicitly given by:
A ( w k ) = [ ( diag ( Φ x w k ) Φ ) + ( diag ( Φ w k ) Φ x ) Φ ( 0 , : ) Φ ( 1 , : ) ] , b k = [ ( Φ w k ) ⊙ ( Φ x w k ) 0 1 ] . (43) A(w^k)=
\begin{bmatrix}
\left(\operatorname{diag}(\Phi_xw^k)\Phi\right)+\left(\operatorname{diag}(\Phi w^k)\Phi_x\right)\\
\Phi(0,:)\\
\Phi(1,:)
\end{bmatrix},
\qquad
b^k=
\begin{bmatrix}
(\Phi w^k)\odot(\Phi_xw^k)\\
0\\
1
\end{bmatrix}. \tag*{(43)} A ( w k ) = ( diag ( Φ x w k ) Φ ) + ( diag ( Φ w k ) Φ x ) Φ ( 0 , : ) Φ ( 1 , : ) , b k = ( Φ w k ) ⊙ ( Φ x w k ) 0 1 . (43)
It can be verified that our A ( w k ) A(w^k) A ( w k ) and b k b^k b k correspond exactly to the classical Gauss-Newton update:
J F ( w k ) w k + 1 = J F ( w k ) w k − F ( w k ) , J_F(w^k)w^{k+1}=J_F(w^k)w^k-F(w^k), J F ( w k ) w k + 1 = J F ( w k ) w k − F ( w k ) ,
i.e.,
A ( w k ) = J F ( w k ) , (44) A(w^k)=J_F(w^k), \tag*{(44)} A ( w k ) = J F ( w k ) , (44)
b k = J F ( w k ) w k − F ( w k ) = [ ( Φ w k ) ⊙ ( Φ x w k ) + ( Φ w k ) ⊙ ( Φ x w k ) − ( Φ w k ) ⊙ ( Φ x w k ) 0 1 ] = [ ( Φ w k ) ⊙ ( Φ x w k ) 0 1 ] . (45) \begin{aligned}
b^k&=J_F(w^k)w^k-F(w^k)\\
&=
\begin{bmatrix}
(\Phi w^k)\odot(\Phi_xw^k)+(\Phi w^k)\odot(\Phi_xw^k)-(\Phi w^k)\odot(\Phi_xw^k)\\
0\\
1
\end{bmatrix}\\
&=
\begin{bmatrix}
(\Phi w^k)\odot(\Phi_xw^k)\\
0\\
1
\end{bmatrix}. \tag*{(45)}
\end{aligned} b k = J F ( w k ) w k − F ( w k ) = ( Φ w k ) ⊙ ( Φ x w k ) + ( Φ w k ) ⊙ ( Φ x w k ) − ( Φ w k ) ⊙ ( Φ x w k ) 0 1 = ( Φ w k ) ⊙ ( Φ x w k ) 0 1 . (45)
Furthermore, by integrating Tikhonov regularization λ r e g ∥ w ∥ 2 \lambda_{\mathrm{reg}}\lVert w\rVert^2 λ reg ∥ w ∥ 2 into our linear solvers to improve conditioning, the objective function is effectively penalized. The corresponding normal equations solved in our algorithm become:
( J F ( w k ) T J F ( w k ) + λ r e g I ) w k + 1 = J F ( w k ) T b k , (46) \left(J_F(w^k)^T J_F(w^k)+\lambda_{\mathrm{reg}}I\right)w^{k+1}=J_F(w^k)^Tb^k, \tag*{(46)} ( J F ( w k ) T J F ( w k ) + λ reg I ) w k + 1 = J F ( w k ) T b k , (46)
which categorizes our Newton-PINet approach as a regularized Gauss-Newton method .
The convergence of the classical regularized Gauss-Newton method depends on both the magnitude of the residuals and the strength of the regularization. Near-quadratic convergence can be achieved when the residuals are sufficiently small and the regularization term is sufficiently weak relative to the Gauss-Newton Hessian approximation. Newton-PINet exhibits near-quadratic convergence, similar to the regularized Gauss-Newton method, only when the meta-learned neural representation is sufficiently accurate, the initial guess is not too far from the solution, and the regularization coefficient is not too large.
PINet: Picard linearization used in Tikhonov-regularized PINN
As a baseline model, we employ Picard linearization in the PINet framework. For the Picard linearization, also referred to as the “lagging-of-coefficients” approach, one factor in the nonlinear term u u x u u_x u u x is kept from the previous iteration k k k . Specifically, the linearization of ( u u x ) k + 1 (u u_x)^{k+1} ( u u x ) k + 1 is given by ( u u x ) k + 1 ≈ u k u x k + 1 (u u_x)^{k+1} \approx u^k u_x^{k+1} ( u u x ) k + 1 ≈ u k u x k + 1 , which can be written in matrix form as
( u u x ) k + 1 ≈ ( Φ w ) k ⊙ ( Φ x w ) ( k + 1 ) = ( diag ( Φ w k ) Φ x ) w ( k + 1 ) . (47) (u u_x)^{k+1} \approx(\boldsymbol{\Phi}w)^k \odot(\boldsymbol{\Phi}_x w)^{(k+1)}
= \left(\operatorname{diag}(\boldsymbol{\Phi}w^k)\boldsymbol{\Phi}_x\right)w^{(k+1)}. \tag*{(47)} ( u u x ) k + 1 ≈ ( Φ w ) k ⊙ ( Φ x w ) ( k + 1 ) = ( diag ( Φ w k ) Φ x ) w ( k + 1 ) . (47)
Consequently, in the linear system A ( w k ) w k + 1 = b k \mathbf{A}(w^k)w^{k+1}=\mathbf{b}^k A ( w k ) w k + 1 = b k , the corresponding matrix and right-hand side are
A ( w k ) = [ ( diag ( Φ w k ) Φ x ) Φ ( 0 , : ) Φ ( 1 , : ) ] , b k = [ 0 0 1 ] . (48) \mathbf{A}(w^k)=
\begin{bmatrix}
\left(\operatorname{diag}(\boldsymbol{\Phi}w^k)\boldsymbol{\Phi}_x\right)\\
\boldsymbol{\Phi}(0,:)\\
\boldsymbol{\Phi}(1,:)
\end{bmatrix},
\qquad
\mathbf{b}^k=
\begin{bmatrix}
0\\
0\\
1
\end{bmatrix}. \tag*{(48)} A ( w k ) = ( diag ( Φ w k ) Φ x ) Φ ( 0 , : ) Φ ( 1 , : ) , b k = 0 0 1 . (48)
Since the Picard method only updates one factor at each iteration while keeping the other fixed, the convergence rate is generally linear rather than quadratic.
Experimental evaluation of convergence
The convergence behavior of Newton-PINet is further verified experimentally. The residual norm ∥ A w − b ∥ \lVert\mathbf{A}w-\mathbf{b}\rVert ∥ A w − b ∥ and the empirical order of convergence (EOC), E O C = log ( ∥ e k + 1 ∥ / ∥ e k ∥ ) log ( ∥ e k ∥ / ∥ e k − 1 ∥ ) \mathrm{EOC}=\frac{\log(\lVert e_{k+1}\rVert/\lVert e_k\rVert)}{\log(\lVert e_k\rVert/\lVert e_{k-1}\rVert)} EOC = l o g (∥ e k ∥ / ∥ e k − 1 ∥) l o g (∥ e k + 1 ∥ / ∥ e k ∥) , are evaluated for both Newton and Picard iterations on the nonlinear Burgers problem. As shown in Table 5 , using the initialization u g u e s s ( t ) = u t = 0 u_{\mathrm{guess}}(t)=u_{t=0} u guess ( t ) = u t = 0 for all time steps, Newton-PINet achieves an observed EOC of 1.7–2.0, a smaller final residual ( 2.98 × 10 − 3 ) (2.98\times10^{-3}) ( 2.98 × 1 0 − 3 ) , and a substantially lower MSE ( 4.59 × 10 − 7 ) (4.59\times10^{-7}) ( 4.59 × 1 0 − 7 ) than Picard (EOC ≈ 1.2 \approx1.2 ≈ 1.2 ), indicating near-quadratic convergence when the initialization is sufficiently close to the solution. Table 6 further shows that poorer initializations, i.e., larger ℓ 2 \ell_2 ℓ 2 distances from the ground-truth solution u g t u_{\mathrm{gt}} u gt , lead to a lower convergence order of Newton-PINet. As discussed in Appendix § 14 .§ 2 , this sensitivity can be mitigated by initializing all time steps with the initial field and employing temporal block decomposition for more challenging nonlinear time-dependent problems such as the K-S equation.
Table 5. Convergence process comparison between Newton and Picard iterations
Table 6. Impact of initial guess on the convergence
The empirical results (Table 7 below) show that increasing λ r e g \lambda_{\mathrm{reg}} λ reg degrades the observed empirical order of convergence (EOC), transitioning Newton-PINet from near-quadratic convergence to linear/sublinear convergence. When λ r e g \lambda_{\mathrm{reg}} λ reg is sufficiently small, it mainly serves to improve numerical conditioning. However, as λ r e g I \lambda_{\mathrm{reg}}I λ reg I begins to dominate, the regularized update ( A T A + λ r e g I ) − 1 A T b (A^{T}A+\lambda_{\mathrm{reg}}I)^{-1}A^{T}b ( A T A + λ reg I ) − 1 A T b approaches 1 λ r e g A T b \frac{1}{\lambda_{\mathrm{reg}}}A^{T}b λ reg 1 A T b , transforming the Newton update into a heavily damped gradient-descent step and thereby degrading the convergence order.
Table 7. Impact of λ r e g \lambda_{\mathrm{reg}} λ reg on convergence order (EOC) and MSE
Problem description and experimental results
Description of nonlinear PDE problems
Nonlinear convection-type PDEs
Burgers’ equation (1D + time)
∂ u ∂ t + u ∂ u ∂ x − γ ∂ 2 u ∂ x 2 = 0 , x ∈ [ − 1 , 1 ] , t ∈ [ 0 , 1 ] . (49) \frac{\partial u}{\partial t}+u\frac{\partial u}{\partial x}-\gamma\frac{\partial^{2}u}{\partial x^{2}}=0,\quad x\in[-1,1],\ t\in[0,1]. \tag*{(49)} ∂ t ∂ u + u ∂ x ∂ u − γ ∂ x 2 ∂ 2 u = 0 , x ∈ [ − 1 , 1 ] , t ∈ [ 0 , 1 ] . (49)
a) Varying PDE parameter:
The temporal and spatial resolutions are set to 51 × \times × 129 with periodic boundary conditions. We fix the initial condition as u ( x , 0 ) = − sin ( π x ) u(x,0)=-\sin(\pi x) u ( x , 0 ) = − sin ( π x ) , and vary the viscosity parameter γ \gamma γ within the range [ 0.001 , 0.05 ] [0.001,0.05] [ 0.001 , 0.05 ] with an increment of 0.001 to generate 50 tasks; 16 of them are randomly selected as training tasks, while the remaining 34 are used for testing. The model architecture consists of 4 hidden layers with 128 neurons per layer. Time block decomposition is not employed (a single time block), and the step of the fine-tuning (nonlinear) iterations is set to 6.
b) Varying initial condition:
The temporal and spatial resolutions are set to 101 × \times × 101 with periodic boundary conditions. Four distinct viscosity values γ ∈ { 0.1 , 0.01 , 0.001 , 0.0001 } \gamma\in\{0.1,0.01,0.001,0.0001\} γ ∈ { 0.1 , 0.01 , 0.001 , 0.0001 } are considered. For each γ \gamma γ , the initial conditions are sampled from a Gaussian random field N ( 0 , 625 ( − Δ + 25 I ) − 4 ) \mathcal{N}(0,625(-\Delta+25I)^{-4}) N ( 0 , 625 ( − Δ + 25 I ) − 4 ) , consistent with Wang et al. [ [41 ] . We generate 50 training tasks and 1000 test tasks for each viscosity, and experiments are conducted independently for each γ \gamma γ . The model architecture consists of 4 hidden layers with 128 neurons per layer, and the nonlinear iteration step is set to 4. We evaluate two model variants: unsupervised Newton-PINet (1 time block) and hybrid Newton-PINet (4 time blocks).
Generalized Korteweg-de Vries (KdV) equation (1D + time)
∂ u ∂ t + β 1 u ∂ u ∂ x + β 2 u 2 ∂ u ∂ x + β 3 u 3 ∂ u ∂ x + δ ∂ 3 u ∂ x 3 = 0 , x ∈ [ − 1 , 1 ] , t ∈ [ 0 , 1 ] (50) \frac{\partial u}{\partial t}+\beta_{1}u\frac{\partial u}{\partial x}+\beta_{2}u^{2}\frac{\partial u}{\partial x}+\beta_{3}u^{3}\frac{\partial u}{\partial x}+\delta\frac{\partial^{3}u}{\partial x^{3}}=0,\quad x\in[-1,1],\ t\in[0,1] \tag*{(50)} ∂ t ∂ u + β 1 u ∂ x ∂ u + β 2 u 2 ∂ x ∂ u + β 3 u 3 ∂ x ∂ u + δ ∂ x 3 ∂ 3 u = 0 , x ∈ [ − 1 , 1 ] , t ∈ [ 0 , 1 ] (50)
The temporal and spatial resolutions are set to 101 × \times × 257 with periodic boundary conditions. We consider three different forms of nonlinear convection terms, corresponding to three combinations of ( β 1 , β 2 , β 3 ) (\beta_{1},\beta_{2},\beta_{3}) ( β 1 , β 2 , β 3 ) . For each combination, 15 tasks are randomly generated by varying δ \delta δ , leading to a total of 45 tasks. Specifically:
K d V : ( β 1 , β 2 , β 3 ) = ( 1 , 0 , 0 ) , δ ∈ [ 0.032 2 , 0.12 2 ] , \mathrm{KdV}:~(\beta_{1},\beta_{2},\beta_{3})=(1,0,0),\quad\delta\in[0.032^{2},0.12^{2}], KdV : ( β 1 , β 2 , β 3 ) = ( 1 , 0 , 0 ) , δ ∈ [ 0.03 2 2 , 0.1 2 2 ] ,
M - K d V : ( β 1 , β 2 , β 3 ) = ( 0 , 1 , 0 ) , δ ∈ [ 0.02 2 , 0.06 2 ] , M\text{-}\mathrm{KdV}:~(\beta_{1},\beta_{2},\beta_{3})=(0,1,0),\quad\delta\in[0.02^{2},0.06^{2}], M - KdV : ( β 1 , β 2 , β 3 ) = ( 0 , 1 , 0 ) , δ ∈ [ 0.0 2 2 , 0.0 6 2 ] ,
G - K d V : ( β 1 , β 2 , β 3 ) = ( 0 , 0 , 1 ) , δ ∈ [ 0.08 2 , 0.15 2 ] . G\text{-}\mathrm{KdV}:~(\beta_{1},\beta_{2},\beta_{3})=(0,0,1),\quad\delta\in[0.08^{2},0.15^{2}]. G - KdV : ( β 1 , β 2 , β 3 ) = ( 0 , 0 , 1 ) , δ ∈ [ 0.0 8 2 , 0.1 5 2 ] .
Among them, 10 tasks are randomly chosen for training and the remaining 35 tasks for testing. The model architecture consists of 4 hidden layers with 256 neurons per layer. 6 time blocks are employed, and the nonlinear iteration step is set to 6.
Kuramoto-Sivashinsky (K-S) equation (1D + time)
∂ u ∂ t + β u ∂ u ∂ x − γ ∂ 2 u ∂ x 2 + σ ∂ 4 u ∂ x 4 = 0 , x ∈ [ 0 , 2 π ] , t ∈ [ 0 , 1 ] (51) \frac{\partial u}{\partial t}+\beta u\frac{\partial u}{\partial x}-\gamma\frac{\partial^{2}u}{\partial x^{2}}+\sigma\frac{\partial^{4}u}{\partial x^{4}}=0,\quad x\in[0,2\pi],\ t\in[0,1] \tag*{(51)} ∂ t ∂ u + β u ∂ x ∂ u − γ ∂ x 2 ∂ 2 u + σ ∂ x 4 ∂ 4 u = 0 , x ∈ [ 0 , 2 π ] , t ∈ [ 0 , 1 ] (51)
The simulated data have a temporal and spatial resolution of 251 × \times × 509, which is downsampled to 63 × \times × 129 for training and evaluation. Periodic boundary conditions are imposed. The fixed coefficients are set as β = 100 / 16 4 \beta=100/16^{4} β = 100/1 6 4 and γ = 100 / 16 2 \gamma=100/16^{2} γ = 100/1 6 2 . The parameter σ \sigma σ varies within the range [ 200 / 16 4 , 300 / 16 4 ] [200/16^{4},\ 300/16^{4}] [ 200/1 6 4 , 300/1 6 4 ] , along with varying initial conditions:
u ( x , 0 ) = ∑ j = 1 J A j sin ( 2 π l j x L + ϕ j ) , u(x,0)=\sum_{j=1}^{J}A_j\sin\left(\frac{2\pi l_jx}{L}+\phi_j\right), u ( x , 0 ) = j = 1 ∑ J A j sin ( L 2 π l j x + ϕ j ) ,
where J = 5 J=5 J = 5 , L = 1 L=1 L = 1 , A j ∈ [ − 0.8 , 0.8 ] A_j\in[-0.8,0.8] A j ∈ [ − 0.8 , 0.8 ] , l j ∈ { 0 , 1 , 2 , 3 , 4 } l_j\in\{0,1,2,3,4\} l j ∈ { 0 , 1 , 2 , 3 , 4 } , and ϕ j ∈ [ − π , π ] \phi_j\in[-\pi,\pi] ϕ j ∈ [ − π , π ] . In total, 150 tasks are generated, of which 100 are used for training and 50 for testing. The model architecture consists of 4 hidden layers with 256 neurons per layer. 15 time blocks are employed, and the nonlinear iteration step is set to 4.
Lid-driven cavity (LDC) equations (2D)
The 2D steady incompressible Navier-Stokes equations in LDC problem are given by
{ ∂ u ∂ x + ∂ v ∂ y = 0 , u ∂ u ∂ x + v ∂ u ∂ y + ∂ p ∂ x − 1 R e ( ∂ 2 u ∂ x 2 + ∂ 2 u ∂ y 2 ) = 0 , u ∂ v ∂ x + v ∂ v ∂ y + ∂ p ∂ y − 1 R e ( ∂ 2 v ∂ x 2 + ∂ 2 v ∂ y 2 ) = 0 , (52) \begin{cases}
\frac{\partial u}{\partial x}+\frac{\partial v}{\partial y}=0,\\
u\frac{\partial u}{\partial x}+v\frac{\partial u}{\partial y}+\frac{\partial p}{\partial x}-\frac{1}{Re}\left(\frac{\partial^2u}{\partial x^2}+\frac{\partial^2u}{\partial y^2}\right)=0,\\
u\frac{\partial v}{\partial x}+v\frac{\partial v}{\partial y}+\frac{\partial p}{\partial y}-\frac{1}{Re}\left(\frac{\partial^2v}{\partial x^2}+\frac{\partial^2v}{\partial y^2}\right)=0, \tag*{(52)}
\end{cases} ⎩ ⎨ ⎧ ∂ x ∂ u + ∂ y ∂ v = 0 , u ∂ x ∂ u + v ∂ y ∂ u + ∂ x ∂ p − R e 1 ( ∂ x 2 ∂ 2 u + ∂ y 2 ∂ 2 u ) = 0 , u ∂ x ∂ v + v ∂ y ∂ v + ∂ y ∂ p − R e 1 ( ∂ x 2 ∂ 2 v + ∂ y 2 ∂ 2 v ) = 0 , (52)
where ( x , y ) ∈ [ 0 , 1 ] 2 (x,y)\in[0,1]^2 ( x , y ) ∈ [ 0 , 1 ] 2 and R e Re R e is the Reynolds number.
The simulated data are generated on a 200 × \times × 200 grid and then downsampled to 51 × \times × 51 for training and testing. The boundary conditions are
top lid: u = 1 u=1 u = 1 , v = 0 v=0 v = 0 ; other walls: u = v = 0 u=v=0 u = v = 0 .
The R e Re R e ranges from 1 to 1000 (specifically 1, 5, 10, 25, 50, 80, 100, ..., 1000 with a step of 50 from 100 to 1000), yielding 25 flow conditions. We use R e = 1 Re=1 R e = 1 , 50, 200, 400, 600, 800 as the training set and the remaining cases as the test set. The model consists of 4 hidden layers with 128 neurons per layer. With three output variables ( u , v , p ) (u,v,p) ( u , v , p ) , the resulting weight dimension of the output layer (i.e., the number of columns in matrix A A A ) is 128 × \times × 4 × \times × 3 = 1536. The nonlinear iteration step is set to 8.
Nonlinear forcing-type PDEs
Convection-diffusion-reaction (CDR) equation (1D + time)
∂ u ∂ t + α ∂ u ∂ x − γ ∂ 2 u ∂ x 2 − ρ 1 u + ρ 2 u 2 + ρ 3 u 3 = 0 , x ∈ [ 0 , 1 ] , t ∈ [ 0 , 1 ] . (53) \frac{\partial u}{\partial t}+\alpha\frac{\partial u}{\partial x}-\gamma\frac{\partial^2u}{\partial x^2}-\rho_1u+\rho_2u^2+\rho_3u^3=0,\quad x\in[0,1],\ t\in[0,1]. \tag*{(53)} ∂ t ∂ u + α ∂ x ∂ u − γ ∂ x 2 ∂ 2 u − ρ 1 u + ρ 2 u 2 + ρ 3 u 3 = 0 , x ∈ [ 0 , 1 ] , t ∈ [ 0 , 1 ] . (53)
The simulated data have a temporal and spatial resolution of 201 × \times × 257, which are downsampled to 101 × \times × 129 for training and evaluation. Periodic boundary conditions are imposed. The parameters are set as α = 1 \alpha=1 α = 1 , γ ∈ { 0.005 , 0.01 , 0.05 } \gamma\in\{0.005,0.01,0.05\} γ ∈ { 0.005 , 0.01 , 0.05 } , ρ 1 , ρ 2 ∈ { 0 , 1 , 3 , 5 } \rho_1,\rho_2\in\{0,1,3,5\} ρ 1 , ρ 2 ∈ { 0 , 1 , 3 , 5 } , and ρ 3 = 5 \rho_3=5 ρ 3 = 5 . The initial conditions are generated from
u ( x , 0 ) = ∑ j = 1 J A j sin ( l j x + ϕ j ) , u(x,0)=\sum_{j=1}^{J}A_j\sin(l_jx+\phi_j), u ( x , 0 ) = j = 1 ∑ J A j sin ( l j x + ϕ j ) ,
where J = 5 J=5 J = 5 , A j ∈ [ 0.1 , 0.5 ] A_j\in[0.1,0.5] A j ∈ [ 0.1 , 0.5 ] , l j ∈ { 1 , 2 , 3 , 4 } l_j\in\{1,2,3,4\} l j ∈ { 1 , 2 , 3 , 4 } , and ϕ j ∈ [ − π , π ] \phi_j\in[-\pi,\pi] ϕ j ∈ [ − π , π ] .
In total, 200 tasks are generated, with 40 used for training and 160 for testing. The model architecture consists of 4 hidden layers with 128 neurons per layer. 4 time blocks are employed, and the nonlinear iteration step is set to 4.
Klein-Gordon equation (1D + time)
∂ 2 u ∂ t 2 − γ ∂ 2 u ∂ x 2 + u 3 = q , x ∈ [ 0 , 1 ] , t ∈ [ 0 , 1 ] (54) \frac{\partial^{2}u}{\partial t^{2}}-\gamma\frac{\partial^{2}u}{\partial x^{2}}+u^{3}=q,\quad x\in[0,1],\ t\in[0,1] \tag*{(54)} ∂ t 2 ∂ 2 u − γ ∂ x 2 ∂ 2 u + u 3 = q , x ∈ [ 0 , 1 ] , t ∈ [ 0 , 1 ] (54)
ICs: ∂ u ∂ t ( x , 0 ) = 0 \frac{\partial u}{\partial t}(x,0)=0 ∂ t ∂ u ( x , 0 ) = 0 .
The solution is defined as u ( x , t ) = k 1 x cos ( k 2 π t ) + k 3 ( x t ) 3 u(x,t)=k_{1}x\cos(k_{2}\pi t)+k_{3}(xt)^{3} u ( x , t ) = k 1 x cos ( k 2 π t ) + k 3 ( x t ) 3 , which is used to derive the corresponding Dirichlet BCs, and source term q q q . The spatio-temporal grid resolution is set to 32 × 32 32\times32 32 × 32 for meta-learning, while a finer grid of 128 × 128 128\times128 128 × 128 is used for testing. The parameters vary within γ , k 1 , k 3 ∈ [ 0.5 , 3 ] \gamma,k_{1},k_{3}\in[0.5,3] γ , k 1 , k 3 ∈ [ 0.5 , 3 ] and k 2 ∈ [ 1 , 7 ] k_{2}\in[1,7] k 2 ∈ [ 1 , 7 ] . A total of 80 tasks are generated, with 16 used for meta-learning and 64 for testing. The model network consists of 4 hidden layers with 128 neurons per layer. 1 time block is employed, and the nonlinear iteration step is set to 4.
Hyperbolic heat equation (1D + time)
∂ u ∂ t − γ ∂ 2 u ∂ x 2 + k 1 sinh ( u ) + k 2 cosh ( u ) + k 3 tanh ( u ) = q , x ∈ [ − 1 , 1 ] , t ∈ [ 0 , 1 ] . (55) \frac{\partial u}{\partial t}-\gamma\frac{\partial^{2}u}{\partial x^{2}}+k_{1}\sinh(u)+k_{2}\cosh(u)+k_{3}\tanh(u)=q,\quad x\in[-1,1],\ t\in[0,1]. \tag*{(55)} ∂ t ∂ u − γ ∂ x 2 ∂ 2 u + k 1 sinh ( u ) + k 2 cosh ( u ) + k 3 tanh ( u ) = q , x ∈ [ − 1 , 1 ] , t ∈ [ 0 , 1 ] . (55)
The solution is defined as u ( x , t , k 1 , k 2 , k 3 ) = sin ( k 1 k 2 π x ) cos ( k 3 π x ) e − π t 2 u(x,t,k_{1},k_{2},k_{3})=\sin(k_{1}k_{2}\pi x)\cos(k_{3}\pi x)e^{-\pi t^{2}} u ( x , t , k 1 , k 2 , k 3 ) = sin ( k 1 k 2 π x ) cos ( k 3 π x ) e − π t 2 , which is used to derive the corresponding ICs, Dirichlet BCs, and source term q q q . The spatio-temporal grid resolution is set to 32 × 32 32\times32 32 × 32 for meta-learning, while a finer grid of 128 × 128 128\times128 128 × 128 is used for testing. The parameters vary within γ ∈ [ 0.2 , 3 ] \gamma\in[0.2,3] γ ∈ [ 0.2 , 3 ] and k 1 , k 2 , k 3 ∈ [ 0.2 , 2.5 ] k_{1},k_{2},k_{3}\in[0.2,2.5] k 1 , k 2 , k 3 ∈ [ 0.2 , 2.5 ] . A total of 80 tasks are generated, with 16 used for training and 64 for testing. The model network consists of 2 hidden layers with 450 neurons per layer. 1 time block is employed, and the nonlinear iteration step is set to 4.
Logarithmic heat equation (1D + time)
∂ u ∂ t − γ ∂ 2 u ∂ x 2 + k 1 u log ( k 2 u ) + k 3 e k 4 u = q , x ∈ [ − 1 , 1 ] , t ∈ [ 0 , 1 ] . (56) \frac{\partial u}{\partial t}-\gamma\frac{\partial^{2}u}{\partial x^{2}}+k_{1}u\log(k_{2}u)+k_{3}e^{k_{4}u}=q,\quad x\in[-1,1],\ t\in[0,1]. \tag*{(56)} ∂ t ∂ u − γ ∂ x 2 ∂ 2 u + k 1 u log ( k 2 u ) + k 3 e k 4 u = q , x ∈ [ − 1 , 1 ] , t ∈ [ 0 , 1 ] . (56)
The solution is defined as u ( x , t , k 2 , k 4 ) = ( sin ( k 2 π x ) + 1.5 ) e − k 4 π x 2 e − π t 2 u(x,t,k_{2},k_{4})=(\sin(k_{2}\pi x)+1.5)e^{-k_{4}\pi x^{2}}e^{-\pi t^{2}} u ( x , t , k 2 , k 4 ) = ( sin ( k 2 π x ) + 1.5 ) e − k 4 π x 2 e − π t 2 , which is used to derive the corresponding ICs, Dirichlet BCs, and source term q q q . The spatio-temporal grid resolution is set to 32 × 32 32\times32 32 × 32 for meta-learning, while a finer grid of 128 × 128 128\times128 128 × 128 for testing. The parameters vary within γ , k 1 , k 3 ∈ [ 0.5 , 5 ] \gamma,k_{1},k_{3}\in[0.5,5] γ , k 1 , k 3 ∈ [ 0.5 , 5 ] and k 2 , k 4 ∈ [ 0.5 , 2 ] k_{2},k_{4}\in[0.5,2] k 2 , k 4 ∈ [ 0.5 , 2 ] . A total of 80 tasks are generated, with 16 used for meta-learning and 64 for testing. The model network consists of 2 hidden layers with 450 neurons per layer. 1 time block is employed, and the nonlinear iteration step is set to 4.
Helmholtz equation (2D)
∂ 2 u ∂ x 2 + ∂ 2 u ∂ y 2 − 100 u + 10 cosh ( u ) = q , x , y ∈ [ 0 , 1.5 ] . (57) \frac{\partial^{2}u}{\partial x^{2}}+\frac{\partial^{2}u}{\partial y^{2}}-100u+10\cosh(u)=q,\quad x,y\in[0,1.5]. \tag*{(57)} ∂ x 2 ∂ 2 u + ∂ y 2 ∂ 2 u − 100 u + 10 cosh ( u ) = q , x , y ∈ [ 0 , 1.5 ] . (57)
The solution is defined as u ( x , y ) = 4 cos ( α 1 π x 2 ) cos ( α 2 π y 2 ) u(x,y)=4\cos(\alpha_{1}\pi x^{2})\cos(\alpha_{2}\pi y^{2}) u ( x , y ) = 4 cos ( α 1 π x 2 ) cos ( α 2 π y 2 ) , which is used to derive the corresponding Dirichlet BCs, and source term q q q . The spatial grid resolution is set to 32 × 32 32\times32 32 × 32 for meta-learning, while a finer grid of 128 × 128 128\times128 128 × 128 is used for testing. The parameters vary within α 1 , α 2 ∈ [ 1 , 3 ] \alpha_{1},\alpha_{2}\in[1,3] α 1 , α 2 ∈ [ 1 , 3 ] . A total of 80 tasks are generated, with 20 used for meta-learning and 60 for testing. The model network consists of 2 hidden layers with 450 neurons per layer. The nonlinear iteration step is set to 4.
Parametric diffusion-reaction equation (2D)
∂ 2 u ∂ x 2 + ∂ 2 u ∂ y 2 + u ( 1 − u 2 ) = q , x , y ∈ [ − 1 , 1 ] . (58) \frac{\partial^{2}u}{\partial x^{2}}+\frac{\partial^{2}u}{\partial y^{2}}+u(1-u^{2})=q,\quad x,y\in[-1,1]. \tag*{(58)} ∂ x 2 ∂ 2 u + ∂ y 2 ∂ 2 u + u ( 1 − u 2 ) = q , x , y ∈ [ − 1 , 1 ] . (58)
The solution is defined as u ( x , y , a 1 , a 2 , w 1 , w 2 , w 3 , w 4 ) = a 1 tanh ( w 1 x ) tanh ( w 2 y ) + a 2 sin ( w 3 x ) sin ( w 4 y ) u(x,y,a_{1},a_{2},w_{1},w_{2},w_{3},w_{4})=a_{1}\tanh(w_{1}x)\tanh(w_{2}y)+a_{2}\sin(w_{3}x)\sin(w_{4}y) u ( x , y , a 1 , a 2 , w 1 , w 2 , w 3 , w 4 ) = a 1 tanh ( w 1 x ) tanh ( w 2 y ) + a 2 sin ( w 3 x ) sin ( w 4 y ) , which is used to derive the corresponding Dirichlet BCs and source term q q q . The spatial grid resolution is set to 32 × 32 32\times32 32 × 32 for meta-learning, while a finer grid of 128 × 128 128\times128 128 × 128 is used for testing. The parameters vary within a 1 , a 2 ∈ [ 0.1 , 1 ] a_{1},a_{2}\in[0.1,1] a 1 , a 2 ∈ [ 0.1 , 1 ] and w 1 , w 2 , w 3 , w 4 ∈ [ 1 , 5 ] w_{1},w_{2},w_{3},w_{4}\in[1,5] w 1 , w 2 , w 3 , w 4 ∈ [ 1 , 5 ] . 17 tasks are generated for meta-learning and 100 for testing. The model network consists of 2 hidden layers with 450 neurons per layer. The nonlinear iteration step is set to 4.
Parametric diffusion-reaction equation (3D)
∂ 2 u ∂ x 2 + ∂ 2 u ∂ y 2 + ∂ 2 u ∂ z 2 + u ( 1 − u 2 ) = q , x , y , z ∈ [ − 1 , 1 ] . (59) \frac{\partial^2 u}{\partial x^2}+\frac{\partial^2 u}{\partial y^2}+\frac{\partial^2 u}{\partial z^2}+u(1-u^2)=q,\quad x,y,z\in[-1,1]. \tag*{(59)} ∂ x 2 ∂ 2 u + ∂ y 2 ∂ 2 u + ∂ z 2 ∂ 2 u + u ( 1 − u 2 ) = q , x , y , z ∈ [ − 1 , 1 ] . (59)
The solution is defined as u ( x , y , z , a 1 , a 2 , w 1 , w 2 , w 3 , w 4 , w 5 , w 6 ) = a 1 tanh ( w 1 x ) tanh ( w 2 y ) tanh ( w 3 z ) + a 2 sin ( w 4 x ) sin ( w 5 y ) sin ( w 6 z ) u(x,y,z,a_1,a_2,w_1,w_2,w_3,w_4,w_5,w_6)=a_1\tanh(w_1x)\tanh(w_2y)\tanh(w_3z)+a_2\sin(w_4x)\sin(w_5y)\sin(w_6z) u ( x , y , z , a 1 , a 2 , w 1 , w 2 , w 3 , w 4 , w 5 , w 6 ) = a 1 tanh ( w 1 x ) tanh ( w 2 y ) tanh ( w 3 z ) + a 2 sin ( w 4 x ) sin ( w 5 y ) sin ( w 6 z ) , which is used to derive the corresponding Dirichlet boundary conditions and source term q q q . The spatial grid resolution is set to 32 × 32 × 32 32\times32\times32 32 × 32 × 32 for both meta-training and testing. The PDE parameters are sampled from a 1 , a 2 ∈ [ 0.1 , 1 ] a_1,a_2\in[0.1,1] a 1 , a 2 ∈ [ 0.1 , 1 ] and w 1 , w 2 , w 3 , w 4 , w 5 , w 6 ∈ [ 1 , 5 ] w_1,w_2,w_3,w_4,w_5,w_6\in[1,5] w 1 , w 2 , w 3 , w 4 , w 5 , w 6 ∈ [ 1 , 5 ] . 17 tasks are generated for meta-learning and 100 for testing. The network consists of two hidden layers with 450 neurons per layer, and the number of nonlinear iterations is set to 4.
The meta-learning configurations for each problem are summarized in Table 8 .
Data generation, error metrics, and computational setup
Datasets for Burgers, generalized KdV, K-S, and CDR problems are generated using the Chebfun package [16 ] with spectral Fourier discretization and a fourth-order exponential time-differencing Runge-Kutta scheme (ETDRK4) [10 ] . Datasets for LDC problem are generated using a computational fluid dynamics (CFD) method [5 ] . For the remaining benchmark problems, analytical solutions are available.
We evaluate the generalization accuracy of the models using both mean squared error (MSE) and relative L 2 L^2 L 2 error. In addition, for the 1D nonlinear diffusion-reaction problem, we adopt the relative MSE metric, consistent with the baseline study [2 ] for comparison (see Table 13 and Figure 13 ).
Table 13. Model comparison on the 1D nonlinear reaction-diffusion problem. Baseline results are from Boudec et al. [ [2 ] . Time is reported in hours (h), minutes (m), and seconds (s). Best and second-best are bold and underlined, respectively.
Figure 13. Prediction results of Newton-PINet for the 1D nonlinear reaction-diffusion benchmark problem.
M S E = 1 n ∑ s = 1 n ( u s l a b e l − u s ) 2 (60) \mathrm{MSE} = \frac{1}{n}\sum_{s=1}^{n}\left(u_s^{\mathrm{label}}-u_s\right)^2 \tag*{(60)} MSE = n 1 s = 1 ∑ n ( u s label − u s ) 2 (60)
R e l a t i v e L 2 = ∥ u l a b e l − u ∥ 2 ∥ u l a b e l ∥ 2 (61) \mathrm{Relative}\ L^2 = \frac{\left\|u^{\mathrm{label}}-u\right\|_2}{\left\|u^{\mathrm{label}}\right\|_2} \tag*{(61)} Relative L 2 = ∥ u label ∥ 2 u label − u 2 (61)
R e l a t i v e M S E = ∑ s = 1 n ( u s l a b e l − u s ) 2 ∑ s = 1 n ( u s l a b e l ) 2 (62) \mathrm{Relative\ MSE} = \frac{\sum_{s=1}^{n}\left(u_s^{\mathrm{label}}-u_s\right)^2}{\sum_{s=1}^{n}\left(u_s^{\mathrm{label}}\right)^2} \tag*{(62)} Relative MSE = ∑ s = 1 n ( u s label ) 2 ∑ s = 1 n ( u s label − u s ) 2 (62)
where u l a b e l u^{\mathrm{label}} u label and u u u denote the ground-truth and predicted solutions for a given PDE instance, and n n n is the total number of collocation points, including those for the PDE residual, IC/BC constraints.
In this study, all models are implemented in the JAX framework [3 ] . For the KdV, K-S, CDR, and LDC problems, model training is parallelized across four Tesla V100 GPUs; all other models are trained on a single Tesla V100 GPU. Inference for all models is performed on a single Tesla V100 GPU.
Table 2 in the main text summarizes the model performance on each problem. For each problem, besides the Newton-PINet model proposed in this study, two additional baseline models were tested: one that uses the Picard method, referred to as PINet, and a purely data-driven DNN baseline which incorporates PDE parameters as inputs. Note that the tables do not include comparisons with other benchmark models on problems such as the Burgers’ equation with varying initial conditions; the detailed results can be found in Table 12 , Table 13 , and Table 15 .
Table 12. Model comparison on the 1D Burgers’ problem. The initial conditions for generating all the training and test data are drawn from GRF ∼ N ( 0 , 625 ( − Δ + 25 I ) − 4 ) \sim\mathcal{N}(0, 625(-\Delta+ 25I)^{-4}) ∼ N ( 0 , 625 ( − Δ + 25 I ) − 4 ) . The lowest errors are highlighted in bold. “–” denotes results not reported in the references.
As shown, Newton-PINet achieves superior test accuracy across nearly all nonlinear PDEs compared to DNN and PINet, except for the Klein-Gordon equation. The Klein-Gordon equation involves second-order temporal dynamics, making the nonlinear iterations more sensitive to the quality of the initial guess. Initializing all time steps with the initial field u ( t = 0 ) u(t=0) u ( t = 0 ) may still be relatively far from the true solution, leading to a slight degradation in convergence for Newton-PINet. However, the difference in prediction accuracy is negligible (MSE: 1.88 × 10 − 7 1.88 \times10^{-7} 1.88 × 1 0 − 7 vs. 2.36 × 10 − 7 2.36 \times10^{-7} 2.36 × 1 0 − 7 ). Notably, for hyperbolic and logarithmic heat problems, the Picard approach employed by PINet fails to converge (leading to NaN values during meta-learning), whereas Newton-PINet remains stable and achieves accurate generalization.
Prediction results of generalized KdV, K-S, and LDC problems: Due to the coexistence of nonlinear convection terms and high-order dispersion operators, solving the generalized KdV and K-S equations is particularly challenging for PINN-based models. To mitigate these difficulties, we adopt a larger number of temporal domain partitions to stabilize training and improve convergence (6 time blocks for the generalized KdV and 15 time blocks for the K-S equations). As shown in Table 2 and Figure 4 , Newton-PINet achieves test errors of MSE = 1.94 × 10 − 3 =1.94 \times10^{-3} = 1.94 × 1 0 − 3 on the generalized KdV, MSE = 3.45 × 10 − 2 =3.45 \times10^{-2} = 3.45 × 1 0 − 2 on the K-S, and MSE = 1.67 × 10 − 4 =1.67 \times10^{-4} = 1.67 × 1 0 − 4 on the LDC problem, outperforming PINet and DNN under the same configuration. Figure 5 , Figure 6 , Figure 7 show the prediction results of the Newton-PINet model on test tasks for the generalized KdV, K-S, and LDC problems, respectively.
Figure 4. MSE distributions across all test tasks for different models on the generalized KdV and K-S problems.
Prediction results of nonlinear forcing-type PDEs: Figure 8 , Figure 9 , Figure 10 show the prediction results of Newton-PINet on some test cases for the convection-diffusion-reaction (CDR) and other nonlinear forcing-type PDEs described in Appendix § 12 .§ 1 .
Figure 8. Prediction results of Newton-PINet for the CDR problem.
Figure 10. Prediction results of Newton-PINet for the 3D parametric diffusion-reaction problem.
Model generalization to unseen boundary conditions: We first investigate the generalization capability of Newton-PINet to unseen boundary condition (BC) based on the 1D+time hyperbolic heat problem defined in Eq. (55) . The model is meta-trained on 16 tasks with Neumann boundary conditions and subsequently evaluated on 64 unseen tasks with Dirichlet and mixed BC. As shown in Table 9 , the Newton-PINet model can effectively generalize to these out-of-distribution boundary conditions that were not encountered during meta-training.
Model generalization to unseen irregular geometries: We further evaluate the generalization capability of Newton-PINet on unseen irregular geometries using two PDE problems, including the 1D+time hyperbolic heat equation and the 3D parametric diffusion-reaction equation.
1D+time hyperbolic heat problem
For the hyperbolic heat equation defined in Eq. (55) , the model is meta-trained on regular geometry with 16 different PDE parameter configurations and evaluated on 64 unseen test cases with irregular curved geometries and simultaneously varying PDE parameters. During testing, the spatial domain is replaced by a time-dependent curved geometry with symmetric moving boundaries:
x l ( t ) = − 1 − A sin ( 2 π f t ) , x u ( t ) = 1 + A sin ( 2 π f t ) , (63) x_l(t) = -1 - A\sin(2\pi f t), \quad x_u(t) = 1 + A\sin(2\pi f t), \tag*{(63)} x l ( t ) = − 1 − A sin ( 2 π f t ) , x u ( t ) = 1 + A sin ( 2 π f t ) , (63)
where the deformation amplitude and frequency are randomly sampled from A ∈ [ 0.1 , 0.5 ] A \in[0.1, 0.5] A ∈ [ 0.1 , 0.5 ] and f ∈ [ 1.0 , 5.5 ] f \in[1.0, 5.5] f ∈ [ 1.0 , 5.5 ] , respectively.
The adaptation is performed solely through physics-informed least-squares fine-tuning of the output layer, without retraining the network parameters. The prediction results are shown in Table 10 and Figure 11 .
3D parametric diffusion-reaction problem
We further examine the generalization capability of Newton-PINet using the 3D parametric diffusion-reaction problem (Eq. (59) ). The model is meta-trained on regular geometry with 17 different PDE parameter configurations and evaluated on 100 unseen test cases with irregular domains of diverse shapes and simultaneously varying PDE parameters. The irregular geometries are generated through a spherical-domain perturbation procedure. Specifically, sampling points are defined by radius r ∈ [ 0.05 , 1.0 ] r \in[0.05, 1.0] r ∈ [ 0.05 , 1.0 ] , azimuth θ ∈ [ 0 , 2 π ) \theta\in[0, 2\pi) θ ∈ [ 0 , 2 π ) , and polar angle ϕ ∈ [ 0.05 , π − 0.05 ] \phi\in[0.05, \pi- 0.05] ϕ ∈ [ 0.05 , π − 0.05 ] to avoid polar singularities. A perturbation scale is then defined as
scale = 1.0 + A cos ( k θ θ + φ θ ) sin ( k ϕ ϕ + φ ϕ ) , (64) \operatorname{scale} = 1.0 + A\cos(k_{\theta}\theta+ \varphi_{\theta})\sin(k_{\phi}\phi+ \varphi_{\phi}), \tag*{(64)} scale = 1.0 + A cos ( k θ θ + φ θ ) sin ( k ϕ ϕ + φ ϕ ) , (64)
where A ∈ [ 0.1 , 0.3 ] A \in[0.1, 0.3] A ∈ [ 0.1 , 0.3 ] controls the perturbation amplitude, k θ , k ϕ ∈ 2 , 3 , 4 , 5 k_{\theta}, k_{\phi} \in2, 3, 4, 5 k θ , k ϕ ∈ 2 , 3 , 4 , 5 determine the spatial frequencies, and φ θ , φ ϕ \varphi_{\theta}, \varphi_{\phi} φ θ , φ ϕ are randomly sampled phases. The perturbed points are mapped to Cartesian coordinates according to
{ x = r ⋅ scale ⋅ sin ( ϕ ) cos ( θ ) , y = r ⋅ scale ⋅ sin ( ϕ ) sin ( θ ) , z = r ⋅ scale ⋅ cos ( ϕ ) . (65) \begin{cases}
x = r \cdot\operatorname{scale} \cdot\sin(\phi)\cos(\theta),\\
y = r \cdot\operatorname{scale} \cdot\sin(\phi)\sin(\theta),\\
z = r \cdot\operatorname{scale} \cdot\cos(\phi). \tag*{(65)}
\end{cases} ⎩ ⎨ ⎧ x = r ⋅ scale ⋅ sin ( ϕ ) cos ( θ ) , y = r ⋅ scale ⋅ sin ( ϕ ) sin ( θ ) , z = r ⋅ scale ⋅ cos ( ϕ ) . (65)
The outermost shell (r = 1.0 r = 1.0 r = 1.0 ) of the 32 × 32 × 32 32 \times32 \times32 32 × 32 × 32 grid is extracted to impose the Dirichlet boundary conditions. This procedure preserves the topology of the original domain while introducing diverse irregular boundary shapes.
The test results are summarized in Table 11 , with representative predictions shown in Figure 12 .
Overall, these results demonstrate that Newton-PINet can generalize to unseen boundary conditions and irregular domain geometries while maintaining high prediction accuracy and computational efficiency. Notably, the adaptation to these out-of-distribution settings requires only physics-informed fine-tuning of the output layer, without retraining the network parameters.
Comprehensive comparison of model performance on nonlinear benchmark problems
Model comparison under varying initial conditions on Burgers’ problem
We evaluate the model on four settings with γ ∈ { 0.1 , 0.01 , 0.001 , 0.0001 } \gamma\in\{0.1, 0.01, 0.001, 0.0001\} γ ∈ { 0.1 , 0.01 , 0.001 , 0.0001 } in Burgers’ problem, where, for each γ \gamma γ , the initial conditions are sampled from a Gaussian random field N ( 0 , 625 ( − Δ + 25 I ) − 4 ) \mathcal{N}(0, 625(-\Delta+ 25I)^{-4}) N ( 0 , 625 ( − Δ + 25 I ) − 4 ) to generate 50 training and 1000 test tasks. Experiments are conducted independently for each γ \gamma γ . We compare (i) unsupervised Newton-PINet (1 time block) and (ii) hybrid Newton-PINet (4 time blocks) against popular neural operator baselines in terms of training time and test error. The unsupervised operator considered is Physics-Only DeepONet (PO-DeepONet) [41 ] . The supervised operators include DeepONet [Lu et al., 2021], Transformer-DeepONet with Trunk net enhanced by Fourier coefficients (T-DeepONet-TF) [47 ] , and FNO [29 ] , while the hybrid operator is PINO (a physics-informed FNO) [30 ] . Note that PINO uses a training loss that combines both data-driven and PDE-based physics losses. The results are summarized in Table 12 . Compared to PO-DeepONet, unsupervised Newton-PINet reduces training time from 9.25h to 291s while improving test accuracy across all γ \gamma γ experiments (in a data-absent scenario). Compared to supervised and hybrid neural operators, including the state-ofthe-art T-DeepONet-TF and PINO, Newton-PINet consistently achieves superior test accuracy under γ = 0.1 , 0.01 , 0.0001 \gamma= 0.1, 0.01, 0.0001 γ = 0.1 , 0.01 , 0.0001 , with significantly reduced training cost (10x less time). Our model consistently achieves higher accuracy while requiring the least training time, whether fully unsupervised (without data) or with a few labeled samples.
Model comparison under varying PDE parameters on a nonlinear reaction-diffusion problem
We further evaluate our method on a widely studied 1D nonlinear reaction-diffusion problem ∂ u ∂ t − γ ∂ 2 u ∂ x 2 − ρ u ( 1 − u ) = 0 \frac{\partial u}{\partial t} - \gamma\frac{\partial^2 u}{\partial x^2} - \rho u(1-u) = 0 ∂ t ∂ u − γ ∂ x 2 ∂ 2 u − ρ u ( 1 − u ) = 0 [26 ] . The benchmark configuration and baseline model results are all taken from [2 ] . The benchmark uses a fixed Gaussian initial condition, with PDE parameters sampled over γ ∈ [ 1 , 5 ] \gamma\in[1, 5] γ ∈ [ 1 , 5 ] and ρ ∈ [ − 5 , 5 ] \rho\in[-5, 5] ρ ∈ [ − 5 , 5 ] . The baseline models are grouped into two categories: (i) unsupervised , including instance-wise PINNs trained with L-BFGS (PINNs+L-BFGS) or Adam+L-BFGS (PINNs-multi-opt), parametric PINNs (PPINNs) [2 ] and P2 ^{2} 2 INNs [9 ] that incorporate PDE parameters as inputs, and Physics-Only DeepONet (PO-DeepONet) [41 ] ; and (ii) hybrid , which combine supervised and physics-informed approaches, such as PI-DeepONet [18 ] , PINO [30 ] , and the physics-informed neural solver (PI-neural-solver) [2 ] . These baselines were trained with 800 tasks and tested on 200 unseen tasks. It can be observed that, among the baseline models, purely physics-based unsupervised training tends to struggle with highly nonlinear problems, whereas hybrid approaches leverage large amounts of labeled data to improve representation learning, thereby enhancing test accuracy. In contrast, our hybrid Newton-PINet achieves state-of-the-art performance using only 50 training tasks, i.e., 16× \times × fewer tasks , and is evaluated on the same test set using a 4-time-block decomposition. For a fair comparison, both models are trained and evaluated on an NVIDIA A6000 GPU. As shown in Table 13 , Newton-PINet reaches a test MSE roughly three orders of magnitude lower than the best-performing baseline (PI-neural-solver) while requiring only 301 seconds of total meta-training wall-clock time, compared with several hours for the baselines. These results highlight the high data efficiency and computational efficiency of Newton-PINet. Model prediction results are provided in Appendix Figure 13.
Model comparison under different GPU configurations
Following the problem setup in Appendix § 13 .§ 2 , Table 14 further reports the model performance under different GPU configurations. It can be observed that Newton-PINet consistently outperforms PI-neural-solver on both A6000 and V100 GPUs.
Table 14. Comparison of model performance under different GPU configurations.
In addition, a comparison of Newton-PINet on A6000 and V100 GPUs shows that training time differs across the two GPUs, while inference time remains comparable. This is likely because GPU configurations mainly affect the efficiency of gradient-based training, whereas our gradient-free Tikhonov-based adaptation is much less sensitive to such differences.
We further compare the performance of Newton-PINet with state-of-the-art gradient-based meta-learning PINNs [33 ] and Baldwinian-PINNs [52 ] . For a fair comparison, we train an unsupervised Newton-PINet with a meta-learning loss of LSE and a hybrid
Newton-PINet with a meta-learning loss of MSE, considering a single time-block setting. Table 15 summarizes the generalization performance and computational cost of Newton-PINet relative to these meta-learned PINN models. Compared to unsupervised gradient-based meta-PINNs, the unsupervised Newton-PINet consistently achieves lower test errors and shorter inference times. For instance, on a parametric diffusion-reaction problem, Newton-PINet exhibits approximately a 179 × 179 \times 179 × improvement in generalization accuracy while requiring 7000 × 7000 \times 7000 × less computational time. Against gradient-free Baldwinian-PINN, which employs Picard to handle PDE nonlinear terms, Newton-PINet which employs the Newton linearization method reduces test MSE by 2 ∼ 5 2\sim5 2 ∼ 5 orders of magnitude across the benchmark problems and shortens single-task fine-tuning time by roughly one order of magnitude.
We also note that, in some cases, the unsupervised Newton-PINet achieves slightly higher accuracy than the hybrid Newton-PINet. This occurs for relatively simple nonlinear PDEs, where using an LSE-based meta-learning loss facilitates better convergence and generalization than MSE. Conversely, for more complex nonlinear PDEs, such as the KdV and K-S equations, adopting an MSE-based meta-learning loss leads to much better performance.
Model comparison with other neural operators on the newly introduced datasets
In the above experiments, we validated the performance of our model on several previously reported challenging datasets. We next further evaluate our method on the newly introduced and more complex PDE benchmarks in this study, including the generalized KdV, Kuramoto-Sivashinsky (K-S), convection-diffusion-reaction (CDR), and the lid-driven cavity (LDC) nonlinear problems. Specifically, we compare the proposed hybrid Newton-PINet with several state-of-the-art neural operator methods, including DeepONet [Lu et al., 2021], Physics-Only DeepONet (PO-DeepONet) [41 ] , Multi-task DeepONet (MT-DeepONet) [27 ] , FNO [29 ] , PINO [30 ] , and Tucker-FNO [54 ] . The hyperparameter settings of these baseline models are summarized in Table 16 . For each problem, all baseline models and our method are trained and evaluated using the same training and test datasets under identical NVIDIA V100 GPU configurations. The comparison results are presented in Table 17 . It can be observed that our model maintains a moderate parameter scale while consistently achieving the lowest test error across all considered benchmarks. Moreover, for the generalized KdV, CDR, and LDC problems, Newton-PINet requires substantially less training time than the competing methods. For the K-S problem, however, the optimization remains relatively challenging for our model, leading to a comparatively longer training time. Overall, on these newly introduced datasets, existing state-of-the-art neural operator methods exhibit relatively poor generalization performance under limited training data settings, primarily because they lack physics-based fine-tuning mechanisms and their generalization capability heavily relies on large amounts of training data. In contrast, the proposed Newton-PINet is able to maintain high generalization accuracy even in data-sparse regimes.
Table 16. Summary of baseline model architectures and hyperparameter configurations.
Runtime comparison with classical numerical solvers
The goal is not to replace classical solvers for solving a single PDE instance, but to provide an efficient framework for offline meta-training followed by online real-time adaptation for parametrized PDE families. Unlike classical methods, which solve each new PDE configuration from scratch, the pretrained Newton-PINet can directly adapt to new configurations through fast inference while achieving comparable accuracy to numerical solvers.
For example, for the Burgers’ equation, a standard Gauss-Newton solver requires approximately 23 s for solving a new instance from scratch, whereas Newton-PINet completes adaptation within only 0.035 s. For the CDR problem, the Chebfun spectral solver (MATLAB) requires approximately 8 s, compared with 0.1 s for Newton-PINet. For the 2D lid-driven cavity flow, a CFD solver requires more than 30 s (excluding meshing), while Newton-PINet achieves prediction within 0.4 s with an MSE of only 1.67 × 10 − 4 1.67 \times10^{-4} 1.67 × 1 0 − 4 compared with the CFD solution.
Ablation studies
Robustness of the meta-learned Tikhonov regularization parameter
In our method, the Tikhonov regularization parameter (λ r e g \lambda_{\mathrm{reg}} λ reg ) is not manually tuned but meta-learned, enabling the model to identify a stable and task-agnostic regularization level across all training tasks. Once learned, λ r e g \lambda_{\mathrm{reg}} λ reg remains fixed during test-time adaptation.
To evaluate its stability, using the Burgers’ problem as an example, we additionally perform per-task optimization by initializing λ r e g \lambda_{\mathrm{reg}} λ reg from the meta-learned value and further updating it for each test task with 300 epochs of gradient-based updates. As shown in Table 18 , the meta-learned λ r e g \lambda_{\mathrm{reg}} λ reg remains close to the per-task optimized values across a wide range of test tasks, and their generalization performance differs only marginally. This indicates that the meta-learned hyperparameter exhibits strong stability and low sensitivity during test-time adaptation.
Table 18. Sensitivity of the meta-learned λ r e g \lambda_{\mathrm{reg}} λ reg to task adaptation in test inference. For meta-learning, λ r e g \lambda_{\mathrm{reg}} λ reg is learned during meta-training and kept fixed during testing. For Per-task optimization, λ r e g \lambda_{\mathrm{reg}} λ reg is initialized from the meta-learned model parameters and then optimized for each test task through gradient-based updates. γ \gamma γ (test tasks of Burgers’ problem)
Effect of initial guess in nonlinear iterations on model accuracy
We conduct empirical ablations to evaluate how the initial field guess (u g u e s s u_{\mathrm{guess}} u guess ) in nonlinear iterations influences model accuracy. Let’s quantify the distance between u g u e s s u_{\mathrm{guess}} u guess and the ground truth solution (u l a b e l u_{\mathrm{label}} u label ) using the ℓ 2 \ell_2 ℓ 2 norm (∥ u g u e s s − u l a b e l ∥ 2 \lVert u_{\mathrm{guess}} - u_{\mathrm{label}}\rVert_2 ∥ u guess − u label ∥ 2 ). To assess sensitivity to initialization, we test several representative choices for u g u e s s u_{\mathrm{guess}} u guess : setting u g u e s s u_{\mathrm{guess}} u guess at all time steps equal to the initial field (u t = 0 u_{t=0} u t = 0 ), all zeros field, all ones field, and normal random field.
As shown in Table 19 , for the time-dependent Burgers’ problem, using the u l a b e l u_{\mathrm{label}} u label as u g u e s s u_{\mathrm{guess}} u guess leads to the lowest error, representing the convergence limit of our method. The u t = 0 u_{t=0} u t = 0 strategy keeps u g u e s s u_{\mathrm{guess}} u guess close to the u l a b e l u_{\mathrm{label}} u label (lowest non-zero distance) and achieves the best accuracy among the evaluated initialization choices. As shown in the Figure 14 , the converged error starting from u t = 0 u_{t=0} u t = 0 is comparable to that obtained when using u l a b e l u_{\mathrm{label}} u label as the initial guess. For the more chaotic K-S system, attempting to solve a single time-block with the initialization of u 0 u_0 u 0 proved very challenging. This is because a simple initial condition can evolve into highly complex structures over time, causing this u g u e s s u_{\mathrm{guess}} u guess to deviate from u l a b e l u_{\mathrm{label}} u label . To address this, we employed a 15-block temporal decomposition strategy. The block-wise inference generally makes the u g u e s s u_{\mathrm{guess}} u guess for each sub-block closer to the $u_{\mathrm{label}}, thereby improving the convergence of the least-squares solve. As Table 19 shows, the test errors under different initial guesses are comparable. This indicates that the temporal block decomposition mitigates the sensitivity of the least-squares solution to the choice of u g u e s s u_{\mathrm{guess}} u guess . We further tested the time-independent lid-driven cavity (LDC) problem and found that zero initialization achieves the lowest test error.
Table 19. Effect of different initial guesses for the nonlinear iterations on test errors. Initial guess for the nonlinear iteration (u g u e s s u_{\mathrm{guess}} u guess )
Figure 14. Convergence comparison of single-task generalization starting from the true field versus u 0 u_0 u 0 as initial guesses.
Our experiments show that for the strongly nonlinear/chaotic K-S equations, even a relatively small initialization distance (10–20) can lead to significantly worse convergence, whereas tasks like the Burgers’ equation and LDC problems are much less sensitive. Crucially, our empirical observations unveil a few strategies to mitigate this sensitivity to initialization. Setting u g u e s s u_{\mathrm{guess}} u guess at all time steps equal to the initial field (u t = 0 u_{t=0} u t = 0 ) can help provide high accuracy for time-dependent problems, while a temporal block decomposition strategy may be employed for increasingly more complex nonlinear time-dependent systems like the K-S problem.
Effect of the number of columns in A on inference time
The scalability of the Tikhonov solution in Newton-PINet is determined by the matrix A A A . Its rows grow with the total number of PDE, IC, and BC residual samples (n p d e n_{\mathrm{pde}} n pde , n i c n_{\mathrm{ic}} n ic , n b c n_{\mathrm{bc}} n bc ). Its column size is determined by the product of the output dimension D o D_o D o and the pre-final feature dimension (i.e., the total number of neurons in the L − 1 L-1 L − 1 hidden layers with each of width N n N_n N n due to skip connections). This results in A ∈ R ( n p d e + n i c + n b c ) × D o N n ( L − 1 ) A \in\mathbb{R}^{(n_{\mathrm{pde}}+n_{\mathrm{ic}}+n_{\mathrm{bc}})\times D_oN_n(L-1)} A ∈ R ( n pde + n ic + n bc ) × D o N n ( L − 1 ) .
However, the Tikhonov update only involves A T A ∈ R D o N n ( L − 1 ) × D o N n ( L − 1 ) A^T A \in\mathbb{R}^{D_oN_n(L-1)\times D_oN_n(L-1)} A T A ∈ R D o N n ( L − 1 ) × D o N n ( L − 1 ) . We conduct experiments on the 2D steady Navier-Stokes equations for the lid-driven cavity (LDC) problem to evaluate how inference time varies with D o D_o D o . For this LDC case, N n = 128 N_n=128 N n = 128 and L = 5 L=5 L = 5 are fixed, D o = 3 D_o=3 D o = 3 corresponds to predicting ( u , v , p ) (u,v,p) ( u , v , p ) , while D o = 2 D_o=2 D o = 2 and D o = 1 D_o=1 D o = 1 correspond to predicting ( u , v ) (u,v) ( u , v ) with known pressure p p p , and u u u with known ( p , v ) (p, v) ( p , v ) , respectively. Table 20 reports the inference time for different D o D_o D o .
Table 20. Inference time versus D o D_o D o , with N n = 128 N_n = 128 N n = 128 and L = 5 L = 5 L = 5 fixed.
We further examine the cost increase due to the column size by varying N n N_n N n (Table 21 ). The results show that increasing either D o D_o D o or N n N_n N n enlarges A \mathbf{A} A , thus increasing the cost of the Tikhonov update and task inference time.
Table 21. Inference time versus N n N_n N n , with D o = 3 D_o = 3 D o = 3 and L = 5 L = 5 L = 5 fixed.
Effect of time-blocking strategies on test error
We evaluate the effect of different numbers of time blocks on the average test error taking the convection-diffusion-reaction (CDR) problem as an example. As shown in Table 22 , without temporal decomposition (i.e., using a single time block), the model achieves fast inference but suffers from large errors. This is because the initial guess in the nonlinear iteration may deviate significantly from the true solution, causing poor least-squares accuracy. To address this issue, we apply a temporal domain decomposition strategy: starting from the known initial condition, the final prediction of each block serves as the initial guess for the next. This ensures that each block starts more closely to the true dynamics, enabling reliable long-horizon prediction. As shown in the table, increasing the number of blocks can substantially reduce error, though at the cost of longer inference time. For this problem, a 4-block configuration is adopted to balance the accuracy and efficiency.
Table 22. Ablation study for different time-blocking strategies.
Effect of initial condition weight on drift at temporal block boundaries
As can be seen in Figure 3 , the CDR prediction results exhibit noticeable drift at temporal block boundaries. Upon investigation, the drift at boundaries is actually due to an under-weighted enforcement of the initial condition (IC) relative to the PDE and boundary condition (BC) in Tikhonov regularization during the meta-learning and test time. After the ablation study, as shown in Table 23 , we found that increasing the IC weight (λ i c \lambda_{\mathrm{ic}} λ ic ) from 1 to 2 can reduce the test error. Figure 15 illustrates that this IC weight adjustment can effectively mitigate the drift at block boundaries.
Table 23. Effect of initial condition (IC) weight in the Tikhonov regularization on test errors.
Figure 15. Mitigation of drift at block boundaries via adjustment of the initial condition (IC) weight in the Tikhonov regularization (4-time-block for CDR inference).
Model performance under other types of boundary conditions
As shown in Table 24 , we test the model performance for Dirichlet, Neumann, and mixed BCs on the hyperbolic heat and the Klein-Gordon problems. As shown in Table 25 , we also evaluate the model for discontinuous BCs on the 2D lid-driven cavity Navier-Stokes problem, where the velocity field exhibits corner discontinuities at the moving lid. These experiments demonstrate that the proposed framework is both theoretically and practically applicable to a broad range of BC types, beyond periodic and Dirichlet settings.
Table 24. Model performance under different boundary conditions.
Table 25. Model performance for 2D lid-driven cavity (LDC) under discontinuous boundary conditions.
Ablation studies on model architecture
Newton-PINet employs a skip-connected multi-layer architecture with Tikhonov regularization, Sinusoidal feature embeddings at the input, and sin ( ⋅ ) \sin(\cdot) sin ( ⋅ ) activations in all hidden layers. In this section, we perform ablation studies to examine the impact of these design choices and other hyperparameters. All test errors in the following tables are aggregated from 5 individual runs.
Effect of network depth: We first study the impact of network depth using the Burgers’ and CDR problems, with and without skip connections (Table 26 ). For comparison, Sinusoidal feature embeddings are always included at the input, and all hidden layers adopt sin ( ⋅ ) \sin(\cdot) sin ( ⋅ ) activations. The number of neurons per layer is set to 512, 256, 170, and 128 for depths of 1∼ \sim ∼ 4, respectively, so that the last-layer width remains around 512 when skip connections are included. The results show that
Table 26. Test relative L 2 L^2 L 2 errors of model with (w/) and without (w/o) skip connections under different network depths.
shallow networks (1 layer) fail to achieve satisfactory accuracy in both problems. Increasing depth without skip connections often deteriorates generalization. In contrast, with skip connections, the model consistently maintains high accuracy across all depths, confirming the stabilizing effect of skip connections and their robustness to depth variations. This property greatly simplifies hyperparameter tuning compared to standard PINNs.
Effect of network width: Next, we fix the depth to 4 layers and compare the effect of different per-layer widths, with and without skip connections (Table 27 ). Across both Burgers’ and CDR problems, incorporating skip connections consistently improves accuracy under almost all width settings. An exception occurs for the case where all hidden layers have 256 neurons, where skip connections slightly degrade performance—likely due to the substantially increased final-layer width, which makes the Tikhonov regularization computation numerically more challenging. Overall, skip connections improve robustness to width variations.
Table 27. Test relative L 2 L^2 L 2 errors with (w/) and without (w/o) skip connections under different network widths (from top to bottom, corresponding to layer 1∼4).
Effect of mesh resolution: We then examine performance under different mesh resolutions, with consistent resolutions applied during meta-training and testing (Table 28 ). With skip connections, the model exhibits stronger adaptability to resolution changes, particularly in the CDR problem. Notably, different resolutions alter the size of the matrices, which can influence numerical stability. Nevertheless, skip connections help preserve accuracy under these settings.
Table 28. Test relative L 2 L^2 L 2 errors with (w/) and without (w/o) skip connections under different mesh resolutions
Effect of activation functions and Sinusoidal embeddings: We further compare activation functions under varying depths, with skip connections included (Table 29 ). For tanh ( ⋅ ) \tanh(\cdot) tanh ( ⋅ ) and Gaussian activations, Sinusoidal embeddings are not used, while for sin ( ⋅ ) \sin(\cdot) sin ( ⋅ ) activations they are applied at the input. The combination of Sinusoidal embeddings with sin ( ⋅ ) \sin(\cdot) sin ( ⋅ ) consistently outperforms other activations across different depths, highlighting the effectiveness of this design choice for nonlinear PDE learning.
Table 29. Test relative L 2 L^2 L 2 errors of models with different activation functions under varying network depths
Effect of weight initialization and learning rate: During meta-learning, we update the hidden-layer weights (w ~ \widetilde{w} w ) and other learning parameters via gradient descent. Here, we investigate the impact of weight initialization schemes and learning rate (η \eta η ) settings on the model generalization (Table 30 ). Results indicate that He initialization together with a smaller initial learning rate leads to the most stable and accurate generalization performance, whereas larger learning rates often cause unstable training or accuracy degradation.
Table 30. Test relative L 2 L^2 L 2 errors under different weight initialization methods and initial learning rates
In summary: Newton-PINet benefits significantly from skip connections, which enhance robustness to depth, width, and mesh resolution, thereby reducing the need for extensive hyperparameter tuning. Sinusoidal feature embeddings with sin ( ⋅ ) \sin(\cdot) sin ( ⋅ ) activations further boost accuracy across depths, and careful initialization and learning rate selection improve stability. Collectively, these results explain why Newton-PINet maintains reliable generalization across diverse nonlinear PDE problems.
Practical scaling discussion of Newton-PINet
Effectiveness of Tikhonov regularization
The scalability of the Tikhonov solution is determined by the matrix A \mathbf{A} A . Its rows grow with the total number of PDE, IC, and BC residual samples. Its column size is determined by the product of the pre-final feature dimension and the output dimension (D o D_o D o ). The pre-final feature dimension equals the total number of neurons in the L − 1 L-1 L − 1 hidden layers (each of width N n N_n N n ) due to the skip connections. This results in A ∈ R ( n p d e + n i c + n b c ) × D o N n ( L − 1 ) \mathbf{A} \in\mathbb{R}^{(n_{\mathrm{pde}}+n_{\mathrm{ic}}+n_{\mathrm{bc}})\times D_oN_n(L-1)} A ∈ R ( n pde + n ic + n bc ) × D o N n ( L − 1 ) .
For 2D/3D or multi-physics systems (e.g., Navier-Stokes), the increase in the number of PDE equations requires more residual sampling points, while a higher output dimensionality enlarges the number of columns, making A \mathbf{A} A larger and more challenging to solve, particularly during Newton iterations for nonlinear problems.
Our strategy is to moderately control the pre-final feature dimension, for example, by decreasing the number of neurons in each layer, thereby reducing the number of columns of A \mathbf{A} A and ensuring that the least-squares problem remains reasonably overdetermined. Since the Tikhonov update w = ( λ r e g I + A T A ) − 1 A T b \mathbf{w}=(\lambda_{\mathrm{reg}}\mathbf{I}+\mathbf{A}^{T}\mathbf{A})^{-1}\mathbf{A}^{T}\mathbf{b} w = ( λ reg I + A T A ) − 1 A T b involves only A T A ∈ R D o N n ( L − 1 ) × D o N n ( L − 1 ) \mathbf{A}^{T}\mathbf{A}\in\mathbb{R}^{D_oN_n(L-1)\times D_oN_n(L-1)} A T A ∈ R D o N n ( L − 1 ) × D o N n ( L − 1 ) , its computational cost depends primarily on the feature dimension, as opposed to the potentially very large number of residual samples in high-dimensional or multiphysics domains. This keeps the Tikhonov solver tractable as long as the feature dimension is properly controlled. In our experiments, the Newton-PINet achieves high accuracy without requiring a large pre-final feature dimension (i.e., no. features ≪ no. collocation points). Appendix F.3 investigates the effect of the column size of A A A on task inference time. The results show that increasing either the output dimension or the number of network width enlarges A A A , thereby increasing the cost of the Tikhonov update and the task inference time.
Although the computational cost of the Tikhonov update can be effectively controlled, the cost of constructing A A A may become considerable and dominate the overall inference time when the number of residual samples is large (e.g., in high-dimensional systems). Nevertheless, even in such cases, the proposed gradient-free adaptation can be more efficient than gradient-based fine-tuning in PINNs.
Effectiveness for solving parametric PDEs
Our model is well suited for solving parametric PDEs, as the number of residual samples need not be large and tasks can be efficiently mini-batched. The parameters define different tasks and can be large but they are not directly used as network inputs. Instead, our model learns shared representations across tasks. During training, we avoid sampling residual points across parameter inputs as in P2INNs [9 ] , and instead construct A A A directly from spatiotemporal points for least-squares adaptation. So, the number of residual samples depends only on the spatiotemporal dimension, not the parameter dimension, which avoids unnecessary growth in the size of A A A . Table 15 demonstrates that the model achieves superior performance on a parametric PDE problem compared to other meta-learning PINN methods.
Grid-free nature and resolution adaptability
Similar to PINNs and neural operators, our model is grid-free: no mesh construction is required; collocation points can be sampled arbitrarily, and the trained model can give predictions at arbitrary locations, even on finer grids. For example, in the five benchmark cases (Table 2 and Table 8 ), trained on a coarse 32 × 32 32 \times32 32 × 32 grid, the model achieves accurate zero-shot generalization on a 128 × 128 128 \times128 128 × 128 grid.
Time-blocking strategy
In addition to the matrix size, the scalability of our model for nonlinear problems also depends on the quality of the initial guess for the iterative updates. Similar to the theoretical behavior of the Newton method, if the initial guess deviates too far from the true solution, more Newton iterations are required to achieve reasonable convergence. This challenge becomes particularly severe for time-dependent chaotic systems, such as the Kuramoto-Sivashinsky (K-S) equation, where even simple initial conditions can quickly evolve into highly complex spatiotemporal patterns. Consequently, it is difficult to obtain an initial guess close to the solution. We can adopt the temporal domain decomposition strategy: starting from the known initial condition, the final prediction of a previous block is used as the initial guess for the next block. This ensures that each block starts more closely to the true dynamics, improving the stability and accuracy of the least-squares updates without using large Newton iterations, and enabling reliable long-horizon prediction.
For complex time-dependent PDEs, temporal domain decomposition is a practical and effective strategy across both AI-based and classical methods. For PINNs methods, training PINNs over the entire spatiotemporal domain in a single-shot usually fails. As noted in prior work [43 ] , gradient-based training tends to fit later-time dynamics before resolving initial conditions, which can lead to premature convergence (and incorrect solutions), especially for chaotic systems such as K-S. Consequently, many recent PINNs adopt temporal decomposition, typically at the expense of high computational cost. Separately, many neural PDE solvers (e.g., neural operators) [21 , 20 ] also rely on autoregressive (one-step) inference for long-time prediction. This strategy is consistent with classical numerical methods, where solutions are advanced sequentially in time to ensure stability.
Low-rank strategy
Although the task adaptation step is formulated as a least-squares problem, the matrix A A A in our method is not sparse in the traditional numerical PDE solver sense. Because the features are generated by a multilayer neural representation with skip connections, each row of A A A contains dense network features rather than the typical discretization-induced sparsity and structure. As a consequence, the rank and conditioning of our least-squares system is determined jointly by (i) the density and distribution of collocation points, (ii) the Tikhonov parameter, and more importantly (iii) the quantity and expressiveness of hidden-layer weights that shape the feature space. In this instance, the practical scaling for large collocation sets may derive more from the fact that one can still control the cost of the solve through meta-learning a better (yet minimal) set of pre-final layer features. For this reason, low-rank or iterative solvers designed for large sparse PDE matrices (e.g., LU decomposition or Krylov solvers) may not offer clear advantages in our setting. We experimented with SVD-based solvers, LU, and Cholesky factorizations; none achieved better accuracy-stability trade-offs than the Tikhonov formulation used in our method. A more comprehensive investigation of scalable low-rank or iterative solvers tailored to dense neural-feature matrices is an interesting direction for future work.