WorksPhysics and Astronomy
Examining the robustness of Physics-Informed Neural Networks to noise for Inverse Problems
Abstract
Approximating solutions to partial differential equations (PDEs) is fundamental for the modeling of dynamical systems in science and engineering. Physics-informed neural networks (PINNs) are a recent machine learning-based approach, for which many properties and limitations remain unknown. PINNs are widely accepted as less computationally efficient and accurate than traditional methods for solving PDEs, such as the finite element method. However, PINNs are commonly claimed to show promise in solving inverse problems and handling noisy or incomplete data. We compare the performance of PINNs in solving inverse problems with that of a traditional approach using the finite element method combined with a numerical optimizer. The models are tested on viscosity identification in 1D Burgers' equation and in 2D/3D Taylor-Green Vortex, in all cases with additive Gaussian noise applied to training and validation data. We find that while PINNs may require less human effort and specialized knowledge, they are outperformed by the traditional approach. For example, for 2D Taylor-Green Vortex with =1 noise, the baseline has a mean prediction RMSE of 0.0013 compared to 0.01 for the best PINN variation. However, PINNs scale better than the baseline with the computational complexity of the problem. We identify failures during training to be addressed if the PINN performance on noisy inverse problems is to become more competitive.
Introduction
Partial differential equations (PDEs) are essential tools for modeling physical systems which are extensively used within engineering [1]. Solving a PDE is called the forward problem. The most widely used methods are backed by a deep theoretical understanding of their limitations and their capabilities on various problems [2]. In many real-life cases, the PDE may not be fully defined. For inverse problems [3], the goal is to identify something missing in the physical model such as a parameter or an initial condition.
Physics-informed neural networks (PINN) leverage the flexibility of neural networks while combining it with physics knowledge to solve both the forward and inverse problems of PDEs within a unified framework [4]. Note that PINNs were introduced to solve inverse problems with static parameters [4]. There has been some limited work to extend PINNs to function-valued inverse problems [5, 6], but we consider this outside of our current scope.
Theoretical understanding of the capabilities and limitations of PINNs remains limited compared to standard methods [7]. Despite extensive literature on the subject [8, 9, 10, 11, 12], PINNs are widely recognized to be inferior to traditional methods for solving the forward problem of PDEs. As an example, Chuang et al. [12] report that PINN takes 32 hours to achieve the same accuracy as a traditional solver running for less than 20 seconds. However, PINNs seem to be less vulnerable to the curse of dimensionality [13]. PINNs may also solve inverse problems many traditional solvers cannot, such as those without known boundary conditions [14, 15].
Despite lower accuracy in the forward problem, PINNs may solve the inverse problem satisfactorily [11, 8, 9], and they have been applied with reasonable success in various fields [16, 17, 18, 19]. CDFbench tests various physics machine learning models on predicting velocity fields of fluid mechanics systems [20] while PINNacle benchmarks variations of PINN on different types of physics problems, including some inverse problems [21]. However, as noted by Karnakov et al. [22], to the best of our knowledge there exists no benchmark to test how PINNs compare with traditional solvers on inverse problems, in particular inverse Navier-Stokes problems with noise.
Numerous efforts have been made to enhance PINN performance [23], such as by balancing the gradients of each training loss, domain decomposition, Fourier space embedding and learning earlier time steps of the physical system first [24, 25, 26, 27]. For the inverse problem, improvement has been demonstrated in parameter identification by using different optimizer for parameter identification and network parameters [28]. Using an ensemble of PINNs has also been suggested to achieve more accurate parameter identification [29]. ODIL is a proposed drop-in replacement for PINNs which directly optimizes discrete formulations of PDEs, removing the neural network [22].
Contributions and outline
We explore the claim that PINNs are effective for inverse problems. As our baseline, we use the general purpose method of combining the finite element method (FEM) with an optimizer. The most efficient solver-based approaches involve significant tailoring to the individual problem. Our purpose is to compare PINNs to a similarly easily accessible but reasonable choice of standard methods, without extensive tuning. We perform tests on parameter identification with varying levels of noise and data availability on a series of increasingly difficult fluid problems leading to the following contributions:
We show that while PINNs require less manual labor and specialized knowledge, a simple baseline with a FEM solver and SLSQP optimizer largely outperforms PINNs on noisy inverse problems.
We find that the relative performance of PINNs appears to improve with the computational complexity of the problem, consistent with existing literature [13].
We identify two particular problems PINNs have on noisy inverse problems:
The PINNs were consistently biased in their parameter estimation, eventually locking the model into a poor solution of the PDE.
With adaptive weights [24], needed for harder problems, the physics loss grew very large for noisy training data. In these cases, the physics loss does not have the desired effect of correcting the noise.
We find that identifying the best model during training is not trivial. New early stopping strategies are needed for noisy inverse problems.
The remainder of the paper is outlined as follows: In Section 2, we provide an overview of PINNs, the finite element method, and the mathematical formulation of the test problems. In Section 3, the shared setup of the experiments is described. Section 4 presents the experimental details and their results. The code can be found on GitHub1. § 5 discusses the results of the experiments and in § 6 we conclude.
Background
Physics-Informed Neural Networks
In a standard neural network, an input vector is passed through layers of weighted sums and non-linear activation functions. , , , are respectively the output, the input and the parameters (biases and weights) of layer . We may consider the full neural network as a parameterized single function [30]:
can be both a scalar function and a vector function . To evaluate the parameters, we define a loss function which measures the discrepancy between the predicted output and the true value for observations available in the training set, totaling . For regression type problems, a common choice is the Mean Squared Error (MSE):
In physics-informed neural networks (PINNs) [4], in addition to the above loss function, we include a second loss function:
is a function which calculates the residual of the differential equation we want to impose on the solution. Leveraging the automatic differentiation capabilities available in popular machine learning libraries, we can easily calculate the derivates of to check adherence with the differential equation. Missing parameters in the PDE can be optimized along with the parameters of the neural network. The two losses in Equation (2) and Equation (3) are summed to get the total loss minimized by the optimizer: , where is used to balance the weights.
In our experiments, we used a popular algorithm which adjusts to ensure that the magnitudes from the two losses are similar [24]. In early stopping [30], we exclude a part of the training set, called the validation set, and use it to approximate when the model has achieved maximum performance.
When training PINNs, the optimizer Adam [31] is often applied first to the loss function since it is considered effective for widely exploring search optimization landscapes. The quasi-Newtonian optimization method L-BFGS (Limited-Memory Broyden-Fletcher-Goldfarb-Shanno) is applied at the end to converge more thoroughly to the closest minimum [32]. To our knowledge, since neural network optimization landscapes are highly nonconvex, we cannot expect monotonous learning for L-BFGS with line search [33].
Finite Element Method
The Finite Element Method (FEM) is a general purpose method for solving differential equations [34]. The procedure works by projecting the solution onto simpler functions, called basis functions, resulting in a system of equations which, when solved, give the closest approximate fit to the solution. In some ways, it is similar to how PINNs estimate the solution by representing it as a neural network and minimizing the losses.
The differential equation in its standard form is known as the strong formulation. We project it by multiplying by a simpler function on both sides, which is referred to as a test function in the literature, and by integrating. This gives us the projected weak form of the problem:
Here is a linear differential operator which encodes the behavior of the system while is the solution we are looking for, defined over a domain on some specific time interval with respect to initial and boundary conditions. This description can be extended to any dimension, but we assume scalar variables here for simplicity.
There are many methods for how to proceed from Equation (4). One of the most well-established methods for deriving and solving the weak form is the Galerkin method [35], in which the same basis functions , are used for both the test functions and trial functions . We get a matrix equation with the following structure to solve:
The matrices are defined by the exact problem and the specific approach taken. There are many variations of FEM which can be applied to solve problems both in space and time. The steps to derive the weak form depend on the equation to be solved and its boundary conditions. Custom algorithms are often necessary for acceptable performance on different problems [36].
Sequential Least Squares Programming
To estimate an unknown parameter in a differential equation using observed data, we first make an initial guess. With this guess, we solve the differential equation using the finite element method. The data loss, measuring the discrepancy between the solution and the training data, is used to update the guess. This process is repeated until convergence [3].
In the experiments, we use Sequential Least Squares Programming (SLSQP) [37], a more robust variation of Sequential Quadratic Programming (SQP). Both algorithms are efficient for non-linear programming problems. Our goal is to find the minimum of the function , which represents the data loss with regards to the unknown parameter . At the function minimum, the gradient will be zero. To seek this zero, the function (and any additional constraint functions) are linearized with Taylor expansions. In SQP, the problem is solved with Newton’s method which uses the gradient to find which direction to move and by how much. However, SLSQP uses a quasi-Newtonian method such as BFGS to avoid calculating the Hessian. Constraints are handled with a nonlinear least squares method.
Test problems
We will perform our experiments on the Burgers’ equation and the Navier-Stokes equations. The systems are related in the sense that Burgers’ equation is a simplification of the Navier-Stokes equations. All the systems used for our experiments are commonly used benchmark problems [4, 8, 20].
The actual implementations of each system used in the experiments, such as boundary conditions and parameters, will be described in Chapter 4.
Although we will specifically work on Burgers’ equation in 1D and Navier-Stokes in 2D and 3D, we will describe the systems in -dimensional form.
Burgers’ equation
Burgers’ equation is a non-linear partial differential equation. It models the evolution of a velocity field as a function of space and time:
Here describes how the velocity field changes in time. is the non-linear advection term which determines how the velocity field changes as a function of its current velocity and its position. is the diffusion term which describes how the velocity is evened out in the flow. Depending on the initial conditions and viscosity, Burgers’ equation can develop shocks when discontinuities arise in the solution as . Intuitively, shocks can be thought of as hard corners forming in the wave. Shocks pose a challenge for common approaches to solving differential equations since standard methods assume differentiable and smooth functions.
Incompressible Navier-Stokes equations
Like Burgers’ equation, the Navier-Stokes equations are a set of non-linear PDEs which model the evolution of a velocity field as a function of time and space. The incompressible form is used to describe liquids whose density does not change with pressure and temperature. The momentum equation in the incompressible Navier-Stokes equations has the general form:
with the requirement of continuity:
As in Burgers’ equation, , and describe the change in time, advection and diffusion, respectively. is the pressure term which evens out differences in pressure. simply states the incompressibility in mathematical terms: The mass of the fluid in any volume does not change. Burgers’ equation can be seen as Navier-Stokes simplified, neglecting pressure and with no compressibility constraint.
Under some conditions, the Navier-Stokes equations exhibit turbulence. In turbulent flow, many small whirls form which interact in complex ways on small scales. We get a chaotic system where small changes in the initial conditions may lead to very large changes in the solution. The Reynolds number is a heuristic used to capture the turbulence of a system. Standard PINNs are not expected to work well with chaos [27]. For this reason, we only consider Navier-Stokes systems with low Reynolds numbers (). However, for Taylor-Green Vortex this is sufficient for some turbulence.
General experimental setup
The most efficient traditional solver-based approaches require significant tailoring to each individual problem. Our goal is to compare PINNs to a traditional general-purpose method which is feasible to run on limited hardware using existing libraries. We want to consider how reasonable implementations would practically measure up against each other in real life application. This is not a controlled scaling study, which is not common in machine learning where the effects of different parameters may be highly irregular. For example, a larger neural network may not give higher accuracy. The results must be understood in light of our methodological limitations. We chose to follow a paper which specifically considered how to compare standard methods to machine learning [8]. To calibrate the models, we matched the models on accuracy when tested on noiseless data as well as we could.
As baseline, we employed a traditional Finite Element Method (FEM) solver combined with an SLSQP optimizer. For the optimizer, we used the SLSQP implementation from SciPy [38]. For the 1D Burgers’ equation, we used the optional FEniCS library pyadjoint [39] to differentiate the solver and calculate gradients for the optimizer. However, this approach required the data and solver grid structure to be identical, which was unfeasible for the Navier-Stokes test problems because of the more complex grids required to solve these problems. In these cases, we employed finite differences with a three-point stencil from SciPy. This produced more accurate gradients than alternative methods discussed in the FEniCS community [40].
The PINN models were implemented using PyTorch [41]. Note that for PINN, we provide results for two types of models. One is a pure PINN model (marked "PINN" in the plots in Figures 1, Figure 3, Figure 5) where we get predictions with a forward pass on the trained neural network. In the "PINN/FEM" model, we collect the estimated parameter from the trained model and solve the differential equation using the same FEM solver as for the baseline. As is standard in machine learning, a small amount of the training data was separated and used as validation data in the PINN experiments. To get prediction RMSEs, we only compared predicted velocities and not the pressure.
Figure 3. Results for 2D Taylor-Green vortex on test set based on 30 runs of each model. Standard deviation () is the noise added to the training data. Top left: Prediction accuracies, . Top right: Prediction accuracies, . Bottom left: Parameter accuracies, . Bottom right: Parameter accuracies, .

Figure 5. Results for 3D Taylor-Green vortex on test set based on 30 runs of each model. For visibility, the x-axes are displayed in a symmetric logarithmic scale with a linear threshold of 0.01. Standard deviation () is the noise added to the training data. Top left: Prediction accuracies with outliers removed. Top right: Parameter accuracies with outliers removed. Bottom left: Prediction accuracies with outliers included. Bottom right: Parameter accuracies with outliers included.
Note that we report the training losses in MSE, the quantity optimized during training. However, we report the predictions in RMSE. This is because the small variation in MSE makes the prediction graphs hard to read. For completeness, the same graphs with MSE are included in Appendix § 13.
The experiments were conducted on the Idun high-performance computing cluster at the Norwegian University of Science and Technology (NTNU) [42]. The compute nodes are mainly equipped with Intel Xeon processors and NVIDIA GPUs. For most of the PINN experiments, we used NVIDIA A100 GPUs. Training PINNs on Navier-Stokes problems may require days depending on the GPU and problem instance, making hyperparameter optimization computationally impractical [12]. For this reason, hyperparameters are commonly selected by manual tuning, and for Navier-Stokes, we used hyperparameter settings consistent with related literature. For the 1D Burgers’ equation, we performed extensive hyperparameter tuning using Optuna [43]. We also investigated the effect of individual hyperparameters. We observed a high degree of robustness with respect to hyperparameter selection. The final hyperparameters were chosen to favor performance on the inverse problem.
For 1D Burgers’ equation, we used the same data as in the original PINN paper2 [4]. For 1D Burgers’ equation with two parameters, we used an analytical solution based on the Cole-Hopf transform, which is described in additional detail in Appendix § 12. For 2D Navier-Stokes, we used an analytical solution. For the specific 3D Navier-Stokes problem that we tested, neither an analytical solution nor pre-existing ground truth data were available. For this reason, we generated reference data using the spectral/hp element framework Nektar++ [44]. We used a scheme similar to the quasi-3D scheme in the official Nektar++ 3D Taylor-Green Vortex tutorial [45].
The noise added to the data was Gaussian with zero mean, using the library NumPy [46]. In all the experiments, we only added noise to the labels in the training data, but never to the initial condition and boundary condition data. Noise was also added to the validation set labels. This is a standard choice in machine learning, since if you were to have data without noise available, it would be much more valuable as training data. By , we denote the standard deviation of the noise. By SNR, we denote the variance of the training data without noise divided by the variance of the noise.
Due to the Central Limit Theorem [47], Gaussian noise can be used to approximate sensor noise. However, many of the noise levels we tested, in particular for the 1D Burgers’ equation, are extreme. For 1D Burgers’ equation, all the values stay within . In other words, with the magnitude of the noise should at least be as large as the original signal. With , noise dominates the original signal by a large amount. In real life, noise on scales such as are more likely. However, we did not want to make any assumptions about the specific application of PINNs. Rather, we wanted to explore the practical properties and limits of PINNs in a general way.
For 1D Burgers’ equation, we chose not to use early stopping based on empirical experience. For the Navier-Stokes problems, we used the validation set to determine which model to use. However, for this to work we had to exclude early models from considerations. Especially with noise, PINNs may do well on the validation set before they have started learning the physics meaningfully. This means the optimizer does not have time to adjust the unknown physics parameter in the physics loss and the inverse problem is not solved. Additionally, both PINN and baseline model were bounded to output minimum . This is because the FEM solver is unable to handle large Reynolds numbers and may break with low viscosity values.
Experiments
1D Burgers’ equation
The ground truth viscosity term to solve for was set to following the original PINN paper [4]. Likewise, and . The following initial conditions and Dirichlet boundary conditions were used:
Under these conditions, the system develops shock-like formations around , which are known to be difficult to solve [4]. Analytical solutions are possible for using the Cole-Hopf transformation [1]. We also tested 1D Burgers’ equation with two unknown parameters, adding a second physical parameter to the equation, described more extensively in Appendix § 12.
Specific experimental setup
For the training and test data for Burgers’ equation, the same data was used as in the original PINN paper [4]. The total dataset consisted of 25600 observations of which 10% was used as training data and 2% as validation data. The training and validation data were randomly sampled from the full dataset. For each model, we trained 30 random initializations for which we present the mean and standard deviation of the performance. The noise levels tested for the inverse problem, both in and SNR, are given in Appendix F.§ 1. Remaining implementation details are in Appendix § 7.§ 1.
Results, 1D Burgers’ equation

Figure 1. Results for 1D Burgers’ equation on test set based on 30 runs of each model. Standard deviation () is the noise added to the training data. Top left: Prediction accuracies, . Top right: Prediction accuracies, . Bottom left: Parameter predictions, . Bottom right: Parameter predictions, .
Figure 1 shows the results of the experiments. By inspecting the tables in Appendix § 9.§ 1, we see that for , the PINN model has the lowest RMSE accuracy of the three models tested. However, the PINN model takes 110 seconds on average to run using GPUs while our not optimized FEM/SLSQP model takes 25 seconds on a single core CPU. The peak memory use of both models are similar, around 500 MB. As the noise increases, the variation in prediction accuracy for both models begins to increase as well. In particular, PINN tends to predict very high parameter values at higher frequency when the noise increases. As can be seen in Appendix C.§ 1, for PINN the skewness of the prediction distribution increase in a right-leaning direction. Without noise, all models perform well with RMSE of the order of .
Despite many of our tests being on extreme levels of noise, we observe that FEM/SLSQP significantly outperforms the classical PINN at all noise levels. However, the results of FEM/SLSQP and the PINN/FEM model cannot be distinguished with statistical significance, aside from at where FEM/SLSQP is highly likely to have lower error than PINN/FEM.
PINN exhibits a small negative bias for all noise levels, including , as seen in Appendix C.§ 1. However, the bias is only statistically significant for noise levels 0, 3, 5, 7, 25. FEM/SLSQP has a positive bias, but these results are only significant for the highest noise levels. With , which can be seen in Appendix § 15, PINNs have the highest mean estimates. (However, PINN also has a very large variation.)
Figure 2 shows the loss history during training. The observed history for is typical for the 1D Burgers’ equation tests without noise. While running the Adam optimizer, we see characteristic spikes in both the data loss and physics loss as the model tries to balance them. When L-BFGS is turned on (after slightly less than 1 minute), we get a mostly smooth convergence towards a local minimum as would be expected from this optimizer.

Figure 2. 1D Burgers’ equation loss history from training and parameter history from training. Note that the validation data, like the training data, has noise added to it which can explain the similar performance of their losses on high noise levels. The test loss only evaluates predictive accuracy, not agreement with differential equation. Top left: Loss history during PINN training, . Top right: Loss history for . Bottom left: Loss history for . Bottom right: Parameter learning history for . All the results of the losses exclude the weights used for loss balancing.
With both and , the data loss is largely monotonic. The physics loss develops in a more complicated way with more erratic spikes. For , the physics loss makes a jump around 0.9 minutes. After the test loss has reached its lowest point, the model ends up converging to a worse state. However, there is no way to tell from the validation loss when this lowest point happened, which supports our choice to not use early stopping for this problem. The remaining loss histories from our experiments not included here show similar behavior in the presence of noise.
The minimum test loss approximately occurs when the parameter estimates are closest to the ground truth. After, PINN tends to estimate a lower viscosity on average and converge on a worse total loss. This is also supported by the previously mentioned statistical tests in Appendix C.§ 1, which show that PINN tends to have a slight negative bias. This behavior is prominent in the example loss history with and . For , this can only be weakly seen. However, as more noise is added to the training data, the model becomes increasingly likely to estimate the viscosity too high.
As described in Appendix § 12, we also tested 1D Burgers’ equation with two unknown parameters. As we can see in Appendix § 9.§ 2, the models had similar performance as with one unknown parameter with slightly worse overall performance. Some of the issues that we shall later see in the other experiments with more complex systems are more pronounced in the problem with two parameters, such as large physics losses. Solving for two parameters produces effects similar to increasing system complexity in other experiments.
2D Taylor-Green Vortex
For the 2D Navier-Stokes experiment, the Taylor-Green vortex problem was selected. This problem is defined by the initial conditions:
The ground truth viscosity term to be solved for was set to while is known. The variables were restricted to , and . This problem roughly corresponds to which is well within laminar flow. We assume is a state vector containing velocities , and the pressure at any given position and time in the 2D domain. The following periodic boundary conditions were used:
The analytical solution for this system is as follows:
Intuitively, the system can be described as a set of four evenly spaced whirls with velocity dissipating according to the value of viscosity.
Specific experimental setup
Following [4], we trained the PINN model to learn the stream function, , along with the pressure field :
Instead of having to learn the velocity for each dimension and , the network only needs to learn a single stream output for all dimensions. This is achieved using the identities and . The formulation assumes incompressibility, so we only need to impose the momentum equation to get the correct physics with this approach.
The total dataset consisted of 412 776 observations uniformly sampled in space and time from the analytical solution. We used 24% for training, 4.8% for validation and imitated hyperparameters used for successful training on 2D Navier-Stokes in the literature [4]. To consistently get acceptable performance, we had to use a gradient balancing algorithm on the weights of the losses [24]. Our implementation is standard, but more information can be found in Appendix § 7.§ 4. We trained 30 random initializations and present the mean and standard deviations of the results. The noise levels tested for the inverse problem, both in and SNR, are given in Appendix F.§ 3. Remaining exact implementation details can be found in Appendix § 7.§ 2.
Results, 2D Taylor-Green Vortex
Figure 3 shows that FEM/SLSQP on average outperforms the other models for 2D Taylor-Green Vortex on all tested noise levels. Where the distributions overlap, we have confirmed that FEM/SLSQP is superior with statistical significance in Appendix § 9.§ 3. On the other hand, PINN takes 6.6 hours on average while FEM takes 11.2 hours on average. PINN also had lower peak memory use than the baseline. We used 10 cores running in parallel for FEM/SLSQP and each individual core required 966 MBs at most. For PINN we only used one core, with a peak memory use of around 1088 MB.
We can see that the parameter estimation for 2D Taylor-Green Vortex has larger variation than for 1D Burgers’ equation on the noise levels for which we did experiments on both systems. As with RMSE predictions, FEM/SLSQP outperforms PINN at predicting the correct unknown parameter. As opposed to 1D Burgers’ equation, the skewness of the estimated parameter distribution leans left rather than right for PINN, and does not grow as evenly with the noise.
As with 1D Burgers’ equation, PINN tends to have a small negative bias on parameter estimation for 2D Taylor-Green Vortex. The bias is statistically significant from noise level to . In contrast with the results for 1D Burgers’ equation, the mean of the predicted parameter continues to decrease with noise rather than to eventually blow up. FEM/SLSQP has a small positive bias on lower noise levels. Interestingly, PINN has the best performance at . However, the advantage is slight and, for this noise level, PINN also has a larger range in accuracy than for lower noise levels.
In Figure 4, the loss histories fluctuate even more than they did for 1D Burgers’ equation. For visibility at , we have applied weighted average exponential smoothing with [48]. On lower noise levels, the loss history resembles the behavior of 1D Burgers’ equation: After switching to L-BFGS, we often see a smooth convergence towards a better model. At lower noise levels, PINNs rapidly converge near ground truth but show little subsequent improvement.
Figure 4. 2D Taylor-Green vortex training history. Note that noise is added to the validation data which may explain poor validation loss performance. The test loss only evaluates predictive accuracy, not agreement with differential equation. Top left: Loss history during PINN training, . Top right: Loss history for . Bottom left: Parameter learning history, . Bottom right: Parameter learning history, . All the results of the losses exclude the weights used for loss balancing.
As noise increases, a different phenomenon becomes increasingly prevalent. While the PINN model first tends to balance physics and data training well, at some point it starts to disregard the physics loss. We can see for in Figure 4 that the physics loss, which is the mean square average over the data, approaches . In the parameter estimation histories for higher numbered , it often stays near the ground truth for a while before training starts to fail. The best models are always from before this transition.
3D Taylor-Green Vortex
For the implementation of 3D Navier-Stokes, the Taylor-Green vortex problem was used. In 3D, it is defined by the initial conditions:
The ground truth viscosity term was chosen to be . This roughly corresponds to the Reynolds number 1000, which for the Taylor-Green Vortex is starting to become turbulent. We wanted to increase the complexity of the problem as well as add a dimension to get more variety in the test systems. The variables were restricted to , , and . We assume is a state vector containing velocities , , and the pressure at any given position and time in the 3D domain. The following periodic boundary conditions were used:
Intuitively, 3D Navier-Stokes consists of layers of the 2D version of the problem where the direction of the flow is opposite on regular intervals depending on the -coordinate. Due to small turbulence, velocity starts to build and creates some velocity in the direction making the layers of whirls interact with each other chaotically. There is no analytical solution for the chosen parameters.
Specific experimental setup
3D Navier-Stokes cannot be reduced to a single stream function, and alternative formulations that decrease the number of required functions are non-trivial. For this reason, we trained the PINN model to learn the individual velocities as outputs. The total dataset consisted of 1 100 000 observations randomly sampled from the data we generated using Nektar++ as described in § 3. We used 25% as training data and 5% as validation data which is similar as what we used for the 2D problem. The proportion of training and validation data was selected through manual experimentation to find what was required to reliably solve the problem with PINN at . Here as well, we used a gradient balancing algorithm for the loss weights [24]. 30 random initializations were trained of each model and we present the mean and standard deviations of the results. The noise levels tested for the inverse problem, both in and SNR, are given in Appendix F.§ 4. Other implementation details can be found in Appendix § 7.§ 3.
Results, 3D Taylor-Green Vortex
For the 3D experiments, both PINN and FEM/SLSQP sometimes catastrophically fail. Both PINN and FEM/SLSQP occasionally get stuck on or near the initial guess. Note that these failures, in a practical setting, could easily be identified and excluded. We define outliers using interquartile range (IQR) [47]. Values below and values above were identified as outliers. For FEM/SLSQP these failures account for about 20% of the runs. For PINN, it is less than 1%. However, in more than 25% of runs, PINN is stuck on the lowest parameter allowed. This is not captured by our outlier definition and relative to the task it is a decent estimation, so we have chosen not to consider these results outliers.
No negative bias can be proven for PINN while FEM has a small positive bias on several noise levels. When we include the outlier runs, the standard deviations of FEM are also very large. For instance, it is 0.38 for . According to our statistical tests in Appendix § 9.§ 4, FEM still outperforms both PINN and PINN/FEM with statistical significance. When we exclude the outliers for FEM, the same effect becomes larger. With outliers excluded, FEM/SLSQP’s parameter identification accuracy also becomes similar as in 2D. However, it must be noted that the baseline’s failure rate is very high. Even though FEM is very likely to outperform PINN in most cases, it is not straightforward to claim that the baseline is better than PINN.
For 3D Taylor-Green Vortex, FEM/SLSQP is a lot more expensive to run than PINN. On average, FEM/SLSQP takes 19.9 hours to solve the inverse problem. PINN takes 10.8 hours on average. The memory costs of FEM/SLSQP are also far greater. PINN’s peak memory use is similar to as for 2D Taylor-Green Vortex, about 1100 MB. FEM/SLSQP has an average peak memory of almost 2000 MB for each of the 55 cores we used. The baseline does not scale well and requires HPC resources for 3D.
Even with outliers removed, all the models generally perform worse than on the 2D problem. This result is expected since by adding a dimension with equal size as the others, the domain becomes much larger. The lower ground truth viscosity problem also increases the Reynolds number, making the system harder, potentially leading to small amounts of chaos as well.
Comparing the loss and parameter estimation histories in Figure 6, we observe the same phenomena as in 2D Taylor-Green Vortex. In the most successful runs, we may get a smooth improvement when L-BFGS is applied in the end. In the less successful runs, the physics loss tends to blow up. In the 2D case, these phenomena seem tied to the level of noise with the physics loss exploding becoming increasingly likely with higher levels of noise. For 3D, the phenomena seem to be independent of the noise.

Figure 6. 3D Taylor-Green vortex training histories. Note that the validation data may contain noise which can explain poor validation loss performance. The test loss only evaluates predictive accuracy, not agreement with differential equation. Left: Loss history for PINN, . Weighted average exponential smoothing with was used [48]. Right: Parameter learning history, . All the results of the losses exclude the weights used for loss balancing.
Discussion
1D Burgers’ equation
The results from the 1D Burgers’ equation experiments raise an interesting question. As seen in Figure 2, for , the PINN test loss exhibits typical overfitting behavior after around 1.25 minutes. Since we have much of the differential equation and noisy data, we could naively expect that minimizing both losses, would always approximate the solution better. The physics equation should prohibit the model from overfitting on the noise.
One contribution to this failure may be the tendency of PINNs to estimate the viscosity as lower than the ground truth. As the physics loss gets sufficiently small, PINNs may eventually decrease the loss significantly by making the unknown viscosity smaller. When the differential equation thus becomes incorrect, it is less compatible with the data. The model may not be able to improve anymore, stuck in a local minima, and it will converge on a worse solution than earlier in training. This is similar to the "reward hacking" problem common in reinforcement learning where the model finds a way to optimize its loss which does not serve the purpose of training [49]. As seen in Appendix C.§ 1, PINN often has a negative bias.
While test loss identifies the optimal model, using only training and validation losses is insufficient. Developing a method to detect optimal stopping points from these losses alone would enable early stopping tailored to PINNs in noisy inverse problems. Since the validation loss only tells us how the model is doing on the forward problem, it is not always indicative of the inverse problem. In many cases already, understanding and accommodating the difference of PINNs from ordinary machine learning has helped develop the competitiveness of PINNs in other aspects [24, 50].
Even if the FEM/SLSQP baseline is not significantly more accurate than PINN/FEM on the 1D Burgers’ equation, it is certainly faster. For some applications this difference may not make a practical difference. At the same time FEM/SLSQP is considerably more difficult to set up and requires more specialized knowledge. We need a weak form of our PDE, the grid must be specified and the matrix solver chosen. In many real-life settings, PINNs may be more appropriate given the existence of user-friendly ML packages.
2D Taylor-Green Vortex
We noted that for this problem PINN was faster than the FEM/SLSQP baseline. As discussed in § 3, a careful comparison of computational cost is outside of our scope. A significant part of the cost of FEM/SLSQP consists in calculating the points of the training set to compare against the solution. The large amount of training data may be biased towards PINN in that FEM/SLSQP may not need as much training data as a neural network to achieve equal results. This is a typical challenge in comparing traditional methods with machine learning methods: Both training on different data sets and training on the same data set may be considered unfair.
Shorter training times may also be achievable for PINN. With , the model finds parameter values close to the ground truth in less than an hour, if we properly know when to stop training. With the applied gradient weighting algorithm, PINN may require less epochs than presented in the literature. This supports our claim in § 5.1 that we may be able to improve PINNs if we develop appropriate early stopping strategies.
For 2D Navier-Stokes, PINN has the same tendency to estimate the viscosity as lower than the ground truth as we observed with 1D Burgers’ equation. An example of this can be seen in the loss histories in Figure 4. This bias is also confirmed by statistical tests found in Appendix § 9.§ 3. However, with higher noise a different behavior begins to dominate. Increasingly with noise, PINNs will at some point during training disregard the physics loss and let it grow by a few magnitudes. For higher noise levels, such as for Burger’s equation in § 2, we can already see some of this tendency although less prominent. We also see this more frequently on lower noise levels in the 1D Burgers’ equation tests with two unknown parameters. However, we hypothesize this phenomena is exacerbated by the adaptive weight algorithm we used for the Navier-Stokes problems [24]. The weight on the data loss, calculated by dividing the maximum gradient of the physics loss by the average gradient of the data loss, may make it more profitable to overfit on the noise than to consider the physics. As the physics become incompatible with the model of the noisy data, the optimizer will eventually disregard the physics loss. Contrary to expectations, the physics loss does not effectively constrain the adverse effects of noise in PINNs, in particular when the adaptive weight gradient balancing algorithm is used.
3D Taylor-Green Vortex
Interestingly, many of the problems which we only saw for higher noise levels with 2D Taylor-Green Vortex are already present without noise with 3D Taylor-Green Vortex. A possible explanation is that increasing the difficulty of the problem has a similar effect as adding noise to the training data. This is also supported by the 1D Burgers’ equations experiments with two parameters instead of only one, where we see more of the problems that appear with 2D and 3D Taylor-Green Vortex. We may hypothesize that overfitting on a simplified model of a difficult physical system can become more profitable than learning the physics, just as when overfitting on noise.
PINN does not show a consistent negative bias for 3D Taylor-Green Vortex as it did with 1D Burgers’ equation and 2D Taylor-Green Vortex. The mean biases go both directions, but are only statistically significant at (where the bias is negative). A possible explanation is that learning issues due to the higher complexity in 3D occlude this simpler learning problem.
We observe that PINN seems to scale better with the computational complexity of the problem compared to FEM/SLSQP. The cost for 3D Taylor-Green Vortex is not much higher than it was for the 2D version of the problem, in particular the memory use. The baseline, FEM/SLSQP, scales much worse. Running it on the 3D problem is significantly more expensive and requires HPC resources. FEM/SLSQP is also highly likely to catastrophically fail compared to PINN. On the other hand, as with FEM, PINNs accuracy degrades with the 3D problem compared to the 2D problem. Our observation that PINNs may scale well supports previous claims in the literature [13].
Conclusion
Through three different experiments, we have observed that a simple baseline using FEM and an optimizer, mostly outperforms PINNs in accuracy on noisy inverse problems. However, this observation is complicated by the baseline’s memory costs for Navier-Stokes and its frequent catastrophic failure rate (about 20%) for the 3D test problem. The limits of our experiments also make discussion about time cost difficult. To the best of our knowledge, this is the first benchmark comparing PINNs with traditional solvers on inverse problems, in particular inverse Navier-Stokes problems with noise. However, PINNs do require less manual labor and specialized knowledge compared to the FEM-based baseline. We have also found that relative to the baseline, PINNs appear to scale better with the computational complexity of the problem as memory costs change little. This is consistent with findings in existing literature where PINNs excel in extremely high dimensions [13].
We have identified two difficulties which appear to limit the performance of PINNs on noisy inverse problems. There is a "reward hacking" phenomenon, the parameter learning is biased by the optimization process. This causes training to converge to a suboptimal solution. Adaptive weighting also tends to break down with noisy training data. Contrary to our expectations, the physics loss fails to constrain the adverse effects of noise.
Furthermore, we have found that it is not trivial to identify the best model during training. Neither the validation loss nor physics loss tends to give a good indication according to the test loss. Developing new early stopping strategies for PINNs may be crucial to make the models more competitive on noisy inverse problems. The future of PINN research appears to lie in better tailoring neural networks to the physics domain in particular [23].
Implementation details
1D Burgers' equation
In the FEM model, we used the following weak formulation:
To represent the solution vector field , we assume the PINN model where contains a single space and time coordinate from the physics training data and . Substituting this into the 1D Burgers’ partial differential equation, we get the following residual to measure:
| Aspect | Details |
| Repetitions | 30 runs per experiment. |
| Noise levels () | 0, 0.5, 1, 2, 3, 5, 7, 10, 25. |
| Spatial resolution | . |
| Temporal resolution | . |
| Dataset | From PINN paper [4]. |
| Total data points | 100 time steps · 256 positions = 25600. |
| Training data | 10 time steps · 256 = 2560 (10%). |
| Validation data | 2 time steps · 256 = 512. |
| Test data | 88 time steps · 256 = 22528 (88%). |
| Initial condition data | 160 randomly generated points. |
| Boundary condition data | 80 randomly generated points. |
| Physics data | 2540 random () points. |
Table A.1. 1D Burgers’ equation, shared setup.
| Aspect | Details |
| Basis functions | 1st degree polynomials. |
| Training data | Training and validation sets. |
| Discretization | Same as training data. |
| Solver | Newton, max 50 iter, no preconditioner. |
| Initial guess | . |
| SLSQP settings | , max iter 100, bounds . |
| Gradient calc. | Adjoint method (pyadjoint). |
Table A.2. 1D Burgers’ model setups, FEM/SLSQP setup.
| Aspect | Details |
| Architecture | 4 layers, 20 neurons/layer. |
| Activation | Tanh. |
| Batch size | Full-batch. |
| Training data | Training data, BC, IC, physics points. |
| Training epochs | 4000 (ADAM) + 4000 (L-BFGS). |
| Early stopping | Best results without. |
| Adaptive weights | Best results without. |
| Parameter search | Logarithmic scale. |
| Initial guess | . |
| Loss weight | . |
| Adam, learning rate | 0.001 (default). |
| L-BFGS, learning rate | 1 (default). |
| L-BFGS, history size | 50. |
| L-BFGS, line search | Strong Wolfe. |
| L-BFGS, rtol and atol | 1e-16, 1e-16. |
Table A.3. 1D Burgers’ model setups, PINN setup.
2D Navier-Stokes equations
For FEM, we used the algorithm Chorin’s projection [51]. Each time step requires solving three systems. First we solve for , ignoring pressure:
Next, we solve for the pressure using the velocity from last step:
where is a test function. With we can get the corrected, real :
For incompressible Navier-Stokes systems with periodic boundary conditions, the pressure is only unique up to an additive constant. In our implementation, this gauge condition is not enforced during the discrete solve. We obtain a pressure field with a fixed gauge by subtracting the spatial average of the pressure, thus enforcing a zero-mean pressure condition. Since we only compare velocity predictions in our study, this choice does not affect the results.
By using the identities from 4.2.1, by substitution we have , and the residual functions for the physics loss:
Here we assumed that each contains the coordinates , and time .
| Aspect | Details shared |
| Repetitions | 30 runs per experiment. |
| Noise levels () | 0, 0.001, 0.01, 0.05, 0.1, 0.25, 0.5, 1. |
| Spatial resolution | . |
| Temporal resolution | . |
| Dataset | Generated from analytical solution. |
| Total data points | 26 time steps 126 126 observations. |
| Training data | 99225 random samples, of total. |
| Validation data | 19845 random samples, of total. |
| Test data | 293706 random samples, of total. |
| Initial condition data | 4000 randomly generated positions. |
| Boundary condition data | 4000 randomly generated positions. |
| Physics data | The inputs from the training data. |
Table A.4. 2D Navier-Stokes equation, shared setup.
| Aspect | Details for FEM/SLSQP |
| Basis functions | Cubic polynomial for velocity, quadratic polynomial for pressure. |
| Training data | Training and validation sets. |
| Discretization | , , . |
| Solver | Gmres solver with amg preconditioner. Abstol = 1e-12, rtol=1e-10, maximum iterations 5000. |
| Initial guess | . |
| SLSQP settings | Ftol=1e−16, max iter 100, bounds [0.003141, 5]. |
| Gradient calc. | 3-point finite difference (SciPy). |
Table A.5. 2D Navier-Stokes, FEM/SLSQP setup.
| Aspect | Details for PINN |
| Architecture | 9 layers, 20 neurons/layer. |
| Activation | Tanh. |
| Batch size | Full-batch. |
| Training data | Training data, BC, IC, physics points. |
| Training epochs | 200000 (ADAM) + 50000 (L-BFGS). |
| Early stopping | Best model, based on the validation set, is loaded at end of training. Models from the first 10,000 epochs are ignored. |
| Adaptive weights | Yes, was needed [24]. |
| Parameter search | Softplus used to bound to [0.003141, ). |
| Initial guess | . |
| Adam, learning rate | 0.001 (default). |
| L-BFGS, learning rate | 1 (default). |
| L-BFGS, history size | 50. |
| L-BFGS, line search | Strong Wolfe. |
| L-BFGS, rtol and atol | 1e-16, 1e-16. |
Table A.6. 2D Navier-Stokes, PINN setup.
3D Navier-Stokes equations
As for 2D Navier-Stokes, we used Chorin’s projection described in -dimensional form. For the physics loss, the residual function was implemented as follows, assuming each has coordinates , , and time :
| Aspect | Details |
| Repetitions | 30 runs per experiment. |
| Noise levels () | 0, 0.001, 0.01, 0.05, 0.1, 0.25, 0.5, 1. |
| Spatial resolution | . |
| Temporal resolution | Solved with , saved with . |
| Dataset | Generated using Nektar++ [44]. |
| Total data points | 1100000 random samples from generated solution. |
| Training data | 275000 random samples, = 25% of total. |
| Test data | 770000 random samples, = 75% of total. |
| Validation data | 55000 random samples, = 5% of total. |
| Initial condition data | 16000 randomly generated positions. |
| Boundary condition data | 8000 random positions from generated solution. |
| None shared with training data. | |
| Physics data | The inputs from the training data. |
Table A.7. 3D Navier-Stokes equation, shared setup.
| Aspect | Details |
| Basis functions | Quadratic polynomial for velocity, 1st degree polynomial for pressure. |
| Training data | Training and validation sets. |
| Discretization | , , . |
| Solver | Gmres solver with amg preconditioner. Abstol = 1e-12, rtol=1e-10, maximum iterations 50000. |
| Initial guess | . |
| SLSQP settings | Ftol=1e-16, max iter 100, bounds [0.003141, 5]. |
| Gradient calc. | 3-point finite difference (SciPy). |
Table A.8. 3D Navier-Stokes equation, FEM/SLSQP setup.
| Aspect | Details |
| Architecture | 9 layers, 20 neurons/layer. |
| Activation | Tanh. |
| Batch size | Full-batch. |
| Training data | Training data, BC, IC, physics points. |
| Training epochs | 200000 (ADAM) + 50000 (L-BFGS). |
| Early stopping | Best model, based on the validation set, is loaded at end of training. Models from the first 10,000 epochs are ignored. |
| Adaptive weights | Yes, was needed. [24]. |
| Parameter search | Softplus used to bound to [0.003141, ). |
| Initial guess | . |
| Adam, learning rate | 0.001 (default). |
| L-BFGS, learning rate | 1 (default). |
| L-BFGS, history size | 50. |
| L-BFGS, line search | Strong Wolfe. |
| L-BFGS, rtol and atol | 1e-16, 1e-16. |
Table A.9. 3D Navier-Stokes equation, PINN setup.
Adaptive weights
Adaptive weights
For the Navier-Stokes problems, we used a gradient balancing algorithm [24]. At regular intervals, the algorithm calculates the difference in magnitude of the maximum physics loss gradients and the average data loss gradients:
The data weight is updated using , with commonly chosen as 0.9:
The weights are updated every 10 epochs, which is the frequency discussed in the paper [24]. We tried lower update frequencies, but this did not give satisfactory performance. Note that the weights are detached from the computational graph. Boundary condition and initial condition data is combined with the rest of the training data and they are all weighted together as one. The physics loss is not weighted. Below are some sample graphs demonstrating how the weights evolve during training.
Figure 7 (PDF p. 38).

Figure 8 (PDF p. 39).

Figure 9 (PDF p. 40).
Visually the graphs are similar to what is shown in the original paper [24], but the weight values are much larger. Note that for 2D Taylor-Green Vortex with and 3D Taylor-Green Vortex, where there was struggle with apparent overfitting, the weights tend to grow during training.
Prediction results
In this section we report the mean and standard deviation for each model’s RMSE performance on the test set for all the problems studied. The difference between each pair of models is quantified using the Vargha-Delaney statistic [52]. Its value indicates a probability between 0 and 1 that the reference model (the first model in each set of tables) has larger RMSE than the other model. To quantify whether the distribution of the results of each pair of models are different to a statistically significant degree, we used the Brunner-Munzel test [53]. This is justified by the results from the tests generally not being normally distributed. Note that Brunner-Munzel is undefined when the distributions of each model do not overlap at all. For the 1D Burgers’ problems with , FEM is deterministic, and variance is 0 which also means Brunner-Munzel becomes undefined. For both these cases, tests are simply reported as N/A. We provide the original values, as well as the values adjusted using the Holm family-wise error rate correction [54] and the Benjamini-Hochberg false discovery rate correction [55]. For the Holm-derived values, all the values below 0.05% have been marked in bold.
1D Burgers' equation
| Noise | PINN | FEM | Vargha–Delaney A |
| 0 | 0.005366 ± 0.000595 | 0.006515 ± 0 | 0.067 |
| 0.001 | 0.022701 ± 0.006844 | 0.010623 ± 0.004350 | 0.956 |
| 0.01 | 0.049035 ± 0.009264 | 0.020314 ± 0.016021 | 0.914 |
| 0.05 | 0.095079 ± 0.023605 | 0.026936 ± 0.019883 | 0.974 |
| 0.1 | 0.144118 ± 0.039172 | 0.058326 ± 0.045087 | 0.914 |
| 0.25 | 0.205870 ± 0.057595 | 0.110683 ± 0.148179 | 0.832 |
| 0.5 | 0.257481 ± 0.080885 | 0.160393 ± 0.171301 | 0.791 |
| 1 | 0.452368 ± 0.212503 | 0.187883 ± 0.192528 | 0.822 |
Table B.10. PINN and FEM results with Vargha-Delaney A effect size.
| Noise | Brunner–Munzel | |||
| 0 | N/A | N/A | N/A | N/A |
| 0.001 | −15.398 | 2.537e − 16 | 1.522e − 15 | 8.879e − 16 |
| 0.01 | −9.032 | 4.370e − 10 | 1.748e − 09 | 7.647e − 10 |
| 0.05 | −20.920 | 1.023e − 19 | 7.160e − 19 | 7.160e − 19 |
| 0.1 | −10.280 | 3.039e − 12 | 1.520e − 11 | 7.091e − 12 |
| 0.25 | −5.207 | 1.280e − 05 | 2.560e − 05 | 1.493e − 05 |
| 0.5 | −4.324 | 1.427e − 04 | 1.427e − 04 | 1.427e − 04 |
| 1 | −5.712 | 1.243e − 06 | 3.729e − 06 | 1.740e − 06 |
Table B.11. Brunner-Munzel tests and adjusted p-values for PINN and FEM results.
| Noise | FEM | PINN/FEM | Vargha–Delaney |
| 0 | 0.033 | ||
| 0.001 | 0.462 | ||
| 0.01 | 0.311 | ||
| 0.05 | 0.263 | ||
| 0.1 | 0.507 | ||
| 0.25 | 0.504 | ||
| 0.5 | 0.677 | ||
| 1 | 0.529 |
Table B.12. FEM and PINN/FEM results with Vargha-Delaney A effect size.
| Noise | Brunner–Munzel | |||
| 0 | N/A | N/A | N/A | N/A |
| 0.001 | 0.482 | 0.6321 | 1.0000 | 0.9565 |
| 0.01 | 2.573 | 0.0127 | 0.0763 | 0.0445 |
| 0.05 | 3.405 | 0.0013 | 0.0088 | |
| 0.1 | 0.9342 | 1.0000 | 0.9565 | |
| 0.25 | 0.9565 | 1.0000 | 0.9565 | |
| 0.5 | 0.0245 | 0.1223 | 0.0571 | |
| 1 | 0.7110 | 1.0000 | 0.9565 |
Table B.13. Brunner-Munzel tests and adjusted p-values for FEM and PINN/FEM results.
| Noise | PINN | PINN/FEM | Vargha–Delaney |
| 0 | 0.013 | ||
| 0.001 | 0.847 | ||
| 0.01 | 0.927 | ||
| 0.05 | 0.969 | ||
| 0.1 | 0.989 | ||
| 0.25 | 0.988 | ||
| 0.5 | 0.998 | ||
| 1 | 0.878 |
Table B.14. PINN and PINN/FEM results with Vargha-Delaney A effect size.
| Noise | Brunner–Munzel | |||
| 0 | 38.304 | |||
| 0.001 | ||||
| 0.01 | ||||
| 0.05 | ||||
| 0.1 | ||||
| 0.25 | ||||
| 0.5 | ||||
| 1 |
Table B.15. Brunner-Munzel tests and adjusted p-values for PINN and PINN/FEM results.
1D Burgers' equation with two parameters
| Noise | PINN | FEM | Vargha–Delaney |
| 0 | 1.000 | ||
| 0.001 | 0.922 | ||
| 0.01 | 0.988 | ||
| 0.05 | 0.991 | ||
| 0.1 | 0.976 | ||
| 0.25 | 0.938 | ||
| 0.5 | 0.870 | ||
| 1 | 0.849 |
Table B.16. PINN and FEM results with Vargha-Delaney A effect size.
| Noise | Brunner–Munzel | |||
| 0 | N/A | N/A | N/A | N/A |
| 0.001 | ||||
| 0.01 | ||||
| 0.05 | ||||
| 0.1 | ||||
| 0.25 | ||||
| 0.5 | ||||
| 1 |
Table B.17. Brunner-Munzel tests and adjusted p-values for PINN and FEM results.
| Noise | FEM | PINN/FEM | Vargha–Delaney |
| 0 | 0.000 | ||
| 0.001 | 0.382 | ||
| 0.01 | 0.180 | ||
| 0.05 | 0.169 | ||
| 0.1 | 0.206 | ||
| 0.25 | 0.428 | ||
| 0.5 | 0.372 | ||
| 1 | 0.517 |
Table B.18. FEM and PINN/FEM results with Vargha-Delaney A effect size.
| Noise | Brunner–Munzel | |||
| 0 | N/A | N/A | N/A | N/A |
| 0.001 | 1.607 | 0.1135 | 0.3974 | 0.1589 |
| 0.01 | 5.467 | |||
| 0.05 | 6.147 | |||
| 0.1 | 5.080 | |||
| 0.25 | 0.959 | 0.3416 | 0.6831 | 0.3985 |
| 0.5 | 1.686 | 0.0993 | 0.3974 | 0.1589 |
| 1 | 0.8329 | 0.8329 | 0.8329 |
Table B.19. Brunner-Munzel tests and adjusted p-values for FEM and PINN/FEM results.
| Noise | PINN | PINN/FEM | Vargha–Delaney |
| 0 | |||
| 0.001 | |||
| 0.01 | |||
| 0.05 | |||
| 0.1 | |||
| 0.25 | |||
| 0.5 | |||
| 1 |
Table B.20. PINN and PINN/FEM results with Vargha-Delaney A effect size.
| Noise | Brunner–Munzel | |||
| 0 | N/A | N/A | N/A | N/A |
| 0.001 | ||||
| 0.01 | ||||
| 0.05 | ||||
| 0.1 | ||||
| 0.25 | ||||
| 0.5 | ||||
| 1 |
Table B.21. Brunner-Munzel tests and adjusted p-values for PINN and PINN/FEM results.
2D Taylor-Green Vortex
| Noise | PINN | FEM | Vargha–Delaney |
| 0 | 0.257 | ||
| 0.001 | 0.268 | ||
| 0.01 | 1.000 | ||
| 0.05 | 1.000 | ||
| 0.1 | 1.000 | ||
| 0.25 | 1.000 | ||
| 0.5 | 1.000 | ||
| 1 | 1.000 |
Table B.22. PINN and FEM results with Vargha-Delaney A effect size.
| Noise | Brunner–Munzel | |||
| 0 | 3.055 | 0.0048 | 0.0083 | |
| 0.001 | 2.832 | 0.0083 | 0.0083 | |
| 0.01 | N/A | N/A | N/A | N/A |
| 0.05 | N/A | N/A | N/A | N/A |
| 0.1 | N/A | N/A | N/A | N/A |
| 0.25 | N/A | N/A | N/A | N/A |
| 0.5 | N/A | N/A | N/A | N/A |
| 1 | N/A | N/A | N/A | N/A |
Table B.23. Brunner-Munzel tests and adjusted p-values for PINN and FEM results.
| Noise | FEM | PINN/FEM | Vargha–Delaney |
Table B.24. FEM and PINN/FEM results with Vargha-Delaney A effect size.
| Noise | Brunner–Munzel | |||
| N/A | N/A | N/A | N/A | |
Table B.25. Brunner-Munzel tests and adjusted p-values for PINN and PINN/FEM results.
| Noise | PINN | PINN/FEM | Vargha–Delaney |
Table B.26. PINN and PINN/FEM results with Vargha-Delaney A effect size.
| Noise | Brunner–Munzel | |||
| 0 | 4.535 | |||
| 0.001 | 4.687 | |||
| 0.01 | −0.385 | 0.7022 | 1.0000 | 0.8330 |
| 0.05 | −0.806 | 0.4260 | 1.0000 | 0.6816 |
| 0.1 | −0.213 | 0.8330 | 1.0000 | 0.8330 |
| 0.25 | −0.314 | 0.7553 | 1.0000 | 0.8330 |
| 0.5 | −0.964 | 0.3424 | 1.0000 | 0.6816 |
| 1 | −6.951 |
Table B.27. Brunner-Munzel tests and adjusted p-values for PINN and PINN/FEM results.
3D Taylor-Green Vortex, with outliers
| Noise | PINN | FEM | Vargha–Delaney |
| 0 | 1.000 | ||
| 0.001 | 0.967 | ||
| 0.01 | 0.896 | ||
| 0.05 | 0.923 | ||
| 0.1 | 0.956 | ||
| 0.25 | 0.933 | ||
| 0.5 | 1.000 | ||
| 1 | 0.967 |
Table B.28. PINN and FEM results with Vargha-Delaney A effect size.
| Noise | Brunner–Munzel | |||
| 0 | N/A | N/A | N/A | N/A |
| 0.001 | −14.000 | |||
| 0.01 | −7.108 | |||
| 0.05 | −9.105 | |||
| 0.1 | −13.339 | |||
| 0.25 | −9.355 | |||
| 0.5 | N/A | N/A | N/A | N/A |
| 1 | −14.000 |
Table B.29. Brunner-Munzel tests and adjusted p-values for PINN and FEM results.
| Noise | FEM | PINN/FEM | Vargha–Delaney |
| 0 | 0.050 | ||
| 0.001 | 0.061 | ||
| 0.01 | 0.194 | ||
| 0.05 | 0.171 | ||
| 0.1 | 0.123 | ||
| 0.25 | 0.138 | ||
| 0.5 | 0.103 | ||
| 1 | 0.247 |
Table B.30. FEM and PINN/FEM results with Vargha-Delaney A effect size.
| Noise | Brunner–Munzel | |||
| 0 | 12.687 | |||
| 0.001 | 11.501 | |||
| 0.01 | 4.533 | |||
| 0.05 | 5.359 | |||
| 0.1 | 7.482 | |||
| 0.25 | 6.107 | |||
| 0.5 | 8.755 | |||
| 1 | 3.963 |
Table B.31. Brunner-Munzel tests and adjusted p-values for PINN and PINN/FEM results.
| Noise | PINN | PINN/FEM | Vargha–Delaney |
| 0 | |||
| 0.001 | |||
| 0.01 | |||
| 0.05 | |||
| 0.1 | |||
| 0.25 | |||
| 0.5 | |||
| 1 |
Table B.32. PINN and PINN/FEM results with Vargha-Delaney A effect size.
| Noise | Brunner–Munzel | |||
| 0 | ||||
| 0.001 | ||||
| 0.01 | ||||
| 0.05 | ||||
| 0.1 | ||||
| 0.25 | ||||
| 0.5 | ||||
| 1 | N/A | N/A | N/A | N/A |
Table B.33. Brunner-Munzel tests and adjusted p-values for PINN and PINN/FEM results.
3D Taylor-Green Vortex, without outliers
| Noise | PINN | FEM | Vargha–Delaney |
| 0 | 0.032690 ± 0.009598 | 0.001923 ± 0.000019 | 1.000 |
| 0.001 | 0.039061 ± 0.009850 | 0.001920 ± 0.000001 | 1.000 |
| 0.01 | 0.036553 ± 0.010552 | 0.001924 ± 0.000018 | 1.000 |
| 0.05 | 0.036072 ± 0.011541 | 0.001919 ± 0.000001 | 1.000 |
| 0.1 | 0.036208 ± 0.010680 | 0.001922 ± 0.000006 | 1.000 |
| 0.25 | 0.037936 ± 0.010059 | 0.001935 ± 0.000022 | 1.000 |
| 0.5 | 0.040479 ± 0.008837 | 0.001996 ± 0.000111 | 1.000 |
| 1 | 0.051235 ± 0.004187 | 0.002609 ± 0.001327 | 1.000 |
Table B.34. PINN and FEM results with Vargha-Delaney A effect size.
| Noise | Brunner–Munzel | |||
| 0 | N/A | N/A | N/A | N/A |
| 0.001 | N/A | N/A | N/A | N/A |
| 0.01 | N/A | N/A | N/A | N/A |
| 0.05 | N/A | N/A | N/A | N/A |
| 0.1 | N/A | N/A | N/A | N/A |
| 0.25 | N/A | N/A | N/A | N/A |
| 0.5 | N/A | N/A | N/A | N/A |
| 1 | N/A | N/A | N/A | N/A |
Table B.35. Brunner-Munzel tests and adjusted p-values for PINN and FEM results.
| Noise | FEM | PINN/FEM | Vargha–Delaney |
| 0 | 0.001923 0.000019 | 0.006938 0.006173 | 0.000 |
| 0.001 | 0.001920 0.000001 | 0.012048 0.009203 | 0.000 |
| 0.01 | 0.001924 0.000018 | 0.009024 0.007570 | 0.036 |
| 0.05 | 0.001919 0.000001 | 0.011518 0.010064 | 0.000 |
| 0.1 | 0.001922 0.000006 | 0.010777 0.008486 | 0.000 |
| 0.25 | 0.001935 0.000022 | 0.013890 0.022805 | 0.033 |
| 0.5 | 0.001996 0.000111 | 0.012939 0.009445 | 0.041 |
| 1 | 0.002609 0.001327 | 0.010349 0.009772 | 0.155 |
Table B.36. FEM and PINN/FEM results with Vargha-Delaney A effect size.
| Noise | Brunner–Munzel | |||
| 0 | N/A | N/A | N/A | N/A |
| 0.001 | N/A | N/A | N/A | N/A |
| 0.01 | 13.868 | 1.880e 14 | 7.518e 14 | 3.927e 14 |
| 0.05 | N/A | N/A | N/A | N/A |
| 0.1 | N/A | N/A | N/A | N/A |
| 0.25 | 14.000 | 1.964e 14 | 7.518e 14 | 3.927e 14 |
| 0.5 | 13.473 | 4.124e 14 | 8.247e 14 | 5.498e 14 |
| 1 | 6.546 | 2.526e 08 | 2.526e 08 | 2.526e 08 |
Table B.37. Brunner-Munzel tests and adjusted p-values for FEM and PINN/FEM results.
| Noise | PINN | PINN/FEM | Vargha–Delaney |
| 0 | 0.032690 0.009598 | 0.006938 0.006173 | 0.982 |
| 0.001 | 0.039061 0.009850 | 0.012048 0.009203 | 0.969 |
| 0.01 | 0.036553 0.010552 | 0.009024 0.007570 | 0.964 |
| 0.05 | 0.036072 0.011541 | 0.011518 0.010064 | 0.940 |
| 0.1 | 0.036208 0.010680 | 0.010777 0.008486 | 0.939 |
| 0.25 | 0.037936 0.010059 | 0.013890 0.022805 | 0.967 |
| 0.5 | 0.040479 0.008837 | 0.012939 0.009445 | 0.985 |
| 1 | 0.051235 0.004187 | 0.010349 0.009772 | 1.000 |
Table B.38. PINN and PINN/FEM results with Vargha-Delaney A effect size.
| Noise | Brunner–Munzel | |||
| 0 | −40.513 | |||
| 0.001 | −24.209 | |||
| 0.01 | −24.061 | |||
| 0.05 | −15.114 | |||
| 0.1 | −14.911 | |||
| 0.25 | −14.000 | |||
| 0.5 | −31.771 | |||
| 1 | N/A | N/A | N/A | N/A |
Table B.39. Brunner-Munzel tests and adjusted p-values for PINN and PINN/FEM results.
Parameter results
In this section we report the mean and median bias of each model. The bias is defined as the deviation between the parameter predicted by each model and the ground truth parameter. Bootstrap 95% confidence intervals [56] are also reported for each bias value as well. We characterize the distribution of the bias by reporting IQR, skewness and minimum/maximum values. To determine whether the bias deviates from 0 in a statistically significant manner, we used a two-sided Wilcoxon signed-rank test [57] as the distributions of the biases are substantially skewed, in particular at high noise levels. The resulting -values are reported both in the original form and adjusted using the Holm family-wise error rate correction [54] as well as the Benjamini-Hochberg false discovery rate correction [55]. The Holm-based values below 0.05% are shown in bold. Note that for the deterministic FEM results for 1D Burgers’ equation with , skewness is undefined and the Wilcoxon test is uninformative. In these cases, we report the results as N/A.
1D Burgers’ equation
Ground truth value for unknown parameter in this problem is .
| Noise | Model | Mean Bias | Median Bias | 95% Bootstrap CI for Bias |
| 0 | PINN | −0.000098 | −0.000115 | [−0.000115, −0.000081] |
| 0.5 | PINN | −0.000342 | −0.000256 | [−0.000670, 0.000002] |
| 1 | PINN | −0.000394 | −0.001424 | [−0.001181, 0.000473] |
| 2 | PINN | −0.000536 | −0.002193 | [−0.001672, 0.000756] |
| 3 | PINN | −0.000644 | −0.002362 | [−0.002166, 0.001495] |
| 5 | PINN | 0.000348 | −0.002549 | [−0.002431, 0.004670] |
| 7 | PINN | −0.001204 | −0.002519 | [−0.002468, 0.000703] |
| 10 | PINN | 0.360167 | −0.001840 | [0.030164, 0.854303] |
| 25 | PINN | 25.445983 | 6.099766 | [9.407605, 49.872131] |
| 0 | FEM | 0.000131 | 0.000131 | [0.000131, 0.000131] |
| 0.5 | FEM | 0.000425 | 0.000489 | [0.000112, 0.000760] |
| 1 | FEM | 0.000276 | −0.000314 | [−0.000482, 0.001151] |
| 2 | FEM | 0.000763 | −0.001043 | [−0.000637, 0.002663] |
| 3 | FEM | 0.006658 | 0.002247 | [0.002790, 0.010983] |
| 5 | FEM | 0.150866 | 0.002336 | [0.009093, 0.359999] |
| 7 | FEM | 0.371069 | 0.007848 | [0.020457, 0.811692] |
| 10 | FEM | 0.487935 | 0.008563 | [0.075662, 1.016190] |
| 25 | FEM | 2.002900 | 0.254178 | [1.200200, 2.825378] |
Table C.40. Bias statistics for 1D Burgers’ equation tests.
| Noise | Model | IQR | Skewness | Minimum | Maximum |
| 0 | PINN | 0.000050 | 0.928 | −0.000188 | 0.000038 |
| 0.5 | PINN | 0.001440 | 0.589 | −0.001841 | 0.002329 |
| 1 | PINN | 0.002127 | 1.410 | −0.002722 | 0.005951 |
| 2 | PINN | 0.002275 | 2.044 | −0.002882 | 0.010696 |
| 3 | PINN | 0.000790 | 3.815 | −0.002874 | 0.023615 |
| 5 | PINN | 0.000558 | 4.101 | −0.002897 | 0.048962 |
| 7 | PINN | 0.000783 | 4.262 | −0.003113 | 0.021295 |
| 10 | PINN | 0.028501 | 4.200 | −0.003022 | 6.106257 |
| 25 | PINN | 20.202263 | 4.443 | −0.003014 | 322.680601 |
| 0 | FEM | 0.000000 | N/A | 0.000131 | 0.000131 |
| 0.5 | FEM | 0.001205 | 0.827 | −0.001199 | 0.003356 |
| 1 | FEM | 0.002442 | 1.824 | −0.002789 | 0.008702 |
| 2 | FEM | 0.003205 | 3.426 | −0.002492 | 0.021940 |
| 3 | FEM | 0.011753 | 1.847 | −0.003101 | 0.043822 |
| 5 | FEM | 0.029121 | 3.654 | −0.003147 | 2.155088 |
| 7 | FEM | 0.046579 | 3.314 | −0.003162 | 4.996817 |
| 10 | FEM | 0.136785 | 3.025 | −0.003150 | 4.996817 |
| 25 | FEM | 4.980715 | 0.463 | −0.003185 | 4.996817 |
Table C.41. Characteristics of distribution for 1D Burgers’ equation test errors.
| Noise | Model | ||||
| 0 | PINN | 5.0 | |||
| 0.5 | PINN | 138.0 | 0.0523 | 0.2613 | 0.0683 |
| 1 | PINN | 170.0 | 0.2054 | 0.7923 | 0.2328 |
| 2 | PINN | 126.0 | 0.0277 | 0.1664 | |
| 3 | PINN | 96.0 | 0.0040 | ||
| 5 | PINN | 59.0 | |||
| 7 | PINN | 61.0 | |||
| 10 | PINN | 169.0 | 0.1981 | 0.7923 | 0.2328 |
| 25 | PINN | 10.0 | |||
| 0 | FEM | N/A | N/A | N/A | N/A |
| 0.5 | FEM | 116.0 | 0.0155 | 0.1082 | |
| 1 | FEM | 223.0 | 0.8553 | 1.0000 | 0.8553 |
| 2 | FEM | 215.0 | 0.7303 | 1.0000 | 0.7760 |
| 3 | FEM | 110.0 | 0.0106 | 0.0894 | |
| 5 | FEM | 109.0 | 0.0099 | 0.0894 | |
| 7 | FEM | 71.0 | |||
| 10 | FEM | 73.0 | |||
| 25 | FEM | 5.0 |
Table C.42. Wilcoxon signed-rank test statistics for 1D Burgers’ equation.
1D Burgers' equation with two parameters
Ground truth value for viscosity parameter in this problem is .
Ground truth value for advection parameter in this problem is 0.55.
| Noise | Model | Mean Bias | Median Bias | 95% Bootstrap CI for Bias |
| 0 | PINN | 0.000193 | 0.000198 | [0.000181, 0.000206] |
| 0.5 | PINN | −0.000583 | −0.000531 | [−0.000947, −0.000226] |
| 1 | PINN | −0.001792 | −0.002553 | [−0.002328, −0.001170] |
| 2 | PINN | −0.002821 | −0.003034 | [−0.003021, −0.002507] |
| 3 | PINN | −0.002983 | −0.003060 | [−0.003058, −0.002897] |
| 5 | PINN | −0.000553 | −0.003103 | [−0.003084, 0.004446] |
| 7 | PINN | 0.061586 | −0.003117 | [−0.003069, 0.186921] |
| 10 | PINN | 0.256946 | −0.002925 | [0.009165, 0.617358] |
| 25 | PINN | 9.494277 | 1.600279 | [3.282049, 17.611974] |
| 0 | FEM | 0.000152 | 0.000152 | [0.000152, 0.000152] |
| 0.5 | FEM | 0.000162 | −0.000221 | [−0.000148, 0.000493] |
| 1 | FEM | 0.000824 | 0.000817 | [0.000287, 0.001370] |
| 2 | FEM | −0.000152 | −0.001336 | [−0.001145, 0.000951] |
| 3 | FEM | 0.001838 | 0.000419 | [0.000374, 0.003564] |
| 5 | FEM | 0.102238 | −0.000375 | [0.002358, 0.297202] |
| 7 | FEM | 0.562331 | −0.000480 | [0.077626, 1.136288] |
| 10 | FEM | 1.182931 | 0.002623 | [0.511359, 1.932753] |
| 25 | FEM | 1.268733 | 0.005378 | [0.566791, 2.052807] |
Table C.43. Bias statistics for viscosity prediction in 1D Burgers’ equation tests.
| Noise | Model | IQR | Skewness | Minimum | Maximum |
| 0 | PINN | 0.000060 | −0.116 | 0.000134 | 0.000247 |
| 0.5 | PINN | 0.001390 | −0.360 | −0.002745 | 0.001270 |
| 1 | PINN | 0.001450 | 2.036 | −0.003182 | 0.003860 |
| 2 | PINN | 0.000392 | 4.604 | −0.003183 | 0.001037 |
| 3 | PINN | 0.000287 | 1.605 | −0.003183 | −0.002260 |
| 5 | PINN | 0.000266 | 5.475 | −0.003183 | 0.071309 |
| 7 | PINN | 0.000263 | 5.469 | −0.003183 | 1.856309 |
| 10 | PINN | 0.005934 | 3.645 | −0.003183 | 3.807769 |
| 25 | PINN | 5.715438 | 3.112 | −0.003183 | 94.327661 |
| 0 | FEM | 0.000000 | N/A | 0.000152 | 0.000152 |
| 0.5 | FEM | 0.001013 | 0.987 | −0.001184 | 0.002327 |
| 1 | FEM | 0.002302 | 0.289 | −0.001783 | 0.003989 |
| 2 | FEM | 0.004060 | 1.225 | −0.002987 | 0.008324 |
| 3 | FEM | 0.005226 | 1.853 | −0.002894 | 0.018063 |
| 5 | FEM | 0.015898 | 5.470 | −0.003346 | 2.883208 |
| 7 | FEM | 0.014215 | 2.555 | −0.003350 | 4.996817 |
| 10 | FEM | 2.289449 | 1.209 | −0.003343 | 4.996817 |
| 25 | FEM | 1.494229 | 1.246 | −0.003355 | 4.996817 |
Table C.44. Characteristics of distribution for viscosity prediction in 1D Burgers’ equation.
| Noise | Model | ||||
| 0 | PINN | 0.0 | |||
| 0.5 | PINN | 103.0 | 0.0066 | 0.0598 | |
| 1 | PINN | 45.0 | |||
| 2 | PINN | 1.0 | |||
| 3 | PINN | 0.0 | |||
| 5 | PINN | 30.0 | |||
| 7 | PINN | 87.0 | 0.0020 | ||
| 10 | PINN | 199.0 | 0.5028 | 1.0000 | 0.5342 |
| 25 | PINN | 27.0 | |||
| 0 | FEM | N/A | N/A | N/A | N/A |
| 0.5 | FEM | 219.0 | 0.7922 | 1.0000 | 0.7922 |
| 1 | FEM | 107.0 | 0.0087 | 0.0696 | |
| 2 | FEM | 191.0 | 0.4045 | 1.0000 | 0.4584 |
| 3 | FEM | 155.0 | 0.1142 | 0.6851 | 0.1618 |
| 5 | FEM | 178.0 | 0.2710 | 1.0000 | 0.3291 |
| 7 | FEM | 173.0 | 0.2286 | 1.0000 | 0.2989 |
| 10 | FEM | 116.0 | 0.0155 | 0.1082 | |
| 25 | FEM | 85.0 | 0.0017 |
Table C.45. Wilcoxon signed-rank tests for viscosity prediction in 1D Burgers’ equation.
| Noise | Model | Mean Bias | Median Bias | 95% Bootstrap CI for Bias |
| 0 | PINN | 0.002167 | 0.002157 | [0.002081, 0.002256] |
| 0.5 | PINN | −0.005446 | 0.001737 | [−0.014797, 0.003965] |
| 1 | PINN | −0.038576 | −0.040225 | [−0.059253, −0.018518] |
| 2 | PINN | −0.080095 | −0.059642 | [−0.115137, −0.046315] |
| 3 | PINN | −0.111381 | −0.106573 | [−0.173821, −0.051607] |
| 5 | PINN | −0.217104 | −0.199926 | [−0.277550, −0.157503] |
| 7 | PINN | −0.231970 | −0.256692 | [−0.314987, −0.148053] |
| 10 | PINN | 0.083602 | −0.322993 | [−0.299589, 0.727331] |
| 25 | PINN | 0.742806 | −0.085778 | [0.031044, 1.713769] |
| 0 | FEM | 0.001291 | 0.001291 | [0.001291, 0.001291] |
| 0.5 | FEM | −0.004102 | −0.006705 | [−0.012382, 0.004651] |
| 1 | FEM | 0.009455 | 0.001445 | [−0.005047, 0.025132] |
| 2 | FEM | −0.002922 | −0.006106 | [−0.034036, 0.028089] |
| 3 | FEM | −0.023035 | −0.038868 | [−0.068635, 0.022375] |
| 5 | FEM | 0.027904 | −0.047465 | [−0.056421, 0.114937] |
| 7 | FEM | −0.017300 | −0.059742 | [−0.109609, 0.075631] |
| 10 | FEM | 0.007344 | 0.007008 | [−0.134890, 0.144440] |
| 25 | FEM | −0.046517 | −0.007033 | [−0.192782, 0.101437] |
Table C.46. Bias statistics for advection prediction in 1D Burgers’ equation tests.
| Noise | Model | IQR | Skewness | Minimum | Maximum |
| 0 | PINN | 0.000439 | 0.036 | 0.001703 | 0.002567 |
| 0.5 | PINN | 0.038467 | 0.064 | −0.052701 | 0.057314 |
| 1 | PINN | 0.071762 | −0.135 | −0.153387 | 0.079692 |
| 2 | PINN | 0.115984 | −0.403 | −0.258359 | 0.068890 |
| 3 | PINN | 0.132001 | −0.376 | −0.540691 | 0.177078 |
| 5 | PINN | 0.190839 | −0.605 | −0.550000 | 0.050067 |
| 7 | PINN | 0.413568 | 0.201 | −0.550000 | 0.203899 |
| 10 | PINN | 0.454597 | 5.029 | −0.550000 | 8.383439 |
| 25 | PINN | 1.174251 | 3.109 | −0.550000 | 10.835713 |
| 0 | FEM | 0.000000 | N/A | 0.001291 | 0.001291 |
| 0.5 | FEM | 0.020729 | 0.996 | −0.049999 | 0.063621 |
| 1 | FEM | 0.063369 | 0.543 | −0.051569 | 0.105989 |
| 2 | FEM | 0.111149 | 0.183 | −0.158631 | 0.171450 |
| 3 | FEM | 0.170158 | −0.113 | −0.349669 | 0.268005 |
| 5 | FEM | 0.365696 | 0.558 | −0.337237 | 0.450000 |
| 7 | FEM | 0.280093 | −0.057 | −0.550000 | 0.450000 |
| 10 | FEM | 0.809269 | −0.266 | −0.550000 | 0.450000 |
| 25 | FEM | 0.991587 | −0.040 | −0.550000 | 0.450000 |
Table C.47. Distribution characteristics for advection prediction in 1D Burgers’ equation.
| Noise | Model | ||||
| 0 | PINN | 0.0 | |||
| 0.5 | PINN | 184.0 | 0.3285 | 1.0000 | 0.5584 |
| 1 | PINN | 83.0 | 0.0015 | ||
| 2 | PINN | 56.0 | |||
| 3 | PINN | 94.0 | 0.0035 | ||
| 5 | PINN | 8.0 | |||
| 7 | PINN | 45.0 | |||
| 10 | PINN | 106.0 | 0.0081 | 0.0896 | |
| 25 | PINN | 209.0 | 0.6408 | 1.0000 | 0.7781 |
| 0 | FEM | N/A | N/A | N/A | N/A |
| 0.5 | FEM | 157.0 | 0.1241 | 1.0000 | 0.2638 |
| 1 | FEM | 191.0 | 0.4045 | 1.0000 | 0.5730 |
| 2 | FEM | 223.0 | 0.8553 | 1.0000 | 0.8553 |
| 3 | FEM | 189.0 | 0.3818 | 1.0000 | 0.5730 |
| 5 | FEM | 220.0 | 0.8078 | 1.0000 | 0.8553 |
| 7 | FEM | 205.0 | 0.5838 | 1.0000 | 0.7634 |
| 10 | FEM | 221.0 | 0.8236 | 1.0000 | 0.8553 |
| 25 | FEM | 177.0 | 0.2621 | 1.0000 | 0.4951 |
Table C.48. Wilcoxon signed-rank tests for advection prediction in 1D Burgers’ equation.
2D Taylor-Green Vortex
Ground truth value for unknown parameter in this problem is 0.1.
| Noise | Model | Mean Bias | Median Bias | 95% Bootstrap CI for Bias |
| 0 | PINN | −0.000558 | 0.000026 | [−0.001053, −0.000148] |
| 0.001 | PINN | −0.000524 | −0.000053 | [−0.001027, −0.000145] |
| 0.01 | PINN | −0.000184 | −0.000039 | [−0.000496, 0.000041] |
| 0.05 | PINN | −0.000889 | −0.000776 | [−0.001333, −0.000462] |
| 0.1 | PINN | −0.002075 | −0.002093 | [−0.002659, −0.001515] |
| 0.25 | PINN | −0.005316 | −0.005385 | [−0.006187, −0.004425] |
| 0.5 | PINN | −0.008795 | −0.007857 | [−0.009940, −0.007670] |
| 1 | PINN | −0.009789 | −0.009772 | [−0.012159, −0.007637] |
| 0 | FEM | 0.000104 | 0.000104 | [0.000104, 0.000104] |
| 0.001 | FEM | 0.000104 | 0.000104 | [0.000103, 0.000104] |
| 0.01 | FEM | 0.000099 | 0.000099 | [0.000092, 0.000106] |
| 0.05 | FEM | 0.000084 | 0.000065 | [0.000041, 0.000126] |
| 0.1 | FEM | 0.000170 | 0.000208 | [0.000101, 0.000237] |
| 0.25 | FEM | 0.000117 | 0.000077 | [−0.000069, 0.000304] |
| 0.5 | FEM | 0.000087 | 0.000007 | [−0.000284, 0.000469] |
| 1 | FEM | 0.000120 | −0.000161 | [−0.000419, 0.000667] |
Table C.49. Bias statistics for viscosity prediction in 2D Taylor-Green Vortex tests.
| Noise | Model | IQR | Skewness | Minimum | Maximum |
| 0 | PINN | 0.000214 | 0.000106 | ||
| 0.001 | PINN | 0.000161 | 0.000245 | ||
| 0.01 | PINN | 0.000244 | 0.000770 | ||
| 0.05 | PINN | 0.001946 | 0.001184 | ||
| 0.1 | PINN | 0.002040 | 0.001228 | ||
| 0.25 | PINN | 0.002774 | 0.181 | 0.000586 | |
| 0.5 | PINN | 0.004669 | |||
| 1 | PINN | 0.006672 | |||
| 0 | FEM | 0.000000 | 0.000103 | 0.000104 | |
| 0.001 | FEM | 0.000002 | 0.816 | 0.000101 | 0.000108 |
| 0.01 | FEM | 0.000018 | 0.000044 | 0.000142 | |
| 0.05 | FEM | 0.000162 | 0.006 | 0.000283 | |
| 0.1 | FEM | 0.000229 | 0.000570 | ||
| 0.25 | FEM | 0.000998 | 0.056 | 0.000950 | |
| 0.5 | FEM | 0.001626 | 0.389 | 0.002386 | |
| 1 | FEM | 0.002415 | 0.416 | 0.002966 |
Table C.50. Distribution characteristics for viscosity prediction in 2D Taylor-Green vortex.
| Noise | Model | ||||
| 0 | PINN | 189.0 | 0.3818 | 1.0000 | 0.4699 |
| 0.001 | PINN | 112.0 | 0.0120 | 0.0723 | |
| 0.01 | PINN | 200.0 | 0.5158 | 1.0000 | 0.5895 |
| 0.05 | PINN | 79.0 | 0.0010 | ||
| 0.1 | PINN | 16.0 | |||
| 0.25 | PINN | 1.0 | |||
| 0.5 | PINN | 0.0 | |||
| 1 | PINN | 0.0 | |||
| 0 | FEM | 0.0 | |||
| 0.001 | FEM | 0.0 | |||
| 0.01 | FEM | 0.0 | |||
| 0.05 | FEM | 72.0 | |||
| 0.1 | FEM | 51.0 | |||
| 0.25 | FEM | 184.0 | 0.3285 | 1.0000 | 0.4380 |
| 0.5 | FEM | 217.0 | 0.7611 | 1.0000 | 0.8118 |
| 1 | FEM | 224.0 | 0.8712 | 1.0000 | 0.8712 |
Table C.51. Wilcoxon signed-rank tests for viscosity prediction in 2D Taylor-Green Vortex.
3D Taylor-Green Vortex, with outliers
Ground truth value for unknown parameter in this problem is 0.01.
| Noise | Model | Mean Bias | Median Bias | 95% Bootstrap CI for Bias |
| 0 | PINN | 0.000635 | 0.001489 | [−0.000784, 0.001951] |
| 0.001 | PINN | −0.001079 | −0.000067 | [−0.002878, 0.000755] |
| 0.01 | PINN | 0.000541 | 0.000964 | [−0.001189, 0.002187] |
| 0.05 | PINN | −0.001477 | −0.000084 | [−0.003103, 0.000115] |
| 0.1 | PINN | −0.001392 | −0.000025 | [−0.003050, 0.000192] |
| 0.25 | PINN | 0.003627 | 0.001124 | [−0.002388, 0.013921] |
| 0.5 | PINN | −0.000971 | −0.001744 | [−0.003294, 0.001756] |
| 1 | PINN | −0.002776 | −0.002551 | [−0.003954, −0.001583] |
| 0 | FEM | −0.000041 | 0.000294 | [−0.000603, 0.000295] |
| 0.001 | FEM | 0.166339 | 0.000294 | [−0.000220, 0.499160] |
| 0.01 | FEM | 0.493919 | 0.000293 | [−0.000174, 0.996085] |
| 0.05 | FEM | 0.165154 | 0.000289 | [−0.001825, 0.498047] |
| 0.1 | FEM | 0.165684 | 0.000313 | [−0.001089, 0.498493] |
| 0.25 | FEM | 0.329266 | 0.000243 | [−0.000306, 0.824930] |
| 0.5 | FEM | −0.000425 | 0.000278 | [−0.001250, 0.000238] |
| 1 | FEM | 0.165420 | 0.000112 | [−0.001460, 0.498441] |
Table C.52. Bias statistics for viscosity prediction in 3D Taylor-Green Vortex tests.
| Noise | Model | IQR | Skewness | Minimum | Maximum |
| 0 | PINN | 0.004957 | −0.759 | −0.006858 | 0.006629 |
| 0.001 | PINN | 0.009734 | 0.065 | −0.006858 | 0.008105 |
| 0.01 | PINN | 0.005261 | −0.349 | −0.006858 | 0.009968 |
| 0.05 | PINN | 0.009177 | −0.185 | −0.006858 | 0.004507 |
| 0.1 | PINN | 0.009413 | −0.173 | −0.006858 | 0.005126 |
| 0.25 | PINN | 0.009551 | 5.233 | −0.006858 | 0.141775 |
| 0.5 | PINN | 0.009039 | 1.981 | −0.006858 | 0.026158 |
| 1 | PINN | 0.006602 | 0.210 | −0.006858 | 0.004680 |
| 0 | FEM | 0.000000 | −4.316 | −0.006475 | 0.000316 |
| 0.001 | FEM | 0.000001 | 5.477 | −0.003751 | 4.988961 |
| 0.01 | FEM | 0.000008 | 2.810 | −0.006858 | 4.987737 |
| 0.05 | FEM | 0.000054 | 5.477 | −0.006858 | 4.985034 |
| 0.1 | FEM | 0.000093 | 5.477 | −0.006858 | 4.985623 |
| 0.25 | FEM | 0.000173 | 3.661 | −0.005090 | 4.989280 |
| 0.5 | FEM | 0.000429 | −2.445 | −0.006858 | 0.001019 |
| 1 | FEM | 0.001526 | 5.477 | −0.006858 | 4.988669 |
Table C.53. Distribution characteristics for viscosity prediction in 3D Taylor-Green vortex.
| Noise | Model | ||||
| 0 | PINN | 173.0 | 0.2286 | 1.0000 | 0.2813 |
| 0.001 | PINN | 167.0 | 0.1840 | 1.0000 | 0.2453 |
| 0.01 | PINN | 192.0 | 0.4161 | 1.0000 | 0.4756 |
| 0.05 | PINN | 160.0 | 0.1403 | 1.0000 | 0.2209 |
| 0.1 | PINN | 162.0 | 0.1519 | 1.0000 | 0.2209 |
| 0.25 | PINN | 205.0 | 0.5838 | 1.0000 | 0.5838 |
| 0.5 | PINN | 155.0 | 0.1142 | 1.0000 | 0.2209 |
| 1 | PINN | 65.0 | |||
| 0 | FEM | 59.0 | |||
| 0.001 | FEM | 57.0 | |||
| 0.01 | FEM | 78.0 | |||
| 0.05 | FEM | 159.0 | 0.1347 | 1.0000 | 0.2209 |
| 0.1 | FEM | 135.0 | 0.0449 | 0.4940 | 0.1198 |
| 0.25 | FEM | 56.0 | |||
| 0.5 | FEM | 144.0 | 0.0699 | 0.6989 | 0.1598 |
| 1 | FEM | 204.0 | 0.5699 | 1.0000 | 0.5838 |
Table C.54. Wilcoxon signed-rank tests for viscosity prediction in 3D Taylor-Green Vortex.
3D Taylor-Green Vortex, without outliers
Ground truth value for unknown parameter in this problem is 0.01.
| Noise | Model | Mean Bias | Median Bias | 95% Bootstrap CI for Bias |
| 0 | PINN | 0.000635 | 0.001489 | [−0.000784, 0.001951] |
| 0.001 | PINN | −0.001079 | −0.000067 | [−0.002878, 0.000755] |
| 0.01 | PINN | 0.000541 | 0.000964 | [−0.001189, 0.002187] |
| 0.05 | PINN | −0.001477 | −0.000084 | [−0.003103, 0.000115] |
| 0.1 | PINN | −0.001392 | −0.000025 | [−0.003050, 0.000192] |
| 0.25 | PINN | −0.001137 | 0.001096 | [−0.002790, 0.000444] |
| 0.5 | PINN | −0.001906 | −0.002592 | [−0.003642, −0.000099] |
| 1 | PINN | −0.002776 | −0.002551 | [−0.003955, −0.001550] |
| 0 | FEM | 0.000294 | 0.000294 | [0.000294, 0.000294] |
| 0.001 | FEM | 0.000294 | 0.000294 | [0.000293, 0.000294] |
| 0.01 | FEM | 0.000294 | 0.000294 | [0.000292, 0.000296] |
| 0.05 | FEM | 0.000298 | 0.000298 | [0.000289, 0.000307] |
| 0.1 | FEM | 0.000312 | 0.000329 | [0.000291, 0.000331] |
| 0.25 | FEM | 0.000246 | 0.000243 | [0.000198, 0.000294] |
| 0.5 | FEM | 0.000350 | 0.000328 | [0.000217, 0.000474] |
| 1 | FEM | −0.000206 | 0.000133 | [−0.000704, 0.000229] |
Table C.55. Bias statistics for viscosity prediction in 3D Taylor-Green Vortex tests.
| Noise | Model | IQR | Skewness | Minimum | Maximum |
| 0 | PINN | 0.004957 | −0.759 | −0.006858 | 0.006629 |
| 0.001 | PINN | 0.009734 | 0.065 | −0.006858 | 0.008105 |
| 0.01 | PINN | 0.005261 | −0.349 | −0.006858 | 0.009968 |
| 0.05 | PINN | 0.009177 | −0.185 | −0.006858 | 0.004507 |
| 0.1 | PINN | 0.009413 | −0.173 | −0.006858 | 0.005126 |
| 0.25 | PINN | 0.009237 | −0.319 | −0.006858 | 0.005572 |
| 0.5 | PINN | 0.009002 | 0.411 | −0.006858 | 0.009137 |
| 1 | PINN | 0.006602 | 0.210 | −0.006858 | 0.004680 |
| 0 | FEM | 0.000000 | 0.658 | 0.000294 | 0.000294 |
| 0.001 | FEM | 0.000001 | −0.474 | 0.000292 | 0.000295 |
| 0.01 | FEM | 0.000005 | 0.298 | 0.000283 | 0.000307 |
| 0.05 | FEM | 0.000031 | −0.264 | 0.000255 | 0.000334 |
| 0.1 | FEM | 0.000074 | −0.200 | 0.000216 | 0.000423 |
| 0.25 | FEM | 0.000105 | 0.141 | −0.000024 | 0.000554 |
| 0.5 | FEM | 0.000356 | −0.336 | −0.000417 | 0.001019 |
| 1 | FEM | 0.000897 | −1.577 | −0.003265 | 0.001259 |
Table C.56. Distribution characteristics for viscosity prediction in 3D Taylor-Green vortex.
| Noise | Model | ||||
| 0 | PINN | 173.0 | 0.2286 | 0.9820 | 0.2813 |
| 0.001 | PINN | 167.0 | 0.1840 | 0.9820 | 0.2453 |
| 0.01 | PINN | 192.0 | 0.4161 | 1.0000 | 0.4439 |
| 0.05 | PINN | 160.0 | 0.1403 | 0.9820 | 0.2209 |
| 0.1 | PINN | 162.0 | 0.1519 | 0.9820 | 0.2209 |
| 0.25 | PINN | 175.0 | 0.3692 | 1.0000 | 0.4220 |
| 0.5 | PINN | 125.0 | 0.0455 | 0.3642 | 0.0809 |
| 1 | PINN | 65.0 | |||
| 0 | FEM | 0.0 | |||
| 0.001 | FEM | 0.0 | |||
| 0.01 | FEM | 0.0 | |||
| 0.05 | FEM | 0.0 | |||
| 0.1 | FEM | 0.0 | |||
| 0.25 | FEM | 1.0 | |||
| 0.5 | FEM | 30.0 | |||
| 1 | FEM | 151.0 | 0.7712 | 1.0000 | 0.7712 |
Table C.57. Wilcoxon signed-rank tests for viscosity prediction in 3D Taylor-Green Vortex.
1D Burgers' equation with two unknown parameters
We also tested 1D Burgers’ equation with two parameters.
The parameter added was an advection parameter, , which scales the advection. These tests may give insight in how the behavior of the models is affected when identifying multiple parameters. The ground truth data was created from an analytical solution using the Cole-Hopf transformation. To be consistent with the scaling, the equation was reformulated in terms of the rescaled variable . After applying the standard Cole-Hopf transform to , the physical velocity field was recovered using . The ground truth solutions for the unknown parameters were and .
The ranking between the models were very similar as with one parameter, however overall all the models performed slightly worse. Compared to the Navier-Stokes experiments, the standard 1D Burgers’ equation tests showed less of the biased parameter learning and exploding physics loss that we saw in the more complex problems. However, in the 1D Burgers’ equation tests with two unknown parameters, these phenomena were more pronounced.

Figure D.7. Results for 1D Burgers’ equation with two unknown parameters on test set based on 30 runs of each model. Standard deviation () is the noise added to the training data. Top left: Prediction accuracies, . Top right: Prediction accuracies, . Center left: Viscosity identification accuracies, . Center right: Viscosity identification accuracies, . Bottom left: Advection identification accuracies, . Bottom right: Advection identification accuracies,
Predictions graphs in MSE
1D Burgers' equation
!Prediction accuracies for 1D Burgers’ equation with bbox=0.227,0.252,0.480,0.378 !Prediction accuracies for 1D Burgers’ equation with bbox=0.541,0.252,0.805,0.378
Figure E.8. Results for 1D Burgers’ equation on test set based on 30 runs of each model. Left: Prediction accuracies, . Right: Prediction accuracies, .
1D Burgers' equation with two parameters
!Prediction accuracies for 1D Burgers’ equation with two parameters and bbox=0.227,0.528,0.480,0.654 !Prediction accuracies for 1D Burgers’ equation with two parameters and bbox=0.541,0.528,0.805,0.654
Figure E.9. Results for 1D Burgers’ equation with two parameters on test set based on 30 runs of each model. Left: Prediction accuracies, . Right: Prediction accuracies, .
2D Taylor-Green Vortex
!2D Taylor-Green Vortex prediction accuracy for standard deviation bbox=0.25,0.22,0.49,0.39
(a)
!2D Taylor-Green Vortex prediction accuracy for standard deviation bbox=0.54,0.22,0.80,0.39
(b)
Figure E.10. Results for 2D Taylor-Green Vortex on test set based on 30 runs of each model. Left: Prediction accuracies, . Right: Prediction accuracies, .
3D Taylor-Green Vortex
(a)
(b)
Figure E.11. Results for 3D Taylor-Green Vortex on test set based on 30 runs of each model. Left: Prediction accuracies with outliers excluded. Right: Prediction accuracies with outliers included.
Noise levels tested
1D Burgers’ equation
| SNR | SNR (dB) | |
| 0 | — | |
| 0.5 | 1.5048 | 1.77 |
| 1 | 0.3762 | -4.25 |
| 2 | 0.0940 | -10.27 |
| 3 | 0.0418 | -13.79 |
| 5 | 0.0150 | -18.23 |
| 7 | 0.0077 | -21.15 |
| 10 | 0.0038 | -24.25 |
| 25 | 0.0006 | -32.20 |
Table 2.
1D Burgers’ equation with two parameters
| SNR | SNR (dB) | |
| 0 | — | |
| 0.5 | 1.8476 | 2.67 |
| 1 | 0.4619 | -3.35 |
| 2 | 0.1155 | -9.38 |
| 3 | 0.0513 | -12.90 |
| 5 | 0.0185 | -17.33 |
| 7 | 0.0094 | -20.26 |
| 10 | 0.0046 | -23.35 |
| 25 | 0.0007 | -31.31 |
Table 3.
2D Taylor-Green Vortex
| SNR | SNR (dB) | |
| 0 | — | |
| 0.001 | 114885.2596 | 50.60 |
| 0.01 | 1148.8526 | 30.60 |
| 0.05 | 45.9541 | 16.62 |
| 0.1 | 11.4885 | 10.60 |
| 0.25 | 1.8382 | 2.64 |
| 0.5 | 0.4595 | -3.38 |
| 1 | 0.1149 | -9.40 |
Table 4.
3D Taylor-Green Vortex
| SNR | SNR (dB) | |
| 0 | — | |
| 0.001 | 69943.5822 | 48.45 |
| 0.01 | 699.4358 | 28.45 |
| 0.05 | 27.9774 | 14.47 |
| 0.1 | 6.9944 | 8.45 |
| 0.25 | 1.1191 | 0.49 |
| 0.5 | 0.2798 | -5.53 |
| 1 | 0.0699 | -11.55 |
Table 1.
Additional graph
Figure 1 (PDF p. 76). 1D Burger's equation, parameter discovered
References
- [1]L. Evans, Partial Differential Equations, Graduate studies in mathematics, 2 ed., American Mathematical Society, 2010.
- [2]V. Ryaben’kii, et al., A Theoretical Introduction to Numerical Analysis, CRC Press, 2006.
- [3]C. R. Vogel, Computational Methods for Inverse Problems, volume 23 of Frontiers in Applied Mathematics, SIAM, 2002.DOI
- [4]M. Raissi, et al., Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, Journal of Computational Physics 378 (2019) 686–707.DOI
- [5]Z. Miao, Y. Chen, Vc-pinn: Variable coefficient physics-informed neural network for forward and inverse problems of pdes with variable coefficient, Physica D: Nonlinear Phenomena 456 (2023) 133945. URL: https://www.sciencedirect.com/science/article/pii/S0167278923002993. doi:https://doi.org/10.1016/j.physd.2023.133945.
- [6]S. Thakur, E. Esmaili, S. Libring, L. Solorio, A. Ardekani, Inverse resolution of spatially varying diffusion coefficient using physics-informed neural networks, Physics of Fluids 36 (2024). doi:10.1063/5.0207453.DOI
- [7]S. Cuomo, et al., Scientific machine learning through physics-informed neural networks: Where we are and what’s next, 2022. URL: https://arxiv.org/abs/2201.05624. arXiv:2201.05624.
- [8]N. McGreivy, et al., Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations, 2024. URL: https://arxiv.org/abs/2407.07218. arXiv:2407.07218.
- [9]G. Karniadakis, et al., Physics-informed machine learning, Nature Reviews Physics 3 (2021) 422–440. doi:10.1038/s42254-021-00314-5.DOI
- [10]T. G. Grossmann, et al., Can physics-informed neural networks beat the finite element method?, 2023. URL: https://arxiv.org/abs/2302.04107. arXiv:2302.04107.
- [11]F. Fernández de la Mata, et al., Physics-informed neural networks for data-driven simulation: Advantages, limitations, and opportunities, Physica A: Statistical Mechanics and its Applications 610 (2023). doi:10.1016/j.physa.2022.128415.DOI
- [12]P.-Y. Chuang, et al., Experience report of physics-informed neural networks in fluid simulations: pitfalls and frustration, 2022. URL: https://arxiv.org/abs/2205.14249. arXiv:2205.14249.
- [13]Z. Hu, et al., Tackling the curse of dimensionality with physics-informed neural networks, Neural Networks 176 (2024). doi:https://doi.org/10.1016/j.neunet.2024.106369.
- [14]X. Jin, et al., Nsf nets (navier-stokes flow nets): Physics-informed neural networks for the incompressible navier-stokes equations, Journal of Computational Physics 426 (2021) 109951. URL: https://www.sciencedirect.com/science/article/pii/S0021999120307257. doi:https://doi.org/10.1016/j.jcp.2020.109951.
- [15]M. Raissi, et al., Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations. Science 367 (2020) 1026–1030. URL: https://www.science.org/doi/abs/10.1126/science.aaw4741. doi:10.1126/science.aaw4741.DOI
- [16]R. Sharma, et al., Inversion of dc resistivity data using physics-informed neural networks, 2024. URL: https://arxiv.org/abs/2408.02420. arXiv:2408.02420.
- [17]X. Yang, et al., A two-stage imaging framework combining cnn and physics-informed neural networks for full-inverse tomography: A case study in electrical impedance tomography (eit), 2025. URL: https://arxiv.org/abs/2407.17721. arXiv:2407.17721.
- [18]K. Yokota, et al., Identification of physical properties in acoustic tubes using physics-informed neural networks, 2024. URL: https://arxiv.org/abs/2406.11119. arXiv:2406.11119.
- [19]M. Berardi, et al., Inverse physics-informed neural networks for transport models in porous materials, 2024. URL: https://arxiv.org/abs/2407.10654. arXiv:2407.10654.
- [20]Y. Luo, et al., Cfdbench: A large-scale benchmark for machine learning methods in fluid dynamics, 2024. URL: https://arxiv.org/abs/2310.05963. arXiv:2310.05963.
- [21]Z. Hao, J. Yao, C. Su, H. Su, Z. Wang, F. Lu, Z. Xia, Y. Zhang, S. Liu, L. Lu, J. Zhu, Pinnacle: A comprehensive benchmark of physics-informed neural networks for solving pdes, 2023. URL: https://arxiv.org/abs/2306.08827. arXiv:2306.08827.
- [22]P. Karnakov, et al., Solving inverse problems in physics by optimizing a discrete loss: Fast and accurate learning without neural networks, PNAS Nexus 3 (2023). URL: http://dx.doi.org/10.1093/pnasnexus/pgae005. doi:10.1093/pnasnexus/pgae005.
- [23]S. Wang, S. Sankaran, H. Wang, P. Perdikaris, An expert’s guide to training physics-informed neural networks, 2023. URL: https://arxiv.org/abs/2308.08468. arXiv:2308.08468.
- [24]S. Wang, et al., Understanding and mitigating gradient pathologies in physics-informed neural networks, 2020. URL: https://arxiv.org/abs/2001.04536. arXiv:2001.04536.
- [25]E. Kharazmi, Z. Zhang, G. E. Karniadakis, hp-vpinns: Variational physics-informed neural networks with domain decomposition, Computer Methods in Applied Mechanics and Engineering 374 (2021) 113547. URL: http://dx.doi.org/10.1016/j.cma.2020.113547. doi:10.1016/j.cma.2020.113547.
- [26]M. Tancik, P. P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ramamoorthi, J. T. Barron, R. Ng, Fourier features let networks learn high frequency functions in low dimensional domains, 2020. URL: https://arxiv.org/abs/2006.10739. arXiv:2006.10739.
- [27]S. Wang, et al., Respecting causality is all you need for training physics-informed neural networks, 2022. URL: https://arxiv.org/abs/2203.07404. arXiv:2203.07404.
- [28]A. New, M. Villafañe-Delgado, C. Shugert, Equation identification for fluid flows via physics-informed neural networks, 2024. URL: https://arxiv.org/abs/2408.17271. arXiv:2408.17271.
- [29]M. Aliakbari, M. Soltany Sadrabadi, P. Vadasz, A. Arzani, Ensemble physics informed neural networks: A framework to improve inverse transport modeling in heterogeneous domains, Physics of Fluids 35 (2023) 053616. URL: https://doi.org/10.1063/5.0150016. doi:10.1063/5.0150016.
- [30]I. Goodfellow, et al., Deep Learning, MIT Press, 2016. http://www.deeplearningbook.org.
- [31]D. P. Kingma, et al., Adam: A method for stochastic optimization, 2017. URL: https://arxiv.org/abs/1412.6980. arXiv:1412.6980.
- [32]J. F. Urbán, P. Stefanou, J. A. Pons, Unveiling the optimization process of physics informed neural networks: How accurate and competitive can pinns be?, Journal of Computational Physics 523 (2025) 113656. URL: https://www.sciencedirect.com/science/article/pii/S0021999124009045. doi:https://doi.org/10.1016/j.jcp.2024.113656.
- [33]A. S. Lewis, S. Zhang, Nonsmoothness and the bfgs method, 2013. URL: https://people.orie.cornell.edu/aslewis/publications/sub-nonsmoothness.pdf, preprint, Cornell University.
- [34]R. Courant, Variational methods for the solution of problems of equilibrium and vibrations, Bulletin of the American Mathematical Society 49 (1943) 1–23.
- [35]P. Seshu, Textbook of Finite Element Analysis, PHI Learning, 2003.
- [36]W. K. Liu, et al., Eighty years of the finite element method: Birth, evolution, and future, Archives of Computational Methods in Engineering 29 (2022) 4431–4453. doi:10.1007/s11831-022-09740-9.DOI
- [37]D. Kraft, A Software Package for Sequential Quadratic Programming, Technical Report DFVLR-FB 88-28, DLR German Aerospace Center, Köln, Germany, 1988.
- [38]P. Virtanen, et al., SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python, Nature Methods 17 (2020) 261–272.
- [39]S. K. Mitusch, et al., dolfin-adjoint 2018.1: automated adjoints for FEniCS and Firedrake, Journal of Open Source Software 4 (2019) 1292. doi:10.21105/joss.01292.DOI
- [40]dolfin-adjoint developers, Inverse parameter identification fails - how to use discrete measurements?, 2021. URL: https://github.com/dolfin-adjoint/dolfin-adjoint/issues/, gitHub issue #7.
- [41]A. Paszke, et al., Pytorch: An imperative style, high-performance deep learning library, 2019. URL: https://arxiv.org/abs/1912.01703.
- [42]Norwegian University of Science and Technology, Idun hpc cluster, https://www.hpc.ntnu.no/idun/, 2025. Accessed: 2025-08-25.
- [43]T. Akiba, S. Sano, T. Yanase, T. Ohta, M. Koyama, Optuna: A next-generation hyperparameter optimization framework, in: KDD ’19: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, 2019, pp. 2623–2631.arxiv.org/abs/1907.10902
- [44]C. D. Cantwell, et al., Nektar++: An open-source spectral/hp element framework, Computer Physics Communications 192 (2015) 205–219. URL: https://www.sciencedirect.com/science/article/pii/S0010465515000533.
- [45]Department of Aeronautics, Imperial College London, UK, et al., Quasi-3D Computation of the Taylor-Green Vortex Flow, 2024. URL: https://doc.nektar.info/tutorials/latest/incns/taylor-green-vortex/incns-taylor-green-vortex.pdf.
- [46]C. R. Harris, et al., Array programming with NumPy, Nature 585 (2020) 357–362. URL: https://doi.org/10.1038/s41586-020-2649-2. doi:10.1038/s41586-020-2649-2.
- [47]R. E. Walpole, R. H. Myers, S. L. Myers, K. Ye, Probability and Statistics for Engineers and Scientists, 9 ed., Pearson, 2016.
- [48]R. J. Hyndman, G. Athanasopoulos, Forecasting: Principles and Practice, 3rd ed., OTexts, Melbourne, Australia, 2021. URL: https://OTexts.com/fpp3, accessed on 2026-02-13.
- [49]A. Irpan, Deep reinforcement learning doesn’t work yet, https://www.alexirpan.com/2018/02/14/rl-hard.html, 2018.
- [50]J. Taylor, et al., Optimizing the optimizer for data driven deep neural networks and physics informed neural networks, 2022. URL: https://arxiv.org/abs/2205.07430. arXiv:2205.07430.
- [51]A. J. Chorin, A numerical method for solving incompressible viscous flow problems, Journal of Computational Physics 2 (1967) 12–26. doi:https://doi.org/10.1016/0021-9991(67)90037-X.
- [52]A. Vargha, H. D. Delaney, A critique and improvement of the cl common language effect size statistics of mcgraw and wong, Journal of Educational and Behavioral Statistics 25 (2000) 101 – 132.
- [53]E. Brunner, U. Munzel, The nonparametric behrens-fisher problem: Asymptotic theory and a small-sample approximation, Biometrical Journal 42 (2000) 17–25.doi.org/10.1002/(sici)1521-4036(200001)42:1<17::aid-bimj17>3.0.co;2-u