WorksComputer Science
Some Best Practices in Operator Learning
Abstract
Hyperparameters searches are computationally expensive. This paper studies some general choices of hyperparameters and training methods specifically for operator learning. It considers the architectures DeepONets, Fourier neural operators and Koopman autoencoders for several differential equations to find robust trends. Some options considered are activation functions, dropout and stochastic weight averaging.
Introduction
A neural operator is a neural network that is intended to approximate an operator between function spaces [1, 2, 3]. An example of an output function for a neural operator is a solution to a differential equation. Examples of input functions for a neural operator are the initial conditions or the boundary conditions for the differential equation. The study of neural operators is called operator learning.
Various choices of hyperparameters and training strategies are made when training a neural network. Limiting such choices to known robust choices can significantly improve the speed of hyperparameters searches. This paper studies the effect of some of these choices in operator learning, that is, it tries to find some best practices in operator learning. This is not a comparison between different architectures. In order, this paper studies the choice of activation function, the use of dropout, the use of stochastic weight averaging and the use of a learning rate finder.
Architectures
This section introduces the architectures used in this paper. These are DeepONets, Fourier neural operators and Koopman autoencoders.
DeepONets
Deep neural operators, which are abbreviated as DeepONets, are a neural operator architecture [4, 5, 6, 7, 8, 9]. They consist of two parts. The first part encodes information about the differential equation into a latent space. It is called the branch network. The second part encodes a position into a latent space. It is called the trunk network. An example input for the branch network is the initial condition for a dynamic differential equation, and an example for the trunk network is the space-time point to evaluate the solution to the differential equation. In practice, there is usually only one forward pass for the branch network, but many forward passes for the trunk network to encode all the desired positions.
The dimensions of the latent spaces of the branch network and the trunk network are the same. And, the output of such a model is the dot product between both latent spaces. Intuitively, the branch network encodes the solution to the differential equation as a set of basis functions, and the trunk network encodes the position as the coefficients of these basis functions.
Figure 1. The DeepONet architecture: The input is the input function, and the input is the point where the output function is evaluated. Their encodings are denoted by and , respectively. The output is denoted by .
Fourier Operator Networks
Fourier neural operators, which are abbreviated as FNO, are a neural operator architecture [9, 10, 11, 12, 13, 14, 15, 16]. They are based on spectral convolution layers.
In a spectral convolution layer, first the Fourier transform is applied. Then, the higher modes are dropped. Then, a linear function is applied. Then, the inverse Fourier transform is applied [17]. Similarly to residuals for a normal convolution layer, a linear function is applied to the original input, and the output is added elementwise to the output of the inverse Fourier transform. Finally, an activation function is applied.
While a normal convolution layer is local, a spectral convolution layer is global. Furthermore, for a spectral convolution layer to be computationally reasonable, the fast Fourier transform needs to be used [18]. Thus, the data needs to be structured as a grid.
First, in a Fourier neural operator, a feed-forward neural network is applied to increase the channel dimension. Then, a sequence of spectral convolution layers are applied. Then, a feed-forward neural network is applied to decrease the channel dimension to the output size.
The original channels include the input function, such as the initial conditions. For time-independent differential equation, the Fourier neural network can be applied for each time step, or the desired time can be included in the channels. In this paper, the time is included in the channels, that is, for each desired time, the Fourier neural network is applied to the initial conditions together with the desired time. When using grids of various resolutions, the spatial step size is also included. And, for inhomogeneous equations, the spatial position is also included.

Figure 2. A Fourier neural operator: The input is denoted by . First, a feed-forward neural network is used to increase the channel dimension. Then, a sequence of spectral convolution layers are applied. Then, a feed-forward neural network is used to decrease the channel dimension. The output is denoted by .
Figure 3. A spectral convolution layer: The input is denoted by . On the bottom, a linear layer is applied to each channel. On the top, first the fast Fourier transform is applied. Then, the higher modes are dropped. Then, a linear layer is applied to each channel. Then, the inverse fast Fourier transform is applied. Then, the top and bottom are added elementwise, and an activation function is applied. The output is denoted by .
Koopman Autoencoders
Koopman autoencoders are a neural operator architecture that are used for time-dependent differential equations [19, 20, 21, 22, 23]. They are popular for dynamic mode decomposition [24, 25, 26, 27, 28] and control [29, 30, 31, 32, 33, 34, 35, 36].
The Koopman formulation of classical mechanics is an alternative framework of classical mechanics [37, 38, 39]. It was inspired by the Hamiltonian formulation of quantum mechanics [40, 41, 42]. In this theory, the physical state at a given time is represented as a state in an infinite-dimensional latent space, and an infinite-dimensional operator governs the time evolution in this latent space. Intuitively, the Koopman formulation can be thought of as removing nonlinearity by infinitely increasing the dimension.
The Koopman formulation of classical mechanics can be discretized to provide a numerical scheme by approximating the time-evolution operator by a finite-dimensional matrix . In this scheme, a physical state can be evolved into a later physical state by encoding it into the latent space, applying the matrix repetitively and then decoding it back to the physical space, that is, the equation
approximately holds, where is a discretized encoder and is a discretized decoder. In this numerical scheme, the dimension of this latent space is called the encoding dimension.

Figure 4. Discretization of the Koopman formulation into a numerical scheme: The physical states at successive time points are denoted by , , , and . The encoded states at successive time points are denoted by , , , and . The function is the true time evolution of the physical state by the time step. The discretized Koopman operator, encoder and decoder are denoted by , and , respectively.
Koopman autoencoders are based on the discretized Koopman formulation where the operator, encoder and decoder are neural networks. The encoder and decoder normally consists of fully connected layers or convolution layers. And, the operator is a fully connected layer without a bias, and an activation function is not used between successive applications of it. Because the operator is applied repeatedly, it can be useful to use gradient clipping.
The loss for DeepONets and Fourier neural operators is the mean squared error between the output of the model and the true solution. For Koopman autoencoders, it is better to use additional loss terms during training [43]. The training loss used for Koopman autoencoders in this paper is
where is the encoder, is the encoder, is the Koopman operator, is the number of time steps and each is a vector in the physical space. Intuitively, the first term is the accuracy, the second term regulates the relation between the encoder and the decoder, and the last term forces the operator to be unitary. Furthermore, a mask is on the Koopman operator so that it is tridiagonal. The intuition of these choices is from the Koopman formulation of classical mechanics.
Differential Equations
This section introduces the differential equations that are used for the numerical experiments in this paper. The ordinary differential equations are the equation for the pendulum, the Lorenz system and a fluid attractor equation. The partial differential equations are Burger’s equation and the Korteweg-de-Vries equation.
Pendulum
The equation for the pendulum is the differential equation
This is a time-dependent second-order nonlinear ordinary differential equation whose dimension is 1. This equation models the motion of a pendulum in a constant gravitational field, where the variable is the angle of the pendulum from vertical.
In the numerical experiments in this paper, the models attempt to learn the solution to this differential equation as a function of the initial condition. To get data, initial positions are generated randomly. Then, this differential equation is numerically solved for these initial conditions using the Runge-Kutta Method [44].
Lorenz System
The Lorenz system is the differential equation
This is a time-dependent first-order nonlinear ordinary differential equation whose dimension is 3. Historically, this equation was used in weather modeling. Now, it provides a benchmark example of a chaotic system.
In the numerical experiments in this paper, the models attempt to learn the solution to this differential equation as a function of the initial condition. To get data, initial positions are generated randomly. Then, this differential equation is numerically solved for for these initial conditions using the Runge-Kutta Method [44].
Fluid Attractor Equation
The differential equation
is used to model fluid flow around a cylinder [45]. It is a time-dependent first-order nonlinear ordinary differential equation whose dimension is 3.
In the numerical experiments in this paper, the models attempt to learn the solution to this differential equation as a function of the initial condition. To get data, initial positions are generated randomly. Then, this differential equation is numerically solved for for these initial conditions using the Runge-Kutta Method [44].
Burger's Equation
Burger's equation is the differential equation
This is a time-dependent first-order nonlinear partial differential equation whose domain dimension and range dimension are both 1. It is used to model some fluids.
In the numerical experiments in this paper, the models attempt to learn the solution to this differential equation as a function of the initial condition. Furthermore, Dirichlet boundary conditions are used for this equation such that the value of the unknown function is 0 on the boundary. To get data, initial conditions are made by generating random symbolic expressions that satisfy the boundary conditions, and these expressions are then numerically sampled. Then, the partial differential equation is numerically solved for these initial conditions [46].
Korteweg-de-Vries Equation
The Korteweg-de-Vries equation, which is abbreviated as the KdV equation, is the differential equation
This is a time-dependent first-order nonlinear partial differential equation whose domain dimension and range dimension are both 1. It is used to model some fluids. In the numerical experiments in this paper, the models attempt to learn the solution to this differential equation as a function of the initial condition. Furthermore, periodic boundary conditions are used for this equation, that is, the value of the unknown function is the same on each endpoint of the spatial domain. To get data, initial conditions are made by generating random symbolic expressions that satisfy the boundary conditions, and these expressions are then numerically sampled. Then, the partial differential equation is numerically solved for these initial conditions [47].
Activation Function
This section studies the use of different activation functions in operator learning. The activation functions that are compared are the hyperbolic tangent function, the rectified linear unit, the gaussian error linear unit and the exponential linear unit. The rectified linear unit is the function
and it is denoted as . The gaussian error linear unit is the function
where is the cumulative distribution function of the standard normal distribution, that is, the function
The gaussian error linear unit is denoted as . The exponential linear unit is the function
and it is denoted as .

Figure 5. Activation functions: The upper left is the hyperbolic tangent. The upper right is the rectified linear unit. The lower left is the gaussian error linear unit. The lower right is the exponential linear unit.
Intuitively, in operator learning, the activation functions should be smooth because they are normally approximating smooth functions. Furthermore, the activation functions tanh and gelu are the most common in practice.
Numerical experiments were done for three architectures and each for two differential equations. DeepONets were used for the Lorenz system and Burger’s equation. Fourier neural operators were used for Burger’s equation and the KdV equation. And, Koopman autoencoders were used for the equation for the pendulum and the fluid attractor equation. The results are in Table 1, Table 2 and Table 3, respectively.
| Lorenz | Burger’s equation | ||
| error | activation | error | activation |
| 1.388e-2 | relu | 3.424e-3 | relu |
| 5.429e-3 | gelu | 2.576e-3 | gelu |
| 1.446e-2 | elu | 4.183e-3 | elu |
| 6.608e-3 | tanh | 3.363e-3 | tanh |
Table 1. Results for activation functions for DeepONets: The left table is for the Lorenz system, and the right table is for Burger’s equation.
| Burger’s | KdV | ||
| error | activation | error | activation |
| 6.428e-4 | relu | 4.116e-3 | relu |
| 5.013e-4 | gelu | 3.731e-3 | gelu |
| 5.355e-4 | elu | 3.909e-3 | elu |
| 1.777e-3 | tanh | 6.220e-3 | tanh |
Table 2. Results for activation functions for Fourier neural operators: The left table is for Burger’s equation, and the right table is for the KdV equation.
| pendulum | fluid attractor | ||
| error | activation | error | activation |
| 7.719e-4 | relu | 6.900e-4 | relu |
| 1.752e-4 | gelu | 1.112e-5 | gelu |
| 1.360e-4 | elu | 3.322e-5 | elu |
| 8.705e-4 | tanh | 4.602e-5 | tanh |
Table 3. Results for activation functions for Koopman autoencoders: The left table is for the equation for the pendulum, and the right table is for the fluid attractor equation.

Figure 6. Progress for different activation functions: Both plots show the trajectories for target and the prediction of a Koopman autoencoder for the fluid attractor equation at 1000 epochs. On the left, the activation function used is relu. On the right, the activation function used is gelu. The prediction on the right is visibly more accurate.
In every experiment, the activation function gelu preformed the best. Thus, is is recommend to use the activation function gelu in operator learning. In three out of the six experiments, the activation function relu preformed the worst, which is intuitively expected because it is not differentiable. In three out of the six experiments, the activation function tanh preformed the worst, which is surprising because it is often used in practice.
Dropout
This section studies the use of dropout in operator learning. In a layer in a neural network, dropout is a training technique where the entries of the matrix and displacement vector of a linear layer are randomly set to 0 [48, 49]. The dropout is the prescribed fraction of entries to set to 0. Dropout often prevents neural networks from overfitting during training.
Numerical experiments were done for three architectures and each for two differential equations. DeepONets were used for the Lorenz system and Burger’s equation. Fourier neural operators were used for Burger’s equation and the KdV equation. And, Koopman autoencoders were used for the equation for the pendulum and the fluid attractor equation. The results are in Table 4, Table 5 and Table 6, respectively. For the Koopman autoencoders, dropout was not experimented with for the operators, but only for the encoders and decoders.
| Lorenz | Burger’s | ||
| error | dropout | error | dropout |
| 6.999e-3 | 0.0 | 3.211e-3 | 0.0 |
| 3.363e-2 | 0.05 | 7.125e-3 | 0.05 |
| 5.396e-2 | 0.1 | 8.874e-3 | 0.1 |
| 7.036e-2 | 0.15 | 1.036e-2 | 0.15 |
Table 4. Results for dropout for DeepONets: The left table is for the Lorenz system, and the right table is for Burger’s equation.
| Burger’s | KdV | ||
| error | dropout | error | dropout |
| 7.585e-4 | 0.0 | 3.090e-3 | 0.0 |
| 1.138e-3 | 0.05 | 4.709e-3 | 0.05 |
| 1.126e-3 | 0.1 | 3.584e-3 | 0.1 |
| 1.958e-3 | 0.15 | 4.286e-3 | 0.15 |
Table 5. Results for dropout for Fourier neural operators: The left table is for the Burger’s equation, and the right table is for the KdV equation.
| pendulum | fluid attractor | ||
| error | dropout | error | dropout |
| 1.325e-3 | 0.0 | 4.696e-5 | 0.0 |
| 1.385e-2 | 0.05 | 2.238e-4 | 0.05 |
| 2.204e-2 | 0.1 | 3.336e-4 | 0.1 |
| 3.369e-2 | 0.15 | 4.982e-4 | 0.15 |
Table 6. Results for dropout for Koopman autoencoders: The left table is for the equation for the pendulum, and the right table is for the fluid attractor equation.
In every experiment, dropout decreased the accuracy of the model. Furthermore, Figure 7 displays the validation loss for a DeepONet for the Lorenz system with various dropout rates, and it demonstrates that increasing the dropout rate decreases the accuracy. Thus, it is recommended to not use dropout in operator learning.

Figure 7. Validation loss using dropout: A DeepONet is used for the Lorenz system with various dropout rates.
Stochastic Weight Averaging
This section studies the use of stochastic weight averaging in operator learning. Stochastic weight averaging is an addition to the optimizer that averages the parameters of a neural networks at different points during training that often improves convergence and prevents overfitting [50, 51, 52, 53]. For stochastic weight averaging, the learning rate strategy is applied normally for the majority of the epochs. Then, during the remaining part, a constant or periodic learning rate is used. The weights during this remaining part are averaged to get the final weights. Normally, gradient descent will tend find minima around the edge of a flat region, and stochastic gradient descent will tend to find minima in the center of such a flat region, which, intuitively, will generalize more to unseen data.

Figure 8. Stochastic weight averaging: The optimizer dictates the learning rate for the majority of the epochs. Towards the end of training, the learning rate is fixed. The final weights of the model are the average of the weights during this latter stage.
Numerical experiments were done for three architectures and each for two differential equations. DeepONets were used for the Lorenz system and Burger’s equation. Fourier neural operators were used for Burger’s equation and the KdV equation. And, Koopman autoencoders were used for the equation for the pendulum and the fluid attractor equation. The results are in Table 7, Table 8 and Table 9, respectively. For each model, the base learning rate was 0.001.
| Lorenz | KdV | ||||
| error | SWA LR | error | SWA LR | ||
| 1.084e-2 | none | 1.261e-2 | none | ||
| 1.045e-2 | 1e-4 | 4.700e-2 | 1e-4 | ||
| 8.323e-3 | 1e-3 | 1.072e-2 | 1e-3 | ||
| 1.673e-2 | 1e-2 | 1.910e-1 | 1e-2 | ||
| 1.699 | 1e-1 | 9.269e-1 | 1e-1 |
Table 7. Results stochastic weight averaging for DeepONets: The left table is for the Lorenz system, and the right table is for the KdV equation. The base learning rate of both models was 0.001.
| Burger’s | KdV | ||||
| error | SWA LR | error | SWA LR | ||
| 7.973e-4 | none | 7.34086e-3 | none | ||
| 4.011e-4 | 1e-4 | 4.36237e-3 | 1e-4 | ||
| 4.856e-4 | 1e-3 | 6.49731e-3 | 1e-3 | ||
| 2.540e-1 | 1e-2 | 2.07142e-1 | 1e-2 | ||
| 2.684e-1 | 1e-1 | 2.40617e-1 | 1e-1 |
Table 8. Results stochastic weight averaging for Fourier neural operators: The left table is for the equation for burger’s equation, and the right table is for the KdV equation. The base learning rate of both models was 0.001.
| pendulum | fluid attractor | ||||
| error | SWA LR | error | SWA LR | ||
| 4.370e-4 | none | 4.595e-5 | none | ||
| 2.122e-4 | 1e-4 | 9.073e-6 | 1e-4 | ||
| 4.919e-4 | 1e-3 | 1.162e-5 | 1e-3 | ||
| 2.536e-3 | 1e-2 | 7.259e-5 | 1e-2 | ||
| 3.665e-1 | 1e-1 | 2.757e-2 | 1e-1 |
Table 9. Results stochastic weight averaging for Koopman autoencoders: The left table is for the equation for the pendulum, and the right table is for the fluid attractor equation. The base learning rate of both models was 0.001.
In each experiment, stochastic weight averaging improved the accuracy of the model when the learning rate was the same or one tenth of the original learning rate. And, it is recommend to use stochastic weight averaging with such a learning rate. When the learning rate was significantly bigger than the original learning rate, the model preformed worse. This is common for stochastic weight averaging. Intuitively, if the learning rate is too big, the model weights will jump from local minimum to local minimum instead of looping around the flat region of a single local minimum, and the average of the weights will then be away from any local minimum instead of the center of a flat region.
Learning Rate Finder
A learning rate finder tries to find the optimal learning automatically instead of specifying it manually [54, 55, 56]. This is done by sweeping through a range of learning rates and then choosing the one corresponding to the greatest descent in the loss. This can be beneficial because it is one less hyperparameter to optimize.

Figure 9. Learning rate finder: The dot represents the optimal learning rate for the neural network. The optimal learning is not the learning rate that corresponds to the smallest loss, but the learning rate that corresponds to the greatest change in the loss, that is, the optimal learning is where the negative of the derivative is greatest.
The authors did experiments with a learning rate finder. Overall, using a learning rate finder was worse for operator learning. The found learning rate was inconsistent and normally was not close to the optimal learning rate. Furthermore, experiments were done where the model was first warmed up before applying the learning rate finder, that is, a few epochs were done beforehand, and using a learning rate finder was still not effective.
Conclusion
For operator learning, it recommended to use gelu as the activation function. It is recommended to not use dropout. It is recommended to use stochastic weight averaging with a learning rate that is at most the original learning rate. And, it is recommended to not use a learning rate finder, but instead find it when hyperparameter tuning.
Code Availability
The code for the numerical experiments is publicly available at https://gitlab.com/dustin_lee/neural-operators. All experiments were done in Python. Neural networks were implemented with PyTorch [57, 58, 59] and Lightning [60]. Hyperparameter configuration was done using Hydra [61].
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgment
GL would like to thank the support of National Science Foundation (DMS-2053746, DMS-2134209, ECCS-2328241, CBET-2347401 and OAC-2311848), and U.S. Department of Energy (DOE) Office of Science Advanced Scientific Computing Research program DE-SC0023161, and DOE–Fusion Energy Science, under grant number: DE-SC0024583.
References
- [1]Nikola B Kovachki, Samuel Lanthaler, and Andrew M Stuart. “Operator learning: Algorithms and analysis”. In: arXiv preprint arXiv:2402.15715 (2024).arxiv.org/abs/2402.15715
- [2]Nicolas Boullé and Alex Townsend. “A mathematical guide to operator learning”. In: arXiv preprint arXiv:2312.14688 (2023).DOI
- [3]Nick Winovich. “Neural Network Approximations to Solution Operators for Partial Differential Equations”. PhD thesis. Purdue University, 2021.DOI
- [4]Lu Lu et al. “Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators”. In: Nature Machine Intelligence 3.3 (Mar. 2021), pp. 218–229. ISSN: 2522-5839. DOI: 10.1038/s42256-021-00302-5. URL: http://dx.doi.org/10.1038/s42256-021-00302-5.
- [5]Samuel Lanthaler, Siddhartha Mishra, and George E Karniadakis. “Error estimates for deeponets: A deep learning framework in infinite dimensions”. In: Transactions of Mathematics and Its Applications 6.1 (2022), tnac001.arxiv.org/abs/2102.09618
- [6]Somdatta Goswami et al. “A physics-informed variational DeepONet for predicting crack path in quasi-brittle materials”. In: Computer Methods in Applied Mechanics and Engineering 391 (2022), p. 114587.DOI
- [7]Junyan He et al. “Novel DeepONet architecture to predict stresses in elastoplastic structures with variable complex geometries and loads”. In: Computer Methods in Applied Mechanics and Engineering 415 (2023), p. 116277.DOI
- [8]Sung Woong Cho, Jae Yong Lee, and Hyung Ju Hwang. “Learning time-dependent PDE via graph neural networks and deep operator network for robust accuracy on irregular grids”. In: arXiv preprint arXiv:2402.08187 (2024).arxiv.org/abs/2402.08187
- [9]Lu Lu et al. “A comprehensive and fair comparison of two neural operators (with practical extensions) based on fair data”. In: Computer Methods in Applied Mechanics and Engineering 393 (2022), p. 114778.DOI
- [10]Zongyi Li et al. “Fourier neural operator for parametric partial differential equations”. In: arXiv preprint arXiv:2010.08895 (2020).arxiv.org/abs/2010.08895
- [11]Shaoxiang Qin et al. “Toward a Better Understanding of Fourier Neural Operators: Analysis and Improvement from a Spectral Perspective”. In: arXiv preprint arXiv:2404.07200 (2024).arxiv.org/abs/2404.07200
- [12]Zongyi Li et al. “Fourier neural operator with learned deformations for pdes on general geometries”. In: Journal of Machine Learning Research 24.388 (2023), pp. 1–26.DOI
- [13]Johannes Brandstetter, Max Welling, and Daniel E Worrall. “Lie point symmetry data augmentation for neural PDE solvers”. In: International Conference on Machine Learning. PMLR. 2022, pp. 2241–2256.arxiv.org/abs/2202.07643
- [14]Nikola Kovachki, Samuel Lanthaler, and Siddhartha Mishra. “On universal approximation and error bounds for Fourier neural operators”. In: Journal of Machine Learning Research 22.290 (2021), pp. 1–76.arxiv.org/abs/2107.07562
- [15]Alasdair Tran et al. “Factorized fourier neural operators”. In: arXiv preprint arXiv:2111.13802 (2021).arxiv.org/abs/2111.13802
- [16]Matthew Tancik et al. “Fourier features let networks learn high frequency functions in low dimensional domains”. In: Advances in neural information processing systems 33 (2020), pp. 7537–7547.arxiv.org/abs/2006.10739
- [17]Elias M Stein and Rami Shakarchi. Fourier analysis: an introduction. Vol. 1. Princeton University Press, 2011.
- [18]Kamisetty Ramamohan Rao, Do Nyeon Kim, and Jae Jeong Hwang. Fast Fourier transform-algorithms and applications. Springer Science & Business Media, 2011.DOI
- [19]Bethany Lusch, J Nathan Kutz, and Steven L Brunton. “Deep learning for universal linear embeddings of nonlinear dynamics”. In: Nature communications 9.1 (2018), p. 4950.DOI
- [20]Nishaal Parmar, Hazem H Refai, and Thordur Runolfsson. “A survey on the methods and results of data-driven koopman analysis in the visualization of dynamical systems”. In: IEEE Transactions on Big Data 8.3 (2020), pp. 723–738.DOI
- [21]Giorgos Mamakoukas, Ian Abraham, and Todd D Murphy. “Learning data-driven stable Koopman operators”. In: Free radical biology & medicine. (2020).
- [22]Bowen Huang, Xu Ma, and Umesh Vaidya. “Data-driven nonlinear stabilization using koopman operator”. In: The Koopman Operator in Systems and Control: Concepts, Methodologies, and Applications (2020), pp. 313–334.arxiv.org/abs/1901.07678
- [23]Stefan Klus et al. “Data-driven approximation of the Koopman generator: Model reduction, system identification, and control”. In: Physica D: Nonlinear Phenomena 406 (2020), p. 132416.DOI
- [24]Steven L Brunton et al. “Koopman invariant subspaces and finite linear representations of nonlinear dynamical systems for control”. In: PloS one 11.2 (2016), e0150171.DOI
- [25]Marko Budišić, Ryan Mohr, and Igor Mezić. “Applied koopmanism”. In: Chaos: An Interdisciplinary Journal of Nonlinear Science 22.4 (2012).DOI
- [26]J Nathan Kutz, Xing Fu, and Steven L Brunton. “Multiresolution dynamic mode decomposition”. In: SIAM Journal on Applied Dynamical Systems 15.2 (2016), pp. 713–735.DOI
- [27]Naoya Takeishi, Yoshinobu Kawahara, and Takehisa Yairi. “Learning Koopman invariant subspaces for dynamic mode decomposition”. In: Advances in neural information processing systems 30 (2017).arxiv.org/abs/1710.04340
- [28]Shervin Bagheri. “Koopman-mode decomposition of the cylinder wake”. In: Journal of Fluid Mechanics 726 (2013), pp. 596–623.DOI
- [29]M Budisić, R Mohr, and I Mezić. The Koopman Operator in Systems and Control: Concepts, Methodologies, and Applications. 2020.
- [30]Yunzhu Li et al. “Learning compositional koopman operators for model-based control”. In: arXiv preprint arXiv:1910.08264 (2019).arxiv.org/abs/1910.08264
- [31]Eurika Kaiser, J Nathan Kutz, and Steven L Brunton. “Data-driven approximations of dynamical systems operators for control”. In: The Koopman Operator in Systems and Control: Concepts, Methodologies, and Applications (2020), pp. 197–234.arxiv.org/abs/1902.10239
- [32]Yiqiang Han, Wenjian Hao, and Umesh Vaidya. “Deep learning of Koopman representation for control”. In: 2020 59th IEEE Conference on Decision and Control (CDC). IEEE. 2020, pp. 1890–1895.arxiv.org/abs/2010.07546
- [33]Daniel Bruder et al. “Koopman-based control of a soft continuum manipulator under variable loading conditions”. In: IEEE robotics and automation letters 6.4 (2021), pp. 6852–6859.arxiv.org/abs/2002.01407
- [34]Daniel Bruder et al. “Modeling and control of soft robots using the koopman operator and model predictive control”. In: arXiv preprint arXiv:1902.02827 (2019).arxiv.org/abs/1902.02827
- [35]Hassan Arbabi, Milan Korda, and Igor Mezić. “A data-driven koopman model predictive control framework for nonlinear partial differential equations”. In: 2018 IEEE Conference on Decision and Control (CDC). IEEE. 2018, pp. 6409–6414.DOI
- [36]Ian Abraham, Gerardo De La Torre, and Todd D Murphey. “Model-based control using Koopman operators”. In: arXiv preprint arXiv:1709.01568 (2017).DOI
- [37]Bernard O Koopman. “Hamiltonian systems and transformation in Hilbert space”. In: Proceedings of the National Academy of Sciences 17.5 (1931), pp. 315–318.pubmed.ncbi.nlm.nih.gov/16577368
- [38]Steven L Brunton et al. “Modern Koopman theory for dynamical systems”. In: arXiv preprint arXiv:2102.12086 (2021).arxiv.org/abs/2102.12086
- [39]Adam L Bruce, Vera M Zeidan, and Dennis S Bernstein. “What is the Koopman operator? a simplified treatment for discrete-time systems”. In: 2019 American Control Conference (ACC). IEEE. 2019, pp. 1912–1917.DOI
- [40]Michael A Morrison. Understanding quantum physics: A user’s manual. 1990.
- [41]Hans C. Ohanian. Principles of Quantum Mechanics. 1989.
- [42]David J. Griffiths. Introduction to Quantum Mechanics. 2018.
- [43]Dustin Enyeart and Guang Lin. Loss Terms and Operator Forms of Koopman Autoencoders. 2024. arXiv: 2412.04578 [cs.LG]. URL: https://arxiv.org/abs/2412.04578.
- [44]Randall J LeVeque. Finite difference methods for ordinary and partial differential equations: steady-state and time-dependent problems. SIAM, 2007.
- [45]Bernd R Noack et al. “A hierarchy of low-dimensional models for the transient and post-transient cylinder wake”. In: Journal of Fluid Mechanics 497 (2003), pp. 335–363.DOI
- [46]Randall J LeVeque and Randall J LeVeque. Numerical methods for conservation laws. Vol. 214. Springer, 1992.
- [47]Norman J Zabusky and Martin D Kruskal. “Interaction of “solitons” in a collisionless plasma and the recurrence of initial states”. In: Physical review letters 15.6 (1965), p. 240.DOI
- [48]GE Hinton. “Improving neural networks by preventing co-adaptation of feature detectors”. In: arXiv preprint arXiv:1207.0580 (2012).arxiv.org/abs/1207.0580
- [49]Nitish Srivastava et al. “Dropout: a simple way to prevent neural networks from overfitting”. In: The journal of machine learning research 15.1 (2014), pp. 1929–1958.
- [50]Pavel Izmailov et al. “Averaging weights leads to wider optima and better generalization”. In: arXiv preprint arXiv:1803.05407 (2018).arxiv.org/abs/1803.05407
- [51]Ben Athiwaratkun et al. “There are many consistent explanations of unlabeled data: Why you should average”. In: arXiv preprint arXiv:1806.05594 (2018).
- [52]Evgenii Nikishin et al. “Improving stability in deep reinforcement learning with weight averaging”. In: Uncertainty in artificial intelligence workshop on uncertainty in Deep learning. 2018.
- [53]Pavel Izmailov and Andew Gordon Wilson. Stochastic Weight Averaging in PyTorch. Accessed: October 2024. 2019. URL: https://pytorch.org/blog/stochastic-weight-averaging-in-pytorch/.
- [54]Leslie N Smith. “Cyclical learning rates for training neural networks”. In: 2017 IEEE winter conference on applications of computer vision (WACV). IEEE. 2017, pp. 464–472.arxiv.org/abs/1506.01186
- [55]Liyuan Liu et al. “On the variance of the adaptive learning rate and beyond”. In: arXiv preprint arXiv:1908.03265 (2019).arxiv.org/abs/1908.03265
- [56]Akhilesh Gotmare et al. “A closer look at deep learning heuristics: Learning rate restarts, warmup and distillation”. In: arXiv preprint arXiv:1810.13243 (2018).arxiv.org/abs/1810.13243
- [57]Adam Paszke et al. “Automatic differentiation in PyTorch”. In: NIPS-W. 2017.
- [58]Ian Pointer. Programming pytorch for deep learning: Creating and deploying deep learning applications. O’Reilly Media, 2019.
- [59]Phillip Lippe. UvA Deep Learning Tutorials. https://uvadlc-notebooks.readthedocs.io/en/latest/. 2024.DOI
- [60]Torch Lightning. URL: https://github.com/Lightning-AI/pytorch-lightning.
- [61]Omry Yadan. Hydra - A framework for elegantly configuring complex applications. Github. 2019. URL: https://github.com/facebookresearch/hydra.

