Optimal higher order convergence rates for the MUON optimizer
1 version
AJ
Arnulf Jentzen
cs.NAmath.NAmath.PRcs.LGmath.OCstat.MLmath.APcs.AImath.STstat.TH
On Valency
published · living versionsPreprints & journals
151 papers in the corpus · 2009–2026Convergence rates for the RMSprop optimizer with full control of the hyperparameters
0 citations
On MUON optimization: From non-convergence to an error analysis with Polar Express and the Newton-Schulz polynomial from implementations
0 citations
Strong error analysis for the stochastic momentum optimizer
0 citations
Unified convergence analysis for gradient descent optimization methods in the training of deep neural networks
0 citations
Central limit theorem for the averaged Adam optimizer
0 citations
Uniform a priori bounds and error analysis for the Adam stochastic gradient descent optimization method
0 citations
Controlled fields, rough stochastic calculus, and Itô-Wentzell-Alekseev-Gröbner identities
0 citations
Physics-informed diffusion models in spectral space
0 citations
Adam symmetry theorem: characterization of the convergence of the stochastic Adam optimizer
0 citations
ODE approximation for the Adam algorithm: General and overparametrized setting
0 citations
Asymptotic stability properties and a priori bounds for Adam and other gradient descent optimization methods
0 citations
Error analysis for the deep Kolmogorov method
0 citations
Deep neural networks can provably solve Bellman equations for Markov decision processes without the curse of dimensionality
0 citations
PADAM: Parallel averaged Adam reduces the error for stochastic optimization in scientific machine learning
0 citations
SAD Neural Networks: Divergent Gradient Flows and Asymptotic Optimality via o-minimal Structures
0 citations
A brief review of the Deep BSDE method for solving high-dimensional partial differential equations
0 citations
Sharp higher order convergence rates for the Adam optimizer
0 citations
Non-convergence to the optimal risk for Adam and stochastic gradient descent optimization in the training of deep neural networks
0 citations
On the logical skills of large language models: evaluations using arbitrarily complex first-order logic problems
0 citations
Mathematical analysis of the gradients in deep learning
0 citations
Averaged Adam accelerates stochastic optimization in the training of deep neural network approximations for partial differential equation and optimal control problems
0 citations
Nonlinear Monte Carlo Methods with Polynomial Runtime for Bellman Equations of Discrete Time High-Dimensional Stochastic Optimal Control Problems.
2 citations
Counterexamples to regularities for the derivative processes associated to stochastic evolution equations.
2 citations
An overview of diffusion models for generative artificial intelligence
0 citations
Non-convergence to global minimizers in data driven supervised deep learning: Adam and stochastic gradient descent optimization provably fail to converge to global minimizers in the training of deep neural networks with ReLU activation
0 citations
An Overview on Machine Learning Methods for Partial Differential Equations: from Physics Informed Neural Networks to Deep Operator Learning
1 citation
Convergence rates for the Adam optimizer
2 citations
Non-convergence of Adam and other adaptive stochastic gradient descent optimization methods for non-vanishing learning rates
1 citation
Learning rate adaptive stochastic gradient descent optimization methods: numerical simulations for deep learning methods for partial differential equations and convergence analyses
0 citations
Deep neural networks with ReLU, leaky ReLU, and softplus activation provably overcome the curse of dimensionality for space-time solutions of semilinear partial differential equations
1 citation
Non-convergence to global minimizers for Adam and stochastic gradient descent optimization and constructions of local minimizers in the training of artificial neural networks
8 citations
Overcoming the curse of dimensionality in the numerical approximation of high-dimensional semilinear elliptic partial differential equations.
34 citations
Gradient Descent Provably Escapes Saddle Points in the Training of Shallow ReLU Networks.
4 citations
Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory
7 citations
Deep neural networks with ReLU, leaky ReLU, and softplus activation provably overcome the curse of dimensionality for Kolmogorov partial differential equations with Lipschitz nonlinearities in the -sense
1 citation
On the existence of minimizers in shallow residual ReLU neural network optimization landscapes
8 citations
Algorithmically Designed Artificial Neural Networks (ADANNs): Higher order deep operator learning for parametric partial differential equations
0 citations
The necessity of depth for artificial neural networks to approximate certain classes of smooth and bounded functions without the curse of dimensionality
0 citations
Convergence to good non-optimal critical points in the training of neural networks: Gradient descent optimization with one random initialization overcomes all bad non-global local minima with high probability
2 citations
Blow up phenomena for gradient descent optimization methods in the training of artificial neural networks
2 citations
An efficient Monte Carlo scheme for Zakai equations
1 citation
Normalized gradient flow optimization in the training of ReLU artificial neural networks
0 citations
Efficient Approximation of High-Dimensional Functions With Neural Networks.
38 citations
On bounds for norms of reparameterized ReLU artificial neural network parameters: sums of fractional powers of the Lipschitz norm control the network parameter vector
0 citations
Deep learning approximations for non-local nonlinear PDEs with Neumann boundary conditions
18 citations
On the existence of infinitely many realization functions of non-global local minima for artificial neural networks with ReLU activation
1 citation
Learning the random variables in Monte Carlo simulations with stochastic gradient descent: Machine learning for parametric PDEs and financial derivative pricing
11 citations
Deep neural network approximation theory for high-dimensional functions
4 citations
On the existence of global minima and convergence analyses for gradient descent methods in the training of deep neural networks
12 citations
Convergence proof for stochastic gradient descent in the training of deep neural networks with ReLU activation for constant target functions
2 citations
Profile built from the corpus for this byline.
Author records are still filling in while the Hub is in beta. If this is your page, you'll be able to claim it soon. Spot a mistake? Tell us.