betaFeedback

WorksPhysics and Astronomy

From Ferminet to PINN. Connections between neural network-based algorithms for high-dimensional Schrödinger Hamiltonian

Abstract

In this note, we establish some connections between standard (data-driven) neural network-based solvers for PDE and eigenvalue problems developed on one side in the applied mathematics and engineering communities (e.g. Deep-Ritz and Physics Informed Neural Networks (PINN)), and on the other side in quantum chemistry (e.g. Variational Monte Carlo algorithms, Ferminet or Paulinet following the pioneer work of Carleo et. al. In particular, we re-formulate a PINN algorithm as a fitting problem with data corresponding to the solution to a standard Diffusion Monte Carlo algorithm initialized thanks to neural network-based Variational Monte Carlo. Connections at the level of the optimization algorithms are also established.

Introduction

In less than 10 years (since 2017), Physics Informed Neural Networks (PINN) has emerged as a major player in the field of computational methods for solving direct and inverse problems involving (high-dimensional) parameterized partial differential equations (PDEs) [23, 26, 19, 1]. The principle is to seek solutions to PDEs in the form of data-driven neural networks, optimized by minimizing PDE residuals. These techniques, along with others such as polynomial regression, Gaussian process, reduced order modeling, or Deep Ritz [25], have become highly popular in several engineering fields (e.g. mechanical engineering, fluid dynamics) as they allow to a certain extent, the tackling of problems that are inaccessible with traditional PDE solvers.

There is a tremendous effort within the mathematics and computer science communities to address several open questions related to convergence, approximability, efficient high-dimensional optimizers, etc., in Scientific Machine Learning, including PINN-like techniques. Independently of the development of PINN-like methods by the mathematics, engineering, computer science, and statistics communities, significant practical progress has been achieved in quantum chemistry and quantum physics in the development of neural network-based Variational Monte Carlo (VMC) and Diffusion Monte Carlo (DMC) methods. This has enabled the calculation of the electronic structure of molecules larger than those usually considered with traditional computational methods. This approach corresponds mathematically to the approximation of the lowest eigenvalues (energies) of high-dimensional Schrödinger operators. Following the construction of Neural Quantum States (NQS) [7, 8], several corresponding libraries have been developed, such as Ferminet by DeepMind/Google [9, 22] or Paulinet [15].

To our knowledge, the two above formalisms have never been unified or even rigorously connected, despite their strong similarities. In this manuscript, we establish some connections potentially relevant for the development of a mathematical formalism unifying these modern techniques. More specifically, we will reformulate the so-called Neural-Variation Diffusion Monte Carlo (N-VDMC), corresponding roughly to a standard DMC method where the trial wavefunction is computed through a VMC method (corresponding to a Deep-Ritz algorithm [25]) as proposed for instance in Ferminet. Let us also mention [14], where a neural network based variational solving for many-body Schroedinger Hamiltonian is proposed. We will also show that at the level of the optimization algorithm (gradient descent-like algorithm) a N-VDMC is indeed “similar” to a PINN method applied to a diffusion equation (PINN-DE) derived from the time-dependent Schrödinger equation.

In the first part of the manuscript, we will propose a short presentation of PINN, VMC, and DMC and N-VDMC algorithms. We then show that we can reformulate the PINN algorithm as a fitting problem with data, corresponding to the solution to a N-VDMC algorithm. We will then show that at the level of the optimization algorithm, a modified version of N-VDMC leads to the same gradient descent algorithm as PINN.

In this manuscript, we do not discuss the (very complex) structure of the neural networks usually used in quantum chemistry which follows standard strategies based on one-electron orbitals and Slater’s determinants [24], and refer to [22, 7, 8] for details. However, the properties and connections presented in this manuscript are largely independent of the structure of the neural networks; the latter will be treated here as a parameterized functions.

Mathematical set-up

We denote by HH the exact NN-particle Schrödinger Hamiltonian of domain D(H)=H2(R3N;C)D(H) = H^2(\mathbb{R}^{3N};\mathbb{C}) and by R=(r1,…,rN)∈R3N\mathbf{R} = (r_1,\ldots,r_N) \in\mathbb{R}^{3N} with rir_i the 3-dimensional coordinates of the iith particle, see [10, 24]. We denote by ϕG(R)\phi_G(\mathbf{R}) the groundstate of HH

HϕG(R)=EGϕG(R),H\phi_G(\mathbf{R}) = E_G\phi_G(\mathbf{R}),

where HH reads

H=−12∇R2+Vp(R)−ET,H = -\frac{1}{2}\nabla_{\mathbf{R}}^2 + V_p(\mathbf{R}) - E_T,

and where ETE_T is a so-called off-set energy and VpV_p the potential energy of the considered molecule/atom. In this work, we consider a relatively general framework, as we will “only” establish high-level connections between PINN-like algorithms and neural network-based VMC and DMC algorithms.

Basics on neural networks

Let us here recall the basics of fully connected neural networks. The latter are parameterized functions constructed as the composition of i) discrete convoluted linear functions, and ii) nonlinear functions. In a dd-dimension the space, we introduce l∈Nl \in\mathbb{N}, ni∈Nn_i \in\mathbb{N} for i=0,1,…,li = 0,1,\ldots,l, with n0=dn_0 = d. We denote by θ∈Θ:=∏i=1lRni×(ni−1+1)\theta\in\Theta:= \prod_{i=1}^{l} \mathbb{R}^{n_i \times(n_{i-1}+1)} the neural network parameters such that: θ={θi}i\theta= \{\theta_i\}_i with θi=(wi,bi)∈Rni×ni−1×Rni\theta_i = (w^i,b^i) \in\mathbb{R}^{n_i \times n_{i-1}} \times\mathbb{R}^{n_i} and i=1,…,li = 1,\ldots,l. A network f^\widehat{f} in Rd\mathbb{R}^d is a function from ∏i=1lRni×(ni−1+1)×Rd\prod_{i=1}^{l} \mathbb{R}^{n_i \times(n_{i-1}+1)} \times\mathbb{R}^d to Rnl\mathbb{R}^{n_l} such that

f^(θ;τ)=wlσ(wl−1(σ(w2σ(w1τ+b1)+b2)⋯ )+bl−1)+bl,(1)\widehat{f}(\theta;\tau) = w^l \sigma\left(w^{l-1}\left(\sigma\left(w^2 \sigma\left(w^1 \tau+ b^1\right) + b^2\right)\cdots\right) + b^{l-1}\right) + b^l, \tag*{(1)}

where σ\sigma is a so-called activation function. We refer to [16, 2] for details on neural networks. The use of neural network-based algorithms for PDE and eigenvalue problems largely rely on approximability theorems such as the celebrated Cybenko’s Universal Approximation theorem [11] and more recently [12, 13]. Let us repeat that here, we do not discuss the very elaborated structure of neural networks used for approximating wavefunctions in high-dimensional electronic structure problems, but refer to [22, 7, 8].

Deep-Ritz / Ferminet

Following the celebrated works of Carleo et al. [7, 8] on neural quantum states, Ferminet [9, 22] has become a standard electronic structure calculation library, which is based on the minimization of the Rayleigh quotient

EG=min⁡ϕ⟨Hϕ,ϕ⟩⟨ϕ,ϕ⟩,E_G = \min_{\phi} \frac{\langle H\phi,\phi\rangle}{\langle\phi,\phi\rangle},

thanks to a Variational Monte-Carlo using a neural network ϕ^(ϑ;R)\widehat{\phi}(\vartheta;R) as ansatz; and where we have used the notation ⟨Hϕ,ϕ⟩\langle H\phi,\phi\rangle where ⟨⋅,⋅⟩\langle\cdot,\cdot\rangle denotes the L2L^2-inner product. The groundstate ϕG\phi_G is approximated by an optimized neural network ψT:R∈R3N↦ϕ^(ϑ∗;R)\psi_T : R \in\mathbb{R}^{3N} \mapsto\widehat{\phi}(\vartheta^*;R), such that

ϑ∗=argmin⁡ϑ⟨Hϕ^(ϑ;⋅),ϕ^(ϑ;⋅)⟩⟨ϕ^(ϑ;⋅),ϕ^(ϑ;⋅)⟩.\vartheta^* = \operatorname*{argmin}_{\vartheta} \frac{\langle H\widehat{\phi}(\vartheta;\cdot),\widehat{\phi}(\vartheta;\cdot)\rangle}{\langle\widehat{\phi}(\vartheta;\cdot),\widehat{\phi}(\vartheta;\cdot)\rangle}.

Practically, the Rayleigh quotient is minimized thanks to automatic differentiation and using Monte-Carlo integration with learning nodes {Rα}α\{R^\alpha\}_\alpha in R3N\mathbb{R}^{3N}

ϑ~∗=argmin⁡ϑ∑αHϕ^(ϑ;Rα)ϕ^(ϑ;Rα)∑αϕ^(ϑ;Rα)ϕ^(ϑ;Rα),\widetilde{\vartheta}^* = \operatorname*{argmin}_{\vartheta} \frac{\sum_\alpha H\widehat{\phi}(\vartheta;R^\alpha)\widehat{\phi}(\vartheta;R^\alpha)}{\sum_\alpha\widehat{\phi}(\vartheta;R^\alpha)\widehat{\phi}(\vartheta;R^\alpha)},

hence defining an approximate groundstate. This approach can be seen as a Deep-Ritz algorithm [25]; corresponding to a class of variational neural network-based solvers for PDE and eigenvalue problems. In the following, the use of neural network as ansatz for minimizing the Rayleigh quotient will be referred as Neural-Variational Monte Carlo (N-VMC).

Diffusion Monte Carlo and Neural Diffusion Monte Carlo

Let us shortly recall the principle of the DMC method [20, 18, 6, 17, 27]. The latter relies on the computation over, say Q:=R3N×R+Q := \mathbb{R}^{3N} \times\mathbb{R}_{+}, of the following function

f(R,t)=ψT(R)ϕ(R,t),f(R,t) = \psi_T(R)\phi(R,t),

where ψT\psi_T is a given trial function approximating ϕG\phi_G for a NN-particle Schrödinger operator. The function ff satisfies the following equation on QQ, and we will refer to the Diffusion Equation (DE)

∂tf−12∇2f+∇R(V0(R)f)−S0(R)f=0,f(R,0)=ψT2(R),(2)\partial_t f - \frac{1}{2}\nabla^2 f + \nabla_R(V_0(R)f) - S_0(R)f = 0,\quad f(R,0) = \psi_T^2(R), \tag*{(2)}

where (for non-null ψT\psi_T)

V0(R)=∇R∣log⁡ψT(R)∣(drift velocity),S0(R)=ET−E0(R)(branching term),E0(R)=ψT(R)−1HψT(R)(local energy).(3)\begin{aligned} V_0(R) &= \nabla_R \lvert\log\psi_T(R)\rvert\quad\text{(drift velocity)},\\ S_0(R) &= E_T - E_0(R) \quad\text{(branching term)},\\ E_0(R) &= \psi_T(R)^{-1}H\psi_T(R) \quad\text{(local energy)}. \tag*{(3)} \end{aligned}

In the following, we simply rewrite (2) as

∂tf+K(f)=0,f(⋅,0)=f0,(4)\partial_t f + K(f) = 0,\quad f(\cdot,0) = f_0, \tag*{(4)}

where f0=ψT2f_0 = \psi_T^2 and where the operator KK is defined as follows.

K(f)=−12∇2f+∇R(V0(R)f)−S0(R)f.(5)K(f) = -\frac{1}{2}\nabla^2 f + \nabla_R(V_0(R)f) - S_0(R)f. \tag*{(5)}

The groundstate and groundstate energy are obtained from

lim⁡t→+∞f(R,t)=ψT(R)ϕG(R),\lim_{t\to+\infty} f(R,t) = \psi_T(R)\phi_G(R),

and the so-called mixed estimator is

Emix=lim⁡t→+∞⟨Hϕ,ψT⟩⟨ϕ,ψT⟩=lim⁡t→+∞∫E0(R)f(R,t) dR∫f(R,t) dR,E_{\mathrm{mix}} = \lim_{t\to+\infty}\frac{\langle H\phi,\psi_T\rangle}{\langle\phi,\psi_T\rangle} = \lim_{t\to+\infty}\frac{\int E_0(R)f(R,t)\,dR}{\int f(R,t)\,dR},

leading to EG=⟨HϕG,ψT⟩/⟨ϕG,ψT⟩E_G = \langle H\phi_G,\psi_T\rangle/\langle\phi_G,\psi_T\rangle, as long as ψT\psi_T has an overlap with ϕG\phi_G. From a practical point of view the above approach is discretized using a sequence of time-dependent random walks {Rα(t)}α\{R^\alpha(t)\}_\alpha from tt to R3N\mathbb{R}^{3N}, and allowing for the construction of a function f~\tilde f approximating ff in the form (see [27])

f~:(R,t)∈Q↦∑αf~(Rα(t),t)δ(R−Rα(t))∑αf~(Rα(t),t)∈R.\tilde f : (R,t) \in Q \mapsto\frac{\sum_\alpha\tilde f(R^\alpha(t),t)\delta(R-R^\alpha(t))}{\sum_\alpha\tilde f(R^\alpha(t),t)} \in\mathbb{R}.

A sequence of discrete times t0=0<t1<⋯<ti<ti+1<⋯t_0 = 0 < t_1 < \cdots< t_i < t_{i+1} < \cdots and dt=ti+1−tidt = t_{i+1} - t_i for any i∈Ni \in\mathbb{N} is introduced. For {R0α,t0}α\{R_0^\alpha,t_0\}_\alpha given, we define f~iα=f~(Riα,ti)\tilde{f}_i^\alpha= \tilde{f}(R_i^\alpha,t_i), such that

Ri+1α=Riα+dtV0(Riα)+dtηi,R_{i+1}^\alpha= R_i^\alpha+ dt V_0(R_i^\alpha) + \sqrt{dt}\eta_i,
f~i+1α=f~iαexp⁡(dt(S0(Ri+1α)+S0(Riα))/2).\tilde{f}_{i+1}^\alpha= \tilde{f}_i^\alpha\exp\left(dt\left(S_0(R_{i+1}^\alpha)+S_0(R_i^\alpha)\right)/2\right).

with ηi=η(ti)\eta_i = \eta(t_i) where η\eta is a 3N3N-dimensional Gaussian random vector with null mean E[η]=0\mathbb{E}[\eta] = 0 (in space and time) and variance equal to dtdt. Moreover E[η2]=3N\mathbb{E}[\eta^2] = 3N, where η2\eta^2 denotes η⋅η\eta\cdot\eta. Acceptance/rejection process is based on Metropolis’ probability [17] and we set for any ii

f~i(R)=f~(R,ti)=∑αf~iαδ(R−Riα)∑αf~iα.\tilde{f}_i(R) = \tilde{f}(R,t_i) = \frac{\sum_\alpha\tilde{f}_i^\alpha\delta(R-R_i^\alpha)}{\sum_\alpha\tilde{f}_i^\alpha}.

Finally, for ii large enough and denoting by ⟨⋅⟩α\langle\cdot\rangle_\alpha the average over the walkers, we get the mixed estimator at any time tit_i, Emixi=⟨E~i−1(Riα)f~iα⟩α/⟨f~iα⟩αE_{\mathrm{mix}}^i = \langle\tilde{E}_{i-1}(R_i^\alpha)\tilde{f}_i^\alpha\rangle_\alpha/\langle\tilde{f}_i^\alpha\rangle_\alpha.

Initialization. In the N-VDMC algorithm, the trial function ψT\psi_T is computed from a neural-network VMC/Deep-Ritz method referred above as a N-VMC method [7, 8, 9, 15, 25]. In this goal, we introduce a neural-network ϕ^T(ϑ;R)\hat{\phi}_T(\vartheta;R) with parameters ϑ\vartheta. Deep-Ritz/Ferminet consists in minimizing

ϑ∗=argmin⁡ϑ⟨Hϕ^T(ϑ;R),ϕ^T(ϑ;R)⟩⟨ϕ^T(ϑ;R),ϕ^T(ϑ;R)⟩.\vartheta^* = \underset{\vartheta}{\operatorname{argmin}}\frac{\langle H\hat{\phi}_T(\vartheta;R),\hat{\phi}_T(\vartheta;R)\rangle}{\langle\hat{\phi}_T(\vartheta;R),\hat{\phi}_T(\vartheta;R)\rangle}.

We denote f0(R)=ϕ^T2(ϑ∗;R)f_0(R)=\hat{\phi}_T^2(\vartheta^*;R) which is taken as the initial data to (2) for both N-VDMC and PINN. In practice, for a fixed set of initial positions {R0α}α\{R_0^\alpha\}_\alpha, we define

f~0(R)=∑αf0(R0α)δ(R−R0α)∑αf0(R0α).\tilde{f}_0(R)=\frac{\sum_\alpha f_0(R_0^\alpha)\delta(R-R_0^\alpha)}{\sum_\alpha f_0(R_0^\alpha)}.

For the sake of notation simplicity, hereafter we will denote fif_i the DMC-solution (in place of f~i\tilde{f}_i) at time tit_i, while f^i\hat{f}_i will refer to the PINN-solution.

Basics on PINN

Physics Informed Neural Networks (PINN) [23, 26, 19, 1] has become a standard methodology in engineering and applied mathematics for the data-driven computation of partial differential equations. A PINN-like solution to a differential system (S)(S) is a (smooth and usually data-driven) neural network optimized thanks to the minimization of the norm of the PDE residual of (S)(S) evaluated at some (randomly chosen) learning nodes. This approach has recently gained a huge interest in scientific machine learning thanks to their simplicity, their nice mathematical and computational properties coming in particular from automatic differentiation and efficient stochastic gradient descent methods [5].

We now discuss the principle of PINN algorithms for solving (2). In order to avoid a null solution (which can occur for large times), it is preferable to decompose the time-domain as follows: R+=⋃i=1∞[ti−1,ti)\mathbb{R}_{+}=\bigcup_{i=1}^{\infty}[t_{i-1},t_i), with t0=0<t1<⋯<ti<ti+1<⋯t_0=0<t_1<\cdots<t_i<t_{i+1}<\cdots and we denote dt=ti+1−tidt=t_{i+1}-t_i and Qi=R3N×[ti−1,ti)Q_i=\mathbb{R}^{3N}\times[t_{i-1},t_i). In general dtdt does not represent a time-step (and the time-derivative is not approximated) and can often be taken much larger than usual time steps in standard computational solvers such as DMC. It is also possible to solve (2) by considering it as a boundary value problem, hence without approximating the time derivative. In this paper, as the objective is to propose links to N-VDMC algorithms, within PINN we propose a first order discretization of ∂t\partial_t, so that dtdt actually represents the same time-step as the one used in DMC.

  • On Q0Q_0 and for f0=ψT2f_0=\psi_T^2 given, we search for a neural network f^:(θ;R)∈Θ×R3N→R\widehat{f}:(\theta;R)\in\Theta\times\mathbb{R}^{3N}\to\mathbb{R} which i) optimized by minimizing the commonly called loss (objective) function

L1(PINN)(θ)=∥f^(θ;⋅)−f0−dt2∇R2f0+dt∇R(V0f0)−dtS0f0∥L2(R3N)2,\mathcal{L}_1^{(\mathrm{PINN})}(\theta)=\left\|\widehat{f}(\theta;\cdot)-f_0-\frac{dt}{2}\nabla_R^2f_0+dt\nabla_R(V_0f_0)-dtS_0f_0\right\|_{L^2(\mathbb{R}^{3N})}^2,

where S0S_0, V0V_0 are defined in (3). The approximate solution to (2) at time t1t_1 is denoted by f^1(R)=f^(θ1∗;R)\widehat{f}_1(R)=\widehat{f}(\theta_1^*;R), where

θ1∗=argmin⁡θL1(θ).\theta_1^*=\operatorname*{argmin}_{\theta}\mathcal{L}_1(\theta).
  • On QiQ_i for i>1i>1, assuming given f^i−1(R)\widehat{f}_{i-1}(R) (initial condition at t=tit=t_i), we search for a neural network f^:(θ;R)∈Θ×R3N→R\widehat{f}:(\theta;R)\in\Theta\times\mathbb{R}^{3N}\to\mathbb{R} optimized by minimizing

Li(PINN)(θ)=∥f^(θ;⋅)−f^i−1−dt2∇R2f^i−1+dt∇R(V^i−1f^i−1)−dtS^i−1f^i−1∥L2(R3N)2,(6)\mathcal{L}_i^{(\mathrm{PINN})}(\theta)=\left\|\widehat{f}(\theta;\cdot)-\widehat{f}_{i-1}-\frac{dt}{2}\nabla_R^2\widehat{f}_{i-1}+dt\nabla_R(\widehat{V}_{i-1}\widehat{f}_{i-1})-dt\widehat{S}_{i-1}\widehat{f}_{i-1}\right\|_{L^2(\mathbb{R}^{3N})}^2, \tag*{(6)}

where f^i−1=ϕ^i−12\widehat{f}_{i-1}=\widehat{\phi}_{i-1}^2 is given and (assuming enough regularity)

V^i−1(R)=∇R∣log⁡ϕ^i−1(R)∣(drift velocity),S^i−1(R)=ET−E^i−1(R)(branching term),E^i−1(R)=ϕ^i−1−1(R)Hϕ^i−1(R)(local energy).\begin{aligned} \widehat{V}_{i-1}(R)&=\nabla_R\left|\log\widehat{\phi}_{i-1}(R)\right| \quad\text{(drift velocity)},\\ \widehat{S}_{i-1}(R)&=E_T-\widehat{E}_{i-1}(R) \quad\text{(branching term)},\\ \widehat{E}_{i-1}(R)&=\widehat{\phi}_{i-1}^{-1}(R)H\widehat{\phi}_{i-1}(R) \quad\text{(local energy)}. \end{aligned}

The approximate solution to (2) at tit_i is then denoted by f^i(R)=f^(θi∗;R)\widehat{f}_i(R)=\widehat{f}(\theta_i^*;R), where

θi∗=argmin⁡θLi(θ).\theta_i^*=\operatorname*{argmin}_{\theta}\mathcal{L}_i(\theta).

In high-dimension, the L2L^2-norm are computed using Monte-Carlo integrations; this allows to connect PINN and DMC. Hence for ii large enough we get an approximation to f(R,ti)f(R,t_i) from which we get can obtain an approximate ground state and approximate ground state energy. More specifically, denoting {Rα}α\{R^\alpha\}_\alpha a set of MM randomly chosen learning nodes over R3N\mathbb{R}^{3N} approximating (6) by the commonly called training loss L~i(PINN)\widetilde{\mathcal{L}}_i^{(\mathrm{PINN})} defined as follows.

L~i(PINN)(θ)=1M∑α∣f^(θ;Rα)−f^i−1(Rα)−dt2∇R2f^i−1(Rα)+dt∇R(V^i−1f^i−1)(Rα)−dtS^i−1(Rα)f^i−1(Rα)∣2.(7)\widetilde{\mathcal{L}}_i^{(\mathrm{PINN})}(\theta)=\frac{1}{M}\sum_\alpha\left|\widehat{f}(\theta;R^\alpha)-\widehat{f}_{i-1}(R^\alpha)-\frac{dt}{2}\nabla_R^2\widehat{f}_{i-1}(R^\alpha)+dt\nabla_R\left(\widehat{V}_{i-1}\widehat{f}_{i-1}\right)(R^\alpha)-dt\widehat{S}_{i-1}(R^\alpha)\widehat{f}_{i-1}(R^\alpha)\right|^2. \tag*{(7)}

By abuse of notation, in the following we will keep the notation Li(PINN)(θ)\mathcal{L}_{i}^{(\mathrm{PINN})}(\theta) to represent the training loss.

Hereafter, we will use the L2L^2-inner product (⟨⋅,⋅⟩L2(R3N))(\langle\cdot,\cdot\rangle_{L^2(\mathbb{R}^{3N})}) and norm (∥⋅∥L2(R3N))(\|\cdot\|_{L^2(\mathbb{R}^{3N})}) to define the loss functions. However when the searched solution has more regularity (say HsH^s for s≥1s \ge1, corresponding to the L2L^2-functions with derivatives (up to the ssth ones) in L2L^2, using the HsH^s-norm can potentially improve the minimization process and/or provide a more precise approximate solution as recently proven in recent several papers, where PINN algorithms are applied for instance to Navier-Stokes equations, geophysical models or nonlinear Schrödinger equations [12, 13]. Further investigations will be conducted in the framework of quantum chemistry.

Inclusion of data

One of the key interests of neural networks is the use of data allowing to drive/supervise the learning of the searched eigenstate(s). It basically consists in adding to the loss function, a data-fitting contribution allowing for a more accurate neural network optimization. As an example of supervised learning, let us consider a Schrödinger operator with parameter-dependent potentials V(R,p)V(R,p) with p∈Πp \in\Pi and Π\Pi a bounded subset of Rπ\mathbb{R}^{\pi} with π∈N∖{0}\pi\in\mathbb{N}\setminus\{0\}. Then we search for a neural network ϕ^:(θ;R,p)∈Θ×R3N×Π↦ϕ^(θ;R,p)∈R\hat{\phi} : (\theta;R,p) \in\Theta\times\mathbb{R}^{3N} \times\Pi\mapsto\hat{\phi}(\theta;R,p) \in\mathbb{R} approximating a pp-dependent groundstate ϕG:(R,p)∈R3N×Π↦ϕG(R,p)∈R\phi_G : (R,p) \in\mathbb{R}^{3N}\times\Pi\mapsto\phi_G(R,p) \in\mathbb{R}. Assuming that the groundstate was computed for qq parameters p1,⋯ ,pqp_1,\cdots,p_q, we add to the loss function the following contribution/constraint.

L(θ)←L(θ)+∑k=1q∑α∣ϕ^(θ;Rα,pk)−ϕG(Rα,pk)∣2.\mathcal{L}(\theta) \leftarrow\mathcal{L}(\theta) + \sum_{k=1}^{q}\sum_{\alpha}\left|\hat{\phi}(\theta;R^\alpha,p_k)-\phi_G(R^\alpha,p_k)\right|^2.

As data inclusion is similarly performed within N-VDMC, Deep-Ritz/ferminet and PINN algorithms, in this paper we do not focus on this aspect.

Organization

The rest of the manuscript is organized as follows. In Section 2, we show that the standard PINN-DE algorithm can be reformulated as a fitting problem for the N-VDMC solution. Section 3 is devoted to the comparison at the optimization level, of a modified N-VDMC algorithm with the standard PINN-DE algorithm. We conclude in Section 4.

Standard N-VDMC vs PINN-DE

In this section, we establish connections between N-VDMC (trial wavefunction estimated by N-VMC) and PINN-DE algorithms. We show that a PINN-DE can be reformulated as a fitting problem for data corresponding to the DMC solution where the nodes used in the Monte Carlo integration are the DMC random walks.

In the following fif_i will refer to the DMC-solution at time tit_i, while f^i\hat{f}_i will refer to the PINN-solution.

Preliminary remark

In order to connect the PINN-DE and DMC methods we will rewrite those methods in a weak form. At iteration i≥1i \ge1, let us recall that the PINN loss function reads

Li(PINN)(θ)=∥f^(θ;⋅)−f^i−1−dt2∇R2f^i−1+dt∇R(V^i−1f^i−1)−dtS^i−1f^i−1∥L2(R3N)2.\mathcal{L}_{i}^{(\mathrm{PINN})}(\theta)=\left\|\widehat{f}(\theta;\cdot)-\widehat{f}_{i-1}-\frac{dt}{2}\nabla_{R}^{2}\widehat{f}_{i-1}+dt\nabla_{R}(\widehat{V}_{i-1}\widehat{f}_{i-1})-dt\widehat{S}_{i-1}\widehat{f}_{i-1}\right\|_{L^{2}(\mathbb{R}^{3N})}^{2}.

We denote by Φ\Phi the smooth function defined as

Φ(θ;⋅)=f^(θ;⋅)−f^i−1−dt2∇R2f^i−1+dt∇R(V^i−1f^i−1)−dtS^i−1f^i−1,\Phi(\theta;\cdot)=\widehat{f}(\theta;\cdot)-\widehat{f}_{i-1}-\frac{dt}{2}\nabla_{R}^{2}\widehat{f}_{i-1}+dt\nabla_{R}(\widehat{V}_{i-1}\widehat{f}_{i-1})-dt\widehat{S}_{i-1}\widehat{f}_{i-1},

and we rewrite Li(PINN)(θ)\mathcal{L}_{i}^{(\mathrm{PINN})}(\theta) in the form

Li(PINN)(θ)=⟨f^(θ;⋅)−f^i−1−dt2∇R2f^i−1+dt∇R(V^i−1f^i−1)−dtS^i−1f^i−1,Φ⟩=⟨f^(θ;⋅)−f^i−1,Φ⟩−dt⟨S^i−1f^i−1−dt2f^i−1,∇R2Φ⟩−dt⟨f^i−1,V^∇RΦ⟩.\begin{aligned} \mathcal{L}_{i}^{(\mathrm{PINN})}(\theta) &=\left\langle\widehat{f}(\theta;\cdot)-\widehat{f}_{i-1}-\frac{dt}{2}\nabla_{R}^{2}\widehat{f}_{i-1}+dt\nabla_{R}(\widehat{V}_{i-1}\widehat{f}_{i-1})-dt\widehat{S}_{i-1}\widehat{f}_{i-1},\Phi\right\rangle\\ &=\left\langle\widehat{f}(\theta;\cdot)-\widehat{f}_{i-1},\Phi\right\rangle-dt\left\langle\widehat{S}_{i-1}\widehat{f}_{i-1}-\frac{dt}{2}\widehat{f}_{i-1},\nabla_{R}^{2}\Phi\right\rangle-dt\left\langle\widehat{f}_{i-1},\widehat{V}\nabla_{R}\Phi\right\rangle. \end{aligned}

By abuse of notation (keeping the same notation for the training loss), and for randomly selecting learning nodes {Rα}α\{R^{\alpha}\}_{\alpha}, we get

Li(PINN)(θ)=1M∑α(f^(θ;Rα)−f^i−1(Rα))Φ(Rα)−dtS^i−1(Rα)f^i−1(Rα)Φ(Rα)−dt2f^i−1(Rα)∇R2Φ(Rα)−dtf^i−1(Rα)V^i−1(Rα)∇RΦ(Rα).(8)\begin{aligned} \mathcal{L}_{i}^{(\mathrm{PINN})}(\theta) ={}&\frac{1}{M}\sum_{\alpha}\left(\widehat{f}(\theta;R^{\alpha})-\widehat{f}_{i-1}(R^{\alpha})\right)\Phi(R^{\alpha})-dt\widehat{S}_{i-1}(R^{\alpha})\widehat{f}_{i-1}(R^{\alpha})\Phi(R^{\alpha})\\ &-\frac{dt}{2}\widehat{f}_{i-1}(R^{\alpha})\nabla_{R}^{2}\Phi(R^{\alpha})-dt\widehat{f}_{i-1}(R^{\alpha})\widehat{V}_{i-1}(R^{\alpha})\nabla_{R}\Phi(R^{\alpha}). \tag*{(8)} \end{aligned}

Then we define f^i=f^(θi∗;⋅)\widehat{f}_{i}=\widehat{f}(\theta_{i}^{*};\cdot) where θi∗=argmin⁡θLi(PINN)(θ)\theta_{i}^{*}=\operatorname*{argmin}_{\theta}\mathcal{L}_{i}^{(\mathrm{PINN})}(\theta).

Unlike PINN-DE, N-VDMC relies on the electron positions {Riα}α\{R_{i}^{\alpha}\}_{\alpha} as follows

fi(R)=∑αfi(Riα)δ(R−Riα)∑αfi(Riα).f_{i}(R)=\frac{\sum_{\alpha}f_{i}(R_{i}^{\alpha})\delta(R-R_{i}^{\alpha})}{\sum_{\alpha}f_{i}(R_{i}^{\alpha})}.

Expanding fif_{i} about RiαR_{i}^{\alpha}, we get

fi(Riα)=fi(Ri−1α)exp⁡(dt(S0(Ri−1α)+S0(Riα)/2))=fi(Ri−1α)(1+dtS0(Ri−1α))+dt3/2ηi−1∇RS0(Ri−1α)+O(dt2).\begin{aligned} f_{i}(R_{i}^{\alpha}) &=f_{i}(R_{i-1}^{\alpha})\exp\left(dt\left(S_{0}(R_{i-1}^{\alpha})+S_{0}(R_{i}^{\alpha})/2\right)\right)\\ &=f_{i}(R_{i-1}^{\alpha})\left(1+dtS_{0}(R_{i-1}^{\alpha})\right)+dt^{3/2}\eta_{i-1}\nabla_{R}S_{0}(R_{i-1}^{\alpha})+O(dt^{2}). \end{aligned}

We simply denote

fi(Riα)=fi(Ri−1α)(1+dtS0(Ri−1α))+O(ηi−1dt3/2)+O(dt2).(9)f_{i}(R_{i}^{\alpha})=f_{i}(R_{i-1}^{\alpha})\left(1+dtS_{0}(R_{i-1}^{\alpha})\right)+O(\eta_{i-1}dt^{3/2})+O(dt^{2}). \tag*{(9)}

Moreover in the distributional sense

δ(R−Riα)=δ(R−Ri−1α)−dtV0(Ri−1α)δ′(R−Ri−1α)+ηi2dt2δ′′(R−Ri−1α)−dtηi−1δ′(R−Ri−1α)+O(ηi−1dt3/2)+O(dt2).\delta(R-R_{i}^{\alpha})=\delta(R-R_{i-1}^{\alpha})-dtV_{0}(R_{i-1}^{\alpha})\delta'(R-R_{i-1}^{\alpha})+\eta_{i}^{2}\frac{dt}{2}\delta''(R-R_{i-1}^{\alpha})-\sqrt{dt}\eta_{i-1}\delta'(R-R_{i-1}^{\alpha})+O(\eta_{i-1}dt^{3/2})+O(dt^{2}).

Hence

fi(Riα)δ(R−Riα)=fi(Ri−1α)δ(R−Ri−1α)−dtV0(Ri−1α)fi(Ri−1α)δ′(R−Ri−1α)+dtS0(Ri−1α)fi(Ri−1α)δ(R−Ri−1α)+ηi−12dt2fi(Ri−1α)δ′′(R−Ri−1α)−dtηi−1fi(Ri−1α)δ′(R−Ri−1α)+O(ηi−1dt3/2)+O(dt2).\begin{aligned} f_i(R_i^\alpha)\delta(R-R_i^\alpha)={}&f_i(R_{i-1}^\alpha)\delta(R-R_{i-1}^\alpha)-dtV_0(R_{i-1}^\alpha)f_i(R_{i-1}^\alpha)\delta'(R-R_{i-1}^\alpha)\\ &+dtS_0(R_{i-1}^\alpha)f_i(R_{i-1}^\alpha)\delta(R-R_{i-1}^\alpha)+\eta_{i-1}^2\frac{dt}{2}f_i(R_{i-1}^\alpha)\delta''(R-R_{i-1}^\alpha)\\ &-\sqrt{dt}\eta_{i-1}f_i(R_{i-1}^\alpha)\delta'(R-R_{i-1}^\alpha)+O(\eta_{i-1}dt^{3/2})+O(dt^2). \end{aligned}

We then deduce that for any Φ\Phi smooth

⟨(fi(Riα)δ(R−Riα),Φ⟩=fi(Ri−1α)Φ(Ri−1α)(1+dtS0(Ri−1α))+dtfi−1(Ri−1α)V0(Ri−1α)∇RΦ(Ri−1α)+ηi−12dt2fi(Ri−1α)∇R2Φ(Ri−1α)+ηi−1dtfi(Ri−1α)∇RΦ(Ri−1α)+O(ηi−1dt3/2)+O(dt2).\begin{aligned} \langle(f_i(R_i^\alpha)\delta(R-R_i^\alpha),\Phi\rangle={}&f_i(R_{i-1}^\alpha)\Phi(R_{i-1}^\alpha)(1+dtS_0(R_{i-1}^\alpha))+dtf_{i-1}(R_{i-1}^\alpha)V_0(R_{i-1}^\alpha)\nabla_R\Phi(R_{i-1}^\alpha)\\ &+\eta_{i-1}^2\frac{dt}{2}f_i(R_{i-1}^\alpha)\nabla_R^2\Phi(R_{i-1}^\alpha)+\eta_{i-1}\sqrt{dt}f_i(R_{i-1}^\alpha)\nabla_R\Phi(R_{i-1}^\alpha)\\ &+O(\eta_{i-1}dt^{3/2})+O(dt^2). \end{aligned}

Then for any smooth function Φ\Phi, we have

fi(Riα)Φ(Riα)=fi(Riα)Φ(Ri−1α)(1+dtS0(Ri−1α))+dtfi−1(Ri−1α)V0(Ri−1α)∇RΦ(Ri−1α)+ηi−12dt2fi(Ri−1α)∇R2Φ(Ri−1α)+ηi−1dtfi(Ri−1α)∇RΦ(Ri−1α)+O(ηi−1dt3/2)+O(dt2).(10)\begin{aligned} f_i(R_i^\alpha)\Phi(R_i^\alpha)={}&f_i(R_i^\alpha)\Phi(R_{i-1}^\alpha)(1+dtS_0(R_{i-1}^\alpha))+dtf_{i-1}(R_{i-1}^\alpha)V_0(R_{i-1}^\alpha)\nabla_R\Phi(R_{i-1}^\alpha)\\ &+\eta_{i-1}^2\frac{dt}{2}f_i(R_{i-1}^\alpha)\nabla_R^2\Phi(R_{i-1}^\alpha)+\eta_{i-1}\sqrt{dt}f_i(R_{i-1}^\alpha)\nabla_R\Phi(R_{i-1}^\alpha)\\ &+O(\eta_{i-1}dt^{3/2})+O(dt^2). \tag*{(10)} \end{aligned}

In the following subsection, we will use (10) to reformulate (8).

PINN as a fitting algorithm for DMC

The objective is here then to rewrite Li(PINN)\mathcal{L}_i^{(\mathrm{PINN})} using {fi}i\{f_i\}_i, the N-VDMC solution. At iteration i−1i-1, we assume given f^i−1(R)\hat{f}_{i-1}(R) the approximate solution to PINN-DE at time tit_i. We construct the smooth function Φ\Phi, as follows

Φ:(θ;R)∈Θ×R3N↦f^(θ;R)−f^i−1(R)(1+dtS^i−1(R))−dt2∇R2f^i−1(R)+dt∇R(V^i−1(R)f^i−1(R)).\Phi:(\theta;R)\in\Theta\times\mathbb{R}^{3N}\mapsto\hat{f}(\theta;R)-\hat{f}_{i-1}(R)(1+dt\hat{S}_{i-1}(R))-\frac{dt}{2}\nabla_R^2\hat{f}_{i-1}(R)+dt\nabla_R(\hat{V}_{i-1}(R)\hat{f}_{i-1}(R)).

In order to connect the PINN and N-VDMC algorithms, in PINN’s loss function, we select the learning nodes {Rα}α\{R^\alpha\}_\alpha over Qi=[ti,ti+1]×R3NQ_i=[t_i,t_{i+1}]\times\mathbb{R}^{3N}, as the electron positions from DMC; that is: {Rα}α={Riα}α\{R^\alpha\}_\alpha=\{R_i^\alpha\}_\alpha. For the sake of readiness, we use hereafter the following notation

f^i−1α=f^i−1(Ri−1α),fi−1α=fi−1(Ri−1α),f0;iα=f0(Riα),\hat{f}_{i-1}^\alpha=\hat{f}_{i-1}(R_{i-1}^\alpha),\qquad f_{i-1}^\alpha=f_{i-1}(R_{i-1}^\alpha),\qquad f_{0;i}^\alpha=f_0(R_i^\alpha),
Vi−1α=V^i−1(Ri−1α),V^i−1αf^i−1α=(Vi−1αf^i−1α)i−1α,V0;i−1α=V0(Ri−1α),V_{i-1}^\alpha=\hat{V}_{i-1}(R_{i-1}^\alpha),\qquad \hat{V}_{i-1}^\alpha\hat{f}_{i-1}^\alpha=(V_{i-1}^\alpha\hat{f}_{i-1}^\alpha)_{i-1}^\alpha,\qquad V_{0;i-1}^\alpha=V_0(R_{i-1}^\alpha),
∇RΦi−1α(θ)=∇RΦ(θ;Riα),S^i−1α=S^i−1(Ri−1α),S0;i−1α=S0(Ri−1α),\nabla_R\Phi_{i-1}^\alpha(\theta)=\nabla_R\Phi(\theta;R_i^\alpha),\qquad \hat{S}_{i-1}^\alpha=\hat{S}_{i-1}(R_{i-1}^\alpha),\qquad S_{0;i-1}^\alpha=S_0(R_{i-1}^\alpha),
Φi−1α(θ)=Φ(θ;Ri−1α).\Phi_{i-1}^\alpha(\theta)=\Phi(\theta;R_{i-1}^\alpha).

We next introduce the following sequence {ϵi−1α}α\{\epsilon_{i-1}^\alpha\}_\alpha and {δi−1α}α\{\delta_{i-1}^\alpha\}_\alpha

ϵi−1α=fi−1α−f^i−1α,δi−1α=f0;i−1α−f^i−1α,\epsilon_{i-1}^\alpha=f_{i-1}^\alpha-\hat{f}_{i-1}^\alpha,\qquad \delta_{i-1}^\alpha=f_{0;i-1}^\alpha-\hat{f}_{i-1}^\alpha,

corresponding to the difference between i) the N-VDMC and PINN-DE solutions and ii) the trial function and the PINN-DE solution, at time ti−1t_{i-1} and at {Ri−1α}α\{R_{i-1}^{\alpha}\}_{\alpha}. Hence using fiα=fi−1α(1+dtS^i−1α)+O(ηi−1dt3/2)+O(dt2)f_{i}^{\alpha} = f_{i-1}^{\alpha}(1+dt\widehat{S}_{i-1}^{\alpha})+O(\eta_{i-1}dt^{3/2})+O(dt^{2}) (see (9)), we get

f^i−1α(1+dtS^i−1α)=fiα(1+dtS0;i−1α)+εi−1α(1+dtS^i−1α)+dtfi−1α(S^i−1α−S0;i−1α)=fiα−εi−1α(1+dtS^i−1α)+dtfi−1α(S^i−1α−S0;i−1α)+O(ηi−1dt3/2)+O(dt2).(11)\begin{aligned} \widehat{f}_{i-1}^{\alpha}(1+dt\widehat{S}_{i-1}^{\alpha}) &= f_{i}^{\alpha}(1+dtS_{0;i-1}^{\alpha})+\varepsilon_{i-1}^{\alpha}(1+dt\widehat{S}_{i-1}^{\alpha})+dtf_{i-1}^{\alpha}(\widehat{S}_{i-1}^{\alpha}-S_{0;i-1}^{\alpha})\\ &=f_{i}^{\alpha}-\varepsilon_{i-1}^{\alpha}(1+dt\widehat{S}_{i-1}^{\alpha})+dtf_{i-1}^{\alpha}(\widehat{S}_{i-1}^{\alpha}-S_{0;i-1}^{\alpha})+O(\eta_{i-1}dt^{3/2})+O(dt^{2}). \tag*{(11)} \end{aligned}

From (8), using Monte-Carlo integration Li(PINN)\mathcal{L}_{i}^{(\mathrm{PINN})} with {Rα}α={Ri−1α}α\{R^{\alpha}\}_{\alpha}=\{R_{i-1}^{\alpha}\}_{\alpha}, we then have

Li(PINN)(θ)=1M∑α{(f^α(θ;Ri−1α)−fiα)Φi−1α(θ)−dt2f^i−1α∇R2Φi−1α(θ)−dtV^i−1αf^i−1α∇RΦi−1α(θ)+εi−1αΦi−1α(θ)(1+dtS^i−1α)−dtfi−1α(S^i−1α−S0;i−1α)Φi−1α(θ)}.\mathcal{L}_{i}^{(\mathrm{PINN})}(\theta)=\frac{1}{M}\sum_{\alpha}\left\{\left(\widehat{f}^{\alpha}(\theta;R_{i-1}^{\alpha})-f_{i}^{\alpha}\right)\Phi_{i-1}^{\alpha}(\theta)-\frac{dt}{2}\widehat{f}_{i-1}^{\alpha}\nabla_{R}^{2}\Phi_{i-1}^{\alpha}(\theta)-dt\widehat{V}_{i-1}^{\alpha}\widehat{f}_{i-1}^{\alpha}\nabla_{R}\Phi_{i-1}^{\alpha}(\theta)+\varepsilon_{i-1}^{\alpha}\Phi_{i-1}^{\alpha}(\theta)(1+dt\widehat{S}_{i-1}^{\alpha})-dtf_{i-1}^{\alpha}(\widehat{S}_{i-1}^{\alpha}-S_{0;i-1}^{\alpha})\Phi_{i-1}^{\alpha}(\theta)\right\}.

Then using (10), we can rewrite (8) as follows

Li(PINN)(θ)=1M∑α{(f^α(θ;Ri−1α)−fiα)Φi−1α(θ)−dt(V^i−1α−V0;i−1α)f^i−1α∇RΦi−1α(θ)+εi−1αΦi−1α(θ)(1+dtS^i−1α+dt2∇R2)∇R2Φi−1α(θ)+dtV^i−1αf^i−1α∇RΦi−1α(θ)−dtfi−1α(S^i−1α−S0;i−1α)Φi−1α(θ)+ηdtf^i−1α∇RΦi−1α(θ)+O(ηi−1dt3/2)+O(dt2)}.(12)\begin{aligned} \mathcal{L}_{i}^{(\mathrm{PINN})}(\theta)=\frac{1}{M}\sum_{\alpha}\Big\{&\left(\widehat{f}^{\alpha}(\theta;R_{i-1}^{\alpha})-f_{i}^{\alpha}\right)\Phi_{i-1}^{\alpha}(\theta)-dt(\widehat{V}_{i-1}^{\alpha}-V_{0;i-1}^{\alpha})\widehat{f}_{i-1}^{\alpha}\nabla_{R}\Phi_{i-1}^{\alpha}(\theta)\\ &+\varepsilon_{i-1}^{\alpha}\Phi_{i-1}^{\alpha}(\theta)\left(1+dt\widehat{S}_{i-1}^{\alpha}+\frac{dt}{2}\nabla_{R}^{2}\right)\nabla_{R}^{2}\Phi_{i-1}^{\alpha}(\theta)+dt\widehat{V}_{i-1}^{\alpha}\widehat{f}_{i-1}^{\alpha}\nabla_{R}\Phi_{i-1}^{\alpha}(\theta)\\ &-dtf_{i-1}^{\alpha}(\widehat{S}_{i-1}^{\alpha}-S_{0;i-1}^{\alpha})\Phi_{i-1}^{\alpha}(\theta)+\eta\sqrt{dt}\widehat{f}_{i-1}^{\alpha}\nabla_{R}\Phi_{i-1}^{\alpha}(\theta)+O(\eta_{i-1}dt^{3/2})+O(dt^{2})\Big\}. \tag*{(12)} \end{aligned}

From (11) and as ∇R2f^i−1/2=∇R(V^i−1f^i−1)\nabla_{R}^{2}\widehat{f}_{i-1}/2=\nabla_{R}(\widehat{V}_{i-1}\widehat{f}_{i-1}) (setting f^i−1=ϕ^i−12\widehat{f}_{i-1}=\widehat{\phi}_{i-1}^{2} and assuming enough regularity) then

Φi−1α(θ)=f^α(θ;Ri−1α)−f^i−1α(1+dtS^i−1α)−dt2∇R2f^i−1α+dt∇R(V^i−1f^i−1)i−1α=f^α(θ;Ri−1α)−fiα+εi−1α(1+dtS0;i−1α)−dtfi−1α(S^i−1α−S0;i−1α)+O(ηi−1dt3/2)+O(dt2).\begin{aligned} \Phi_{i-1}^{\alpha}(\theta)&=\widehat{f}^{\alpha}(\theta;R_{i-1}^{\alpha})-\widehat{f}_{i-1}^{\alpha}(1+dt\widehat{S}_{i-1}^{\alpha})-\frac{dt}{2}\nabla_{R}^{2}\widehat{f}_{i-1}^{\alpha}+dt\nabla_{R}(\widehat{V}_{i-1}\widehat{f}_{i-1})_{i-1}^{\alpha}\\ &=\widehat{f}^{\alpha}(\theta;R_{i-1}^{\alpha})-f_{i}^{\alpha}+\varepsilon_{i-1}^{\alpha}(1+dtS_{0;i-1}^{\alpha})-dtf_{i-1}^{\alpha}(\widehat{S}_{i-1}^{\alpha}-S_{0;i-1}^{\alpha})+O(\eta_{i-1}dt^{3/2})+O(dt^{2}). \end{aligned}

We deduce that

(f^α(θ;Ri−1α)−fiα)Φi−1α(θ)=(f^α(θ;Ri−1α)−fiα)2+(f^α(θ;Ri−1α)−fiα)(εi−1α(1+dtS0;i−1α)−dtfi−1α(S^i−1α−S0;i−1α))+O(ηi−1dt3/2)+O(dt2).\begin{aligned} \left(\widehat{f}^{\alpha}(\theta;R_{i-1}^{\alpha})-f_{i}^{\alpha}\right)\Phi_{i-1}^{\alpha}(\theta) ={}&\left(\widehat{f}^{\alpha}(\theta;R_{i-1}^{\alpha})-f_{i}^{\alpha}\right)^{2}\\ &+\left(\widehat{f}^{\alpha}(\theta;R_{i-1}^{\alpha})-f_{i}^{\alpha}\right)\left(\varepsilon_{i-1}^{\alpha}(1+dtS_{0;i-1}^{\alpha})-dtf_{i-1}^{\alpha}(\widehat{S}_{i-1}^{\alpha}-S_{0;i-1}^{\alpha})\right)\\ &+O(\eta_{i-1}dt^{3/2})+O(dt^{2}). \end{aligned}

We next use that η\eta is a Gaussian random process, such that

1M∑αηi−1=O(1M),1M∑αηi−12=3N+O(1M).\frac{1}{M}\sum_{\alpha}\eta_{i-1}=O\left(\frac{1}{\sqrt{M}}\right),\qquad\frac{1}{M}\sum_{\alpha}\eta_{i-1}^{2}=3N+O\left(\frac{1}{\sqrt{M}}\right).

This allows to simplify the expression of the loss function and getting rid of the ηi−1\eta_{i-1} terms, the PINN-DE loss function can be rewritten as a fitting problem.

Li(PINN)(θ)=1M∑α(f^α(θ;Ri−1α)−fiα)2+1M∑αεiα(θ)+O(1M).\mathcal{L}_{i}^{(\mathrm{PINN})}(\theta)=\frac{1}{M}\sum_{\alpha}\left(\widehat{f}^{\alpha}(\theta;R_{i-1}^{\alpha})-f_{i}^{\alpha}\right)^{2}+\frac{1}{M}\sum_{\alpha}\varepsilon_{i}^{\alpha}(\theta)+O\left(\frac{1}{\sqrt{M}}\right).

where

εiα(θ)=(f^α(θ;Ri−1α)−fiα)(εi−1α(1+dtS0;i−1α)−dtfi−1α(S^i−1α−S0;i−1α))−dt(V^i−1α−V0;i−1α)f^i−1α∇RΦi−1α(θ)−dtfi−1α(S^i−1α−S0;i−1α)Φi−1α(θ)+εi−1αΦi−1α(θ)(1+dtS^i−1α+dt2∇R2Φi−1α(θ)+dtV^i−1αf^i−1α∇RΦi−1α(θ))+O(dt2).\begin{aligned} \varepsilon_i^\alpha(\theta) ={}& \left(\widehat{f}^\alpha(\theta;R^\alpha_{i-1})-f_i^\alpha\right)\left(\varepsilon^\alpha_{i-1}\left(1+dtS^\alpha_{0;i-1}\right)-dt f^\alpha_{i-1}\left(\widehat{S}^\alpha_{i-1}-S^\alpha_{0;i-1}\right)\right) \\ &-dt\left(\widehat{V}^\alpha_{i-1}-V^\alpha_{0;i-1}\right)\widehat{f}^\alpha_{i-1}\nabla_R\Phi^\alpha_{i-1}(\theta)-dt f^\alpha_{i-1}\left(\widehat{S}^\alpha_{i-1}-S^\alpha_{0;i-1}\right)\Phi^\alpha_{i-1}(\theta) \\ &+\varepsilon^\alpha_{i-1}\Phi^\alpha_{i-1}(\theta)\left(1+dt\widehat{S}^\alpha_{i-1}+\frac{dt}{2}\nabla_R^2\Phi^\alpha_{i-1}(\theta)+dt\widehat{V}^\alpha_{i-1}\widehat{f}^\alpha_{i-1}\nabla_R\Phi^\alpha_{i-1}(\theta)\right)+O(dt^2). \end{aligned}

We easily get the existence of Ci−1>0C_{i-1}>0, such that

∣εiα(θ)∣≤(∣εi−1α∣+Ci−1dt∣δi−1α∣)∣f^α(θ;Ri−1α)−fiα∣+O(dt2).\left|\varepsilon_i^\alpha(\theta)\right|\leq\left(\left|\varepsilon^\alpha_{i-1}\right|+C_{i-1}dt\left|\delta^\alpha_{i-1}\right|\right)\left|\widehat{f}^\alpha(\theta;R^\alpha_{i-1})-f_i^\alpha\right|+O(dt^2).

We can actually refine this result. For i=1i=1, as the initial data in PINN and DMC are taken identical then ε0α\varepsilon^\alpha_0 and δ0α\delta^\alpha_0 are null, so that S^0α=S0;0α\widehat{S}^\alpha_0=S^\alpha_{0;0}, V^0α=V0;0α\widehat{V}^\alpha_0=V^\alpha_{0;0} and as a consequence ε0α(θ)=O(dt2)\varepsilon^\alpha_0(\theta)=O(dt^2). Hence, assuming that the loss can be as small as wanted (which is however in general, a very strong assumption), we can deduce by induction that εiα(θ)=O(dt2)\varepsilon^\alpha_i(\theta)=O(dt^2). Thus, PINN can be seen as a data-fitting algorithm N-VDMC from data {fiα}α\{f_i^\alpha\}_\alpha up to a (small) correction εi−1α\varepsilon^\alpha_{i-1}. More specifically, we can state the following proposition.

Proposition 2.1. Let us denote by {fiα}α\{f_i^\alpha\}_\alpha the N-VDMC solution at time iteration i≥1i\geq1. Assume that the PINN training loss function can be as small as possible up to iteration i−1i-1. Then, at iteration ii, the PINN training loss can be reformulated as a fitting problem on {fiα}α\{f_i^\alpha\}_\alpha:

Li(θ)=1M∑α(f^α(θ;Ri−1α)−fiα)2+1M∑αεiα(θ)+O(1M),\mathcal{L}_i(\theta)=\frac{1}{M}\sum_\alpha\left(\widehat{f}^\alpha(\theta;R^\alpha_{i-1})-f_i^\alpha\right)^2+\frac{1}{M}\sum_\alpha\varepsilon_i^\alpha(\theta)+O\left(\frac{1}{\sqrt{M}}\right),

where εiα(θ)=O(dt2)\varepsilon_i^\alpha(\theta)=O(dt^2) and MM is the number of learning nodes.

Optimization algorithm for Modified N-VDMC and PINN-DE

In this section we show that both the N-VDMC and PINN-DE algorithms lead at first order, to comparable optimization algorithms; hence lead to comparable optimized neural networks approximating the groundstate to the Schrödinger operator. More specifically, we show that the PINN method i) applied to the Imaginary Time Method (ITM) referred here as PINN-ITM, and ii) applied to the diffusion equation (2) PINN-DE, lead to a gradient descent algorithm similar to the one of a modified N-VDMC method. In the previous section, the N-VDMC algorithm (1.4) is initialized once for all at t=t0t=t_0 thanks to a N-VMC algorithm (Ferminet/Deep-Ritz). Hereafter, on each time interval [ti,ti+1][t_i,t_{i+1}], the so-called modified N-VDMC is reinitialized (at t=tit=t_i) from a N-VMC algorithm, as long as ϕ^i−1\widehat{\phi}_{i-1} is smooth and strictly positive; we refer to § 2. Moreover within the PINN algorithms, the learning nodes are taken as the electron positions from DMC. For simplicity, in the following we also set ET=0E_T=0.

On QiQ_i and setting f^(θ;ti)=f^i−1=ϕ^i−12\widehat{f}(\theta;t_i)=\widehat{f}_{i-1}=\widehat{\phi}_{i-1}^2 and f^(θ;⋅)=ψ^(θ;⋅)ϕ^i−1\widehat{f}(\theta;\cdot)=\widehat{\psi}(\theta;\cdot)\widehat{\phi}_{i-1}, we rewrite

f^(θ;⋅)=f^i−1−dtK(f^(θ;⋅)),\widehat{f}(\theta;\cdot)=\widehat{f}_{i-1}-dtK(\widehat{f}(\theta;\cdot)),

as follows

ψ^ϕ^i−1=ϕ^i−12+dt2∇R2(ψ^ϕ^i−1)−dt∇R(ψ^ϕ^i−1∇R∣log⁡ϕ^i−1∣)−dtψ^Hϕ^i−1.\widehat{\psi}\widehat{\phi}_{i-1} = \widehat{\phi}_{i-1}^{2} +\frac{dt}{2}\nabla_{R}^{2}(\widehat{\psi}\widehat{\phi}_{i-1}) -dt\nabla_{R}(\widehat{\psi}\widehat{\phi}_{i-1}\nabla_{R}|\log\widehat{\phi}_{i-1}|) -dt\widehat{\psi}H\widehat{\phi}_{i-1}.

Then we get

ψ^ϕ^i−1=ϕ^i−12+dt2ψ^∇R2ϕ^i−1+dt2ϕ^i−1∇R2ψ^+dt∇Rψ^∇Rϕ^i−1−dt∇R(ψ^∇Rϕ^i−1)−dtψ^Hϕ^i−1,\widehat{\psi}\widehat{\phi}_{i-1} = \widehat{\phi}_{i-1}^{2} +\frac{dt}{2}\widehat{\psi}\nabla_{R}^{2}\widehat{\phi}_{i-1} +\frac{dt}{2}\widehat{\phi}_{i-1}\nabla_{R}^{2}\widehat{\psi} +dt\nabla_{R}\widehat{\psi}\nabla_{R}\widehat{\phi}_{i-1} -dt\nabla_{R}(\widehat{\psi}\nabla_{R}\widehat{\phi}_{i-1}) -dt\widehat{\psi}H\widehat{\phi}_{i-1},

then

ψ^ϕ^i−1=ϕ^i−12+dt2ϕ^i−1∇R2ψ^−dt2ψ^∇R2ϕ^i−1−dtϕ^i−1Vp(R)ψ^+dt2ψ^∇R2ϕ^i−1=ϕ^i−1(ϕ^i−1−dtHψ^).(13)\begin{aligned} \widehat{\psi}\widehat{\phi}_{i-1} &=\widehat{\phi}_{i-1}^{2} +\frac{dt}{2}\widehat{\phi}_{i-1}\nabla_{R}^{2}\widehat{\psi} -\frac{dt}{2}\widehat{\psi}\nabla_{R}^{2}\widehat{\phi}_{i-1} -dt\widehat{\phi}_{i-1}V_{p}(R)\widehat{\psi} +\frac{dt}{2}\widehat{\psi}\nabla_{R}^{2}\widehat{\phi}_{i-1}\\ &=\widehat{\phi}_{i-1}\left(\widehat{\phi}_{i-1}-dtH\widehat{\psi}\right). \tag*{(13)} \end{aligned}

Neural network optimization

In this subsection, we derive the optimization algorithm for the 3 different approaches: PINN-ITM, PINN-DE and N-VDMC. We here assume that on each QiQ_i the same learning nodes/randoms walks {Riα}α={Riα}α\{R_i^\alpha\}_\alpha=\{R_i^\alpha\}_\alpha are used in the 3 different algorithms.

  • Neural network optimization for PINN-ITM. Let us first recall that the imaginary time method can be reformulated as a minimizing the energy associated to the Hamiltonian HH.

EG:=min⁡χ∈L2(R3N;R)R(χ),whereR(χ)=⟨Hχ,χ⟩⟨χ,χ⟩.(14)E_G:=\min_{\chi\in L^2(\mathbb{R}^{3N};\mathbb{R})}\mathcal{R}(\chi),\quad\text{where}\quad\mathcal{R}(\chi)=\frac{\langle H\chi,\chi\rangle}{\langle\chi,\chi\rangle}. \tag*{(14)}

Minimizing R\mathcal{R} the Rayleigh quotient, is equivalent under technical conditions to the imaginary time method. From for any tt and assuming given a normalized function ϕ0\phi_0 in R3N\mathbb{R}^{3N}, we search for ψ\psi in Q=R3N×R+Q=\mathbb{R}^{3N}\times\mathbb{R}_+ solution to the time-dependent Schrödinger equation in imaginary time, such that

∂tψ(R,t)=−Hψ(R,t),(R,t)∈Q,ψ(R,0)=ϕ0(R),R∈R3N,ϕ(R,t)=ψ(R,t)∥ψ(⋅,t)∥L2(R3N;R),(R,t)∈Q.(15)\begin{aligned} \partial_t\psi(R,t)&=-H\psi(R,t),\quad(R,t)\in Q,\\ \psi(R,0)&=\phi_0(R),\quad R\in\mathbb{R}^{3N},\\ \phi(R,t)&=\frac{\psi(R,t)}{\|\psi(\cdot,t)\|_{L^2(\mathbb{R}^{3N};\mathbb{R})}},\quad(R,t)\in Q. \tag*{(15)} \end{aligned}

By showing

ddtE(ψ(⋅,t)∥ψ(⋅,t)∥L2(R3N;R))≤0,\frac{d}{dt}E\left(\frac{\psi(\cdot,t)}{\|\psi(\cdot,t)\|_{L^2(\mathbb{R}^{3N};\mathbb{R})}}\right)\leq0,

for negative potentials (see [4, 3]), then lim⁡t→∞ψ(⋅,t)=ϕG\lim_{t\to\infty}\psi(\cdot,t)=\phi_G. In other words under technical assumptions, solving (14) is equivalent to solving (15).

In practice, in order to solve (15) the imaginary time method requires a time-discretization t0=0<t1<⋯<ti<ti+1<⋯t_0 = 0 < t_1 < \cdots< t_i < t_{i+1} < \cdots. Then, assumed given a normalized function ϕi−1\phi_{i-1} in R3N\mathbb{R}^{3N}, we search for ϕi\phi_i in R3N\mathbb{R}^{3N} such that on Qi=R3N×[ti−1,ti]Q_i = \mathbb{R}^{3N} \times[t_{i-1}, t_i]

∂tψ(R,t)=−Hψ(R,t),(R,t)∈Qi,ψ(R,ti−1)=ϕi−1(R),R∈R3N,ϕi(R)=ψ(R,ti)∥ψ(⋅,ti)∥L2(R3N),R∈R3N.(16)\begin{aligned} \partial_t \psi(R,t) &= -H\psi(R,t), \quad(R,t) \in Q_i,\\ \psi(R,t_{i-1}) &= \phi_{i-1}(R), \quad R \in\mathbb{R}^{3N},\\ \phi_i(R) &= \frac{\psi(R,t_i)}{\lVert\psi(\cdot,t_i)\rVert_{L^2(\mathbb{R}^{3N})}}, \quad R \in\mathbb{R}^{3N}. \tag*{(16)} \end{aligned}

Semi-discretizing (16) in time, we get

ψi(R)=ϕi−1(R)−dtHϕi−1(R),R∈R3N,ϕi(R)=ψi(R)∥ψi∥L2(R3N),R∈R3N.(17)\begin{aligned} \psi_i(R) &= \phi_{i-1}(R) - dtH\phi_{i-1}(R), \quad R \in\mathbb{R}^{3N},\\ \phi_i(R) &= \frac{\psi_i(R)}{\lVert\psi_i\rVert_{L^2(\mathbb{R}^{3N})}}, \quad R \in\mathbb{R}^{3N}. \tag*{(17)} \end{aligned}

Solving (17) with PINN is equivalent to searching for a neural network using PINN ψ^(ϑi;R)\widehat{\psi}(\vartheta_i; R) such that ϑi=argmin⁡ϑMi(ϑ)\vartheta_i = \operatorname*{argmin}_{\vartheta}\mathcal{M}_i(\vartheta), where (at the continuous level)

Mi(PINN)(ϑ)=∥ψ^(ϑ;⋅)−(1−dtH)ϕ^i−1∥L2(R3N)2.\mathcal{M}_i^{(\mathrm{PINN})}(\vartheta) = \left\lVert\widehat{\psi}(\vartheta;\cdot) - (1-dtH)\widehat{\phi}_{i-1}\right\rVert_{L^2(\mathbb{R}^{3N})}^2.

Hence

∇ϑMi(PINN)(ϑ)=2⟨∇ϑψ^(ϑ;⋅),ψ^(ϑ;⋅)−(1−dtH)ϕ^i−1⟩.\nabla_{\vartheta}\mathcal{M}_i^{(\mathrm{PINN})}(\vartheta) = 2\left\langle\nabla_{\vartheta}\widehat{\psi}(\vartheta;\cdot), \widehat{\psi}(\vartheta;\cdot) - (1-dtH)\widehat{\phi}_{i-1}\right\rangle.

Thus, we get

Proposition 3.1. The gradient descent algorithm applied to the training loss approximating the continuous PINN-ITM loss Mi(PINN)\mathcal{M}_i^{(\mathrm{PINN})}, using learning nodes {Riα}α\{R_i^\alpha\}_\alpha on R3N\mathbb{R}^{3N}, reads as follows

ϑk+1=ϑk−2νkM∑α{∇ϑψ^(ϑk;Riα)[ψ^(ϑ;Riα)−ϕ^i−1(Riα)+dtHϕ^i−1(Riα)]},(18)\vartheta_{k+1} = \vartheta_k - \frac{2\nu_k}{M}\sum_{\alpha}\left\{\nabla_{\vartheta}\widehat{\psi}(\vartheta_k;R_i^\alpha)\left[\widehat{\psi}(\vartheta;R_i^\alpha)-\widehat{\phi}_{i-1}(R_i^\alpha)+dtH\widehat{\phi}_{i-1}(R_i^\alpha)\right]\right\}, \tag*{(18)}

with learning rate νk\nu_k.

  • Neural network optimization for PINN-DE. We here reformulate the DE presented in Subsection 1.4 from (16) on QiQ_i by multiplying the PDE by ϕi−1\phi_{i-1}. We define f:=ϕi−1ψf := \phi_{i-1}\psi which satisfies on QiQ_i

∂tf(R,t)=−K(fi−1)(R),(R,t)∈Qi,f(R,ti−1)=fi−1(R),R∈R3N.(19)\begin{aligned} \partial_t f(R,t) &= -K(f_{i-1})(R), \quad(R,t) \in Q_i,\\ f(R,t_{i-1}) &= f_{i-1}(R), \quad R \in\mathbb{R}^{3N}. \tag*{(19)} \end{aligned}

Then assuming f^i−1\widehat{f}_{i-1} given, a PINN method consists in searching for a neural network f^(θi∗;⋅)\widehat{f}(\theta_i^*;\cdot) with θi∗=argmin⁡θLi(PINN)(θ)\theta_i^* = \operatorname*{argmin}_{\theta}\mathcal{L}_i^{(\mathrm{PINN})}(\theta), where (at the continuous level)

Li(PINN)(θ)=∥f^(θ;⋅)−f^i−1−dt2∇R2f^i−1+dt∇R(V^i−1f^i−1)−dtS^i−1f^i−1∥L2(R3N)2=∥f^(θ;⋅)−f^i−1+dtK(f^i−1)∥L2(R3N)2.\begin{aligned} \mathcal{L}_i^{(\mathrm{PINN})}(\theta) &= \left\lVert\widehat{f}(\theta;\cdot)-\widehat{f}_{i-1}-\frac{dt}{2}\nabla_R^2\widehat{f}_{i-1}+dt\nabla_R(\widehat{V}_{i-1}\widehat{f}_{i-1})-dt\widehat{S}_{i-1}\widehat{f}_{i-1}\right\rVert_{L^2(\mathbb{R}^{3N})}^2\\ &= \left\lVert\widehat{f}(\theta;\cdot)-\widehat{f}_{i-1}+dtK(\widehat{f}_{i-1})\right\rVert_{L^2(\mathbb{R}^{3N})}^2. \end{aligned}

The gradient descent method requires the gradient of Li(PINN)\mathcal{L}_i^{(\mathrm{PINN})}

∇θLi(PINN)(θ)=2⟨∇θf^(θ;⋅),f^(θ;⋅)−f^i−1+dtK(f^i−1)⟩.\nabla_\theta\mathcal{L}_i^{(\mathrm{PINN})}(\theta) = 2\langle\nabla_\theta\widehat{f}(\theta;\cdot),\widehat{f}(\theta;\cdot)-\widehat{f}_{i-1}+dtK(\widehat{f}_{i-1})\rangle.

If we set

f^(θ;⋅)=ϕ^i−1ϕ^(θ;⋅),f^i−1=ϕ^i−12,\widehat{f}(\theta;\cdot)=\widehat{\phi}_{i-1}\widehat{\phi}(\theta;\cdot),\quad\widehat{f}_{i-1}=\widehat{\phi}_{i-1}^{2},

then, from (13) we get

∇θLi(PINN)(θ)=2⟨ϕ^i−1∇θϕ^(θ;⋅),ϕ^i−1ϕ^(θ;⋅)−ϕ^i−12+dtK(ϕ^i−12)⟩=2⟨ϕ^i−12∇θϕ^(θ;⋅),ϕ^(θ;⋅)−ϕ^i−1+dtHϕ^i−1⟩.\begin{aligned} \nabla_\theta\mathcal{L}_i^{(\mathrm{PINN})}(\theta) &=2\langle\widehat{\phi}_{i-1}\nabla_\theta\widehat{\phi}(\theta;\cdot),\widehat{\phi}_{i-1}\widehat{\phi}(\theta;\cdot)-\widehat{\phi}_{i-1}^{2}+dtK(\widehat{\phi}_{i-1}^{2})\rangle\\ &=2\langle\widehat{\phi}_{i-1}^{2}\nabla_\theta\widehat{\phi}(\theta;\cdot),\widehat{\phi}(\theta;\cdot)-\widehat{\phi}_{i-1}+dtH\widehat{\phi}_{i-1}\rangle. \end{aligned}

Setting ⟨u,v⟩ϕ^i−1=⟨ϕ^i−12u,v⟩\langle u,v\rangle_{\widehat{\phi}_{i-1}}=\langle\widehat{\phi}_{i-1}^{2}u,v\rangle, we can in particular we rewrite the loss function as follows

∇θLi(PINN)(θ)=2⟨∇θϕ^(θ;⋅),ϕ^(θ;⋅)−ϕ^i−1+dtHϕ^i−1⟩ϕ^i−1.\nabla_\theta\mathcal{L}_i^{(\mathrm{PINN})}(\theta)=2\langle\nabla_\theta\widehat{\phi}(\theta;\cdot),\widehat{\phi}(\theta;\cdot)-\widehat{\phi}_{i-1}+dtH\widehat{\phi}_{i-1}\rangle_{\widehat{\phi}_{i-1}}.

Hence, we get

Proposition 3.2. The gradient descent algorithm applied to the training loss approximating the continuous PINN-DE loss Li(PINN)\mathcal{L}_i^{(\mathrm{PINN})}, using learning nodes {Riα}α\{R_i^\alpha\}_\alpha on R3N\mathbb{R}^{3N}, reads as follows

θk+1=θk−2νkM∑α{∑αϕ^i−12(Riα)∇θϕ^(θ;Riα)[ϕ^(θ;Riα)−ϕ^i−1(Riα)+dtHϕ^i−1(Riα)]},(20)\theta_{k+1}=\theta_k-\frac{2\nu_k}{M}\sum_\alpha\left\{\sum_\alpha\widehat{\phi}_{i-1}^{2}(R_i^\alpha)\nabla_\theta\widehat{\phi}(\theta;R_i^\alpha)\left[\widehat{\phi}(\theta;R_i^\alpha)-\widehat{\phi}_{i-1}(R_i^\alpha)+dtH\widehat{\phi}_{i-1}(R_i^\alpha)\right]\right\}, \tag*{(20)}

with learning rate νk\nu_k.

  • Neural network optimization for N-VDMC. The N-VDMC method is a combination of N-VCM and DMC methods. More specifically, unlike the “standard” N-VDMC presented in the previous section where N-VMC was only applied once at t=t0t=t_0, we here assume that VMC is applied at each tit_i with i≥1i\geq1. In other words, on QiQ_i, we initialize the wavefunction with a N-VMC algorithm (Ferminet), as long as the initial states ϕ^i−1\widehat{\phi}_{i-1} are smooth and positive. Then we apply a DMC algorithm from ti−1t_{i-1} to tit_i. More specifically, we minimize Ri(ψ^(ϑ))\mathcal{R}_i(\widehat{\psi}(\vartheta)) where

Ri(ψ^(ϑ))=⟨Hψ^(ϑ;⋅),ψ^(ϑ;⋅)⟩⟨ψ^(ϑ;⋅),ψ^(ϑ;⋅)⟩,\mathcal{R}_i(\widehat{\psi}(\vartheta))=\frac{\langle H\widehat{\psi}(\vartheta;\cdot),\widehat{\psi}(\vartheta;\cdot)\rangle}{\langle\widehat{\psi}(\vartheta;\cdot),\widehat{\psi}(\vartheta;\cdot)\rangle},

and where the corresponding gradient descent method is initialized with the solution to the DMC solution at time ti−1t_{i-1} (which differs from ϕ^i−1\widehat{\phi}_{i-1}). The converged solution is then denoted by ϕ^i−1\widehat{\phi}_{i-1}. Let us consider VMC/DMC on QiQ_i assuming that ψ^\widehat{\psi} (from DMC) is such that

ψ^(ϑ;⋅)=ϕ^i−1−dt2Hψ^(ϑ;⋅)−dt2Hϕ^i−1,\widehat{\psi}(\vartheta;\cdot)=\widehat{\phi}_{i-1}-\frac{dt}{2}H\widehat{\psi}(\vartheta;\cdot)-\frac{dt}{2}H\widehat{\phi}_{i-1},

and ⟨ϕ^i−1,ϕ^i−1⟩=1\langle\widehat{\phi}_{i-1},\widehat{\phi}_{i-1}\rangle= 1.

We have (for dtdt small enough)

⟨ψ(ϑ;⋅),ψ(ϑ;⋅)⟩=1−dt⟨ϕ^i−1,H(ψ(ϑ;⋅)+ϕ^i−1)⟩+dt24∥H(ψ(ϑ;⋅)+ϕ^i−1)∥2=1−dt⟨Hϕ^i−1,ϕ^i−1⟩−dt⟨ψ(ϑ;⋅),Hϕ^i−1⟩+O(dt2)=1−2dt⟨ψ(ϑ;⋅),Hϕ^i−1⟩+O(dt2).\begin{aligned} \langle\psi(\vartheta;\cdot),\psi(\vartheta;\cdot)\rangle &=1-dt\langle\widehat{\phi}_{i-1},H(\psi(\vartheta;\cdot)+\widehat{\phi}_{i-1})\rangle+\frac{dt^{2}}{4}\lVert H(\psi(\vartheta;\cdot)+\widehat{\phi}_{i-1})\rVert^{2}\\ &=1-dt\langle H\widehat{\phi}_{i-1},\widehat{\phi}_{i-1}\rangle-dt\langle\psi(\vartheta;\cdot),H\widehat{\phi}_{i-1}\rangle+O(dt^{2})\\ &=1-2dt\langle\psi(\vartheta;\cdot),H\widehat{\phi}_{i-1}\rangle+O(dt^{2}). \end{aligned}

Then ∇ϑ⟨ψ(ϑ;⋅),ψ(ϑ;⋅)⟩=−dt⟨∇ϑψ(ϑ;⋅),Hϕ^i−1⟩+O(dt2)\nabla_{\vartheta}\langle\psi(\vartheta;\cdot),\psi(\vartheta;\cdot)\rangle=-dt\langle\nabla_{\vartheta}\psi(\vartheta;\cdot),H\widehat{\phi}_{i-1}\rangle+O(dt^{2}). Hence

⟨Hψ(ϑ;⋅),ψ(ϑ;⋅)⟩⟨ψ(ϑ;⋅),ψ(ϑ;⋅)⟩=2dt⟨ϕ^i−1−ψ(ϑ;⋅),ψ(ϑ;⋅)⟩⟨ψ(ϑ;⋅),ψ(ϑ;⋅)⟩−⟨Hϕ^i−1,ψ(ϑ;⋅)⟩⟨ψ(ϑ;⋅),ψ(ϑ;⋅)⟩=2dt(⟨ϕ^i−1,ψ(ϑ;⋅)⟩⟨ψ(ϑ;⋅),ψ(ϑ;⋅)⟩−1)−⟨ψ(ϑ;⋅),Hϕ^i−1⟩⟨ψ(ϑ;⋅),ψ(ϑ;⋅)⟩=2dt(1−dt⟨ψ(ϑ;⋅),Hϕ^i−1⟩/2−dt⟨Hϕ^i−1,ϕ^i−1⟩/2⟨ψ(ϑ;⋅),ψ(ϑ;⋅)⟩−1)−⟨ψ(ϑ;⋅),Hϕ^i−1⟩⟨ψ(ϑ;⋅),ψ(ϑ;⋅)⟩=2dt(1⟨ψ(ϑ;⋅),ψ(ϑ;⋅)⟩−1)−2⟨ψ(ϑ;⋅),Hϕ^i−1⟩⟨ψ(ϑ;⋅),ψ(ϑ;⋅)⟩−⟨Hϕ^i−1,ϕ^i−1⟩⟨ψ(ϑ;⋅),ψ(ϑ;⋅)⟩.\begin{aligned} \frac{\langle H\psi(\vartheta;\cdot),\psi(\vartheta;\cdot)\rangle}{\langle\psi(\vartheta;\cdot),\psi(\vartheta;\cdot)\rangle} &=\frac{2}{dt}\frac{\langle\widehat{\phi}_{i-1}-\psi(\vartheta;\cdot),\psi(\vartheta;\cdot)\rangle}{\langle\psi(\vartheta;\cdot),\psi(\vartheta;\cdot)\rangle} -\frac{\langle H\widehat{\phi}_{i-1},\psi(\vartheta;\cdot)\rangle}{\langle\psi(\vartheta;\cdot),\psi(\vartheta;\cdot)\rangle}\\ &=\frac{2}{dt}\left(\frac{\langle\widehat{\phi}_{i-1},\psi(\vartheta;\cdot)\rangle}{\langle\psi(\vartheta;\cdot),\psi(\vartheta;\cdot)\rangle}-1\right) -\frac{\langle\psi(\vartheta;\cdot),H\widehat{\phi}_{i-1}\rangle}{\langle\psi(\vartheta;\cdot),\psi(\vartheta;\cdot)\rangle}\\ &=\frac{2}{dt}\left(\frac{1-dt\langle\psi(\vartheta;\cdot),H\widehat{\phi}_{i-1}\rangle/2-dt\langle H\widehat{\phi}_{i-1},\widehat{\phi}_{i-1}\rangle/2}{\langle\psi(\vartheta;\cdot),\psi(\vartheta;\cdot)\rangle}-1\right)\\ &\qquad-\frac{\langle\psi(\vartheta;\cdot),H\widehat{\phi}_{i-1}\rangle}{\langle\psi(\vartheta;\cdot),\psi(\vartheta;\cdot)\rangle}\\ &=\frac{2}{dt}\left(\frac{1}{\langle\psi(\vartheta;\cdot),\psi(\vartheta;\cdot)\rangle}-1\right) -2\frac{\langle\psi(\vartheta;\cdot),H\widehat{\phi}_{i-1}\rangle}{\langle\psi(\vartheta;\cdot),\psi(\vartheta;\cdot)\rangle} -\frac{\langle H\widehat{\phi}_{i-1},\widehat{\phi}_{i-1}\rangle}{\langle\psi(\vartheta;\cdot),\psi(\vartheta;\cdot)\rangle}. \end{aligned}

Then

⟨Hψ^(ϑ;⋅),ψ^(ϑ;⋅)⟩⟨ψ(ϑ;⋅),ψ(ϑ;⋅)⟩=4⟨ψ(ϑ;⋅),Hϕ^i−1⟩−2⟨ψ(ϑ;⋅),Hϕ^i−1⟩−⟨Hϕ^i−1,ϕ^i−1⟩+O(dt)=2⟨ψ^(ϑ;⋅),Hϕ^i−1⟩−⟨Hϕ^i−1,ϕ^i−1⟩+O(dt).\begin{aligned} \frac{\langle H\widehat{\psi}(\vartheta;\cdot),\widehat{\psi}(\vartheta;\cdot)\rangle}{\langle\psi(\vartheta;\cdot),\psi(\vartheta;\cdot)\rangle} &=4\langle\psi(\vartheta;\cdot),H\widehat{\phi}_{i-1}\rangle-2\langle\psi(\vartheta;\cdot),H\widehat{\phi}_{i-1}\rangle\\ &\qquad-\langle H\widehat{\phi}_{i-1},\widehat{\phi}_{i-1}\rangle+O(dt)\\ &=2\langle\widehat{\psi}(\vartheta;\cdot),H\widehat{\phi}_{i-1}\rangle-\langle H\widehat{\phi}_{i-1},\widehat{\phi}_{i-1}\rangle+O(dt). \end{aligned}

Hence the minimization of R\mathcal{R} is obtained from

∇ϑRi(ϑ)=2⟨∇ϑψ^(ϑ;⋅),Hϕ^i−1⟩+O(dt)=−2dt⟨∇ϑψ^(ϑ;⋅),ψ^(ϑ;⋅)−ϕ^i−1+dtHψ^(ϑ;⋅)⟩+O(dt).\begin{aligned} \nabla_{\vartheta}\mathcal{R}_{i}(\vartheta) &=2\langle\nabla_{\vartheta}\widehat{\psi}(\vartheta;\cdot),H\widehat{\phi}_{i-1}\rangle+O(dt)\\ &=-\frac{2}{dt}\langle\nabla_{\vartheta}\widehat{\psi}(\vartheta;\cdot),\widehat{\psi}(\vartheta;\cdot)-\widehat{\phi}_{i-1}+dtH\widehat{\psi}(\vartheta;\cdot)\rangle+O(dt). \end{aligned}

Thus we can state the following proposition.

Proposition 3.3. The gradient descent algorithm applied to N-VDMC where the continuous loss function Ri\mathcal{R}_{i} is approximated thanks to the learning nodes {Riα}α\{R_{i}^{\alpha}\}_{\alpha} on R3N\mathbb{R}^{3N}, reads as follows

ϑk+1=ϑk−2νkM∑α{∇ϑψ^(ϑk;Riα)[ψ^(ϑk;Riα)−ϕ^i−1(Riα)+dtHϕ^i−1(Riα)+O(dt2)]}.(21)\vartheta_{k+1}=\vartheta_{k}-\frac{2\nu_{k}}{M}\sum_{\alpha}\left\{\nabla_{\vartheta}\widehat{\psi}(\vartheta_{k};R_{i}^{\alpha})\left[\widehat{\psi}(\vartheta_{k};R_{i}^{\alpha})-\widehat{\phi}_{i-1}(R_{i}^{\alpha})+dtH\widehat{\phi}_{i-1}(R_{i}^{\alpha})+O(dt^{2})\right]\right\}. \tag*{(21)}

with learning rate νk\nu_{k}.

Conclusion

The above discussion allows to deduce that up to a O(dt2)O(dt^{2}), the PINN-DE and N-VDMC algorithms lead to the same optimization algorithms (20) and (21). In addition, by choosing ⟨⋅,⋅⟩ϕ^i−1\langle\cdot,\cdot\rangle_{\hat{\phi}_{i-1}} as the inner product (at iteration ii), still up to a O(dt2)O(dt^{2}) the PINN-ITM algorithm also leads to the same optimization algorithm, see (18).

Recall that the “direct” N-VDMC as described in § 1.4 (see [27]) and unlike the modified version used above, does not require to the use of VMC at each time iteration tit_i and as a consequence is much less computationally complex. Notice that PINN-like algorithms do not necessarily require the discretization of the time-derivative. Instead we can search for a neural network on QT=R3N×[0,T]Q_T = \mathbb{R}^{3N} \times[0,T] (for TT large enough) with f0=ψT2f_0 = \psi_T^2 given, for a neural network f^:(θ;R,t)∈Θ×Q→R\hat{f} : (\theta; R,t) \in\Theta\times Q \to\mathbb{R} (and designed such that f^:(θ;R,0)=f0(R)\hat{f} : (\theta; R,0) = f_0(R)) by minimizing

L(PINN)(θ)=∥∂tf^−∇R2f^+∇R(V0f^)−dtS0f^∥L2(Q)2.\mathcal{L}^{(\mathrm{PINN})}(\theta)=\left\|\partial_t\hat{f}-\nabla_R^2\hat{f}+\nabla_R(V_0\hat{f})-dtS_0\hat{f}\right\|_{L^2(Q)}^2 .

In principle, the computation is performed once for all over QQ, hence avoiding the use of N-VMC at intermediate times tit_i. In practice, it may still however be necessary to decompose the time-domain [0,T][0,T] in several subdomains ([Ti,Ti+1]([T_i,T_{i+1}] of length larger than dt)dt) used in DMC to avoid the computation of a null network.

Conclusion

NQS, Ferminet and PINN, Deep-Ritz have become very popular libraries and algorithms which were developed independently to each other in different fields of research. In this manuscript we have established non-trivial bridges between these different methodologies. We think that these connections may be beneficial for future developments in the different involved scientific communities. Naturally, PINN algorithms for solving (2) are more computationally complex than DMC-like algorithms, as they approximate PDE solutions, unlike DMC algorithms which only evaluate “trajectories”. In future works, we plan to develop efficient PINN-like algorithms for eigenvalue problems in the spirit of DMC algorithms. Data-driven PINN algorithms will be explored for high-dimensional quantum chemistry evolution problems (for instance in the spirit of [21]), such as laser-molecule interactions.

References

  1. [1]T. Alt, K. Schrader, M. Augustin, P. Peter, and J. Weickert. Connections between numerical algorithms for PDE and neural networks. Journal of Mathematical Imaging and Vision, 65(1):185–208, 2023.arxiv.org/abs/2107.14742
  2. [2]M. Anthony and P. L. Bartlett. Neural network learning: theoretical foundations. Cambridge University Press, Cambridge, 1999.
  3. [3]W. Bao and Y. Cai. Mathematical theory and numerical methods for Bose-Einstein condensation. Kinetic and Related Models, 6(1):1–135, 2013.DOI
  4. [4]W. Bao and Q. Du. Computing the ground state solution of Bose-Einstein condensates by a normalized gradient flow. SIAM Journal on Scientific Computing, 25(5):1674–1697, 2004.arxiv.org/abs/cond-mat/0303241
  5. [5]L. Bottou, F. E. Curtis, and J. Nocedal. Optimization methods for large-scale machine learning. SIAM Rev., 60(2):223–311, 2018.arxiv.org/abs/1606.04838
  6. [6]M. Caffarel, P. Del Moral, and L. de Montella. On the mathematical foundations of diffusion Monte Carlo, 2024.arxiv.org/abs/2402.04642
  7. [7]G Carlo, I. Cirac, K. Cranmer, L. Daudet, M. Schuld, N. Tishby, L. Vogt-Maranto, and L. Zdeborová. Machine learning and the physical sciences. Rev. Mod. Phys., 91:045002, Dec 2019.DOI
  8. [8]G. Carleo and M. Troyer. Solving the quantum many-body problem with artificial neural networks. Science, 355(6325):602–606, 2017.DOI
  9. [9]K. Choo, A. Mezzacapo, and G. Carleo. Fermionic neural-network states for ab-initio electronic struc ture. Nature Communications, 11(1), 2020.DOI
  10. [10]C. Cohen-Tannoudji, J. Dupont-Roc, and G. Grynberg. Atom-photon interactions. Wiley Interscience, N.Y., 1992.
  11. [11]G. Cybenko. Approximation by superpositions of a sigmoidal function. Math. Control. Signals Syst., 2(4):303–314, 1989.
  12. [12]T. De Ryck, A. D. Jagtap, and S. Mishra. Error estimates for physics-informed neural networks approximating the navier–stokes equations. IMA Journal of Numerical Analysis, 44(1):83–119, 2024.
  13. [13]T. De Ryck, S. Lanthaler, and S. Mishra. On the approximation of functions by tanh neural networks. Neural Networks, 143:732–750, 2021.DOI
  14. [14]Jiequn H., Linfeng Z., and Weinan E. Solving many-electron schrödinger equation using deep neural networks. J. of Comput. Phys., 399:108929, 2019.DOI
  15. [15]J. Hermann, Z. Schätzle, and F. Noé. Deep-neural-network solution of the electronic Schroedinger equation. Nature Chemistry, 12(10):891 – 897, 2020.
  16. [16]J. J. Hopfield. Neural networks and physical systems with emergent collective computational abilities. Proc. Nat. Acad. Sci. U.S.A., 79(8):2554–2558, 1982.
  17. [17]M. H. Kalos. Monte Carlo calculations of the ground state of three- and four-body nuclei. Phys. Rev., 128:1791–1795, Nov 1962.
  18. [18]I. Kosztin, B. Faber, and K. Schulten. Introduction to the diffusion Monte Carlo method. American Journal of Physics, 64(5):633–644, May 1996.DOI
  19. [19]I.E. Lagaris, A. Likas, and D.I. Fotiadis. Artificial neural networks for solving ordinary and partial differential equations. IEEE Transactions on Neural Networks, 9(5):987–1000, 1998.DOI
  20. [20]N. Metropolis and S. Ulam. The Monte Carlo method. Journal of the American Statistical Association, 44(247):335–341, 1949.
  21. [21]Jannes Nys, Zakari Denis, and Giuseppe Carleo. Real-time quantum dynamics of thermal states with neural thermofields, 2024.arxiv.org/abs/2309.07063
  22. [22]D. Pfau, J. S. Spencer, A. G. D. G. Matthews, and W. M. C. Foulkes. Ab initio solution of the many-electron Schroedinger equation with deep neural networks. Physical Review Research, 2(3), September 2020.
  23. [23]M. Raissi, P. Perdikaris, and G.E. Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707, 2019.DOI
  24. [24]A. Szabo and N.S. Ostlund. Modern Quantum Chemistry: Introduction to Advanced Electronic Structure Theory. Dover Books on Chemistry Series. Dover Publications, 1996.
  25. [25]E. Weinan and Bing Yu. The deep Ritz method: A deep learning-based numerical algorithm for solving variational problems. Communications in Mathematics and Statistics, 6(1):1 – 12, 2018.
  26. [26]L. Yang, D. Zhang, and G. E. Karniadakis. Physics-informed generative adversarial networks for stochastic differential equations. SIAM J. Sci. Comput., 42(1):A292–A317, 2020.arxiv.org/abs/1811.02033
  27. [27]A. Zen, S. Sorella, M. J. Gillan, A. Michaelides, and D. Alfè. Boosting the accuracy and speed of quantum Monte Carlo: Size consistency and time step. Physical Review B, 93(24), jun 2016.DOI

Paper details

vlnc:W_qfzeu479
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents