betaFeedback

WorksComputer Science

Neural Pfaffians: Solving Many Many-Electron Schrödinger Equations

Abstract

Neural wave functions accomplished unprecedented accuracies in approximating the ground state of many-electron systems, though at a high computational cost. Recent works proposed amortizing the cost by learning generalized wave functions across different structures and compounds instead of solving each problem independently. Enforcing the permutation antisymmetry of electrons in such generalized neural wave functions remained challenging as existing methods require discrete orbital selection via non-learnable hand-crafted algorithms. This work tackles the problem by defining overparametrized, fully learnable neural wave functions suitable for generalization across molecules. We achieve this by relying on Pfaffians rather than Slater determinants. The Pfaffian allows us to enforce the antisymmetry on arbitrary electronic systems without any constraint on electronic spin configurations or molecular structure. Our empirical evaluation finds that a single neural Pfaffian calculates the ground state and ionization energies with chemical accuracy across various systems. On the TinyMol dataset, we outperform the `gold-standard' CCSD(T) CBS reference energies by 1.9mEhE_h and reduce energy errors compared to previous generalized neural wave functions by up to an order of magnitude.

Introduction

Solving the electronic Schrödinger equation is at the heart of computational chemistry and drug discovery. Its solution provides a molecule’s or material’s electronic structure and energy [53]. While the exact solution is infeasible, neural networks have recently shown unprecedentedly accurate approximations [25]. These neural networks approximate the system’s ground-state wave function Ψ:RNe×3→R\Psi: \mathbb{R}^{N_{\mathrm{e}}\times3} \to\mathbb{R}, the lowest energy state, by minimizing the energy ⟨Ψ∣H^∣Ψ⟩\langle\Psi| \hat{H} | \Psi\rangle, where H^\hat{H} is the Hamiltonian operator, a mathematical description of the system. While such neural wave functions are highly accurate, training has proven computationally intensive.

Gao & Günnemann [14] have shown that training a generalized neural wave function on a large class of systems amortizes the cost. However, their approach is limited to different geometric arrangements of the same molecule. Subsequent works eliminated this limitation by introducing hand-crafted algorithms [15] or heavily relying on classical Hartree-Fock calculations [42]. Both impose strict, non-learnable mathematical constraints and prior assumptions that may not always hold, limiting their generalization and accuracies. Hand-crafted algorithms only work for a limited set of molecules, in particular organic molecules near equilibrium, while the reliance on Hartreee-Fock empirically results in degraded accuracies.

In this work, we propose the Neural Pfaffian (NeurPf) to overcome these limitations. As suggested by its name, NeurPf uses Pfaffians to define a superset of the previously used Slater determinants to enforce the fermionic antisymmetry. The Pfaffian lifts the constraint on the number of molecular orbitals from Slater determinants [47], enabling overparametrized wave functions

Preprint. Under review. with simpler and more accurate generalization. Compared to Globe [15], the absence of hand-crafted algorithms enables the modeling of non-equilibrium, ionized, or excited systems. By being fully learnable without fixed Hartree-Fock calculations like TAO [43], NeurPf achieves significantly lower variational energies. Our empirical results show that NeurPf can learn all second-row elements’ ground-state, ionization, and electron affinity potentials with a single wave function. Further, we demonstrate that NeurPf’s accuracy surpasses Globe on the challenging nitrogen dimer with seven times fewer parameters while not suffering from performance degradations when adding structures to the training set. On the TinyMol dataset, NeurPf surpasses the highly accurate reference CCSD(T) CBS energies on the small structures by 1.9 mEh and reduces errors compared to TAO by factors of 10 and 6 on the small and large structures, respectively.

Quantum chemistry

Quantum chemistry aims to solve the time-independent Schrödinger equation [11]

H^∣Ψ⟩=E∣Ψ⟩(1)\hat{H} \lvert\Psi\rangle= E \lvert\Psi\rangle \tag*{(1)}

where Ψ:RN↑×3×RN↓×3→R\Psi: \mathbb{R}^{N_{\uparrow} \times3} \times\mathbb{R}^{N_{\downarrow} \times3} \to\mathbb{R} is the electronic wave function for N↑N_{\uparrow} spin-up and N↓N_{\downarrow} spin-down electrons, H^\hat{H} is the Hamiltonian operator, and EE is the system’s energy. To ease notation, if not necessary, we omit spins in Ψ\Psi and treat it as Ψ:RNe×3→R\Psi: \mathbb{R}^{N_{\mathrm{e}} \times3} \to\mathbb{R} where Ne=N↑+N↓N_{\mathrm{e}} = N_{\uparrow} + N_{\downarrow}. The Hamiltonian H^\hat{H} for molecular systems, which we are concerned with in this work, is given by

H^=−12∑i=1Ne∑k=13∂2∂rik2+∑j>iNe1∥r⃗i−r⃗j∥−∑i=1Ne∑m=1NnZm∥r⃗i−R⃗m∥+∑n>mNnZmZn∥R⃗m−R⃗n∥(2)\hat{H} = -\frac{1}{2}\sum_{i=1}^{N_{\mathrm{e}}}\sum_{k=1}^{3}\frac{\partial^{2}}{\partial r_{ik}^{2}} + \sum_{j>i}^{N_{\mathrm{e}}}\frac{1}{\lVert\vec{r}_{i} - \vec{r}_{j} \rVert} - \sum_{i=1}^{N_{\mathrm{e}}}\sum_{m=1}^{N_{\mathrm{n}}}\frac{Z_{m}}{\lVert\vec{r}_{i} - \vec{R}_{m} \rVert} + \sum_{n>m}^{N_{\mathrm{n}}}\frac{Z_{m} Z_{n}}{\lVert\vec{R}_{m} - \vec{R}_{n} \rVert} \tag*{(2)}

with r⃗i∈R3\vec{r}_{i} \in\mathbb{R}^{3} being the iith electron’s position, and R⃗m∈R3\vec{R}_{m} \in\mathbb{R}^{3}, Zm∈N+Z_{m} \in\mathbb{N}_{+} being the mmth nucleus’ position and charge. The wave function Ψ\Psi describes the behavior of electrons in the system defined by the Hamiltonian H^\hat{H}. As the square of the wave function Ψ2\Psi^{2} is proportional to the probability density p(r⃗)∝Ψ2(r⃗)p(\vec{r}) \propto\Psi^{2}(\vec{r}) of finding the electrons at positions r⃗∈RNe×3\vec{r} \in\mathbb{R}^{N_{\mathrm{e}} \times3}, its integral must be finite:

∫Ψ(r⃗)2 dr⃗<∞.(3)\int\Psi(\vec{r})^{2} \,\mathrm{d}\vec{r} < \infty. \tag*{(3)}

Further, as electrons are indistinguishable half-spin fermionic particles, the wave function must be antisymmetric under any same-spin electron permutation τ\tau:

Ψ(τ↑(r⃗↑),τ↓(r⃗↓))=sgn⁡(τ↑)sgn⁡(τ↓)Ψ(r⃗).(4)\Psi\left(\tau^{\uparrow}(\vec{r}^{\uparrow}),\tau^{\downarrow}(\vec{r}^{\downarrow})\right) = \operatorname{sgn}(\tau^{\uparrow})\operatorname{sgn}(\tau^{\downarrow})\Psi(\vec{r}). \tag*{(4)}

To enforce this constraint, the wave function is typically defined as a so-called Slater determinant of N↑+N↓N_{\uparrow} + N_{\downarrow} integrable so-called orbital functions ϕi:R3→R\phi_i : \mathbb{R}^{3} \to\mathbb{R}:

ΨSlater(r⃗)=det⁡[ϕj↑(r⃗i↑)]det⁡[ϕj↓(r⃗i↓)]=det⁡Φ↑(r⃗↑)det⁡Φ↓(r⃗↓).(5)\Psi_{\mathrm{Slater}}(\vec{r}) = \det\left[\phi_{j}^{\uparrow}(\vec{r}_{i}^{\uparrow})\right]\det\left[\phi_{j}^{\downarrow}(\vec{r}_{i}^{\downarrow})\right] = \det\Phi^{\uparrow}(\vec{r}^{\uparrow})\det\Phi^{\downarrow}(\vec{r}^{\downarrow}). \tag*{(5)}

Note that for the determinant to exist, one needs exactly N↑N_{\uparrow} up and N↓N_{\downarrow} down orbitals ϕi↑\phi_{i}^{\uparrow} and ϕi↓\phi_{i}^{\downarrow}.

In linear algebra, Eq. (1) is an eigenvalue problem, where we look for the eigenfunction Ψ0\Psi_{0} with the lowest eigenvalue E0E_{0}. In Variational Monte Carlo (VMC), this is solved by applying the variational principle, which states that the energy of any trial wave function Ψ\Psi upper bounds E0E_{0}:

E0≤⟨Ψ∣H^∣Ψ⟩⟨Ψ2⟩=∫Ψ(r⃗)H^Ψ(r⃗) dr⃗∫Ψ2(r⃗) dr⃗.(6)E_{0} \leq\frac{\langle\Psi\lvert\hat{H} \rvert\Psi\rangle}{\langle\Psi^{2} \rangle} = \frac{\int\Psi(\vec{r}) \hat{H}\Psi(\vec{r}) \,\mathrm{d}\vec{r}}{\int\Psi^{2}(\vec{r}) \,\mathrm{d}\vec{r}}. \tag*{(6)}

By plugging in the probability distribution from Eq. (3), we can rewrite Eq. (6) as

E0≤Ep(r⃗)[Ψ−1(r⃗)H^Ψ(r⃗)]=Ep(r⃗)[EL(r⃗)],(7)E_{0} \leq\mathbb{E}_{p(\vec{r})}\left[\Psi^{-1}(\vec{r})\hat{H}\Psi(\vec{r})\right] = \mathbb{E}_{p(\vec{r})}\left[E_{\mathrm{L}}(\vec{r})\right], \tag*{(7)}

with EL(r⃗)=Ψ(r⃗)−1H^Ψ(r⃗)E_{\mathrm{L}}(\vec{r}) = \Psi(\vec{r})^{-1}\hat{H}\Psi(\vec{r}) being the so-called local energy. The right-hand side of Eq. (7) is known as the variational energy. As Eq. (7) does not require Ψ\Psi to be an analytic function, we can approximate the energy of any valid wave function Ψ\Psi with samples drawn from p(r⃗)p(\vec{r}). If we pick a parametrized family of wave functions Ψθ\Psi_\theta, we can optimize the parameters θ\theta to minimize the VMC energy by following the gradient of the variational energy

∇θ=Ep(r⃗)[(EL(r⃗)−Ep(r⃗)[EL(r⃗)])∇θlog⁡Ψθ(r⃗)](8)\nabla_\theta= \mathbb{E}_{p(\vec{r})}\left[\left(E_L(\vec{r})-\mathbb{E}_{p(\vec{r})}\left[E_L(\vec{r})\right]\right)\nabla_\theta\log\Psi_\theta(\vec{r})\right] \tag*{(8)}

where we approximate all expectations by Monte Carlo sampling [7].

Neural wave functions typically keep the functional form of Eq. (5) but replace the orbitals ϕi\phi_i with learned many-electron orbitals ϕiNN:R3×RNe×3→R\phi_i^{\mathrm{NN}}:\mathbb{R}^{3} \times\mathbb{R}^{N_e \times3} \to\mathbb{R} [25]. These many-electron orbitals ϕiNN\phi_i^{\mathrm{NN}} are implemented as different readouts of the same permutation-equivariant neural network. Multiplying each orbital by an envelope function χi:R3→R\chi_i:\mathbb{R}^{3} \to\mathbb{R} that decays exponentially to zero at large distances enforces the finite integral requirement in Eq. (3).

Generalized wave functions solve the more general problem where the nucleus positions R⃗\vec{R} and charges ZZ are not fixed. Since the Hamiltonian H^R⃗,Z\hat{H}_{\vec{R},Z} depends on the molecular structure (R⃗,Z)(\vec{R}, Z), so does the corresponding ground state wave function ΨR⃗,Z\Psi_{\vec{R},Z}. Note that we still work in the Born-Oppenheimer approximation, i.e., we treat the nuclei as classical point charges [53]. Given a dataset of molecular structures D={(R⃗1,Z1),…}\mathcal{D} = \{(\vec{R}_1, Z_1), \ldots\}, the total energy

∑(R⃗,Z)∈D⟨ΨR⃗,Z∣H^R⃗,Z∣ΨR⃗,Z⟩⟨ΨR⃗,Z∣ΨR⃗,Z⟩\sum_{(\vec{R},Z)\in\mathcal{D}} \frac{\langle\Psi_{\vec{R},Z}|\hat{H}_{\vec{R},Z}|\Psi_{\vec{R},Z}\rangle}{\langle\Psi_{\vec{R},Z}|\Psi_{\vec{R},Z}\rangle}

is minimized to approximate the ground state for each structure. Typically, the dependence on R⃗,Z\vec{R}, Z is implemented by using a meta network that takes R⃗,Z\vec{R}, Z as inputs and outputs the parameters of the electronic wave function [14].

Related work

While attempts to enforce the fermionic antisymmetry in neural wave functions in less than O(Ne3)O(N_e^3) operations promise faster runtime than Slater determinants, the accuracy of these methods is limited [23, 1, 40]. Pfau et al. [37] and Hermann et al. [24] established Slater determinants for neural wave functions by demonstrating chemical accuracy on small molecules. Note, Eq. (5) may also be written via a block-diagonal matrix, i.e., Ψ(r⃗)=det⁡(diag⁡(Φ↑,Φ↓))\Psi(\vec{r}) = \det\left(\operatorname{diag}(\Phi^\uparrow, \Phi^\downarrow)\right). Spencer et al. [46]’s implementation further increased accuracies by parametrizing the off diagonals that were implicitly set to 0 before, with additional orbitals Φ~\tilde{\Phi}:

ΨSlater(r⃗)=det⁡(Φ^(r⃗))=det⁡(Φ↑(r⃗)Φ~↑(r⃗)Φ~↓(r⃗)Φ↓(r⃗))(9)\Psi_{\mathrm{Slater}}(\vec{r}) = \det(\hat{\Phi}(\vec{r})) = \det\begin{pmatrix}\Phi^\uparrow(\vec{r}) & \tilde{\Phi}^\uparrow(\vec{r}) \\ \tilde{\Phi}^\downarrow(\vec{r}) & \Phi^\downarrow(\vec{r})\end{pmatrix} \tag*{(9)}

Several works confirmed the improved empirical accuracy of this approach [21, 31, 16, 17] (Ren et al. 2023). While later works refined the architecture to increase accuracy [48, 49, 50], the use of Slater determinants mostly remained a constant, with two notable exceptions: Firstly, Lou et al. (2023) use AGP wave functions [5, 6] to formulate the wave function as Ψ(r⃗)=det⁡(Φ↑)det⁡(Φ↓)=det⁡(Φ↑TΦ↓)\Psi(\vec{r}) = \det(\Phi^\uparrow) \det(\Phi^\downarrow) = \det(\Phi^{\uparrow\mathsf{T}}\Phi^\downarrow). This avoids picking exactly N↑/N↓N_\uparrow/N_\downarrow orbitals as Φ↑\Phi^\uparrow and Φ↓\Phi^\downarrow may be non-square but fails to generalize Eq. (9), we empirically verify the impact of this limitation in App. I. Secondly, Kim et al. (2023) introduced the combination of neural networks and Pfaffians, who demonstrated its performance on the ultra-cold Fermi gas. Though universal in theory, their parametrization yields no trivial adaption to molecular systems. In classical quantum chemistry, Bajdich et al. [2, 3] reported promising early results with Pfaffians in single-structure calculations for small molecules. In this work, we generalize Eq. (9) to Pfaffian wave functions that permit pretraining with Hartree-Fock calculations and generalization across molecules.

Generalized wave functions. Scherbela et al. [41] started this research with a weight-sharing scheme between wave functions. These still had to be reoptimized for each structure. Later, Gao & Günnemann (2022, 2023b) proposed PESNet, a generalized wave function for energy surfaces allowing joint training without reoptimization. Subsequent works extended PESNet to different compounds where the main challenge is parametrizing exactly N↑+N↓N_\uparrow+ N_\downarrow orbitals, such that the orbital matrix in Eq. (9) stays square. The problem of finding these orbitals was formulated into a discrete orbital selection problem. Gao & Günnemann [15]’s hand-crafted algorithm accomplishes this by selecting orbitals via a greedy nearest neighbor search. In contrast, Scherbela et al. [43, 42] use the lowest eigenvalues of the Fock matrix as a selection criteria. Both introduce non-learnable constraints, limiting generalization or sacrificing accuracy. NeurPf avoids the selection problem by introducing an overparameterization when enforcing the exchange antisymmetry.

Neural Pfaffian

Previous generalized wave functions build on Slater wave functions and attempt to adjust the orbitals ϕi\phi_i to the molecule. Slater determinants were chosen due to their previously demonstrated high accuracy. However, they require exactly N↑+N↓N_{\uparrow}+N_{\downarrow} orbitals. While the nuclei allow inferring the total number of electrons NeN_{\mathrm{e}} of any stable, singlet state system, the spin distribution into N↑N_{\uparrow} and N↓N_{\downarrow} orbitals per atom is not readily available. Previous works implement this via a discrete selection of orbitals via non-learnable prior assumptions and constraints on the wave function; see § 3.

Here, we present the Neural Pfaffian (NeurPf), a superset of Slater wave functions that preserves accuracy while relaxing the orbital number constraint. By not enforcing an exact number of orbitals, NeurPf is overparametrized with No≥max⁡{N↑,N↓}N_{\mathrm{o}} \geq\max\{N_{\uparrow},N_{\downarrow}\} orbitals, avoiding discrete selections and making it a natural choice for generalized wave functions. Importantly, NeurPf can be pretrained with Hartree-Fock, which accounts for >99%>99\% of the total energy [47]. We introduce NeurPf in four steps: (1) We introduce the Pfaffian and use it to define a superset of Slater wave functions. (2) We present memory-efficient envelopes that additionally accelerate convergence. (3) We introduce a new pretraining scheme for matching Pfaffian and Slater wave functions. (4) We discuss combining our developments to build a generalized wave function.

Pfaffian wave function

The Pfaffian of a skew-symmetric 2n×2n2n \times2n matrix AA, i.e., A=−ATA=-A^{T}, is defined as

Pf⁡(A)=12nn!∑τ∈S2nsgn⁡(τ)∏i=1nAτ(2i−1),τ(2i)(10)\operatorname{Pf}(A)=\frac{1}{2^{n}n!}\sum_{\tau\in S_{2n}}\operatorname{sgn}(\tau)\prod_{i=1}^{n}A_{\tau(2i-1),\tau(2i)} \tag*{(10)}

where S2nS_{2n} is the symmetric group of 2n2n elements. One may consider it a square root of the determinant of AA since Pf⁡(A)2=det⁡(A)\operatorname{Pf}(A)^{2}=\det(A). An important property of the Pfaffian is Pf⁡(BABT)=det⁡(B)Pf⁡(A)\operatorname{Pf}(BAB^{T})=\det(B)\operatorname{Pf}(A) for any invertible matrix BB and skew-symmetric matrix AA. In the context of neural wave functions, this means that if AA is an along both dimensions permutation equivariant function of the electron positions r⃗\vec{r}, A(τ(r⃗))=PτA(r⃗)PτTA(\tau(\vec{r}))=P_{\tau}A(\vec{r})P_{\tau}^{T}, the Pfaffian of AA is a valid wave function that fulfills the antisymmetry requirement from Eq. (4):

Ψ(τ(r⃗))=Pf⁡(A(τ(r⃗)))=Pf⁡(PτA(r⃗)PτT)=det⁡(Pτ)Pf⁡(A(r⃗))=sign⁡(τ)Ψ(r⃗).(11)\Psi(\tau(\vec{r}))=\operatorname{Pf}(A(\tau(\vec{r})))=\operatorname{Pf}(P_{\tau}A(\vec{r})P_{\tau}^{T})=\det(P_{\tau})\operatorname{Pf}(A(\vec{r}))=\operatorname{sign}(\tau)\Psi(\vec{r}). \tag*{(11)}

To compute the Pfaffian without evaluating the 2n!2n! terms in Eq. (10), we implement the Pfaffian via a tridiagonalization with the Householder transformation as in Wimmer [51].

There are various ways to construct AA [2, 3, 26]. Here, we introduce a superset of Slater wave functions, enabling high accuracy on molecular systems. If AA is a skew-symmetric matrix, so is BABTBAB^{T} for any arbitrary matrix BB. Thus, we can construct ΨPfaffian\Psi_{\mathrm{Pfaffian}} as

ΨPfaffian(r⃗)=1Pf⁡(APf)Pf⁡(Φ^Pf(r⃗)APfΦ^Pf(r⃗)T)(12)\Psi_{\mathrm{Pfaffian}}(\vec{r})=\frac{1}{\operatorname{Pf}(A_{\mathrm{Pf}})}\operatorname{Pf}\left(\hat{\Phi}_{\mathrm{Pf}}(\vec{r})A_{\mathrm{Pf}}\hat{\Phi}_{\mathrm{Pf}}(\vec{r})^{T}\right) \tag*{(12)}

where APf∈RNo×NoA_{\mathrm{Pf}}\in\mathbb{R}^{N_{\mathrm{o}}\times N_{\mathrm{o}}} is a learnable skew-symmetric matrix and Φ^Pf:RNe×3→RNe×No\hat{\Phi}_{\mathrm{Pf}}:\mathbb{R}^{N_{\mathrm{e}}\times3}\to\mathbb{R}^{N_{\mathrm{e}}\times N_{\mathrm{o}}} is a permutation equivariant function like in Eq. (9). This construction elevates the need for having exactly N↑/N↓N_{\uparrow}/N_{\downarrow} orbitals as in Slater determinants. We may now overparametrize the wave function with No≥max⁡{N↑,N↓}N_{\mathrm{o}}\geq\max\{N_{\uparrow},N_{\downarrow}\} orbitals, allowing for a more flexible and simpler implementation without needing discrete orbital selection. By choosing Φ^Pf=Φ^\hat{\Phi}_{\mathrm{Pf}}=\hat{\Phi}, it is straightforward to see that Eq. (12) is a superset of the Slater determinant wave function in Eq. (9). Note that, like in Eq. (9), we parametrize two sets of orbital functions ΦPf\Phi_{\mathrm{Pf}} and Φ^Pf\hat{\Phi}_{\mathrm{Pf}} and change their order for spin-down electrons to not enforce the exchange antisymmetry between different-spin electrons. As the normalizer Pf⁡(APf)\operatorname{Pf}(A_{\mathrm{Pf}}) is constant, we drop it going forward. As it is common in quantum chemistry [47, 24], we use linear combinations of wave functions to increase expressiveness:

ΨPfaffian(r⃗)=∑k=1NkckΨPfaffian,k(r⃗).(13)\Psi_{\mathrm{Pfaffian}}(\vec{r})=\sum_{k=1}^{N_{\mathrm{k}}}c_{k}\Psi_{\mathrm{Pfaffian},k}(\vec{r}). \tag*{(13)}

We visually compare the schematic of the Slater determinant and Pfaffian wave functions in Figure 1. In App. § 8, we discuss how to handle odd numbers of electrons such that Φ^PfAPfΦ^PfT\hat{\Phi}_{\mathrm{Pf}}A_{\mathrm{Pf}}\hat{\Phi}_{\mathrm{Pf}}^{T} has even dimensions. Like previous work [37], we parametrize the orbital functions ϕi\phi_i as a product of a permutation equivariant neural network h:R3×RN×3→RNfh : \mathbb{R}^3 \times\mathbb{R}^{N \times3} \to\mathbb{R}^{N_f} and an envelope function χ:R3→R\chi: \mathbb{R}^3 \to\mathbb{R}:

Figure 1. Schematic of the Slater determinant (1a) and our NeurPf (1b). Where the Slater formulation requires exactly NeN_e orbital functions, the Pfaffian formulation works for any number No≥max⁡{N↑,N↓}N_o \ge\max\{N_\uparrow, N_\downarrow\} of orbital functions, indicated by the rectangular orbital blocks.

ϕki(r⃗j∣r⃗)=χki(r⃗j)⋅h(r⃗j∣r⃗)Twki⋅ηkiN↑−N↓(14)\phi_{ki}(\vec{r}_j \mid\vec{r}) = \chi_{ki}(\vec{r}_j) \cdot h(\vec{r}_j \mid\vec{r})^{T} w_{ki} \cdot\eta_{ki}^{N_\uparrow-N_\downarrow} \tag*{(14)}

with wki∈RNfw_{ki} \in\mathbb{R}^{N_f} being a learnable weight vector, and ηkiN↑−N↓∈R\eta_{ki}^{N_\uparrow-N_\downarrow} \in\mathbb{R} being a scalar depending on the spin state of the system, i.e., the difference between the number of up and down electrons. The envelope function χ\chi ensures that the integral of the squared wave function is finite. For hh, we use Moon from Gao & Günnemann [15] thanks to its size consistency.

Memory-efficient envelopes

To satisfy the finite integral requirement on the square of Ψ\Psi in Eq. (3), the orbitals ϕ\phi are multiplied by an envelope function χ:R3→R\chi: \mathbb{R}^3 \to\mathbb{R} that exponentially decays to zero at large distances. We do not split spins here and work with Ne=N↑+N↓N_e = N_\uparrow+ N_\downarrow to simplify the discussion, but, in practice, we would split the envelopes into two sets, one for ΦPf\Phi_{\mathrm{Pf}} and one for Φ~Pf\widetilde{\Phi}_{\mathrm{Pf}}. The envelope function is typically a sum of exponentials centered on the nuclei [46]. In Einstein’s summation notation, the envelope function can be written as

χki(r⃗bj)=πkmi⏟Nk×Nn×No⋅exp⁡(−σkmi∥r⃗bj−R⃗m∥)bkmji⏟Nb×Nk×Nn×Ne×No(15)\chi_{ki}(\vec{r}_{bj}) = \underbrace{\pi_{kmi}}_{N_k \times N_n \times N_o} \cdot \underbrace{\exp\left(-\sigma_{kmi}\left\lVert\vec{r}_{bj} - \vec{R}_m \right\rVert\right)b_{kmji}}_{N_b \times N_k \times N_n \times N_e \times N_o} \tag*{(15)}

where NbN_b denotes the batch size. Empirically, we found the tensor on the right side containing many redundant entries. Further, due to the nonlinearity of the exponential function, one cannot implement the envelope in a simple matrix contraction but has to materialize the full five-dimensional tensor. NeurPf amplifies this problem as No≥NeN_o \ge N_e whereas Slater determinants constraint No=NeN_o = N_e.

We use a single set of exponentials per nucleus instead of having one for each combination of orbital and nucleus. This reduces the number of envelopes per electron from Nk×Nn×NoN_k \times N_n \times N_o to Nk×NenvN_k \times N_{\mathrm{env}}, where Nenv=Nn×Nenv/nucN_{\mathrm{env}} = N_n \times N_{\mathrm{env/nuc}} is the number of envelope functions. In general, we pick Nenv/nucN_{\mathrm{env/nuc}} such that Nenv≈NoN_{\mathrm{env}} \approx N_{\mathrm{o}}. These atomic envelopes are linearly recombined into molecular envelopes, effectively enlarging π\pi to a Nk×No×NenvN_k \times N_{\mathrm{o}} \times N_{\mathrm{env}} tensor. Thanks to these rearrangements, we avoid constructing a five-dimensional tensor. Instead, we define the envelopes as

χki(r⃗bj)=πkni⋅exp⁡(−σkn∥r⃗bj−R⃗n∥)kbnj.(16)\chi_{ki}(\vec{r}_{b_j}) = \pi_{kni} \cdot\exp\left(-\sigma_{kn}\left\|\vec{r}_{b_j} - \vec{R}_n\right\|\right)^{k b_{nj}}. \tag*{(16)}

Concurrently, Pfau et al. [38] presented similar bottleneck envelopes. However, we found ours to converge faster and not yield numerical instabilities. We discuss this further in App. § 9 and I.

Pretraining Pfaffian wave functions

Pretraining is essential in training neural wave functions and has frequently been observed to critically affect final energies [15, 48, 21]. The pretraining aims to find orbital functions close to the ground state to stabilize the optimization. Traditionally, this is done by matching the orbitals of the neural wave function to the orbitals of a baseline wave function, typically a Hartree-Fock wave function ΨHF=det⁡(ΦHF)\Psi_{\mathrm{HF}} = \det(\Phi_{\mathrm{HF}}), by solving

min⁡θ∥Φθ−ΦHF∥22,(17)\min_{\theta} \left\|\Phi_{\theta} - \Phi_{\mathrm{HF}}\right\|_2^2, \tag*{(17)}

for the neural network parameters θ\theta [37]. Since our Pfaffian has NoN_{\mathrm{o}} orbitals while Hartree-Fock has NeN_{\mathrm{e}}, we cannot directly apply this to our Pfaffian wave function. Further, as we predict orbitals per nucleus, our arbitrary orbital order may not align with Hartree-Fock.

We propose two alternative pretraining schemes for neural Pfaffian wave functions: one based on matching single-electron orbitals and one based on matching geminals, effectively two-electron orbitals. We need to expand the Hartree-Fock orbitals ΦHF\Phi^{\mathrm{HF}} to NoN_{\mathrm{o}} orbitals to match the single-electron orbitals directly. We construct ΦˉHF\bar{\Phi}^{\mathrm{HF}} by padding the extra No−NeN_{\mathrm{o}} - N_{\mathrm{e}} orbitals with zeros. It can easily be verified that the wave function ΨHF−Pf=1Pf⁡AHFPf⁡(ΦˉHFAˉHFΦˉHFT)\Psi_{\mathrm{HF-Pf}} = \frac{1}{\operatorname{Pf}A_{\mathrm{HF}}}\operatorname{Pf}(\bar{\Phi}_{\mathrm{HF}} \bar{A}_{\mathrm{HF}} \bar{\Phi}_{\mathrm{HF}}^{T}), is equivalent to the original Hartree-Fock wave function, i.e., ΨHF−Pf=ΨHF=det⁡(ΦHF)\Psi_{\mathrm{HF-Pf}} = \Psi_{\mathrm{HF}} = \det(\Phi_{\mathrm{HF}}) for any invertible skew-symmetric AHFA_{\mathrm{HF}}. Further, note that the multiplication of ΦˉHF\bar{\Phi}_{\mathrm{HF}} with any matrix T∈SO(No)T \in SO(N_{\mathrm{o}}) from the special orthogonal group does not change ΨHF−Pf\Psi_{\mathrm{HF-Pf}}. Thus, it suffices to match the single electron orbitals of Φ^Pf\hat{\Phi}_{\mathrm{Pf}} and ΦˉHF\bar{\Phi}_{\mathrm{HF}} up to a rotation T∈SO(No)T \in SO(N_{\mathrm{o}}), yielding the following optimization problem:

min⁡θmin⁡T∈SO(No)∥Φ^Pf−ΦˉHFT∥22.(18)\min_{\theta}\min_{T \in SO(N_{\mathrm{o}})} \left\|\hat{\Phi}_{\mathrm{Pf}} - \bar{\Phi}_{\mathrm{HF}}T\right\|_2^2. \tag*{(18)}

We solve this alternatingly for TT and θ\theta. To match the geminals Φ^PfAPfΦ^PfT\hat{\Phi}_{\mathrm{Pf}}A_{\mathrm{Pf}}\hat{\Phi}_{\mathrm{Pf}}^{T} and ΦHFAHFΦHFT\Phi_{\mathrm{HF}}A_{\mathrm{HF}}\Phi_{\mathrm{HF}}^{T}, we have to account for the fact that the choice of AHFA_{\mathrm{HF}} is arbitrary as long as it is skew-symmetric and invertible. Again, we solve this optimization problem alternatingly by solving for AHF∈S={A∈SO(Ne):A=−AT}A_{\mathrm{HF}} \in\mathcal{S} = \{A \in SO(N_{\mathrm{e}}) : A = -A^{T}\} and θ\theta:

min⁡θmin⁡AHF∈S∥Φ^PfAPfΦ^PfT−ΦHFAHFΦHFT∥22.(19)\min_{\theta}\min_{A_{\mathrm{HF}} \in\mathcal{S}} \left\|\hat{\Phi}_{\mathrm{Pf}}A_{\mathrm{Pf}}\hat{\Phi}_{\mathrm{Pf}}^{T} - \Phi_{\mathrm{HF}}A_{\mathrm{HF}}\Phi_{\mathrm{HF}}^{T}\right\|_2^2. \tag*{(19)}

While both formulations share the same minimizer, combining both yields the most stable results. We hypothesize that this is because the single-electron orbitals are more stable than the geminals and thus provide a better starting point for the optimization. In contrast, the latter provides a closer formulation of the neural network orbitals. Thus, we pretrain our neural Pfaffian wave functions by solving the optimization problem

min⁡θ(αmin⁡T∈SO(No)∥Φ^Pf−ΦˉHFT∥22+βmin⁡AHF∈S∥Φ^PfAPfΦ^PfT−ΦHFAHFΦHFT∥22)(20)\min_{\theta}\left(\alpha\min_{T \in SO(N_{\mathrm{o}})}\left\|\hat{\Phi}_{\mathrm{Pf}}-\bar{\Phi}_{\mathrm{HF}}T\right\|_2^2+\beta\min_{A_{\mathrm{HF}}\in\mathcal{S}}\left\|\hat{\Phi}_{\mathrm{Pf}}A_{\mathrm{Pf}}\hat{\Phi}_{\mathrm{Pf}}^{T}-\Phi_{\mathrm{HF}}A_{\mathrm{HF}}\Phi_{\mathrm{HF}}^{T}\right\|_2^2\right) \tag*{(20)}

with weights α,β∈[0,1]\alpha,\beta\in[0,1]. To optimize over the special orthogonal group SO(No)SO(N_{\mathrm{o}}), we use the Cayley transform [13]. App. § 10 further details the procedure.

Generalizing over systems

We now focus on generalizing the construction of our Pfaffian wave function for different systems. We accomplish the generalization similar to PESNet [14] by introducing a second neural network, the MetaGNN M:(R3×R+)Nn→Θ\mathcal{M} : (\mathbb{R}^3 \times\mathbb{R}_{+})^{N_{\mathrm{n}}} \to\Theta that acts upon the molecular structure, i.e., nuclei positions and charges, and parametrizes the electronic wave function ΨPfaffian:RNe×3×Θ→R\Psi_{\mathrm{Pfaffian}} : \mathbb{R}^{N_{\mathrm{e}} \times3} \times\Theta\to\mathbb{R} for the system of interest. As architecture for the wave function and MetaGNN, we use the same architecture as in Gao et al. [15] with the exception being that we replace the Slater determinant with the Pfaffian as described in § 4 and minor tweaks highlighted in App. § 11.§ 4.

Pfaffian. To represent wave functions of different systems within a single NeurPf, we need to adapt the orbitals Φ^Pf\hat{\Phi}_{\mathrm{Pf}} and antisymmetrizer APfA_{\mathrm{Pf}} from Eq. (12) to the molecule. In doing so, we must ensure No≥max⁡{N↑,N↓}N_{\mathrm{o}} \ge\max\{N_{\uparrow}, N_{\downarrow}\}. Otherwise, Φ^PfAPfΦ^PfT\hat{\Phi}_{\mathrm{Pf}}A_{\mathrm{Pf}}\hat{\Phi}_{\mathrm{Pf}}^{T} is singular, and the wave function is zero. One may solve this by picking NoN_{\mathrm{o}} large enough that No≥max⁡{N↑,N↓}N_{\mathrm{o}} \ge\max\{N_{\uparrow}, N_{\downarrow}\} for all molecules in the dataset. However, this is computationally expensive, does not reuse known orbitals in the problem, and simply moves the problem to even larger systems. Instead, we grow the number of orbitals NoN_{\mathrm{o}} with the system size by defining Norb/nucN_{\mathrm{orb/nuc}} orbitals per nucleus, as depicted in Figure 2. This allows us to transfer orbitals from smaller systems to larger systems. We only need to ensure that Norb/nucN_{\mathrm{orb/nuc}} is larger than half the maximum number of electrons in a period, e.g., for the first period Norb/nuc≥1N_{\mathrm{orb/nuc}} \ge1, for the second period Norb/nuc≥5N_{\mathrm{orb/nuc}} \ge5.

Figure 2. Orbital parametrization per nucleus. ∘\circ, ∘\circ, ∘\circ indicate electrons and nuclei, respectively.

The projection W\mathbf{W} from Eq. (14) and the envelope decays σ\boldsymbol{\sigma} are parametrized by node embeddings, while the envelope weights π\boldsymbol{\pi} and the antisymmetrizer APfA_{\mathrm{Pf}} are derived from edge embeddings. We predict a Norb/nuc×NfN_{\mathrm{orb/nuc}} \times N_{\mathrm{f}} matrix per nucleus for W\mathbf{W} and a Nenv/nucN_{\mathrm{env/nuc}} vector per nucleus for σ\boldsymbol{\sigma}. For the edge parameters π\boldsymbol{\pi} and APfA_{\mathrm{Pf}}, we predict a Nenv/nuc×Norb/nucN_{\mathrm{env/nuc}} \times N_{\mathrm{orb/nuc}} and a Norb/nuc×Norb/nucN_{\mathrm{orb/nuc}} \times N_{\mathrm{orb/nuc}} matrix per edge, respectively. These are concatenated into the Nenv×NoN_{\mathrm{env}} \times N_{\mathrm{o}} and No×NoN_{\mathrm{o}} \times N_{\mathrm{o}} matrices π\boldsymbol{\pi} and A^Pf\hat{A}_{\mathrm{Pf}}. The latter is antisymmetrized to get APf=12(A^Pf−A^PfT)A_{\mathrm{Pf}} = \frac{1}{2}(\hat{A}_{\mathrm{Pf}} - \hat{A}_{\mathrm{Pf}}^{T}). We parametrize the spin-dependent scalars η\eta as node outputs for a fixed number of spin configurations NsN_{\mathrm{s}}. Because the change in spin configuration does not grow with system size, NsN_{\mathrm{s}} is fixed. We generate two sets of these parameters, one for ΦPf\Phi_{\mathrm{Pf}} and one for Φ~Pf\widetilde{\Phi}_{\mathrm{Pf}}. App. D provides definitions for the wave function, the MetaGNN, and the parametrization.

Pretraining. Previous work like Gao & Günnemann [15] needed to canonicalize the Hartree-Fock solutions for different systems before pretraining to ensure that the orbitals fit the neural network. Alternatively, Scherbela et al. [42] relied on traditional quantum chemistry methods like Foster & Boys [10]’s localization to canonicalize their orbitals in conjunction with sign equivariant neural networks. In contrast, we ensure that the transformed Hartree-Fock orbitals are similar across structures as we optimize T∈SO(No)T \in SO(N_{\mathrm{o}}) and AHF∈SA_{\mathrm{HF}} \in\mathcal{S} for each structure separately, which simultaneously also accounts for arbitrary rotations and flips of the orbitals produced by Hartree-Fock.

Limitations. While our Pfaffian-based generalized wave function significantly improves accuracy on organic chemistry, we leave the transfer to periodic systems for future work ([28]). Further, due to the lack of low-level hardware/software support for the Pfaffian and the increased number of orbitals No≥max⁡{N↑,N↓}N_{\mathrm{o}} \ge\max\{N_{\uparrow}, N_{\downarrow}\}, our Pfaffian is slower than a comparably-sized Slater determinant. While we solve the issue of enforcing the fermionic antisymmetry, our neural wave functions are still unaware of any symmetries of the wave function itself. These are challenging to describe and largely unknown, but their integration may improve generalization performance ([44]). Finally, in classical single-structure calculations, NeurPf may not improve accuracies. App. P discusses the broader impact of our work.

Experiments

In the following, we evaluate NeurPf on several atomic and molecular systems by comparing it to Globe (Gao & Günnemann, [15]) and TAO (Scherbela et al., [43]). Concretely, we investigate the following: (1) Second-row elements and their ionization potentials and electron affinities. Globe cannot compute these due to its restriction to singlet state systems. (2) The challenging nitrogen potential energy surface where Globe significantly degraded performance when enlarging their training set with additional molecules. (3) The TinyMol dataset (Scherbela et al., [43]) to evaluate NeurPf’s generalization capabilities across biochemical molecules. In interpreting the following results, one should mind the variational principle, i.e., lower energies are better for neural wave functions. Further, 1 kcal mol−1≈1.6 mEh1\ \mathrm{kcal\ mol}^{-1} \approx1.6\ \mathrm{m}E_{\mathrm{h}} is the typical threshold for chemical accuracy.

Like previous work, we optimize the neural wave function using the VMC framework from Sec. 2. We precondition the gradient with the Spring optimizer (Goldshlager et al., [22]). App. E details the setup further. App. F, I and J show an experiment on extensity and additional ablations.

Atomic systems and spin configurations

We evaluate NeurPf on second-row elements and their ionization potentials and electron affinities. These systems are particularly interesting as they represent

a wide range of spin configurations. We cannot use Globe on such systems because they differ from the singlet state assumption. Instead, we compare our results to the single-structure calculations from Pfau et al. [37]’s FermiNet and the exact results from Chakravorty et al. [8]; Klopper et al. [27]. In App. G, we repeat this experiment for metals.

Figure 3 displays the ground state energy, electron affinity, and ionization potential errors of NeurPf during training compared to the reference energies from Pfau et al. [37]; Chakravorty et al. [8]; Klopper et al. [27]. It is apparent that NeurPf reaches chemical accuracy relative to the exact results while only training a single neural network for all systems. While separately optimized FermiNets may achieve lower errors, Pfau et al. [37] trained 21 neural networks for 200k steps each compared to a single NeurPf trained for 200k steps, i.e., 21 times fewer steps and samples. Whereas Gao & Günnemann [15]; Scherbela et al. [42] focus on singlet state systems or stable biochemical molecules, NeurPf demonstrates that a generalized wave function need not be restricted to such simple systems and can even generalize to a wide range of electronic configurations.

Figure 3. Ground state, electron affinity, and ionization potential errors of second-row elements during training. A single NeurPf has been trained on all systems jointly while references [37] were calculated separately for each system. Energies are averaged over the last 10% of steps.

Effect of uncorrelated data. Next, we evaluate NeurPf on the nitrogen potential energy surface, a traditionally challenging system due to its high electron correlation effects [33]. This is particularly interesting as Gao & Günnemann [15] observed a significant accuracy degradation when reformulating their wave function to generalize over different systems. In particular, they found that training only on the nitrogen dimer leads to significantly lower errors than training with an ethene-augmented dataset, indicating an accuracy penalty in generalization. We replicate their setup and compare the performance of NeurPf trained on the nitrogen energy surface with and without additional ethene structures. Like Gao & Günnemann [15], the nitrogen structures are taken from Pfau et al. [37] and the ethene structures from Scherbela et al. [41]. As additional references, we plot Gao & Günnemann [14]’s PESNet and Fu et al. (2023)’s FermiNet results.

Figure 4 shows the error potential energy surface relative to the experimental results from Le Roy et al. [29]. NeurPf reduces the average error on the energy surface from Globe’s 2.7 mEhmE_h to 2 mEhmE_h when training solely on nitrogen structures. When adding the ethene structures, Globe’s error increases to 5.3 mEhmE_h while NeurPf’s error stays constant at 2 mEhmE_h, a lower error than the Globe without the augmented dataset. These results indicate NeurPf’s strong capabilities in approximating ground states while allowing for generalization across different systems without a significant loss in accuracy.

Potential energy surface of nitrogen

Figure 4. Potential energy surface of nitrogen. Energies are relative to Le Roy et al. [29].

TinyMol dataset. Finally, we look at learning a generalized wave function over different molecules and structures. We use the TinyMol dataset [43], consisting of a small and large dataset. The dataset includes ‘gold-standard’ CCSD(T) CBS energies. The small set consists of 3 molecules with 2 heavy atoms, while the large set covers 4 molecules with 3 heavy atoms. For each molecule, 10 structures are provided. Here, we compare again both Globe (+Moon) and TAO to NeurPf. All models are directly trained on the small and large test sets.

Figure 5 shows the mean energy difference to CCSD(T) at different stages of the training. We refer to App. § 18 for a per molecule error attribution. It is apparent that NeurPf yields lower errors than the TAO and Globe after at least 500 steps. On the small structures, NeurPf even matches the CCSD(T) baseline after 16k steps and achieves 1.9 mEh lower energies after 32k steps. Since VMC methods are variational, i.e., lower energies are always better, NeurPf is more accurate than the CCSD(T) CBS reference. Compared to TAO and Globe, NeurPf reports 5.9 mEh and 11.3 mEh lower energies, respectively. On the large structures, we observe a similar pattern where we find NeurPf having a 25 times smaller error than TAO during the early stages of training and reaching 21.1 mEh lower energies after 32k steps – a 6 times lower error compared to the CCSD(T) baseline. Note that since the CCSD(T) (CBS) energies are neither exact nor variational, the true error to the ground state is unknown. Still, we provide additional numbers for a NeurPf trained for 128k steps in App. § 18. There, we find NeurPf yielding 4.4 mEh lower energies on the large structures.

Figure 5. Convergence of mean energy difference on the TinyMol dataset from Scherbela et al. [43]. The y-axis is linear < 1 and logarithmic ≥ 1. Due to the variational principle, NeurPf is better than the reference CCSD(T) on the small molecules.

These results show that a generalized wave function can achieve high accuracy on various molecular structures without pretraining when not relying on hand-crafted algorithms or Hartree-Fock calculations. For additional experiments, we refer the reader to App. § 19 where we first pretrain TAO and NeurPf on a separate training set and, then, finetune on the small and large test sets and App. § 20 for a comparison of joint and separate optimization.

Conclusion

In this work, we established a new way of parametrizing neural network wave functions for generalization across molecules via overparameterization with Pfaffians. Our Neural Pfaffian is more accurate, simpler to implement, fully learnable, and applicable to any molecular system compared to previous work. The wave function changes smoothly with the structure, avoiding the discrete orbital selection problem previously solved via hand-crafted algorithms or Hartree-Fock. Additionally, we introduced a memory-efficient implementation of the exponential envelopes, reducing memory requirements while accelerating convergence. Further, we presented a pretraining scheme for Pfaffians enabling initialization with Hartree-Fock – a crucial step for molecular systems. Our experimental evaluation demonstrated that our Neural Pfaffian can generalize across different ionizations of various systems, stay accurate when enlarging datasets, and set a new state of the art by outperforming previous neural wave functions and the reference CCSD(T) CBS on the TinyMol dataset. These developments open the door for new neural wave functions applications, e.g., to generate reference data for machine-learning force fields or density functional theory [9, 19].

Acknowledgments

Acknowledgments. We greatly thank Simon Geisler for our valuable discussions. Further, we thank Valerie Engelmayer, Leo Schwinn, and Aman Saxena for their invaluable feedback on the manuscript. Funded by the Federal Ministry of Education and Research (BMBF) and the Free State of Bavaria under the Excellence Strategy of the Federal Government and the Länder.

Odd numbers of electrons

To handle odd numbers of electrons, we extend the electron pair matrix Φ^PfAPfΦ^PfT\hat{\Phi}_{\mathrm{PfAPf}}\hat{\Phi}_{\mathrm{Pf}}^{T} to even dimensions. We accomplish this by augmenting Φ^PfAPfΦ^PfT\hat{\Phi}_{\mathrm{PfAPf}}\hat{\Phi}_{\mathrm{Pf}}^{T} with a learnable single-electron orbital ϕodd\phi_{\mathrm{odd}} to

Φ^PfAPfΦ^PfT=(Φ^PfAPfΦ^PfTϕodd−ϕoddT0).(21)\hat{\Phi}_{\mathrm{PfAPf}}\hat{\Phi}_{\mathrm{Pf}}^{T} = \begin{pmatrix} \hat{\Phi}_{\mathrm{PfAPf}}\hat{\Phi}_{\mathrm{Pf}}^{T} & \phi_{\mathrm{odd}} \\ -\phi_{\mathrm{odd}}^{T} & 0 \end{pmatrix}. \tag*{(21)}

To obtain a single additional orbital for the whole molecule, we parameterize one orbital ϕodd,m\phi_{\mathrm{odd},m} for each nucleus as in Eq. (14) and sum them up to obtain ϕodd=∑m=1Nnϕodd,m\phi_{\mathrm{odd}} = \sum_{m=1}^{N_n} \phi_{\mathrm{odd},m}.

Difference to bottleneck envelopes

Similar to the bottleneck envelope from Pfau et al. [38], our efficient envelopes aim at reducing memory requirements. The bottleneck envelopes are defined as

χbottleneckk(rbi)j=∑l=1Lwjlk∑m=1Nnπlmexp⁡(−σlm∥ri−Rm∥)(22)\chi_{\mathrm{bottleneck}}^{k}(r_{bi})_j = \sum_{l=1}^{L} w_{jl}^{k} \sum_{m=1}^{N_n} \pi_{lm} \exp\left(-\sigma_{lm}\lVert r_i - R_m\rVert\right) \tag*{(22)}

While both methods share the idea of reducing the number of parameters, they differ in their implementation. Whereas the bottleneck envelopes construct a full set of LL many-nuclei envelopes and then linearly recombine these to the final envelopes for each of K×NoK \times N_o orbitals, our efficient envelopes construct the final envelopes directly from a set of single-nuclei exponentials. Further, we use a different set of basis functions for each of the KK determinants. In terms of computational complexity, the bottleneck envelopes require O(NeNnL)+O(KLNeNo)O(N_eN_nL) + O(KLN_eN_o) operations to compute the envelopes, while our efficient envelopes require O(KNenvNeNo)O(KN_{\mathrm{env}}N_eN_o) operations. In practice, we found our efficient envelopes to be faster and converge better on all systems we tested. An ablation study is presented in App. § 16. Further, we observed no numerical instabilities in our envelopes as reported by Pfau et al. [38].

Compared to the full envelopes, we find our memory efficient ones to be slower but yielding better performance. This is likely due to the increased number of wave function parameters. The number of parameters for the full envelopes and our memory efficient envelopes is shown in Table 1 for an example with Ne=No=20N_e = N_o = 20, Nn=5N_n = 5, Nd=16N_d = 16, Natomenv=8N_{\mathrm{atom}}^{\mathrm{env}} = 8. The full envelopes’ σ\sigma, π\pi scale both O(NdNnNo)O(N_dN_nN_o) while our memory efficient envelopes’ σ\sigma scales O(NdNnNenv/nuc)O(N_dN_nN_{\mathrm{env}/\mathrm{nuc}}) and π\pi scales O(NdNnNenv/nucNo)O(N_dN_nN_{\mathrm{env}/\mathrm{nuc}}N_o). In runtime, the full envelopes require O(NdNnNeNo)O(N_dN_nN_eN_o) operations, while our memory efficient envelopes require O(NdNnNatomenvNeNo)O(N_dN_nN_{\mathrm{atom}}^{\mathrm{env}}N_eN_o) operations. In memory complexity, the full envelopes require O(NdNnNe2)O(N_dN_nN_e^2), while our memory efficient envelopes require O(NdNnNenvenvNatomNe)O(N_dN_n\frac{N_{\mathrm{env}}^{\mathrm{env}}}{N_{\mathrm{atom}}}N_e).

σ\sigmaπ\piTotal
full160016003200
our6401280013400

Table 1. Number of envelope parameters for the full envelope and our memory efficient envelopes for an explanatory system.

Pretraining

To pretrain NeurPf, we solve the optimization problem from Eq. (20). The nested optimization problems are solved iteratively, where we first solve for T∈SO(N)T \in SO(N) and AHF∈SA_{\mathrm{HF}} \in S and then for the parameters of the wave function θ\theta. We describe how we parametrize the special orthogonal group SO(N)SO(N) and the antisymmetric special orthogonal group SS and then how we solve the optimization problems.

To optimize over the special orthogonal group SO(N)SO(N), we parametrize TT via some arbitrary matrix T~∈RN×N\widetilde{T} \in\mathbb{R}^{N \times N}. Next, we obtain an antisymmetrized version of T~\widetilde{T} via

T^=12(T~−T~T).(23)\hat{T} = \frac{1}{2}\left(\widetilde{T} - \widetilde{T}^{T}\right). \tag*{(23)}

We now may use T^\widehat{T} with the Cayley transform to obtain a special orthogonal matrix

T‾=(T^−I)−1(T^+I)(24)\overline{T}=\left(\widehat{T}-I\right)^{-1}\left(\widehat{T}+I\right) \tag*{(24)}

where II is the identity matrix. T‾\overline{T} is now a special orthogonal matrix where all eigenvalues are 11. To parametrize matrices with an even number of eigenvalues −1-1 as well, we simply multiply T‾\overline{T} with itself:

T=T‾T‾(25)T=\overline{T}\overline{T} \tag*{(25)}

which gives us our final parametrization of the special orthogonal group SO(N)SO(N) [13].

We follow Gallier [13], to parametrize antisymmetric special orthogonal matrices S\mathbb{S}. In particular, we parametrize some TT using the procedure outlined above. To parametrize AHFA_{\mathrm{HF}}, it remains to antisymmetrize TT while preserving the special orthogonal property. We accomplish this by defining

AHF=TI~TT(26)A_{\mathrm{HF}}=T\widetilde{I}T^{T} \tag*{(26)}

where

I~=diag⁡([01−10],…)=[0100⋯−1000000100−10⋮⋱](27)\widetilde{I}=\operatorname{diag}\left(\begin{bmatrix}0 & 1 \\ -1 & 0\end{bmatrix},\ldots\right)= \begin{bmatrix} 0 & 1 & 0 & 0 & \cdots\\ -1 & 0 & 0 & 0 & \\ 0 & 0 & 0 & 1 & \\ 0 & 0 & -1 & 0 & \\ \vdots& & & & \ddots \end{bmatrix} \tag*{(27)}

is the antisymmetric identity matrix. Since the product of special orthogonal matrices is special orthogonal and BABTBAB^{T} yielding an antisymmetric matrix for any special orthogonal matrix BB, we have that AHF∈SA_{\mathrm{HF}}\in\mathbb{S} is an antisymmetric special orthogonal matrix.

Now that we can parametrize both groups with real matrices, we can simplify the optimization problem by performing gradient optimization for both TT, AHFA_{\mathrm{HF}}, and θ\theta. We solve this problem alternatively, where we first solve for TT and AHFA_{\mathrm{HF}} by doing NpreN_{\mathrm{pre}} steps of gradient optimization with the prodigy optimizer [35] and then perform a single outer step on θ\theta with the lamb optimizer [52] like previous works [14, 48].

Model architectures

We largely reuse the same architecture for the MetaGNN M:(R3×N+)Nn→Θ\mathcal{M}:(\mathbb{R}^{3}\times\mathbb{N}_{+})^{N_{\mathrm{n}}}\to\Theta and wave function ΨPfaffian:RNe×3×Θ→R\Psi_{\mathrm{Pfaffian}}:\mathbb{R}^{N_{\mathrm{e}}\times3}\times\Theta\to\mathbb{R} as Gao & Günnemann [15]. We canonicalize all molecular structures using the equivariant coordinate frame from Gao & Günnemann [14].

Wave function

Similar to Gao & Günnemann [15], we use bars above functions and parameters to indicate that the MetaGNN M\mathcal{M} parameterizes these and that they vary by structure. We define our wave function as a Jastrow-Pfaffian wave function like Kim et al. [26]:

Ψ(r)=exp⁡(J(r))∑k=1KckPf⁡(Φ^Pfk(r)APfΦ^Pfk(r)T).(28)\Psi(\mathbf{r})=\exp\left(J(\mathbf{r})\right)\sum_{k=1}^{K}c_{k}\operatorname{Pf}\left(\widehat{\Phi}_{\mathrm{Pf}}^{k}(\mathbf{r})A_{\mathrm{Pf}}\widehat{\Phi}_{\mathrm{Pf}}^{k}(\mathbf{r})^{T}\right). \tag*{(28)}

As Jastrow factor J:RN×3→RJ:\mathbb{R}^{N\times3}\to\mathbb{R} we use a linear combination of a learnable MLP of electron embeddings and the fixed electronic cusp Jastrow from von Glehn et al. [48]:

J(r)=∑i=1NMLP⁡(h(r⃗i∣r⃗))+βpar∑i,j;αi=αj−14αpar2αpar+∥ri−rj∥+βanti∑i,j;αi≠αj−12αanti2αanti+∥ri−rj∥(29)\begin{aligned} J(\mathbf{r})={}&\sum_{i=1}^{N}\operatorname{MLP}\left(h(\vec{r}_{i}\mid\vec{r})\right)\\ &+\beta_{\mathrm{par}}\sum_{i,j;\alpha_{i}=\alpha_{j}}-\frac{1}{4}\frac{\alpha_{\mathrm{par}}^{2}}{\alpha_{\mathrm{par}}+\lVert r_{i}-r_{j}\rVert}\\ &+\beta_{\mathrm{anti}}\sum_{i,j;\alpha_{i}\ne\alpha_{j}}-\frac{1}{2}\frac{\alpha_{\mathrm{anti}}^{2}}{\alpha_{\mathrm{anti}}+\lVert r_{i}-r_{j}\rVert} \tag*{(29)} \end{aligned}

where h:R3×RN×3→RNfh:\mathbb{R}^{3}\times\mathbb{R}^{N\times3}\to\mathbb{R}^{N_f} is the iith output of the permutation equivariant neural network, implemented via the Molecular orbital network (Moon) Gao & Günnemann, [15], βpar,βanti,αpar,αanti∈R\beta_{\mathrm{par}},\beta_{\mathrm{anti}},\alpha_{\mathrm{par}},\alpha_{\mathrm{anti}}\in\mathbb{R} are learnable scalars, and αi\alpha_i is the spin of the iith electron.

The orbitals Φ^Pf\hat{\Phi}_{P_f} are defined as in Eq. (14) with Moon performing the following steps: We start with constructing electron embeddings based on electron-electron distances and then proceed to aggregate these embeddings to the orbitals. The nuclei are updated through MLPs and finally diffused to the electrons, yielding the final electron embeddings.

The initial embedding hi(0)h_i^{(0)} of the iith electron is constructed as

hi(0)=1μ(r⃗i)(∑j=1Nσ(gije−eWδαiαj)∘Γαiαj(∥r⃗i−r⃗j∥))W(30)h_i^{(0)}=\frac{1}{\mu(\vec{r}_i)}\left(\sum_{j=1}^{N}\sigma\left(g_{ij}^{\mathrm{e-e}}\mathbf{W}\delta_{\alpha_i}^{\alpha_j}\right)\circ\Gamma^{\alpha_i\alpha_j}\left(\lVert\vec{r}_i-\vec{r}_j\rVert\right)\right)\mathbf{W} \tag*{(30)}

where ∘\circ denotes the Hadamard product and the Kronecker delta δαiαj\delta_{\alpha_i}^{\alpha_j} as superscript indicates different parameters depending on the identity between spin αi\alpha_i and αj\alpha_j. Γ:RI→RD\Gamma:\mathbb{R}^{I}\to\mathbb{R}^{D} is a learnable radial filter function, and σ\sigma is the activation function. gije−e∈R4g_{ij}^{\mathrm{e-e}}\in\mathbb{R}^{4} are the rescaled electron-electron distances von Glehn et al., [48]:

gij=log⁡(1+∥r⃗i−r⃗j∥)∥r⃗i−r⃗j∥[r⃗i−r⃗j,∥r⃗i−r⃗j∥].(31)g_{ij}=\frac{\log\left(1+\lVert\vec{r}_i-\vec{r}_j\rVert\right)}{\lVert\vec{r}_i-\vec{r}_j\rVert}\left[\vec{r}_i-\vec{r}_j,\lVert\vec{r}_i-\vec{r}_j\rVert\right]. \tag*{(31)}

μ\mu is a normalization factor:

μ(r⃗)=1+∑m=1MZm2exp⁡(−∥r⃗−R⃗m∥2σnorm2).(32)\mu(\vec{r})=1+\sum_{m=1}^{M}\frac{Z_m}{2}\exp\left(-\frac{\lVert\vec{r}-\vec{R}_m\rVert^2}{\sigma_{\mathrm{norm}}^2}\right). \tag*{(32)}

We use the initial electron embeddings with nuclei embeddings and electron-nuclei distances to construct pairwise nuclei-electron embeddings representing edges in a fully connected graph:

hime−n=σ(hi(0)+zˉm+gime−nWˉm).(33)h_{im}^{\mathrm{e-n}}=\sigma\left(h_i^{(0)}+\bar{z}_m+g_{im}^{\mathrm{e-n}}\bar{\mathbf{W}}_m\right). \tag*{(33)}

where zˉm\bar{z}_m is the mmth nucleus embedding, gime−n∈R4g_{im}^{\mathrm{e-n}}\in\mathbb{R}^{4} are the rescaled electron-nuclei distances like in Eq. (31). These embeddings are then aggregated with spatial filters twice: once towards the nuclei and once towards the electrons:

hmnα(1)=1μ(R⃗m)∑i∈Aαhi,me−n∘Γˉmn(r⃗i−R⃗m),(34)h_m^{\mathrm{n}\alpha(1)}=\frac{1}{\mu(\vec{R}_m)}\sum_{i\in A^\alpha}h_{i,m}^{\mathrm{e-n}}\circ\bar{\Gamma}_m^{\mathrm{n}}\left(\vec{r}_i-\vec{R}_m\right), \tag*{(34)}
mi(1)=1μ(r⃗i)∑m=1Mhi,me−n∘Γˉme(r⃗i−R⃗m),(35)m_i^{(1)}=\frac{1}{\mu(\vec{r}_i)}\sum_{m=1}^{M}h_{i,m}^{\mathrm{e-n}}\circ\bar{\Gamma}_m^{\mathrm{e}}\left(\vec{r}_i-\vec{R}_m\right), \tag*{(35)}
hi(1)=σ(mi(1)W+b).(36)h_i^{(1)}=\sigma\left(m_i^{(1)}\mathbf{W}+b\right). \tag*{(36)}

We update the nuclei embeddings with LL update layers:

hmnα(l+1)=hmnα(l)+σ([hmnα(l),hmnα^(l)]W(l)+b(l)),(37)h_m^{\mathrm{n}\alpha(l+1)}=h_m^{\mathrm{n}\alpha(l)}+\sigma\left(\left[h_m^{\mathrm{n}\alpha(l)},h_m^{\mathrm{n}\hat{\alpha}(l)}\right]\mathbf{W}^{(l)}+b^{(l)}\right), \tag*{(37)}

where α^\hat{\alpha} denotes the opposite spin of α\alpha, to obtain the final nuclei embeddings hmnα(L)h_m^{\mathrm{n}\alpha(L)}. The final electron embeddings hie(L)h_i^{\mathrm{e}(L)} are constructed by combining the message from the nuclei and the previous electron embedding:

hie(L)=σ(σ(hi(1)W+mi(L)+b1)W+b2)+hi(1)(38)h_i^{\mathrm{e}(L)}=\sigma\left(\sigma\left(h_i^{(1)}\mathbf{W}+m_i^{(L)}+b_1\right)\mathbf{W}+b_2\right)+h_i^{(1)} \tag*{(38)}

where mim_i is the message from the nuclei to the iith electron:

mi(L)=1μ(r⃗i)∑m=1Mσ([hmnαi(L),hmnα^i(L)]W+b)∘Γˉmdiff(r⃗i−R⃗m).(39)m_i^{(L)}=\frac{1}{\mu(\vec{r}_i)}\sum_{m=1}^{M}\sigma\left(\left[h_m^{\mathrm{n}\alpha_i(L)},h_m^{\mathrm{n}\hat{\alpha}_i(L)}\right]\mathbf{W}+b\right)\circ\bar{\Gamma}_m^{\mathrm{diff}}\left(\vec{r}_i-\vec{R}_m\right). \tag*{(39)}

The spatial filters Γˉ\bar{\Gamma} are defined as:

Γˉm(l)(x)=βˉm(x)W(l),(40)\bar{\Gamma}^{(l)}_{m}(x)=\bar{\beta}_{m}(x)W^{(l)}, \tag*{(40)}
βˉm(x)=[exp⁡(−(∥x∥sˉmi)2)]i=1DWenv∘(σ(xWˉm(1)+bˉm(1))W(2)+b(2)).(41)\bar{\beta}_{m}(x)=\left[\exp\left(-\left(\frac{\lVert x\rVert}{\bar{s}_{mi}}\right)^2\right)\right]_{i=1}^{D}W^{\mathrm{env}}\circ\left(\sigma\left(x\bar{W}^{(1)}_{m}+\bar{b}^{(1)}_{m}\right)W^{(2)}+b^{(2)}\right). \tag*{(41)}

Note that βˉ\bar{\beta} is shared across all instances of Γˉ\bar{\Gamma}. Γ\Gamma is defined analogously to Γˉ\bar{\Gamma} but with fixed learnable parameters instead of MetaGNN parametrized ones.

MetaGNN

The MetaGNN M:(R3×N+)Nn→Θ\mathcal{M}:(\mathbb{R}^{3}\times\mathbb{N}_{+})^{N_{n}}\to\Theta takes the nucleus position R⃗\vec{R} and charges ZZ as input and outputs parameters of the electronic wave function to adapt the solution to the system of interest. We follow Gao & Günnemann [14, 15] and implement it as a graph neural network (GNN) where nuclei are represented as nodes and edges are constructed based on inter-particle distances. The charge of the nucleus determines the initial node embeddings:

ki(0)=EZi(42)k_{i}^{(0)}=EZ_{i} \tag*{(42)}

where EE is an embedding matrix and ZiZ_i is the charge of the iith nucleus. These embeddings are iteratively updated via message passing in the following way:

ki(l+1)=f(l)(ki(l),ti(l)),(43)k_{i}^{(l+1)}=f^{(l)}\left(k_{i}^{(l)},t_{i}^{(l)}\right), \tag*{(43)}
ti(l)=1∣NR⃗i∣∑j=1Mg(l)(ki(l),kj(l))∘Γ(l)(R⃗i−R⃗j),(44)t_{i}^{(l)}=\frac{1}{\left|\mathcal{N}_{\vec{R}_{i}}\right|}\sum_{j=1}^{M}g^{(l)}\left(k_{i}^{(l)},k_{j}^{(l)}\right)\circ\Gamma^{(l)}\left(\vec{R}_{i}-\vec{R}_{j}\right), \tag*{(44)}
νxN=1+∑y∈Nexp⁡(−∥x−y∥2σnorm2)(45)\nu_{x}^{\mathcal{N}}=1+\sum_{y\in\mathcal{N}}\exp\left(-\frac{\lVert x-y\rVert^{2}}{\sigma_{\mathrm{norm}}^{2}}\right) \tag*{(45)}

where Eq. (43) describes the update function, Eq. (44) the message construction, and Eq. (45) a learnable normalization coefficient. We implement the functions ff and gg via Gated Linear Units (GLU) ([45]). As spatial filters, we use the same as in the wave function but additionally multiply the filters with radial Bessel functions from Gasteiger et al. [20]:

Γ(l)(x)=β(x)W(l),(46)\Gamma^{(l)}(x)=\beta(x)W^{(l)}, \tag*{(46)}
β(x)=[2csin⁡(fixc)xexp⁡(−(∥x∥si)2)]i=1DWenv∘(σ(xW(1)+b(1))W(2)+b(2)).(47)\beta(x)=\left[\sqrt{\frac{2}{c}}\frac{\sin\left(\frac{f_i x}{c}\right)}{x}\exp\left(-\left(\frac{\lVert x\rVert}{s_i}\right)^2\right)\right]_{i=1}^{D}W^{\mathrm{env}}\circ\left(\sigma\left(xW^{(1)}+b^{(1)}\right)W^{(2)}+b^{(2)}\right). \tag*{(47)}

where fif_i are learnable frequencies, and cc is a smooth cutoff for the Bessel functions.

After LL layers, we take the final node embeddings, pass them through another GLU, and then use a different GLU as head for each distinct parameter tensor of the wave function we want to predict. For edge-dependent parameters, like π\pi or AA, we first construct edge embeddings by concatenating all combinations of node embeddings. We pass these through a GLU and then proceed like for node embeddings. For all outputs, we add a default charge-dependent parameter tensor such that the MetaGNN only learns a delta to an initial guess depending on the charge of the nucleus.

Orbital parametrization

Our Pfaffian wave function enables us to simply parametrize a NO≥max⁡{N↑,N↓}N_O\geq\max\{N_{\uparrow},N_{\downarrow}\} orbitals rather than parametrizing exactly N↑/N↓N_{\uparrow}/N_{\downarrow}. As discussed in § 4.4, we accomplish this by associating a fixed number of orbitals with each nucleus. Here, we provide detailed construction for all parameters of the orbital construction. For simplicity, we do not explicitly show the dependence on the kkth Pfaffian. Note that we simply extend the readout by an NkN_k sized dimension for each of the NkN_k Pfaffians from Eq. (13). Further, we predict two sets of parameters, one for ΦPf\Phi_{\mathrm{Pf}} and one for ΦˉPf\bar{\Phi}_{\mathrm{Pf}} in Eq. (9). To parametrize the orbitals, we predict Norb/nucN_{\mathrm{orb}/\mathrm{nuc}} orbital parameters for each of the NnN_{\mathrm{n}} nuclei. Concretely, the linear projection to WkW_k from Eq. (14) are constructed as

W=[ω1(k1)⋮ωNorb/nuc(k1)ω1(k2)⋮ωNorb/nuc(kNn)]∈RNo×Nf(48)W = \begin{bmatrix} \omega_1(\mathbf{k}_1) \\ \vdots\\ \omega_{N_{\mathrm{orb}/\mathrm{nuc}}}(\mathbf{k}_1) \\ \omega_1(\mathbf{k}_2) \\ \vdots\\ \omega_{N_{\mathrm{orb}/\mathrm{nuc}}}(\mathbf{k}_{N_{\mathrm{n}}}) \end{bmatrix} \in\mathbb{R}^{N_{\mathrm{o}} \times N_{\mathrm{f}}} \tag*{(48)}

where ωi:RD→RNf\omega_i : \mathbb{R}^{D} \to\mathbb{R}^{N_{\mathrm{f}}} learnable readouts of our MetaGNN. Similarly, we parametrize the envelope coefficients σk\sigma_k from Eq. (16):

σ=[ς1(k1)⋮ςNenv/nuc(k1)ς1(k2)⋮ςNenv/nuc(kNn)]∈R+Nenv(49)\boldsymbol{\sigma} = \begin{bmatrix} \varsigma_1(\mathbf{k}_1) \\ \vdots\\ \varsigma_{N_{\mathrm{env}/\mathrm{nuc}}}(\mathbf{k}_1) \\ \varsigma_1(\mathbf{k}_2) \\ \vdots\\ \varsigma_{N_{\mathrm{env}/\mathrm{nuc}}}(\mathbf{k}_{N_{\mathrm{n}}}) \end{bmatrix} \in\mathbb{R}_{+}^{N_{\mathrm{env}}} \tag*{(49)}

where ςi:RD→R+\varsigma_i : \mathbb{R}^{D} \to\mathbb{R}_{+} are learnable readouts of our MetaGNN. The linear orbital weights π\pi connect each nuclei-centered envelope to the non-atom-centered orbitals. For this, we need to find a mapping from each of the NenvN_{\mathrm{env}} envelopes to each of the NoN_{\mathrm{o}} orbitals. Since Nenv=Nenv/nuc×NnN_{\mathrm{env}} = N_{\mathrm{env}/\mathrm{nuc}} \times N_{\mathrm{n}} and No=Norb/nuc×NnN_{\mathrm{o}} = N_{\mathrm{orb}/\mathrm{nuc}} \times N_{\mathrm{n}} are predicted per nuclei, a natural connection is established via a pair-wise atom function:

π=[ϖ1,1(k1,k1)⋯ϖ1,Norb/nuc(k1,k1)ϖ1,1(k2,k1)⋯⋮⋱⋮⋮⋱ϖNenv/nuc,1(k1,k1)⋯ϖNenv/nuc,Norb/nuc(k1,k1)ϖNenv/nuc,1(k2,k1)⋯ϖ1,1(k1,k2)⋯ϖ1,Norb/nuc(k1,k2)ϖ1,1(k2,k2)⋯⋮⋱⋮⋮⋱]∈RNenv×No(50)\boldsymbol{\pi} = \begin{bmatrix} \varpi_{1,1}(\mathbf{k}_1,\mathbf{k}_1) & \cdots& \varpi_{1,N_{\mathrm{orb}/\mathrm{nuc}}}(\mathbf{k}_1,\mathbf{k}_1) & \varpi_{1,1}(\mathbf{k}_2,\mathbf{k}_1) & \cdots\\ \vdots& \ddots& \vdots& \vdots& \ddots\\ \varpi_{N_{\mathrm{env}/\mathrm{nuc}},1}(\mathbf{k}_1,\mathbf{k}_1) & \cdots& \varpi_{N_{\mathrm{env}/\mathrm{nuc}},N_{\mathrm{orb}/\mathrm{nuc}}}(\mathbf{k}_1,\mathbf{k}_1) & \varpi_{N_{\mathrm{env}/\mathrm{nuc}},1}(\mathbf{k}_2,\mathbf{k}_1) & \cdots\\ \varpi_{1,1}(\mathbf{k}_1,\mathbf{k}_2) & \cdots& \varpi_{1,N_{\mathrm{orb}/\mathrm{nuc}}}(\mathbf{k}_1,\mathbf{k}_2) & \varpi_{1,1}(\mathbf{k}_2,\mathbf{k}_2) & \cdots\\ \vdots& \ddots& \vdots& \vdots& \ddots \end{bmatrix} \in\mathbb{R}^{N_{\mathrm{env}} \times N_{\mathrm{o}}} \tag*{(50)}

where ϖi,j:RD×RD→R\varpi_{i,j} : \mathbb{R}^{D} \times\mathbb{R}^{D} \to\mathbb{R} are learnable readouts of our MetaGNN. Similarly, we establish the orbital correlations AA from Eq. (14) by connecting each of the NoN_{\mathrm{o}} orbitals to each other:

A^pf=[α1,1(k1,k1)⋯α1,Norb/nuc(k1,k2)α1,1(k2,k1)⋯⋮⋱⋮⋮⋱αNorb/nuc,1(k1,k1)⋯αNorb/nuc,Norb/nuc(k1,k1)αNorb/nuc,1(k2,k1)⋯α1,1(k1,k2)⋯α1,Norb/nuc(k1,k2)α1,1(k2,k2)⋯⋮⋱⋮⋮⋱]∈RNo×No(51)\hat{A}_{\mathrm{pf}} = \begin{bmatrix} \alpha_{1,1}(\mathbf{k}_1,\mathbf{k}_1) & \cdots& \alpha_{1,N_{\mathrm{orb}/\mathrm{nuc}}}(\mathbf{k}_1,\mathbf{k}_2) & \alpha_{1,1}(\mathbf{k}_2,\mathbf{k}_1) & \cdots\\ \vdots& \ddots& \vdots& \vdots& \ddots\\ \alpha_{N_{\mathrm{orb}/\mathrm{nuc}},1}(\mathbf{k}_1,\mathbf{k}_1) & \cdots& \alpha_{N_{\mathrm{orb}/\mathrm{nuc}},N_{\mathrm{orb}/\mathrm{nuc}}}(\mathbf{k}_1,\mathbf{k}_1) & \alpha_{N_{\mathrm{orb}/\mathrm{nuc}},1}(\mathbf{k}_2,\mathbf{k}_1) & \cdots\\ \alpha_{1,1}(\mathbf{k}_1,\mathbf{k}_2) & \cdots& \alpha_{1,N_{\mathrm{orb}/\mathrm{nuc}}}(\mathbf{k}_1,\mathbf{k}_2) & \alpha_{1,1}(\mathbf{k}_2,\mathbf{k}_2) & \cdots\\ \vdots& \ddots& \vdots& \vdots& \ddots \end{bmatrix} \in\mathbb{R}^{N_{\mathrm{o}} \times N_{\mathrm{o}}} \tag*{(51)}
Apf=12(A^pf−A^pfT)(52)A_{\mathrm{pf}} = \frac{1}{2}\left(\hat{A}_{\mathrm{pf}} - \hat{A}_{\mathrm{pf}}^{\mathsf{T}}\right) \tag*{(52)}

where αi,j:RD×RD→R\alpha_{i,j} : \mathbb{R}^{D} \times\mathbb{R}^{D} \to\mathbb{R} are learnable readouts of our MetaGNN and Eq. (52) enforcing the antisymmetry requirements on AA.

Changes to the MetaGNN

We performed several optimizations on the MetaGNN from Gao & Günnemann [15] that primarily reduce the number of parameters while keeping accuracy. In particular, we changed the following:

  • We replace all MLPs with gated linear units (GLU) (Shazeer [45]).

  • We reduced the hidden dimension from 128 to 64.

  • We reduced the message dimension from 64 to 32.

  • We use bessel basis functions [20] on the radius for edge filters.

  • We remove the hand-crafted orbital locations and the associated network.

  • We added a LayerNorm before every GLU.

Together, these changes reduce the number of parameters from 13M to 1M for the MetaGNN while outperforming Gao & Günnemann [15] as demonstrated in § 5.

Experimental setup

Hyperparameters

We list the default parameters used for the experiments in Table 2. Most of them were taken directly from Gao & Günnemann [15]. We may have used different parameters for the experiments in § 5 if explicitly stated so. We implement everything in JAX [4]. To compute the laplacian ∇2Ψ\nabla^{2}\Psi, we use the forward laplacian algorithm (Li et al., 2024) implemented in the folx library [18].

HyperparameterValue
Structurebatch sizefull batch
StructureTotal electron samples4096
PretrainingEpochs10000
PretrainingLearning rate10−3∗(1+t∗10−4)−110^{-3} * (1 + t * 10^{-4})^{-1}
PretrainingOptimizerLamb
PretrainingMCMC steps5
PretrainingBasisSTO-6G
PretrainingSubproblem steps50
PretrainingSubproblem optimizerProdigy
PretrainingSubproblem α\alpha1.0
PretrainingSubproblem β\beta10−410^{-4}
OptimizationSteps60000
OptimizationLearning rate0.02∗(1+t∗10−4)−10.02 * (1 + t * 10^{-4})^{-1}
OptimizationOptimizerSpring
OptimizationMCMC steps20
OptimizationNorm constraint10−310^{-3}
OptimizationDamping0.001
OptimizationMomentum0.99
OptimizationEnergy clipping5 times mean deviation from median
AnsatzHidden dim256
AnsatzE-E int dim32
AnsatzLayers4
AnsatzActivationSiLU
AnsatzDeterminants/Pfaffians16
AnsatzJastrow layers3
AnsatzFilter hidden dims[16, 8]
PfaffianNorb/nucN_{\mathrm{orb/nuc}} (H, He)2
PfaffianNorb/nucN_{\mathrm{orb/nuc}} (Li, Be)6
PfaffianNorb/nucN_{\mathrm{orb/nuc}} (B, C)7
PfaffianNorb/nucN_{\mathrm{orb/nuc}} (N, O)8
PfaffianNorb/nucN_{\mathrm{orb/nuc}} (F, Ne)10
PfaffianNenv/nucN_{\mathrm{env/nuc}}8
MetaGNNEmbedding dim64
MetaGNNMessage dim32
MetaGNNLayers3
MetaGNNActivationSiLU
MetaGNNFilter hidden dims[32, 16]

Table 2. Hyperparameters used for the experiments.

Source code

We provide the source code publicly on GitHub 1.

Compute time

Table 3 lists the compute times required for conducting our experiments measured in Nvidia A100 GPU hours. Depending on the experiment, we use between 1 and 4 GPUs per experiment via data parallelism. We typically allocated 32GB of system memory and 16 CPU cores per experiment. In terms of the number of parameters, the Moon wave function is as large as in Gao & Günnemann [15] at 1M parameters, and the MetaGNN shrank from 13M parameters to just 1M parameters.

ExperimentTime (GPU hours)
Ionization & affinity224
N2116
N2 + Ethene124
TinyMol small78
TinyMol large96

Table 3. Compute time per experiment measured in Nvidia A100 GPU hours.

Preconditioning

The Spring optimizer [22] is a natural gradient descent optimizer for electronic wave functions Ψ\Psi with the following update rule

θt=θt−ηδt(53)\theta^{t} = \theta^{t} - \eta\delta^{t} \tag*{(53)}
δt=(OˉTOˉ+λI)−1(∇θt+λμδt−1)(54)\delta^{t} = \left(\bar{O}^{T}\bar{O} + \lambda I\right)^{-1}\left(\nabla\theta^{t} + \lambda\mu\delta^{t-1}\right) \tag*{(54)}

where λ\lambda is the damping factor, μ\mu is the momentum, η\eta is the learning rate, and Oˉ\bar{O} is the zero-centered Jacobian:

Oˉ=O−1N∑i=1NOi,(55)\bar{O} = O - \frac{1}{N}\sum_{i=1}^{N}O_i, \tag*{(55)}
O=[∂log⁡ψ(x1)∂θ⋮∂log⁡ψ(xN)∂θ].(56)O = \begin{bmatrix} \frac{\partial\log\psi(x_1)}{\partial\theta} \\ \vdots\\ \frac{\partial\log\psi(x_N)}{\partial\theta} \end{bmatrix}. \tag*{(56)}

Since Oˉ∈RN×P\bar{O} \in\mathbb{R}^{N \times P} where NN is the batch size and PP the number of parameters, the update in Eq. (54) can be efficiently computed using the Woodbury matrix identity, which after some simplifications yields

δt=Oˉ(OˉOˉT+λI)−1(ϵ+μOˉδt−1)+μδt−1.(57)\delta^{t} = \bar{O}\left(\bar{O}\bar{O}^{T} + \lambda I\right)^{-1}\left(\epsilon+ \mu\bar{O}\delta^{t-1}\right) + \mu\delta^{t-1}. \tag*{(57)}

Our early experiment found it necessary to center the jacobian Oˉ\bar{O} per molecule rather than once for all. In single-structure VMC, the centering eliminates the gradient of the wave function along the direction where the amplitude of the wave function increases for all inputs. This direction does not affect energies. Thus, instead of restricting the gradient from increasing in magnitude for all samples, we constrain it to not increase in magnitude for each molecule separately. Note that the latter implies the first but not vice versa. For multi-structure VMC, we compute Oˉ\bar{O} as

Oˉ=O−[1N1∑i=1N1Oi⋮1NM∑i=N−NMNOi](58)\bar{O} = O - \begin{bmatrix} \frac{1}{N_1}\sum_{i=1}^{N_1} O_i \\ \vdots\\ \frac{1}{N_M}\sum_{i=N-N_M}^{N} O_i \end{bmatrix} \tag*{(58)}

where N1,…,NMN_1,\ldots,N_M are the index limits between molecular structures.

To stabilize computations, we performed preconditioning in float64.

Extensivity on hydrogen chains

Gao & Günnemann [15] and Scherbela et al. [43] analyzed the behavior of their wave functions on hydrogen chains to investigate the extensivity of their wave functions. They did so by training the generalized wave functions on a set of hydrogen chains with 6 and 10 elements. Then, they evaluated the energy per atom on hydrogen chains with different lengths. We replicated their experiment and trained a single NeurPf on the hydrogen chains with 6 and 10 atoms and evaluated the energy per atom on hydrogen chains of increasing lengths.

Figure 6. Energy per atom of hydrogen chains with different lengths. The energy is computed with a single NeurPf trained on the hydrogen chains with 6 and 10 atoms.

Fig. 6 shows the energy per atom of hydrogen chains with different lengths for various methods, Globe+Moon and Globe+FermiNet from Gao & Günnemann [15], Scherbela et al. [43], Hartree-Fock (CBS), the AFQMC limit for an infinitely long chain [36], and NeurPf. It is apparent that NeurPf outperforms Globe+Moon and Globe+FermiNet significantly by achieving significantly lower energies outside of the training regime. Compared to Scherbela et al. [43], NeurPf generally performs better on longer chains, achieving errors below the Hartree-Fock baseline. However, we observe significantly higher errors in the shortest chains in NeurPf.

These results indicate that NeurPf is better at generalizing to longer chains than previous works despite not including additional Hartree-Fock calculations like Scherbela et al. [43].

Metal ionization energies

In addition to the results in Sec. 5, where we train on all second-row elements and their ionization and affinity potentials, we here train a single NeurPf on a set of metals and their ionization energies. This demonstrates that Neural Pfaffians also scale to heavier 3rd and 4th row elements. Fig. 7 shows the ionization energy during training. It is apparent, that NeurPf can learn a solution for all states simultaneously.

Ionization energies of metal atoms

Figure 7. Ionization energies of metal atoms. The ionization energies are computed with a single NeurPf trained on the neutral and ionized atoms. Reference energies are taken from Martin & Musgrove [34].

TinyMol convergence in time

In Figure 8, we show the runtime effect of choosing different embedding and antisymmetrizer. We test our default model, our (FermiNet), our (PsiFormer) and Globe + Moon on both TinyMol datasets. For any time budget, all variants of NeurPf converge to lower energies than Globe.

Energy convergence as a function of time

Figure 8. Energy convergence as a function of time.

Convergence ablation studies

Here, we provide additional ablation studies to further investigate the performance of NeurPf and our efficient envelopes. In particular, we train four different models on the small TinyMol dataset: NeurPf, NeurPf with the envelopes from Spencer et al. [46], NeurPf with the envelopes from Pfau et al. [38], and an AGP-based generalized wave function.

The total energy during training is shown in Figure 9. The left plot shows the convergence regarding the number of steps, and the right plot shows the convergence in terms of time. We observe that NeurPf convergence is consistently faster than the other methods in terms of the number of steps and time. One further sees the importance of generalizing Eq. (9) via the Pfaffian as the AGP-based wave function does not converge to the same accuracy as NeurPf. The bottleneck envelopes from Pfau et al. [38] do not only converge to worse energies but are also slower per step than our efficient envelopes from § 4.2.

Figure 9. Ablation study on the small TinyMol dataset. The y-axis shows the sum of all energies in the dataset. The left plot shows the convergence in terms of the number of steps. The right plot shows the convergence in terms of time. our + Full Env. shows a NeurPf with the envelopes from Spencer et al. [46] and our + Bottleneck Env. uses the bottleneck envelopes from Pfau et al. [38].

Model ablation studies

Learnable antisymmetrizer

We picked Pf⁡(ΦAΦT)\operatorname{Pf}(\Phi A \Phi^{\mathsf{T}}) as parametrization because it generalizes Slater determinants and many alternative parametrizations. For instance, by choosing A=(0I−I0)A = \begin{pmatrix} 0 & I \\ -I & 0 \end{pmatrix} and Φ=(Φ1 Φ2)=Pf⁡(ΦAΦT)⟹Pf⁡(Φ1Φ2T−Φ2Φ1T)\Phi= (\Phi_1\ \Phi_2) = \operatorname{Pf}(\Phi A \Phi^{\mathsf{T}}) \Longrightarrow\operatorname{Pf}(\Phi_1\Phi_2^{\mathsf{T}} - \Phi_2\Phi_1^{\mathsf{T}}). We investigate the impact of having AA being fixed/learnable in Figure 10. The results suggest that having AA being learnable is a significant factor in our Neural Pfaffian’s accuracy.

TinyMol ablation with fixed and learnable antisymmetrizer

Figure 10. TinyMol ablation with fixed and learnable antisymmetrizer.

Embedding network

Since NeurPf is not limited to Moon, we performed additional ablations with FermiNet (Pfau et al. [37]) and PsiFormer (von Glehn et al. [48]) as the embedding. The results in Figure 11 show Neural Pfaffians outperforming Globe and TAO with any of the three equivariant embedding models. Consistent with Gao & Günnemann [15], Moon is the best choice for generalized wave functions.

Ablation study on the small TinyMol dataset with different embedding networks

Figure 11. Ablation study on the small TinyMol dataset with different embedding networks.

TinyMol results

Here, we provide additional data analysis and error metrics for the TinyMol dataset. First, we show in Table 4 the energy per molecule for the small and large TinyMol datasets for NeurPf, Globe, and TAO. To estimate the remaining error, we also train another NeurPf for 128k steps. The results show that NeurPf consistently outperforms TAO and Globe on all molecules in both datasets.

Method (Steps)SmallLarge
CNHC2_2H4_4COH2_2C3_3H4_4CN2_2H2_2CNOHCO2_2
Globe (32k)5.212.310.762.345.840.442.7
TAO (32k)1.14.56.618.721.041.919.6
our (32k)-3.70.1-2.112.75.53.15.0
our (128k)-4.2-1.5-3.71.4-3.8-6.9-8.2

Table 4. TinyMol energies compared to CCSD(T) in mEh_\mathrm{h}.

Second, we show the error per molecule for both the small and large TinyMol datasets in Figure 12. We plot all models after 32k steps of training. It is apparent that NeurPf consistently results in lower, i.e., better, energies than TAO on all molecules in both datasets. Even the pretrained TAO is outperformed by NeurPf on all but four structures of C3H4C_3H_4 in the large TinyMol dataset.

Figure 12. Boxplot of the energy per molecule on both TinyMol small and large datasets for NeurPf, TAO, and the pretrained TAO from Scherbela et al. [43]. Each boxplot contains results from 10 structures for the given molecule. The line indicates the mean, the box the interquartile range, and the whiskers the 1.5 times the interquartile range.

Pretraining on the TinyMol dataset

The TinyMol provides an additional pretraining set of 360 structures (18 molecules, 20 structures each). Like Scherbela et al. [43], we pretrain our model on the training set of the TinyMol dataset and then finetune on the two test sets. Interestingly, we find the Spring optimizer to be unstable when swapping molecules from step to step and, thus, use CG-preconditioning like Gao & Günnemann [15] during pretraining. While yielding a small benefit on the small molecules, we find no notable difference to the Hartree-Fock pretrained model on the large molecules as shown in Figure 13. On the small structures, the unpretrained NeurPf’s energies are 5.7 mEH\mathrm{m}E_H lower. NeurPf also surpasses the pretrained TAO after just 8k steps. Compared to the pretrained TAO on the large structures, NeurPf surpasses TAO after 16k steps and achieves 5.4 mEH\mathrm{m}E_H lower energies after 32k steps.

Figure 13. TinyMol results with pretraining on the training set.

Joint vs separate training

To estimate the benefit of training a generalized wave function compared to training a model per molecule, we compare the convergence of the total energy on the TinyMol dataset for both approaches depending on the total number of training steps. As training a separate model for each of the 70 TinyMole test molecules is computationally beyond the scope of this work, we select on structure per molecules and train a model for each of the 7 molecules. We use the same NeurPf with MetaGNN for both approaches. The results are shown in Figure 14. We observe that for lower step numbers, it is quite beneficial to train a generalized model. Though, this benefit vanishes for higher step numbers, and training a model per molecule yields lower energies. We attribute this to the fact that the generalized model has to learn a more complex representation that is not necessary for each molecule individually. Further, the per-molecule energy estimates are quite unstable due to the small shared batch size. Developments like Scherbela et al. [42] may improve NeurPf training as well.

Figure 14. Comparison of the energy per molecule on the TinyMol dataset for training jointly on all structures vs training a model per structure.

Training time by batch composition

Here, we benchmark the total time per step for a two-molecule batch. We test all combinations of two molecules with Ne1,Ne2∈2,4,8,16,32N_e^1, N_e^2 \in2, 4, 8, 16, 32. While we find a small runtime increase when processing small molecules jointly in Figure 15, for larger systems, we see the runtime per step converge to the geometric mean of the individual runtimes.

Figure 15. Time per training step depending on the number of electrons in two molecules.

Pfaffian runtime

In Figure 16, we benchmark our implementation for Pf⁡(ΦAΦT)\operatorname{Pf}(\Phi A \Phi^{T}) (incl. the matrix multiplications) against the standard operation of det⁡Φ\det\Phi for 10 to 100 electrons. We implement the Pfaffian in JAX while highly optimized CUDA kernels are available for the determinant. In summary, both share the same complexity of O(N3)O(N^{3}), but the Pfaffian is approximately 5 times slower.

Figure 16. Time for the forward pass, gradient and Laplacian computation of determinant vs our Pfaffian implementation.

Broader impact

Highly accurate quantum chemical calculations are essential for understanding chemical reactions and materials properties. Our work contributes to this development by providing accurate neural network quantum Monte Carlo calculations at broader scales thanks to generalized wave functions. While this may be used to distill more accurate force fields or exchange-correlation functionals for DFT, the societal impact of our work is primarily in the scientific domain due to the high computational cost of neural network VMC. To the best of our knowledge, our work does not promote any negative societal impact more than general theoretical chemistry research does.

References

  1. [1]Acevedo, A., Curry, M., Joshi, S. H., Leroux, B., and Malaya, N. Vandermonde Wave Function Ansatz for Improved Variational Monte Carlo. In 2020 IEEE/ACM Fourth Workshop on Deep Learning on Supercomputers (DLS), pp. 40–47, November 2020. doi: 10.1109/DLS51937.2020.00010.
  2. [2]Bajdich, M., Mitas, L., Drobný, G., Wagner, L. K., and Schmidt, K. E. Pfaffian Pairing Wave Functions in Electronic-Structure Quantum Monte Carlo Simulations. Physical Review Letters, 96 (13):130201, April 2006. ISSN 0091-9007, 1079-7114. doi: 10.1103/PhysRevLett.96.130201.DOI
  3. [3]Bajdich, M., Mitas, L., Wagner, L. K., and Schmidt, K. E. Pfaffian pairing and backflow wavefunctions for electronic structure quantum Monte Carlo methods. Physical Review B, 77(11):115112, March 2008. doi: 10.1103/PhysRevB.77.115112.DOI
  4. [4]Bradbury, J., Frostig, R., Hawkins, P., Johnson, M. J., Leary, C., Maclaurin, D., Necula, G., Paszke, A., VanderPlas, J., Wanderman-Milne, S., and Zhang, Q. JAX: Composable transformations of Python+NumPy programs, 2018.
  5. [5]Casula, M. and Sorella, S. Geminal wavefunctions with Jastrow correlation: A first application to atoms. The Journal of Chemical Physics, 119(13):6500–6511, October 2003. ISSN 0021-9606, 1089-7690. doi: 10.1063/1.1604379.DOI
  6. [6]Casula, M., Attaccalite, C., and Sorella, S. Correlated geminal wave function for molecules: An efficient resonating valence bond approach. The Journal of Chemical Physics, 121(15):7110–7126, October 2004. ISSN 0021-9606. doi: 10.1063/1.1794632.
  7. [7]Ceperley, D., Chester, G. V., and Kalos, M. H. Monte Carlo simulation of a many-fermion study. Physical Review B, 16(7):3081–3099, October 1977. doi: 10.1103/PhysRevB.16.3081.DOI
  8. [8]Chakravorty, S. J., Gwaltney, S. R., Davidson, E. R., Parpia, F. A., and p Fischer, C. F. Ground-state correlation energies for atomic ions with 3 to 18 electrons. Physical Review A, 47(5):3649–3670, May 1993. doi: 10.1103/PhysRevA.47.3649.
  9. [9]Cheng, L., Szabó, P. B., Schätzle, Z., Kooi, D., Köhler, J., Giesbertz, K. J. H., Noé, F., Hermann, J., Gori-Giorgi, P., and Foster, A. Highly Accurate Real-space Electron Densities with Neural Networks, September 2024.DOI
  10. [10]Foster, J. M. and Boys, S. F. Canonical Configurational Interaction Procedure. Reviews of Modern Physics, 32(2):300–302, April 1960. ISSN 0034-6861. doi: 10.1103/RevModPhys.32.300.
  11. [11]Foulkes, W. M. C., Mitas, L., Needs, R. J., and Rajagopal, G. Quantum Monte Carlo simulations of solids. Reviews of Modern Physics, 73(1):33–83, January 2001. doi: 10.1103/RevModPhys.73.33.DOI
  12. [13]Gallier, J. Remarks on the Cayley Representation of Orthogonal Matrices and on Perturbing the Diagonal of a Matrix to Make it Invertible, November 2013.
  13. [14]Gao, N. and Günnemann, S. Ab-Initio Potential Energy Surfaces by Pairing GNNs with Neural Wave Functions. In International Conference on Learning Representations, April 2022.
  14. [15]Gao, N. and Günnemann, S. Generalizing Neural Wave Functions. In International Conference on Machine Learning, February 2023a. doi: 10.48550/arXiv.2302.04168.DOI
  15. [16]Gao, N. and Günnemann, S. Sampling-free Inference for Ab-Initio Potential Energy Surface Networks. In The Eleventh International Conference on Learning Representations, February 2023b.arxiv.org/abs/2205.14962
  16. [17]Gao, N. and Günnemann, S. On Representing Electronic Wave Functions with Sign Equivariant Neural Networks. In ICLR 2024 Workshop on AI4DifferentialEquations In Science, March 2024.arxiv.org/abs/2403.05249
  17. [18]Gao, N., Köhler, J., and Foster, A. Folx - Forward Laplacian for JAX, 2023.
  18. [19]Gao, N., Eberhard, E., and Günnemann, S. Learning Equivariant Non-Local Electron Density Functionals, October 2024.arxiv.org/abs/2410.07972
  19. [20]Gasteiger, J., Groß, J., and Günnemann, S. Directional Message Passing for Molecular Graphs. In International Conference on Learning Representations, September 2019.arxiv.org/abs/2003.03123
  20. [21]Gerard, L., Scherbela, M., Marquetand, P., and Grohs, P. Gold-standard solutions to the Schrödinger equation using deep learning: How much physics do we need? Advances in Neural Information Processing Systems, May 2022.arxiv.org/abs/2205.09438
  21. [22]Goldshlager, G., Abrahamsen, N., and Lin, L. A Kaczmarz-inspired approach to accelerate the optimization of neural network wavefunctions, January 2024.DOI
  22. [23]Han, J., Zhang, L., and E, W. Solving many-electron Schrödinger equation using deep neural networks. Journal of Computational Physics, 399:108929, December 2019. ISSN 0021-9991. doi: 10.1016/j.jcp.2019.108929.DOI
  23. [24]Hermann, J., Schätzle, Z., and Noé, F. Deep-neural-network solution of the electronic Schrödinger equation. Nature Chemistry, 12(10):891–897, October 2020. ISSN 1755-4330, 1755-4349. doi: 10.1038/s41557-020-0544-y.DOI
  24. [25]Hermann, J., Spencer, J., Choo, K., Mezzacapo, A., Foulkes, W. M. C., Pfau, D., Carleo, G., and Noé, F. Ab initio quantum chemistry with neural-network wavefunctions. Nature Reviews Chemistry, 7(10):692–709, October 2023. ISSN 2397-3358. doi: 10.1038/s41570-023-00516-8.DOI
  25. [26]Kim, J., Pescia, G., Fore, B., Nys, J., Carleo, G., Gandolfi, S., Hjorth-Jensen, M., and Lovato, A. Neural-network quantum states for ultra-cold Fermi gases, May 2023.
  26. [27]Klopper, W., Bachorz, R. A., Tew, D. P., and Hä́ttig, C. Sub-meV accuracy in first-principles computations of the ionization potentials and electron affinities of the atoms H to Ne. Physical Review A, 81(2):022503, February 2010. ISSN 1050-2947, 1094-1622. doi: 10.1103/PhysRevA.81.022503.
  27. [28]Kosmala, A., Gasteiger, J., Gao, N., and Günnemann, S. Ewald-based Long-Range Message Passing for Molecular Graphs. In Proceedings of the 40th International Conference on Machine Learning, pp. 17544–17563. PMLR, July 2023.arxiv.org/abs/2303.04791
  28. [29]Le Roy, R. J., Huang, Y., and Jary, C. An accurate analytic potential function for ground-state N2 from a direct-potential-fit analysis of spectroscopic data. The Journal of Chemical Physics, 125 (16):164310, October 2006. ISSN 0021-9606, 1089-7690. doi: 10.1063/1.2354502.
  29. [31]Lin, J., Goldshlager, G., and Lin, L. Explicitly antisymmetrized neural network layers for variational Monte Carlo simulation. arXiv:2112.03491 [physics], December 2021.
  30. [33]Lyakh, D. I., Musial, M., Lotrich, V. F., and Bartlett, R. J. Multireference Nature of Chemistry: The Coupled-Cluster View. Chemical Reviews, 112(1):182–243, January 2012. ISSN 0009-2665, 1520-6890. doi: 10.1021/cr2001417.DOI
  31. [34]Martin, W. C. and Musgrove, A. Ground levels and ionization energies for the neutral atoms. 1998.
  32. [35]Mishchenko, K. and Defazio, A. Prodigy: An Expeditiously Adaptive Parameter-Free Learner, October 2023.arxiv.org/abs/2306.06101
  33. [36]Motta, M., Ceperley, D. M., Chan, G. K.-L., Gomez, J. A., Gull, E., Guo, S., Jiménez-Hoyos, C. A., Lan, T. N., Li, J., Ma, F., Millis, A. J., Prokof’ev, N. V., Ray, U., Scuseria, G. E., Sorella, S., Stoudenmire, E. M., Sun, Q., Tupitsyn, I. S., White, S. R., Zgid, D., Zhang, S., and Simons Collaboration on the Many-Electron Problem. Collaboration on the Many-Electron Problem. Towards the Solution of the Many-Electron Problem in Real Materials: Equation of State of the Hydrogen Chain with State-of-the-Art Many-Body Methods. Physical Review X, 7(3):031059, September 2017. ISSN 2160-3308. doi: 10.1103/PhysRevX.7.031059.DOI
  34. [37]Pfau, D., Spencer, J. S., Matthews, A. G. D. G., and Foulkes, W. M. C. Ab initio solution of the many-electron Schrödinger equation with deep neural networks. Physical Review Research, 2(3):033429, September 2020. doi: 10.1103/PhysRevResearch.2.033429.DOI
  35. [38]Pfau, D., Axelrod, S., Sutterud, H., von Glehn, I., and Spencer, J. S. Accurate computation of quantum excited states with neural networks. Science, 385(6711):eadn0137, August 2024. doi: 10.1126/science.adn0137.DOI
  36. [40]Richter-Powell, J., Thiede, L., Asparu-Guzik, A., and Duvenaud, D. Sorting Out Quantum Monte Carlo, November 2023.
  37. [41]Scherbela, M., Reisenhofer, R., Gerard, L., Marquetand, P., and Grohs, P. Solving the electronic Schrödinger equation for multiple nuclear geometries with weight-sharing deep neural networks. Nature Computational Science, 2(5):331–341, May 2022. ISSN 2662-8457. doi: 10.1038/s43588-022-00228-x.DOI
  38. [42]Scherbela, M., Gerard, L., and Grohs, P. Variational Monte Carlo on a Budget — Fine-tuning pre-trained Neural Wavefunctions. In Thirty-Seventh Conference on Neural Information Processing Systems, November 2023.arxiv.org/abs/2307.09337
  39. [43]Scherbela, M., Gerard, L., and Grohs, P. Towards a transferable fermionic neural wavefunction for molecules. Nature Communications, 15(1):120, January 2024. ISSN 2041-1723. doi: 10.1038/s41467-023-44216-9.DOI
  40. [44]Schütt, K. T., Sauceda, H. E., Kindermans, P.-J., Tkatchenko, A., and Müller, K.-R. SchNet – A deep learning architecture for molecules and materials. The Journal of Chemical Physics, 148(24):241722, June 2018. ISSN 0021-9606, 1089-7690. doi: 10.1063/1.5019779.DOI
  41. [45]Shazeer, N. GLU Variants Improve Transformer, February 2020.arxiv.org/abs/2002.05202
  42. [46]Spencer, J. S., Pfau, D., Botev, A., and Foulkes, W. M. C. Better, Faster Fermionic Neural Networks. 3rd NeurIPS Workshop on Machine Learning and Physical Science, November 2020.arxiv.org/abs/2011.07125
  43. [47]Szabo, A. and Ostlund, N. S. Modern Quantum Chemistry: Introduction to Advanced Electronic Structure Theory. Courier Corporation, 2012.
  44. [48]von Glehn, I., Spencer, J. S., and Pfau, D. A Self-Attention Ansatz for Ab-initio Quantum Chemistry. In The Eleventh International Conference on Learning Representations, February 2023.arxiv.org/abs/2211.13672
  45. [49]Wilson, M., Gao, N., Wudarski, F., Rieffel, E., and Tubman, N. M. Simulations of state-of-the-art fermionic neural network wave functions with diffusion Monte Carlo, March 2021.arxiv.org/abs/2103.12570
  46. [50]Wilson, M., Moroni, S., Holzmann, M., Gao, N., Wudarski, F., Vegge, T., and Bhowmik, A. Neural network ansatz for periodic wave functions and the homogeneous electron gas. Physical Review B, 107(23):235139, June 2023. doi: 10.1103/PhysRevB.107.235139.DOI
  47. [51]Wimmer, M. Algorithm 923: Efficient Numerical Computation of the Pfaffian for Dense and Banded Skew-Symmetric Matrices. ACM Transactions on Mathematical Software, 38(4):30:1–30:17, August 2012. ISSN 0098-3500. doi: 10.1145/2331130.2331138.DOI
  48. [52]You, Y., Li, J., Reddi, S., Hseu, J., Kumar, S., Bhojanapalli, S., Song, X., Demmel, J., Keutzer, K., and Hsieh, C.-J. Large Batch Optimization for Deep Learning: Training BERT in 76 minutes. In Eighth International Conference on Learning Representations, April 2020.arxiv.org/abs/1904.00962
  49. [53]Zhang, X., Wang, L., Helwig, J., Luo, Y., Fu, C., Xie, Y., Liu, M., Lin, Y., Xu, Z., Yan, K., Adams, K., Weiler, M., Li, X., Fu, T., Wang, Y., Yu, H., Xie, Y., Fu, X., Strasser, A., Xu, S., Liu, Y., Du, Y., Saxton, A., Ling, H., Lawrence, H., Stärk, H., Gui, S., Edwards, C., Gao, N., Lader, A., Wu, T., Hofgard, E. F., Tehrani, A. M., Wang, R., Daigavane, A., Bohde, M., Kurtin, J., Huang, Q., Phung, T., Xu, M., Joshi, C. K., Mathis, S. V., Azizzadenesheli, K., Fang, A., Aspuru-Guzik, A., Bekkers, E., Bronstein, M., Zitnik, M., Anandakumar, A., Ermon, S., Liò, P., Yu, R., Günnemann, S., Leskovec, J., Ji, H., Sun, J., Barzilay, R., Jaakkola, T., Coley, C. W., Qian, X., Qian, X., Smidt, T., and Ji, S. Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systems, November 2023.DOI

Paper details

vlnc:W_yz6ypdfj
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 3

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents