betaFeedback

WorksPhysics and Astronomy

Learning a reactive potential for silica-water through uncertainty attribution

Abstract

The reactivity of silicates in an aqueous solution is relevant to various chemistries ranging from silicate minerals in geology, to the C-S-H phase in cement, nanoporous zeolite catalysts, or highly porous precipitated silica. While simulations of chemical reactions can provide insight at the molecular level, balancing accuracy and scale in reactive simulations in the condensed phase is a challenge. Here, we demonstrate how a machine-learning reactive interatomic potential can accurately capture silicate-water reactivity. The model was trained on a new dataset comprising 400,000 energies and forces of molecular clusters at the ωω-B97XD def2-TVZP level. To ensure the robustness of the model, we introduce a new and general active learning strategy based on the attribution of the model uncertainty, that automatically isolates uncertain regions of bulk simulations to be calculated as small-sized clusters. Our trained potential is found to reproduce static and dynamic properties of liquid water and solid crystalline silicates, despite having been trained exclusively on cluster data. Furthermore, we utilize enhanced sampling simulations to recover the self-ionization reactivity of water accurately, and the acidity of silicate oligomers, and lastly study the silicate dimerization reaction in a water solution at neutral conditions and find that the reaction occurs through a flanking mechanism.

Introduction

One of the most abundant materials on the planet, silica[1] and its many polymorphs are used in applications like catalysts[2], pharmaceuticals[3], nanotechnology[4], and additives[5]. Amorphous silica, in particular, has dozens of commercial applications and the global precipitated silica market is expected to reach US$3.49 billion by 2023[6].

Precipitated silica or silica gel is produced by the polymerization of water-soluble silicate salts in the presence of mineral acids, resulting in a precipitate or gel[7]. Similarly, zeolites are crystallized from a precursor gel in hydrothermal conditions[8]. While various experimental techniques can characterize nanosized crystallites and precipitates, they cannot resolve the individual silicate aggregates or the network of silicon connectivity [9, 10].

Molecular simulations of reactivity in the aqueous condensed phase can bridge this knowledge gap on silica condensation. The key challenge is describing covalent bond breaking and formation accurately while being able to simulate large length- and time-scales. Numerous classical interatomic potentials for anhydrous silica have been generated to simulate silica glasses or other polymorphs[11, 12], including the well-known BKS[11] pair potential for α\alpha-quartz and silica glasses, and more recent reparametrizations for nanoclusters and amorphous silica[12, 13]. These models, along with ReaxFF models parameterized for water-silica[14, 15] cannot accurately replicate the condensation reactions in aqueous solution. The recent advances in machine learning (ML) techniques and the expansion of neural networks (NN) and automatic differentiation routines[16, 17] have provided a branch between physical sciences and statistical learning. NN-based interatomic potentials (NNIP) built on a variety of molecular representations retain the accuracy of the ab initio training data and can be executed at a computational cost much lower than first principle calculations. Several NNIP have been constructed for pure water[18, 19, 20], amorphous and liquid silica[21, 22] and siliceous zeolites[23]. The authors of the latter study demonstrated superior performance to ReaxFF. However, no NNIP exists that models reactive silica-water interactions in acid or bases, which are required to understand the fundamental steps of silica precipitation, crystallization and gel formation.

The initial step of silicate polymerization or oligomerization is dimer formation, whose mechanism depends on pH and temperature[24, 25]. In neutral conditions or in a slightly acidic medium, the dimerization can occur through neutral monomers[24]. Aqueous reactions such as these are hard to understand at an atomic level, given the need to simulate the role of explicit solvent in the condensed phase, various bond breaking and formation, and proton-transfers in the solvent and reactants. Quantum chemical simulations can provide accurate energies for systems up to hundreds of atoms but are too costly to address free energy questions, that arise in the presence of full explicit solvation. A couple of mechanisms have been reported for the neutral dimerization reaction[26]. One involves a SN2S_N2-backside reaction[27] whereas the other involves a lateral "flank-side" attack[28].

In this study, we trained a reactive water-silica NNIP through an active learning loop. Our ground truth data consists of orbital-based hybrid DFT calculations on molecular clusters of various sizes and the trained NNIP generalizes to condensed phase simulation of thousands of atoms, both aqueous and solid silica. Since production simulations contain many more atoms than we can simulate with DFT - we introduce a new uncertainty attribution technique that builds on adversarial attacks on uncertainty[29] to locate clusters of uncertainty within large simulation boxes for annotating with DFT. The NNIP was validated successfully against experimental properties of liquid water, and silica, both physical (diffusivity, structure, vibrational) and chemical (dissociation constants). We then used the potential to evaluate the dimerization of orthosilicic acid with over 2 ns of enhanced-sampling molecular dynamics (MD) simulations and resolve the mechanism, which had not been resolved until now in an aqueous solution.

Results and discussions

Reference Data Set

A preliminary training data set was constructed containing molecular silica clusters with explicit aqueous solvation. Molecular graphs for linear and cyclic SixOyHzSi_xO_yH_z were generated with x<10x < 10 programmatically, and embedded into 3D conformers with RDkit. Solvating water molecules were added at random with geometrical constraints to avoid clashes and to maximize hydrogen bonding. A subset of structures were deprotonated in one or more O-H bonds and calculated as anion, or balanced with Na+\mathrm{Na}^{+} cations. We also added secondary and common building units of zeolites from the international zeolite database[30]. All structures were relaxed with xTB and then refined with ω\omega-B97xD3/TZVP level of theory in Orca, which served as our ground truth forces and energies. Reactive geometries for proton transfer and covalent Si-O-Si reactivity were obtained using nudged elastic band at the xTB level, followed by single-point DFT calculations. Figure 1a shows the distribution of the data set comprising 210K geometries based on sources as well as based on the stoichiometry of the molecule clusters.

Figure 1. Training dataset and active learning method. a) Distribution of data based on data source - Cluster: silicate and water clusters based on random position guesses. Opt: molecule clusters optimized at a lower level of DFT theory or X-TB. AL: data produced through active learning and NEB: transition states using NEB; and based on stoichiometry - SiOn\mathrm{SiO}_n: silicate clusters in a vacuum. SiOn\mathrm{SiO}_n-H2_2O: silicate clusters surrounded by water molecules, H2_2O: water molecules only and with Na: molecular clusters of silicate or water with Na in it. b) Schematic diagram of attribution. The potential energy and forces and their uncertainty are calculated for a molecular system using a NN ensemble. The uncertainty is then differentiated with respect to atom positions to obtain individual per-atom uncertainty attributes. The neighborhood of the hydrogen atom in orthosilicic acid which is attracted by the adjacent water molecule has high attributes. This shows that the schematic potential is bad at hydrogen transfer from orthosilicic acid to adjacent water. c) Two examples of attribution. The color bar is based on attributes. The red circles show the atomic environments added for active learning with atoms with high attributes at the center. The H3_3O+−H+ - H_2O,HO, H_3OO+ − H3_3O+,H+, H_3OO+ − OH−^-, Si(OH)4_4 − H2_2O interactions and Si(OH)3_3OH2+_2^+ complex lead to high attributes as they were not present in the training data. Parity plot of d) forces and e) energies of a set of molecular clusters obtained through attribution-based active learning. The first-generation NNIP performs poorly on them. f) Consecutive generation-wise improvement of attributes on a held-out condensed phase system. Attribution-based active learning thus improves our model.

Hybrid DFT is more accurate for reactivity than the GGA functional, typically used to train interatomic potentials on solids, but is restricted to a few hundred atoms in molecule clusters. However, considering the local, atom-centered architecture of most NNIPs, it is possible for them to learn from cluster data and run predictions on periodic systems. Most of the interactions in our concerning phase space are short-range coupled with electrostatic long-range interactions which are relatively shielded in a high dielectric medium like water[20, 31].

Active learning strategies

Our active learning methods rely on prediction uncertainty. We trained an ensemble of three NNIP models with varying parameters initialized with different random seeds. We used the variance of forces predictions as the uncertainty metric. We implemented an uncertainty-based adversarial attack by performing gradient-based optimization of the differentiable uncertainty metric[29]. New molecular conformations from the data set were sampled by back-propagating atomic displacements to find local optima that maximize the uncertainty of the NNIP committee while balancing thermodynamic likelihood. These new configurations are then evaluated using quantum mechanical calculations and used to retrain the NNIPs in an active learning loop.

Differentiable uncertainty attribution

After early loops of adversarial attacks, the NNIP is stable enough to run MD simulations of condensed phases for a decent amount of simulation time. An adversarial attack on condensed phases will only generate condensed phase configurations, while our data set comprises molecular clusters. Hence, we adapted the idea of attribution. Attribution is a technique employed to test how a change in a certain input neuron can impact an output neuron, thus making it somewhat interpretable[32, 33]. Similarly, we calculate per-atom attributes on uncertainty which qualitatively interpret which atoms will contribute most to the uncertainty of the NNIP committee. It is calculated as the derivative of the variance of forces or energy with respect to atomic positions. Thus, from atoms with higher attributes, we can detect inter-atomic interactions that the NNIP is highly uncertain about. A schematic of attribution with an example is shown in Figure 1b.

We developed an active learning strategy around attribution that help us choose uncertain atomic clusters from MD frames of condensed phases without manual inspection. Atoms with attributes higher than two times the standard deviation from the mean are chosen as atomic centers and all neighboring molecules within a sphere of radius equal to the cutoff, the distance the NNIP model uses to generate neighbors (6Å), are chosen as the molecule cluster to add to training data. Thus a molecule in the structure remains intact in the cluster. Only the first nearest neighbors are chosen as they have the major contributions to the atomic energies or forces. Each MD frame of condensed phases is used to obtain molecule clusters that can be put back in training data after Quantum mechanical calculations. This active learning loop can work automatically and thus provides an end-to-end framework for improved training of NNIPs employed for condensed phase systems but trained on molecular cluster data. Examples with active learning method are shown in Figure 1c. The NNIP which is used to sample the first generation of molecular clusters using attribution is tested on the same set of geometries after their DFT energies and forces were calculated. It performs very poorly as shown in Figure 1d and Figure 1e, thus proving the efficiency of attribution to choose atomic environments with high errors. We further tested consecutive improvement of attributes on a held-out set of condensed phases which was not put into training data. Figure 1f shows a consecutive decrease in attributes over three generations showing the generation-based improvement skills of our active learning technique. Thus, the attribution-based technique helps sample configurations with high error as well as improves the NNIP model on consecutive generations, which proves the technique’s efficiency as an active learning tool.

Properties of water

It has been seen based on a series of experiments that benchmarking force error is not sufficient[34] and simulation-based statistics should be used. Hence, it is imperative for us to begin by testing the properties of liquid water.

Radial Distribution Functions

We compared the oxygen-oxygen (gOOg_{\mathrm{OO}}) and oxygen-hydrogen (gOHg_{\mathrm{OH}}) radial distribution functions (RDFs) to experimental RDFs obtained from X-ray diffraction measurements[35] and those obtained by reported Gaussian moment neural network (GM-NN) potentials[20]. Thus, we compared our results with GM-NNs trained on periodic systems of water as well as those trained on water clusters similar to our NNIP. Figure 2a shows good agreement of gOOg_{\mathrm{OO}} with the experimental results with the first peak slightly overestimated similar to the ones obtained by BLYP and B3LYP potentials trained on clusters. Both the peak positions in gOHg_{\mathrm{OH}} match experimental results as seen in Figure 2b but are overestimated for all the NN potentials, which can be attributed to the lack of Quantum nuclear effects (QNE)[36]. It is mentioned in the literature that the O-O bond structure of water at ambient conditions is not as affected by QNEs as the bonds involving hydrogen[18, 36] and we notice a similar behavior for our NNIP.

Figure 2. Properties of water. a) O-O and b) O-H radial distribution functions at 300 K calculated by NVT simulations with our NNIP and compared to experimental and GM-NN potentials’ results.NNIP: our Neural network inter-atomic potential; EXP: experimental data; revPBE-bulk: GM-NN potential trained on periodic water box calculated at the revPBE-D3 level of theory; revPBE-cluster: GM-NN potential trained on water clusters calculated at the revPBE-D3 level of theory; B3LYP: GM-NN potential trained on water clusters calculated at the B3YLP-D3 level of theory; BLYP: GM-NN potential trained on water clusters calculated at the BYLP-D3 level of theory. c) Mean squared displacements vs time at different temperatures from 260K to 360K. d) diffusivity vs temperature obtained using our NNIP compared with results from a DP-MP2 potential. e) Vibrational density of states of liquid water obtained from classical (MD) and Ring polymer (RPMD) molecular simulations with our NNIP. The dashed vertical lines represent the experimental infrared spectrum.

Diffusion coefficient

The mean square displacements (MSDs) at different temperatures vs time are shown in Figure 2c. At 300 K, we obtained our diffusion coefficient (D) as 0.21 ± 0.005 Å2^2/ps, close to an experimental value of 0.24 ± 0.015 Å2^2/ps[37]. Similar to RDFs, we compare our results to the GM-NN potentials whose predicted values are 0.182 ± 0.006 Å2^2/ps, 0.215 ± 0.007 Å2^2/ps, 0.143 ± 0.004 Å2^2/ps, 0.139 ± 0.006 Å2^2/ps respectively. We plotted our diffusion coefficients at different temperatures in Figue 2d and compare our results with those obtained by another reported NN potential with MP2 accuracy (DP-MP2)[19]. Our results are higher than those predicted by the DP-MP2 model for most of the temperatures and are closer to experimental results. These underestimated D values for the DP-MP2 model are attributed to the overestimation of inter-molecular energy by MP2 with the aug-cc-pVDZ basis set. Thus our potential can perform better at predicting the diffusion properties of water at different temperatures and provide results comparable to experimental results.

Vibrational Density of States

The vibrational density of states of water (VDOS) is shown in Figure 2e, where we could observe that the peaks corresponding to O-H bond stretch, bending, and libration of water at 300 K obtained using our NNIP are quite close to the experimental observations[18]. We further employed Ring Polymer molecular dynamics (RPMD) coupled with path integral Langevin equation thermostat (PiLE)[38] to introduce NQE. The RPMD simulations shifted the O-H stretch and bending peaks towards the left. The O-H stretch peak is shifted from 3670 to 3600 cm−1\mathrm{cm}^{-1} (3380 cm−1\mathrm{cm}^{-1} from the experiment) and the bending motion peak is shifted from 1664 to 1628 cm−1\mathrm{cm}^{-1} (1637 cm−1\mathrm{cm}^{-1} from the experiment). The libration has very minor changes due to NQE, also observed for VDOS obtained by DP-MP2 model[19] and in previous studies[18].

Equilibrium Density

Our NNIP predicted an equilibrium density of 1.08 g/cc. The GM-NN potentials produced liquid water with densities 0.86 g/cc, 1.02 g/cc, 1.1 g/cc, and 1.12g/cc respectively[20]. Our NNIP performed equally well as the GM-NN potentials trained on clusters. Considering our results we can claim that our NNIP though trained on molecular clusters can predict properties of bulk periodic water systems.

Crystalline Silica

We next tested our ability to predict the relative formation energies of crystalline silica; pure siliceous zeolites to take it a notch further and establish the success of our NNIP on periodic systems. We compared the relative formation energies of 236 zeolites with respect to α\alpha-Quartz predicted by our NNIP to the ones calculated using a periodic PBE-D3 level of DFT theory. Figure 3a depicts our NNIP performing quite closely to periodic DFT results. We also collected experimental relative transition enthalpies for fifteen siliceous zeolites[23] and compared them to the relative formation energies calculated using periodic PBE-D3 DFT theory and our NNIP in Figure 3b. The NNIP performed better than the periodic DFT. This better prediction can be due to the fact that our NNIP is trained on data obtained using a higher level of hybrid functional DFT theory than PBE-D3. Thus, training on cluster data has a lesser impact than the theory of DFT employed to obtain the training data and our NNIP serves better in predicting the relative formation energies of crystalline zeolites.

Figure 3. Properties of crystalline silica. a) Relative formation energies of pure siliceous zeolites with respect to α\alpha-Quartz calculated using our NNIP compared to those obtained by PBE-D3 level of DFT theory. b) Relative formation energies of fifteen zeolites obtained by NNIP and PBE-D3 compared to their experimental transition enthalpies.

Elastic constants

We also calculated elastic constants of α\alpha-Quartz and siliceous FAU zeolite framework to further solidify our claim on NNIP’s capability to predict properties of periodic systems involving silicates. We then compared the elastic constants with those from published literature[39]. The calculated elastic constants compared to results found in the literature are shown in Table 1. It can be seen the NNIP ensemble is quite close to obtaining the elastic constants for both α\alpha-Quartz and FAU zeolite framework. Thus, our NNIP can capture the symmetry in periodic silicates adequately and can be used for predicting their properties.

Crystalline silicaBulk Modulus (GPa)Young Modulus (GPa)Shear Modulus (GPa)Poisson’s ratio
α\alpha-QuartzNNIP ensemble51.92±1051.92\pm10112.16±8112.16\pm849.20±349.20\pm30.14±0.060.14\pm0.06
α\alpha-Quartzliterature38.2338.23101.41101.4147.9347.930.060.06
FAUNNIP ensemble44.34±544.34\pm542.11±442.11\pm415.69±515.69\pm50.34±0.020.34\pm0.02
FAUliterature61.3761.3750.2450.2418.4218.420.360.36

Table 1. Elastic constants of α\alpha-Quartz and FAU siliceous zeolite framework calculated by our NNIP ensemble and compared to results published in the literature.

Chemical reactions

pKa of water

To obtain the pKa_{\mathrm{a}} of water or pKw_{\mathrm{w}}, we chose a single H2_2O molecule in a cubic box of 100 water molecules and varied one of the O-H bond lengths as our reaction coordinate from 0.8 to 1.9 Å with an increment of 0.05 Å. The deprotonation reaction is shown in equation 1

H2O(aq)+99H2O(l)↔OH−+H3O++98H2O(l)(1)\mathrm{H_2O(aq)} + 99\mathrm{H_2O(l)} \leftrightarrow\mathrm{OH^-} + \mathrm{H_3O^+} + 98\mathrm{H_2O(l)} \tag*{(1)}

Where H2_2O (aq) is the water molecule undergoing dissociation. The potential of mean force (PMF) vs O-H bond length (rOHr_{\mathrm{OH}}) is shown in Figure 4a. As seen from the figure, the PMF falls into a minimum between rOH=0.95r_{\mathrm{OH}} = 0.95 and 1 Å. This is the equilibrium O-H bond length in the neutral water molecule, thus signifying the reactant state. We find another equilibrium point at rOH=1.85r_{\mathrm{OH}} = 1.85 Å. Here, the H+^+ has left the dissociated water molecule and has been successfully transferred to H3_3O+^+. The only interaction between OH−^- and the H3_3O+^+ is long-range Coulombic now and hence this state signifies the product state where the dissociation is complete. From the difference in PMF, we obtain our pKw\mathrm{p}K_{\mathrm{w}} as 15.4. The reference pKw\mathrm{p}K_{\mathrm{w}} value is 14[40] but considering the NQE the expected pKw\mathrm{p}K_{\mathrm{w}} becomes 17[41]. Our result is also comparable to a reported value of 16 obtained by first principle calculations using SCAN functional[42]. They used the same procedure as us to find the pKa\mathrm{p}K_{\mathrm{a}} of water.

Figure 4. Simulations of chemical reactions. a) PMF of water deprotonation. The insets show reactant state (0.950.95 Å <rOH<1< r_{\mathrm{OH}} < 1 Å) and product state (rOH=1.85r_{\mathrm{OH}} = 1.85 Å). The free energy of the reactant state is set to 0. b) PMF of orthosilicic acid deprotonation. The insets show reactant state (0.950.95 Å <rOH<1< r_{\mathrm{OH}} < 1 Å) and product state (rOH=1.55r_{\mathrm{OH}} = 1.55 Å). c) PMF of dimer deprotonation. d) PMF of trimer deprotonation. e) Free energy profile of silica dimerization, coordination between attacking O and transferring hydrogen and that between leaving oxygen and the transferring hydrogen along the reaction coordinate d1−d2d_1 - d_2 (Å). The insets show the reactant, product, and transition states.

pKa of silicate oligomers

We also calculated the pKa\mathrm{p}K_{\mathrm{a}} of orthosilicic acid, silicate dimer, and trimer. Orthosilicic acid and dimer have all identical O-H bonds. Hence dissociating any one of them is fine. However, for trimer two O-H bonds attached to the middle Si are different than the end ones. We chose one of the end O-H bond dissociations for trimer to get its pKa\mathrm{p}K_{\mathrm{a}}. Similar to water, we took a single orthosilicic acid, dimer, and trimer in a cubic box of 100 water molecules and varied their chosen O-H bond lengths. The individual PMFs vs rOHr_{\mathrm{OH}} are shown in Figure 4b-Figure 4d. The deprotonation reaction of orthosilicic acid looks as shown in equation 2.

Si(OH)4(aq)+100H2O(l)⟷SiO4H3−+H3O++99H2O(l)(2)\mathrm{Si(OH)}_{4}(\mathrm{aq}) + 100\mathrm{H}_{2}\mathrm{O}(\mathrm{l}) \longleftrightarrow\mathrm{SiO}_{4}\mathrm{H}_{3}^{-} + \mathrm{H}_{3}\mathrm{O}^{+} + 99\mathrm{H}_{2}\mathrm{O}(\mathrm{l}) \tag*{(2)}

We found the equilibrium reactant state at rOHr_{\mathrm{OH}} between 0.95 and 1 Å and product state at rOH=1.55r_{\mathrm{OH}} = 1.55 Å for orthosilicic acid and between 1.5 and 1.55 Å for the dimer and the trimer. From the PMF, we obtain the pKa\mathrm{p}K_{\mathrm{a}} of orthosilicic acid as 10.45. Orthosilicic acid is a weak acid with a known pKa\mathrm{p}K_{\mathrm{a}} of 9.8[43]. Further, for dimer and trimer, we obtained a pKa\mathrm{p}K_{\mathrm{a}} of 10.43 and 10.42 respectively, whose pKa\mathrm{p}K_{\mathrm{a}} values have been reported to vary between 9.5 and 10.7[44]. Our reactive NNIP is thus shown to be capable of replicating complex reactions involving pure water as well as amorphous silicates in water.

Silicate dimerization

We next simulated a dimerization reaction of two silicate monomers in a neutral solution with 100 water molecules at 300K. We ran umbrella sampling on a reaction coordinate, d1−d2d_{1} - d_{2} varying it from -5 Å to 5 Å with an increment of 0.05 Å. d2d_{2} signifies the bond distance between attacking oxygen and the attacked silicon, whereas d1d_{1} signifies the distance between the leaving oxygen and the attacked silicon. Our attacking oxygen, attacked silicon, and the leaving oxygen atoms for the SN2S_N2 and the lateral attack mechanisms are shown in Figure S1. The reported SN2S_N2 mechanism and our observed one are shown in Figure S1-Figure S1. In the case of the lateral attack mechanism, we saw the reaction happening as explained in literature[24, 25, 26], where the leaving oxygen leaves with the hydrogen from the attacking oxygen (Figure S1). From the free energy profile in Figure 4e, we see that at the coordinate close to 55 Å, the product dimer is formed. The reactant state with two monomers is at the coordinates between −3-3 and −5-5 Å where the free energy is almost flat. The reaction energy was obtained by subtracting these two energies which shows that the product phase is only 0.2 kJ/mole higher in energy than the reactant phase which can be noise. The activation energy is obtained as 103 kJ/mole which is closer to 129 kJ/mole found in a DFT study, with solvent effect incorporated and product phase found less stable with free energy 9 kJ/mole higher[24]. From the plot of the coordination between the attacking oxygen with the transferring hydrogen and that of the leaving oxygen with the transferring hydrogen in Figure 4e, we observed that at the transition state, the hydrogen leaves the attacking oxygen with a drop in their coordination value and moves to the leaving oxygen with an increase in their coordination value almost immediately as expected in the lateral flank-side mechanism. Hence, we can conclude that silicate dimerization in an aqueous solution at neutral conditions occurs through a lateral flank-side attack mechanism with pentavalent silicon as a transition state.

Figure S1. Schemes of Silicate dimerization reactions. a) Expected scheme of SN2S_N2 back side attack mechanism. The hydrogen from the attacking oxygen goes to the lateral oxygen, which then loses hydrogen to the leaving oxygen. b) Observed scheme of back-side attack mechanism. When we chose the back side oxygen as leaving oxygen to replicate SN2S_N2 mechanism, a surrounding water molecule interfered and took up hydrogen from the attacking oxygen forming a H3O+\mathrm{H_3O^+} and OH−\mathrm{OH^-}. c) Observed lateral flank-side attack mechanism. This mechanism was not hampered by surrounding water molecules and occurred as reported in the literature. The transition state has a pentavalent Silicon atom.

Conclusion

Neural Network potentials have proven to work as a bridge between first principle calculations and classical potentials several times. In this work, we have proved the same by developing a reactive equivariant NNIP trained on a complex domain of silicate-water interactions. Our NNIP though trained on molecule clusters can predict properties of bulk periodic systems including liquid water as well as crystalline solids. We further proposed a new active learning strategy based on attribution of differentiable uncertainty which further displays highly uncertain inter-atomic interactions in amorphous systems. This method not only serves as a qualitative measure for unsure atomic environments but also serves to minimize computational costs by extracting only the uncertain atomic environments. The NNIP trained on this strategy then proved to be successful in predicting the properties of bulk water as well as crystalline silicates. The reactive potential is further adept at predicting the reaction path of deprotonation of water, small silicate oligomers, and silica dimerization reaction in a water solution. This NNIP can further be used to probe into other silicate polymerization reactions leading to precipitated silica. We can also study the impact of pH and temperature on these reactions in the future as we have sodium in our data set to maintain charge neutrality or we can add aluminum in our data set to train a potential on preliminary stages of the synthesis of zeolites.

Methodology

Model Architecture

Our NN potential is based on PaiNN architecture[45], which is an equivariant neural network with message passing as its backbone. Equivariant models[45, 46, 47] can act on non-invariant inputs like displacement vectors in a symmetry-respecting way and thus offer good accuracy with a low amount of training data for properties that are equivariant in Euclidean space like forces, thus being more data sufficient than invariant potentials like Schnet[48]. In this method, an invariant feature vector for each atom is generated with their atomic numbers which are then updated through convolutions with "messages" from neighboring atoms which consist of distance, orientation, and features of nearest neighbor atoms within a cutoff distance. Each atomic feature is updated with information from neighbors and through convolutions obtains information from atoms far away as well, thereby generating a representation vector as a function of atomic positions and orientations, which is then mapped to atomic energies through feed-forward NN and summed to obtain the energy of the system. Through automatic differentiation, the forces on each atom can also be obtained. We further modified the model to predict stresses of a system based on the virial theorem[49] as given in equation (3) for pair potentials

σijV=1V∑α[∑β=1β>αN(riβ−riα)fjαβ−mαviαvjα],fjαβ=∂E∂(rjβ−rjα)(3)\sigma^{V}_{ij} = \frac{1}{V} \sum_{\alpha} \left[ \sum_{\substack{\beta=1 \\ \beta>\alpha}}^{N} \left(r^{\beta}_{i} - r^{\alpha}_{i}\right) f^{\alpha\beta}_{j} - m^{\alpha}v^{\alpha}_{i}v^{\alpha}_{j} \right], \qquad f^{\alpha\beta}_{j} = \frac{\partial E}{\partial\left(r^{\beta}_{j} - r^{\alpha}_{j}\right)} \tag*{(3)}

where (i,j) signifies x,y, and z directions, β\beta varies from 1 to N neighbors of atom α\alpha, riβr^{\beta}_{i}, and riαr^{\alpha}_{i} are positions of atom β\beta and α\alpha along direction i respectively, fjαβf^{\alpha\beta}_{j} is the force on α\alpha by β\beta along direction j and E is total energy, V is total volume, mαm^{\alpha} is the mass of atom α\alpha and viαv^{\alpha}_{i} is the thermal velocity of atom α\alpha.

Training details

Molecular geometries of the same stoichiometry are split at a ratio of 3:1:1 into training, validation, and test sets. We employed a mini-batch gradient descent optimization with Adam optimizer. The learning drops from 10−410^{-4} to 10−610^{-6} at a rate of 0.5 when the validation loss hits a plateau for 20 epochs. The loss function is the same as described elsewhere[45], with a loss coefficient of 0.95 for forces and 0.05 for energies. We fixed the number of convolutional layers at 4 and the cutoff for nearest neighbors at 6 A˚\mathrm{\AA}. We optimized the other hyperparameters with Sigopt on validation loss. The optimized dimension of the feature vector for each atom is 310; the number of radial basis functions used are 20 and the batch size is chosen as 9.

Active learning

We ran adversarial attacks on molecular geometries randomly chosen from the preliminary data set for three generations. Then we used our model to run MD simulations on 10-100 orthosilicates with different ratios of water molecules at different temperatures of 300, 500, 750, and 1000 K and ran attribution-based active learning on them. We also ran simulations replicating the deprotonation of water, orthosilicic acid, and other silicates. We further ran attributions on zeolite crystals and dimerization reactions to extract uncertain molecule clusters from them. Our active learning consists of nine cycles, involving three from adversarial attacks and then six generations of attribution-based active learning.

Simulation details

We used the atomic simulation environment (ASE) package coupled with our NNIP to run simulations. All periodic structures with silicates and water molecules are generated using Packmol and rendered using OVITO. The simulation boxes were generated with the equilibrium density of water predicted by our NNIP of 1.08 g/cc. All MD simulations were run with canonical (NVT) ensemble with Nosé-Hoover thermostat and timestep of 0.5 fs.

Radial distribution functions of water

We obtain the oxygen-oxygen (gOOg_{\mathrm{OO}}) and oxygen-hydrogen (gOHg_{\mathrm{OH}}) RDFs from 100 ps long MD simulations of a periodic box containing 400 H2O\mathrm{H_2O} molecules in canonical (NVT) ensemble at 300 K after 20 ps equilibration period. We divide the whole trajectory into ten parts and take the average RDF curve as the estimated one.

Diffusion coefficients of water

The diffusion coefficients were calculated from the MSDs by Einstein’s diffusion equation as given in equation 4

D=MSD6Δt=⟨∣x(t+Δt)−x(t)∣2⟩6Δt(4)D = \frac{MSD}{6\Delta t} = \frac{\left\langle\lvert x(t+\Delta t)-x(t)\rvert^2\right\rangle}{6\Delta t} \tag*{(4)}

Where D is the diffusion coefficient, Δt\Delta t is the time interval used for measuring MSDs. We used 10 ps as the time interval. We further obtained the slope of ln(MSD) vs ln(t) and choose the region for linear fitting where the slope is within 1±10−41 \pm10^{-4}. The MSDs were calculated based on the relative squared displacement of oxygen atoms and averaged over trajectories for all water molecules. We further needed to extrapolate the diffusion coefficient to infinite-system size owing to the finite-size effect[50]. The extrapolated diffusion coefficient (D(∞\infty)) is obtained from finite-system size diffusion coefficient (D(L)) with an extra correction term as shown in equation 5

D(∞)=D(L)+ζ6πβηL(5)D(\infty) = D(L) + \frac{\zeta}{6\pi\beta\eta L} \tag*{(5)}

Where L is the simulation box size, ζ\zeta depends on the geometry of the simulation box (2.837297 for a cubic box), β\beta is 1kBT\frac{1}{k_{\mathrm{B}}T}, and η\eta is experimental shear viscosity = 0.8925 x 10−310^{-3} Pa s. We calculated the diffusion coefficients by first running an NVT simulation for 50 ps to equilibrate the system and then ran five independent NVE simulations for 150 ps at different temperatures of 260, 280, 300, 320, 340, and 360 K and chose the average MSDs.

Vibrational density of states of water

The VDOS of water can be procured from the Fourier transform of the velocity-velocity autocorrelation function as given in equation 6

Cvv(ω)=∫⟨v(τ)v(τ+t)⟩τe−iωt dt(6)C_{vv}(\omega) = \int\left\langle v(\tau)v(\tau+t)\right\rangle_{\tau}e^{-i\omega t}\,dt \tag*{(6)}

Where v(τ\tau) is the centroid velocity of the atom and ω\omega is the vibrational frequency. The RPMD simulations were implemented using SchnetPack[51] with 5 beads and 0.25fs timestep.

Equilibrium density of water

We ran MD simulations in NPT (constant pressure) ensemble using the isobaric-isothermal form of the Nosé–Hoover dynamics at a temperature of 300K and a pressure of 1 atm. We chose a cubic box of 400 water molecules and the box was allowed to relax in a hydrostatic manner. A 1 ns long trajectory after 100 ps of equilibration period was divided into ten parts and the time-averaged volume of the simulation box was taken as equilibrium volume. The equilibrium density was then calculated from it.

Elastic constants of silicates

We provided normal strains of -1% to 1% in steps of 0.5% and shear strains of -4% to 4% in steps of 2% to the equilibrium structures of α\alpha-Quartz and FAU zeolite and obtained their deformed structures using Pymatgen. We then optimized each deformed structure by the BFGS algorithm implemented in the ASE package. We then obtained the stresses at different strains and their corresponding slopes to measure the elastic tensors for each system. We then used ELATE software[39] to obtain elastic constants from the tensors. We used our NNIP ensemble to procure the average elastic constants and their standard deviation.

Chemical reactions

We performed constrained MD simulations with umbrella sampling varying our reaction coordinates chosen for different reactions. We then obtained a smooth reaction path from reactant to product state by calculating the PMF as a function of our chosen reaction coordinates. The PMF was calculated using the Multistate Bannett’s Acceptance Ratio (MBAR)[52] on the sampled trajectories. The MBAR analysis was carried out using codes implemented in adaptive sampling package[53, 54]. We can use the PMF of the deprotonation reactions of water and silicates to obtain their pKa\mathrm{p}K_a values. The difference in the PMF of reactant and product phase (ΔF\Delta F) is related to the acid dissociation constant (KcK_c) as shown in equation 7 and the relation between pKa\mathrm{p}K_a and KcK_c is shown in equation 8, Combining both we can get pKa\mathrm{p}K_a from ΔF\Delta F as shown in equation 9.

ΔF=−kBTln⁡kc(7)\Delta F = -k_B T \ln k_c \tag*{(7)}
pKa=−log⁡kc(8)\mathrm{p}K_a = -\log k_c \tag*{(8)}
pKa=ΔFkBT×2.3026(9)\mathrm{p}K_a = \frac{\Delta F}{k_B T \times2.3026} \tag*{(9)}

Where kBk_B is the Boltzmann constant.

Methodology

Acknowledgement

R.G-B. and S.R. acknowledge funding support from Evonik AG and the MIT-IBM Watson AI Lab. We acknowledge the MIT Engaging cluster at the Massachusetts Green High-Performance Computing Center (MGHPCC) for providing high-performance computing resources.

Data Availability

The silica-water data set can be obtained from the authors on request and the NNIP model generated during this study is available at Materials Cloud Archive under accession code https://doi.org/10.24435/materialscloud:61-x8. The codes used for this study can be downloaded from https://github.com/learningmatter-mit/NeuralForceField.

References

Supplementary Materials

References

  1. [1]Heaney, P. J., Prewitt, C. T. & Gibbs, G. V. Silica: Physical behavior, geochemistry, and materials applications (Walter de Gruyter GmbH & Co KG, 2018).
  2. [2]Weitkamp, J. Zeolites and catalysis. Solid state ionics 131, 175–188 (2000).
  3. [3]Gao, Y. et al. Multifunctional Role of Silica in Pharmaceutical Formulations. AAPS PharmSciTech 2022 23:4, 23, 1–18 (Mar. 2022).
  4. [4]Halas, N. J. Nanoscience under glass: The versatile chemistry of silica nanostructures. ACS Nano 2, 179–183 (Feb. 2008).DOI
  5. [5]Bergna, H. & Roberts, W. Colloidal silica: fundamentals and applications (2005).
  6. [6]Raza, N. et al. Synthesis and characterization of amorphous precipitated silica from alkaline dissolution of olivine. RSC Advances 8, 32651–32658 (Sept. 2018).DOI
  7. [7]Dewati, R. et al. Precipitated Silica from Pumice and Carbon Dioxide Gas (Co2) in Bubble Column Reactor. Journal of Physics: Conference Series 953, 012226 (Jan. 2018).DOI
  8. [8]Cundy, C. S. & Cox, P. A. The hydrothermal synthesis of zeolites: History and development from the earliest days to the present time. Chemical Reviews 103, 663–701 (2003).
  9. [9]Rai, D. K., Beaucage, G., Vogtt, K., Ilavsky, J. & Kammler, H. K. In situ study of aggregate topology during growth of pyrolytic silica. Journal of Aerosol Science 118, 34–44 (Apr. 2018).DOI
  10. [10]Meier, M., Sonnick, S., Asylbekov, E., Rädle, M. & Nirschl, H. Multi-scale characterization of precipitated silica. Powder Technology 354, 45–51 (Sept. 2019).DOI
  11. [11]Van Beest, B. W., Kramer, G. J. & Van Santen, R. A. Force fields for silicas and aluminophosphates based on ab initio calculations. Physical Review Letters 64, 1955 (Apr. 1990).
  12. [12]Carré, A., Horbach, J., Ispas, S. & Kob, W. New fitting scheme to obtain effective potential from Car-Parrinello molecular-dynamics simulations: Application to silica. Europhysics Letters 82, 17001 (Mar. 2008).DOI
  13. [13]Flikkema, E. & Bromley, S. T. A new interatomic potential for nanoscale silica. Chemical Physics Letters 378, 622–629 (Sept. 2003).
  14. [14]Fogarty, J. C., Aktulga, H. M., Grama, A. Y., Van Duin, A. C. & Pandit, S. A. A reactive molecular dynamics simulation of the silica-water interface. The Journal of Chemical Physics 132, 174704 (May 2010).
  15. [15]Rimsza, J. M., Yeon, J., Van Duin, A. C. & Du, J. Water Interactions with Nanoporous Silica: Comparison of ReaxFF and ab Initio based Molecular Dynamics Simulations. Journal of Physical Chemistry C 120, 24803–24816 (Nov. 2016).DOI
  16. [16]Musil, F. et al. Physics-inspired structural representations for molecules and materials. Chemical Reviews 121, 1–60 (Aug. 2021).DOI
  17. [17]Axelrod, S. et al. Learning Matter: Materials Design with Machine Learning and Atomistic Simulations. Accounts of Materials Research 3, 343–357 (Mar. 2022).DOI
  18. [18]Yao, Y. & Kanai, Y. Nuclear Quantum Effect and Its Temperature Dependence in Liquid Water from Random Phase Approximation via Artificial Neural Network. Journal of Physical Chemistry Letters 12, 6354–6362 (July 2021).
  19. [19]Liu, J., Lan, J. & He, X. Toward High-level Machine Learning Potential for Water Based on Quantum Fragmentation and Neural Networks. Journal of Physical Chemistry A 126, 3926–3936 (June 2022).DOI
  20. [20]Zaverkin, V., Holzmüller, D., Schuldt, R. & Kästner, J. Predicting properties of periodic systems from cluster data: A case study of liquid water. Journal of Chemical Physics 156, 114103 (Mar. 2022).
  21. [21]Erhard, L. C., Rohrer, J., Albe, K. & Deringer, V. L. A machine-learned interatomic potential for silica and its relation to empirical models. npj Computational Materials 2022 8:1 8, 1–12 (Apr. 2022).
  22. [22]Balyakin, I. A., Rempel, S. V., Ryl’tsev, R. E. & Rempel, A. A. Deep machine learning interatomic potential for liquid silica. Physical Review E 102, 52125 (Nov. 2020).DOI
  23. [23]Erlebach, A., Nachtigall, P. & Grajciar, L. Accurate large-scale simulations of siliceous zeolites by neural network potentials. arXiv preprint arXiv:2102.12404, 1–33 (Feb. 2021).DOI
  24. [24]Trinh, T. T., Jansen, A. P. & Van Santen, R. A. Mechanism of oligomerization reactions of silica. Journal of Physical Chemistry B 110, 23099–23106 (Nov. 2006).
  25. [25]Zhang, X. Q., Trinh, T. T., Van Santen, R. A. & Jansen, A. P. Mechanism of the initial stage of silicate oligomerization. Journal of the American Chemical Society 133, 6613–6625 (May 2011).DOI
  26. [26]Schaffer, C. L. & Thomson, K. T. Density functional theory investigation into structure and reactivity of prenucleation silica species. Journal of Physical Chemistry C 112, 12653–12662 (Aug. 2008).DOI
  27. [27]Pereira, J. C., Catlow, C. R. & Price, G. D. Silica condensation reaction: an ab initio study. Chemical Communications, 1387–1388 (Jan. 1998).DOI
  28. [28]Elanany, M. et al. A quantum molecular dynamics simulation study of the initial hydrolysis step in sol-gel process. Journal of Physical Chemistry B 107, 1518–1524 (Feb. 2003).DOI
  29. [29]Schwalbe-Koda, D., Tan, A. R. & Gómez-Bombarelli, R. Differentiable sampling of molecular geometries with uncertainty-based adversarial attacks. Nature Communications 12, 1–12 (Aug. 2021).
  30. [30]Ch. Baerlocher and L.B. McCusker. Database of Zeolite Structures. http://www.iza-structure.org/databases/ (2021).
  31. [31]Morawietz, T., Singraber, A., Dellago, C. & Behler, J. How van der waals interactions determine the unique properties of water. Proceedings of the National Academy of Sciences of the United States of America 113, 8368–8373 (July 2016).
  32. [32]Shrikumar, A., Greenside, P. & Kundaje, A. Learning important features through propagating activation differences. 34th International Conference on Machine Learning, ICML 7, 4844–4866 (2017).
  33. [33]Sundararajan, M., Taly, A. & Yan, Q. Axiomatic Attribution for Deep Networks. 34th International Conference on Machine Learning, ICML 7, 5109–5118 (Mar. 2017).
  34. [34]Fu, X. et al. Forces are not Enough: Benchmark and Critical Evaluation for Machine Learning Force Fields with Molecular Simulations (Oct. 2022).arxiv.org/abs/2210.07237
  35. [35]Skinner, L. B. et al. Benchmark oxygen-oxygen pair-distribution function of ambient water from x-ray diffraction measurements with a wide Q-range. 138, 074506 (Feb. 2013).DOI
  36. [36]Marsalek, O. & Markland, T. E. Quantum Dynamics and Spectroscopy of Ab Initio Liquid Water: The Interplay of Nuclear and Electronic Quantum Effects. Journal of Physical Chemistry Letters 8, 1545–1551 (Apr. 2017).
  37. [37]Holz, M., Heil, S. R. & Sacco, A. Temperature-dependent self-diffusion coefficients of water and six selected molecular liquids for calibration in accurate 1H NMR PFG measurements. Physical Chemistry Chemical Physics 2, 4740–4742 (Jan. 2000).DOI
  38. [38]Ceriotti, M., Parrinello, M., Markland, T. E. & Manolopoulos, D. E. Efficient stochastic thermostatting of path integral molecular dynamics. The Journal of Chemical Physics 133, 124104 (Sept. 2010).DOI
  39. [39]Gaillac, R., Pullumbi, P. & Coudert, F. X. ELATE: an open-source online application for analysis and visualization of elastic tensors. Journal of Physics: Condensed Matter 28, 275201 (May 2016).DOI
  40. [40]Silverstein, T. P. & Heller, S. T. PKa Values in the Undergraduate Curriculum: What Is the Real pKa of Water? Journal of Chemical Education 94, 690–695 (June 2017).DOI
  41. [41]Ceriotti, M. et al. Nuclear Quantum Effects in Water and Aqueous Systems: Experiment, Theory, and Current Challenges. Chemical Reviews 116, 7529–7550 (July 2016).
  42. [42]Wang, R., Carnevale, V., Klein, M. L. & Borguet, E. First-Principles Calculation of Water p Ka Using the Newly Developed SCAN Functional. Journal of Physical Chemistry Letters 11, 54–59 (Jan. 2020).
  43. [43]Perry, C. C. Biogenic Silica: A Model of Amorphous Structure Control. Growth, Dissolution and Pattern Formation in Geosystems, 237–251 (1999).DOI
  44. [44]Belton, D. J., Deschaume, O. & Perry, C. C. An overview of the fundamentals of the chemistry of silica with relevance to biosilicification and technological advances. The FEBS journal 279, 1710–1720 (May 2012).DOI
  45. [45]Schütt, K. T., Unke, O. T. & Gastegger, M. Equivariant message passing for the prediction of tensorial properties and molecular spectra. arXiv preprint arXiv:2102.03150 (2021).arxiv.org/abs/2102.03150
  46. [46]Batzner, S. et al. E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials. Nature Communications 13, 1–11 (Jan. 2021).
  47. [47]Musaelian, A. et al. Learning Local Equivariant Representations for Large-Scale Atomistic Dynamics (Apr. 2022).
  48. [48]Schütt, K. T. et al. SchNet: A continuous-filter convolutional neural network for modeling quantum interactions. Advances in Neural Information Processing Systems 2017-Decem, 992–1002 (2017).arxiv.org/abs/1706.08566
  49. [49]Subramaniyan, A. K. & Sun, C. T. Continuum interpretation of virial stress in molecular simulations. International Journal of Solids and Structures 45, 4340–4346 (July 2008).DOI
  50. [50]Yeh, I. C. & Hummer, G. System-size dependence of diffusion coefficients and viscosities from molecular dynamics simulations with periodic boundary conditions. Journal of Physical Chemistry B 108, 15873–15879 (Oct. 2004).DOI
  51. [51]Schütt, K. T. et al. SchNetPack: A Deep Learning Toolbox for Atomistic Systems. Journal of Chemical Theory and Computation 15, 448–455 (Jan. 2019).DOI
  52. [52]Shirts, M. R. & Chodera, J. D. Statistically optimal analysis of samples from multiple equilibrium states (2008).DOI
  53. [53]Dietschreit, J. C. B., Diestler, D. J., Hulm, A., Ochsenfeld, C. & Gómez-Bombarelli, R. From Free-Energy Profiles to Activation Free Energies (June 2022).
  54. [54]Hulm, A., Dietschreit, J. C. & Ochsenfeld, C. Statistically optimal analysis of the extended-system adaptive biasing force (eABF) method. Journal of Chemical Physics 157, 24110 (July 2022).

Paper details

vlnc:W_pup4mdn7
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents