WorksPhysics and Astronomy
Predicting tensorial molecular properties with equivariant machine-learning models
Abstract
Embedding molecular symmetries into machine-learning models is key for efficient learning of chemico-physical scalar properties, but little evidence on how to extend the same strategy to tensorial quantities exists. Here we formulate a scalable equivariant machine-learning model based on local atomic environment descriptors. We apply it to a series of molecules and show that accurate predictions can be achieved for a comprehensive list of dielectric and magnetic tensorial properties of different ranks. These results show that equivariant models are a promising platform to extend the scope of machine learning in materials modelling.
Machine learning is revolutionizing materials science by enabling applications such as molecular properties prediction[1, 2], accelerated design of new compounds[3, 4, 5, 6], and automated AI-driven laboratories[7]. The success of machine learning is due to its ability to understand the underlying features of a distribution of data from examples drawn from it, even in absence of physical insights on the problem[8, 9]. However, large improvements in the predictive power of the model are achieved by making it aware of the data’s properties[10]. In the context of learning molecular properties, this is realized by including symmetries into the model’s architecture. Invariance by rigid translations or swaps among identical atoms, and rotational equivariance are topical examples.
A general physical property is described by a combination of spherical tensors of order and -components. When an arbitrary rotation is applied to a molecule with coordinates , the tensor rotates as
where is the Wigner -matrix. Eq. (1) expresses the rotational equivariance condition that any physical property must obey. In the case of scalar quantities (), Eq. (1) reduces to the definition of invariance. This latter scenario has been thoroughly investigated in the context of machine learning force fields[11] and molecular properties prediction[12, 13, 14], where atomic environments are described in terms of translationally and rotationally invariant functions[10, 15, 16, 17, 18, 19, 20, 21]. However, only a few attempts at designing machine-learning models able to capture equivariance for tensorial properties so far exist.[22, 23, 24, 25, 26]
Gaussian Process Regression (GPR) has been the first machine-learning model to be applied to this problem by means of a symmetry-adapted definition of the Smooth Overlap of Atomic Position (SOAP) kernel[22, 23]. In this contribution, we will instead focus on a different machine-learning architecture, with the aim of identifying the principles for equivariance learning of tensorial properties amenable to both linear and deep learning-based models. In ref. [24], it was proposed to extend the spectral neighbour analysis potential framework[27, 28]. Accordingly, a tensorial property is decomposed in a sum of atomic tensors, each one linearly dependent on the atom’s bispectrum components[29], i.e. rotationally and translationally invariant descriptors of an atomic environment. Given the order of the tensor , the equivariance was then obtained by determining sets of coefficients and by reorienting any given molecular structure with respect to a reference one. Although successfully applied to the prediction of spin-phonon coupling coefficients[24, 25, 30], this approach is hard to generalize to multi-molecule data sets or to condensed-phase.
In this contribution, we explore the possibility to impose the equivariance condition by augmenting invariant atomic features with spherical harmonics. The latter naturally transform as in Eq. (1), , and therefore, provide a more natural basis for decomposing the tensorial property. This ansatz is also the basis of equivariant models recently appeared in literature. Euclidean neural networks[31, 32, 33, 34, 35], moment-tensor potentials[36], and Gaussian-moment neural networks (GMNN)[26] are a few notable examples[37]. The basic idea underlying these methods is to use contractions of high-order tensors, either spherical or Cartesian, to build invariant features. Once combined with convolutional neural networks (CNN), they have shown improved performance with respect to conventional deep learning models for the prediction of scalar quantities[38]. Interestingly, equivariant models have also been applied to the learning of Hamiltonian operators and wave-functions, which could in principle be used to predict molecular properties[39, 40, 41]. However, despite their ideal features, these models have never been applied to the direct learning of tensorial properties. The only exception is represented by GMNN, which have only recently been applied to the learning of the magnetic anisotropy tensor[26].
Inspired by these works, we here demonstrate the generality and accuracy of this framework. Instead of using deep-learning architectures, we decided to use linear regression to introduce our method for a two-fold reason. On the one hand, this makes it possible to clearly individuate the principles underlying equivariant models for tensors by removing the dependency of results with respect to the model’s complexity. This comes at no loss of generality, as the model can be readily scaled-up with neural networks (vide infra). On the other hand, linear machine-learning models have shown an excellent compromise between learning rate and accuracy with respect to complex architectures[28], which instead perform better for large data sets[42].
Following the approach of ref. [24], we write each component of the spherical tensor as a sum of atomic tensors, where is the -th atom’s contribution in a system with atoms. The total contribution reads
where runs over bispectrum components, , and are coefficients that need to be determined. The use of complex or real coefficients does not change the results. are the spherical harmonics of the -th atom’s environment and are defined as , where runs over neighboring atoms within from atom , is the standard definition of a complex spherical harmonics, and are the coordinates of the atom rescaled by the coordinates of the atom . The bispectrum components for each atom are also computed up to the same cutoff radius . The coefficients are determined by minimizing the root mean squared error (RMSE) with respect to a training set of reference values . Here we use ridge regression, which includes a -regularization. The definitions of and enforce the model’s translational invariance, while the symmetry with respect to the swap of identical atoms is imposed by using the same set of coefficients for atoms of the same chemical element. The sum over atomic contributions in Eq. (2) makes the model independent on the atoms’ order. Finally, the model also satisfies the equivariance condition of Eq. (1), as the right-hand side automatically transforms as a spherical tensor. This is made possible by the fact that the coefficients multiplying do not depend on the index .
In order to illustrate the power of our model, we apply it to series of dielectric and magnetic molecular properties of key importance for various spectroscopies, namely the dipole moment, the polarizability tensor, magnetic anisotropy and the coefficients of effective crystal field Hamiltonian. The dipole moment and the polarizability tensor are needed for the modelling of infra-red and Raman spectra[43, 44], while the molecular magnetic anisotropy and the effective crystal field Hamiltonian underpin electron paramagnetic resonance (EPR) spectroscopy and spin relaxation[30, 45, 46].
Let us begin by discussing the results over dielectric properties for three data sets presented in ref. [23]: a water monomer, a water dimer, and the Zundel cation.
The structure of these compounds is reported in Figure 1. The target properties for our model are the dipole moment vector , which is equivariant to a spherical tensor with (), and the polarizability tensor , which is equivalent to the sum of two spherical tensors with and , respectively. Here we focus on learning the non-scalar component with (). The three data sets each contains 1000 configurations with arbitrary distortion and orientation in space[23]. Both the dipole and the polarizability tensor were computed at the CCSD/6d-aug-cc-pvtz level. Figure 1b shows the learning curve our model for the water monomer data set, namely the plot of the test set’s RMSE as function of the number of training points. The learning curve for water dimer and Zundel cation data sets show similar trends and they are reported in Fig. S1. It is clear that with more training points, the test error goes down and the accuracy of the model’s prediction improves. Although a thorough comparison of different methods is beyond the scope of this work, we found that symmetry-adapted Gaussian process regression (SA-GPR)[23] performs similarly or slightly better than our linear model (see Table S1).

Figure 1. Learning of dielectric properties. (a) The molecular structure of (i) the water monomer, (ii) the water dimer, and (iii) the Zundel cation. The Oxygen atom is in red and Hydrogen atoms are in white. (b) The learning curve of our model over the water monomer data set for the polarizability spherical tensor and the dipole spherical tensor . The test set always contains 500 configurations and all quantities are expressed in atomic units.
Next, we apply our model to the magnetic properties of coordination compounds. The target property for the following tests is the magnetic anisotropy tensor , which can be accurately computed using Complete Active Space (CASSCF) and valence state perturbation theory (NEVPT2) methods[47]. is a symmetric trace-less tensor and can thus be converted into a spherical tensors of order .
Firstly, we study a data set that contains only local distortions of a single molecular structure, namely the one of the top-performance single molecule magnet CoL (with HL=1,2-bis- (methanesulfonamido)benzene)[48]. The data set, presented in ref. [24], contains at total of 1500 configurations obtained by applying a random distortion of maximum displacements of Å to each atom’s Cartesian components. 1200 configurations were used to train the model, 150 were used as validation to tune the hyper-parameters, and 150 were used as test set. Figure 2a illustrates the accuracy of our model in predicting the magnetic anisotropy, with an error of 1.6 cm for all . Most importantly, it should be noted that the orientation of the configurations in the training and test sets does not affect the accuracy of the model. The error of the model remains identical when the configurations in the data set are all oriented in the same direction or when the configurations are randomly rotated. To further demonstrate that our method is in fact equivariant, a configuration of the test set is randomly rotated by an angle . The predicted spherical tensor for this configuration is rotated using the corresponding Wigner -matrix. Then, our model is used to predict the tensor

Figure 2. Learning of magnetic anisotropy. (a) Plot of the predicted spherical tensor’s components, , using our model against spherical tensors calculated from the tensor obtained by CASSCF calculation for different configurations of CoL. (b) Plot of the predicted spherical tensor component with when the molecule is rotated by an angle and comparison to the rotated tensor component calculated by applying the Wigner -matrix to the predicted tensor of the unrotated structure . The inset shows the molecular structure of CoL with the Co atom in pink, the N atoms in blue, the S atoms in yellow, the O atoms in red, the C atoms in cyan, and the H atoms in white. The plots for all other spherical tensor components are provided as Supplementary Information.
values for the rotated configuration . Based on Eq. (1), these two values should be identical for an equivariant model, which is indeed the case (see Figure 2b and Fig. S8).
Next we apply our model to a data set that contains multiple molecules. The data set includes the six molecules [Co(HO)], [Co(HO)], [Co(HO)], [Co(HO)], [Co(HO)], and [Co(HO)]. Each molecule was randomly distorted 100 times by applying a maximum displacements of Å to each atom’s Cartesian components. 80 configurations per molecules were used for training and 20 configurations per molecule as test set. The tensor was then computed with CASSCF for each molecular frame. The accuracy of our predictions is illustrated in Figure 3a, with an error of 5.9 cm for all components. Similarly to the previous test with CoL, the accuracy of the model does not depend on the specific orientation of the molecules. To further test the reliability of the model’s predictions, we use it to predict how the tensor changes as function of small distortions of the Co ion in [Co(HO)]. Although none of these configurations explicitly appear in the training set, Fig. S4 shows that the correct profile of as a function of the atomic displacement is well reproduced. The positive outcome of this test enables applications in spin relaxation, where the derivatives of are needed[30, 49]. As a second test, we re-train the model by adding an additional spectator water molecule at great distance from the Co complexes. We compute the contribution to the total axial magnetic anisotropy for a water molecule bonded to one of the Co(HO) molecules and for the spectator one. Fig. S3 shows that the model correctly captures the physics of the problem by assigning a vanishing value, within the model’s RMSE, to the non-bonded water molecule, and a finite value to the water molecules coordinating the Co ion.

Figure 3. (a) Plot of predicted spherical tensors using our model against spherical tensors calculated from CASSCF tensor values of the Co(HO) set. (b) The molecular structure of (i) [Co(HO)], (ii) [Co(HO)], (iii) [Co(HO)], (iv) [Co(HO)], (v) [Co(HO)], and (vi) [Co(OH)] where the Co atom is coloured in cyan, the O atoms are coloured in red, and the H atoms are coloured in white.
Next we present the results for the magnetic properties of the [Dy(HO)] coordination complex, whose structure is depicted in the inset of Figure 4. The Dy ion posses 5 unpaired electrons in the -shell and an unquenched electronic angular momentum, resulting in a ground state with a total angular moment . The 16 electronic levels associated with this multiplet can be described with the effective crystal field Hamiltonian
!Learning curves for crystal field parameters and molecular structure of [Dy(H2O)9]3+ bbox=0.16,0.43,0.47,0.74
Figure 4. Learning of crystal field parameters. The learning curve of our model over the [Dy(HO)] data set for the spherical tensors of crystal field parameters of order . The test set always contains 430 configurations. Continuous lines and filled circles refer to the results of the linear model, dashed lines and filled circles to the results obtained with a neural network architecture. The inset shows the molecular structure of [Dy(HO)] where the Dy atom is in cyan, the O atoms are in red, and H atoms are in white.
where are coefficients adjusted to reproduce CASSCF simulations[50]. The operators are tesseral tensor operators, which can be transformed into spherical tensor operators[51]. We prepare a training set of over 3000 entries by distorting the DFT-optimized structure of Dy[HO]. Random distortions of a maximum value of Å are applied to each atom’s Cartesian coordinates. The crystal field coefficients appearing in Eq. (3) are then computed with CASSCF methods as detailed in the ESI. We apply our method to the coefficients , converted into the spherical components, and report the learning curve in Figure 4, demonstrating that tensors up to the sixth order can be well reproduced. Graphical representation of predictions vs reference data and proof of model equivariance for this data set are reported in Figs. S5 and S9-S11.
Finally, we use the Dy[HO] data sets to also show that the model of Eq. (2) can be easily generalized to deep learning architectures by feeding the bispectrum components, , to a neural network with weights/biases , and real scalar output (see Fig. S6)
Results for a model with one independent neural net for each chemical species, each with three hidden layers and 32:32:16 nodes, are reported in Figure 4 and Fig. S7 and highlight the fact that the linear model provides better results for a small training set. However, as expected, as larger training sets are employed, the deep-learning model is able to keep leaning from data, while the performance of the linear model reaches a plateau due to the limited flexibility of its architecture. Similar results are obtained for tensors of higher order and for neural networks with different numbers of layers or nodes (see ESI). The excellent results obtained with Eq. (4) demonstrate the generality of our approach, where models designed to predict scalar properties can be extended to the case of tensors by simply learning the coefficients of a combination of spherical harmonics functions.
In conclusion, we have presented a general procedure to build equivariant machine-learning models able to predict tensorial properties. Even in its simple linear form, the model accurately predicts properties of different type and rank, and it is able to handle realistic compounds as well as multi-molecule data sets. Moreover, we have shown that higher accuracy can be achieved be using deep-learning architectures and large data sets. This model can find immediate application in magnetism[46, 52], magnetic resonance[25, 53], and vibrational spectroscopy[44], or be further optimized by combining it with the wealth of methods already developed for the prediction of scalar properties[1, 2, 10, 12].
Acknowledgements
This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. [948493]). Computational resources were provided by the Trinity College Research IT and the Irish Centre for High-End Computing (ICHEC).
References
References
- [1]K. T. Butler, D. W. Davies, H. Cartwright, O. Isayev, and A. Walsh, Nature 559, 547–555 (2018).
- [2]P. Friederich, F. Häse, J. Proppe, and A. Aspuru-Guzik, Nat. Mater. 20, 750 (2021).
- [3]P. Raccuglia, K. C. Elbert, P. D. F. Adler, C. Falk, M. B. Wenny, A. Mollo, M. Zeller, S. A. Friedler, J. Schrier, and A. J. Norquist, Nature 533, 73–76 (2016).
- [4]J. E. Saal, A. O. Oliynyk, and B. Meredig, Annu. Rev. Mater. Res. 50, 49 (2020).
- [5]C. Suh, C. Fare, J. A. Warren, and E. O. Pyzer-Knapp, Annu. Rev. Mater. Res. 50, 1 (2020).
- [6]H. Zhang, K. Hippalgaonkar, T. Buonassisi, O. M. Lovvik, E. Sagvolden, D. Ding, and et al., ES Energy & Environment (2018).
- [7]A. F. de Almeida, R. Moreira, and T. Rodrigues, Nat. Rev. Chem. 3, 589–604 (2019).
- [8]A. F. de Almeida, R. Moreira, and T. Rodrigues, J. Chem. Phys. 103, 4129 (1995).
- [9]S. Lorenz, A. Groß, and M. Scheffler, Chem. Phys. Lett. 395, 210 (2004).
- [10]J. Behler and M. Parrinello, Phys. Rev. Lett. 98 (2007).
- [11]I. Poltavsky and A. Tkatchenko, J. Phys. Chem. Lett. 12, 6551–6564 (2021).
- [12]K. T. Schütt, P. Kessel, M. Gastegger, K. A. Nicoli, A. Tkatchenko, and K.-R. Müller, J. Chem. Theory Comput. 15, 448 (2019).
- [13]F. M. Paruzzo, A. Hofstetter, F. Musil, S. De, M. Ceriotti, and L. Emsley, Nat. Commun. 9, 4501 (2018).
- [14]K. T. Schütt, M. Gastegger, A. Tkatchenko, K.-R. Müller, and R. J. Maurer, Nat. Commun. 10, 5024 (2019).
- [15]T. W. Ko, J. A. Finkler, S. Geodecker, and J. Behler, Nat. Commun. 12 (2021).
- [16]Y. Shao, M. Hellström, P. D. Mitev, L. Knijff, and C. Zhang, J. Chem. Inf. Model. 60, 1184 (2020).
- [17]S. Chmiela, H. E. Sauceda, I. Poltavsky, K.-R. Müller, and A. Tkatchenko, Comput. Phys. Commun. 240, 38 (2019).
- [18]X. Gao, F. Ramezanghorbani, O. Isayev, J. S. Smith, and A. E. Roitberg, J. Chem. Inf. Model 60, 3408 (2020).
- [19]C. Devereux, J. S. Smith, K. K. Huddleston, K. Barros, R. Zubatyuk, O. Isayev, and A. E. Roitberg, J. Chem. Theory Comput. 16, 4192 (2020).
- [20]J. S. Smith, B. T. Nebgen, R. Zubatyuk, N. Lubbers, C. Devereux, K. Barros, S. Tretiak, O. Isayev, and A. E. Roitberg, Nat. Commun 10 (2019).
- [21]J. S. Smith, B. Nebgen, and N. Lubbers, J. Chem. Phys. 148 (2018).
- [22]A. Gleimlo, P. Sollich, and A. De Vita, Phys. Rev. B 95, 214302 (2017).
- [23]A. Grisafi, D. M. Wilkins, G. Csányi, and M. Ceriotti, Phys. Rev. Lett. 120 (2018).
- [24]A. Lunghi and S. Sanvito, J. Phys. Chem. C 124, 5802–5806 (2020).
- [25]A. Lunghi, Appl. Mag. Reson. 51, 1343 (2020).
- [26]V. Zaverkin, J. Netz, F. Zills, A. Köhn, and J. Kästner, J. Chem. Theory Comput. (2021).
- [27]A. P. Thompson, L. P. Swiler, C. R. Trott, S. M. Foiles, and G. J. Tucker, J. Comput. Phys. 285, 316 (2015).
- [28]A. Lunghi and S. Sanvito, Sci. Adv. 5 (2019).
- [29]A. P. Bartók, R. Kondor, and G. Csányi, Phys. Rev. B 87 (2013).
- [30]A. Lunghi and S. Sanvito, J. Phys. Chem. Lett. 11, 6273 (2020).
- [31]N. Thomas, T. Smidt, S. Kearnes, L. Yang, L. Li, K. Kohlhoff, and P. Riley, “Tensor field networks: Rotation- and translation-equivariant neural networks for 3d point clouds,” (2018), arXiv:1802.08219.arxiv.org/abs/1802.08219
- [32]M. Weiler, M. Geiger, M. Welling, W. Boomsma, and T. Cohen, “3d steerable cnns: Learning rotationally equivariant features in volumetric data,” (2018), arXiv:1807.02547.arxiv.org/abs/1807.02547
- [33]R. Kondor, Z. Lin, and S. Trivedi, in NeurIPS (2018) pp. 10138–10147.
- [34]B. Anderson, T.-S. Hy, and R. Kondor, “Cormorant: Covariant molecular neural networks,” (2019), arXiv:1906.04015.arxiv.org/abs/1906.04015
- [35]T. Cohen and M. Welling, in Proceedings of The 33rd International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 48, edited by M. F. Balcan and K. Q. Weinberger (PMLR, New York, New York, USA, 2016) pp. 2990–2999.
- [36]A. V. Shapeev, Multiscale Model. Simul. 14, 1153–1173 (2016).
- [37]M. F. Langer, A. Goëßmann, and M. Rupp, “Representations of molecules and materials for interpolation of quantum-mechanical simulations via machine learning,” (2021), arXiv:2003.12081 [physics.comp-ph].DOI
- [38]B. K. Miller, M. Geiger, T. E. Smidt, and F. Noé, “Relevance of rotationally equivariant convolutions for predicting molecular properties,” (2020), arXiv:2008.08461 [cs.LG].arxiv.org/abs/2008.08461
- [39]L. Zhang, B. Onat, G. Dusson, G. Anand, R. J. Maurer, C. Ortner, and J. R. Kermode, arXiv preprint arXiv:2111.13736 (2021).DOI
- [40]O. Unke, M. Bojorgeski, M. Gastegger, M. Geiger, T. Smidt, and K.-R. Müller, Advances in Neural Information Processing Systems 34 (2021).
- [41]J. Nigam, M. J. Willatt, and M. Ceriotti, J. Chem. Phys. 156, 014115 (2022).
- [42]M. M. Najafabadi, F. Villanustre, T. M. Khoshgoftaar, N. Seliya, R. Wald, and E. Muharemagic, J. Big Data 2 (2015).
- [43]F. Perakis, L. D. Marco, A. Shalit, F. Tang, Z. R. Kann, T. D. Kühne, R. Torre, M. Bonn, and Y. Nagata, Chem. Rev. 116, 7590–7607 (2016).
- [44]S. Lubera, L. Mannuzzi, and J. Hutter, J. Chem. Phys. 141, 094503 (2014).
- [45]M. Briganti, F. Santanni, L. Tesi, F. Totti, R. Sessoli, and A. Lunghi, J. Am. Chem. Soc. 143, 13633–13645 (2021).
- [46]L. Escalera-Moreno, J. J. Baldoví, A. G.-A. no, and E. Coronado, Chem. Sci. 9, 3265 (2018).
- [47]F. Neese, M. Atanasov, G. Bistoni, D. Maganas, and S. Ye, J. Am. Chem. Soc. 141, 2814 (2019).
- [48]Y. Rechkemmer, F. D. Breitgoff, M. van der Meer, M. Atanasov, M. Hakl, M. Orlita, P. Neugebauer, F. Neese, B. Sarkar, and J. van Slageren, Nat. Commun. 7 (2016).
- [49]A. Lunghi and S. Sanvito, Journal Chem. Phys. 153, 174113 (2020).
- [50]L. F. Chibotaru and L. Ungur, J. Chem. Phys. 137, 064112 (2012).
- [51]I. D. Ryabov, J. Magn. Reson. 140, 141 (1999).
- [52]C. J. Court and J. M. Cole, Npj Comput. Mater. 6 (2020).
- [53]F. M. Paruzzo, A. Hofstetter, F. Musil, S. De, M. Ceriotti, and L. Emsley, Nat. Commun. 9, 4501 (2018).