betaFeedback

WorksPhysics and Astronomy

Physically Interpretable Machine Learning for nuclear masses

Abstract

We present a novel approach to modeling the ground state mass of atomic nuclei based directly on a probabilistic neural network constrained by relevant physics. Our Physically Interpretable Machine Learning (PIML) approach incorporates knowledge of physics by using a physically motivated feature space in addition to a soft physics constraint that is implemented as a penalty to the loss function. We train our PIML model on a random set of ∼\sim20% of the Atomic Mass Evaluation (AME) and predict the remaining ∼\sim80%. The success of our methodology is exhibited by the unprecedented σRMS∼186σ_\textrm{RMS}\sim186 keV match to data for the training set and σRMS∼316σ_\textrm{RMS}\sim316 keV for the entire AME with Z≥20Z \geq 20. We show that our general methodology can be interpreted using feature importance.

Introduction - The minimal energy required to break up a nucleus into its constituent nucleons is one of the fundamental properties of an atomic nucleus. This quantity, which is equivalent to the mass, features prominently as an input for the theoretical prediction of a number of nuclear properties which are important for both scientific and technological applications [1, 2, 3]. This effect is perhaps most apparent in the important role that masses play in predicting the reaction and decay properties of atomic nuclei [4, 5]. Masses also serve as critical inputs for the study of astrophysical phenomena, from influencing the composition of neutron star crusts to impacting heavy element synthesis and its potential observable consequences [6, 7, 8, 9, 10].

The many-body Hamiltonian that describes atomic nuclei is exceedingly complex and remains unknown, therefore the lowest energy state cannot be calculated directly from first principles for heavy nuclei. This state of affairs has led to the development of many theoretical descriptions of atomic masses, including semi-classical approaches [11], microscopic approaches [12] and more recently, models enhanced by considering improvements to model discrepancies with data [13, 14]. An inherent limitation in contemporary modeling is that the model itself remains fixed with optimization focused on parameters. This can be overcome with application of Machine Learning (ML) algorithms in which the model itself is optimized [15].

In this letter we present a novel approach to modeling masses directly from a ML model constrained by physics. Our ‘Physically Interpretable Machine Learning’ or PIML approach builds physically meaningful feature spaces and applies soft constraints to ensure relevant physics is being obeyed. It can be generalized to any problem in which physical constraints may need to be applied to a machine learned model. In a drastic improvement to our previous work [16], we train a probabilistic network on a fraction of available data and predict the vast majority of masses for thousands of nuclear species measured to date. We achieve unprecedented model accuracy and retain predictive power when extrapolating. We show that our model can be interpreted using a standard measure for feature importance, and this interpretation fits within the context of the well-established picture of atomic nuclei.

Methods - We use a probabilistic ML technique, the Mixture Density Network [17]. Our Probabilistic network is built on the PyTorch [18] framework and can be run on either CPU or GPU architectures. This type of modeling has been shown to be successful in describing nuclear properties while providing well-quantified uncertainties [19].

Lovell et al. [16] reported that a combination of macroscopic and microscopic features is suitable for describing masses across the chart of nuclides. Based on this previous analysis, we use eight features: the proton number (ZZ), the neutron number (NN), the mass number (AA), the odd-even nature of protons (ZeoZ_{\mathrm{eo}}), the odd-even nature of neutrons (NeoN_{\mathrm{eo}}), the valence number of protons as measured from the nearest closed shell (VpV_p), the valence number of neutrons as measured from the nearest closed shell (VnV_n) and a measure of isospin asymmetry (Pasym=N−ZAP_{\mathrm{asym}} = \frac{N-Z}{A}).

The last five features inform the model on quantum mechanical effects. Pairing effects manifest from the inclusion of the ZeoZ_{\mathrm{eo}} and NeoN_{\mathrm{eo}} terms which are binary, taking the value of 0 or 1. Valence terms characterize the counting of particles (or holes) between major closed neutron and proton shells. As the valence number increases up to the mid-shell, more complex excitations, including collective modes may appear [20]. The success of this picture can be related to nuclear promiscuity (a measure of the strength of proton-neutron interactions per valence nucleon) [21]. The final feature informs the model about the Pauli exclusion principle.

We take as our training set a random selection of the masses of 450 nuclei in the AME2016 [22] with Z≥20Z \ge20. The same set of 450 nuclei is fixed throughout training and does not change. The match to this data is computed with a log loss function that we denote by L1\mathcal{L}_1 (see Ref. [16]).

In addition to the physics-based feature space, we seek to encode physical constraints into model training. For this work, we chose to enforce one such possibility, the Garvey-Kelson relations [23]. This well-known series of formulas involves a judicious choice of mass differences of neighboring nuclei that minimizes the interactions between nucleons to first order, resulting in particular linear combinations that strategically sum to zero. We implement this as a soft constraint, a second loss function, L2\mathcal{L}_2, in our training. This additional loss function is calculated over the entire AME and serves as a penalty for model solutions that do not obey well-established physical law. Note that because this is a soft constraint, training may ensue that temporarily increases L2\mathcal{L}_2 in pursuit of the global minimum. We revisit this important point shortly.

The model hyperparameters are as follows: the number of hidden layers is six, the number of hidden nodes is eight, the number of Gaussian ad-mixtures is one, the weight of the physics constraint is λphys=1\lambda_{\mathrm{phys}} = 1, and we implement the Adam optimizer with initial learning rate 0.0002 [24]. To avoid overfitting, we implement regularization with a weight decay set to 0.01. These hyperparameters were determined from a select set of runs where the values were varied.

The weight of the physics constraint, λphys=1\lambda_{\mathrm{phys}} = 1 is especially noteworthy. We found that if the physics constraint was weighted too heavily (large values of $\lambda_{\mathrm{phys}}), training often failed as sharp cusps were encountered in the evolution of the total loss function which was prohibitive to optimization. As λphys\lambda_{\mathrm{phys}} tends to zero, the physics constraint becomes less influential on training and we return to the previous results of Ref. [16].

In training we seek to minimize the total log loss that consists of a sum of the loss for the match to our training set, as well as the physical constraint: Ltotal=L1+λphysL2\mathcal{L}_{\mathrm{total}} = \mathcal{L}_1 + \lambda_{\mathrm{phys}}\mathcal{L}_2. Each training epoch attempts to improve the total log loss function with respect to previous solutions. We allow this process to continue for roughly 10710^7 epochs.

The log losses for a sample training run used in this work is shown in Figure 1. The loss with respect to data, L1\mathcal{L}_1, decreases monotonically by virtue of the minimization algorithm. The loss with respect to physics, L2\mathcal{L}_2, however, exhibits a complex and highly non-linear structure. This results in a total loss function that displays similarly complex behavior, e.g. briefly rising (ca. epoch no. 10510^5), before establishing a global minimum for the entire run.

Figure 1. The log loss as a function of training epoch for the match to data, L1\mathcal{L}_1, physics constraint, L2\mathcal{L}_2 and the sum of the two, Ltotal=L1+λphysL2\mathcal{L}_{\mathrm{total}} = \mathcal{L}_1 + \lambda_{\mathrm{phys}}\mathcal{L}_2. The learning rate is 0.0002.

Figure 1 demonstrates this very general but often overlooked attribute of optimization with multiple constraints [25, 26]. If we consider epoch numbers ≈1−8×105\approx1 - 8 \times10^5, for example, we clearly see that while training based on fit to data alone is generally quite good (and improving), the same fit’s ability to satisfy basic physical requirements is quite poor by comparison. Global minima across all epochs, e.g. in the case of a different training set, may not necessarily be the final epoch at which a stop condition is reached (although the two points do coincide by happenstance in Figure 1). Consequently, it is important that past work that has applied ML in physics without implementing or considering physically motivated constraints should be approached with caution.

Results - In Figure 2, we compare the PIML model predictions against our training data (AME2016 atomic masses) over a range of neodymium isotopes with existing measured data. The entirety of the AME dataset lies within the 3σ3\sigma uncertainty interval, with a majority (all but ≈3\approx3) data points lying within a 1σ1\sigma or 2σ2\sigma interval. The overall excellent agreement with AME data shown here is indicative of the results across the entire chart of nuclides. We report a root-mean-square error σRMS\sigma_{\mathrm{RMS}} for the PIML model to be σRMS∼186\sigma_{\mathrm{RMS}} \sim186 keV for the training set (20% coverage of AME2016 data) and σRMS∼316\sigma_{\mathrm{RMS}} \sim316 keV for the entirety of AME2016 with Z≥20Z \ge20. We retain our predictive capability (σRMS∼336\sigma_{\mathrm{RMS}} \sim336 keV) when comparing to the latest 2020 release of the evaluation [27]. These results are competitive with global mass models available today.

Figure 2. Predictions of masses along the neodymium (Z=60Z = 60) isotopic chain using our PIML model. The PIML average and 11-3σ3\sigma uncertainties are shown in progressively lighter shading along with AME2016 [22] and recent data from Ref. [28]. Masses are plotted relative to FRDM2012.

Of primary interest to the nuclear physics community is not just the ability to model the properties of known nuclei (here, masses), but also the ability to apply these same models to predict the properties which currently cannot be produced or otherwise studied in laboratory settings. This has, to date, been a particularly difficult situation for nuclear theory, because while, e.g., microscopic, macroscopic-microscopic, or phenomenological approaches to modeling atomic nuclei may be applied to nuclei well outside the range of those used for parameter calibrating, it has proven challenging to develop a robust picture of the overall uncertainty in the extrapolated predictions. Recent efforts have included the application of Gaussian processes and Bayesian methods to better quantify the uncertainties of model parameters, particularly those of microscopic nuclear models (see Ref. [29, 30, 31] and references therein).

Figure 2 gives some insight into how we may proceed with regards to understanding both the quality of PIML extrapolations beyond available data, as well as how these predictions can assist in our understanding of total (statistical, systematic, and/or model) uncertainties of nuclear mass predictions. In particular we note that the results of our calculations were derived from both training and testing against AME2016 data. In the time that has since passed, there have been a number of experiments which have provided data that extends beyond the limits of this dataset, and in particular, the results of Ref. [28] extended the measurements of the neodymium isotopic chain shown in Figure 2 towards a series of more neutron-rich isotopes, up to atomic mass number A=160A = 160.

On the question of extrapolation, we see that PIML predictions (which were constructed with no information concerning the Orford et al. data) would appear to generally follow the trends in nuclear mass suggested by the additional measurements, and indeed lying well within the 2σ2\sigma uncertainty intervals. This suggests the reasonable expectation that the PIML model (or other models founded on similar principles) will reasonably extrapolate towards the more exotic, short-lived nuclei without any grotesque violation of basic nuclear physics principles, consistent with the design goals of our approach as laid out in the Methods section.

Furthermore, the PIML approach naturally provides thoroughly robust estimates with respect to the uncertainties point-wise for each individual prediction, i.e., each individual mass prediction is associated with its own, uniquely inferred uncertainty. Indeed, the uncertainties shown in Figure 2 by the shaded bands clearly reflect their expected behavior, insofar as the error bands are narrowly focused about their mean (but not unreasonably so) where AME data exists, while as the predictions begin to drift past the limits of the training/testing dataset, the same uncertainty bands begin to diverge, up to about 3σ∼13\sigma\sim1 MeV for the isotopes shown. This opens up the possibility for more thorough uncertainty quantification analyses based on this general approach, which we intend to explore in future works.

An important consequence of PIML modeling is that the output of the network may be interpreted and understood, as with any useful theoretical model. To this end, we compute Shapley Additive Explanations (SHAP) values [32] to measure feature importance. Figure 3 ranks the eight features when applied to the more recent AME2020 [27]. We find that macroscopic quantities rank the highest in determining the masses, followed by the features which control the quantum effects. Figure 3 reflects the long held belief that, to first order, the atomic nucleus is well-described by bulk, macroscopic features, while microscopic features induce subtle, yet extremely important corrections [33, 34]. This is evident when comparing a nucleus with its close neighbors, where the macroscopic features may be very nearly equal, but the quantum effects in these nuclei can lead to dramatic differences in nearly every nuclear observable.

Figure 3. The ranking of feature importance measured by SHAP value over the entire AME2020 [27]. Macroscopic-like terms rank the highest followed by the quantum effects of Pauli exclusion, valence nucleons and pairing.

Conclusions - We present a feed-forward neural network with the capacity to directly predict the nuclear binding of atomic nuclei. We achieve an unprecedented match to training data with a root-mean-square error of 186 keV (∼20%\sim20\% of the AME) while utilizing a single physical constraint and only eight parameters defining the feature space. Our model is capable of predicting all of the AME with Z≥20Z \ge20 at σRMS∼316\sigma_{\mathrm{RMS}} \sim316 keV.

In contrast to previous Machine Learning methods, our Physically Interpretable Machine Learning (PIML) approach affords the ability to analyze manifestations of physical phenomena. We demonstrate this by establishing relative importance of macroscopic and microscopic input features which is consistent with the traditional understanding of the atomic nuclear binding energy viz. the semi-empirical mass formula, as well as with macroscopic-microscopic and fully microscopic methods.

The technology developed here is general, and may in principle be applied to the study of any physical observable, or combination thereof. A concerted effort in this approach opens new possibilities to capture complex physics that is not currently viable via other contemporary approaches to computational physics. This may prove especially valuable in advancing the study of a wide range of many-body problems that permeate physics.

References

Acknowledgements

We thank P. Talou for his helpful comments and review of manuscript. M.R.M., T.M.S., A.E.L., A.T.M. were supported by the US Department of Energy through the Los Alamos National Laboratory (LANL). LANL is operated by Triad National Security, LLC, for the National Nuclear Security Administration of U.S. Department of Energy (Contract No. 89233218CNA000001).

References

  1. [1]M. Thoennessen, Plans for the facility for rare isotope beams, Nuclear Physics A 834, 688c (2010), the 10th International Conference on Nucleus-Nucleus Collisions (NN2009).DOI
  2. [2]P. Talou, I. Stetcu, and T. Kawano, Modeling the emission of prompt fission gamma-rays for fundamental physics and applications, Physics Procedia 59, 83 (2014), gAMMA-2 Scientific Workshop on the Emission of Prompt Gamma-Rays in Fission and Related Topics.DOI
  3. [3]F. Kondev, M. Wang, W. Huang, S. Naimi, and G. Audi, The NUBASE2020 evaluation of nuclear physics properties, Chinese Physics C 45, 030001 (2021).
  4. [4]T. Kawano, R. Capote, S. Hilaire, and P. Chau Huu-Tai, Statistical hauser-feshbach theory with width-fluctuation correction including direct reaction channels for neutron-induced reactions at low energies, Physical Review C 94, 10.1103/physrevc.94.014612 (2016).DOI
  5. [5]J. Randrup and R. Vogt, Generation of Fragment Angular Momentum in Fission, Phys. Rev. Lett. 127, 062502 (2021), arXiv:2103.14778 [nucl-th].DOI
  6. [6]nent accreted neutron star crust, Physical Review C 85, 10.1103/physrevc.85.055804 (2012).
  7. [7]G. W. Misch, B. A. Brown, and G. M. Fuller, Neutrino-pair emission from hot nuclei during stellar collapse, Physical Review C 88, 10.1103/phys-revc.88.015807 (2013).
  8. [8]M. Mumpower, R. Surman, G. McLaughlin, and A. Aprahamian, The impact of individual nuclear properties on r-process nucleosynthesis, Progress in Particle and Nuclear Physics 86, 86–126 (2016).DOI
  9. [9]T. M. Sprouse, R. Navarro Perez, R. Surman, M. R. Mumpower, G. C. McLaughlin, and N. Schunck, Propagation of statistical uncertainties of skyrme mass models to simulations of r-process nucleosynthesis, Physical Review C 101, 10.1103/physrevc.101.055803 (2020).DOI
  10. [10]Y. L. Zhu, K. A. Lund, J. Barnes, T. M. Sprouse, N. Vassh, G. C. McLaughlin, M. R. Mumpower, and R. Surman, Modeling Kilonova Light Curves: Dependence on Nuclear Inputs, ApJ 906, 94 (2021), arXiv:2010.03668 [astro-ph.HE].
  11. [11]P. Möller, A. J. Sierk, T. Ichikawa, and H. Sagawa, Nuclear ground-state masses and deformations: FRDM(2012), Atomic Data and Nuclear Data Tables 109, 1 (2016), arXiv:1508.06294 [nucl-th].DOI
  12. [12]S. Goriely, S. Hilaire, M. Girod, and S. Péru, First gogny-hartree-fock-bogoliubov nuclear mass model, Phys. Rev. Lett. 102, 242501 (2009).pubmed.ncbi.nlm.nih.gov/19659002
  13. [13]R. Utama, J. Piekarewicz, and H. B. Prosper, Nuclear mass predictions for the crustal composition of neutron stars: A bayesian neural network approach, Phys. Rev. C 93, 014311 (2016).DOI
  14. [14]L. Neufcourt and et al., Neutron drip line in the ca region from bayesian model averaging, PRL 122, 062502 (2019).DOI
  15. [15]J. Wu, X.-Y. Chen, H. Zhang, L.-D. Xiong, H. Lei, and S.-H. Deng, Hyperparameter optimization for machine learning models based on bayesian optimization, Journal of Electronic Science and Technology 17, 26 (2019).DOI
  16. [16]A. E. Lovell, A. T. Mohan, T. M. Sprouse, and M. R. Mumpower, Nuclear masses learned from a probabilistic neural network (2022), arXiv:2201.00676 [nucl-th].DOI
  17. [17]C. M. Bishop, Mixture Density Networks, Tech. Rep. (Aston University, Department of Computer Science and Applied Mathematics, 1994).
  18. [18]A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopa, F. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, Pytorch: An imperative style, high-performance deep learning library, in Advances in Neural Information Processing Systems 32, edited by H. Wallach, H. Larochelle, A. Beygelzimer, F. d’Alche-Buc, E. Fox, and R. Garnett (Curran Associates, Inc., 2019) pp. 8024–8035.arxiv.org/abs/1912.01703
  19. [19]A. E. Lovell, A. T. Mohan, and P. Talou, Quantifying uncertainties on fission fragment mass yields with mixture density networks, Journal of Physics G: Nuclear and Particle Physics 47, 114001 (2020).DOI
  20. [20]R. Casten and N. Zamfir, The evolution of nuclear structure: The npn scheme and related correlations, Journal of Physics G: Nuclear and Particle Physics 22, 1521 (1999).
  21. [21]R. Casten, D. S. Brenner, and P. E. Haustein, Valence p-n interactions and the development of collectivity in heavy nuclei, Phys. Rev. Lett. 58, 658 (1987).pubmed.ncbi.nlm.nih.gov/10035002
  22. [22]M. Wang, G. Audi, F. G. Kondev, W. Huang, S. Naimi, and X. Xu, The AME2016 atomic mass evaluation (II): tables, graphs and references, Chinese Physics C 41, 030003 (2017).
  23. [23]G. T. Garvey, W. J. Gerace, R. L. Jaffe, I. Talmi, and I. Kelson, Set of nuclear-mass relations and a resultant mass table, Rev. Mod. Phys. 41, S1 (1969).DOI
  24. [24]D. P. Kingma and J. Ba, Adam: A method for stochastic optimization (2017), arXiv:1412.6980 [cs.LG].arxiv.org/abs/1412.6980
  25. [25]K. Deb, Multi-objective optimization, in Search Methodologies: Introductory Tutorials in Optimization and Decision Support Techniques, edited by E. K. Burke and G. Kendall (Springer US, Boston, MA, 2005) pp. 273–316.
  26. [26]N. Gunantara, A review of multi-objective optimization: Methods and its applications, Cogent Engineering 5, 1502242 (2018), https://doi.org/10.1080/23311916.2018.1502242.
  27. [27]M. Wang, W. Huang, F. Kondev, G. Audi, and S. Naimi, The AME 2020 atomic mass evaluation (II): tables, graphs and references, Chinese Physics C 45, 030003 (2021).
  28. [28]R. Orford, N. Vassh, J. A. Clark, G. C. McLaughlin, M. R. Mumpower, G. Savard, R. Surman, A. Apprahamian, F. Buchinger, M. T. Burkey, D. A. Gorelov, T. Y. Hirsh, J. W. Klimes, G. E. Morgan, A. Nystrom, and K. S. Sharma, Precision Mass Measurements of Neutron-Rich Neodymium and Samarium Isotopes and Their Role in Understanding Rare-Earth Peak Formation, Phys. Rev. Lett. 120, 262702 (2018).
  29. [29]L. Neufcourt, Y. Cao, W. Nazarewicz, and F. Viens, Bayesian approach to model-based extrapolation of nuclear observables, Phys. Rev. C 98, 034318 (2018).DOI
  30. [30]J. A. Melendez, R. J. Furnstahl, D. R. Phillips, M. T. Pratola, and S. Wesolowski, Quantifying correlated truncation errors in effective field theory, Phys. Rev. C 100, 044001 (2019).DOI
  31. [31]N. Schunck, J. O’Neal, M. Grosskopf, E. Lawrence, and S. M. Wild, Calibration of energy density functionals with deformed nuclei, Journal of Physics G: Nuclear and Particle Physics 47, 074001 (2020).DOI
  32. [32]S. Lundberg and S.-I. Lee, A unified approach to interpreting model predictions, arxiv:1705.07874 (2017), arXiv:1705.07874 [cs.AI].arxiv.org/abs/1705.07874
  33. [33]C. F. Weizsäcker, Zur Theorie der Kernmassen, Zeitschrift für Physik 96, 431 (1935).
  34. [34]J. W. Negele, The mean-field theory of nuclear structure and dynamics, Rev. Mod. Phys. 54, 913 (1982).DOI

Paper details

vlnc:W_8dxyjb9j
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents