In this paper
AbstractIntroductionProjected neural additive modelsUniversal approximation and approximation rateExtension to multiple outputsLearning problem and constraints for sparsityPost-processing and symbolic regressionNumerical experimentsBenchmarking with knot theoryLimitationsConclusionAcknowledgments and Disclosure of FundingReferencesAppendixPolynomial reproducingAdditional results for knot theory: classificationAdditional results for knot theory: regressionHigh-dimensional image classification with the MNIST datasetPhase field theory for fracture of brittle solidsWorksComputer Science
Projected Neural Additive Models as Universal Approximators
Abstract
Neural additive models (NAMs) offer interpretability but lack expressivity due to their rigid additive structure, whereas multi-layer perceptrons (MLPs) achieve universal approximation at the cost of transparency. We establish the first theoretical foundation for projected neural additive models (PNAMs), which augment NAMs with a learnable linear transformation to capture cross-variable interactions. While standard NAMs cannot approximate even the simplest interaction term because their cross-derivatives are identically zero, we prove that PNAMs are universal approximators and characterize the minimum number of projections required for exact polynomial representation. Furthermore, we derive an approximation rate of for functions (i.e., functions with continuous derivatives), where denotes the number of projections and the input dimension; this rate is known to be optimal for ridge function approximation. To recover interpretability from the projected inputs, we introduce structured sparsity regularizers and post-hoc symbolic regression techniques that enable global input ranking, parameter pruning, and the conversion of learned bases into compact analytical expressions. Experiments on knot theory invariants, MNIST, and phase field fracture mechanics demonstrate that PNAMs match or surpass similar architectures in accuracy while discovering salient features and yielding symbolic models with orders of magnitude fewer parameters.
Introduction
While deep neural networks have been widely adopted across diverse tasks due to their expressivity, they inherently lack interpretability [1]. Among the myriad methods introduced to address this issue, Agarwal et al. [2] propose the neural additive model (NAM), an architecture constrained to a linear combination of single-variable functions parameterized by independent multi-layer perceptrons (MLPs). Although this decoupled structure provides feature-wise interpretability, the strictly additive nature of NAMs limits their expressivity; in particular, their cross-derivatives are identically zero, rendering them incapable of representing functions with nontrivial input interactions.
To overcome this limitation, Phan et al. [3] introduce a learnable linear transformation preceding the NAM (see Figure 1)—an architecture we refer to as the projected neural additive model (PNAM). A complementary line of work—GA²M [4], NODE-GA²M [5], GAMI-Net [6], SPAM [7], and SIAN [8]—augments generalized additive models (GAMs) [9] and NAMs with explicit interaction terms such as acting on pairs of inputs. Interpretability is then pairwise, but the number of interaction terms grows combinatorially with the number of inputs , and the interaction order is capped by construction: a model limited to pairwise terms cannot represent, e.g., and is therefore not a universal approximator. If the ground truth is dominated by a few low-order interactions of raw features, such explicit models are more directly interpretable. The PNAM is instead preferable when interactions are higher-order, dense in a few directions, or unknown a priori, as it captures them along learned directions, with far below the pairs that a pairwise model enumerates. While the transformation empirically enhances expressivity by capturing cross-variable couplings, the PNAM architecture was introduced without a rigorous theoretical foundation or a formal proof of universal approximation.

Figure 1. Architecture of the PNAM. The output is predicted via a linear combination of single-variable functions, parameterized by independent MLPs, which are evaluated on linear combinations of the inputs . The transformation and weighting coefficients (both denoted with dashed lines) can be optimized to yield sparser (and more interpretable) solutions.
Contributions. Our core contributions establish this missing foundation alongside novel mechanisms for interpretability and sparsity. First, we prove that PNAMs are universal approximators (Theorem 2.1)—a fundamental guarantee previously missing for this architecture class. Second, we characterize the projection complexity of (Appendix § 8.§ 1) by establishing constructive upper bounds and a parameter-counting lower bound on the minimum number of projections required to represent any degree- polynomial in variables exactly. Third, building on these results, we derive an approximation rate of for functions (Theorem 2.2), a rate known to be optimal for ridge function approximation [10, 11].
To mitigate the reduction in interpretability induced by , we introduce structured regularization techniques that enable global feature ranking, penalize unnecessary input interactions, and promote sparse solutions. While standard NAMs provide feature-level interpretability via their shape functions, our regularized PNAMs offer global input ranking across the full dataset [12, 13]. Moreover, we leverage the modularity of PNAMs to prune nonessential parameters and extract compact symbolic expressions from the learned single-variable bases. Empirically, experiments on knot theory invariants confirm that PNAMs recover accurate analytical expressions comparable to Kolmogorov–Arnold networks (KANs) [14], entirely without prior knowledge of the relevant invariants.
Related work. The PNAM belongs to a growing class of architecture-based models designed to enhance the interpretability of deep neural networks without compromising expressivity. Prominent alternatives include KANs [14], deep polynomial neural networks [15], and graph neural networks with inductive biases [16], alongside related interpretability methods such as SINDy [17], structural pruning [18], and gradient-based attribution [19]. Compared to KANs—perhaps their closest architectural relative3—PNAMs offer a distinct and complementary theoretical profile. While KANs leverage the Kolmogorov–Arnold representation theorem (which necessitates two layers of nonlinearities), PNAMs utilize a single linear transformation followed by 1D nonlinearities. This simpler structure enables the approximation rate of Theorem 2.2.
In addition, we emphasize that although we have equipped PNAMs with post-hoc symbolic regression (SR) capabilities, this technique is primarily employed for pruning. Like most additive architectures, PNAMs are not designed to recover exact physical laws with precise functional forms; their additive nature prevents them from compactly approximating operations such as division,4 and the number of projections required to represent complex interactions can be prohibitively large. Consequently, we reserve the recovery of exact physical laws for dedicated SR algorithms leveraging reinforcement learning [21], physics-inspired strategies [22], and genetic programming [23]. Further discussion on interpretability and optimization constraints is provided in § 4.
Projected neural additive models
Given a training dataset containing input-output pairs, where is an input vector of independent variables and is the corresponding scalar label, the goal of supervised learning is to construct an approximating function . We hypothesize that this multi-variable function can be approximated by a weighted sum of single-variable functions acting on a projected space:
where is the predicted value and represents an additive noise term. The projected variables in Eq. (1) result from a linear transformation of the inputs, such that . For compactness, each feature function and its corresponding weighting coefficient are combined into a composite function .
The single-variable functions can be implemented using various bases (e.g., polynomials or neural networks); here, we parameterize the output as a linear combination of MLPs:
where each is an -layer MLP with a set of learnable weights and biases and an element-wise activation function :
To effectively capture high-frequency components, we leverage a Fourier feature mapping [24, 25] that transforms each projected variable into
where entries in are sampled from a Gaussian distribution , with standard deviation , and fixed after initialization. When this mapping is used, the MLP in Eq. (3) is evaluated on rather than the scalar . As illustrated in Figure 1, the transformation , weighting coefficients , and error term in Eq. (2) are encoded as two weight matrices and a bias term, respectively. This construction formalizes the PNAM architecture, extending the foundational and empirical frameworks established by Agarwal et al. [2] and Phan et al. [3].
Remark 1 (Comparison with NAMs and MLPs). In a PNAM, each basis in Eq. (1) is a function of a transformed variable , rather than a raw input as in NAMs or other GAMs [9, 26]. The linear transformation enables the model to capture cross-variable couplings (e.g., in Example A.1) that standard NAMs cannot. Although this projection precludes the direct visualization of input contributions typical of NAMs, we demonstrate in § 2.4 how the static matrix can be examined to perform global input ranking and identify critical interactions. This structured interpretability—encompassing feature pruning and dimensionality reduction—is generally absent in black-box, fully connected MLPs.
Universal approximation and approximation rate
By the Stone–Weierstrass theorem [27, 28, 29], polynomials are dense in the space of continuous functions for a compact domain , meaning they can approximate any continuous function to arbitrary precision. Consequently, we can establish universal approximation for PNAMs by demonstrating that they can reproduce arbitrary multi-variable polynomials. The key insight is that the learnable linear transformation enables PNAMs to express these polynomials via a superposition of ridge functions—functions of the form —whereas NAMs, restricted to (the identity projection), cannot represent even the degree-2 cross-term and therefore fail to reproduce arbitrary polynomials. We show in Theorem A.1 that single-variable polynomials of projected inputs can serve as the bases to reproduce any multi-variable polynomial, providing the constructive foundation for the general approximation results that follow.
Alternatively, instead of single-variable polynomials, one can employ a sum of 1D neural networks acting on the projected inputs to achieve universal approximation. Since these networks satisfy the 1D universal approximation property [30, 31, 32], they can readily approximate the required single-variable polynomials used to construct the multi-variable approximation. This, by extension, guarantees the universal approximation property of the resulting multi-dimensional PNAM architecture. This capability is formally stated in the following theorem.
Theorem 2.1 (Universal approximation). Let be a set of single-variable polynomials or 1D neural networks. Given any function in the set of continuous real-valued functions for a compact domain and any , there exist a linear transformation , where , and a set of functions in such that
for all .
Proof. Since is compact, the Stone–Weierstrass theorem applies [27, 28, 29], that is, polynomials are dense in . As a result, there exists a (multi-variable) polynomial such that
Then for , by Theorem A.1, we have
with single-variable polynomials . Note that Theorem A.1 covers any polynomial , establishing the existence of a transformation and functions that reproduce it exactly. If is a set of polynomials, we are done; if represents neural networks, we treat the polynomials as intermediate target functions. By the universal approximation theorem for neural networks [30, 31, 32], for each , there exists a network with
where denotes the supremum norm over , the image of under the th projection. Summing overall and applying the triangle inequality yield
Combining this result with Eq. (6) via the triangle inequality gives Eq. (5), completing the proof.
Theorem 2.1 establishes that PNAMs can approximate any continuous function to arbitrary precision, but does not quantify the rate at which the approximation error decreases with the number of projections . Leveraging the polynomial reproducing property of Theorem A.1 and the classical Jackson theorem [33], the following theorem derives an explicit approximation rate.
Theorem 2.2 (Approximation rate). Let for a compact domain with and . For any , there exists a PNAM with projections satisfying
where depends on , , , and .
Proof. By Jackson’s theorem [33], there exists a polynomial of degree such that
where depends on , , , and . By Theorem A.1, can be represented exactly by a PNAM with projections. Inverting this relation gives . Thus, for large , the asymptotic scaling satisfies . Substituting this result into the Jackson bound from Eq. (10) yields
where and factors from the inversion are absorbed into . Because is represented exactly by the PNAM , the approximation error vanishes. Applying the triangle inequality:
which completes the proof.5
Remark 2 (Choice of projection bound). The approximation rate of in Theorem 2.2 follows directly from the bound on established in Theorem A.1. Using the alternative bounds from Theorems A.2 or A.3 would produce weaker results: Theorem A.3 gives , which recovers the classical polynomial approximation rate of [11, 33], while the exponential scaling of Theorem A.2 yields only a logarithmic rate of . Theorem A.1 therefore provides the best approximation rate among the three bounds, since it requires the fewest projections for a given polynomial degree in the large- regime. Moreover, Maiorov [10] proves that is the optimal rate for ridge function approximation of Sobolev-class functions, confirming that the bound in Theorem A.1 is tight in this asymptotic sense.
Remark 3 (Parameter complexity). To achieve approximation error , Theorem 2.2 requires projections. To achieve a matching error, each approximating a single-variable function in is a network with width . Thus, the parameter complexity of PNAMs is
for fixed depth , where the term is dominated by as . This result matches the optimal complexity established for standard MLPs [34].
Extension to multiple outputs
To accommodate multiple outputs, let , , and , with denoting the number of dependent variables. We extend Eq. (1) to represent each output as
where is the single-variable function associated with the projection for the output [35]. To see that Eq. (11) is a natural extension of Eq. (1), consider the case where the bases share no common inputs across different outputs . In this scenario, the total number of projections is , with denoting the number of projections required to approximate each independently. Equivalently, for each index , both the coefficients and functions are nonzero only for a disjoint subset of indices of size , yielding independent basis sets. By extension of Theorem 2.1, can be chosen sufficiently large to approximate any -tuple of continuous functions on a compact domain.
Learning problem and constraints for sparsity
Let denote all learnable parameters of the 1D MLPs. We optimize the parameters of PNAMs by minimizing the following loss function (in parentheses) for training samples:
where the empirical loss is defined as
for classification (cross-entropy), or
for regression (mean squared error). The term optionally imposes physical constraints that restrict the space of admissible solutions [36, 37, 38].
To prevent overfitting and produce interpretable solutions, we employ two structured regularizers:
where is the standard regularization of the network weights and biases [39].6 and discourages the bases from taking on disproportionately large values [2]. Given that the inputs and outputs are typically normalized, this constraint underpins the global input ranking of § 2.4, in which the columns of the linear transformation serve as a proxy for predictive importance.
To further promote sparsity and disentanglement, we analyze the singular value decomposition (SVD) of :
where and are orthonormal matrices of dimensions and , respectively, and is an diagonal matrix containing the singular values. Drawing on the metric
used to measure the distance between two 3D rotations [40], we define a joint penalty for the deviation of and from the identity matrix :
This term effectively penalizes unnecessary couplings between the inputs; in isolation, is minimized when the transformation is a rectangular diagonal matrix. This shrinkage toward the axis-aligned NAM subclass—the most interpretable member of the PNAM family—acts as a structured prior: since is dominated by the empirical loss , cross-variable couplings are introduced only when they reduce by more than the cost they incur.7 Finally, we apply sparsity constraints on and to drive nonessential interactions and bases to zero [25, 35, 41]:
Post-processing and symbolic regression
Following training, we apply three post-processing techniques to prune the PNAM and enhance interpretability [1, 42].
The first technique exploits the regularizer in Eq. (13). The column-wise mean of the absolute entries in the linear transformation —defined as and termed the mean absolute coefficients—measures how strongly each input is used in the learned projections. When the bases are constrained by and the variables are normalized, become comparable in magnitude across projections, so a column of is large only if this reduces the empirical loss . The mean absolute coefficients then serve as an empirical proxy for predictive importance (validated in Appendix § 8.§ 2) rather than a guaranteed property of the PNAM architecture. We can thus retain only the top features and zero out the columns of associated with the least significant inputs. The second technique applies hard thresholding to the transformation and weighting coefficients , zeroing out entries where and for user-defined thresholds and .8
Finally, the third technique leverages the single-variable nature of the bases to convert them into explicit symbolic expressions. While various SR algorithms are compatible (such as DSR [21] or AI Feynman [22]), we employ PySR [23], which utilizes genetic programming [43, 44], for its efficiency and ease of use. To distill the output of the MLP into symbolic form, we minimize the following objective for fixed and :
where is a mathematical expression of a single variable , and controls the weight placed on matching the derivatives of the learned MLP. Summing the resulting over and adding the bias recovers the global output as defined in Eq. (11). These post-processing steps can reduce tens of thousands of network parameters to as few as tens or hundreds, effectively bypassing the NP-hardness of multi-variable SR [21, 45] while retaining the predictive power of deep neural networks.
Numerical experiments
We illustrate the expressivity and interpretability of PNAMs through a benchmark study on an extensive dataset of mathematical knots from [46], covering (i) multi-class classification and (ii) single-output regression. In both tasks, PNAMs recover the same key invariants identified by previous studies [14, 46] and produce compact symbolic expressions without prior knowledge of the relevant invariants. Additional results for knot theory are provided in Appendices § 8.§ 2 and § 8.§ 3. Experiments on the MNIST dataset [47], provided in Appendix § 8.§ 4, further demonstrate the feature selection and dimensionality reduction capabilities of PNAMs for high-dimensional image classification. Finally, we examine the benign overfitting behavior of PNAMs as the projection dimension increases using a phase field simulation of fracture propagation [48] with a limited dataset (Appendix § 8.§ 5).
Unless otherwise stated, we employ an 80/20 train-test split; the training set is further divided into training and validation subsets (80/20). The PNAM is implemented in PyTorch [49] using SiLU9 [50, 51] as the activation function in Eq. (3). We initialize the projection dimension to and adjust it based on the complexity of the task.
Unless otherwise stated, the regularization weights in Eq. (12) are set to 0.01 for classification and 0.001 for regression tasks.10 By default, the optional physical constraint term is omitted, and the regularization weight in Eq. (16) is set to zero. We utilize a batch size of 256 samples11 and the Adam optimizer [52] with an initial learning rate of 0.001 that decays by a factor of 0.995 per epoch to train all networks. All training is conducted on a single NVIDIA A100-SXM4-40GB GPU. Once trained, each basis is evolved for one minute using 30 populations of 30 expressions with a maximum complexity of 30; all operators and leaf nodes are assigned a complexity of one. Other relevant hyperparameters and training details are reported alongside the results.
Benchmarking with knot theory
Established by a team of researchers mostly from Google DeepMind (DM) [46], the dataset of mathematical knots consists of 243,746 samples, each possessing 17 geometric invariants: adjoint torsion degree, torsion degree, short geodesic (real part), short geodesic (imaginary part), injectivity radius, Chern–Simons invariant, cusp volume, longitudinal translation, meridional translation (imaginary part), meridional translation (real part), volume, and six symmetry groups. The goal is to use these 17 invariants to predict the knot signature, which takes on one of 14 discrete values (even integers ranging from −12 to 14). To assess the versatility of PNAMs, we explore two distinct formulations: (i) a multi-class classification problem with 14 classes and (ii) a single-output regression problem.
Classification. The first two rows of Table 1 are reproduced from [14] and compare the performance of the MLP12 implemented by Davies et al. [46] against that of the KAN. The subsequent rows present our implementations of the MLP, the NAM, and four parameterizations of the PNAM, all utilizing two hidden layers of 64 and 32 neurons.13 We scale all inputs to have zero mean and unit variance and train the models for 50 epochs without early stopping. All six models achieve test accuracies exceeding 90%, with the NAM performing the worst and the PNAM with matching our implementation of the MLP. The accuracy of the PNAM increases monotonically with (from 8 to 64), consistent with Theorem 2.2. Moreover, the accuracy gain of the PNAM over the NAM for all three values of in Table 1 suggests that the penalty in Eq. (14) does not collapse the PNAM into a NAM in practice (see, e.g., the learned projection matrix in Fig. A.2(b)). Although the accuracy of PNAMs can be improved by increasing or by replacing the first hidden layer of the basis MLPs with a Fourier feature mapping (e.g., the last row of Table 1, with and in Eq. (4)), these modifications increase the complexity of the learned functions.
Table 1. Performance comparison of different neural network architectures for the multi-class classification problem of predicting the signature of mathematical knots. The first two rows are reproduced from Table 3 of Liu et al. [14]; see [14] for definitions of and . For the remaining rows, the mean and standard deviation of the test accuracy are computed over 10 runs. The last row incorporates the Fourier feature mapping described in Eq. (4).
Table 2 summarizes the three most important features identified by the mean absolute coefficients of the linear transformation for the PNAM with and without the Fourier feature mapping.
| Rank | Input feature | Symbol | Frequency | Test acc. |
| 1 | Re(meridional translation) | 10/10 | 55.1 3.5% | |
| 2 | Longitudinal translation | 10/10 | 74.2 1.5% | |
| 3 | Im(meridional translation) | 7/10 | 78.1 2.4% | |
| 3 | Cusp volume | 1/10 | 80.5% | |
| 3 | Im(short geodesic) | 1/10 | 73.5% | |
| 3 | Volume | 1/10 | 70.5% |
Table 2. Ranking of important features based on the mean absolute coefficients for the PNAM with (without the Fourier feature mapping) in Table 1. The mean and standard deviation of the test accuracy (associated with retaining only the top features) are computed over the runs in which the top features appeared.
Across all 10 runs, the meridional translation (real part) and the longitudinal translation consistently have the largest and second largest mean absolute coefficients, respectively; the meridional translation (imaginary part) ranks third in seven runs. The top two invariants are likewise ranked first and second in 10/10 runs for and , indicating that the feature ranking remains stable as the capacity grows. Retaining only the columns of associated with the top inputs in Table 2, the PNAM still achieves a test accuracy of 78.1% (refer to Fig. A.1 for additional details). These findings are consistent with Fig. 3 of Davies et al. [46]14 and Fig. 4.3 of Liu et al. [14]. Interestingly, PNAMs recover these key invariants through direct inspection of the projection weights, whereas Davies et al. [46] employ gradient-based attribution [19] and Liu et al. [14] leverage a specific KAN architecture with a hidden dimension of one to extract feature importance.
Regression. Davies et al. [46] and Liu et al. [14] leverage insights from the classification task to construct explicit mathematical expressions for the knot signature, now formulated as a single-output regression problem. In Table A.5, expression A corresponds to the equation handcrafted by Davies et al. [46], while expressions B to F are derived from post-processing steps of KANs trained specifically on the three most relevant invariants [14]. Although Davies et al. [46] originally report a test accuracy between 70 and 80% for expression A, Liu et al. [14] report a test accuracy of 83.1%. Since the accuracy appears sensitive to the random data split employed, we evaluate all candidate expressions on our specific test set and the entire knot dataset (denoted as “Our” and “Total acc.” in Table A.5), in addition to the values reported by Liu et al. [14] for expressions A to F.
We demonstrate that the PNAM with regularization weight in Table A.3—trained on the full set of invariants without prior knowledge of their relevance—can be distilled into symbolic expressions with as few as tens of parameters. We emphasize that expressions G to I in Table A.5 emerge from the PNAM pipeline after pruning and SR; their particular forms depend on the available data and would likely change as more invariants are considered. The key contribution here is the ability of PNAMs to discover relevant relationships as new data and invariants are introduced.
Among the results summarized in Table A.4, expressions H and I are obtained from the run depicted in Fig. A.2, using features with the cusp volume15 ranked as the third most significant. Although the cusp volume is not explicitly identified in [14, 46] as a primary invariant for predictive accuracy, the PNAM independently identifies it as an invariant that potentially enhances performance. Moreover, comparing expressions E and G suggests that PNAMs can match KANs in accuracy while utilizing fewer invariants without relying on additional assumptions. For an analysis of how the regularization weights in Eq. (12) influence performance, see Appendix § 8.§ 3.
Limitations
Interpretability scope. Although PNAMs provide global interpretability, they do not offer local interpretability. Because the bases operate on the projected variables rather than the original inputs , examining in isolation does not directly reveal the contribution of a specific input to the output. Furthermore, the mean absolute coefficients of the linear transformation measure feature usage in the learned projections, and they serve as a proxy for predictive importance only when the regularizer in Eq. (13) is active and appropriately calibrated. In the absence of such a constraint, the resulting ranking may be unstable or erroneous (see, e.g., Fig. A.5(a)). Even with this constraint, the ranking is an empirical property of the regularized models, validated in the settings evaluated here, as opposed to a general guarantee of the architecture. Finally, we emphasize that the PNAM is a predictive, rather than a causal, model. Large mean absolute coefficients indicate a strong predictive influence but do not imply a causal relationship, particularly when the model is trained on highly correlated inputs.
An alternative approach to interpretability combines MLPs with post-hoc attribution, pruning, or symbolic distillation. Attribution explains the predictions and pruning reduces the model size, but the model itself remains a black box. Symbolic distillation, moreover, amounts to the multi-variable SR problem whose intractability motivates our design. The PNAM instead provides an integrated pipeline in which the same trained model supports feature ranking, pruning, and conversion into a closed-form expression, because the additive structure reduces SR to independent 1D problems. However, the GAM/NAM extensions with explicit interaction terms [4, 5, 6, 7, 8] are more directly interpretable when the ground truth is dominated by a few low-order interactions of raw features.
Optimization and training. The universal approximation and rate results of § 2.1 are only existential: they assert that accurate transformations and bases exist, not that gradient-based optimization recovers them—a characteristic shared with classical universal approximation theorems for MLPs [30, 31]. Two observations nevertheless connect the theory to the learned models. First, the transformation achieving an exact representation is highly nonunique (e.g., Example A.1), so the optimization target is a large set of admissible transformations rather than a single isolated construction. Second, performance improves monotonically with the projection dimension on average across all our experiments (see Table 1, A.7, and A.9), tracking the capacity scaling predicted by Theorem 2.2.
In practice, the joint optimization over , , , and in Eq. (12) is non-convex, and performance is sensitive to both and the regularization weights . Over-regularization leads to sparse but inaccurate models, while under-regularization results in dense projections that are significantly harder to interpret. In addition, the SR step in Section 2.4 is run for only one minute per basis in the numerical experiments. While this ensures computational efficiency, it may be insufficient for capturing the nuances of highly complex functions. Consequently, the resulting symbolic expressions are approximations and should be rigorously validated before use in safety-critical applications.
Conclusion
In this work, we establish the first theoretical foundation for PNAMs, proving universal approximation, characterizing the minimum number of projections , and deriving an approximation rate of for functions. Building on this foundation, we introduce structured regularization, input pruning, and symbolic regression techniques to recover sparse and interpretable models. Across a diverse set of experiments, PNAMs consistently match or surpass similar architectures in predictive accuracy while promoting sparsity. In the knot theory benchmark, PNAMs recover the same key invariants identified by prior studies and produce compact symbolic expressions with orders of magnitude fewer parameters than the trained networks. Additional experiments on MNIST and phase field fracture mechanics demonstrate the feature selection capability of PNAMs for high-dimensional image classification and their robustness in the limited-data regime, respectively. As a stand-alone model, the PNAM is not meant to replace the MLP, the NAM, or classical SR. Instead, it offers a principled middle ground: achieving the accuracy of the MLP while retaining the additive structure of the NAM, at the cost of increased memory and training time proportional to . Future work includes parallelizing the evaluation of the 1D bases to reduce this overhead and exploring the use of PNAMs as backbones in graph neural networks and transformers, where their modularity could enable structural pruning of network parameters and accelerate inference.
Acknowledgments and Disclosure of Funding
NNP is supported by the SMART Scholarship Program and the SMART SEED Innovation Award, both administered by the U.S. Army DEVCOM Army Research Laboratory. WCS is supported by the Department of Energy (DOE), National Nuclear Security Administration, Predictive Science Academic Alliance Program (PSAAP) under Award Number DE-NA0003962. Additional support is provided for NNP and WCS through the Dynamic Materials and Interactions Program from the Air Force Office of Scientific Research under grant contract FA9550-22-1-0310 and through the Army Research Office under grant contract W911NF-24-1-0116. QD is supported in part by the DOE-ASCR award DE-SC0025347 and the National Science Foundation (NSF) award DMS-2309245. Support is also provided by the NSF under grant contract CMMI-1846875 for the high-performance computing facility. All of this support is gratefully acknowledged. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the sponsors, including the Army Research Laboratory or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes, notwithstanding any copyright notation herein. The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
References
[1] W. J. Murdoch, C. Singh, K. Kumbier, R. Abbasi-Asl, and B. Yu. Definitions, methods, and applications in interpretable machine learning. Proceedings of the National Academy of Sciences, 116(44):22071–22080, 2019.
[2] R. Agarwal, L. Melnick, N. Frosst, X. Zhang, B. Lengerich, R. Caruana, and G. E. Hinton. Neural additive models: Interpretable machine learning with neural nets. Advances in Neural Information Processing Systems, 34(359):4699–4711, 2021.
[3] N. N. Phan, W.C. Sun, and J. D. Clayton. HYDRA: Symbolic feature engineering of overparameterized Eulerian hyperelasticity models for fast inference time. Computer Methods in Applied Mechanics and Engineering, 437:117792, 2025.
[4] Y. Lou, R. Caruana, J. Gehrke, and G. Hooker. Accurate intelligible models with pairwise interactions. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 623–631, 2013.
[5] C.-H. Chang, R. Caruana, and A. Goldenberg. NODE-GAM: Neural generalized additive model for interpretable deep learning. arXiv preprint arXiv:2106.01613, 2021.
[6] Z. Yang, A. Zhang, and A. Sudjianto. GAMI-Net: An explainable neural network based on generalized additive models with structured interactions. Pattern Recognition, 120:108192, 2021.
[7] A. Dubey, F. Radenovic, and D. Mahajan. Scalable interpretability via polynomials. Advances in Neural Information Processing Systems, 35:36748–36761, 2022.
[8] J. Enouen and Y. Liu. Sparse interaction additive networks via feature interaction detection and sparse selection. Advances in Neural Information Processing Systems, 35:13908–13920, 2022.
[9] T. Hastie and R. Tibshirani. Generalized additive models. Statistical Science, 1(3):297–310, 1986.
[10] V. E. Maiorov. On best approximation by ridge functions. Journal of Approximation Theory, 99(1):68–94, 1999.
[11] A. Pinkus. Approximation theory of the MLP model in neural networks. Acta Numerica, 8:143–195, 1999.
[12] C. Molnar. Interpretable Machine Learning: A Guide for Making Black Box Models Explainable. Leanpub, 2019.
Appendix
Polynomial reproducing
In this section, we establish the polynomial reproducing property, namely, that any multi-variable polynomial of can be represented by a superposition of ridge functions. Specifically, we show that there exists a linear transformation such that , where each is a single-variable polynomial of the projected input . This result is proved for arbitrary dimensions via mathematical induction on the input dimension , starting with the base case for two dimensions ().
To facilitate the inductive step, we first present a lemma demonstrating the reproducing property for a specific bivariate case. This lemma provides the necessary constructive foundation for the general -dimensional result that follows.
Lemma A.1 (Monomial representation). For any bivariate monomial , there exist a linear transformation , where , with and , and a set of coefficients such that
Proof. Applying the binomial expansion to each term on the right-hand side of Eq. (A.1) and reordering the summations, we obtain
To satisfy the equality in Eq. (A.1), the coefficients of the resulting polynomial must match the target monomial . Specifically, the coefficient must be 1 when and 0 otherwise. This leads to the following system of linear equations for :
where is the Kronecker delta. Written in matrix form, this system is
We note that the coefficient matrix forms a so-called Vandermonde matrix [53, 54]. It is a standard result that a Vandermonde matrix is invertible if and only if the constants are distinct. By choosing mutually different values for , we guarantee a unique solution for the coefficients . This establishes that projections suffice, thus proving the bound .
Even for this simple case, we may observe the nonuniqueness in the construction of the linear transformation . Consequently, it is reasonable to expect that for specific machine learning applications, may be optimized to achieve effective representations in a reduced-dimensional feature space. This potential for dimensionality reduction is illustrated in the following example.
Example A.1 (Dimensionality reduction). We present instances where the required projection dimension is strictly smaller than the upper bound established in Lemma A.1. For , a variation of the polarization identity (see Theorem A.2) gives
which corresponds to , whereas the lemma suggests an upper bound of 3. In the trivial case where or , the monomial is univariate, and we may simply use the identity transformation with . Furthermore, can be smaller than the input dimension . For example, the quadratic form can be represented as with , yielding . These cases demonstrate the efficacy of utilizing a learned transformation to achieve dimensionality reduction through an appropriate latent space.
Next, we consider the extension of Lemma A.1. Since a product of two single-variable polynomials and can be expressed as a linear combination of bivariate monomials , Lemma A.1 can be generalized to represent polynomials with a product form.
Lemma A.2 (Product-form representation). For any bivariate polynomial of having the product form , there exist a linear transformation , where , with denoting the degree of a polynomial, and a set of single-variable polynomials such that
Proof. We begin by expanding the single-variable polynomials in their respective power bases:
Multiplying these expressions yields a bivariate polynomial represented as a linear combination of monomials:
To apply Lemma A.1 using a single unified linear transformation for all terms, we fix distinct constants . For any given monomial in the expansion, the total degree satisfies .
The matching of coefficients for a specific monomial generates a Vandermonde system of equations with unknowns . Since and the constants are distinct, the resulting Vandermonde matrix has full row rank. This guarantees that the system is consistent and admits at least one valid solution set for each pair . While the solution is nonunique for , we may fix any particular solution, such as the minimum-norm solution obtained via the Moore–Penrose pseudoinverse [55] of the Vandermonde matrix.
By solving for using the same constants for every monomial, we can employ a global transformation with and . Substituting the monomial representations back into the expansion and reordering the summations, we obtain
This result matches the form of Eq. (A.2) by defining each single-variable polynomial as
where represents a projected input. This completes the proof. □
Another consequence is the reproducing property for arbitrary polynomials in two dimensions, which follows from Lemma A.2 by observing that each monomial is itself a product of two single-variable polynomials.
Lemma A.3 (Bivariate polynomial representation). For any bivariate polynomial of , there exist a linear transformation , where , and a set of single-variable polynomials such that
Proof. Note that refers to the degree of the multi-variable polynomial, defined as the maximum value of among the monomials appearing in with nonzero coefficients. Expanding in the monomial basis, we have
The key observation is that each monomial is a specific instance of the product form treated in Lemma A.2, where . Lemma A.2 therefore applies to each monomial individually, utilizing a unified set of projections shared globally across all monomials. The remainder of the proof follows identically to that of Lemma A.2. □
Finally, we employ mathematical induction to extend the result to any input dimension.
Theorem A.1 (Polynomial representation via induction). For any multi-variable polynomial of , there exist a linear transformation , for some $M that can be chosen to depend only on and (e.g., ), and a set of single-variable polynomials such that
Proof. We proceed by mathematical induction on the input dimension . For the base case , Lemma A.3 establishes that a single transformation suffices for any bivariate polynomial of degree , as the choice of projection constants depends only on .
Now, assume that Theorem A.1 holds uniformly for the space of polynomials of degree in variables, that is, there exists a global transformation that can represent any such polynomial by varying only the single-variable functions. We then consider a polynomial of degree defined on . It can be expanded as
for some polynomials , where . Since has degree , each term must satisfy . In particular, because for all , the inductive hypothesis allows us to use the same global transformation for all . Thus, there exists a set of single-variable polynomials such that
Substituting Eq. (A.5) into the expansion and rearranging the summations, we obtain
Let . Each is a bivariate polynomial in the variables and of total degree at most . By Lemma A.3, each admits a representation involving at most projections of the form .
Given that the choice of is independent of the specific polynomials , we can share these constants across all , resulting in a global transformation for dimensions. Therefore, the total number of projections is
Solving this recurrence relation with the base case yields
As a result, Theorem A.1 holds for the input dimension . This completes the induction process and the proof.
Theorem A.1 shows that to represent a polynomial of degree in variables, the number of required projections satisfies ; thus, for large , the bound scales as . This bound is consistent with classical ridge function approximation theory (see, e.g., [11]), which establishes the same scaling. While Theorem A.1 achieves this scaling constructively via mathematical induction using a Vandermonde system of equations, Pinkus [11] leverages a non-constructive dimension argument on the space of homogeneous polynomials to guarantee the existence of suitable directions without specifying them. Alternatively, an estimate on can be obtained via a complementary construction based on the polarization identity [56].
Theorem A.2 (Polynomial representation via polarization). Let be the space of multi-variable polynomials in variables of degree at most . Any can be represented exactly by a PNAM with projections, where is bounded by the total number of terms generated by the polarization of each basis monomial, that is,
Proof. Any polynomial of degree , where , can be written as a linear combination of monomials using multi-index notation:
The number of monomials of degree exactly is , and the total dimension of the polynomial space is . Note that constant monomials () require no projections, as their contribution can be absorbed into the constant term of any single-variable polynomial in the representation; hence, they do not contribute to .
Step 1: Polarization of a single monomial. For each monomial of degree , we apply the polarization identity. We encode the factored structure of as a product of linear factors by introducing vectors , where of them equal the th standard basis vector , so that for the corresponding factor. The polarization identity then gives
Pairing each sign vector with its negation yields equal contributions, since
As a result, the terms collapse to distinct projected power functions of the form , where and are the scalar weight and projection direction associated with the th sign vector pair , respectively.
Step 2: Summation over the polynomial. Substituting the polarized form of each monomial back into the polynomial expansion yields
By linearity, rearranging the summations gives
where is a single-variable polynomial (specifically a monomial).
Step 3: Consolidation. Based on the above construction, the total number of projections is bounded by the sum over all monomial degrees of the number of projections per monomial:
This completes the proof.
Theorems A.1 and A.2 each provide a constructive upper bound for , but with complementary limitations: Theorem A.1 scales as and is exponential in , while Theorem A.2 scales as and is exponential in . A bound that is simultaneously polynomial in both and can be obtained by a non-constructive argument via the Waring rank [56].
Theorem A.3 (Bound via Waring rank). The minimum number of projections required to represent any multi-variable polynomial in variables of degree is bounded by the dimension of the polynomial space:
Proof. The set of all multi-variable polynomials of degree at most in variables, , is a vector space with dimension . The set of projected power functions spans , as every monomial can be recovered via the polarization identity in Theorem A.2. Since this spanning set covers the entire space, one can extract a basis of size . Any can therefore be expressed as a linear combination of at most projected functions.
As noted, the Waring bound scales polynomially in both and , making it tighter than Theorem A.2 for large regardless of . This bound is looser than Theorem A.1 when is large relative to , but tighter than Theorem A.1 when is large relative to . For large , the Waring bound can be further refined for generic polynomials via the notion of generic Waring rank, as formalized by the Alexander–Hirschowitz theorem [56, 57]. The following corollary summarizes the resulting bounds on , including a parameter-counting lower bound.
Corollary A.4 (Projection complexity). For the space of multi-variable polynomials in variables of degree at most , the minimum number of projections in the exact polynomial representation satisfies
where the lower bound follows from a parameter-counting argument and the upper bound from Theorem A.3. In the exact polynomial representation setting, each of the projections contributes free parameters from the linear transformation and coefficients from the single-variable polynomial , giving total free parameters, which must be at least . For generic polynomials with large , the Alexander–Hirschowitz theorem [56, 57] tightens the upper bound to
which differs from the lower bound by a factor of .
The bounds established in this section—from the constructive bound of Theorem A.1 to the parameter-counting lower bound of Corollary A.4—characterize the minimum required for exact polynomial representation. In practice, however, PNAMs do not aim to represent a fixed polynomial exactly, but rather to learn an effective approximation of an unknown function from data. As demonstrated in Example A.1, the nonuniqueness of the transformation means that the theoretical upper bounds on are not tight in general; different constructions can achieve the same representation with fewer projections. This suggests that, through learning and optimization, PNAMs can discover representations that are more compact than the theoretical bounds would predict, adapting to the specific structure of the target function rather than to the worst case.
Additional results for knot theory: classification
Figure A.1 shows a complete ranking of the inputs and the associated test accuracy of using the top inputs from one of the runs summarized in Table 2.
Figure A.1. One complete instance of the ranking and accuracy reported in Table 2. (a) Relative importance of the input features based on the mean absolute coefficients, and (b) test accuracy associated with using the top features in (a).
Validation of the importance ranking. To test whether the ranking by the mean absolute coefficients of the linear transformation captures predictive importance as opposed to mere feature usage, we perform four post-training checks on the PNAM with and without the Fourier feature mapping in Table 1. First, columns of are zeroed out according to the ranking of each run. Table A.1 reveals that removing the single top-ranked feature reduces the test accuracy from 93.6 to 30.9%, whereas removing the three bottom-ranked features has no measurable effect. Second, starting from the transformation with all columns zeroed out, a greedy selection reactivates one column at a time, taking the column with the largest gain in accuracy as the next most significant input. This procedure measures predictive importance without reference to the coefficients of , and it recovers the same top two invariants as the mean absolute coefficients in 10/10 runs and the same third invariant in 5/10 runs, where the third position varies between the meridional translation (imaginary part) and the cusp volume (which also appears at rank 3 in Table 2). However, the greedy selection requires evaluations of the test set, so the ranking by the mean absolute coefficients remains the practical choice when the number of inputs is large (e.g., for the MNIST dataset).
| Remove | Test acc. | |
| Top | Bottom | |
| 0 (baseline) | 93.6 ± 0.4% | 93.6 ± 0.4% |
| 1 | 30.9 ± 2.1% | 93.7 ± 0.4% |
| 2 | 28.8 ± 1.2% | 93.6 ± 0.4% |
| 3 | 25.2 ± 2.8% | 93.5 ± 0.4% |
Table A.1. Test accuracy of the PNAM with (without the Fourier feature mapping) in Table 1 after zeroing out the columns of the linear transformation associated with the top or bottom features of each run. The mean and standard deviation are computed over 10 runs without retraining.
Third, permutation importance—the drop in accuracy when the values of one input are shuffled across the test set—correlates with the mean absolute coefficients, yielding a Spearman rank correlation of ; the two rankings agree on the most important feature in 10/10 runs. Fourth, the gradient sensitivity over the test set, which incorporates the derivatives of the bases and cancellations across projections, gives , agreeing on the top-ranked feature in 9/10 runs, where the single disagreement selects the second-ranked feature. These checks complement the accuracies of retaining the top features in Table 2 and Fig. A.1(b).
Group-lasso MLP baseline. As an alternative to the ranking by the mean absolute coefficients, we retrain the MLP in Table 1 under an otherwise identical protocol with the added penalty , where is the th column of the first-layer weight matrix , rank the input features by , and sweep over . At every , this baseline is a weaker feature selector than the PNAM. At the representative value , it matches the unregularized MLP in accuracy (95.7 ± 0.1%) and its most important feature agrees with that of the PNAM in 10/10 runs, but the features ranked second and third vary across runs and never include the longitudinal translation or the meridional translation (imaginary part). Retaining its top three features produces a test accuracy of 50.2 ± 3.8%, against 78.1 ± 2.4% for the PNAM under the same input pruning (Table 2). This behavior persists for all . At , the selection improves, recovering the longitudinal translation among its top three features in 9/10 runs with a top-three accuracy of 75.5 ± 6.3%, but the accuracy with the full feature set falls to 86.8 ± 1.2%, below even that of the NAM. In the MLP, the deeper layers freely recombine the first-layer outputs, so the column norms become informative only once a sufficiently strong penalty concentrates the first-layer weights on a few inputs, by which point the accuracy has already degraded.
Ablation of the projection matrix. Table A.2 isolates the roles of learning and regularizing the linear transformation in one matched setting: knot classification with , where all models use 17 basis MLPs of identical architecture under the protocol of Table 1. First, random mixing is worse than no mixing at all, so the performance gain of PNAMs cannot be attributed to the projection mechanism alone or to the additional parameters of . The gain comes from learning the projection itself: the learned transformation with regularization weights outperforms the random transformation fixed at initialization by 6.5% and the identity transformation by 2.9%. Second, regularizing costs 0.3% in accuracy but, in exchange, stabilizes the feature ranking. In the regularized models, the three most significant invariants of Table 2 occupy ranks 1, 2, and 3 in 10/10, 10/10, and 8/10 runs, respectively. In the unregularized models, the top-ranked feature is stable, but the second-ranked feature varies across runs. The regularization weight thus allows the user to navigate the trade-off between accuracy and ranking reliability discussed in § 4.
| Test acc. | |
| Random, fixed at initialization | 88.8 ± 0.5% |
| Identity (= NAM, Table 1) | 92.4 ± 0.2% |
| Learned, regularized (Table 1) | 95.0 ± 0.2% |
| Learned, unregularized () | 95.3 ± 0.2% |
Table A.2. Ablation of the projection matrix on knot classification with . All models use 17 basis MLPs of identical architecture under the protocol of Table 1; the mean and standard deviation of the test accuracy are computed over 10 runs.
Alternative to the penalty . To assess the sensitivity of our results to the form of the penalty , we retrain the PNAM with (without the Fourier feature mapping) in Table 1 under an otherwise identical protocol, with replaced by in footnote 7. This substitution yields the same test accuracy (93.6 ± 0.4%) and the same top two invariants in 10/10 runs, with the invariant ranked third slightly more stable under (7/10 runs compared to 4/10). Our conclusions are thus insensitive to this design choice.
Additional results for knot theory: regression
In Table A.3, we compare the performance of the MLP and the NAM with 17 bases against that of the PNAM with eight bases under three values of the regularization weights . Both the NAM and the PNAM can be converted into symbolic expressions after training. Without regularization (), the PNAM is within 3.2% of the MLP. The larger gap at is the price of a sparser model with a more reliable ranking, and this model is chosen for symbolic distillation below. Given the poor performance of the NAM, we proceed with just the PNAM. Similar to Table 2, Table A.4 suggests that the meridional translation (real part) and the longitudinal translation are the most and second most important invariants, respectively. Nevertheless, solely comparing the mean absolute coefficients may not be sufficient, as unimportant features could be incorrectly ranked as important. We can, however, eliminate these features by examining the accuracy associated with including them, since including an unimportant feature does not improve accuracy compared to the corresponding case with one fewer input.
Table A.3: Performance comparison of different neural network architectures for the single-output regression problem of predicting the signature of mathematical knots. The mean and standard deviation of the test accuracy are computed over 10 runs. The three PNAMs use regularization weights ; Figs. A.4 and A.5 show their corresponding training and validation histories and the effect of on the input ranking, respectively.
| Method | Architecture | Parameter count | Test acc. |
| MLP | 3 layers: [17, 64, 32, 1] | ||
| NAM | 3 layers: [1, 64, 32, 1] | ||
| PNAM () | 3 layers: [1, 64, 32, 1] | ||
| PNAM () | 3 layers: [1, 64, 32, 1] | ||
| PNAM () | 3 layers: [1, 64, 32, 1] |
Table A.3.
Table A.4: Ranking of important features based on the mean absolute coefficients for the PNAM with in Table A.3. The mean and standard deviation of the test accuracy (associated with retaining only the top features) are computed over the runs in which the top features appeared.
| Rank | Input feature | Symbol | Frequency | Test acc. |
| 1 | Re(meridional translation) | 10/10 | ||
| 2 | Longitudinal translation | 6/10 | ||
| 2 | Torsion degree | 4/10 | ||
| 3 | Torsion degree | 3/10 | ||
| 3 | Re(short geodesic) | 3/10 | ||
| 3 | Longitudinal translation | 2/10 | ||
| 3 | Cusp volume | 1/10 | ||
| 3 | Im(short geodesic) | 1/10 |
Table A.4.
Figure A.2 shows a complete ranking of the inputs and the associated test accuracy of using the top inputs from one of the runs summarized in Table A.4. The ranking in Fig. A.2(d) is determined from the mean absolute coefficients of the linear transformation in Fig. A.2(b). As intended by the regularization in § 2.3, the learned (i) is predominantly diagonal due to the penalty in Eq. (14) and (ii) is sparse due to the constraint in Eq. (15), while retaining the essential off-diagonal entries. The post-processed transformation with in Fig. A.2(c) and the weighting coefficients of just two nonzero bases in Fig. A.2(a) are then leveraged to construct expressions and in Table A.5. Although each basis is evolved for only one minute using a few basic operators, Fig. A.3 demonstrates that SR can consistently approximate the single-variable bases because of their simple 1D nature. Expressions and achieve test accuracies of 81.2 and 81.4%, respectively, comparable to the test accuracy of 81.3% in Fig. A.2(e) for inputs. Moreover, Table A.6 reveals that both expressions are more than two times faster for inference than the fully connected MLP.
Recall that the above results correspond to the regularization weight , chosen to balance the trade-off between accuracy and sparsity. While increasing can make PNAMs sparser, Fig. A.4 indicates that overprioritizing sparsity can lead to unstable training, resulting in poorer performance. Specifically, setting results in such a sparse (see the small coefficient values in Fig. A.5(c)) that the accuracy in Fig. A.5(d) is poor regardless of . On the other hand, Fig. A.5(b) suggests that setting —thereby deactivating the regularization constraints—improves the accuracy for large , due to the extensive number of coefficients with large values in Fig. A.5(a), but reduces it for small , due to the erroneous ranking of the input features. Although the exact ranking may not be correct, this PNAM can still narrow down a set of variables (e.g., ) that is relevant for predictive accuracy while remaining competitive with the MLP in Table A.3.
High-dimensional image classification with the MNIST dataset
To demonstrate that PNAMs can efficiently handle high-dimensional problems with numerous inputs, we evaluate their performance on the MNIST dataset [47], highlighting their dimensionality reduction capability and providing visual insight into their input pruning mechanism. The MNIST dataset contains 70,000 images of handwritten digits from 0 to 9, each of size pixels, partitioned into
Figure A.2. One complete instance of the ranking and accuracy reported in Table A.4, along with optimized parameters of the PNAM. (a) Weighting coefficients prior to post-processing, (b) linear transformation prior to post-processing, (c) post-processed , retaining only the top features and zeroing out the rows that correspond to the zero-valued columns of in (a), (d) relative importance of the input features based on the mean absolute coefficients of in (b), and (e) test accuracy associated with using the top features in (d).
Figure A.3. Reproducibility of SR for learning 1D functions. Both sets of bases (PNAM) are from the run in Fig. A.2, with (a) the two bases (SR) comprising expression H and (b) the two bases (SR) comprising expression I in Table A.5. The neural network parameterization of the PNAM, expression H, and expression I all use features and achieve test accuracies of about 81%.
60,000 images for training/validation and 10,000 images for testing. Each image is reshaped into a 784-dimensional feature vector of raw pixel intensities, resulting in a high-dimensional classification
| ID | Expression | PC† | Discovered by | Eval. of test acc. Reported | Eval. of test acc. Our | Total acc. |
| A | 3 | Human (DM) | 83.1% | 74.5% | 73.8% | |
| B | 12 | [3, 1] KAN | 62.6% | 27.0% | 26.8% | |
| C | 17 | [3, 1, 1] KAN | 71.9% | 41.7% | 41.5% | |
| D | 29 | [3, 2, 1] KAN | 84.0% | – | – | |
| E | 7 | [3, 2, 1] KAN + Padé approx. | 82.8% | 79.3% | 79.3% | |
| F | 13 | [3, 1] KAN [3, 1] KAN | 77.8% | 27.0% | 26.8% | |
| G | 31 | PNAM () | – | 75.9% | 75.7% | |
| H | 52 | PNAM () | – | 81.2% | 80.9% | |
| I | 34 | PNAM () | – | 81.4% | 81.0% |
Table A.5. Mathematical expressions for the knot dataset. Inputs , , , and are the cusp volume, longitudinal translation, meridional translation (imaginary part), and meridional translation (real part), respectively. Expressions A to F are reproduced from Table 4 of Liu et al. [14]. A factor of is added to expression A for consistency with DeepMind’s findings [46, 58]. Expression D has missing parentheses, so it cannot be evaluated. A factor of is added to expression E for consistency with expression A. For expressions G and H, bases in Eq. (16) use addition, subtraction, multiplication, and square as operators; they additionally use exponential, sine, and tangent for expression I. Every constant in the expressions is counted as one parameter. See our code (footnote 2) to reproduce these results.
| Method | Training time (s/epoch) | Inference time (ns/sample) | Test acc. |
| MLP | 2.1 | 290 | |
| NAM | 14 | 4300 | |
| PNAM | 8.9 | 2100 | |
| Expression G | – | 78 | 75.9% |
| Expression H | – | 130 | 81.2% |
| Expression I | – | 120 | 81.4% |
Table A.6. Training and inference times of the neural networks in Table A.3, along with inference times of expressions G to I in Table A.5. Each basis comprising expressions G to I is evolved for one minute. Training times are obtained from an NVIDIA A100-SXM4-40GB GPU, while inference times are obtained from an Apple M1 CPU with 8 GB of memory. The test accuracy is repeated to aid comparison.
task where the model must map dense feature vectors to their corresponding digit labels. This experiment illustrates the ability of PNAMs to preserve interpretability while achieving strong predictive accuracy by identifying and utilizing only the most informative inputs, effectively pruning irrelevant features.
Figure A.4. Effects of regularization weights on the convergence of PNAMs for the knot dataset. Total loss corresponds to the loss function in Eq. (12). The three PNAMs use the same projection dimension , and each basis MLP has two hidden layers of 64 and 32 neurons.
Figure A.5. Effects of regularization weights on the input ranking and test accuracy of PNAMs for the knot dataset. Regularization weight for (a) and (b); for (c) and (d). (a) and (c): Relative importance of the input features based on the mean absolute coefficients. (b) and (d): Test accuracy associated with using the top features in (a) and (c), respectively. Results for are provided in Fig. A.2(d) and (e).
In Table A.7, we compare PNAMs against a standard NAM and fully connected MLPs, all of which are trained for up to 1000 epochs with an early stopping patience of 50 epochs. First, the linear transformation allows us to project the original 784-dimensional vector into a lower-dimensional feature space of size . Even with a small , the resulting PNAM outperforms the standard NAM, despite the latter requiring one basis per input and having nearly 100 times more parameters. Second, increasing enhances the expressivity of PNAMs, allowing them to fit the training data perfectly (see Fig. A.6) and achieve a test accuracy of 98.1% for . Third, although PNAMs perform similarly to MLPs, suggesting comparable expressive power in practice, they retain the key advantage of interpretability. Unlike black-box MLPs, PNAMs offer insight into which variables are crucial for predictive accuracy, enabling model transparency and post-hoc analysis.
| Method | Architecture | Parameter count | Training acc. | Test acc. |
| MLP | 3 layers: [784, 64, 32, 10] | 99.9% | 96.1% | |
| MLP | 3 layers: [784, 128, 64, 10] | 99.9% | 96.3% | |
| MLP | 4 layers: [784, 128, 64, 32, 10] | 99.9% | 96.7% | |
| MLP | 4 layers: [784, 256, 128, 64, 10] | 99.9% | 97.1% | |
| NAM | 3 layers: [1, 64, 32, 10] | 95.0% | 93.0% | |
| PNAM | 3 layers: [1, 64, 32, 10] | 96.0% | 94.7% | |
| PNAM | 3 layers: [1, 64, 32, 10] | 98.5% | 96.7% | |
| PNAM | 3 layers: [1, 64, 32, 10] | 100% | 97.5% | |
| PNAM | 3 layers: [1, 64, 32, 10] | 100% | 98.1% |
Table A.7. Performance comparison of different neural network architectures for the multi-class classification problem of predicting handwritten digits from the MNIST dataset. Training and validation histories of the PNAMs are provided in Fig. A.6.
Figure A.6. Effects of the projection dimension on the convergence and accuracy of PNAMs for the MNIST dataset. Training and validation histories of (a) the loss function in Eq. (12) and (b) the accuracy. Each basis MLP has two hidden layers of 64 and 32 neurons; all models are trained using regularization weights .
Figure A.7 illustrates that PNAMs produce a consistent ranking of feature importance across different values of , demonstrating robustness in their learned feature sensitivity. This consistency supports effective pruning of nonessential inputs with minimal impact on accuracy. Specifically, for the PNAM with , we remove the least important pixels and visualize two progressively smaller subsets in Fig. A.8. Retaining only the top 400 and 200 relevant pixels, this model still achieves test accuracies of 98.0 and 94.0%, respectively. Visual inspection of the remaining pixels reveals that they occupy semantically meaningful regions, allowing human observers to easily identify the underlying digit classes. Further insight is provided by Fig. A.9, which shows that the majority of predictive information is concentrated in a small number of dominant singular values, enabling both dimensionality reduction and feature pruning.
Finally, we examine the impact of the regularization weights on the performance of PNAMs. Similar to the trend observed in Fig. A.4, Fig. A.10 indicates that increasing the regularization weights leads to less stable training as the optimization objective shifts from prioritizing predictive accuracy to enforcing sparsity. Nevertheless, as shown in Table A.8, these regularization weights can be tuned to obtain a desired trade-off between accuracy and sparsity, tailored to the specific needs of the user.
Figure A.7. Impact of input pruning on the test accuracy for the PNAMs in Table A.7. Unimportant inputs are pruned according to the mean absolute coefficients (see § 2.4).
Figure A.8. Demonstration of important inputs for the PNAM with in Fig. A.7. Shown are the first 10 unique handwritten digits in the MNIST test set, arranged from 0 to 9, using (a) the () and (b) the () most relevant pixels, which achieve test accuracies of 98.0 and 94.0%, respectively.
Figure A.9. Singular values of the linear transformation for the PNAMs in Table A.7. The linear decay and sudden drop in magnitude of the singular values on a log scale suggest that we can prune coefficients in without severely degrading performance.
Phase field theory for fracture of brittle solids
We consider a phase field simulation of fracture in boron carbide () with isotropic elasticity and isotropic fracture energy from [48], for quasi-static extension up to peak load. The dataset contains 96 samples,17 homogenized from this simulation. These samples are snapshots along a single loading path, and the 80/20 train-test split of § 3 is sampled randomly along this path, so the reported mean squared error (MSE) measures interpolation within the loading path. Given the homogenized values of the axial strain, order parameter, and magnitude of the material gradient of the order parameter, the goal is to predict the average strain energy, phase energy, and axial stress. Among the inputs, the order parameter measures local damage from 0 (undamaged) to 1 (fully fractured), and the magnitude of its material gradient measures how localized the damage is. Physically, the strain energy is the elastic energy stored by deformation, the phase energy is the energy expended in creating fracture surfaces, and the stress is their total rate of change with strain.
Figure A.10. Effects of regularization weights on the convergence and accuracy of PNAMs for the MNIST dataset. Training and validation histories of (a) the loss function in Eq. (12) and (b) the accuracy. The four PNAMs use the same projection dimension , and each basis MLP has two hidden layers of 64 and 32 neurons.
| Training acc. | Test acc. | Percent of zero | |
| 0.001 | 95.6% | 94.3% | 14.1% |
| 0.01 | 96.0% | 94.7% | 32.8% |
| 0.1 | 95.7% | 94.7% | 68.9% |
| 1 | 90.9% | 91.3% | 76.5% |
Table A.8. Trade-off between accuracy and sparsity due to regularization weights for the PNAMs in Fig. A.10. An element with is considered zero, since setting such an element to zero has little to no effect on the test accuracy.
Since the stress is obtained by differentiating the sum of the two energies, we frame this task as a two-output regression problem. The MSE is employed as in Eq. (12) to predict the strain energy and phase energy; to predict the stress, we use the following form for the physical constraint .18
where and are the derivatives of the strain energy and phase energy, respectively, with respect to the strain . We set the regularization weight .19 In addition, every training and validation batch is augmented with one reference point at the undeformed state (zero inputs with zero targets), anchoring the predicted energies and stress at the origin.
All variables are scaled to have zero min and unit max. Table A.9 summarizes the performance of the MLP, the NAM, and four parameterizations of the PNAM, all utilizing three hidden layers of 256 neurons and trained for up to 5000 epochs with an early stopping patience of 50 epochs. On average, the test MSE of PNAMs decreases as the projection dimension increases [3]. Nevertheless, a PNAM with achieves the lowest validation MSE among all 60 networks underlying Table A.9, and we select this network for symbolic distillation. We then set in Eq. (16) to ensure accurate stress predictions and convert this network with approximately parameters into a symbolic form with roughly 100 parameters. As illustrated in Figs. A.11 and A.12, by employing SiLU as the activation function in Eq. (3), all bases can be approximated as single-variable polynomials, listed in Table A.10. Predictions of the strain energy, phase energy, and stress for the linear combinations of these polynomials are shown in Fig. A.13, achieving a test MSE of .
| Method | Architecture | Parameter count | Test MSE |
| MLP | 4 layers: [3, 256, 256, 256, 2] | ||
| NAM | 4 layers: | ||
| PNAM | 4 layers: | ||
| PNAM | 4 layers: | ||
| PNAM | 4 layers: | ||
| PNAM | 4 layers: |
Table A.9. Performance comparison of different neural network architectures for the two-output regression problem of predicting the strain energy and phase energy, from which the stress is derived. The mean and standard deviation of the combined test MSE for all three variables are computed over 10 runs.
Figure A.11. Bases and their derivatives. The expressions of are presented in Table A.10.
Table A.11 reports training and inference times for the neural networks in Table A.9, alongside the inference time of the symbolic form in Table A.10. As expected, training and inference times increase with . However, converting the PNAM into a compact symbolic expression yields faster inference, consistent with the knot theory results in Table A.6. Finally, Fig. A.14 reveals that increasing improves performance but may lead to overfitting after saturation.
Figure A.12. Bases and their derivatives. The expressions of are presented in Table A.10.
| Bases | Projected variables | ||
| Basis | Expression | Variable | Expression |
Table A.10. Mathematical expressions for the phase field dataset. Inputs , , and are the strain, order parameter, and magnitude of the material gradient of the order parameter, respectively. To convert the PNAM with in Table A.9 and (see § 2.4) into symbolic form, bases in Eq. (16) use addition, subtraction, multiplication, and square as operators. Counting every constant in and as one parameter, the symbolic form has 112 parameters and achieves a test MSE of .
Figure A.13. Test predictions for the linear combinations of the polynomials in Table A.10. (a) Linear combination of , (b) linear combination of , and (c) derivative of the sum of the linear combinations of and with respect to the strain.
| Method | Training time (s/epoch) | Inference time (s/sample) | Test MSE |
| MLP | 1.1 | 27 | |
| NAM | 3.0 | 61 | |
| PNAM () | 3.2 | 58 | |
| PNAM () | 6.8 | 150 | |
| PNAM () | 13 | 320 | |
| PNAM () | 26 | 610 | |
| Expression (Table A.10) | – | 17 |
Table A.11. Training and inference times of the neural networks in Table A.9, along with the inference time of the combined expression (i.e., the sum of the linear combinations of and ) in Table A.10. Each nonzero basis is evolved for one minute. Training times are obtained from an NVIDIA A100-SXM4-40GB GPU, while inference times are obtained from an Apple M1 CPU with 8 GB of memory. The test MSE is repeated to aid comparison.
Figure A.14. Effects of the projection dimension on the convergence of PNAMs for the phase field dataset. Total loss corresponds to the loss function in Eq. (12). Each basis MLP has three hidden layers of 256 neurons; all models are trained using regularization weights .
References
- [1]W. J. Murdoch, C. Singh, K. Kumbier, R. Abbasi-Asl, and B. Yu. Definitions, methods, and applications in interpretable machine learning. Proceedings of the National Academy of Sciences, 116(44):22071–22080, 2019.DOI
- [2]R. Agarwal, L. Melnick, N. Frosst, X. Zhang, B. Lengerich, R. Caruana, and G. E. Hinton. Neural additive models: Interpretable machine learning with neural nets. Advances in Neural Information Processing Systems, 34(359):4699–4711, 2021.arxiv.org/abs/2004.13912
- [3]N. N. Phan, W.C. Sun, and J. D. Clayton. HYDRA: Symbolic feature engineering of overparameterized Eulerian hyperelasticity models for fast inference time. Computer Methods in Applied Mechanics and Engineering, 437:117792, 2025.DOI
- [4]Y. Lou, R. Caruana, J. Gehrke, and G. Hooker. Accurate intelligible models with pairwise interactions. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 623–631, 2013.DOI
- [5]C.-H. Chang, R. Caruana, and A. Goldenberg. NODE-GAM: Neural generalized additive model for interpretable deep learning. arXiv preprint arXiv:2106.01613, 2021.arxiv.org/abs/2106.01613
- [6]Z. Yang, A. Zhang, and A. Sudjianto. GAMI-Net: An explainable neural network based on generalized additive models with structured interactions. Pattern Recognition, 120:108192, 2021.arxiv.org/abs/2003.07132
- [7]A. Dubey, F. Radenovic, and D. Mahajan. Scalable interpretability via polynomials. Advances in Neural Information Processing Systems, 35:36748–36761, 2022.arxiv.org/abs/2205.14108
- [8]J. Enouen and Y. Liu. Sparse interaction additive networks via feature interaction detection and sparse selection. Advances in Neural Information Processing Systems, 35:13908–13920, 2022.arxiv.org/abs/2209.09326
- [9]T. Hastie and R. Tibshirani. Generalized additive models. Statistical Science, 1(3):297–310, 1986.
- [10]V. E. Maiorov. On best approximation by ridge functions. Journal of Approximation Theory, 99(1):68–94, 1999.DOI
- [11]A. Pinkus. Approximation theory of the MLP model in neural networks. Acta Numerica, 8:143–195, 1999.DOI
- [12]C. Molnar. Interpretable Machine Learning: A Guide for Making Black Box Models Explainable. Leanpub, 2019.
- [13]Rudin, C. Chen, Z. Chen, H. Huang, L. Semenova, and C. Zhong. Interpretable machine learning: Fundamental principles and 10 grand challenges. Statistics Surveys, 16:1–85, 2022.arxiv.org/abs/2103.11251
- [14]Liu, Y. Wang, S. Vaidya, F. Ruehle, J. Halverson, M. Soljacic, T. Hou, and M. Tegmark. KAN: Kolmogorov–Arnold networks. In International Conference on Learning Representations, pages 70367–70413, 2025.DOI
- [15]G. G. Chrysos, S. Moschoglou, G. Bouritsas, J. Deng, Y. Panagakis, and S. Zafeiriou. Deep polynomial neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(8):4021–4034, 2022.DOI
- [16]M. Cranmer, A. Sanchez-Gonzalez, P. Battaglia, R. Xu, K. Cranmer, D. Spergel, and S. Ho. Discovering symbolic models from deep learning with inductive biases. Advances in Neural Information Processing Systems, 33(1462):17429–17442, 2020.
- [17]S. L. Brunton, J. L. Proctor, and J. N. Kutz. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the National Academy of Sciences, 113(15):3932–3937, 2016.DOI
- [18]G. Fang, X. Ma, M. Song, M. B. Mi, and X. Wang. DepGraph: Towards any structural pruning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16091–16101, 2023.arxiv.org/abs/2301.12900
- [19]M. Sundararajan, A. Taly, and Q. Yan. Axiomatic attribution for deep networks. In International Conference on Machine Learning, volume 70, pages 3319–3328. PMLR, 2017.arxiv.org/abs/1703.01365
- [20]A. Odrzywolek. All elementary functions from a single binary operator. arXiv preprint arXiv:2603.21852, 2026.
- [21]B. K. Petersen, M. Landajuela, T. N. Mundhenk, C. P. Santiago, S. K. Kim, and J. T. Kim. Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients. arXiv preprint arXiv:1912.04871, 2019.arxiv.org/abs/1912.04871
- [22]S.-M. Udrescu and M. Tegmark. AI Feynman: A physics-inspired method for symbolic regression. Science Advances, 6(16):eaay2631, 2020.DOI
- [23]M. Cranmer. Interpretable machine learning for science with PySR and SymbolicRegression.jl. arXiv preprint arXiv:2305.01582, 2023.arxiv.org/abs/2305.01582
- [24]M. Tancik, P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ramamoorthi, J. Barron, and R. Ng. Fourier features let networks learn high frequency functions in low dimensional domains. Advances in Neural Information Processing Systems, 33(632):7537–7547, 2020.arxiv.org/abs/2006.10739
- [25]B. Bahmani, H. S. Suh, and W.C. Sun. Discovering interpretable elastoplasticity models via the neural polynomial method enabled symbolic regressions. Computer Methods in Applied Mechanics and Engineering, 422:116827, 2024.DOI
- [26]F. Radenovic, A. Dubey, and D. Mahajan. Neural basis models for interpretability. Advances in Neural Information Processing Systems, 35(612):8414–8426, 2022.arxiv.org/abs/2205.14120
- [27]M. H. Stone. Applications of the theory of Boolean rings to general topology. Transactions of the American Mathematical Society, 41(3):375–481, 1937.
- [28]M. H. Stone. The generalized Weierstrass approximation theorem. Mathematics Magazine, 21 (4):167–184, 1948.
- [29]N. E. Cotter. The Stone–Weierstrass theorem and its application to neural networks. IEEE Transactions on Neural Networks, 1(4):290–295, 1990.pubmed.ncbi.nlm.nih.gov/18282849
- [30]K. Hornik, M. Stinchcombe, and H. White. Multilayer feedforward networks are universal approximators. Neural Networks, 2(5):359–366, 1989.DOI
- [31]M. Leshno, V. Y. Lin, A. Pinkus, and S. Schocken. Multilayer feedforward networks with a nonpolynomial activation function can approximate any function. Neural Networks, 6(6): 861–867, 1993.DOI
- [32]Z. Lu, H. Pu, F. Wang, Z. Hu, and L. Wang. The expressive power of neural networks: A view from the width. Advances in Neural Information Processing Systems, 30:6232–6240, 2017.arxiv.org/abs/1709.02540
- [33]R. A. DeVore and G. G. Lorentz. Constructive Approximation, volume 303 of A Series of Comprehensive Studies in Mathematics. Springer, Berlin, 1993.
- [34]D. Yarotsky. Error bounds for approximations with deep ReLU networks. Neural Networks, 94: 103–114, 2017.DOI
- [35]S. Xu, Z. Bu, P. Chaudhari, and I. J. Barnett. Sparse neural additive model: Interpretable deep learning with feature selection via group sparsity. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, volume 3, pages 343–359. Springer, 2023.arxiv.org/abs/2202.12482
- [36]W. M. Czarnecki, S. Osindero, M. Jaderberg, G. Swirszcz, and R. Pascanu. Sobolev training for neural networks. Advances in Neural Information Processing Systems, 30:4281–4290, 2017.arxiv.org/abs/1706.04859
- [37]M. Raissi, P. Perdikaris, and G. E. Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707, 2019.DOI
- [38]J.-H. Bastek, W.C. Sun, and D. Kochmann. Physics-informed diffusion models. In International Conference on Learning Representations, pages 3360–3385, 2025.arxiv.org/abs/2403.14404
- [39]A. Krogh and J. Hertz. A simple weight decay can improve generalization. Advances in Neural Information Processing Systems, 4:950–957, 1991.
- [40]D. Q. Huynh. Metrics for 3D rotations: Comparison and analysis. Journal of Mathematical Imaging and Vision, 35:155–164, 2009.DOI
- [41]R. Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B: Statistical Methodology, 58(1):267–288, 1996.
- [42]H. Cheng, M. Zhang, and J. Q. Shi. A survey on deep neural network pruning: Taxonomy, comparison, analysis, and recommendations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12):10558–10578, 2024.DOI
- [43]J. H. Holland. Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence. MIT Press, Cambridge, MA, 1992.
- [44]J. R. Koza. Genetic programming as a means for programming computers by natural selection. Statistics and Computing, 4:87–112, 1994.DOI
- [45]M. Virgolin and S. P. Pissis. Symbolic regression is NP-hard. arXiv preprint arXiv:2207.01018, 2022.arxiv.org/abs/2207.01018
- [46]A. Davies, P. Veličković, L. Buesing, S. Blackwell, D. Zheng, N. Tomašev, R. Tanburn, P. Battaglia, C. Blundell, A. Juhász, et al. Advancing mathematics by guiding human intuition with AI. Nature, 600:70–74, 2021.DOI
- [47]Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
- [48]J. D. Clayton, J. Knap, and R. B. Leavy. On rate dependence and anisotropy in phase field modeling of polycrystalline fracture. Mechanics of Materials, 180:104606, 2023.DOI
- [49]A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al. PyTorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems, 32(721):8026–8037, 2019.arxiv.org/abs/1912.01703
- [50]D. Hendrycks and K. Gimpel. Gaussian error linear units (GELUs). arXiv preprint arXiv:1606.08415, 2016.
- [51]S. Elfwing, E. Uchibe, and K. Doya. Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. Neural Networks, 107:3–11, 2018.DOI
- [52]D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.arxiv.org/abs/1412.6980
- [53]N. Macon and A. Spitzbart. Inverses of Vandermonde matrices. The American Mathematical Monthly, 65(2):95–100, 1958.DOI
- [54]L. R. Turner. Inverse of the Vandermonde matrix with applications. Technical report, NASA Lewis Research Center, Cleveland, OH (United States), 1966.
- [55]J. C. A. Barata and M. S. Hussein. The Moore–Penrose pseudoinverse: A tutorial review of the theory. Brazilian Journal of Physics, 42:146–165, 2012.DOI
- [56]J. M. Landsberg. Tensors: Geometry and Applications, volume 128 of Graduate Studies in Mathematics. American Mathematical Society, Providence, RI, 2012.DOI
- [57]J. Alexander and A. Hirschowitz. Polynomial interpolation in several variables. Journal of Algebraic Geometry, 4(2):201–222, 1995.
- [58]A. Davies, A. Juhász, M. Lackenby, and N. Tomašev. The signature and cusp geometry of hyperbolic knots. Geometry & Topology, 28(5):2313–2343, 2024.DOI
- [59]N. Mallinar, J. Simon, A. Abedsoltan, P. Pandit, M. Belkin, and P. Nakkiran. Being, tempered, or catastrophic: A taxonomy of overfitting. Advances in Neural Information Processing Systems, 35(87):1182–1195, 2022.arxiv.org/abs/2207.06569
- [60]M. E. Gurtin. Generalized Ginzburg–Landau and Cahn–Hilliard equations based on a microforce balance. Physica D: Nonlinear Phenomena, 92(3-4):178–192, 1996.DOI
- [61]C. Miehe, F. Welschinger, and M. Hofacker. Thermodynamically consistent phase-field models of fracture: Variational principles and multi-field FE implementations. International Journal for Numerical Methods in Engineering, 83(10):1273–1311, 2010.DOI













