betaFeedback

WorksComputer Science

Connectivity Optimized Nested Graph Networks for Crystal Structures

Abstract

Graph neural networks (GNNs) have been applied to a large variety of applications in materials science and chemistry. Here, we recapitulate the graph construction for crystalline (periodic) materials and investigate its impact on the GNNs model performance. We suggest the asymmetric unit cell as a representation to reduce the number of atoms by using all symmetries of the system. This substantially reduced the computational cost and thus time needed to train large graph neural networks without any loss in accuracy. Furthermore, with a simple but systematically built GNN architecture based on message passing and line graph templates, we introduce a general architecture (Nested Graph Network, NGN) that is applicable to a wide range of tasks. We show that our suggested models systematically improve state-of-the-art results across all tasks within the MatBench benchmark. Further analysis shows that optimized connectivity and deeper message functions are responsible for the improvement. Asymmetric unit cells and connectivity optimization can be generally applied to (crystal) graph networks, while our suggested nested graph framework will open new ways of systematic comparison of GNN architectures.

Introduction

Since seminal work by Duvenaud et al. [ [1], graph neural networks (GNNs) developed into one of the most versatile and accurate classes of machine learning models for the prediction of molecular and material properties. Consequently, GNNs find increasing application in materials sciences for structure-property predictions [2], materials screening [3] and high-throughput simulations [4]. Learning on experimental or simulated databases [5, 6, 7, 8], GNNs show promising potential to develop new materials to tackle societies growing demand for high-performance materials in the fields of catalysis, renewable energies, energy conversion or functional materials [9, 11].

Graph convolutional neural networks operate on the (spatial) graph structure to transform node embeddings and have been suggested for semi-supervised node classification [12, 13]. With the focus on molecular graphs, the message passing framework (MPNN) was proposed by Gilmer et al. [ [14] in order to group and generalize many GNN architectures that update node representations by iteratively exchanging information, or messages, across edges in the graph [15, 16]. Neural networks designed to predict the potential energy surface of molecules for molecular dynamics simulations [17], such as the continuous-filter convolutional network SchNet [18], can also be interpreted as MPNN graph networks. Significant progress was achieved by incorporating further geometric information such as bond [19] and dihedral angles [20] into the MPNN scheme. To compute explicit angle information, the graph network has to operate on higher order pathways or connect edge information in a so-called line graph, which sets up a graph on edges L(G)L(G), i.e. on-top of the underlying graph GG [21]. A state-of-the-art model which makes use of this principle and reaches good performance for materials is ALIGNN [22]. Recently, equivariant graph networks have been introduced [23], which build on irreducible representations of the euclidean symmetry groups to achieve equivariance under e.g. rotation and translation [24]. Although many of the previously mentioned GNNs can or have been applied to materials, fewer architectures are developed with a primary focus on crystalline systems. The crystal-graph convolution neural network (CGCNN) first introduced a GNN architecture on a crystalline system by constructing a multi-graph that correctly represents the atomic neighbors in a periodic system [25]. Its improved version iCGCNN incorporates information on the Voronoi tessellated crystal structure and explicit three-body correlations of neighboring constituent atoms [26]. MEGNet further leverages global state information and added edge updates in the convolution process [27]. With GeoCGNN a geometric GNN was introduced [28], which encodes the local geometrical information using an attention mask composed of Gaussian radial basis functions and plane waves from a k-point mesh of Monkhorst Pack special points [29]. Although many GNN model architectures have been proposed in this context [25, 27, 30, 26, 28, 22, 31], there does not yet seem to be unanimous consent in the literature on which is the best method or the most decisive tool in geometric deep learning to process crystalline materials.

In most cases, newly introduced GNNs make specific improvements over previous architectures and propose multiple reasonable new design decisions inspired by chemical domain knowledge. While this approach has so far led to a consistent improvement of model accuracy, we choose a more systematic approach inspired by the work of You et al. [2020]: First, we stake out a new design space based on an extension of the original graph network (GN) framework [33], namely nested graph networks (NGNs). Then we navigate through the design space to find suitable NGN architectures.

In this work, we re-evaluate edge selection methods to build a multi-graph as suggested by Xie and Grossman [2018] and compare their performance on a wide set of parameters. In this context, we introduce the asymmetric unit cell as a representation to further exploit crystal symmetries and effectively reduce the number of edges. Next, we develop a connectivity-optimized crystal graph network (coGN) from message passing and line-graph templates which are intentionally kept as general as possible. By optimizing within the generalized family of GNNs, we improve state-of-the-art results on 6 out of 9 of the MatBench benchmark datasets [34] and achieve close to parity results with the best models on the remaining datasets, making our model the best general model on the entire set of diverse tasks in the benchmark.

Crystal Graph Construction

There are two main challenges when trying to build graph representations of crystal structures in contrast to organic molecules: (a) Bonds between atoms in crystals have more diverse types (covalent, ionic, metallic bonds), or are often not well defined at all. (b) Crystal structures have no fixed finite size, as they are defined as periodic repetitions of a unit cell.

Edge selection The first aspect raises the question of which edges to add to the graph that describes the crystal. To circumvent this problem one relies on the geometrical properties of the atom arrangement instead of chemically informed bonds.

Figure 1 schematically shows a two-dimensional crystal pattern and three different methods for selecting edges between atoms. The k-nearest-neighbors approach (Figure 1a) depends on the number of neighbors kk, but can lead to largely different edge distances when the crystal density varies. The radius-based approach (Figure 1b) limits the distance between two nodes by a hyperparameter rr, but the number of neighbors is unbounded and the method can lead to either disconnected or overly dense graphs if rr is chosen inappropriately. The parameter-free Voronoi-based approach (Figure 1c) leads to an intuitive edge selection where edges are drawn between two atoms if there is a Voronoi cell ridge between them. However, at least in theory, the number of edges and their distances are also unbounded for this approach.

Edge selection for a single atom in a crystal structure

Figure 1. Edge selection for a single atom in a crystal structure. Red atoms mark the cubic unit cell.

All three edge selection methods have been used previously in the context of crystals and GNNs [27, 26, 25, 35]. But to our knowledge, there is no detailed comparison between the methods and hyperparameters.

Exploiting crystal symmetries In contrast to molecules, crystal structures are modeled as (infinite) periodic repetitions of the unit cell atom motif. A direct approach to extracting a finite graph for a given crystal structure is to simply select all unit cell atoms with their respective neighbors (Figure 2a). In an information-theoretical sense, this representation captures the entire information of the crystal for a given edge selection method. However, from a naive message-passing perspective, only the nodes representing atoms inside of the central unit cell receive messages from all their neighboring atom nodes, which would model a finite graph rather than an infinite periodic graph.

Graph representations of crystal structures exploiting different levels of symmetries

Figure 2. Graph representations of crystal structures exploiting different levels of symmetries. The asymmetric unit cell is shown in (c)

To solve this issue, Xie and Grossman [2018] proposed a multigraph crystal representation (Figure 2b) introducing periodic/cyclic boundary conditions, which we will refer to as a unit cell graph. In this representation, one node represents all the shift-equivalent atoms of the crystal, which usually results in multiple edges with distance information matching their translated lattice positions. As a consequence, GNNs will always provide equivalent node embedding for periodic atoms, consistent with Bloch’s theorem.

The periodicity of the unit cell graph represents translation symmetry. Since crystals often exhibit more symmetries, we propose the asymmetric unit graph representation for crystals, which considers all symmetries of a crystal structure.1 In this representation, all symmetry-equivalent atoms are represented by a single node in the graph. The example crystal in Figure 2 exhibits a horizontal reflection symmetry. The two yellow atoms in the crystal unit cell are symmetry-equivalent and therefore merged into one node (with multiplicity two) for the asymmetric unit cell (Figure 2c). Just like in unit cell graphs, self-loops and multi-edges can occur in asymmetric unit cell graphs.

Since physical target properties in ML tasks are often invariant under E(3) symmetry operations, many GNNs are designed to be E(3)-invariant, but as consequence, yield equal node embeddings for symmetrical atoms in the unit cell graph, leading to redundant computations in the message passing step. The asymmetric unit graph representation can further remove these redundancies and yet maintain the same node embeddings xvx_v.2 However, global readout operations have to be adapted to handle asymmetric unit cells, since atoms can have different symmetry-related multiplicities mvm_v (the number of symmetry equivalent atoms for each equivalence class) A simple adaptation of the readout function can fix the issue and restore equal results for unit cell graphs and asymmetric unit graphs:

agg⁡′(xv)={agg⁡v∈V(xv⋅mv)⋅∣V∣∑vmvfor mean or attentionagg⁡v∈V(xv⋅mv)for sumagg⁡v∈V(xv)for min or max\operatorname{agg}^{\prime}(x_v)= \begin{cases} \operatorname{agg}_{v\in V}(x_v\cdot m_v)\cdot\frac{|V|}{\sum_{v}m_v} & \text{for mean or attention}\\ \operatorname{agg}_{v\in V}(x_v\cdot m_v) & \text{for sum}\\ \operatorname{agg}_{v\in V}(x_v) & \text{for min or max} \end{cases}

Nested Graph Network Framework

In addition to the choice of input representation, the GNN model architecture has a substantial impact on the quality of crystal graph property predictions.

To find the best GNN architecture for a certain task, it is instructive to systematically explore (see e.g. [32]) the design space of GNN modules and building blocks. Moreover, a framework has to be chosen on how to define and process GNN modules. The Message Passing Framework by Gilmer et al. [ [14], for example, has shown that a framework can unify and accelerate the efforts of the scientific community.

For a combinatorial generalization of GNNs, we build upon the graph network (GN) framework of Battaglia et al. [ [33] in this work. In this framework, one GNN layer is described by a GN block, which transforms a generic attributed graph with edge, node, and global graph features via three update functions ϕ\phi and three aggregation functions ρ\rho.

However, many state-of-the-art models such as DimeNet [43 b], ALIGNN [22] and M3GNet [31], which incorporate many-body interactions between atoms, are only indirectly captured by GNs. Therefore we propose an extension to the framework which we term nested graph networks (NGNs). In the following, we introduce NGNs by first discussing how angle information is incorporated via the line graph concept [22] and then explaining the flow of NGN calculations, before we will show concrete examples of implementations of NGNs in § 5.1.

To achieve rotation invariance, popular models such as SchNet [18] or MEGNet [27] only include scalar distances between two atoms as edge features. However, to incorporate more geometric information, models such as DimeNet [43] or ALIGNN [22] additionally use E(3)-invariant angle information, represented by combinations of edges (see Figure 9). In the line graph L(G)L(G), which can be constructed uniquely based on GG (see Figure 3 and [38]), there is an edge eeij,ejkL(G)e_{e_{ij},e_{jk}}^{L(G)} for every two incident edges eij,ejke_{ij},e_{jk} in GG. This enables the assignment of angular information between the adjacent atoms to the edges of the line graph. The same applies to (generalized) dihedral angles (4-node or 3-edge objects) in the second-order line graph L(L(G))L(L(G)).

Figure 3. On the left: A graph GG and the construction of its line graph L(G)L(G). On the right: Nested graph network architecture with GN blocks working on GG and containing other GN blocks working on the line graph L(G)L(G) as edge update function ϕE\phi_E. The line graph is able to process multi-node geometric features, such as angles (see Table 1)

Figure 9. Two different geometric graphs which are undistinguishable from distance edge features, but become distinguishable with angle information between edges or by adding the dashed edge to the graphs.

NGNs operate on the graph GG as well as the line graph L(G)L(G) (potentially also L(L(G))L(L(G)) etc.), exploiting the one-to-one mapping of edges in GG and nodes in L(G)L(G) (see Table 1). Each edge update function ϕE\phi_E in GN blocks that operate on GG can be instantiated as a nested GN (see Figure 3). A more detailed description of the algorithm can be found in Appendix 7.3. The NGN framework extends the usual sequential compositionality of GN blocks with a hierarchical/nested compositionality, thereby increasing the expressiveness [39]. The composition of simple and well-known building blocks facilitates the implementation and the ease of understanding of the framework. Furthermore, NGNs generalize existing models such as SchNet [18], DimeNet [43] and ALIGNN [22], making them more comparable and extensible (see Figure 4).

Entity in CrystalGGL(G)L(G)L(L(G))L(L(G))
AtomsNodes
BondsEdgesNodes
AnglesEdgesNodes
DihedralsEdges

Table 1. Correspondence between entities in the crystal, the Crystal Graph GG, its Line Graph L(G)L(G), etc.

Figure 4. Other models in the NGN framework.

Line Graph Construction

Mathematically a line graph for a directed graph is defined as follows: An edge exists in L(G)L(G) for each corresponding edge pair (eij,ejk)(e_{ij}, e_{jk}), which forms a path of length two in GG (Figure 5a). This definition coincides with the angles ∠eijejk\angle e_{ij}e_{jk} as used in in DimeNet Klicpera et al., [43], GemNet Gasteiger et al., [40] and ALIGNN Choudhary and DeCost, [22]. In this work, we propose a deviation from the original line graph definition and use angles ∠eijekj\angle e_{ij}e_{kj} between edges that have the same destination node, instead of edges that form a path of length two (see Figure 5b). The intuition behind this deviation is based on the interpretation of edge messages in the GNN as force contributions that act on an atom from different directions Defferrard et al., [41].

Line graph variants

Figure 5. Line graph variants.

Note that we will use the term line graph very loosely in this work. Even though the variant is strictly speaking not a line graph we will still refer to it as such. In general, we will use the term line graph for every graph L(G)L(G) that has a bijection between edges in GG and nodes in L(G)L(G) and a deterministic method for constructing edges in L(G)L(G) based on the topology of GG.

This generalized definition would also allow for incorporating dihedral angles by adding edges ∠eijekl\angle e_{ij}e_{kl} to the line graph for every path (eij,ejk,ekl)(e_{ij}, e_{jk}, e_{kl}) of length three in the graph GG [42, 43, 20]. Another way to incorporate information of dihedral edges into the model would be to construct the second-order line graph L(L(G))L(L(G)) and add another layer of nesting to the model architecture.

Evaluation

Datasets To evaluate the preprocessing methods and GN architectures we relied on the MatBench benchmark [34], which can be considered to be the ImageNet [44] for machine learning models in materials science. The benchmark currently consists of 13 strictly standardized supervised crystal property prediction tasks curated from different data sources [45, 46, 47, 48]. Out of the 13 tasks, four provide only the crystal composition as input and the other nine incorporate the crystal structure with the geometric arrangement of atoms. Since this work focuses on crystal structures specifically, we only use the nine datasets that rely on structural information for property prediction. The size of the datasets ranges from 636 to 132,752 crystal instances.

Implementations We used the Keras Graph Convolution Neural Networks (KGCNN) library [49] to implement the GN and NGNs used in the experiments of this work. The code for crystal preprocessing and GNN models is available online3

GNN Architecture Search

To find a suitable GN architecture, we conduct an architecture search within the NGN framework. In order to generalize previous models, our NGN framework is very flexible and spans a large hyperparameter space for architectural decisions but which makes the exploration of the entire space infeasible. If no nesting is explicitly required, the NGN framework falls back to the GN framework of Battaglia et al. [ [33]. For comparison, we search architectures with and without nesting, which lead to the connectivity optimized graph architectures coGN and coNGN, respectively. The search procedure and architecture details is discussed in the following.

Graph Network Exploration We start with a simple non-nested architecture shown in Figure 6, which consists of sequentially connected GN blocks, which can be divided into three phases.

Figure 6. Basic graph network structure.

Moreover, we optimize the hyperparameter space on one dataset only, namely the log_gvrh MatBench dataset, and tested the model on all other datasets of MatBench without changing hyperparameters to verify that the found architecture is indeed a suitable candidate for multiple diverse tasks on crystal graphs.

The first block embeds atom/node features and bond/edge features independent from one another. In the second phase, we sequentially connect TT processing blocks with identical architecture, but independent learnable parameters. To restrict the search space we do not include global graph features and do not allow edge updates between blocks. This resembles a conventional message-passing architecture. In the third phase, we have a single readout block, which aggregates node features into graph-level features to make the crystal property prediction.

Figure 6 only specifies the high-level architecture of the GNN and not the concrete instantiations of update functions ϕE\phi_E, ϕV\phi_V, ϕG\phi_G and aggregation functions ρE→V\rho_{E\to V}, ρV→G\rho_{V\to G}. To narrow down suitable concrete implementations, we conducted a two-part hyperparameter search on the log_gvrh MatBench dataset. First, we searched for categorical hyperparameters, in a greedy stepwise search. In the second step, we used the Optuna hyperparameter optimization framework [50] and its implementation of the Tree-structured Parzen Estimator (TPE) to find ordinal hyperparameters.4

We assume independence between categorical hyperparameters and optimize for each parameter individually, to keep runtimes within limits. We initially instantiated all update functions ϕE\phi_E, ϕV\phi_V, ϕG\phi_G with MLPs of depth 3. Extending ϕV\phi_V with residual ϕV(xv,x^v,xG)=xv+MLPV(x^v)\phi_V(x_v,\hat{x}_v,x_G)=x_v+\mathrm{MLP}_V(\hat{x}_v) and gated node updates ϕV(xv,x^v,xG)=GRU(xv,MLPV(x^v))\phi_V(x_v,\hat{x}_v,x_G)=\mathrm{GRU}(x_v,\mathrm{MLP}_V(\hat{x}_v)) both improve performance. For aggregation function ρV\rho_V, ρG\rho_G we tried mean, sum, and attention-based functions. Batch and graph normalization [51] increased training time significantly without considerable benefits for predictive performance. Prediction accuracy further increases when including atom features (atomic mass, radius, electronegativity, ionization, and oxidation states) in addition to the atomic number. The influence of the choice of edge selection methods shown in Figure 8 is discussed in detail in § 5.5.

MAE for GNs, NGNs, and different edge selection methods on the log_gvrh dataset

Figure 8. MAE for GNs, NGNs, and different edge selection methods on the log_gvrh dataset.

For the kkNN-based edge selection preprocessing with k=24k=24, we searched for ordinal hyperparameters with a hyperparameter optimization.

Embedding Block

ϕE\phi_E: Gauss basis expansion of edge distances with 32 Gaussians evenly spread on the [0,8][0,8] Å interval. As only distance information is used, this embedding is E(3)-invariant. The initial representation is projected into a 128-dimensional embedding space with a single perceptron layer.

ϕV\phi_V: Embedding of atom features (atomic number, atomic mass, atomic radius, electronegativity, ionization states, and oxidation states) into a 128-dimensional space.

Processing Blocks

Five processing blocks are concatenated with identical configurations (T=5T=5).

ϕE(xeij,xvi,xvj)=MLPE(xeij∥xvi∥xvj)=xeij′\phi_E(\mathbf{x}_{e_{ij}},\mathbf{x}_{v_i},\mathbf{x}_{v_j})=\mathrm{MLP}_E(\mathbf{x}_{e_{ij}}\mathbin\Vert\mathbf{x}_{v_i}\mathbin\Vert\mathbf{x}_{v_j})=\mathbf{x}'_{e_{ij}} The edge update function, which constructs the message between two nodes, is a five-layer MLP and takes the concatenation of edge, receiver, and sender node features.

ρE→V({xeij′∣j=k})=∑j=kxeij′=x^vk\rho_{E\to V}(\{\mathbf{x}'_{e_{ij}}\mid j=k\})=\sum_{j=k}\mathbf{x}'_{e_{ij}}=\hat{\mathbf{x}}_{v_k} For each node the incoming messages are sum-aggregated.

ϕV(xv,x^v)=xv+MLPV(x^v)\phi_V(\mathbf{x}_v,\hat{\mathbf{x}}_v)=\mathbf{x}_v+\mathrm{MLP}_V(\hat{\mathbf{x}}_v) A residual node update function transforms the aggregated messages with a single-layer perceptron.

Readout Block

ϕV(xv)=xv\phi_V(\mathbf{x}_v)=\mathbf{x}_v The readout block does not update node features. Its node update function is the identity function.

ρV→G({xv′∣v∈V})=mean⁡v∈V(xv′)=x^G\rho_{V\to G}(\{\mathbf{x}'_v\mid v\in V\})=\operatorname{mean}_{v\in V}(\mathbf{x}'_v)=\hat{\mathbf{x}}_G The mean of the node features is aggregated to compute the graph-level prediction.

ϕG(x^G)=MLPG(x^G)=xG′\phi_G(\hat{\mathbf{x}}_G)=\mathrm{MLP}_G(\hat{\mathbf{x}}_G)=\mathbf{x}'_G A single-layer MLP with a linear activation function creates the final prediction for the crystal property.

We used the same dimensionality (128) for all hidden representations of edges, nodes, and graphs. Unless mentioned otherwise we use the commonly used swish activation function in MLPs. The GNN is trained with an Adam optimizer with a linear learning rate scheduler for 800 epochs.

Overall, coGN is comparably simple, as it only contains MLPs as update functions, mean or sum aggregation functions and no sophisticated message-passing scheme, such as edge-gated or attention-based message passing used in CGCNN, ALIGNN or GeoCGNN. In this regard, coGN also does not incorporate any domain-specific design decisions, which are not justified by the hyperparameter optimization.

When compared with previous models, we find that along with a connectivity optimization, discussed in section 5.2, a much deeper edge update network ϕE\phi_E, in our case five layers, yields better results. We attribute this observation to the increased complexity of edge information in a periodic multi-graph that in practice exhibits a large number of edges per node.

Nested Graph Network Exploration. Based on the results of the first part of the hyperparameter search for parameters with categorical values, we augment the processing blocks with nested GN blocks shown in Figure 7.

Nested graph network variants

Figure 7. (a) NGN variant 1. (b) NGN variant 2. (c) NGN variant 3. Nested graph network variants.

The first variant (Figure 7a) has a single Nested GN block at the beginning, which updates node and edge features, followed by non-nested GN blocks. The reasoning behind this architecture is that the Nested GN block might be able to encode geometrical constellations of edge angles into edge features. The second variant (Figure 7b) is a generalized version of ALIGNN-d [42] and DimeNet(++) [43 c]. Each GN block on the graph level (GG) has two GN blocks on the line graph level (L(G)L(G)). Only node features are updated between GN blocks. The third variant (Figure 7c) moves most of the computation to the line graph level (L(G)L(G)). It consists of only one GN block on the graph level with 4 consecutive nested GN blocks (TL(G)=4T^{L(G)} = 4).

E(3)E(3)-invariant angle features between two edges are encoded with a 16-dimensional Gauss basis expansion on the [0,π][0, \pi] rad interval and attached to line graph edges.

Training on preprocessed crystals with kkNN edge selection and k=24k = 24 and both line graph variants from § 4, we obtain the results displayed in Table 2. The results were obtained before the final ordinal hyperparameter optimization, which explains the discrepancy with Table 3. For comparison, the MAE for the corresponding non-nested GN is 0.0788.

Nested GNLine GraphMAE (log_gvrh)
Variant 1∠eijejk\angle e_{ij}e_{jk}0.0809±0.00220.0809 \pm0.0022
Variant 1∠eijekj\angle e_{ij}e_{kj}0.0787±0.00190.0787 \pm0.0019
Variant 2∠eijejk\angle e_{ij}e_{jk}0.0801±0.00200.0801 \pm0.0020
Variant 2∠eijekj\angle e_{ij}e_{kj}0.0783±0.00330.0783 \pm0.0033
Variant 3∠eijejk\angle e_{ij}e_{jk}0.0805±0.00150.0805 \pm0.0015
Variant 3∠eijekj\angle e_{ij}e_{kj}0.0799±0.00160.0799 \pm0.0016

Table 2. Results on the log_gvrh dataset with NGN variants.

DatasetcoGN (ours)coNGN (ours)ALIGNNMODNetCGCNNM3GNetMatformer
e_form ↓17.0 ± 0.317.8 ± 0.421.5 ± 0.544.8 ± 3.933.7 ± 0.619.5 ± 0.221.232 ± 0.302
is_metal* ↑0.9124 ± 0.00230.9089 ± 0.00190.9128 ± 0.00150.9038 ± 0.01060.9520 ± 0.00740.958 ± 0.0010.906 ± 0.002
gap ↓155.9 ± 1.7169.7 ± 3.5186.1 ± 3.0219.9 ± 5.9297.2 ± 3.5183 ± 5187.825 ± 3.817
perovskites ↓26.9 ± 0.829.0 ± 1.128.8 ± 0.990.8 ± 2.845.2 ± 0.733 ± 1.031.514 ± 0.71
log_kvrh ↓0.0535 ± 0.00280.0491 ± 0.00260.0568 ± 0.00280.0548 ± 0.00250.0712 ± 0.00280.058 ± 0.0030.063 ± 0.0027
log_gvrh ↓0.0689 ± 0.00090.0670 ± 0.00060.0715 ± 0.00060.0731 ± 0.00070.0895 ± 0.00160.086 ± 0.0020.077 ± 0.0016
dielectric ↓0.3088 ± 0.08590.3142 ± 0.07400.3449 ± 0.08710.2711 ± 0.07140.5988 ± 0.08330.312 ± 0.0630.634 ± 0.131
phonons ↓_29.712 ± 1.997_28.887 ± 3.284_29.539 ± 2.115_34.275 ± 2.078157.7635 ± 12.31134.1 ± 4.542.526 ± 11.886
jdft2d ↓_37.165 ± 13.683__36.170 ± 11.597_43.424 ± 8.94933.192 ± 7.34349.244 ± 11.58750.1 ± 11.942.827 ± 12.281

Table 3. Comparison with results of state-of-the-art model on MatBench structure datasets and splits [34] (status 2023-08-08). Ordered by descending cardinality, these are e_form (meV/atom), is_metal (AUC(ROC)), gap (meV), perovskites (meV/unit cell), log_kvrh (log10(GPa)), log_gvrh (log10(GPa)), dielectric (unitless), phonons (1/cm) and jdft2d (meV/atom). Current benchmark holders on MatBench, namely, ALIGNN [22], MODNet [52] and CGCNN [25] are listed. Additionally, recent models M3GNet [31] and Matformer [53] are added, which have been published during the preparation of this work. Since Matformer was not trained on the official benchmark, we retrained the original model. The best results are indicated in bold font, while other results within one standard deviation are underlined. *For is_metal there is a discrepancy between the submissions for AUC computation (which is why the classification metric is likely to change in future version).

Despite the theoretically greater expressiveness of NGNs, we do not achieve substantially better prediction results. The proposed line graph variant with angles between edges with the same target node (∠eijekj\angle e_{ij}e_{kj}) leads to a small but consistent improvement across all line graph variants.

We optimized the ordinal hyperparameters of the DimeNet-like architecture (Variant 2) with TPE and reached a MAE of 0.0705 on the log_gvrh MatBench dataset which yields better performance than the current leader on MatBench [22], even though the graph preprocessing, i.e. the connectivity was not optimized yet. Hyperparameter optimization of plain GNs yields a similar error of 0.0693, making them comparable to Nested GNs.

Crystal Graph Connectivity

For GNNs in particular, there is a strong interdependency between input representation and model, because the topology of the input graph also affects the computational graph. The interdependency between preprocessing of crystals and model architecture also occurs with respect to crystal property predictions and should therefore be considered.

Figure 8 shows the effect of different preprocessing methods for a non-nested GN. Again, results were obtained before the final ordinal hyperparameter optimization, which explains the discrepancy with Table 3. The MAE is plotted as a function of the average degree over the resulting graphs of the log_gvrh dataset for different edge selection methods. The accuracy of GNN models tends to fall for increasing graph connectivity up to an average degree of about 30. For a higher average degree the accuracy either gets worse or quickly converges depending on the dataset. Interestingly, for the kkNN edge selection the minimum corresponds to k=24k = 24, whereas other works use a value of 12 [25, 22]. The Voronoi-based edge selection results in graphs with an average degree of around 12. Adding the area of the Voronoi cell ridge to each edge as an additional edge feature can improve the predictive power of the GNN. We found, however, that this effect is much less pronounced after optimization of ordinal hyperparameters, making the k=24k = 24 nearest-neighbor method the preferred choice for edge selection in our experiments.

The observations for non-nested GNs do not apply to NGNs, when comparing their behavior in Figure 8. For NGNs, the minimum test error occurs at a lower average degree of around 12 edges per node and is followed by a steeper increase for higher connectivity. Higher-order nesting could potentially lead to further improvement at yet lower connectivity of the base graph, which should be systematically explored in the future.

At the same time, our observations raise the issue of a trade-off between nesting and graph connectivity. This trade-off can also be explained from an analytical point of view. The reason for including angle information with NGNs is shown in the example in Figure 9. The two geometric graphs are not distinguishable for GNNs from relative distance information alone. Incorporating angles between edges into the GNN architecture increases expressiveness and allows for the distinction of the graphs. Yet, similar enhancements of expressiveness can also be achieved by increasing graph connectivity. In the example, adding the single dashed edge between two red nodes makes the graphs distinguishable for GNNs, without any angle information.

From our experiments, we can not conclude that NGNs offer a systematic advantage over simple GNs when used on graphs with high connectivity. Inspired by the results of Figure 8, we optimized an NGN for less connected Voronoi (+ ridge area) preprocessed graphs and achieved the results displayed in Table 3. Although NGNs showed the best performance on the specific dataset they have been (hyperparameter) optimized on, they can not maintain their advantage consistently on all other datasets without re-optimizing. Unfortunately, NGNs require the construction of line graphs and tend to have significantly more trainable parameters. Consequently, training is more than three times as computationally expensive as for non-nested GNs.

Finally, we were able to demonstrate the effectiveness of densely connected crystal graphs with the coGN model, by surpassing current state-of-the-art models on most of the MatBench datasets as shown in Table 3. The results also support that hyperparameters, which originate from the optimization on the log_gvrh dataset, generalize to other datasets and tasks. Further results and statistics can be found in the Appendix § 8.4.

Asymmetric Unit Graphs

In Section 2, we discussed different graph representations for crystals and found that asymmetric unit graphs are smaller than unit cell graphs and yet lead to identical predictions of GNNs with E(3)-invariant layers. The exact reduction factor for asymmetric unit graphs depends on the symmetries each specific crystal exhibits. Since the maximal space group order for crystals is 48, we can specify theoretical lower and upper bounds for the number of nodes nasun_{\mathrm{asu}} in asymmetric unit graphs in relation to unit cell graphs:

nunit≥nasu≥148⋅nunitn_{\mathrm{unit}} \ge n_{\mathrm{asu}} \ge\frac{1}{48} \cdot n_{\mathrm{unit}}

We found that for the MatBench datasets, the empirical average reduction factor for the number of nodes (nn), edges (m=nL(G)m = n^{L(G)}) as well as line graph edges (mL(G)m^{L(G)}) is approximately 2.15:

nunitnasu≈munitmasu≈munitL(G)masuL(G)≈2.1\frac{n_{\mathrm{unit}}}{n_{\mathrm{asu}}} \approx\frac{m_{\mathrm{unit}}}{m_{\mathrm{asu}}} \approx\frac{m_{\mathrm{unit}}^{L(G)}}{m_{\mathrm{asu}}^{L(G)}} \approx2.1

Approximately the same factor can be observed for the GPU memory footprint during training. Due to parallelization and batching the acceleration of the training runtimes is somewhat smaller and depends on the selected batch size. For batch sizes of 32, 64, and 128, for example, we get a speedup of 1.2, 1.3, and 1.8, respectively.

Conclusion

This paper discusses fundamental aspects of crystal property prediction with GNNs: the incorporation of symmetries in GNN models, the interdependence of input graph generation from crystal structures (i.e. preprocessing) with the systematic exploration of a suitable GNN architecture, as well as the generalization of model architectures as nested graph networks. We conclude that these aspects cannot be considered separately.

Our contribution to the first aspect includes the proposal of the asymmetric unit graph representation, which decreases training time and memory consumption without effects on predictive performance for E(3)E(3)-invariant GNNs. To use asymmetric unit graphs with equivariant GNNs some adaptations to message passing and readout operations are required, which will be addressed in future work. We furthermore compared different edge selection methods and discovered that graphs with higher connectivity (compared to previous works) can yield better performance. From this insight, we construct the coGN, which achieves state-of-the-art results on the MatBench benchmark.

To explore the space of GNN architectures, we introduced the nested graph network (NGN) framework, which subsumes state-of-the-art GNN models which incorporate angle information in line graph form. We observed that a variation in the line graph definition for angles is likely to improve predictions by a small amount. Although we could not find an architecture for nested graph networks that substantially outperforms graph networks without nesting, we still encourage further research in this direction, specifically into the scaling laws of accuracy as a function of data set size and complexity as well as the depth of line-graph nesting. Future work might overcome the mentioned trade-off between high connectivity and nesting and potentially explore specific nested graph network architectures, which we did not consider due to hyperparameter space restrictions.

Acknowledgement

The authors acknowledge support by the state of Baden-Württemberg through bwHPC.

Appendix

Crystal Preprocessing: Edge Selection

Figure 10. Distributions of edge-to-node ratios per graph (average degree) in the mp_e_form dataset for different preprocessing methods. Due to the symmetrical and regular structure of crystals, it often happens that there are many neighbors with equal distances. Therefore our kNN-based edge selection implementation, also allows to toggle whether edges with a ϵ\epsilon-distance difference as the kk-th nearest neighbor are also added as connections. We included an ϵ=10−9\epsilon= 10^{-9} Åthreshold to make up for numerical inaccuracies.

Figure 11. Distribution of edge distances in the mp_e_form dataset for different edge selection methods.

Crystal Preprocessing: Symmetries

Figure 12. Average number of nodes in each dataset for unit cell graphs (unit) and asymmetric unit graphs (asu).

Figure 13. Histogram of number of nodes in mp_e_form dataset for unit cell graphs (unit) and asymmetric unit graphs (asu). The distribution for asymmetric unit graphs is skewed towards lower number of nodes. Clearly pronounced periodic peaks at even numbers for the unit cell graph distribution can be explained with symmetric atom pairs.

Nested Graph Network Algorithm

Function NGN-Block(X_E, X_V, x_G, G; θ):
    // X_E = {x_e₁, …, x_eₘ}
    // X_V = {x_v₁, …, x_vₙ}
    for eᵢⱼ ∈ E do
        x_eᵢⱼ ← ϕ̂_E(x_eᵢⱼ, x_vᵢ, x_vⱼ, x_G)                         // Edge update
    end
    X_∡ ← get_line_graph_edge_features(X_E, X_V, x_G, G)              // i.e. angles between edges
    for t ← 1 to T^{L(G)} do
        _, X′_E, _, _ ← GN-Block(X_∡, X′_E, x_G, L(G); θ^t)           // Treat X′_E as node features of L(G)
    end
    for v_k ∈ V do
        x̂_vₖ ← ρ_{E→V}({x′_eᵢⱼ | eᵢⱼ ∈ E ∧ j = k})                 // Local edge aggregation
        x′_vₖ ← ϕ_V(x_vₖ, x̂_vₖ, x_G)                                 // Node update
    end
    x̂_G ← ρ_{V→G}({x′_v | v ∈ V})                                    // Node aggregation
    x̃_G ← ρ_{E→G}({x′_e | e ∈ E})                                    // Global edge aggregation
    x′_G ← ϕ_G(x_G, x̂_G, x̃_G)                                       // Global update
    // X′_E = {x′_e₁, …, x′_eₘ}
    // X′_V = {x′_v₁, …, x′_vₙ}
    return X′_E, X′_V, x′_G, G

    // Iterate over sequentially composed NGN blocks
    for t ← 1 to T do
        X_E, X_V, x_G, G ← NGN-Block(X_E, X_V, x_G, G; θ_t)
    end

Algorithm 1: Graph network algorithm adapted from Battaglia et al., [33] with additional blue parts that indicate the modifications for the nested graph networks. GN blocks are parameterized by parameters θ\theta and receive edge features XEX_E, node features XVX_V, graph level features xGx_G as well as the graph topology GG as input.

Open Catalyst Project: OC22

Table 4: Test error for the initial structure to relaxed energies (IS2RES) task of the OC22 challenge Tran et al., [7] (status 2023-08-08). For comparison, this table includes solely direct OC22-only predictions. Models trained additionally on OC20 Chanussot et al., [6] and other data sources or indirect predictions using relaxations yield lower errors and better performance. Baseline models are SchNet Schütt et al., [18], DimeNet++ Klicpera et al., [19], PaiNN Schütt et al., [54] and GemNet Klicpera et al., [20].

ModelMAE (ID)MAE (OOD)Average
coGN (Direct OC22-only, r=5.0)1.61832.80582.2121
coGN (Direct OC22-only, k=32)1.62782.97062.2992
GemNet-dT (Direct OC22-only)1.67713.08372.3804
PaiNN (Direct OC22-only)1.7163.68352.6997
DimeNet++ (Direct OC22-only)1.963.51862.7393
SchNet (Direct OC22-only)2.00124.84683.424

Table 4. Test error for the initial structure to relaxed energies (IS2RES) task of the OC22 challenge Tran et al., [7] (status 2023-08-08). For comparison, this table includes solely direct OC22-only predictions. Models trained additionally on OC20 Chanussot et al., [6] and other data sources or indirect predictions using relaxations yield lower errors and better performance. Baseline models are SchNet Schütt et al., [18], DimeNet++ Klicpera et al., [19], PaiNN Schütt et al., [54] and GemNet Klicpera et al., [20].

Table 5: Test error for multiple tasks of the JARVIS benchmark [8]. Values are copied from the JARVIS leaderboard (status 2023-08-08). Models for comparison are DimeNet++ (DN++) [19], ALIGNN [22], CGCNN [25], MatMiner (MM) Li et al., Ward et al. [2018] and CFID [57].

TaskcoGNcoNGNDN++ALIGNNCGCNNMMCFID
dft_3d_mepsz24.108122.84230.364423.731336.053824.665129.3445
dft_3d_exfoliation_energy47.697946.27246.151752.703352.703340.88762.1169
dft_3d_shear_modulus_gv8.66128.488126.08179.47616.045910.541511.9164
dft_3d_spillage0.36090.34630.41370.3510.39650.35920.3867
dft_3d_optb88vdw_total_energy0.02620.02730.0510.03670.08150.09360.2436
dft_3d_mepsx24.228923.380131.956824.045833.659725.293230.261
dft_3d_epsz19.619217.810433.837919.567833.659725.293224.781
dft_3d_dfpt_piezo_max_dij15.223513.886813.988920.570516.013521.5729-
dft_3d_mepsy24.189123.329931.021523.648232.457725.070630.0578
qe_tb_indir_gap0.0474--0.1167-0.0351-
dft_3d_kpoint_length_unit9.57229.345911.88759.514613.21459.0479.7085
dft_3d_n_powerfact452.235456.6118568.8357442.2993-469.6279-
dft_3d_ph_heat_capacity6.11257.812723.36189.6064-5.2757-
dft_3d_formation_energy_peratom0.02710.02910.05280.03310.06250.07340.1419
dft_3d_epsx20.000418.573827.251120.394231.474421.259724.8408
dft_3d_optb88vdw_bandgap0.12190.12670.22470.14230.19080.18730.299
qe_tb_energy_per_atom0.0636-0.7515--1.5049-
dft_3d_max_efg20.441719.549526.955219.121124.669519.4382-
dft_3d_epsy24.189123.329931.021523.648232.457725.070630.0578
dft_3d_encut133.8915129.8266164.315133.7962190.3857138.2769139.4357
dft_3d_n_Seebeck39.269240.097754.275940.921449.317244.2229-
dft_3d_ehull0.04660.04850.36850.07630.1730.0601-
dft_3d_bulk_modulus_kv8.9928.702213.374310.398819.302812.741114.1999
dft_3d_avg_hole_mass0.13720.12850.17090.1239-0.1529-
dft_3d_avg_elec_mass0.09170.08760.1120.0853-0.107-
dft_3d_mbj_bandgap0.2640.27190.47640.31040.40670.33920.5313
dft_3d_dfpt_piezo_max_dielectric30.292325.555330.335828.151432.558936.6913-
dft_3d_slme4.45074.44285.64034.52075.66034.92556.2607
qe_tb_f_enp0.0956--0.1016-0.3219-
dft_3d_magmom_oszicar0.25020.24370.39950.25740.35430.36450.4748
qe_tb_final_energy1.3185----1.4714-

Table 5.

References

  1. [1]David K Duvenaud, Dougal Maclaurin, Jorge Iparraguirre, Rafael Bombarell, Timothy Hirzel, Alán Aspuru-Guzik, and Ryan P Adams. Convolutional networks on graphs for learning molecular fingerprints. Advances in neural information processing systems, 28, 2015.
  2. [2]Mohammadreza Karamad, Rishikesh Magar, Yuting Shi, Samira Siahrostami, Ian D. Gates, and Amir Barati Farimani. Orbital graph convolutional neural network for material property prediction. Phys. Rev. Mater., 4:093801, Sep 2020. doi:10.1103/PhysRevMaterials.4.093801. URL https://link.aps.org/doi/10.1103/PhysRevMaterials.4.093801.DOI
  3. [3]Jonathan Schmidt, Love Pettersson, Claudio Verdozzi, Silvana Botti, and Miguel A. L. Marques. Crystal graph attention networks for the prediction of stable materials. Science Advances, 7(49):eabi7948, 2021. doi:10.1126/sciadv.abi7948. URL https://www.science.org/doi/abs/10.1126/sciadv.abi7948.DOI
  4. [4]Jörg Behler. Atom-centered symmetry functions for constructing high-dimensional neural network potentials. The Journal of Chemical Physics, 134(7):074106, 2011a. doi:10.1063/1.3553717. URL https://doi.org/10.1063/1.3553717.
  5. [5]Scott Kirklin, James E Saal, Bryce Meredig, Alex Thompson, Jeff W Doak, Muratahan Aykol, Stephan Rühl, and Chris Wolverton. The Open Quantum Materials Database (OQMD): assessing the accuracy of DFT formation energies. npj Computational Materials, 1(1):15010, December 2015. ISSN 2057-3960. doi:10.1038/npjcompumats.2015.10. URL https://doi.org/10.1038/npjcompumats.2015.10.
  6. [6]Lowik Chanussot, Abhishek Das, Siddharth Goyal, Thibaut Lavril, Muhammed Shuaibi, Morgane Riviere, Kevin Tran, Javier Heras-Domingo, Caleb Ho, Weihua Hu, Aini Palizhati, Anuroop Sriram, Brandon Wood, Junwoong Yoon, Devi Parikh, C. Lawrence Zitnick, and Zachary Ulissi. Open catalyst 2020 (oc20) dataset and community challenges. ACS Catalysis, 2021. doi:10.1021/acscatal.0c04525.DOI
  7. [7]Richard Tran, Janice Lan, Muhammed Shuaibi, Brandon Wood, Siddharth Goyal, Abhishek Das, Javier Heras-Domingo, Adeesh Kolluru, Ammar Rizvi, Nima Shoghi, Anuroop Sriram, Zachary Ulissi, and C. Lawrence Zitnick. The open catalyst 2022 (oc22) dataset and challenges for oxide electrocatalysis. arXiv preprint arXiv:2206.08917, 2022.DOI
  8. [8]Kamal Choudhary, Kevin F. Garrity, Andrew C. E. Reid, Brian DeCost, Adam J. Biacchi, Angela R. Hight Walker, Zachary Trautt, Jason Hattrick-Simpers, A. Gilad Kusne, Andrea Centrone, Albert Davydov, Jie Jiang, Ruth Pachter, Gowoon Cheon, Evan Reed, Ankit Agrawal, Xiaofeng Qian, Vinit Sharma, Houlong Zhuang, Sergei V. Kalinin, Bobby G. Sumpter, Ghanshyam Pilania, Pinar Acar, Subhasish Mandal, Kristjan Haule, David Vanderbilt, Karin Rabe, and Francesca Tavazza. The joint automated repository for various integrated simulations (JARVIS) for data-driven materials design. 6(1):173, 2020. ISSN 2057-3960. doi:10.1038/s41524-020-00440-1. URL https://doi.org/10.1038/s41524-020-00440-1.
  9. [9]Dolf Gielen, Francisco Boshell, and Deger Saygin. Climate and energy challenges for materials science. 15(2):117–120. ISSN 1476-4660. doi:10.1038/nmat4545. URL https://doi.org/10.1038/nmat4545.
  10. [11]Kamal Choudhary, Marnik Bercx, Jie Jiang, Ruth Pachter, Dirk Lamoen, and Francesca Tavazza. Accelerated discovery of efficient solar cell materials using quantum and machine-learning methods. Chemistry of Materials, 31(15):5900–5908, 2019. doi:10.1021/acs.chemmater.9b02166. URL https://doi.org/10.1021/acs.chemmater.9b02166.
  11. [12]Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017. Conference Track Proceedings. OpenReview.net, 2017. URL https://openreview.net/forum?id=SJU4ayYgl.
  12. [13]Michael Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. In Aldo Gangemi, Roberto Navigli, Maria-Esther Vidal, Pascal Hitzler, Raphaël Troncy, Laura Hollink, Anna Tordai, and Mehwish Alam, editors, The Semantic Web, pages 593–607, Cham, 2018. Springer International Publishing. ISBN 978-3-319-93417-4.
  13. [14]Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International conference on machine learning, pages 1263–1272. PMLR, 2017.arxiv.org/abs/1704.01212
  14. [15]Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph Attention Networks. International Conference on Learning Representations, 2018. URL https://openreview.net/forum?id=rJXMpikCZ.arxiv.org/abs/1710.10903
  15. [16]Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https://proceedings.neurips.cc/paper/2017/file/5dd9db5e033da9c6fb5ba83c7a7ebea9-Paper.pdf.
  16. [17]Jörg Behler. Neural network potential-energy surfaces in chemistry: a tool for large-scale simulations. Phys. Chem. Chem. Phys., 13:17930–17955, 2011b. doi:10.1039/C1CP21668F. URL http://dx.doi.org/10.1039/C1CP21668F.
  17. [18]Kristof Schütt, Pieter-Jan Kindermans, Huziel Enoc Sauceda Felix, Stefan Chmiela, Alexandre Tkatchenko, and Klaus-Robert Müller. Schnet: A continuous-filter convolutional neural network for modeling quantum interactions. Advances in neural information processing systems, 30, 2017.
  18. [19]Johannes Klicpera, Janek Groß, and Stephan Güne­mann. Directional message passing for molecular graphs. CoRR, abs/2003.03123, 2020a. URL https://arxiv.org/abs/2003.03123.
  19. [20]Johannes Klicpera, Florian Becker, and Stephan Güne­mann. Gemnet: Universal directional graph neural networks for molecules. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021. URL https://openreview.net/forum?id=HS_sOaxS9K-.
  20. [21]Zhengdao Chen, Lisha Li, and Joan Bruna. Supervised community detection with line graph neural networks. In International Conference on Learning Representations, 2019a. URL https://openreview.net/forum?id=H1gOZ3A9Fm.
  21. [22]Kamal Choudhary and Brian DeCost. Atomistic line graph neural network for improved materials property predictions. npj Computational Materials, 7(1):1–8, 2021.
  22. [23]Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation- and translation-equivariant neural networks for 3d point clouds, 2018. URL https://arxiv.org/abs/1802.08219.
  23. [24]Simon Batzner, Albert Musaelian, Lixin Sun, Mario Geiger, Jonathan P Mailoa, Mordechai Kornbluth, Nicola Molinari, Tess E Smidt, and Boris Kozinsky. E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nature communications, 13(1):1–11, 2022.
  24. [25]Tian Xie and Jeffrey C Grossman. Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties. Physical review letters, 120(14):145301, 2018.
  25. [26]Cheol Woo Park and Chris Wolverton. Developing an improved crystal graph convolutional neural network framework for accelerated materials discovery. Physical Review Materials, 4(6):063801, 2020.
  26. [27]Chi Chen, Weike Ye, Yunxing Zuo, Chen Zheng, and Shyue Ping Ong. Graph networks as a universal machine learning framework for molecules and crystals. Chemistry of Materials, 31(9):3564–3572, 2019b.DOI
  27. [28]Jiucheng Cheng, Chunkai Zhang, and Lifeng Dong. A geometric information-enhanced crystal graph network for predicting properties of materials. Communications Materials, 2(1):92, September 2021. ISSN 2662-4443. doi:10.1038/s43246-021-00194-3. URL https://doi.org/10.1038/s43246-021-00194-3.
  28. [29]Hendrik J. Monkhorst and James D. Pack. Special points for brillouin-zone integrations. Phys. Rev. B, 13:5188–5192, Jun 1976. doi:10.1103/PhysRevB.13.5188. URL https://link.aps.org/doi/10.1103/PhysRevB.13.5188.DOI
  29. [30]Takenori Yamamoto. Crystal graph neural networks for data mining in materials science. Research Institute for Mathematical and Computational Sciences, LLC, 2019.
  30. [31]Chi Chen and Shyue Ping Ong. A universal graph deep learning interatomic potential for the periodic table. Nature Computational Science, 2(11):718–728, 2022. ISSN 2662-8457. doi:10.1038/s43588-022-00349-3. URL https://doi.org/10.1038/s43588-022-00349-3.
  31. [32]Jiaxuan You, Rex Ying, and Jure Leskovec. Design space for graph neural networks. In NeurIPS, 2020.arxiv.org/abs/2011.08843
  32. [33]Peter W Battaglia, Jessica B Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, et al. Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv:1806.01261, 2018.arxiv.org/abs/1806.01261
  33. [34]Alexander Dunn, Qi Wang, Alex Ganose, Daniel Dopp, and Anubhav Jain. Benchmarking materials property prediction methods: the matbench test set and automatminer reference algorithm. npj Computational Materials, 6(1):138, 2020. ISSN 2057-3960. doi:10.1038/s41524-020-00406-3. URL https://doi.org/10.1038/s41524-020-00406-3.
  34. [35]Olexandr Isayev, Denis Fourches, Eugene N Muratov, Corey Oses, Kevin Rasch, Alexander Tropsha, and Stefano Curtarolo. Materials cartography: representing and mining materials space using structural and electronic fingerprints. Chemistry of Materials, 27(3):735–743, 2015.DOI
  35. [36]Atsushi Togo and Isao Tanaka. Spglib: a software library for crystal symmetry search. arXiv preprint arXiv:1808.01590, 2018.arxiv.org/abs/1808.01590
  36. [38]Frank Harary and Robert Z Norman. Some properties of line digraphs. Rendiconti del circolo matematico di palermo, 9(2):161–168, 1960.DOI
  37. [39]Haggai Maron, Heli Ben-Hamu, Hadar Serviansky, and Yaron Lipman. Provably powerful graph networks. Advances in neural information processing systems, 32, 2019.arxiv.org/abs/1905.11136
  38. [40]Johannes Gasteiger, Florian Becker, and Stephan G"unnemann. Gemnet: Universal directional graph neural networks for molecules. Advances in Neural Information Processing Systems, 34:6790–6802, 2021.arxiv.org/abs/2106.08903
  39. [41]Michaël Defferrard, Martino Milani, Frédérick Gusset, and Nathanaël Perraudin. Deepsphere: a graph-based spherical cnn. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=B1e3OlStPB.arxiv.org/abs/2012.15000
  40. [42]Tim Hsu, Nathan Keilbart, Stephen Weitzner, James Chapman, Penghao Xiao, Tuan Anh Pham, S Roger Qiu, Xiao Chen, and Brandon C Wood. Efficient, interpretable atomistic graph neural network representation for angle-dependent properties and its application to optical spectroscopy prediction. arXiv preprint arXiv:2109.11576, 2021.arxiv.org/abs/2109.11576
  41. [43]Johannes Klicpera, Shankari Giri, Johannes T Margraf, and Stephan G"unnemann. Fast and uncertainty-aware directional message passing for non-equilibrium molecules. arXiv preprint arXiv:2011.14115, 2020c.
  42. [44]Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115(3):211–252, 2015. doi:10.1007/s11263-015-0816-y.DOI
  43. [45]Anubhav Jain, Shyue Ping Ong, Geoffroy Hautier, Wei Chen, William Davidson Richards, Stephen Dacek, Shreyas Cholia, Dan Gunter, David Skinner, Gerbrand Ceder, and Kristin A. Persson. Commentary: The materials project: A materials genome approach to accelerating materials innovation. APL Materials, 1(1):011002, 2013. doi:10.1063/1.4812323. URL https://doi.org/10.1063/1.4812323.
  44. [46]Maarten de Jong, Wei Chen, Thomas Angsten, Anubhav Jain, Randy Notestine, Anthony Gamst, Marcel Sluiter, Chaitanya Krishna Ande, Sybrand van der Zwaag, Jose J Plata, Cormac Toher, Stefano Curtarolo, Gerbrand Ceder, Kristin A. Persson, and Mark Asta. Charting the complete elastic properties of inorganic crystalline compounds. Scientific Data, 2(1):150009, 2015. ISSN 2052-4463. doi:10.1038/sdata.2015.9. URL https://doi.org/10.1038/sdata.2015.9.
  45. [47]Kamal Choudhary, Irina Kalish, Ryan Beams, and Francesca Tavazza. High-throughput identification and characterization of two-dimensional materials using density functional theory. Scientific Reports, 7(1):5179, 2017. ISSN 2045-2322. doi:10.1038/s41598-017-05402-0. URL https://doi.org/10.1038/s41598-017-05402-0.
  46. [48]Ivano E. Castelli, David D. Landis, Kristian S. Thygesen, Søren Dahl, Ib Chorkendorff, Thomas F. Jaramillo, and Karsten W. Jacobsen. New cubic perovskites for one- and two-photon water splitting using the computational materials repository. Energy Environ. Sci., 5:9034–9043, 2012. doi:10.1039/C2EE22341D. URL http://dx.doi.org/10.1039/C2EE22341D.
  47. [49]Patrick Reiser, André Eberhard, and Pascal Friederich. Graph neural networks in tensorflow-keras with raggedtensor representation (kgcnn). Software Impacts, 9:100095, 2021.DOI
  48. [50]Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2019.arxiv.org/abs/1907.10902
  49. [51]Tianle Cai, Shengjie Luo, Keyulu Xu, Di He, Tie-yan Liu, and Liwei Wang. Graphnorm: A principled approach to accelerating graph neural network training. In International Conference on Machine Learning, pages 1204–1215. PMLR, 2021.arxiv.org/abs/2009.03294
  50. [52]Pierre-Paul De Breuck, Geoffroy Hautier, and Gian-Marco Rignanese. Materials property prediction for limited datasets enabled by feature selection and joint learning with MODNet. npj Computational Materials, 7(1):83, 2021. ISSN 2057-3960. doi:10.1038/s41524-021-00552-2. URL https://doi.org/10.1038/s41524-021-00552-2.
  51. [53]Keqiang Yan, Yi Liu, Yuchao Lin, and Shuiwang Ji. Periodic graph transformers for crystal material property prediction. arXiv preprint arXiv:2209.11807, 2022.arxiv.org/abs/2209.11807
  52. [54]Kristof Schütt, Oliver Unke, and Michael Gastegger. Equivariant message passing for the prediction of tensorial properties and molecular spectra. In International Conference on Machine Learning, pages 9377–9388. PMLR, 2021.arxiv.org/abs/2102.03150
  53. [57]Kamal Choudhary, Brian DeCost, and Francesca Tavazza. Machine learning with force-field-inspired descriptors for materials: Fast screening and mapping energy landscape. Phys. Rev. Mater., 2:083801, Aug 2018. doi:10.1103/PhysRevMaterials.2.083801. URL https://link.aps.org/doi/10.1103/PhysRevMaterials.2.083801.DOI

Paper details

vlnc:W_z9u5k9k3
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 10

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents