WorksPhysics and Astronomy
Breaking the Exascale Barrier for the Electronic Structure Problem in Ab-Initio Molecular Dynamics
Abstract
The non-orthogonal local submatrix method applied to electronic-structure based molecular dynamics simulations is shown to exceed 1.1 EFLOP/s in FP16/FP32 mixed floating-point arithmetic when using 4,400 NVIDIA A100 GPUs of the Perlmutter system. This is enabled by a modification of the original method that pushes the sustained fraction of the peak performance to about 80%. Example calculations are performed for SARS-CoV-2 spike proteins with up to 83 million atoms.
I Introduction
Electronic-structure based ab-initio molecular dynamics simulations (AIMD, [1, 2, 3]) are an important tool in solid-state physics, chemistry and material science. The explicit treatment of quantum-mechanical effects in the electronic structure is required in situations, where empirical model potentials used in classical molecular dynamics fail to describe the relevant physical or chemical phenomena.
To derive the forces acting on the atoms the electronic-structure problem has to be solved in every time step during the propagation of the atoms. To make this possible, linear-scaling methods have been developed, where the computational complexity scales only linearly with the number of atoms in the system [4, 5, 6, 7, 8]. We have proposed the non-orthogonal local submatrix method (NOLSM, [9]) as a massively parallel method to solve the electronic-structure problem via an approximate solution of the required matrix functions. The local nature of the method avoids inter-node communication in the solution phase and has been shown to scale extremely well to more than one thousand GPUs, while efficiently using the mixed-precision tensor cores for linear algebra operations.
This paper is building on the implementation described in [9], but since we are focusing on improvements to increase the sustained peak performance, aspects like the compensation of noise from numerical approximations with an appropriately modified Langevin-type equation to obtain accurate thermodynamical expectation values are not touched here, but have been discussed in previous work [7], [10]. Instead, section II summarizes the tackled problem, whereas Section III puts the achievement in relation to the performance of related large-scale electronic-structure-based structure relaxations or AIMD simulations. The innovations beyond those presented in [9] are described in Section IV. In Section V-B, we discuss our evaluation and define the performance measurements. Finally, Section VI discusses the achieved performance.
II Overview of the Problem
Molecular dynamics calculations simulate the movement of atoms in molecules, surfaces or solids by integrating Newtons equation of motion
where is the mass of atom , its position in space and the force acting on it. Thus, the evaluation of the forces acting on the atoms is required in every time step. In electronic-structure based molecular dynamics these forces are evaluated on-the-fly from the total energy of the system via
In turn, the total energy is not obtained from an empirical model as in classical molecular dynamics, but directly from the quantum-mechanical problem of electrons in the electrostatic field of the nuclei. The total energy of a system can be written as
where is the electronic energy, the double counting terms and the nuclear Coulomb repulsion energy of the atoms. The bulk of the computational effort is required to obtain the electronic energy. While this can be done efficiently for small and medium-sized systems by solving a high-dimensional eigenvalue problem [11, 12], very large systems require methods that scale at most linearly with the size of the system in terms of number of atoms. Such linear-scaling electronic-structure methods have been developed [4, 5, 6, 7] for example based on the one-particle reduced density matrix [13], which allows to obtain the electronic energy via
At zero electronic temperature, the density matrix can be written as a matrix function
where is the one-particle Hamiltonian matrix of the system and is the overlap matrix between the basis functions that are used to describe the wave functions of the electrons. Furthermore, denotes the chemical potential. The evaluation of the matrix-sign function in Eq. (5) can be performed iteratively for example with the Newton-Schulz iteration [14]
or other iteration schemes [15] that only require scalings, additions and multiplications of matrices.
The submatrix method [16, 17] instead views the density matrix as a matrix function to be evaluated. Therein, the evaluation of a matrix function is performed in three steps. The steps are performed for every column of the matrix independently and are schematically shown in Figure 1:
Figure 1. Schematic representation of the steps of the submatrix method for the approximate calculation of a matrix function of a large sparse matrix . The first step is the construction of a submatrix for every column of the matrix . Then the matrix function is applied to the dense submatrices, i.e., and finally the relevant result columns are inserted into the sparse result matrix. Figure from [9].
In the first step a submatrix is generated for column of the input matrix by removing all rows and corresponding columns for which the matrix has vanishing or negligible elements in column . The result is a smaller and much denser submatrix .
The matrix function is applied to the submatrix , i.e., is evaluated.
The matrix elements of that correspond to the column are written to the matrix in the sense that the submatrix construction of step 1) is applied in a reverse way.
The resulting matrix is an approximation of the matrix function and by construction has the same sparsity pattern as . The non-orthogonal submatrix method, where is a matrix function of two matrices, combines the sparsity patterns and before the submatrix construction. Thus, it builds two submatrices, and and in step 2) the matrix function is
Please note, that the efficiency and accuracy of the submatrix method can be improved by generating one submatrix for multiple columns instead of just a single column as described in detail in [9] together with the GPU implementation. Moreover, the use of the submatrix approximation and low-precision numerics can be compensated by an modified Langevin-type equation that replaces Newtons equation of motion so that exact ensemble-averaged expectation values can be obtained [7, 10].
Current State of the Art
Table I lists previous attempts to extend the boundaries of electronic-structure based structure relaxation or molecular dynamics simulations.
Innovations Realized
Summary of Contributions
This work uses the previously reported algorithmic innovations like the use of approximate computing techniques, the non-orthogonal local submatrix method and its realization with GPUs, while minimizing the communication, as well as the heuristic combination of columns in the submatrix creation described in [9]. A new development beyond the implementation innovations already shown in [9] like the efficient iterative evaluation of matrix functions for dense matrices on GPU tensor cores is introduced in section IV-B: The matrix-size dependency of the GPU-performance is now also considered for the combination of submatrices and yields an additional speedup.
Implementation Innovations
Submatrix Combination Heuristics
The combination of columns for the generation of submatrices introduced in [9] used a cubic metric, i.e., the combination of two columns yields an improvement if and only if
where is the dimension of the submatrix for column , for column and the number of overlapping columns. This cubic metric represents the number of floating-point operations during the evaluation of the matrix function for the submatrices, which is based on dense matrix multiplications. We propose to modify the cubic metric by including the performance characteristic of the used GPUs. For this purpose, the matrix multiplication performance of the GPUs is measured for different matrix sizes and interpolated. The combination criterion then compares the predicted runtime for the matrix functions of the submatrix instead of the number of floating-point operations, i.e.,
This criterion effectively increases the dimension of the submatrices and the achievable portion of the peak performance. Results are shown for the SARS-CoV-2 Spike protein in aqueous solution with approx. 1.7 mio. atoms in Table II and histograms for the submatrix dimensions in Figure 2. The criterion of Eq. (10) approximately doubles the average submatrix dimension and slightly increases the total number of floating point operations while drastically increasing the estimated floating-point throughput and leading to an overall estimated speedup compared to the previous criterion of Eq. (9).

Figure 2. Histogram for the submatrix sizes in the case of the SARS-CoV-2 Spike protein in aqueous solution with approx. 1.7 mio. atoms: without combining submatrices (blue), combination using criterion Eq. 9 (orange) and using criterion Eq. 10 (green). A discussion of the further structure can be found in [9] and also applies to the spike protein system.
Performance of previously conducted electronic structure-based structure relaxation or AIMD simulations. Therein, the employed electronic structure method is abbreviated by DFT, NSC-DFT, LS-DFT and SS-DFT, which stands for density functional theory and its non-self-consistent, linear-scaling and subsystem variants, respectively. The corresponding basis set to represent the single-particle orbitals are denoted by PW for conventional plane waves, RMG-PW for real-space multigrid plane waves, GPW for Gaussian and plane waves, GTO for Gaussian-type orbitals, FD for finite difference, RS-FD for real-space finite difference, FEM for finite element method, NGWF for non-orthogonal generalized Wannier functions and PAO for polarized atomic orbitals. If the calculation was conducted involving trivial K-point parallelism, the total number of atoms is given as the product of number of independent instances time the number of atoms in anyone of them. The sustained efficiency is either given with respect to the corresponding peak performance, or estimated in terms of parallel efficiency and identified by the “≈” sign. This table has been published previously in [9] and is included here with additional results for comparison.
| Code | Year | Method | Basis | System | # Atoms | # Cores | Machine | Peak Performance | Efficiency |
| CPMD [18] | 2005 | DFT | PW | bulk SiC | 1k | 1.2k CPU | IBM p690 | 1.087 TFLOP/s | ≈ 20% |
| Qbox [19] | 2006 | DFT | PW | bulk Mo | 8*1k | 128k CPU | IBM BlueGene/L | 207.3 TFLOP/s | 56.5% |
| LS3DF [20] | 2009 | DFT | PW | bulk ZnTeO | 36k | 147k CPU | Cray Jaguar | 442 TFLOP/s | ≈ 33% |
| CONQUEST [21] | 2010 | NSC-DFT | PAO | bulk Si | 2.1M | 4k CPU | Cray XT4 | ≈ 60% | |
| CP2K [22] | 2012 | LS-DFT | GPW | bulk H₂ | 1M | 47k CPU | Cray XT5 | ||
| ONETEP [23] | 2014 | LS-DFT | NGWF | amyloid fibril trimer | 42k | 115k CPU | IBM BlueGene/Q | ||
| CONQUEST [24] | 2014 | LS-DFT | PAO | bulk Si | 786k | 200k CPU | K-Computer | ||
| RSDFT [25] | 2014 | DFT | RS-FD | Si nanowire | 107k | 664k CPU | K-Computer | 5.48 PFLOP/s | 51.67% |
| CP2K [26] | 2016 | SS-DFT | GPW | satellite tobacco mosaic virus | 1M | 20k CPU | Cray XC30 | ||
| LDC-DFT [27] | 2014 | SS-DFT | RMG-PW | bulk SiC | 6.3M | 786k CPU | IBM BlueGene/Q | 5.08 PFLOP/s | 50.5% |
| OpenAtom [28] | 2016 | DFT | PW | periodic MOF | 32*424 | 262k CPU | IBM BlueGene/Q | ≈ 52% | |
| MGmol [29] | 2016 | LS-DFT | FD | bulk H₂O | 1.2M | 1.6m CPU | IBM BlueGene/Q | ≈ 39% | |
| DFT-FE [30] | 2019 | DFT | FEM | Mg cluster | 10.5k | 159k CPU +22.8k GPUs | IBM Summit | 46 PFLOP/s | 27.8% |
| CP2K [9] | 2021 | LS-DFT | GTO | bulk water | 102M | 18.4k CPU +1.5k GPUs | JUWELS Booster | 206 PFLOP/s | 43% |
| CP2K [9] | 2021 | LS-DFT | GTO | HIV-1 capsid in solution | 62.5M | 18.4k CPU +1.5k GPUs | JUWELS Booster | 324 PFLOP/s | 67.7% |
| CP2K, this work | 2022 | LS-DFT | GTO | spike proteins in solution | 82.9M | 70.4k CPU +4.4k GPUs | NERSC Perlmutter | 1127 PFLOP/s | 82.1% |
Table 1.
| Combination of submatrices | no | Eq. 9 | Eq. 10 |
| Number of submatrices | 1693134 | 89784 | 28577 |
| Smallest submatrix | 396 | 694 | 708 |
| Largest submatrix | 10627 | 10637 | 10737 |
| Average submatrix dimension | 715 | 1361 | 2282 |
| FLOP count in PFLOP for one mult. | 2.02 | 1.54 | 1.66 |
| Estimated performance in TFLOP/s | 103 | 224 | 270 |
| Estimated speedup | 2.9 | 3.2 |
Table 2. Influence of the two different submatrix combination criteria, Eq. 9 and Eq. 10, on the submatrix sizes, number of floating-point operations for one matrix multiplication of each submatrix, the estimated performance per NVIDIA A100 GPU and the estimated speedup considering the matrix-multiplication performance of an NVIDIA A100.
V How Performance Was Measured
V-A Computational Details
V-A1 SARS-CoV-2 Spike Protein in Aqueous Solution
As our benchmark system, we have used the full-length SARS-CoV-2 spike protein in the open state, anchored in a lipid bilayer (Reference PDB structure: 6VSB, and pre-equilibrated with all-atom MD using NAMD [31], [32]. The system was solvated in aqueous solution in a simulation cell with dimensions Å, and including 1693134 atoms. The single cell shown in Fig. 3 can be easily repeated in a two-dimensional grid of spike proteins as a scalable benchmark system.
V-A2 Simulation Details
The electronic structure is simulated with the GFN-xTB approach in conjunction with a London dispersion correction based on the rational Becke–Johnson damping function [33]. Further details can be found in [9].

Figure 3. SARS-CoV-2 spike protein in aqueous solution: full cell (left) and without hydrogen and oxygen atoms (right).
For the sake of benchmark resources, we have restricted each simulation run to one SCF iteration in the spirit of the second-generation Car–Parrinello AIMD method [11, 34], but included the iterations for finding an appropriate chemical potential that produces a charge-neutral system.
Measurements
The main measurements presented here are:
Wall clock time of the NOLSM method
The wall clock time of the NOLSM method on node is measured for each iteration of the chemical potential. Each iteration includes all transfers between host and GPU. The overall wall clock time wall clock time is defined as the maximum over all node wall-clock times.
FLOPs in the NOLSM method
The per-node floating-point operations in the FP16/FP32-mixed-precision matrix iterations in the NOLSM method are estimated as for a gemm-operation with for each iteration of the chemical potential. The construction of the matrix elements of the submatrices and other operations scaling like are neglected in the count.
Node-Performance of NOLSM method
The node performances of the NOLSM method are defined as for each node .
Performance of NOLSM method
The performance of the NOLSM method is defined as the sum of the node performances.
HPC System and Environment
The benchmark runs presented here have been performed on the Perlmutter system at the National Energy Research Scientific Computing Center (NERSC). The Perlmutter system consists of 1,536 GPU nodes with one AMD EPYC 7763 64-core CPU with 256 GB DDR4 memory and four NVIDIA A100 GPUs with 40 GB of HBM2 memory each. The peak performance of the tensor cores in one NVIDIA A100 GPU is 312 TFLOP/s in FP16 with FP32-based accumulate [35]. The system uses HPE Cray Slingshot as node interconnect.
The software environment used in this work consisted of GCC 11.2.0, Cray-MPICH 8.1.10, CUDA NVCC 11.5.119, and CUBLAS 11.5. One MPI-rank per node and 64 CPU-threads per rank were used as well as four CUDA streams per GPU. Each stream was controlled by a single CPU-thread.
Performance Measurements and Results
Performance of the NOLSM Method for the Spike Protein
We have performed calculations for three different grid sizes of spike proteins: (51 mio. atoms), (61 mio. atoms) and (83 mio. atoms). All three example calculations have been performed with 1,100 nodes of the Perlmutter system, i.e., 4,400 NVIDIA A100 GPUs.
The wall clock time of the NOLSM method is shown in Figure 4. The distribution of the performances of the individual node is shown in Figure 5 for 7 × 7 spike proteins (83 mio. atoms) in relation to the peak performance of the GPUs. The performances of the nodes with 4 NVIDIA A100 GPUs mainly fall in the range between 1 PFLOP/s and 1.07 PFLOP/s with an average of 1.03 PFLOP/s. This represents about 80% of the peak performance of 1.248 PFLOP/s = 4 · 0.312 PFLOP/s per node.
Figure 4. Wall time of the NOLSM method for a grid of SARS-CoV-2 spike proteins in aqueous solution on 1,100 nodes of the Perlmutter system.

Figure 5. Distribution of node performances for 83 mio. atoms (7×7 grid of SARS-CoV-2 spike proteins in aqueous solution) on 1,100 nodes of the Perlmutter system.
Finally, Figure 6 shows the floating-point performance in mixed FP16/FP32 of the NOLSM method. The floating-point throughput of 1.106 to 1.127 EFLOP/s with 4,400 NVIDIA A100 GPUs achieving about 80% of the theoretical peak performance of the tensor cores.
Figure 6. Distribution of node performances for 83 mio. atoms (7×7 grid of SARS-CoV-2 spike proteins in aqueous solution) on 1,100 nodes (4,400 NVIDIA A100 GPUs) of the Perlmutter system.
Conclusion
To the best of our knowledge, the achieved ∼1.1 EFLOP/s in FP16/FP32 floating-point arithmetic positions electronic-structure based molecular dynamics calculations with the non-orthogonal local submatrix method in CP2K [36] represents one of the first algorithms in computational natural science that has broken the exaflop barrier within a scientific application [37, 38, 39]. The massively parallel nature of the method allows for an efficient use of many thousand GPUs. The method can not only be applied to electronic-structure based molecular dynamics, but also in other situations where a matrix function needs to be evaluated for a large sparse matrix or problems that can be transformed to such an operation.
Acknowledgments
This research used resources of the National Energy Research Scientific Computing Center (NERSC), a U.S. Department of Energy Office of Science User Facility located at Lawrence Berkeley National Laboratory, operated under Contract No. DE-AC02-05CH11231 using NERSC award DDR-ERCAP0022240.
Additionally, we would like to thank for funding of this project by computing time provided by the Paderborn Center for Parallel Computing (PC2). This work is partially funded by Paderborn University’s research award for “GreenIT”, as well as the Federal Ministry of Education and Research (BMBF) and the state of North Rhine-Westphalia as part of the NHR Program. T.D.K. received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (Grant Agreement No. 716142).
References
- [1]R. Car and M. Parrinello, “Unified Approach for Molecular Dynamics and Density-Functional Theory”, Phys. Rev. Lett., vol. 55, no. 22, pp. 2471–2474, 1985.pubmed.ncbi.nlm.nih.gov/10032153
- [2]M. C. Payne, M. P. Teter, D. C. Allan, T. A. Arias, and J. D. Joannopoulos, “Iterative minimization techniques for ab initio total-energy calculations: molecular dynamics and conjugate gradients”, Rev. Mod. Phys., vol. 64, no. 4, pp. 1045–1097, 1992.DOI
- [3]T. D. Kühne, “Second generation Car–Parrinello molecular dynamics”, WIREs Comput. Mol. Sci., vol. 4, pp. 391–406, 2014.DOI
- [4]S. Goedecker, “Linear scaling electronic structure methods”, Rev. Mod. Phys., vol. 71, no. 4, pp. 1085–1123, 1999.DOI
- [5]W. Yang, “Direct calculation of electron density in density-functional theory”, Phys. Rev. Lett., vol. 66, no. 11, pp. 1438–1441, 1991.
- [6]G. Galli and M. Parrinello, “Large scale electronic structure calculations”, Phys. Rev. Lett., vol. 69, no. 24, pp. 3547–3550, 1992.pubmed.ncbi.nlm.nih.gov/10046849
- [7]D. Richters and T. D. Kühne, “Self-consistent field theory based molecular dynamics with linear system-sizing”, J. Chem. Phys., vol. 140, no. 13, p. 134 109, 2014.
- [8]A. M. N. Niklasson, S. M. Miszczewski, C. F. A. Negre, M. J. Cawkwell, P. J. Swart, J. Mohd-Yusof, T. C. Germann, M. E. Wall, N. Bock, E. H. Rubensson, and H. D. Djidjev, “Graph-based linear scaling electronic structure theory”, The Journal of Chemical Physics, vol. 144, no. 23, p. 234 101, 2016. DOI: 10.1063/1.4952650. eprint: https://doi.org/10.1063/1.4952650. [Online]. Available: https://doi.org/10.1063/1.4952650.
- [9]R. Schade, T. Kenter, H. Elgabarty, M. Lass, O. Schütt, A. Lazzaro, H. Pabst, S. Mohr, J. Hutter, T. D. Kühne, and C. Plessl, “Towards electronic structure-based ab-initio molecular dynamics simulations with hundreds of millions of atoms”, Parallel Computing, vol. 111, p. 102 920, 2022, ISSN: 0167-8191. DOI: https://doi.org/10.1016/j.parco.2022.102920. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0167819122000242.
- [10]V. Rengaraj, M. Lass, C. Plessl, and T. D. Kühne, “Accurate Sampling with Noisy Forces from Approximate Computing”, Computation, vol. 8, no. 39, pp. 1–11, 2020.DOI
- [11]T. D. Kühne, M. Krack, F. R. Mohamed, and M. Parrinello, “Efficient and Accurate Car-Parrinello-like Approach to Born-Oppenheimer Molecular Dynamics”, Phys. Rev. Lett., vol. 98, no. 6, p. 066 401, 2007.pubmed.ncbi.nlm.nih.gov/17358962
- [12]T. D. Kühne, J. Hesse, and E. Prodan, “Disordered crystals from first principles II: Transport coefficients”, Annals of Physics, vol. 421, p. 168 290, 2020.DOI
- [13]R. McWeeny, “Some recent advances in density matrix theory”, Reviews of Modern Physics, vol. 32, no. 2, p. 335, 1960.
- [14]G. Schulz, “Iterative Berechnung der reziproken Matrix”, ZAMM - Journal of Applied Mathematics and Mechanics / Zeitschrift für Angewandte Mathematik und Mechanik, vol. 13, no. 1, pp. 57–59, 1933.DOI
- [15]D. Richters, M. Lass, A. Walther, C. Plessl, and T. Kühne, “A General Algorithm to Calculate the Inverse Principal p-th Root of Symmetric Positive Definite Matrices”, Communications in Computational Physics, vol. 25, no. 2, pp. 564–585, 2019.DOI
- [16]M. Lass, S. Mohr, H. Wiebeler, T. Kühne, and C. Plessl, “A Massively Parallel Algorithm for the Approximate Calculation of Inverse p-th Roots of Large Sparse Matrices”, in Proc. Platform for Advanced Scientific Computing (PASC) Conference, ACM, 2018.
- [17]M. Lass, R. Schade, T. Kühne, and C. Plessl, “A Submatrix-Based Method for Approximate Matrix Function Evaluation in the Quantum Chemistry Code CP2K”, in Proc. International Conference for High Performance Computing, Networking, Storage and Analysis (SC), IEEE Computer Society, 2020, pp. 1127–1140.DOI
- [18]J. Hutter and A. Curioni, “Dual-level parallelism for ab initio molecular dynamics: Reaching teraflop performance with the CPMD code”, Parallel Computing, vol. 31, no. 1, pp. 1–17, 2005.DOI
- [19]F. Gygi et al., “Large-scale electronic structure calculations of high-Z metals on the BlueGene/L platform”, in Proceedings of the 2006 ACM/IEEE conference on Supercomputing, 2006, p. 45.DOI
- [20]Z. Zhao et al., “The linearly scaling 3D fragment method for large scale electronic structure calculations”, in Journal of Physics: Conference Series, IOP Publishing, vol. 180, 2009, p. 012079.
- [21]D. R. Bowler and T. Miyazaki, “Calculations for millions of atoms with density functional theory: linear scaling shows its potential”, Journal of Physics: Condensed Matter, vol. 22, no. 7, p. 074207, 2010.
- [22]J. VandeVondele, U. Bostrnik, and J. Hutter, “Linear scaling self-consistent field calculations with millions of atoms in the condensed phase”, Journal of chemical theory and computation, vol. 8, no. 10, pp. 3565–3573, 2012.DOI
- [23]K. A. Wilkinson, N. D. Hine, and C.-K. Skylaris, “Hybrid MPI-OpenMP parallelism in the ONETEP linear-scaling electronic structure code: Application to the delamination of cellulose nanofibrils”, Journal of chemical theory and computation, vol. 10, no. 11, pp. 4782–4794, 2014.DOI
- [24]M. Arita, S. Arapan, D. R. Bowler, and T. Miyazaki, “Large-scale DFT simulations with a linear-scaling DFT code CONQUEST on K-computer”, Journal of Advanced Simulation in Science and Engineering, vol. 1, no. 1, pp. 87–97, 2014.DOI
- [25]Y. Hasegawa et al., “Performance evaluation of ultra-large-scale first-principles electronic structure calculation code on the K computer”, The International journal of high performance computing applications, vol. 28, no. 3, pp. 335–355, 2014.DOI
- [26]S. A. Draeger, J. A. Schiffmann, and J. VandeVondele, “Combining linear-scaling DFT with subsystem DFT in Born–Oppenheimer and Ehrenfest molecular dynamics simulations: from molecules to a virus in solution”, Journal of chemical theory and computation, vol. 12, no. 7, pp. 3214–3227, 2016.
- [27]K.-i. Nomura et al., “Metascalable quantum molecular dynamics simulations of hydrogen-on-demand”, in SC’14: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, IEEE, 2014, pp. 661–673.DOI
- [28]N. Jain et al., “Openatom: Scalable ab-initio molecular dynamics with diverse capabilities”, in International Conference on High Performance Computing, Springer, 2016, pp. 139–158.DOI
- [29]J.-L. Lefebvre, D. Osei-Kuffuor, E. W. Draeger, T. Ogitsu, and W. D. Krauss, “Modeling dilute solutions using first-principles molecular dynamics: computing more than a million atoms with over a million cores”, in SC’16: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, IEEE, 2016, pp. 12–22.
- [30]S. Das, P. Motamarri, V. Gavini, B. Turcksin, Y. W. Li, and B. Leback, “Fast, scalable and accurate finite-element based ab initio calculations using mixed precision computing: 46 PFLOPS simulation of a metallic dislocation system”, in Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2019, pp. 1–11.DOI
- [31]D. Wrapp, N. Wang, K. S. Corbett, J. A. Goldsmith, C.-L. Hsieh, O. Abiona, B. S. Graham, and J. S. McLellan, “Cryo-EM structure of the 2019-nCoV spike in the prefusion conformation”, Science, vol. 367, no. 6483, pp. 1260–1263, 2020. DOI: 10.1126/science.abb2507. eprint: https://www.science.org/doi/pdf/10.1126/science.abb2507. [Online]. Available: https://www.science.org/doi/abs/10.1126/science.abb2507.DOI
- [32]L. Casalino, A. C. Dommer, Z. Gaieb, E. P. Barros, T. Sztain, S.-H. Ahn, A. Trifan, A. Brace, A. T. Bogetti, A. Clyde, H. Ma, H. Lee, M. Turilli, S. Khalid, L. T. Chong, C. Simmerling, D. J. Hardy, J. D. Maia, J. C. Phillips, T. Kurth, A. C. Stern, L. Huang, J. D. McCalpin, M. Tatinni, T. Gibbs, J. E. Stone, S. Jha, A. Ramanathan, and R. E. Amaro, “AI-driven multiscale simulations illuminate mechanisms of SARS-CoV-2 spike dynamics”, The International Journal of High Performance Computing Applications, vol. 35, no. 5, pp. 432–451, 2021. DOI: 10.1177/10943420211006452. eprint: https://doi.org/10.1177/10943420211006452. [Online]. Available: https://doi.org/10.1177/10943420211006452.
- [33]S. Grimme, S. Ehrlich, and L. Goerigk, “Effect of the damping function in dispersion corrected density functional theory”, Journal of computational chemistry, vol. 32, no. 7, pp. 1456–1465, 2011.DOI
- [34]T. D. Kühne and E. Prodan, “Disordered crystals from first principles I: Quantifying the configuration space”, Annals of Physics, vol. 391, pp. 120–149, 2018.DOI
- [35][Online]. Available: https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/nvidia-ampere-architecture-whitepaper.pdf.
- [36]T. Kühne et al., “CP2K: An electronic structure and molecular dynamics software package - Quickstep: Efficient and accurate electronic structure calculations”, J. Chem. Phys., vol. 152, no. 1, p. 194103, 2020.DOI
- [37]T. Kurth, S. Treichler, J. Romero, M. Mudigonda, N. Luehr, E. Phillips, A. Mahesh, M. Matheson, J. Deslippe, M. F. Patica, P. Prabhat, and M. Houston, “Exascale Deep Learning for Climate Analytics”, in SC18: International Conference for High Performance Computing, Networking, Storage and Analysis, 2018, pp. 649–660. DOI: 10.1109/SC.2018.00054.DOI
- [38]W. Joubert, D. Weighill, D. Kainer, S. Climer, A. Justice, K. Fagnan, and D. Jacobson, “Attacking the Opioid Epidemic: Determining the Epistatic and Pleiotropic Genetic Architectures for Chronic Pain and Opioid Addiction”, in Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis, ser. SC ’18, Dallas, Texas: IEEE Press, 2018. DOI: 10.1109/SC.2018.00060. [Online]. Available: https://doi.org/10.1109/SC.2018.00060.
- [39]Y. A. Liu, X. L. Liu, F. N. Li, H. Fu, Y. Yang, J. Song, P. Zhao, Z. Wang, D. Peng, H. Chen, C. Guo, H. Huang, W. Wu, and D. Chen, “Closing the “Quantum Supremacy” Gap: Achieving Real-Time Simulation of a Random Quantum Circuit Using a New Sunway Supercomputer”, in Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, ser. SC ’21, St. Louis, Missouri: Association for Computing Machinery, 2021, ISBN: 9781450384421. DOI: 10.1145/3458817.3487399. [Online]. Available: https://doi.org/10.1145/3458817.3487399.


