betaFeedback

WorksPhysics and Astronomy

VASP Plugins: Linking the Vienna ab-initio Simulation Package with Python

Abstract

Implementing novel features and experimental algorithms into widely adopted density functional theory (DFT) codes is frequently hindered by complex legacy architectures and the use of compiled languages such as Fortran. These production codes, while optimised for high-performance computing clusters, present significant hurdles for software development and rapid prototyping, often requiring deep expertise in the code's internal structure to modify. To address this challenge, we present a Python plugin infrastructure for the Vienna ab-initio Simulation Package (VASP) that combines computational efficiency with the flexibility of high-level scripting. Our architecture uses a C++ intermediate layer and pybind11 to expose VASP data as NumPy arrays via shared memory buffers, ensuring high performance without data duplication. We implement two categories of plugins: those that modify quantities at the end of each converged self-consistent field (SCF) cycle, such as structure and force_and_stress, and those that operate during the SCF cycle, such as local_potential and occupancies. We demonstrate the utility of our implementation through three applications, structure relaxation using the scipy library, implementing an implicit solvent model, and adding the DFT-D4 dispersion corrections. This infrastructure effectively bridges the gap between high-performance electronic structure routines and the widespread scientific Python ecosystem.

Introduction

Materials discovery and property prediction rely on accurate and rapid first principles computational methods such as density functional theory (DFT).[1, 2] These calculations are often performed on high-performance computing clusters.[3, 4] Given this focus on performance, most production DFT codes are written in compiled languages such as Fortran[3, 5] and often focus on improving computational speed and numerical stability of their underlying codebase.

These large-scale Fortran projects present significant hurdles for modern software development and rapid prototyping. Programs such as the Vienna ab-initio Simulation Package (VASP),[3] a widely used DFT code,[6] are often characterized by complex memory management and architectures that make it difficult for researchers to implement specific features or experimental algorithms. Consequently, the barrier to entry for modifying the source code is high, frequently requiring deep expertise in the legacy structure of the program.

To circumvent the need for direct source code modifications, researchers frequently rely on Python-based frameworks such as the Atomic Simulation Environment (ASE),[7] Aiida,[8, 9] and Phonopy.[10] These packages operate as wrappers, automating complex computational workflows by generating input files, executing the compiled binaries, and parsing the resulting output files. While these tools are highly effective for high-throughput screening and post-processing, their reliance on file input/output (I/O) restricts them to an external, black-box operation of the DFT engine. Due to this black-box nature of their implementation, they cannot interact dynamically with the computations during runtime.

To implement an interface with more direct control, there is a growing trend toward hybrid software architectures that combine the computational efficiency of Fortran with the flexibility of high-level scripting languages.[11, 12, 13] Python has become the language of choice for the computational physics community, offering an extensive ecosystem of libraries for data analysis, plotting, numerical methods, and machine learning. By implementing an interface between the VASP and Python, it is possible to maintain the speed of the optimized electronic structure routines while providing ready customisation.

In this work, we present the Python plugin infrastructure for VASP. This implementation allows users to write Python functions to modify the behaviour of VASP at specific locations in the source code. We implement two sets of plugins. The first set modifies or replaces selected quantities at the end of each converged self-consistent field (SCF) cycle. For example, the structure plugin modifies the coordinates and lattice vectors and the force_and_stress plugin modifies or replaces the forces and the stress tensor. The second set of plugins modifies selected quantities during the SCF cycle. For example, the local_potential plugin modifies the total electrostatic potential and the occupancies plugin modifies the number of electrons, the Fermi energy, and smearing parameters. The manuscript is structured as follows: in Section II, we describe the VASP Python plugin infrastructure. In Section III we illustrate its utility with a few common use-cases in electronic structure calculations.

Architecture

Figure 1 shows a schematic of the architecture of VASP Python plugins. The top layer of this architecture consists of the VASP Fortran code. All integer, real, and complex arrays are created in this layer. This layer operates as a regular VASP calculation, with no change until it reaches critical “hooks” in the source code.

Image not extracted

Figure 1. Architecture of the Fortran-Python interface. Data is loaded into a buffer in the Fortran layer and exposed to Python via pybind11 as NumPy arrays. Python modifies the shared memory directly, ensuring high performance without data duplication.

A shared memory buffer is created once these hooks are reached. This shared memory buffer is composed entirely of data types compatible with C using the ISO_C_BIND module. These C-bind attributes prepare the data to be transferred from Fortran to Python through C++ and back. Pointers are stored instead of a copy of the data for large grid quantities (such as the charge density). This strategy prevents any memory overhead when transferring data between Fortran and Python.

We separate data into two classes, depending on their intended use. Data that should not be modified through the interface are stored in the constants data structure and data that is expected to be modified are stored in the additions data structure. For example, in the structure plugin, the constants data structure consists of unmodifiable input quantities such as number of ions, ion types and unmodifiable output quantities such as the total energies and forces. The additions data structure consists of modifiable quantities such as lattice vectors and the atom positions.

A C++ layer is used as an intermediate to transfer constants and additions data structures between Fortran and Python. Data is transferred between Fortran and C++ using a Fortran interface function. All interface functions take only constants and additions data structures as inputs.

The constants and additions data structures are converted to numpy arrays using the pybind11 package.[14] Three dimensional grids, such as the charge density, are reshaped to the appropriate dimensions and converted to numpy arrays. All numpy arrays are stored in Python dataclasses with the same arguments. These dataclasses hold all constants and additions data structures, analogous to their Fortran and C++ counterparts.

Listing 1. Constants for the structure plugin; the frozen keyword argument is added to prevent inadvertent modification of constants.
@dataclass(frozen=True)
class ConstantsStructure:
number_ions: int
number_ion_types: int
ion_types: IndexArray
atomic_numbers: IntArray
lattice_vectors: DoubleArray
positions: DoubleArray
POMASS: DoubleArray
total_energy: float
forces: DoubleArray
stress: DoubleArray
shape_grid: IntArray
charge_density: DoubleArray
neighbor_list: List[Neighbors]

Table 1. Constants for the structure plugin; the frozen keyword argument is added to prevent inadvertent modification of constants.

Analogous to the Fortran and C++ interface functions, the Python functions also take constants and additions data structures as input. For example, the constants for the structure plugin (see Listing 1) shows the type and data structures which are accessible to the structure Python function.

To use this plugin infrastructure, users simply write a Python function in a file called vasp_plugin.py with a predefined name that takes in constants and additions. For instance, to modify the structure, we need to specify a function: structure(constants, additions). Any modification to the additions dataclass must be done through the += or -= operators.

Note that the vasp_plugin.py file is treated as a regular Python file to be processed by the Python interpreter. This file offers all the flexibility associated with programming in Python, with the only constraint being that the pre-determined functions take constants and additions as inputs.

Illustrations

In this section, we discuss three applications of our VASP Plugins implementation. First, we discuss the combination of the VASP SCF procedure with external structure relaxation tools using the structure plugin. As an illustration, we relax the atom-centred forces using implementations from the scipy Python package.[15, 16] Second, we implement an implicit solvent package entirely in Python and link it with the local_potential plugin interface. Finally, we discuss how the atom-centred forces can be modified using the force_and_stress plugin. We illustrate this modification by adding dispersion corrections through the DFT–D4[15, 16] package.

Relaxation using the scipy library and the structure plugin

In this illustration, we use two optimisers implemented within the scipy Python package to relax the forces on a structure. We implement a link between two optimisers implemented in the scipy.minimize function, conjugate gradient (CG) and the Broyden-Fletcher-Goldfarb-Shanno (BFGS).

Listing 2 shows the code to be added to the vasp_plugin.py file to link scipy as a back-end for structure relaxation. The history dictionary stores a tuple of the energies and its gradient. In the structure function, positions and lattice vectors at the end of each SCF cycle is accessed through the constants dataclass. Note that the positions are sent as fractional coordinates and would need to be converted to Cartesian coordinates to be commensurate with the forces during relaxation. The initial structure is stored in x_start, which will eventually be passed to the minimize function. Two functions, fun and jac return the total energy and its gradient respectively, if it is stored in history. The minimize function is called with x_init, fun, jac as arguments. Each call to structure restarts this minimize function until it finds a structure that does not exist in history. Once this point is reached, a StopIteration exception is launched from within fun. This exception passes along the next set of positions. Note that the positions are sent as additions to the positions as a shift. Since it is an addition to the existing object, it is required to set changes with either += or -= operators.

Listing 2. Illustration of linking scipy optimisers with VASP Python Plugins.

import numpy as np
from scipy.optimize import minimize

history = {}
x_start = None

def structure(constants, additions):
    global x_start
    x_curr = constants.positions @ constants.lattice_vectors
    history[tuple(x_curr)] = (constants.total_energy, -constants.forces)
    if x_start is None: x_start = x_curr

def fun(x):
    if tuple(x) in history: return history[tuple(x)][0]
    raise StopIteration(x)

def jac(x):
    return history[tuple(x)][1]

try:
    minimize(fun, x0=x_curr, jac=jac,
             method="CG")
except StopIteration as next_step:
    new_x = next_step.args[0]
    shift = (new_x - x_curr) @ np.linalg.inv(constants.lattice_vectors)
    additions.positions += shift

Figure 2 shows the performance of four optimisers, RMM-DIIS implemented in VASP (red), CG implemented in VASP (green), CG implemented in scipy (orange) and BFGS implemented in scipy (blue). For this test, we choose a structure from the Open Catalyst Dataset (index: random585454). We find that the manually optimised implementations within VASP converge in fewer iterations (less than 20) as compared with generic implementations such as scipy (37 or higher) for this particular example.

Image not extracted

Figure 2. Comparison of the norm of the forces between existing optimisers in VASP and those linked with scipy.

Implicit solvent model using the local potential plugin

In this illustration, we implement the self-consistent continuum solvent (SCCS) model[17] and interface it with VASP. Listing 3 shows a sketch of the code required to perform this link in practice. First, a cavity is created based on the charge density using the algorithm in Ref 17. This cavity and the charge density is shown in Figure 3 for an Al(111) face-centered cubic slab at the center of the cell. The dielectric (in orange) is close to the bulk dielectric in areas where the charge density (green) is approximately zero (Lz<2L_z < 2 Å and Lz>13L_z > 13 Å). Similarly, where the charge density is significant, the dielectric is close to 1 (indicating the absence of solvent).

Image not extracted

Figure 3. SCCS generated dielectric (orange), charge density (green) and computed vacuum (dashed blue) and solvent (solid blue) potential.

This constructed dielectric and the charge density, is passed along to a generalised Poisson solver. v_vac (dashed blue, Figure 3) stores the solution to the Poisson equation under vacuum conditions, i.e. the solvent dielectric is one. v_solv (solid blue, Figure 3) stores the solution to the Poisson equation under solvent boundary conditions. Note that the field (the gradient of the potential) flattens out for the solvent (Lz<2L_z < 2 Å and Lz>13L_z > 13 Å). This flattening is a consequence of the comparatively large bulk dielectric constant (78.6) used in this work to mimic water.

Once v_vac and v_solv are computed, their difference is sent back to VASP by adding it to the additions.total_potential object. Note that we subtract v_vac from v_solv as VASP already computes and stores v_vac. The subtraction ensures that v_vac is removed and v_solv takes its place. The resulting potential used by VASP will correspond to that generated in the presence of an implicit solvent.

def apply_solvation(constants, additions):
    lattice = constants.lattice_vectors
    rho = constants.charge_density
    sccs = SCCSDielectric(rho_min=1e-4,
                          rho_max=1e-2, eps_bulk=78.6)
    eps_grid = sccs.get_dielectric_grid(rho)

    # A poisson solver with different
    # boundary conditions
    solver = Poisson(lattice)
    v_solv = solver.solve(rho,
                          dielectric_grid=eps_grid, solver="cg"
                         )
    v_vac = solver.solve(rho,
                         boundary_condition="tin-foil-periodic"
                        )

    v_diff = (v_solv - v_vac)
    additions.total_potential += v_diff

Dispersion corrections with the force_and_stress plugin

Finally, we discuss how the force_and_stress plugin interfaces with external pair-wise force modifiers. For this illustration, we link the force_and_stress plugin with the DFT–D4[15, 16].

from vasp.force_field import AseForCeField

def force_and_stress(constants, additions):
    calculator = DFTD4(method="pbe")
    force_field = AseForCeField(calculator)
    force_field.force_and_stress(constants,
                                 additions)

Listing 4 shows the code required to perform a minimal link between DFT–D4 and VASP. First, we import an AseForCeField from the VASP Plugin package. This AseForCeField takes in any ASE calculator that can compute the forces and optionally, the stress tensor. Finally, the force field adds the forces and the stress tensor generated by the calculator to those computed by VASP. Figure 4 shows the change in the total energy as a function of the interlayer distance between graphite. This simple test is a commonly used benchmark to determine the accuracy of dispersion corrections such as DFT–D4. Using the PBE exchange correlation functional and DFT–D4 linked through the VASP Plugin infrastructure, we find that the interlayer distance is between 6.6–6.7 Å.

Image not extracted

Figure 4. Change in the total energy with graphite interlayer spacing generated using the force_and_stress plugin.

The diverse applications presented in this section demonstrate the extensibility of the VASP Plugin infrastructure across distinct stages the SCF and relaxation cycles. Using the structure, local_potential, and force_and_stress interfaces, we have shown how external Python libraries, ranging from generic implementation such as scipy to specialised physical models like DFT–D4 and custom implicit solvation implementations can be integrated into existing VASP workflows.

Conclusions

In this work, we develop a Python plugin infrastructure to link VASP, a DFT code written in Fortran, with Python functions. This plugins infrastructure uses a C++ interface, enabled by the pybind11 library, to convert data structures in Fortran to those in Python. Central to this implementation is the use of shared memory buffers and Python dataclasses, which provide a safe and efficient method for users to access unmodifiable constants and apply changes via additions. We describe the flexibility of this approach by applying it to three common illustrations in electronic structure calculations. First, we show that the structure plugin allows for the integration of external optimisation packages like scipy. Second, we illustrate that the local_potential plugin

enables the development of complex models, such as implicit solvation, entirely within the Python environment while remaining fully coupled to the VASP SCF cycle. Finally, we show that the force_and_stress plugin simplifies the inclusion of energy terms, such as DFT-D4 dispersion corrections.

By lowering the barrier to entry for source-code modification, this VASP Python plugin system allows researchers to implement custom algorithms without requiring deep expertise in legacy Fortran structure. Overall, we believe this framework provides a robust foundation for future developments in materials discovery, allowing the community to develop and expand the scientific Python ecosystem alongside established DFT routines.

Computational Methods

All density functional theory calculations were performed with the Vienna ab-initio Simulation Package (VASP).[3] The Python plugins feature described in this manuscript is implemented in VASP and is available as of version 6.5.0.

Relaxation calculations to illustrate the differences between ionic solvers were performed a plane-wave cutoff of 520 eV. For this illustration we use a cell with stoichiometry Zr24Sc8\mathrm{Zr}_{24}\mathrm{Sc}_{8} with an NHO molecule placed as an adsorbate on a top site. The energy convergence threshold for each SCF cycle was set to 10−610^{-6} eV. The Zr_sv, Sc_sv, H, N and O pseudopotentials were used for Zr, Sc, H, N and O respectively. An Al(111) slab was used to illustrate the SCCS solvent implementation. A plane-wave cutoff of 241 eV was used. The Al pseudopotential was used for this calculation. To compute the graphite interlayer distance, we use the C pseudopotential with its default cutoff of 400 eV and a 16×16×816 \times16 \times8 Γ\Gamma-centered kk-point mesh. We converge the energy to 10−610^{-6} eV with PREC = Accurate. The PBE functional was used for all calculations. Gaussian smearing was used with a broadening of 0.1 eV for all calculations except for the graphite interlayer distance where the width was reduced to 0.01 eV.

Acknowledgments

We acknowledge support form EuroHPC Joint Undertaking for awarding us access to Leonardo at CINECA, Italy. S.V. acknowledges funding from ANRF MATRICS (ANRF/ARGM/2025/000139/TS) and seed grant (RD/0524-IRCCSH0-023) at IIT Bombay.

Author Contribution

Sudarshan Vijay: Conceptualization, Data curation, Formal analysis, Investigation, Software, Methodology, Validation, Visualization, Writing – original draft, Writing – review & editing. Martijn Marsman: Formal analysis, Software, Writing – review & editing. Georg Kresse: Conceptualization, Funding acquisition, Methodology, Resources Martin Schlipf: Conceptualization, Data curation, Formal analysis, Investigation, Software, Methodology, Validation, Visualization, Writing – original draft, Writing – review & editing.

Use of AI Tools

We used AI tools for grammar and language checking in the preparation of this manuscript. We have reviewed and edited the content as needed and take full responsibility for the content of this publication.

References

  1. [1]H. Eckert, S. Divilov, M. J. Mehl, D. Hicks, A. C. Zettel, M. Esters, X. Campilongo, and S. Curtarolo, “The AFLOW Library of Crystallographic Prototypes: Part 4,” (2024), arXiv:2401.06875 [cond-mat].DOI
  2. [2]A. Jain, G. Hautier, S. P. Ong, C. J. Moore, C. C. Fischer, K. A. Persson, and G. Ceder, Physical Review B 84, 045115 (2011).
  3. [3]G. Kresse and J. Furthmüller, Physical Review B - Condensed Matter and Materials Physics 54, 11169 (1996), arXiv: 10.1016/0927-0256(96)00008 8: ISBN: 1098-80121.
  4. [4]Z. Zhao, B. Austin, M. Marsman, and S. Maintz, .
  5. [5]P. Giannozzi, S. Baroni, N. Bonini, M. Calandra, R. Car, C. Cavazzoni, D. Ceresoli, G. L. Chiarotti, M. Cococcioni, I. Dabo, A. Dal Corso, S. De Gironcoli, S. Fabris, G. Fratesi, R. Gebauer, U. Gerstmann, C. Gougoussis, A. Kokalj, M. Lazzeri, L. Martin-Samos, N. Marzari, F. Mauri, R. Mazzarello, S. Paolini, A. Pasquarello, L. Paulatto, C. Sbraccia, S. Scandolo, G. Sclauzero, A. P. Seitsonen, A. Smogunov, P. Umari, and R. M. Wentzcovitch, Journal of Physics Condensed Matter 21, 395502 (2009), arXiv: 0906.2569.DOI
  6. [6]L. Talirz, L. M. Ghiringhelli, and B. Smit, Living Journal of Computational Molecular Science 3, 1483 (2021).
  7. [7]A. H. Larsen, J. Kleis, K. S. Thygesen, J. K. Nørskov, and K. W. Jacobsen, Physical Review B 84, 245429 (2011).
  8. [8]S. P. Huber, S. Zoupanos, M. Uhrin, L. Talirz, L. Kahle, R. Häuselmann, D. Gresch, T. Müller, A. V. Yakutovich, C. W. Andersen, F. F. Ramirez, C. S. Adorf, F. Gargiulo, S. Kumbhar, E. Passaro, C. Johnston, A. Merkys, A. Cepellotti, N. Mounet, N. Marzari, B. Kozinsky, and G. Pizzi, Scientific Data 7 (2020), 10.1038/s41597-020-00638-4, arXiv: 2003.12476.DOI
  9. [9]S. P. Huber, E. Bosoni, M. Bercx, J. Bröder, A. Degomme, V. Dikan, K. Eimre, E. Flage-Larsen, A. Garcia, L. Genovese, D. Gresch, C. Johnston, G. Petretto, S. Poncé, G. M. Rignanese, C. J. Sewell, B. Smit, V. Tsyplyaev, M. Uhrin, D. Wortmann, A. V. Yakutovich, A. Zadoks, P. Zarabadi-Poor, B. Zhu, N. Marzari, and G. Pizzi, Common workflows for computing material properties using different quantum engines, Tech. Rep. (2021) arXiv: 2105.05063 Publication Title: npj Computational Materials Volume: 7 Issue: 1.DOI
  10. [10]A. Togo and I. Tanaka, Scripta Materialia 108, 1 (2015).
  11. [11]S. Van Der Walt, S. C. Colbert, and G. Varoquaux, Computing in Science and Engineering 13, 22 (2011), arXiv: 1102.1523.
  12. [12]P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, S. J. van der Walt, M. Brett, J. Wilson, K. J. Millman, N. Mayorov, A. R. Nelson, E. Jones, R. Kern, E. Larson, C. J. Carey, I. Polat, Y. Feng, E. W. Moore, J. VanderPlas, D. Laxalde, J. Perktold, R. Cimmarm, I. Henriksen, E. A. Quintero, C. R. Harris, A. M. Archibald, A. H. Ribeiro, F. Pedregosa, P. van Mulbregt, A. Vijayskumar, A. P. Bardelli, A. Rothberg, A. Hilboll, A. Kloeckner, A. Scopatz, A. Lee, A. Rokem, C. N. Woods, C. Fulton, C. Masson, C. Häggström, C. Fitzgerald, D. A. Nicholson, D. R. Hagen, D. V. Pasechnik, E. Olivetti, E. Martin, E. Wieser, F. Silva, F. Lenders, F. Wilhelm, G. Young, G. A. Price, G. L. Ingold, G. E. Allen, G. R. Lee, H. Audren, I. Probst, J. P. Dietrich, J. Silterra, J. T. Webber, J. Slavič, J. Nothman, J. Buchner, J. Kulick, J. L. Schönberger, J. V. de Miranda Cardoso, J. Reimer, J. Harrington, J. L. C. Rodríguez, J. Nunez-Iglesias, J. Kuczynski, K. Tritz, M. Thoma, M. Newville, M. Kümmerer, M. Bolingbroke, M. Tartre, M. Pak, N. J. Smith, N. Nowaczyk, N. She- banov, Q. Pavlyk, P. A. Brodtkorb, P. Lee, R. T. McGib-bon, R. Feldbauer, S. Lewis, S. Tygier, S. Sievert, S. Vigna, S. Peterson, S. More, T. Pudlik, T. Oshima, T. J. Pingel, T. P. Robitaille, T. Spura, T. R. Jones, T. Cera, T. Leslie, T. Zito, T. Krauss, U. Upadhyay, Y. O. Halchenko, and Y. Vazquez-Baeza, Nature Methods 17, 261 (2020), arXiv: 1907.10121.
  13. [13]J. J. Mortensen, A. H. Larsen, M. Kuisma, A. V. Ivanov, A. Taghizadeh, A. Peterson, A. Haldar, A. O. Dohn, C. Schäfer, E. O. Jönsson, E. D. Hermes, F. A. Nilsson, G. Kastlunger, G. Levi, H. Jönsson, H. Häkkinen, J. Fojt, J. Kangsabanik, J. Sødequist, J. Lehtomäki, J. Heske, J. Enkovaara, K. T. Winther, M. Dulak, M. M. Melander, M. Ovesen, M. Louhivuori, M. Walter, M. Gjerding, O. Lopez-Acevedo, P. Erhart, R. Warmbier, R. Würdemann, S. Kaappa, S. Latini, T. M. Boland, T. Bligaard, T. Skovhus, T. Susi, T. Maxson, T. Rossi, X. Chen, Y. L. A. Schmerwitz, J. Schiøtz, T. Olsen, K. W. Jacobsen, and K. S. Thygesen, The Journal of Chemical Physics 160, 092503 (2024).
  14. [14]“pybind/pybind11,” (2026), original-date: 2015-07-05T19:46:48Z.
  15. [15]E. Caldeweyher, S. Ehlert, A. Hansen, H. Neugebauer, S. Spicher, C. Bannwarth, and S. Grimme, The Journal of Chemical Physics 150, 154122 (2019).
  16. [16]E. Caldeweyher, J.-M. Mewes, S. Ehlert, and S. Grimme, Physical Chemistry Chemical Physics 22, 8499 (2020).
  17. [17]O. Andreussi, I. Dabo, and N. Marzari, in Journal of Chemical Physics, Vol. 136 (2012) p. 64102.

Paper details

vlnc:W_428ctbym
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents