betaFeedback

WorksPhysics and Astronomy

H-NESSi: The Hierarchical Non-Equilibrium Systems Simulation package

Abstract

We present H-NESSi (The Hierarchical Non-Equilibrium Systems Simulation package), an open-source software package for solving the Kadanoff-Baym equations (KBE) of nonequilibrium Green's function (NEGF) theory using hierarchical low-rank compression techniques. The simulation of strongly correlated quantum systems out of equilibrium is severely limited by the cubic scaling in propagation time and quadratic memory growth associated with conventional two-time formulations. H-NESSi overcomes these limitations by combining high-order time-stepping schemes with hierarchical off-diagonal low-rank (HODLR) representations of the retarded and lesser Green's functions, enabling controllable accuracy at substantially reduced computational cost and memory usage. Imaginary time quantities are efficiently represented using the discrete Lehmann representation (DLR), allowing compact and accurate treatment of thermal initial states. The implementation supports multiorbital systems, adaptive singular value truncation, and both shared-memory (OpenMP) and distributed-memory (MPI) parallelization strategies suitable for large-scale lattice calculations. The workflow closely mirrors established NEGF frameworks while introducing compression transparently into the propagation procedure. Benchmark applications to driven superconductors within dynamical mean-field theory and to the two-dimensional Hubbard model demonstrate favorable scaling compared to conventional implementations, with asymptotic time complexity significantly below the cubic scaling of uncompressed approaches. H-NESSi thus enables long-time and large-system nonequilibrium simulations of correlated quantum materials which were previously computationally prohibitive.

Program Summary

Manuscript Title: H-NESSi: The Hierarchical Non-Equilibrium Systems Simulation package

Authors: Thomas Blommel, Jeremija Kovačević, Jason Kaye, Emanuel Gull, Jakša Vučicević, Denis Golež

Program Title: H-NESSi

Journal Reference:

Catalogue identifier:

Licensing provisions: MIT

Programming language: C++

Computer: Any architecture with suitable compilers including PCs and clusters.

Operating system: Unix, Linux, OSX

RAM: Highly problem dependent

Classification:

External routines/libraries: cmake, eigen3, hdf5, fftw, libddr, mpi, openmp

Nature of problem: Solves equations of motion of time-dependent Green’s functions on the Kadanoff-Baym contour within compressed representation

Solution method: High-order timestepping method for integrodifferential equations within the compressed representation on the Kadanoff-Baym contour.

Introduction

The theoretical description of nonequilibrium phenomena in quantum materials and simulators requires numerical methods capable of treating strong correlations, long-time dynamics, and ultrafast driving on an equal footing. Many of the most intriguing experimental observations—such as transient superconductivity [1, 2] or long-lived metastable phases [3, 4, 5, 6, 7, 8]—occur on picosecond to nanosecond timescales, motivating the development of propagation schemes that extend well beyond the femtosecond regime characteristic of intrinsic electronic processes. This large separation of timescales poses a significant computational challenge for any numerical ends_mid=1 cal method [9, 10, 11, 12, 13, 14]. In this work, we focus on advancing the capabilities of nonequilibrium Green’s function (NEGF) approaches, which have emerged as a promising tool to capture rapid electronic evolution and correlations over extended timescales [15, 16, 17, 18, 6, 19, 20]. NEGFs also present a natural way around the ill-defined analytic continuation of Matsubara-domain data [21, 22, 23, 24, 25, 26, 27, 28, 29], which has proven useful in computing dynamical response functions in equilibrium [30, 31, 32]. Another motivation comes from the need to study realistic multiorbital materials: despite their experimental relevance [6, 33, 34, 35, 36, 37], multiorbital NEGF simulations remain limited due to the computational and memory costs associated with solving the Kadanoff–Baym equations, with long-time simulations particularly demanding [17, 38, 39, 40].

A wide range of algorithmic strategies have been developed to address these challenges. The first important class consists of approximate formulations of the Kadanoff–Baym equations, such as the generalized Kadanoff–Baym ansatz (GKBA) [41, 42] and its G1–G2 variants [43, 44]. These approaches significantly reduce computational complexity and have enabled long-time simulations in both electron and electron–boson systems [45, 46, 47, 48], although their accuracy can deteriorate in strongly correlated regimes [18, 6]. Data-driven methods, including dynamic mode decomposition [49, 50, 51], and recurrent neural networks [52, 53], yield efficient extrapolation schemes, but systematic error control remains challenging for propagation times far beyond the training interval; however, recently developed exponential approximations [54] show promise.

The second class consists of numerically exact approaches, which exploit specific structures of the two-time Green’s functions to reduce computational cost. Memory-cutting techniques leverage the rapid decay of the history kernel [55, 56, 57, 58, 59, 60] and can achieve linear scaling when applicable, but they fail in regimes with long-range memory. The quantics tensor train (QTT) scheme [61, 62, 63] provides an alternative global-in-time perspective, though convergence issues at long times require patching [64] or predictor–corrector methods [65]. Hierarchical matrix strategies, including hierarchical off-diagonal low-rank (HODLR) compression [66, 67], offer another robust approach, and the possibility of combining compression with timestepping. These methods provide controllable accuracy and a systematic, model-agnostic route to reducing computational cost and memory usage by exploiting low-rank structures in two-time Green’s functions. There is also the possibility of combining them with global iterations [68, 69], enabling fast hierarchical matrix algebra [70].

In this paper, we present an open-source and high-order accurate implementation of the HODLR scheme presented in Ref. [66], generalized to multiorbital systems. It features adaptive rank selection and user-controlled accuracy without introducing additional approximations beyond the hierarchical compression itself. The library offers a flexible workflow that can be readily integrated with applications such as dynamical mean-field theory (DMFT) [71, 18, 6] and impurity solvers. The code was designed with a user experience similar to the NESSi package [17] while enabling multiorbital simulations at significantly longer propagation times. We illustrate the capabilities of the library with benchmark calculations on driven superconductors within DMFT following Ref. [72], as well as with the second Born approximation for the two-dimensional Hubbard model. The latter example follows Ref. [32] and combines the solution of the Kadanoff–Baym equations with efficient parallelization schemes, enabling simulations of large lattice systems that preserve translation invariance.

Overview of multiorbital HODLR timestepping

The non-equilibrium single-particle Green’s function is defined as

Gij(z,z′)=1iTr⁡[T{e−i∫γzdzˉH^(zˉ)d^i(z)d^j†(z′)}]Tr⁡[T{e−i∫γdzˉH^(zˉ)}].(1)G_{ij}(z,z') = \frac{1}{i}\frac{\operatorname{Tr}\left[\mathcal{T}\left\{e^{-i\int_{\gamma}^{z}d\bar{z}\hat{H}(\bar{z})}\hat{d}_{i}(z)\hat{d}_{j}^{\dagger}(z')\right\}\right]}{\operatorname{Tr}\left[\mathcal{T}\left\{e^{-i\int_{\gamma}d\bar{z}\hat{H}(\bar{z})}\right\}\right]}. \tag*{(1)}

where γ\gamma is the Konstantinov-Perel’ contour [73] shown in Figure 1, T\mathcal{T} is the contour ordering operator, and d^j†(z′)\hat{d}_{j}^{\dagger}(z') creates a particle in orbital jj at contour time z′z'. In what follows, we suppress the orbital indices; all propagators should be understood as matrices in orbital space. It is useful to define Keldysh components—functions of real-valued arguments in which the contour locations are made explicit—as shown in Table 1. It is also convenient to define the retarded and advanced components

Figure 1. The Konstantinov-Perel’ contour.

ComponentName
G<(t,t′)=G(t−,t+)G^{<}(t,t') = G(t_{-},t_{+})lesser
G>(t,t′)=G(t+,t−)G^{>}(t,t') = G(t_{+},t_{-})greater
G⌈(t,τ)=G(t±,t0−iτ)G^{\lceil}(t,\tau) = G(t_{\pm},t_{0}-i\tau)left-mixed
G⌉(τ,t)=G(t0−iτ,t±)G^{\rceil}(\tau,t) = G(t_{0}-i\tau,t_{\pm})right-mixed
GM(τ)=−iG(−iτ,t0)G^{M}(\tau) = -iG(-i\tau,t_{0})Matsubara

Table 1. Keldysh components of the Green’s function and their locations on the contour. t±t_{\pm} explicitly places the argument on the γ±\gamma_{\pm} horizontal leg of the contour.

GR(t,t′)=θ(t−t′)[G>(t,t′)−G<(t,t′)](2)G^{R}(t,t') = \theta(t-t')[G^{>}(t,t')-G^{<}(t,t')] \tag*{(2)}
GA(t,t′)=θ(t′−t)[G<(t,t′)−G>(t,t′)].(3)G^{A}(t,t') = \theta(t'-t)[G^{<}(t,t')-G^{>}(t,t')]. \tag*{(3)}

These Keldysh components are not all independent; a minimal set of four suffices to capture all information in the single-particle Green’s function. There are multiple possible choices; following Ref. [17], we choose the components {M,⌈,R,<}\{M,\lceil,R,<\} and refer to the left-mixed component simply as the mixed component. From this set, we can obtain the advanced and right-mixed components from

GA(t,t′)=[GR(t′,t)]†G^{A}(t,t') = [G^{R}(t',t)]^\dagger
G⌈(τ,t)=−ξ[G⌈(t,β−τ)]†G^{\lceil}(\tau,t) = -\xi[G^{\lceil}(t,\beta-\tau)]^\dagger

where ξ=±\xi= \pm for bosons (upper sign) and fermions (lower sign), and β\beta is the inverse temperature. We only need the lesser and greater components on one side of the (t,t′)(t,t') diagonal, due to their anti-Hermitian symmetry

G≷(t,t′)=−[G≷(t′,t)]†(4)G^{\gtrless}(t,t') = -[G^{\gtrless}(t',t)]^\dagger \tag*{(4)}

Finally, all imaginary time functions need only be known on τ∈[0,β)\tau\in[0,\beta), due to (anti-)periodicity

GM(τ)=ξGM(τ+β)G^{M}(\tau) = \xi G^{M}(\tau+\beta)
G⌈(t,τ)=ξG⌈(t,τ+β).G^{\lceil}(t,\tau) = \xi G^{\lceil}(t,\tau+\beta).

In this work, we discretize the time axis into equidistant points with spacing hh. The lesser and retarded components are functions of two times and can therefore be viewed as matrices with entries Gt1t2R=GR(t1h,t2h)G^{R}_{t_{1}t_{2}}=G^{R}(t_{1}h,t_{2}h) and Gt1t2<=G<(t2h,t1h)G^{<}_{t_{1}t_{2}}=G^{<}(t_{2}h,t_{1}h) (here suppressing the orbital indices). We often drop the factor of hh and write G(t1,t2)G(t_{1},t_{2}) for the corresponding matrix element, with integers t1,t2t_{1},t_{2}. The definition of the retarded component in Eq. (2) contains a Heaviside function that restricts this matrix to being lower triangular. The lesser component can similarly be stored as a lower triangular matrix, as the anti-Hermitian symmetry (4) allows us to recover information above the diagonal. For further details on the numerical discretization, anti-Hermitian symmetries, and related aspects, we refer the reader to Ref. [17]. These triangular representations apply equally to Green’s functions, self-energies, and hybridization functions in the context of DMFT.

Kadanoff-Baym equations

The KBE are a coupled set of integro-differential equations for the Keldysh components of the Green’s function, here written in matrix form:

−∂τGM(τ)=δ(τ)1+ϵ(0∣)⋅GM(τ)+∫0βdτˉ ΣM(τ−τˉ)⋅GM(τˉ)(5)-\partial_{\tau}G^{M}(\tau)=\delta(\tau)\mathbf{1}+\epsilon(0|)\cdot G^{M}(\tau) +\int_{0}^{\beta}\mathrm{d}\bar{\tau}\,\Sigma^{M}(\tau-\bar{\tau})\cdot G^{M}(\bar{\tau}) \tag*{(5)}
i∂tGR(t,t−t′)=GR(t,t−t′)⋅ϵ(t−t′)+∫0t′dtˉ GR(t,t−tˉ)⋅ΣR(t−tˉ,t−t′)(6)i\partial_{t}G^{R}(t,t-t')=G^{R}(t,t-t')\cdot\epsilon(t-t') +\int_{0}^{t'}\mathrm{d}\bar{t}\,G^{R}(t,t-\bar{t})\cdot\Sigma^{R}(t-\bar{t},t-t') \tag*{(6)}
i∂tG⌈(t,τ)=ϵ(t)⋅G⌈(t,τ)+∫0tdtˉ ΣR(t,tˉ)⋅G⌈(tˉ,τ)+∫0βdτˉ Σ⌈(t,τˉ)⋅GM(τˉ−τ)(7)i\partial_{t}G^{\lceil}(t,\tau)=\epsilon(t)\cdot G^{\lceil}(t,\tau) +\int_{0}^{t}\mathrm{d}\bar{t}\,\Sigma^{R}(t,\bar{t})\cdot G^{\lceil}(\bar{t},\tau) +\int_{0}^{\beta}\mathrm{d}\bar{\tau}\,\Sigma^{\lceil}(t,\bar{\tau})\cdot G^{M}(\bar{\tau}-\tau) \tag*{(7)}
i∂tG<(t,t′)=ϵ(t)⋅G<(t,t′)+∫0tdtˉ ΣR(t,tˉ)⋅G<(tˉ,t′)+∫0t′dtˉ Σ<(t,tˉ)⋅GA(tˉ,t′)−i∫0βdτˉ Σ⌈(t,τˉ)⋅G⌈(τˉ,t′).(8)i\partial_{t}G^{<}(t,t')=\epsilon(t)\cdot G^{<}(t,t') +\int_{0}^{t}\mathrm{d}\bar{t}\,\Sigma^{R}(t,\bar{t})\cdot G^{<}(\bar{t},t') +\int_{0}^{t'}\mathrm{d}\bar{t}\,\Sigma^{<}(t,\bar{t})\cdot G^{A}(\bar{t},t') -i\int_{0}^{\beta}\mathrm{d}\bar{\tau}\,\Sigma^{\lceil}(t,\bar{\tau})\cdot G^{\lceil}(\bar{\tau},t'). \tag*{(8)}

Here ⋅\cdot marks matrix multiplication in the orbital space and ϵ(0∣)\epsilon(0|) denotes the equilibrium value of ϵ(t)\epsilon(t) on the thermal branch. The function ϵ(t)=h0(t)+ΣMF(t)−μ\epsilon(t)=h_{0}(t)+\Sigma^{\mathrm{MF}}(t)-\mu is the quadratic mean-field Hamiltonian, which contains the quadratic terms of the Hamiltonian, the mean-field self-energy, and the chemical potential, respectively.

The Keldysh components of the correlated (beyond mean-field) self-energy Σ\Sigma obey the same symmetries as the Green’s functions. Since the self-energy is typically a nonlinear functional of the Green’s function, the KBE Eqs. (5)–(8) must be solved self-consistently. We use an iterative scheme in which, at iteration ii, the self-energy is evaluated from the current Green’s function, Σ(i)[G(i)]\Sigma^{(i)}[G^{(i)}], and the KBE are then solved to obtain the updated G(i+1)[Σ(i)]G^{(i+1)}[\Sigma^{(i)}]. Following Ref. [17], the nonlinear equation is converged self-consistently at each timestep before proceeding to the next, except during bootstrapping, where global iterations are used (see § 4.2 and § 5.3). In practice, we find this procedure to be very robust, in contrast to global iterative schemes which can require considerable effort to stabilize [74, 63, 64, 65, 69, 75].

Hierarchical decomposition

The favorable scaling of H-NEssI is a consequence of the use of compressed representations of the two-time Green’s functions and self-energies [66], achieved through a hierarchical partitioning of the lower-triangular retarded and lesser components, as shown in Figure 2. The user supplies the number of timesteps NtN_{t} and hierarchical levels NlN_{l}, which together determine the partitioning, and these parameters remain fixed throughout the calculation. In the example shown, Nl=4N_{l}=4, as evidenced by the four distinct block sizes. In general, a level ll consists of 2l−12^{l-1} blocks, each of approximate size Nt2l\frac{N_{t}}{2^{l}}. If Nt2l\frac{N_{t}}{2^{l}} is not an integer, some blocks will be rectangular rather than square; however, this does not affect the efficiency of the algorithm or the user interface.

Figure 2. Snapshot of HODLR representation of CR(t1,t2)C^{R}(t_1,t_2) and C<(t2,t1)C^{<}(t_2,t_1) for C∈{G,Σ}C \in\{G,\Sigma\} at timestep TT. The blue areas are stored in the truncated SVD representation; all other colored regions are stored directly. The horizontal red slice is the timestep currently being solved for. The green region consists of the previous kk timesteps, which are stored directly and will eventually be used to update the blue and yellow regions. White regions are yet to be solved for.

For efficient storage, each block BB is stored as a truncated SVD (TSVD) B≈UΣV†B\approx U\Sigma V^\dagger, where the diagonal matrix Σ\Sigma contains singular values and UU, VV are the corresponding singular vectors. Singular values below a user-provided threshold εSVD\varepsilon_{\mathrm{SVD}}, along with their associated vectors, are discarded, yielding accuracy ∣B−USV∣<εSVD\lvert B - USV \rvert< \varepsilon_{\mathrm{SVD}}. The number of retained singular values, NσεN^{\varepsilon}_{\sigma}, is the ε\varepsilon-rank of the block, which is typically much smaller than the block size. These ranks often grow sublinearly with block size, improving compression efficiency at long integration times.

Integrating the KBE amounts to incrementally filling in rows of the lower triangular matrices of G<G^{<} and GRG^{R}. This is shown in Figure 2, where the red row at timestep TT is the current row being solved for, and all the rows above have already been computed. In the illustrated example, two blue blocks are partially built, as only some of their rows have been calculated. As described in Ref. [66], these blocks are still stored as a truncated SVD. As new rows are obtained via integration of the KBE, the truncated SVD is incrementally updated using an efficient rank-1 update algorithm.

Multiorbital compression

The Green’s function and self-energy each carry two indices enumerating degrees of freedom such as spin, orbital, and lattice site (the latter is needed for inhomogeneous lattice models; otherwise a single momentum index suffices). We refer to these as orbital indices and denote their number by NoN_o. Multiorbital compression is accomplished by separately storing No2N_o^2 compressed representations of the two-time functions. The complexity of managing these separate compressed functions is hidden from the user, as can be seen in Table 2-Table 5, which show No×NoN_o \times N_o matrices passed between the user and the library. Offline tests on simple multiorbital systems indicate that more sophisticated compression techniques combining orbital and time indices do not yield significant efficiency improvements over the current approach and can reduce compression efficiency while increasing the complexity of evaluating history integrals.

get_les_currCopy No×NoN_o \times N_o lesser data Cij<(t,t′)C^<_{ij}(t,t') at given (t,t′)(t,t') to/from provided Eigen matrix.
set_les_currCopy No×NoN_o \times N_o lesser data Cij<(t,t′)C^<_{ij}(t,t') at given (t,t′)(t,t') to/from provided Eigen matrix.
map_les_currReturn No×NoN_o \times N_o Eigen matrix map of Cij<(t,t′)C^<_{ij}(t,t') at given (t,t′)(t,t')
get_ret_currCopy No×NoN_o \times N_o retarded data CijR(t,t′)C^R_{ij}(t,t') at given (t,t′)(t,t') to/from provided Eigen matrix.
set_ret_currCopy No×NoN_o \times N_o retarded data CijR(t,t′)C^R_{ij}(t,t') at given (t,t′)(t,t') to/from provided Eigen matrix.
map_ret_currReturn No×NoN_o \times N_o Eigen matrix map of CijR(t,t′)C^R_{ij}(t,t') at given (t,t′)(t,t')

Table 2. get, set, and map functions for the retarded and lesser Keldysh components in the red and green regions of Figure 2. These functions will fail if used outside of the red or green regions of Figure 2.

get_mat_tauEvaluate DLR expansion of CijM(τ)C_{ij}^{M}(\tau) for arbitrary 0≤τ≤β0 \le\tau\le\beta into provided Eigen matrix.
get_tv_tauEvaluate DLR expansion of Cij(t,τ)C_{ij}(t,\tau) for arbitrary 0≤τ≤β0 \le\tau\le\beta and given timestep into provided Eigen matrix.
get_mat_reversedEvaluate CijM(β−τl)C_{ij}^{M}(\beta-\tau_l) into provided Nτ×No2N_\tau\times N_o^2 Eigen matrix, where τl\tau_l are DLR nodes
get_tv_reversedEvaluate Cij(t,β−τl)C_{ij}(t,\beta-\tau_l) into provided Nτ×No2N_\tau\times N_o^2 Eigen matrix, where τl\tau_l are DLR nodes

Table 5. Functions to evaluate DLR representations at points not on DLR nodes.

Numerical integration

At timestep TT, we must solve for the retarded, mixed, and lesser components. This requires us to approximate the derivatives and history integrals evaluated on an equidistant grid with spacing hh. These approximations are implemented in the integration class, which supports discretization error scaling as hk+1h^{k+1}, where 1≤k≤51 \leq k \leq5 is a user-specified parameter. In Appendix A, we provide a brief discussion of the approximation schemes we use for evaluating derivatives and integrals. For an in-depth discussion on how these high-order integrators are implemented, we refer the reader to Refs. [17, 76].

Discrete Lehmann representation

H-NESSi uses the discrete Lehmann representation (DLR) [77], via the libdlr library [78], to discretize imaginary time functions on [0,β][0,\beta]. The dlr_info class provides a simple interface, managing the internal data and accessor functions. We give an overview of the method and its user-facing parameters.

The libdlr library requires two parameters, Λ\Lambda and ϵdlr\epsilon_{\mathrm{dlr}}, which control the accuracy of the representation. Λ=βωmax⁡\Lambda= \beta\omega_{\max} sets the maximum spectral support, scaled by the inverse temperature; it can typically be estimated from the energy scales of the system under study. The second parameter, ϵdlr\epsilon_{\mathrm{dlr}}, sets the numerical precision of the representation. Given these parameters, the DLR is constructed by selecting a sparse grid τi∈(0,β)\tau_i \in(0,\beta) on which the imaginary time axis is sampled. The number of sampling points rr has been shown to scale as r=O(log⁡(Λ)log⁡(1/ϵdlr))r = O(\log(\Lambda)\log(1/\epsilon_{\mathrm{dlr}})) [77]. From these nodes, a sum-of-exponentials representation of the function—the DLR expansion—can be constructed.

H-NESSi stores all imaginary time functions exclusively on this sparse DLR grid, so the user is responsible for evaluating the self-energy at these nodes. Helper functions in the dlr_info and herm_matrix_hodlr classes enable evaluation at arbitrary τ∈[0,β]\tau\in[0,\beta]; see Table 4 and Table 5 for examples. Example self-energy evaluations for the Matsubara and mixed components are provided in Appendix B.

get_mat<br>set_matCopy No×NoN_o \times N_o Matsubara data CijM(τ)C_{ij}^{M}(\tau) at given DLR node τi\tau_i to/from provided Eigen matrix.
map_matReturn No×NoN_o \times N_o Eigen matrix map of CijM(τ)C_{ij}^{M}(\tau) at given DLR node τi\tau_i
get_tv<br>set_tvCopy No×NoN_o \times N_o mixed data Cij(t,τ)C_{ij}(t,\tau) at given DLR node τi\tau_i and timestep tt to/from provided Eigen matrix.
map_tvReturn No×NoN_o \times N_o Eigen matrix map of Cij(t,τ)C_{ij}(t,\tau) at given DLR node τi\tau_i and timestep tt

Table 4. Functions to access imaginary time functions at DLR nodes.

Features and layout of the library

Here we list the main capabilities of H-NESSi:

  • Store and incrementally build two-time finite-temperature Green’s functions and self-energies in compressed representation using the herm_matrix_hodlr class

  • Numerically integrate the KBE, leveraging the compressed representations of GG and Σ\Sigma, using the dyson class, with first- to sixth-order integrators implemented in the integration class

  • Manipulate imaginary-time Green’s functions in the DLR format using the DLR class

  • Store one-time contour functions in the function class

H-NESSi is implemented in C++ and includes Python scripts for reading, plotting, and profiling. The documentation provides build instructions, API information, and example programs. The library is available as a public Git repository at https://github.com/KBE-hodlr/H-NESSi. The folder H-NESSi/test contains unit tests to verify installation and output of example programs. H-NESSi uses the Eigen linear algebra package to perform linear algebra routines and interface with the data structures. All data is stored using row-major ordering.

The H-NESSi interface is designed to be nearly identical to the NESSi interface [17]. The main difference is the underlying compressed representation of the two-time quantities. As long as the user only modifies or retrieves values at the current timestep, a H-NESSi program is essentially the same as a NESSi one. Retrieval of compressed data is possible through the same interface, but the reconstruction is expensive, and can be avoided in almost all cases. The second difference is the use of the DLR grid instead of equidistant imaginary-time points, which requires minimal code modifications via the functions in Table 4 and Table 5.

The final difference is the introduction of four additional parameters, which we now list along with guidelines for their selection. NlN_l is the number of levels in the hierarchical decomposition of the Green’s functions and self-energies. In Figure 2 we have Nl=4N_l = 4, as the decomposition contains four different block sizes. We suggest choosing this value so that Nt/2Nl≈10N_t/2^{N_l} \approx10. Next is the SVD truncation parameter, εSVD\varepsilon_{SVD} which represents the desired precision of the solution. In our experience, 10−5<εSVD10^{-5} < \varepsilon_{SVD} can lead to amplification of errors due to inaccurate evaluation of history integrals. For small values of εSVD\varepsilon_{SVD}, care must be taken to ensure that noise arising from unconverged self-consistency loops or discretization error from the finite integration order does not enter the compressed representations. Lastly, the two DLR parameters, εdlr\varepsilon_{\mathrm{dlr}} and Λ\Lambda, are discussed in § 3.5.

The herm_matrix_hodlr class

In this section, we provide an overview of functionality contained within the herm_matrix_hodlr class, including accessing and updating the data it contains, as well as updating the compressed representations at each timestep.

herm_matrix_hodlr is the class responsible for storing two-time Green’s functions and self-energies used in the integration of the KBE. Each object stores four Keldysh components: Matsubara (MM), retarded (RR), lesser (<<), and mixed (⌉\rceil). The Matsubara component is stored on the DLR grid. The mixed component has two arguments: the imaginary axis is discretized on the DLR grid, and the real axis on an equispaced grid. Neither of these two components are stored in an SVD-compressed format. The retarded and lesser components are stored in the HODLR-compressed representation; see Figure 2. A core responsibility of the herm_matrix_hodlr class is managing the HODLR representation and updating the TSVD of each block as new rows are computed. This is done by calling update_blocks once per timestep.

Because of the high-order integration routines, the compressed representation lags the timestepping by k+1k + 1 steps, where kk is the integration order. At timestep TT, the data GR(T−k≤t≤T,0≤t′≤t)G^R(T-k \le t \le T,0 \le t' \le t) and G<(0≤t≤t′,T−k≤t′≤T)G^<(0 \le t \le t',T-k \le t' \le T)—corresponding to the red and green regions in Figure 2—has been computed but not yet compressed. These regions are stored directly, allowing efficient access and modification via the functions in Table 2, which provide Eigen maps or copy data to/from Eigen matrices. These functions will fail if used to access data outside the red and green regions.

The blue and yellow regions of Figure 2 correspond to data stored in TSVDs and directly stored triangles, respectively; these are read-only and must be accessed via the functions in Table 3, valid for any (t,t′)(t,t') up to the current timestep TT. For points in the yellow, green, and red regions, these functions simply copy directly stored data. Accessing data in the TSVD (blue) region requires reconstruction of the function values via the TSVD, which requires O(Ns2)O(N_s^2) operations. If used often, the cost of this reconstruction procedure can become significant compared to cases in which data can only be accessed while in the directly stored regions. Solving the KBE only requires the user to modify the self-energy at the current timestep (red region), meaning that for almost all cases, reconstruction of Green’s function and self-energy values from the TSVD can be avoided.

get_lesEvaluate compressed representation of Cij<(t,t′)C^<_{ij}(t,t') at given (t,t′)(t,t') into provided Eigen matrix.
get_retEvaluate compressed representation of CijR(t,t′)C^R_{ij}(t,t') at given (t,t′)(t,t') into provided Eigen matrix.

Table 3. General get functions for retarded and lesser components that are valid for any (t,t′)(t,t') point up to the current timestep. For a point in the blue region of Figure 2, this involves reconstructing the value from the TSVD representation. Points in the yellow, green, and red regions simply involve copying data.

As discussed in § 3.5, imaginary-time functions are stored on the sparse DLR grid τi∈(0,β)\tau_i \in(0,\beta). The functions in Table 4 allow retrieval and manipulation of this data. It is often necessary, however, to evaluate the Matsubara and mixed components at points off the DLR grid—for example, to compute the equilibrium density matrix −GM(β)-G^M(\beta) (the DLR grid does not include endpoints). The functions in Table 5 evaluate imaginary-time functions at arbitrary 0≤τ≤β0 \le\tau\le\beta using the DLR expansion. One can also obtain imaginary-time quantities on the “reversed” DLR grid τi→β−τi\tau_i \to\beta-\tau_i, which is needed for self-energy diagrams involving quantities such as GM(−τ)=ξGM(β−τ)G^{M}(-\tau)=\xi G^{M}(\beta-\tau).

The dyson class

The dyson class solves the KBE in three steps: (1) solving the Matsubara problem, (2) bootstrapping, and (3) timestepping. In this section we will discuss the functions provided that solve the KBE for the respective steps. In all three cases, these functions will return the norm ∣G(i)−G(i−1)∣\lvert G^{(i)}-G^{(i-1)}\rvert between successive self-consistent iterations, which can be used to terminate the loop once the desired accuracy is reached.

First, the Matsubara problem is solved to obtain the initial thermal state. This step is optional: systems may alternatively be initialized via adiabatic switching on the two-legged contour or with an uncorrelated density matrix, in which case the user begins directly at bootstrapping.

functiondescription
dyson_matPerform a Matsubara iteration.

Table 6. Matsubara functions in the dyson class

The bootstrap process initializes the kthk^{\text{th}}-order accurate multistep timestepper by handling the first kk timesteps. The first kk timesteps are solved simultaneously, meaning that at each iteration the user must update hMFh^{MF} and Σ\Sigma for all 0≤t≤k0 \le t \le k before calling one of the bootstrap functions listed in Table 7. To initialize the self-consistency loop, we must provide an initial guess for GG. This can be done using the green_from_H function, which evaluates the mean-field real-time Green’s function for the first kk timesteps.

functiondescription
dyson_startPerform a Bootstrap iteration. Σ\Sigma and hMFh^{MF} must be TTI for first kk timesteps.
dyson_start_nttiPerform a Bootstrap iteration. No requirements on Σ\Sigma or hMFh^{MF}.
dyson_start_2legPerform a Bootstrap iteration. Solve without ⌈\lceil or MM component.

Table 7. Bootstrap functions in the dyson class.

If the system is driven out of equilibrium in the first kk timesteps, for example, by an interaction quench or an electric field pulse, the user should use the non-time-translation invariant (ntti) solver dyson_start_ntti, which makes no assumptions regarding the dynamics of the system, and directly solves the KBE. This function implements the same bootstrap procedure as laid out in Ref. [17]. If the system remains in the translationally invariant state relative to the thermal preparation for at least the first kk timesteps, we provide the function dyson_start, which only solves the mixed component equation and uses time-translation invariance to obtain G<G^{<} and G>G^{>} . Lastly, the function dyson_start_2leg is for cases when the system is not prepared in an initial thermal state and therefore has no imaginary-time leg. In such cases, only the retarded and lesser components are solved.

After the bootstrap is complete, the system is integrated using the timestepping functions. Before each timestep, an initial guess for the self-consistency loop may be obtained by extrapolating from the previous kk timesteps using the extrapolate function. After this, the timestep may be solved self-consistently using one of the two timestep functions shown in Table 8.

functiondescription
dyson_timestepPerform a timestep iteration. No requirements on Σ\Sigma or hMFh^{MF}.
dyson_timestep_2legPerform a timestep iteration. Solve without ⌈\lceil or MM component.

Table 8. Timestepping functions in the dyson class.

The density matrix, ρ(t)=ξiG<(t,t)\rho(t)=\xi iG^{<}(t,t), describes the particle distribution, and enters the KBE via the mean-field Hamiltonian, making its accurate propagation essential. When constructing the dyson class, one of two arguments must be provided: either RHO_HORIZONTAL or RHO_DIAGONAL. The former provides no special treatment of the density matrix and simply integrates the lesser Green’s function as laid out in Ref. [17].

The latter treats the density matrix in a special manner by integrating the equation

∂tρ(t)∣t=T=ξi∂tG<(t,T)+ξi∂tG>(T,t)\left.\partial_t\rho(t)\right|_{t=T}=\xi_i\partial_tG^{<}(t,T)+\xi_i\partial_tG^{>}(T,t)

along the t=t′t=t^{\prime} diagonal, which incurs no extra computational expense. While both schemes solve the KBE, the diagonal integration provided by RHO_DIAGONAL often results in more strictly preserved conservation laws.

To compare the two schemes, we plot population dynamics for an initially occupied site in a simple two-band, two-site Hubbard model in Figure 3. The state is initially fully occupied before a driving field is turned on at t=10t=10 to excite a portion of the population into the conduction band. We consider two variations of this setup: a short pulse excitation where the field is quickly turned off, Figure 3a, and continuous periodic driving leading to sustained population oscillations, Figure 3b. In both instances, the two methods converge to the same physical result as the timestep size hh is reduced. In the continuous driving case, this convergence is demonstrated by the rolling averages for h=0.0125h=0.0125, for which the diagonal method (solid dark line) and the horizontal method (dashed light line) perfectly overlap.

Figure 3. Comparison of the dynamics for a diagonal element in the density matrix for RHO_DIAGONAL (solid lines) and RHO_HORIZONTAL (dashed lines) after (a) a short pulse excitation and (b) continuous periodic driving. The solid dark lines and light dashed lines overlaid in (b) are rolling averages meant to guide the eye and demonstrate convergence, as the underlying data is highly oscillatory. Real (c) and imaginary (d) part of the density matrix trace for continuous driving. All calculations use ϵSVD=10−10\epsilon_{SVD}=10^{-10}.

Advantages of the diagonal integration are shown in (c), where we demonstrate that the diagonal integration very accurately preserves the total number of particles in the system, whereas the horizontal integration is worse by several orders of magnitude and has a continually compounding loss of particle number. Furthermore, (d) illustrates that the horizontal integration method causes a non-zero imaginary component in the density matrix, which in turn leads to a small non-Hermitian part of the mean-field Hamiltonian. While one could manually force the mean-field Hamiltonian to be Hermitian at each timestep, this ad-hoc intervention only corrects the time-evolution operator; it does not fix the underlying non-Hermiticity of the density matrix itself, meaning physical observables will still be corrupted. The diagonal method naturally avoids this flaw entirely.

A notable caveat to and failure of the diagonal method occurs in steady-state regimes; in situations where the system remains in a steady-state for long periods of time, as in panel (a), the diagonal integration may fail and rapidly diverge. Therefore, in simulation regimes where the system is expected to remain in a steady state for long times, the diagonal method should be avoided, and RHO_HORIZONTAL, with a manual Hermitian correction to the mean-field Hamiltonian, should be used instead.

Example program: driven superconductor

Here we provide an example implementation that showcases how to use H-NESSi to integrate the KBE. We solve a driven superconducting system modeled using the attractive Hubbard model, which is solved self-consistently using the DMFT approximation and a second-Born self-energy. We provide a brief overview of the problem setup; a more in-depth discussion is available in Ref. [67].

The attractive Hubbard Hamiltonian is given by

H=−t0∑⟨j,k⟩σeiqA(t)djσ†dkσ−U∑j(nj↑−1/2)(nj↓−1/2),(9)H=-t_0\sum_{\langle j,k\rangle\sigma}e^{iqA(t)}d_{j\sigma}^{\dagger}d_{k\sigma}-U\sum_j(n_{j\uparrow}-1/2)(n_{j\downarrow}-1/2), \tag*{(9)}

where diσd_{i\sigma} is the annihilation operator at site ii and spin σ\sigma, niσn_{i\sigma} is the spin-dependent density operator and qq is the charge. To describe the dynamics in the superconducting state, it is convenient to rewrite the problem using Nambu spinors

ψi=(di↑di↓†).(10)\psi_i=\begin{pmatrix}d_{i\uparrow}\\d_{i\downarrow}^{\dagger}\end{pmatrix}. \tag*{(10)}

The Hamiltonian now takes the form

H(t)=−t0∑⟨j,k⟩αeiqA(t)αψjα†ψkα(11)H(t)=-t_0\sum_{\langle j,k\rangle\alpha}e^{iqA(t)\alpha}\psi_{j\alpha}^{\dagger}\psi_{k\alpha} \tag*{(11)}
+U(t)∑iψi↑†ψi↓†ψi↓ψi↑−U(t)2∑iαψiα†ψiα,(12){}+U(t)\sum_i\psi_{i\uparrow}^{\dagger}\psi_{i\downarrow}^{\dagger}\psi_{i\downarrow}\psi_{i\uparrow}-\frac{U(t)}{2}\sum_{i\alpha}\psi_{i\alpha}^{\dagger}\psi_{i\alpha}, \tag*{(12)}

where α∈{−1,1}\alpha\in\{-1,1\} runs through the first and second components of the Nambu spinor. We solve the KBE for the normal and anomalous Green’s functions

G(t,t′)=⟨ψ(t)ψ†(t′)⟩=(⟨d↑d↑†⟩(t,t′)⟨d↑d↓⟩(t,t′)⟨d↓†d↑†⟩(t,t′)⟨d↓†d↓⟩(t,t′)).(13)G(t,t')=\langle\psi(t)\psi^\dagger(t')\rangle= \begin{pmatrix} \langle d_{\uparrow}d_{\uparrow}^{\dagger}\rangle(t,t') & \langle d_{\uparrow}d_{\downarrow}\rangle(t,t')\\ \langle d_{\downarrow}^{\dagger}d_{\uparrow}^{\dagger}\rangle(t,t') & \langle d_{\downarrow}^{\dagger}d_{\downarrow}\rangle(t,t') \end{pmatrix}. \tag*{(13)}

The system is then mapped to a single site using DMFT on a Bethe lattice, which introduces the hybridization function for the driven situation [79],

Δ(t,t′)=ΔR(t,t′)+ΔL(t,t′)\Delta(t,t')=\Delta_R(t,t')+\Delta_L(t,t')
ΔR(t,t′)=12tˉ0(t)σzG(t,t′)σztˉ0∗(t′)\Delta_R(t,t')=\frac{1}{2}\bar{t}_0(t)\sigma_zG(t,t')\sigma_z\bar{t}_0^\ast(t')
ΔL(t,t′)=12tˉ0∗(t)σzG(t,t′)σztˉ0(t′),(14)\Delta_L(t,t')=\frac{1}{2}\bar{t}_0^\ast(t)\sigma_zG(t,t')\sigma_z\bar{t}_0(t'), \tag*{(14)}

with hopping matrix elements given by

tˉ0(t)=(eiA(t)00e−iA(t)),(15)\bar{t}_0(t)= \begin{pmatrix} e^{iA(t)} & 0\\ 0 & e^{-iA(t)} \end{pmatrix}, \tag*{(15)}
A(t)=−∫0tE(s) ds,(16)A(t)=-\int_0^t E(s)\,\mathrm{d}s, \tag*{(16)}

due to the Peierls substitution. Here EE is the driving electric field E(t)=E0exp⁡(−(t−tc)2σ2)sin⁡(ω(t−tc))E(t)=E_0\exp\left(-\frac{(t-t_c)^2}{\sigma^2}\right)\sin(\omega(t-t_c)), with pulse center tct_c, pulse width σ\sigma, driving frequency ω\omega, and amplitude E0E_0. The hybridization function enters the KBE in the same manner as the self-energy, so the KBE in Eqs. (5)-(8) are solved with Σ→Σ+Δ\Sigma\to\Sigma+\Delta. This introduces no algorithmic complications: in the example below, we simply define DeltaPlusSigma and compress the sum as a single object. The impurity problem is solved using the fully self-consistent second Born approximation. In this approximation, the Fock term is given by

ΣijF(t)=−U(t)ρij(t)δijˉ,(17)\Sigma^F_{ij}(t)=-U(t)\rho_{ij}(t)\delta_{i\bar{j}}, \tag*{(17)}

where iˉ=1−i\bar{i}=1-i. We work at half-filling, so that the Hartree term cancels the chemical potential. For the first several iterations of the self-consistency loop in equilibrium, we apply a small source field hijB=ηδijˉh^B_{ij}=\eta\delta_{i\bar{j}} to break symmetry and possibly induce a phase transition into the superconducting state. The time-dependent bubble and exchange diagrams are given by

ΣijB(t,t′)=U(t)U(t′)Giˉjˉ>(t,t′)Giˉjˉ<(t,t′)Gjˉiˉ>(t′,t),\Sigma^B_{ij}(t,t')=U(t)U(t')G^{>}_{\bar{i}\bar{j}}(t,t')G^{<}_{\bar{i}\bar{j}}(t,t')G^{>}_{\bar{j}\bar{i}}(t',t),
ΣijE(t,t′)=−U(t)U(t′)Giˉj>(t,t′)Giˉj<(t,t′)Gjˉiˉ>(t′,t).(18)\Sigma^E_{ij}(t,t')=-U(t)U(t')G^{>}_{\bar{i}j}(t,t')G^{<}_{\bar{i}j}(t,t')G^{>}_{\bar{j}\bar{i}}(t',t). \tag*{(18)}

Appendix B contains code implementing these expressions.

Initialization of objects

To begin the program, we define the scalar parameters for the calculation. The number of timesteps nt and the number of HODLR levels nlvl define the geometry of the hierarchical decomposition, while svdtol is the truncation tolerance of the TSVD of each block. The parameters h and k are the timestep size and integration order, respectively. For systems initialized in thermal equilibrium, we must provide the inverse temperature beta, as well as the DLR parameters epsdlr, and lambda discussed in § 3.5. The self-consistent iteration is controlled by the MaxIter and MaxErr parameters, which set the maximum number of iterations and the tolerance at which solutions are considered converged. Lastly, the matrix dimension of the Green’s function and self-energy is set by nao, and the particle statistics (+1 for bosons, -1 for fermions) is set by xi.

// Hodlr parameters
int nlvl, nt, k;
double svdtol, h;

// Matsubara parameters
double epsdlr, lambda, beta;
int ntau;

// Self-Consistency Parameters
int BootMaxIter, StepMaxIter, MatsMaxIter;
double BootMaxErr, StepMaxErr, MatsMaxErr;

// Number of orbitals and particle statistics
int nao, xi;

These parameters can be read from an input file, which is provided as a command line argument:

char * flin;
flin=argv[1];
find_param(flin,"__nlvl=",nlvl);

We then initialize the required objects using classes defined in H-NESSi, namely the DLR, integration weights, Dyson solver, Green’s function, self-energy, self-energy evaluator, and quadratic Hamiltonian. Note that the dlr_info class takes the argument ntau by reference and updates it to the number of DLR nodes required to represent imaginary-time functions to the accuracy set by lambda and epsdlr. The self-energy and hybridization functions, Eqs. (14), (17), and (18), are implemented in the class SC_gf2; a demonstration of their implementation is shown in Appendix B.

// Dyson
hodlr::dlr_info dlr(ntau, lambda, epsdlr, beta,
    nao, xi);
Integration::Integrator I(k);
int r_v = RHO_DIAGONAL;
hodlr::dyson dyson_sol(nt, nao, k, dlr, r_v);
// Green’s function storage
hodlr::herm_matrix_hodlr G(nt, ntau, nlvl, svdtol
    nao, nao, xi, k);
hodlr::herm_matrix_hodlr DeltaPlusSigma(nt, ntau,
    nlvl, svdtol, nao, nao, xi, k);
// Self Energy evaluator
hodlr::SC_gf2 SC_gf2_solver(dlr);
// Time dependent Hamiltonian & chem. pot.
double mu = 0;
hodlr::function Hmf(nt, nao, nao);
hodlr::function U_func(nt, 1, 1);
hodlr::function t0_func(nt, nao, nao);

Matsubara problem

Once all objects are initialized, we solve the Matsubara problem self-consistently. At each iteration, the Hartree-Fock self-energy, second-Born self-energy, and hybridization function are evaluated, and the Dyson equation is solved in the DLR basis [77, 80]. The difference between the previous and updated GMG^{M} is returned, and iterations continue until convergence. After the Matsubara problem is solved, we must call the initGMConvTensor function of the Green’s function object to calculate the imaginary-time convolution in the equation of the mixed component. Note that the value of a function object on the equilibrium branch f(t=0∣)f(t = 0|) can be accessed by setting tstp=-1.

// Do Matsubara iterations
double MatErr;
double eta = 0.0001;
int eta_steps = 5;
int tstp = -1;
for(int iter = 0; iter < MatsMaxIter; iter++) {
  // Reset Self-Energy
  DeltaPlusSigma.set_tstp_zero(-1);

  // Evaluate Delta via increment
  SC_gf2_solver.solve_Delta(tstp, t0_func, G,
                            DeltaPlusSigma);

  // Evaluate Sigma via increment
  SC_gf2_solver.solve_Sigma(tstp, U_func, G,
                            DeltaPlusSigma);

  // Evaluate Hmf
  SC_gf2_solver.solve_Sigma_Fock(tstp, U_func, G,
                                 Hmf);

  // For first several iters, add small field to
  // break symmetry
  if(iter < eta_steps) {
    Hmf(tstp,0,1) += eta;
    Hmf(tstp,1,0) += eta;
  }

  // Solve Dyson Equation for G Matsubara
  MatErr = dyson_sol.dyson_mat(G, mu, Hmf,
                               DeltaPlusSigma, false);

  // Exit if iteration difference is within
  // tolerance.
  // Only exit after magnetic field is off
  if(MatErr < MatsMaxErr && iter > eta_steps)
    break;
}
// Matsubara is converged, we set the convolution
// tensor
G.initGMConvTensor(dlr);

Bootstrapping

After solving the Matsubara problem, we begin the bootstrapping portion of the integration. To obtain a starting point for the self-consistency loop, we evaluate the mean-field Green’s functions as

GR,HF(t,t′)=−iθ(t−t′)exp⁡[−iϵ(0∣)(t−t′)](19)G^{R,\mathrm{HF}}(t,t')=-i\theta(t-t')\exp\left[-i\epsilon^{(0|)}(t-t')\right] \tag*{(19)}
G<,HF(t,t′)=iexp⁡[−iϵ(0∣)t]ρMexp⁡[iϵ(0∣)t′](20)G^{<,\mathrm{HF}}(t,t')=i\exp\left[-i\epsilon^{(0|)}t\right]\rho^{M}\exp\left[i\epsilon^{(0|)}t'\right] \tag*{(20)}
G⌉,HF(t,τ)=ξiexp⁡[−iϵ(0∣)t]ρMexp⁡[ϵ(0∣)τ](21)G^{\rceil,\mathrm{HF}}(t,\tau)=\xi i\exp\left[-i\epsilon^{(0|)}t\right]\rho^{M}\exp\left[\epsilon^{(0|)}\tau\right] \tag*{(21)}

using the mean-field Matsubara Hamiltonian. At each iteration, we first evaluate the self-energy and hybridization, Eqs. (14), (17), and (18), for 0≤t≤k0 \leq t \leq k (with kk the order of accuracy), before solving the KBE in this same region. Just as in the Matsubara case, the dyson integrator returns the difference between the previous and current iterations of GG, which is used to monitor convergence.

// Self-consistency for bootstrapping
dyson_sol.green_from_H(G, mu, Hmf, h, k);
for(int iter = 0; iter < BootMaxIter; iter++) {

  // Evaluate Hmf
  for(int tstp = 0; tstp <= k; tstp++) {
    SC_gf2_solver.solve_Sigma_Fock(tstp, U_func,
                                    G, Hmf);
  }

  // Evaluate Self-Energy
  for(int tstp = 0; tstp <= k; tstp++) {
    DeltaPlusSigma.set_tstp_zero(tstp);
    SC_gf2_solver.solve_Delta(tstp, t0_func, G,
                              DeltaPlusSigma);
    SC_gf2_solver.solve_Sigma(tstp, U_func, G,
                              DeltaPlusSigma);
  }

  // Solve Dyson
  double err = dyson_sol.dyson_start_ntti(G, mu,
                                          Hmf, DeltaPlusSigma, I, h);

  if(err < BootMaxErr) break;
}

Timestepping and SVD updates

After the bootstrapping procedure has converged, we proceed to timestepping. To begin each step, we update the Kadanoff–Baym representations of the Green’s function and self-energy. We then extrapolate the Green’s function to get an initial guess for the self-consistent iteration. Within the self-consistency loop, we first evaluate the mean-field and correlated self-energies before calling the timestep function provided by the dyson class, which again returns the difference between GG and its previous iteration.

for(int tstp = k+1; tstp < nt; tstp++) {
  G.update_blocks(I);
  DeltaPlusSigma.update_blocks(I);

  dyson_sol.Extrapolate(G, I);

  for(int iter = 0; iter < StepMaxIter; iter++)
  {
    // HF
    SC_gf2_solver.solve_Sigma_Fock(tstp, U_func,
                                   G, Hmf);

    // 2B
    DeltaPlusSigma.set_tstp_zero(tstp);
    SC_gf2_solver.solve_Delta(tstp, t0_func, G,
                              DeltaPlusSigma);
    SC_gf2_solver.solve_Sigma(tstp, U_func, G,
                              DeltaPlusSigma);

    // Dyson
    double err = dyson_sol.dyson_timestep(tstp, G,
                                          mu, Hmf, DeltaPlusSigma, I, h);

    if(err < CorrectorMaxErr) break;
  }
}

Outputting results

Once the calculations are complete, the data may be printed to an hdf5 file using the following functions. A python notebook is included in the Plotting directory which reads in GRG^{R} and G<G^{<} printed using the write_to_hdf5 function.

// Open output file
h5e::File out_file("out.hdf5", h5e::File:::
    Overwrite | h5e::File::ReadWrite | h5e::File
    ::Create);
// Output for Green’s function and self-energy
G.write_to_hdf5(out_file, "G");
DeltaPlusSigma.write_to_hdf5(out_file, "S");
// Output single-time functions
Hmf.write_to_hdf5(out_file, "H");
// Write timing information is profiling if
    enabled
dyson_sol.write_timing(out_file, "dyson");

Checkpoint files

Calculations that must be interrupted and restarted can make use of hdf5 checkpoint files, which store the full simulation state. These files can initialize objects via specialized constructors, and are supported for the herm_matrix_hodlr, function, and dyson classes. An example of writing a checkpoint file is as follows:

// Open checkpoint file
h5e::File check_file("check.hdf5", h5e::File::
    Overwrite | h5e::File::ReadWrite | h5e::File
    ::Create);
// Checkpoints for Green’s function and self-
    energy
G.write_checkpoint_hdf5(check_file, "G");
DeltaPlusSigma.write_checkpoint_hdf5(check_file,
    "S");
// Checkpoints for single-time functions
Hmf.write_checkpoint_hdf5(check_file, "H");
// Checkpoints for dyson integrator only
    necessary for timing information
dyson_sol.write_checkpoint_hdf5(check_file, "
    dyson");

A calculation can then be restarted by reading from this file:

h5e::File check_file("check.hdf5", h5e::File::
    ReadOnly);
hodlr::herm_matrix_hodlr G(check_file, "G");
hodlr::herm_matrix_hodlr DeltaPlusSigma(
    check_file, "S");
hodlr::function Hmf(check_file, "H");
hodlr::dyson dyson_sol(check_file, "dyson", dlr);

Timing results

To demonstrate the efficiency of the H-NESSi code, we plot the wall clock time required to reach a given timestep and the ε\varepsilon-rank for growing block sizes in Figure 4(a) and (Figure 4b), respectively. We compare three different calculations: a reference implementation in the NESSi package, a calculation from H-NESSi in the superconducting state, and a calculation from H-NESSi in the normal state. From (b), we can see that the ε\varepsilon-rank of the Green’s function has very different behavior in the two different phases. The normal state is highly compressible, and the ranks of the blocks saturate at small block sizes, whereas in the superconducting state, the ranks grow as N1/2N^{1/2}, where NN is the size of the block. The behavior of the ranks affects the efficiency of our integration algorithm, as shown in (a). For the Dyson solver, both the superconducting and normal states scale approximately as T2+αT^{2+\alpha}, where α\alpha is the exponent characterizing the growth of the ε\varepsilon-rank with block size (e.g., α≈0.5\alpha\approx0.5 for the superconducting state and α≈0\alpha\approx0 for the normal state). This is significantly better than the T3T^3 scaling of the NESSi solver (dot-dashed line in (a)). Furthermore, the scaling prefactors are observed to be significantly smaller in H-NESSi. We also show timings for the TSVD update step in orange. The observed scaling matches the expected scaling of T2+2αT^{2+2\alpha} for both the normal and superconducting phases.

Scaling of wall clock time and epsilon-rank with timestep and block size

Figure 4. (a) Wall clock time for superconducting (solid lines) and disordered (dashed lines) attractive Hubbard model with asymptotic scaling fits (black lines). Blue and orange lines show contribution of solving timesteps and updating SVD blocks, respectively. The timing of the NESSi solver [17] (dot-dashed line) is shown for comparison. (b) Scaling of the ε\varepsilon-ranks versus the block size in the superconducting state at β=18\beta= 18 (SC) and the disordered state at β=1\beta= 1 (normal). Hamiltonian parameters are U=2U = 2, E0=0.02E_0 = 0.02, α=6.5\alpha= 6.5, tc=16t_c = 16, ω=1.3\omega= 1.3. H-NESSi data obtained using εSVD=10−6\varepsilon_{\mathrm{SVD}} = 10^{-6}.

MPI parallelization for generic lattice

Many systems of interest possess translational invariance, so that the Green’s functions Gk,ijG_{k,ij} and self-energies Σk,ij\Sigma_{k,ij} are block-diagonal in momentum kk, with ijij indexing non-diagonalizable orbital degrees of freedom. The Dyson equation then decouples across k-points, enabling straightforward parallelization. However, the self-energy is typically a functional of the Green’s function over the entire Brillouin zone, not just at the local k-point. In a distributed-memory setting, inter-process communication of Gk,ijG_{k,ij} is therefore required so that each process can evaluate Σk,ij\Sigma_{k,ij} for its assigned k-points.

This section is organized as follows. § 6.1 introduces the mpi_comm class, which manages inter-node communication with minimal user effort and an interface mirroring that of herm_matrix_hodlr. § 6.2 presents example calculations for the two-dimensional Hubbard model using a massively parallel implementation, comparing performance with NESSi and analyzing Green’s function compression. § 6.3 provides implementation details of our hybrid MPI + OpenMP parallelization, and § 6.4 discusses scaling behavior.

The mpi_comm class

The mpi_comm class in H-NESSi manages workload distribution across MPI processes, communication of Green’s function and self-energy data, and intraprocess parallelization via OpenMP threading. Upon initialization, the mpi_comm object exposes my_Nk, the number of Brillouin zone points assigned to the local process, and my_global_k_list, the corresponding global indices. These quantities are then used to initialize the Green’s functions, self-energies, and any auxiliary containers such as a function object storing the k-dependent Hamiltonian:

mpi_comm comm(global_Nk, Nt, r, nao);
int my_Nk = comm.my_Nk;
std::vector<herm_matrix_hodlr> G_vec;
std::vector<herm_matrix_hodlr> S_vec;
std::vector<function> H_vec;
G_vec.reserve(my_Nk);
S_vec.reserve(my_Nk);
H_vec.reserve(my_Nk);
for(int i=0;i<my_Nk;i++){
    G_vec.emplace_back(Nt,r,nlvl,svdtol,nao,nao,
                       xi,SolverOrder);
    S_vec.emplace_back(Nt,r,nlvl,svdtol,nao,nao,
                       xi,SolverOrder);
    H_vec.emplace_back(Nt,nao,nao);
}

At each timestep, the self-consistency loop shown in Figure 5 iterates the self-energy and Green’s function until convergence. Evaluating the self-energy at a given k-point requires knowledge of Gk′G_{k'} for all k′k' in the Brillouin zone. This communication is handled by two functions: mpi_get_and_comm, which loads a vector of the locally stored Green’s functions into MPI communication buffers and distributes them to all processes, and mpi_comm_and_set, which communicates the evaluated self-energies back and extracts them into a vector of locally-stored self-energies. For both functions, only the Matsubara component is communicated when tstep==-1; otherwise the lesser, retarded, and mixed components at the current timestep are all communicated. For imaginary-time functions, the “reversed” values GM(β−τi)G^{\mathrm{M}}(\beta-\tau_i) and GI(t,β−τi)G^{\mathrm{I}}(t,\beta-\tau_i) are also communicated, as they are often needed in the self-energy evaluation. Each communication function comes in two variants: a _spawn variant, which creates OpenMP threads at each call via #pragma omp parallel to accelerate buffer loading and unloading, and a _nospawn variant, which assumes it is called from within an existing OpenMP parallel region.

Figure 5. (a) Schematic of the MPI communication performed by the mpi_comm class between the alltp_buff and allk_buff data buffers; see the main text for details. Squares of the same color represent the same data, and colored dashed lines indicate their distribution between the two buffers. White squares denote allocated buffer space for future time points or additional momentum points beyond the active block. For the specific parameters shown (Nk=6N_k = 6, Nmpi=3N_{\mathrm{mpi}} = 3, and communication up to t4′t_4'), the colored blocks account for the entirety of the actively communicated data, illustrating the complete data transposition. (b) Typical self-consistency loop for lattice calculations.

The left-hand side of Figure 5 illustrates the communication scheme. Before communication, each rank stores all time points (and DLR nodes, although only the real-time buffers are depicted) at the current timestep for its locally assigned k-points, in the buffer alltp_buff. After communication, each rank instead holds every k-point but only a subset of time points, in the buffer allk_buff. In the illustrated example, at time t=4t = 4, rank 1 receives GkR(t=4,t′)G_k^{\mathrm{R}}(t = 4,t') and Gk<(t,t=4)G_k^{<}(t,t = 4) for t′=2,3t' = 2,3 and every kk. Each rank is then responsible for evaluating the self-energy at its assigned subset of time points (and DLR nodes, for the mixed component) using the communicated Green’s function data. The number of assigned time points is given by my_Nt, and the first assigned time point by my_first_t. The mixed and Matsubara components are split analogously, with my_Ntau and my_first_tau specifying the local imaginary-time nodes. The communicated data is accessed through an interface that returns Eigen matrix maps of the internal buffers; these functions, listed in Table 9, closely mirror those of the herm_matrix_hodlr class.

map_retReturn No×NoN_o \times N_o Eigen matrix map of CijR(t,t′)C^{R}_{ij}(t,t') at given t′t' and k\mathbf{k} for communicated timestep tt
map_lesReturn No×NoN_o \times N_o Eigen matrix map of Cij<(t′,t)C^{<}_{ij}(t',t) at given t′t' and k\mathbf{k} for communicated timestep tt
map_tvReturn No×NoN_o \times N_o Eigen matrix map of Cij⌈(t,τ)C^{\lceil}_{ij}(t,\tau) at given DLR node τi\tau_i and k\mathbf{k} for communicated timestep tt
map_tv_revReturn No×NoN_o \times N_o Eigen matrix map of Cij⌉(t,β−τ)C^{\rceil}_{ij}(t,\beta-\tau) at given DLR node τi\tau_i and k\mathbf{k} for communicated timestep tt
map_matReturn No×NoN_o \times N_o Eigen matrix map of CijM(τ)C^{M}_{ij}(\tau) at given DLR node τi\tau_i and k\mathbf{k}
map_mat_revReturn No×NoN_o \times N_o Eigen matrix map of CijM(β−τ)C^{M}_{ij}(\beta-\tau) at given DLR node τi\tau_i and k\mathbf{k}

Table 9. Functions for reading and writing to the communication buffers internal to mpi_comm.

Since the mpi_comm class handles all communication and buffer management, the user’s only responsibility is to evaluate the self-energy Σk\Sigma_k from GkG_k. In practice, this amounts to using the map functions in Table 9 to replace the communicated GkG_k data with the corresponding Σk\Sigma_k data. A typical implementation has the following structure:

// Communicate G
comm.mpi_get_and_comm_spawn(tstep,G_vec,dlr);
// mpi_comm assigns each mpi rank a range for t’
int init_t = comm.my_first_t;
int last_t = init_t + comm.my_Nt;
// each rank must do assigned t’ and all k
for(int t = init_t; t < last_t; t++) {
    for(int k = 0; k < global_Nk; k++) {
        // Read in G ret,les before overwriting
        ZMatrix GR = comm.map_ret(k,t);
        ZMatrix GL = comm.map_les(k,t);
        // Evaluate Sigma into comm buffers
        comm.map_ret(k,t) = ...;
        comm.map_les(k,t) = ...
    }
}
// mpi_comm assigned range for dlr points
int init_tau = comm.my_first_tau;
int last_tau = init_tau + comm.my_Ntau;
// each rank must do assigned tau and all k
for (int tau = init_tau; tau < last_tau; tau++) {
    for(int k = 0; k < global_Nk; k++) {
        // Read in G mix before overwriting
        ZMatrix GTV = comm.map_tv(k,tau);
        // Evaluate Sigma into comm buffers
        comm.map_tv(k,tau) = ...
    }
}
// Communicate Sigma
comm.mpi_comm_and_set_spawn(tstp, S_vec);

This default interface should suffice for nearly all use cases. For situations requiring communication of additional Keldysh components or only a subset of orbital indices, a low-level interface to the underlying communication buffers is described in Appendix § 11.

Driven 2D Hubbard example

We demonstrate the parallel implementation by computing the optical conductivity of the Hubbard model. The system is probed with a short electric field pulse, and the optical conductivity is extracted by inverting the linear-response relation. The pulse preserves translational invariance, enabling large-scale parallelization of the Dyson solver. Such parallelization is essential: the lattice must be large enough to suppress finite-size effects and the propagation long enough for the current response to fully decay, so that in physically relevant parameter regimes RAM usage easily exceeds 10 TB even with HODLR compression. This example is therefore well suited to illustrate how hierarchical compression enables otherwise infeasible calculations. We require a highly parallel implementation for this example: for the largest lattices and lowest temperatures considered here, we require more than 15 thousand CPU cores across 60 nodes.

Below, we summarize the physical setup, then outline the hybrid MPI + OpenMP parallelization scheme, which combines our distributed data containers with efficient global communication and shared-memory parallelism on each MPI rank. Finally, we compare memory usage and runtimes with NESii and study the scaling behavior of the algorithm.

We consider the square lattice Hubbard model, described by the Hamiltonian

H=−J∑⟨i,j⟩,σcσ,i†cσ,j−μ∑σ,inσ,i+U∑in↑,in↓,i(22)H=-J\sum_{\langle i,j\rangle,\sigma}c^\dagger_{\sigma,i}c_{\sigma,j}-\mu\sum_{\sigma,i}n_{\sigma,i}+U\sum_i n_{\uparrow,i}n_{\downarrow,i} \tag*{(22)}

where ii, jj enumerate lattice sites, c†c^\dagger and cc are creation and annihilation operators, σ∈{↑,↓}\sigma\in\{\uparrow,\downarrow\} denotes spin, JJ is the nearest-neighbor hopping amplitude, nσ,in_{\sigma,i} is the particle-number operator, μ\mu is the chemical potential, and UU is the on-site interaction strength. In practice, the Hartree shift is absorbed into the chemical potential, μ~=μ−U⟨nσ,i⟩\tilde{\mu}=\mu-U\langle n_{\sigma,i}\rangle. The system is probed by a spatially uniform electric field in the xx-direction, introduced through the vector potential A(t)A(t) via the Peierls substitution εk→εk−A(t)\varepsilon_{\mathbf{k}}\to\varepsilon_{\mathbf{k}-A(t)}, where εk\varepsilon_{\mathbf{k}} is the non-interacting dispersion relation. For a given self-energy approximation, we solve the KB equations for the Green’s function and compute the current, kinetic energy, and average occupancy per spin as

⟨j(t)⟩=−2iNk∑kvk−A(t)xGk<(t,t)(23)\langle j(t)\rangle=-\frac{2i}{N_k}\sum_k v^x_{\mathbf{k}-A(t)}G^<_{\mathbf{k}}(t,t) \tag*{(23)}
Ekin=−2iNk∑kεkGk<(t,t)(24)E_{\mathrm{kin}}=-\frac{2i}{N_k}\sum_k\varepsilon_{\mathbf{k}}G^<_{\mathbf{k}}(t,t) \tag*{(24)}
⟨n⟩=−iNk∑kGk<(t,t),(25)\langle n\rangle=-\frac{i}{N_k}\sum_kG^<_{\mathbf{k}}(t,t), \tag*{(25)}

where Nk=L2N_k=L^2 is the number of k\mathbf{k} points, LL is the linear lattice size, and v=∇kεk−A(t)\mathbf{v}=\nabla_{\mathbf{k}}\varepsilon_{\mathbf{k}-A(t)} is the time-dependent velocity. We choose a step-function vector potential,

A(t)=−Amax⁡θ(t)⟹E(t)=Amax⁡δ(t)(26)A(t)=-A_{\max}\theta(t)\quad\Longrightarrow\quad E(t)=A_{\max}\delta(t) \tag*{(26)}

where Amax⁡A_{\max} is the amplitude of the abrupt shift at t=0t=0 and E(t)=−∂tA(t)E(t)=-\partial_tA(t). The δ\delta-function form of the electric field allows us to invert the linear-response relation,

⟨j(t)⟩=∫0tdt′σ(t−t′)E(t′)⟹σ(t)=⟨j(t)⟩Amax⁡,(27)\langle j(t)\rangle= \int_{0}^{t} dt^{\prime}\sigma(t-t^{\prime})E(t^{\prime}) \Longrightarrow\sigma(t) = \frac{\langle j(t)\rangle}{A_{\max}}, \tag*{(27)}

and extract the optical conductivity σ(t)\sigma(t) directly from the computed current. For a short pulse of the electric field in the xx-direction, the Green’s functions and the self-energy have the reflection symmetry

G(kx,ky)(t,t′)=G(kx,−ky)(t,t′),(28)G_{(k_x,k_y)}(t,t^{\prime}) = G_{(k_x,-k_y)}(t,t^{\prime}), \tag*{(28)}

allowing us to consider a smaller number of kk-points, L2→L(L2+1)L^2 \to L\left(\frac{L}{2}+1\right).

To close the system of KB equations, we approximate the self-energy with the self-consistent Born approximation, which for the Hubbard Hamiltonian reads

ΣrM(τ)=U2GrM(τ)GrM(τ)G−rM(−τ)(29)\Sigma^{M}_{\mathbf r}(\tau) = U^2G^{M}_{\mathbf r}(\tau)G^{M}_{\mathbf r}(\tau)G^{M}_{-\mathbf r}(-\tau) \tag*{(29)}
Σr≷(t,t′)=U2Gr≷(t,t′)Gr≷(t,t′)G−r≶(t′,t)(30)\Sigma^{\gtrless}_{\mathbf r}(t,t^{\prime}) = U^2G^{\gtrless}_{\mathbf r}(t,t^{\prime})G^{\gtrless}_{\mathbf r}(t,t^{\prime})G^{\lessgtr}_{-\mathbf r}(t^{\prime},t) \tag*{(30)}
Σr⌉(t,τ)=U2Gr⌉(t,τ)Gr⌉(t,τ)G−r⌈(τ,t),(31)\Sigma^{\rceil}_{\mathbf r}(t,\tau) = U^2G^{\rceil}_{\mathbf r}(t,\tau)G^{\rceil}_{\mathbf r}(t,\tau)G^{\lceil}_{-\mathbf r}(\tau,t), \tag*{(31)}

where r\mathbf r denotes real-space lattice sites. We note that these expressions are much more expensive to evaluate when written in the momentum basis. We therefore use FFTW [81] to Fourier transform Gk→GrG_k \to G_r and inverse-transform Σr→Σk\Sigma_r \to\Sigma_k. Performing these transforms requires every kk-point on each MPI process, which is precisely the communication protocol implemented in the mpi_comm class.

Figure 6 shows an example current obtained with both NESSi and H-NESSi. Memory constraints limit the NESSi calculation to Nt=2048N_t = 2048 time points, whereas H-NESSi reaches Nt=6289N_t = 6289 time points with the same memory budget (green curve in Figure 6). For an equal number of CPU hours, H-NESSi propagates to longer physical times than NESSi (orange curve). In this example, H-NESSi reaches a propagation time roughly three times larger than NESSi under identical memory constraints. For larger NtN_t the advantage is even more pronounced, reflecting the improved memory scaling: O(Nt2)O(N_t^2) for NESSi versus O(NtNξ8)O(N_tN_\xi^8) for H-NESSi. At lower temperatures the current decays substantially more slowly, requiring longer propagation times.

Figure 6. Current obtained from the solution of the KB equations via Eq. (23). The NESii calculation is limited to Nt=2048N_t = 2048 time points, whereas H-NESii reaches Nt=6289N_t = 6289 time points under the same memory constraints (green curve). For the same number of CPU hours, H-NESii propagates to longer physical times, as shown by the orange curve.

times. Simultaneously, larger lattices are needed to suppress finite-size effects. Both factors increase memory requirements and make the calculations more demanding.

Within this setup, equilibrium single-particle correlators in the real-time domain can be obtained by simply setting the external field to zero. The self-consistent Born approximation is less expensive when formulated in real time, scaling as O(Nt)O(N_t), than in real frequency, where a straightforward implementation scales as O(Nω2)O(N_\omega^2). Although the Dyson solver is typically more expensive in real time, the overall calculation remains inexpensive and is straightforward to set up with our code. As an example, the equilibrium spectral function is shown in Figure 7.

Figure 7. Equilibrium spectral function A(k,ω)A(k,\omega) for the Hubbard Hamiltonian at half-filling, with U=0.5U = 0.5 and β=10\beta= 10.

Figure 8(a) shows that the compressibility of the Green’s function varies significantly across kk-points: the least compressible points require nearly twice as many singular values as the most compressible ones. At lower temperatures, this spread appears to narrow, but the overall number of required singular values increases, reflecting the richer structure of low-temperature correlators. Despite these differences, the ϵ\epsilon-rank at all k-points saturates at relatively small block sizes, consistent with optimal quadratic scaling of the integration routines. At block sizes of 8129, the memory savings relative to dense storage reach nearly a factor of 100 at β=5\beta= 5.

Implementation details

The computation of the self-energy is made drastically cheaper by translation symmetry (even in the presence of a uniform electric field encoded in the time-dependent vector potential). As seen from Eqs. (29)–(31), the self-energy is diagonal in the position basis, unlike in the momentum representation. Moreover, the expression is time-local, enabling efficient parallelization through the communication protocol of mpi_comm described in § 6.1.

The self-energy is most efficiently evaluated in real space, whereas the KB equations decouple in the kk-space representation. We bridge the two domains as follows:

  1. Distribution of Green’s functions: mpi_comm distributes the Green’s functions via mpi_get_and_comm, providing each MPI process with all kk-points for its assigned range of time points.

  1. Real-space self-energy evaluation: Each MPI process loops over its assigned time (and DLR) points and performs the following steps:

  • Inverse 2D Fourier transform Gk→GrG_k \to G_r using FFTW.

  • Evaluate the self-energy in the real-space basis via Eqs. (29)–(31).

  • Forward 2D Fourier transform Σr→Σk\Sigma_r \to\Sigma_k using FFTW.

Since these operations are independent at each time (and DLR) point, they can be parallelized over threads with OpenMP.

  1. Distribution of self-energies: mpi_comm redistributes the self-energies via mpi_comm_and_set, returning each process’s local kk-points.

This self-energy evaluation scheme follows exactly the example implementation in § 6.1. The most expensive step in the procedure (aside from internode communication) is the Fourier transforms, which scale as O(Nklog⁡Nk)O(N_k\log N_k), compared to the O(Nk2)O(N_k^2) cost of evaluating the self-energy directly in momentum space. This communication and evaluation process is encapsulated in the born_approx_se class, which also exploits the time-locality of the self-energy to parallelize over time and DLR points using OpenMP:

born_approx_se se_eval(U,L,global_Nk,size,
    nthreads);

The lattice object stores information about all global k-points in the Brillouin zone, accounting for the reflection symmetry of Eq. (28), and provides the time-dependent non-interacting dispersion relation εk(t)\varepsilon_{\mathbf{k}}(t):

lattice_2d_ysymm lattice(L, Nt, dt, J, Amax, mu,
    nao);

The Dyson solver is parallelized over local k-points using OpenMP. Because the dyson class uses internal buffers, it is not thread-safe; each thread must therefore have its own solver instance. The same applies to the dlr_info class, whose off-node evaluation functions (Table 5) are likewise not thread-safe:

// dlr vector
std::vector<hodlr::dlr_info> dlr_vec;
dlr_vec.reserve(nthreads);
for (int i = 0; i < nthreads; i++) dlr_vec.
    emplace_back
    (r, dlrlambda, epsdlr, beta, nao, xi);

std::vector<hodlr::dyson> dyson_sol_vec;
dyson_sol_vec.reserve(nthreads);
    for (int i =0; i<nthreads; i++) dyson_sol_vec.
    emplace_back(Nt, nao, SolverOrder, dlr_vec[
    i], rho_version, profile);

Each k-point is represented by an instance of the kpoint class, which stores two herm_matrix_hodlr objects for the Green’s function and self-energy, the non-interacting dispersion εk(t)\varepsilon_{\mathbf{k}}(t), and methods that perform the Dyson propagation at that k-point. Each MPI rank holds its assigned k-points in a vector corrK_rank of kpoint objects:

 // kpoint contains Gs and Sigmas
 std::vector<std::unique_ptr<kpoint>> corrK_rank;
 corrK_rank.resize(my_Nk);

 // create kpoints either from scratch or from
 // checkpoint
 if(!checkpoint_exists){
    for(int i=0;i<my_Nk;i++){
       int global_k = comm.my_global_k_list[i];
       corrK_rank[i] = std::make_unique<kpoint>(Nt,r
       ,nLvl,svdtol,nao,beta,dt,SolverOrder,
       lattice.kpoints_[global_k],lattice,mu,
       Ainitx);
    }
 }
 else{
    for(int i=0;i<my_Nk;i++){
       int global_k = comm.my_global_k_list[i];
       h5e::File checkpoint_file(checkpoint_dir+
       "GSigma"+std::to_string(i)+".h5", h5e::
       File::ReadOnly);
       corrK_rank[i] = std::make_unique<kpoint>(Nt,r
       ,nLvl,svdtol,nao,beta,dt,SolverOrder,
       lattice.kpoints_[global_k],lattice,mu,
       Ainitx,checkpoint_file);
    }
 }

Observables are computed on each MPI rank for their local k-blocks during timestepping and accumulated across all ranks via MPI_Reduce at the end of the calculation.

The Dyson integration is carried out by the step_dyson method of the kpoint class, which calls `dyson_mat, dyson_start_ntti, or dyson_timestep for the Matsubara, bootstrapping, and timestepping stages, respectively. Since each k\mathbf{k}-point is independent, this loop is parallelized with OpenMP; as noted above, each thread must use its own dyson and dlr_info instances:

std::vector<double> errk(my_Nk);
#pragma omp parallel
{
  int thread_id = omp_get_thread_num();
  #pragma omp for schedule(static)
  for(int k=0;k<my_Nk;k++){
    errk[k] = corrK_rank[k]->step_dyson(tstp,
        SolverOrder, lattice, I, dyson_sol_vec[
        thread_id], dlr_vec[thread_id]);
  }
}
// gather all errors from each k point
double err;
double tot_err;
err = std::reduce(errk.begin(),errk.end());
MPI_Allreduce(&err, &tot_err, 1, MPI_DOUBLE,
    MPI_SUM, MPI_COMM_WORLD);

Performance

In this section, we benchmark the hybrid MPI + OpenMP implementation. To isolate the contributions of OpenMP and MPI, we first consider a smaller system with L=48L = 48 and Nt=2048N_t = 2048, which can still be solved without parallelization. These results are shown in Figure 9. We then consider a large calculation with L=64L = 64 and Nt=16384N_t = 16384, which requires the full hybrid implementation. These results are shown in Figure 10 and correspond to the calculations in Figure 8. In both cases, the integration scales as Nt2.05N_t^{2.05}, consistent with the expected scaling when the ε\varepsilon-ranks saturate at a maximal value, as confirmed in Figure 8. This behavior closely mirrors that of the single-site calculations in § 5.

Four-panel plots showing maximum $\varepsilon$-rank, imaginary parts of the lesser Green’s function, and $\varepsilon$-rank as a function of block size

Figure 8. (a) Maximum ε\varepsilon-rank across different k-points. The red line indicates the Fermi surface. (b, c) Largest block of the imaginary part of the lesser Green’s function G<G^{<} at k=(π,π)\mathbf{k}=(\pi,\pi) and k=(π/2,π/2)\mathbf{k}=(\pi/2,\pi/2). (d) ε\varepsilon-rank of G<G^{<} as a function of block size. All results are shown for two temperatures, with lattice size L=64L=64 and Nt=16384N_t=16384 timesteps.

Figure 9. Scaling of the Dyson solver (a, d) and the self-energy evaluator (b, e) for different numbers of MPI ranks and OpenMP threads, together with the relative speedup with the number of CPUs (c, f). Upper panels: scaling with MPI ranks at Nth=1N_{\rm th}=1. Lower panels: scaling with OpenMP threads at Nmpi=1N_{\rm mpi}=1. The speedup is nearly linear in both cases.

Figure 10. Performance scaling of the hybrid MPI/OpenMP implementation for a fixed total number of CPUs (NCPU=2048N_{\rm CPU}=2048). Cumulative computation time as a function of simulated timestep is shown for (a) the Dyson solver and (b) the self-energy evaluator across various OpenMP thread counts per MPI rank (NthN_{\rm th}). (c) Computational metrics as a function of NthN_{\rm th}. The left axis (blue) shows the total CPU hours normalized to the optimal case (Nth=8N_{\rm th}=8) for two different simulation lengths. The right axis (orange) shows the total memory footprint for the Nt=16384N_{\rm t}=16384 calculation. For these hardware and calculation parameters, the optimal configuration that minimizes computational cost is Nth=8N_{\rm th}=8.

The speedup plots in Figure 9 show the wall time required to reach a given timestep with the Dyson solver and the self-energy evaluator, as a function of CPU count, for an example where memory requirements do not necessitate parallelization. The speedup relative to a single CPU is nearly perfectly linear up to 32 MPI processes on a single node. The OpenMP scaling is slightly less efficient, but still reaches N0.95N^{0.95} up to 16 threads before saturating.

Figure 10 shows results for a system size and propagation time that require a large number of nodes and a hybrid parallelization strategy. The efficiency depends sensitively on the total number of MPI ranks NmpiN_{\mathrm{mpi}}, the number of threads per rank NthN_{\mathrm{th}}, and their ratio. At lower temperatures, for which the Green’s functions decay slowly, a large number of time points (Nt>10000N_t > 10000) is needed for the current to fully decay. Finite-size effects are also more prominent in this regime, requiring large lattice sizes (Nk>8000N_k > 8000), and memory constraints correspondingly demand more computational nodes. For a fixed total number of CPUs, the ratio Nmpi/NthN_{\mathrm{mpi}}/N_{\mathrm{th}} can be tuned to minimize computational cost. Figure 10(c) shows the CPU hours and memory usage for different ratios at a fixed CPU count. Memory grows slowly with NmpiN_{\mathrm{mpi}} because each rank maintains its own copy of several auxiliary communication buffers (see Table C.11). The scalings of the Dyson solver and self-energy evaluator are shown in Figure 10(a) and (b), respectively. The best performance is achieved with Nth=8N_{\mathrm{th}} = 8 threads per rank, a finding we have reproduced across two different examples and two computing clusters.

Conclusions

We have presented H-NESSi, an open-source implementation of nonequilibrium Green’s function propagation based on hierarchical low-rank compression of two-time objects. By combining high-order integration schemes with HODLR representations of the retarded and lesser components, the method reduces the cubic cost scaling and quadratic memory growth that traditionally limit KBE simulations. The discrete Lehmann representation provides an efficient treatment of imaginary-time quantities and thermal initial states. The resulting framework maintains controllable accuracy while substantially reducing computational cost, extending the accessible simulation times and system sizes for nonequilibrium correlated-electron problems.

We have demonstrated these capabilities in two representative applications: a driven superconducting system treated within dynamical mean-field theory, and the two-dimensional Hubbard model solved with the second Born approximation. In both cases, the compressed propagation exhibits favorable cost and memory requirements compared to conventional approaches while preserving the dynamical features of the underlying many-body problem. The modular design supports multi-orbital systems, adaptive singular-value truncation, and hybrid MPI + OpenMP parallelization, making the code suitable for large-scale simulations on modern computing architectures.

The implementation provides a foundation for further extensions, including more advanced self-energy approximations, embedding schemes, and coupling to additional degrees of freedom. By reducing the computational barrier to two-time nonequilibrium simulations, H-NESSi opens the door to systematic studies of long-time dynamics, strong driving protocols, and complex multiorbital materials within a controlled many-body framework.

Acknowledgments

T.B. was supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, Department of Energy Computational Science Graduate Fellowship under Award No. DE-SC0020347 until Aug. 2023. T.B. (after Aug 2023), and E.G. were supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research and Office of Basic Energy Sciences, Scientific Discovery through Advanced Computing (SciDAC) program under Award Number DE-SC0022088. As of June 1, 2025, E.G. was supported by the European Research Council, grant ERC-2023-AdG: 101142136 (Quantum Algorithms). The Flatiron Institute is a division of the Simons Foundation. J. V. and J. Kov. acknowledge funding provided by the Institute of Physics Belgrade, through the grant by the Ministry of Science, Technological Development and Innovation of the Republic of Serbia. J. V. and J. Kov. acknowledge funding by the European Research Council, grant ERC-2022-StG: 101076100. D.G. is supported by the Slovenian Research and Innovation Agency (ARIS) under Programs No. P1-0044, No. J1-2455, and No. MN-0016-106. The authors gratefully acknowledge the HPC RIVR consortium (www.hpc-rivr.si··񎟿)

and EuroHPC JU [(eurohpc-ju.europa.eu)]https://eurohpc−ju.europa.eu/https://eurohpc-ju.europa.eu/ for funding this research by providing computing resources of the HPC system Vega at the Institute of Information Science [(www.izum.si)]https://www.izum.si/en/home/https://www.izum.si/en/home/. Computations were also performed on the PARADOX supercomputing facility (Scientific Computing Laboratory, Center for the Study of Complex Systems, Institute of Physics Belgrade).

Integral and derivative approximations

The numerical evaluation of derivatives, ddty(t)∣T\left.\frac{d}{dt}y(t)\right|_{T}, and integrals, ∫0Tdt y(t)\int_{0}^{T}dt\,y(t), is split into two cases: T≤hkT \le hk and T>hkT > hk. In the former, we use kthk^{\mathrm{th}}-order polynomial differentiation and integration weights; in the latter, backward differentiation and Gregory integration [82]. To obtain the polynomial differentiation and integration weights, we begin with the polynomial interpolant of y(t)y(t) that takes the values yj≡y(jh)y_j \equiv y(jh) on the equidistant grid:

P[y0,…,yk](t)=∑a,l=0kh−ataPalyl(32)\mathcal{P}[y_0,\ldots,y_k](t)=\sum_{a,l=0}^{k}h^{-a}t^aP_{al}y_l \tag*{(32)}
Pal=(M−1)alfor Mja=ja(33)P_{al}=(M^{-1})_{al}\quad\text{for }M_{ja}=j^a \tag*{(33)}

for 0≤j≤k0 \le j \le k. The polynomial differentiation weights DmlD_{ml} are obtained by explicitly differentiating this interpolant:

dydt∣t=mh≈h−1∑l=0kDmlyl,with(34)\left.\frac{dy}{dt}\right|_{t=mh}\approx h^{-1}\sum_{l=0}^{k}D_{ml}y_l,\quad\text{with} \tag*{(34)}
Dml=∑a=1kPalama−1.(35)D_{ml}=\sum_{a=1}^{k}P_{al}am^{a-1}. \tag*{(35)}

The polynomial integration weights ImnlI_{mnl} are similarly obtained by explicitly integrating this interpolant:

∫mhnhdt y(t)≈h∑l=0kImnlyl,with(36)\int_{mh}^{nh}dt\,y(t)\approx h\sum_{l=0}^{k}I_{mnl}y_l,\quad\text{with} \tag*{(36)}
Imnl=∑a=0kPalna+1−ma+1a+1.(37)I_{mnl}=\sum_{a=0}^{k}P_{al}\frac{n^{a+1}-m^{a+1}}{a+1}. \tag*{(37)}

These weights are then used for T≤hkT \le hk to set up the so-called bootstrapping procedure, which is discussed in § 4.2 and implemented in the functions shown in Table 7.

For the T>hkT > hk case, we use the backward differentiation weights and Gregory weights for approximating the derivatives and integrals, respectively. The backward differentiation weights aja_j are obtained from the polynomial differentiation weights via

dydt∣t=nh≈h−1∑j=0kajyn−j,where(38)\left.\frac{dy}{dt}\right|_{t=nh}\approx h^{-1}\sum_{j=0}^{k}a_jy_{n-j},\quad\text{where} \tag*{(38)}
aj=−D0j.(39)a_j=-D_{0j}. \tag*{(39)}

Lastly, the Gregory weights ωi\omega_i are best thought of as high-order edge corrections to the Riemann sum, where the corrections occur for k+1k+1 points on each end of the sum

∫0nhdt y(t)≈h∑i=0kωiyi+h∑i=k+1n−k−1yi+h∑i=0kωiyn−i.(40)\int_{0}^{nh}dt\,y(t)\approx h\sum_{i=0}^{k}\omega_i y_i+h\sum_{i=k+1}^{n-k-1}y_i+h\sum_{i=0}^{k}\omega_i y_{n-i}. \tag*{(40)}

Since the integration weights differ from unity only near the endpoints, the evaluation of history integrals using the SVD representation is greatly simplified. These weights are given by the sum

ωi=∑j=0ibk+1−i(41)\omega_i=\sum_{j=0}^{i}b_{k+1-i} \tag*{(41)}

where bib_i are the weights defining the k+1k+1 order Adams–Moulton method [82]. These weights are then used for T>hkT > hk for timestepping. The details of this procedure are described in Ref. [17].

Superconducting self-energy evaluation

Here we provide an example self-energy and hybridization evaluation class for the DMFT superconducting problem of § 5. The Fock diagram evaluation (Eq. (17)) requires the density matrix, obtained from the Green’s function. In general the interaction UU may vary in time, and the implementation supports this case, although UU is constant in the example of § 5. Because UU is a scalar and the function class stores matrix-valued functions, we access U(t)U(t) as U(tstp,0,0)U(\mathrm{tstp},0,0).

void SC_gf2::solve_Sigma_Fock(int tstp, function &U, herm_matrix_hodlr &G, function &H) {
    ZMatrix rho(2,2);
    G.density_matrix(tstp, dlr_, rho);

    for(int i = 0; i < 2; i++) {
        H(tstp,i,1-i) = -U(tstp,0,0) * rho(i,1-i);
    }
}

Next, we implement the second-order diagrams of Eq. (18). When tstp==-1, only the Matsubara component is obtained; otherwise the retarded, lesser, and mixed components are computed.

void SC_gf2::solve_Sigma(int tstp, function &U, herm_matrix_hodlr &G, herm_matrix_hodlr &Sigma) {
    if(tstp == -1) { Sigma_mat(U, G, Sigma); }
    else { Sigma_tstp(tstp, U, G, Sigma); }
}

void SC_gf2::Sigma_mat(function &U, herm_matrix_hodlr &G, herm_matrix_hodlr &Sigma) {
    DMatrix GM_reversed(G.ntau(), 4);
    G.get_mat_reversed(dlr_, GM_reversed);

    for(int tau = 0; tau < G.ntau(); tau++) {
        auto SM_map = Sigma.map_mat(tau);
        auto GM_map = G.map_mat(tau);
        for(int i = 0; i < 2; i++) {
            for(int j = 0; j < 2; j++) {
                // For reversed we index (i,j) using rowmajor ordering
                SM_map(i,j) += (U(-1,0,0) * U(-1,0,0) * GM_map(i,j) * GM_map(1-i,1-j) * GM_reversed(tau, (1-j)*2 + (1-i))).real();
                SM_map(i,j) -= (U(-1,0,0) * U(-1,0,0) * GM_map(i,1-j) * GM_map(1-i,j) * GM_reversed(tau, (1-j)*2 + (1-i))).real();
            }
        }
    }
}

void SC_gf2::Sigma_tstp(int tstp, function &U, herm_matrix_hodlr &G, herm_matrix_hodlr &Sigma) {
    for(int t = 0; t <= tstp; t++) {
        auto SR = Sigma.map_ret_curr(tstp, t);
        auto GR = G.map_ret_curr(tstp, t);
        auto SL = Sigma.map_les_curr(t, tstp);
        auto GL = G.map_les_curr(t, tstp);
        auto GG = GR - GL.adjoint();

        for(int i = 0; i < 2; i++) {
            for(int j = 0; j < 2; j++) {
                SL(i,j) += U(tstp,0,0) * U(t,0,0) * GL(i,j) * GL(1-i,1-j) * GG(1-j,1-i);
                SL(i,j) -= U(tstp,0,0) * U(t,0,0) * GL(i,1-j) * GG(1-j,1-i) * GL(1-i,j);
                // R(t,t’) = >(t,t’) - <(t,t’) = >(t,t’) + <(t’,t)^\dagger
                SR(i,j) += U(tstp,0,0) * U(t,0,0) * ( GG(i,j) * GG(1-i,1-j) * GL(1-j,1-i)
                    + std::conj( GL(j,i) * GL(1-j,1-i) * GG(1-i,1-j) ) );
                SR(i,j) -= U(tstp,0,0) * U(t,0,0) * ( GG(i,1-j) * GL(1-j,1-i) * GG(1-i,j)
                    + std::conj( GL(1-j,i) * GG(1-i,1-j) * GL(j,1-i) ) );
            }
        }
    }

    ZMatrix GVT(G.ntau(), 4);
    G.get_vt(tstp, dlr_, GVT);
    for(int tau = 0; tau < G.ntau(); tau++) {
        auto GVT_map = GVT.map(tstp, tau);
        auto STV_map = Sigma.map_tv(tstp, tau);
        for(int i = 0; i < 2; i++) {
            for(int j = 0; j < 2; j++) {
                // For reversed we index (i,j) using rowmajor ordering
                STV_map(i,j) += U(tstp,0,0) * U(-1,0,0) * GTV_map(i,j) * GTV_map(1-i,1-j) * GVT(tau,(1-j)*2 + (1-i));
                STV_map(i,j) -= U(tstp,0,0) * U(-1,0,0) * GTV_map(i,1-j) * GVT(tau,(1-j)*2 + (1-i)) * GTV_map(1-i,j);
            }
        }
    }
}

}

Finally, we implement the hybridization function of Eq. (14). sigma3_ denotes the third Pauli matrix.

void SC_gf2::solve_Delta(int tstp, function &t0, herm_matrix_hodlr &G, herm_matrix_hodlr &Sigma) {
  if(tstp == -1) { Delta_mat(t0, G, Sigma); }
  else { Delta_tstp(tstp, t0, G, Sigma); }
}

void SC_gf2::Delta_mat(function &t0, herm_matrix_hodlr &G, herm_matrix_hodlr &Sigma) {
  for(int tau = 0; tau < G.ntau(); tau++) {
    Sigma.map_mat(tau).noalias() += (t0.get_map(-1) * sigma3_ * G.map_mat(tau) * sigma3_ * t0.get_map
      (-1).conjugate()).real();
  }
}

void SC_gf2::Delta_tstp(int tstp, function &t0, herm_matrix_hodlr &G, herm_matrix_hodlr &Sigma) {
  ZMatrix sGs = ZMatrix::Zero(2,2);
  for(int t = 0; t <= tstp; t++) {
    sGs.noalias() = sigma3_ * G.map_ret_curr(tstp, t) * sigma3_;
    Sigma.map_ret_curr(tstp, t).noalias() += 0.5 * t0.get_map(tstp) * sGs * t0.get_map(t).conjugate();
    Sigma.map_ret_curr(tstp, t).noalias() += 0.5 * t0.get_map(tstp).conjugate() * sGs * t0.get_map(t);

    sGs.noalias() = sigma3_ * G.map_les_curr(t, tstp) * sigma3_;
    Sigma.map_les_curr(t, tstp).noalias() += 0.5 * t0.get_map(t) * sGs * t0.get_map(tstp).conjugate();
    Sigma.map_les_curr(t, tstp).noalias() += 0.5 * t0.get_map(t).conjugate() * sGs * t0.get_map(tstp);
  }

  for(int tau = 0; tau < G.ntau(); tau++) {
    sGs.noalias() = sigma3_ * G.map_tv(tstp, tau) * sigma3_;
    Sigma.map_tv(tstp, tau).noalias() += 0.5 * t0.get_map(tstp) * sGs * t0.get_map(-1).conjugate();
    Sigma.map_tv(tstp, tau).noalias() += 0.5 * t0.get_map(tstp).conjugate() * sGs * t0.get_map(-1);
  }
}

Low-level mpi_comm interface

By default, mpi_comm handles loading, unloading, and communication of Green’s functions and self-energies automatically. In cases where only a subset of orbital or Keldysh components needs to be communicated, a low-level interface allows users to implement custom protocols, as described below.

k_rank_mapVector of size NkN_k that maps a global k index to the MPI rank responsible for that k-point.
my_global_k_listVector of size NklocN_k^{\mathrm{loc}} that maps local k-point indices at each MPI rank to the global k-point indices.
Nk_per_rankVector of size NmpiN_{\mathrm{mpi}} that contains the number of k points handled by each MPI rank
init_tau_per_rankVector of size NmpiN_{\mathrm{mpi}} that contains the initial τ\tau index at each rank.
Ntau_per_rankVector of size NmpiN_{\mathrm{mpi}} that contains the number of τ\tau points handled by each MPI rank

Table C.10. Vectors inside mpi_comm describing the workload distribution among MPI ranks.

The workload distribution and communication details are managed by the mpi_comm class, whose main components are summarized in Tables C.10–C.12. At the start of the program, each MPI rank initializes an instance with

mpi_comm comm(global_Nk, Nt, r, nao, max_component_size);

where Nk = NkN_k is the total number of k-points, Nt = NtN_t is the total number of time points, r is the number of DLR imaginary-time nodes used by the dyson class, nao is the number of orbitals, and max_component_size is the maximum number of Keldysh and/or orbital components sent per communication (default 2). During initialization, the communicator assigns blocks of k-points and τ\tau points to each rank (Table C.10), whereas real-time t′t' points are distributed on the fly at each timestep tt, since the number of τ\tau points is fixed but the number of t′t' points grows with each timestep. Several buffers are also allocated for communication and self-energy evaluation (Table C.11). In the self-consistent loop, the Green’s functions are extracted from the herm_matrix_hodlr objects into alltp_buff and communicated to allk_buff in the downward direction shown in Figure 5; both steps are handled by mpi_get_and_comm. The self-energy is then evaluated inside allk_buff and communicated back to alltp_buff in the upward direction (Figure 5), after which the results are written into the herm_matrix_hodlr objects; these steps are handled by mpi_comm_and_set. Because MPI_Alltoallv requires one-dimensional arrays, the indexing into the communication buffers is nontrivial; the helper functions in Table C.12 perform the necessary index calculations.

tau_alltp_buffBuffer that contains the data at all τ\tau points and a set of k-points local to the MPI rank.
t_alltp_buffBuffer that contains the data at all t′t' points and a set of k-points local to the MPI rank.
tau_allk_buffBuffer that contains the data at all k-points and a set of τ\tau points local to the MPI rank.
t_allk_buffBuffer that contains the data at all k-points and a set of t′t' points local to the MPI rank.

Table C.11. Intermediate communication buffers inside mpi_comm.

alltp_buff_indexComputes the index in the alltp_buff buffers.
t_allk_buff_indexComputes the index in the t_allk_buff buffer.
tau_allk_buff_indexComputes the index in the tau_allk_buff buffer.

Table C.12. Helper indexing functions inside mpi_comm.

The MPI communicator is agnostic to the internal layout of the Green’s functions and self-energies. The user therefore supplies lambda expressions specifying how to read and write each component; these lambdas are collected into vectors and passed to the communicator methods, which iterate over them to perform the extraction or writing. The user may define any number of lambdas to access any Keldysh component or orbital of interest. The only constraint is that max_component_size must be set to at least the largest number of components sent in a single communication. Therefore, if the user wants to send nn Keldysh components of the Green’s function in one communication, max_component_size must be at least nn. If fewer components are sent in a subsequent call, the existing buffer space suffices; if more are sent than max_component_size, an error is thrown.

For example, computing the lesser self-energy from Eq. (30) requires both G>G^{>} and G<G^{<}. The corresponding get lambdas are

//returns G^<>(t,t’) at local k index and real times t,t’
auto getLess = [&corrK_rank, &size]
(int local_ki, int ti, int tpi)
{
    hodlr::ZMatrix tmp(size,size);
    corrK_rank[local_ki]->G_.get_les(ti,tpi,tmp);
    return tmp(0,0);
};

//returns G^>(t,t’) at local k index and real times t,t’
auto getGreat = [&corrK_rank, &size](int local_ki, int ti, int tpi){
    hodlr::ZMatrix tmpRet(size,size),tmpLess(size,size),tmpGreat(size,size);

    corrK_rank[local_ki]->G_.get_les(ti,tpi,tmpLess);
    corrK_rank[local_ki]->G_.get_ret(ti,tpi,tmpRet);
    tmpGreat(0,0) = tmpRet(0,0)+tmpLess(0,0);

    return tmpGreat(0,0);
};
// vector of lambdas for Lesser and Greater components
std::vector<std::function<std::complex<double>(int, int, int)>> getsLG = {getLess, getGreat};

After computing and communicating the self-energy, the communicator needs to write the results back into the self-energy objects. The following set lambdas write ΣkR(t,t′)\Sigma^{\mathrm{R}}_k(t,t') and Σk<(t,t′)\Sigma^{<}_k(t,t'):

// sets Sigma^<(t,t') at local k index and real time t for all t'<=t
auto setLess = [&corrK_rank, &size](int k, int ti, std::vector<std::complex<double>> &Sigma){
    std::vector<std::complex<double>> Sigmac(ti+1);
    for(int j = 0; j<=ti; j++){
        Sigmac[j] = -std::conj(Sigma[j]);
    }
    hodlr::ZMatrixMap(corrK_rank[k]->Sigma_.curr_timestep_les_ptr(0,ti), (ti + 1) * size, size).noalias
        () = hodlr::ZMatrixMap(Sigmac.data(), (ti + 1) * size, size);
};

// sets Sigma^R(t,t') at local k index and real time t for all t'>=t
auto setRet = [&corrK_rank, &size](int k, int ti, std::vector<std::complex<double>> &Sigma)
{
    hodlr::ZMatrixMap(corrK_rank[k]->Sigma_.curr_timestep_ret_ptr(ti,0), (ti + 1) * size, size).noalias
        () = hodlr::ZMatrixMap(Sigma.data(), (ti + 1) * size, size);
};

std::vector<std::function<void(int, int, std::vector<std::complex<double>>&)>> setsLR = {setLess,
    setRet};

The vectors getsLG and setsLR are passed to the communicator methods, which iterate over their elements and perform the extraction or writing in the appropriate communication direction (Figure 5). In this example, max_component_size must be at least 2, since two components of each quantity are communicated.

For general complex time indices tt and t′t' and Keldysh component index η\eta, the get and set lambdas must satisfy the following requirements:

  • A get lambda returns Gkη(t,t′)G_k^\eta(t,t') at a local k index, with signature std::complex<double>(int local_ki, int ti, int tpi).

  • A set lambda writes Σkη(t,t′)\Sigma_k^\eta(t,t') at a local k index for an array of t′≤tt' \leq t points, with signature void(int local_ki, int ti, std::vector<std::complex<double>> &Sigma_tp).

  • Both get and set lambdas must use the timestep tt as their first time index, including for the Matsubara case where t=−1t=-1.

References

  1. [1]Daniele Fausti, RI Tobey, Nicky Dean, Stefan Kaiser, A Dienst, Matthias C Hoffmann, S Pyon, T Takayama, H Takagi, and Andrea Cavalleri. Light-induced superconductivity in a stripe-ordered cuprate. Science, 331(6014):189–191, 2011.DOI
  2. [2]Matteo Mitrano, Alice Cantaluppi, Daniele Nicoletti, Stefan Kaiser, A Perucchi, S Lupi, P Di Pietro, D Pontiroli, M Riccò, Stephen R Clark, et al. Possible light-induced superconductivity in k 3 c 60 at high temperature. Nature, 530(7591):461–464, 2016.
  3. [3]Claudio Giannetti, Massimo Capone, Daniele Fausti, Michele Fabrizio, Fulvio Parmigiani, and Dragan Mihailovic. Ultrafast optical spectroscopy of strongly correlated materials and high-temperature superconductors: a non-equilibrium approach. Adv. Phys., 65(2):58–238, 2016.
  4. [4]Alberto de la Torre, Dante M. Kennes, Martin Claassen, Simon Gerber, James W. McIver, and Michael A. Sentef. Colloquium: Nonthermal pathways to ultrafast control in quantum materials. Rev. Mod. Phys., 93:041002, Oct 2021.DOI
  5. [5]Faran Zhou, Joseph Williams, Christos D Malliakas, Mercouri G Kanatzidis, Alexander F Kemper, and Chong-Yu Ruan. Nonequilibrium dynamics of spontaneous symmetry breaking into a hidden state of charge-density wave. arXiv preprint arXiv:1904.07120, 2019.arxiv.org/abs/1904.07120
  6. [6]Yuta Murakami, Denis Golež, Martin Eckstein, and Philipp Werner. Photoinduced nonequilibrium states in mott insulators. Rev. Mod. Phys., 97:035001, Jul 2025.DOI
  7. [7]DN Basov, RD Averitt, and D Hsieh. Towards properties on demand in quantum materials. Nat. Mater., 16(11):1077–1088, 2017.DOI
  8. [8]Manuel Ligges, Isabella Avigo, Denis Golež, HUR Strand, Y Beyazit, K Hanff, F Diekmann, L Stojchevska, M Kalläne, P Zhou, et al. Ultrafast doublon dynamics in photoexcited 1 t-tas 2. Phys. Rev. Lett., 120(16):166401, 2018.DOI
  9. [9]Miguel AL Marques, Neepa T Maitra, Fernando MS Nogueira, Eberhard KU Gross, and Angel Rubio. Fundamentals of time-dependent density functional theory. Springer, 2012.
  10. [10]Ulrich Schollwöck. The density-matrix renormalization group in the age of matrix product states. Ann. Phys., 326(1):96–192, 2011.DOI
  11. [11]Hartmut Haug and Antti-Pekka Jauho. Quantum kinetics in transport and optics of semiconductors. Springer, 2 edition, 2008.
  12. [12]Philipp Werner, Takashi Oka, Martin Eckstein, and Andrew J. Millis. Weak-coupling quantum monte carlo calculations on the keldysh contour: Theory and application to the current-voltage characteristics of the anderson model. Phys. Rev. B, 81:035108, Jan 2010.DOI
  13. [13]Guy Cohen, Emanuel Gull, David R. Reichman, and Andrew J. Millis. Taming the dynamical sign problem in real-time evolution of quantum many-body problems. Phys. Rev. Lett., 115:266802, Dec 2015.DOI
  14. [14]Emanuel Gull, David R Reichman, and Andrew J. Millis. Numerically exact long-time behavior of nonequilibrium quantum impurity models. Phys. Rev. B, 84:085134, Aug 2011.DOI
  15. [15]Leo P Kadanoff and Gordon A Baym. Quantum Statistical Mechanics. W.A. Benjamin, 1962.
  16. [16]Gianluca Stefanucci and Robert Van Leeuwen. Nonequilibrium many-body theory of quantum systems: a modern introduction. Cambridge University Press, 2013.
  17. [17]Michael Schü̈ler, Denis Golež, Yuta Murakami, Nikolaj Bittner, Andreas Herrmann, Hugo UR Strand, Philipp Werner, and Martin Eckstein. Nessi: The non-equilibrium systems simulation package. Comput. Phys. Commun., 257:107484, 2020.DOI
  18. [18]Hideo Aoki, Naoto Tsuji, Martin Eckstein, Marcus Kollar, Takashi Oka, and Philipp Werner. Nonequilibrium dynamical mean-field theory and its applications. Rev. Mod. Phys., 86(2):779, 2014.DOI
  19. [19]Michael Sentef, Alexander F. Kemper, Brian Moritz, James K. Freericks, Zhi-Xun Shen, and Thomas P. Devereaux. Examining electron-boson coupling using time-resolved spectroscopy. Phys. Rev. X, 3:041033, Dec 2013.DOI
  20. [20]J. K. Freericks, V. M. Turkowski, and V. Zlatić. Nonequilibrium dynamical mean-field theory. Phys. Rev. Lett., 97:266408, Dec 2006.DOI
  21. [21]Mark Jarrell and J.E. Gubernatis. Bayesian inference and the analytic continuation of imaginary-time quantum monte carlo data. Physics Reports, 269(3):133–195, May 1996.
  22. [22]Anders W. Sandvik. Stochastic method for analytic continuation of quantum monte carlo data. Phys. Rev. B, 57:10287–10290, May 1998.DOI
  23. [23]Kazuyoshi Yoshimi, Junya Otsuki, Yuichi Motoyama, Masayuki Ohzeki, and Hiroshi Shinaoka. Spm: Sparse modeling tool for analytic continuation of imaginary-time green’s function. Computer Physics Communications, 244:319–323, November 2019.DOI
  24. [24]Romain Fournier, Lei Wang, Oleg V. Yazyev, and QuanSheng Wu. Artificial neural network approach to the analytic continuation problem. Phys. Rev. Lett., 124:056401, Feb 2020.DOI
  25. [25]Hongkee Yoon, Jae-Hoon Sim, and Myung Joon Han. Analytic continuation via domain knowledge free machine learning. Phys. Rev. B, 98:245101, Dec 2018.DOI
  26. [26]Lexing Ying. Analytic continuation from limited noisy matsubara data. Journal of Computational Physics, 469:111549, November 2022.DOI
  27. [27]Edwin W. Huang, Ryan Sheppard, Brian Moritz, and Thomas P. Devereaux. Strange metallicity in the doped hubbard model. Science, 366(6468):987–990, November 2019.DOI
  28. [28]J. Vučicević, J. Kokalj, R. Žitko, N. Wentzell, D. Tanasković, and J. Mravlje. Conductivity in the square lattice hubbard model at high temperatures: Importance of vertex corrections. Phys. Rev. Lett., 123:036601, Jul 2019.DOI
  29. [29]Lei Zhang, Yang Yu, and Emanuel Gull. Minimal pole representation and analytic continuation of matrix-valued correlation functions. Phys. Rev. B, 110:235131, Dec 2024.DOI
  30. [30]A. Erpenbeck, E. Gull, and G. Cohen. Quantum monte carlo method in the steady state. Physical Review Letters, 130(18):186301, May 2023.DOI
  31. [31]Xinyang Dong, Emanuel Gull, and Hugo U. R. Strand. Excitations and spectra from equilibrium real-time green’s functions. Physical Review B, 106(12):125153, 2022.DOI
  32. [32]Jeremija Kovačević, Michel Ferrero, and Jak ša Vučicević. Toward numerically exact computation of conductivity in the thermodynamic limit of interacting lattice models. Phys. Rev. Lett., 135:016502, Jul 2025.DOI
  33. [33]E Perfetto, D Sangalli, A Marini, and G Stefanucci. Ultrafast charge migration in xuv photoexcited phenylalanine: A first-principles study based on real-time nonequilibrium green’s functions. J. Phys. Chem. Lett., 9(6):1353–1358, 2018.arxiv.org/abs/1803.05681
  34. [34]Alejandro Molina-Sánchez, Davide Sangalli, Ludger Wirtz, and Andrea Marini. Ab initio calculations of ultrashort carrier dynamics in two-dimensional materials: valley depolarization in single-layer wse2. Nano Lett., 17(8):4549–4555, 2017.
  35. [35]D Sangalli, A Ferretti, H Miranda, Claudio Attaccalite, I Marri, Elena Cannuccia, P Melo, M Marsili, F Paleari, A Marrazzo, et al. Many-body perturbation theory calculations using the yambo code. J. Phys. Condens. Matter, 31(32):325902, 2019.DOI
  36. [36]Andrea Marini, Conor Hogan, Myrta Grüning, and Daniele Varsano. Yambo: an ab initio tool for excited state calculations. Comput. Phys. Commun., 180(8):1392–1403, 2009.DOI
  37. [37]Enrico Perfetto and Gianluca Stefanucci. Cheers: a tool for correlated hole-electron evolution from real-time simulations. J. Phys. Condens. Matter, 30(46):465901, 2018.DOI
  38. [38]Karsten Balzer and Michael Bonitz. Nonequilibrium Green’s Functions Approach to Inhomogeneous Systems. Springer, 2012.
  39. [39]Karsten Balzer. Solving the two-time Kadanoff-Baym equations: Application to model atoms and molecules. PhD thesis, 2011.
  40. [40]A. Erpenbeck, T. Blommel, L. Zhang, W.-T. Lin, G. Cohen, and E. Gull. Steady-state properties of multi-orbital systems using quantum monte carlo. The Journal of Chemical Physics, 161(9):094104, 2024.arxiv.org/abs/2407.00771
  41. [41]P Lipavský, V Špička, and B Velický. Generalized kadanoff-baym ansatz for deriving quantum transport equations. Phys. Rev. B, 34(10):6933, 1986.pubmed.ncbi.nlm.nih.gov/9939345
  42. [42]Anděla Kalvová, Bedřich Velický, and Václav Špička. Beyond the generalized kadanoff-baym ansatz. Phys. Status Solidi B, 256(7):1800594, 2019.DOI
  43. [43]Niclas Schlünzen, Jan-Philip Joost, and Michael Bonitz. Achieving the scaling limit for nonequilibrium green functions simulations. Phys. Rev. Lett., 124:076601, Feb 2020.DOI
  44. [44]Jan-Philip Joost, Niclas Schlünzen, and Michael Bonitz. G1-g2 scheme: Dramatic acceleration of nonequilibrium green functions simulations within the hartree-fock generalized kadanoff-baym ansatz. Phys. Rev. B, 101:245101, Jun 2020.DOI
  45. [45]Riku Tuovinen, Denis Golež, Martin Eckstein, and Michael A. Sentef. Comparing the generalized kadanoff-baym ansatz with the full kadanoff-baym equations for an excitonic insulator out of equilibrium. Phys. Rev. B, 102(11):115157, 2020.DOI
  46. [46]Cian C. Reeves, Yuanran Zhu, Chao Yang, and Vojtěch Vlček. Unimportance of memory for the time nonlocal components of the kadanoff-baym equations. Phys. Rev. B, 108:115152, Sep 2023.DOI
  47. [47]Daniel Karlsson, Robert van Leeuwen, Yaroslav Pavlyukh, Enrico Perfetto, and Gianluca Stefanucci. Fast green’s function method for ultrafast electron-boson dynamics. Phys. Rev. Lett., 127:036402, Jul 2021.DOI
  48. [48]Gianluca Stefanucci and Enrico Perfetto. Semiconductor electron-phonon equations: a rung above boltzmann in the many-body ladder, 2023.arxiv.org/abs/2311.03980
  49. [49]Jia Yin, Yang hao Chan, Felipe H. da Jornada, Diana Y. Qiu, Steven G. Louie, and Chao Yang. Using dynamic mode decomposition to predict the dynamics of a two-time non-equilibrium Green’s function. J. Comput. Sci., 64:101843, 2022.arxiv.org/abs/2203.14892
  50. [50]Jia Yin, Yang hao Chan, Felipe H. da Jornada, Diana Y. Qiu, Chao Yang, and Steven G. Louie. Analyzing and predicting non-equilibrium many-body dynamics via dynamic mode decomposition. J. Comput. Phys., 477:111909, 2023.arxiv.org/abs/2107.09635
  51. [51]Cian C. Reeves, Jia Yin, Yuanran Zhu, Khaled Z. Ibrahim, Chao Yang, and Vojtěch Vlček. Dynamic mode decomposition for extrapolating nonequilibrium Green’s-function dynamics. Phys. Rev. B, 107:075107, Feb 2023.DOI
  52. [52]Hardeep Bassi, Yuanran Zhu, Senwei Liang, Jia Yin, Cian C Reeves, Vojtěch Vlček, and Chao Yang. Learning nonlinear integral operators via recurrent neural networks and its application in solving integro-differential equations. Machine Learning with Applications, 15:100524, 2024.arxiv.org/abs/2310.09434
  53. [53]Yuanran Zhu, Jia Yin, Cian C. Reeves, Chao Yang, and Vojtech Vlcek. Predicting nonequilibrium Green’s function dynamics and photoemission spectra via nonlinear integral operator learning. 2024.
  54. [54]Andre Erpenbeck, Yuanran Zhu, Yang Yu, Lei Zhang, Richard Gerum, Olga Goulko, Chao Yang, Guy Cohen, and Emanuel Gull. Compact representation and long-time extrapolation of real-time data for quantum systems. (arXiv:2506.13760), 2025. arXiv:2506.13760 [cond-mat].DOI
  55. [55]Guy Cohen and Eran Rabani. Memory effects in nonequilibrium quantum impurity models. Physical Review B, 84(7):075150, August 2011.DOI
  56. [56]Guy Cohen, Emanuel Gull, David R. Reichman, Andrew J. Millis, and Eran Rabani. Numerically exact long-time magnetization dynamics at the nonequilibrium kondo crossover of the anderson impurity model. Physical Review B, 87(19):195108, May 2013.DOI
  57. [57]Michael Schüler, Martin Eckstein, and Philipp Werner. Truncating the memory time in nonequilibrium dynamical mean field theory calculations. Phys. Rev. B, 97:245129, Jun 2018.DOI
  58. [58]Christopher Stahl, Nagamalleswararao Dasari, Jiajun Li, Antonio Picano, Philipp Werner, and Martin Eckstein. Memory truncated kadanoff-baym equations. Phys. Rev. B, 105:115146, Mar 2022.DOI
  59. [59]Antonio Picano and Martin Eckstein. Accelerated gap collapse in a slater antiferromagnet. Phys. Rev. B, 103:165118, Apr 2021.DOI
  60. [60]Nagamalleswararao Dasari, Jiajun Li, Philipp Werner, and Martin Eckstein. Photoinduced strange metal with electron and hole quasiparticles. Phys. Rev. B, 103:L201116, May 2021.DOI
  61. [61]Hiroshi Shinaoka, Markus Wallerberger, Yuta Murakami, Kosuke Nogaki, Rihito Sakurai, Philipp Werner, and Anna Kauch. Multiscale space-time ansatz for correlation functions of quantum systems based on quantics tensor trains. Phys. Rev. X, 13:021015, Apr 2023.DOI
  62. [62]Matthias Murray, Hiroshi Shinaoka, and Philipp Werner. Nonequilibrium diagrammatic many-body simulations with quantics tensor trains. Phys. Rev. B, 109:165135, Apr 2024.DOI
  63. [63]Maksymilián Šroda, Ken Inayoshi, Hiroshi Shinaoka, and Philipp Werner. High-resolution nonequilibrium gw calculations based on quantics tensor trains, 2024.
  64. [64]Ken Inayoshi, Maksymilián Šroda, Anna Kauch, Philipp Werner, and Hiroshi Shinaoka. A causality-based divide-and-conquer algorithm for nonequilibrium Green’s function calculations with quantics tensor trains, September 2025. arXiv:2509.15028 [cond-mat].arxiv.org/abs/2509.15028
  65. [65]Maksymilián Šroda, Ken Inayoshi, Michael Schüler, Hiroshi Shinaoka, and Philipp Werner. Predictor-corrector method based on dynamic mode decomposition for tensor-train nonequilibrium Green’s function calculations, September 2025. arXiv:2509.22177 [cond-mat].arxiv.org/abs/2509.22177
  66. [66]Jason Kaye and Denis Golež. Low rank compression in the numerical solution of the nonequilibrium Dyson equation. SciPost Phys., 10:091, 2021.DOI
  67. [67]Thomas Blommel, Jason Kaye, Yuta Murakami, Emanuel Gull, and Denis Golež. Chirped amplitude mode in photoexcited superconductors. Phys. Rev. B, 111:094502, Mar 2025.DOI
  68. [68]Baptiste Lamic. Solving the transient dyson equation with quasilinear complexity via matrix compression. arXiv preprint arXiv:2410.11057, 2024.DOI
  69. [69]Jože Gašperlin, Denis Golež, and Jason Kaye. Stability and complexity of global iterative solvers for the Kadanoff-Baym equations, 2025. arXiv:2512.11371 [cond-mat].DOI
  70. [70]Jonas Ballani and Daniel Kressner. Matrices with Hierarchical Low-Rank Structures, pages 161–209. Springer, 2016.DOI
  71. [71]Antoine Georges, Gabriel Kotliar, Werner Krauth, and Marcelo J Rozenberg. Dynamical mean-field theory of strongly correlated fermion systems and the limit of infinite dimensions. Rev. Mod. Phys., 68(1):13, 1996.DOI
  72. [72]Thomas Blommel, David J. Gardner, Carol S. Woodward, and Emanuel Gull. Adaptive time stepping for the two-time integro-differential kadanoff-baym equations. Phys. Rev. B, 110:205134, Nov 2024.DOI
  73. [73]O. V. Konstantinov and V. I. Perel’. A diagram technique for evaluating transport quantities. Sov. Phys. JETP, 12(1):142–149, January 1961.
  74. [74]Qiaoyuan Dong, Igor Krivenko, Joseph Kleinhenz, Andrey E. Antipov, Guy Cohen, and Emanuel Gull. Quantum monte carlo solution of the dynamical mean field equations in real time. Physical Review B, 96(15):155126, October 2017.
  75. [75]Antonio Picano, Giulio Biroli, and Marco Schirò. Quantum thermalization via travelling waves. Phys. Rev. Lett., 134:116503, Mar 2025.DOI
  76. [76]Thomas Blommel. Numerical Integration of the Kadanoff-Baym Equations. PhD thesis, University of Michigan, 2024.DOI
  77. [77]Jason Kaye, Kun Chen, and Olivier Parcollet. Discrete Lehmann representation of imaginary time Green’s functions. Phys. Rev. B, 105:235115, Jun 2022.DOI
  78. [78]Jason Kaye, Kun Chen, and Hugo U.R. Strand. libldr: Efficient imaginary time calculations using the discrete Lehmann representation. Comput. Phys. Commun., 280:108458, 2022.DOI
  79. [79]Jiajun Li, Denis Golež, Philipp Werner, and Martin Eckstein. Long-range η-pairing in photodoped mott insulators. arXiv preprint arXiv:1908.08693, 2019.
  80. [80]Jason Kaye and Hugo U. R. Strand. A fast time domain solver for the equilibrium Dyson equation. Adv. Comput. Math., 49(4), 2023.DOI
  81. [81]Matteo Frigo and Steven G. Johnson. The design and implementation of FFTW3. Proceedings of the IEEE, 93(2):216–231, 2005.DOI
  82. [82]P. H. M. WOLKENFELT. The construction of reducible quadrature rules for volterra integral and integro-differential equations. IMA Journal of Numerical Analysis, 2(2):131–152, 04 1982.DOI

Paper details

vlnc:W_4wvn9bpt
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents