WorksPhysics and Astronomy
LEDDS: Portable LBM-DEM simulations on GPUs
Abstract
Algorithmic formulations of GPU programs provide a high-level alternative to device-specific code by expressing computations as compositions of well-defined parallel primitives (e.g., map, sort, reduce), rather than through handcrafted GPU kernels. In this work, we demonstrate that this paradigm can be extended to complex and challenging problems in computational physics: the simulation of granular flows and fluid-particle interactions. LEDDS, our open-source framework, performs fully coupled Lattice Boltzmann – Discrete Element Method (LBM-DEM) simulations using only algorithmic primitives, and runs efficiently on single-GPU platforms. The entire workflow, including neighbor search, collision detection, and fluid-particle coupling, is expressed as a sequence of portable primitives. While the current implementation illustrates these principles primarily through algorithms from the C++ Standard Library, with selective use of Thrust primitives for performance, the underlying concept is compatible with any HPC environment offering a rich set of parallel algorithms and is therefore applicable across a wide range of modern GPU systems and future accelerators. LEDDS is validated through benchmarks spanning both DEM and LBM-DEM configurations, including sphere and ellipsoid collisions, wall friction tests, single-particle settling, Jeffery's orbits, and particle-laden shear flows. Despite its high level of abstraction, LEDDS achieves performances comparable to those of hand-tuned CUDA solvers, while maintaining portability and code clarity. These results show that high-performance LBM-DEM coupling can be achieved without sacrificing generality or readability, establishing LEDDS as a blueprint for portable multiphysics frameworks based on algorithmic primitives.
PROGRAM SUMMARY
Program Title: LBM-Enhanced Device-independent DEM Solver (LEDDS)
Developer’s repository link: https://gitlab.com/unigehpfs/ledds
Licensing provisions: MIT
Programming language: C++
Nature of problem: Fully resolved particle-laden flows involving granular materials and fluid–particle interactions. Achieving particle-scale accuracy in contacts and fluid–solid interactions is computationally demanding, and it requires a careful design and implementation strategy to simulate such flows on limited resources.
Solution method: In LEDDS, the fluid flow evolution is modelled using the Lattice Boltzmann Method (LBM), while particle motion and contacts are handled by a Discrete Element Method (DEM). Fluid–particle coupling is achieved using a diffuse-interface formulation: the partially saturated method. Simulations are performed on uniform Cartesian grids, and support both spherical and ellipsoidal particles. Parallelization relies on algorithmic primitives implemented in modern C++, and selectively in Thrust, which enables portable high-performance execution on multi-core CPUs and single GPUs.
Additional comments including restrictions and unusual features: The LBM and DEM modules can be used independently or fully coupled through a consistent data structure. LEDDS is currently limited to single-phase, isothermal flows and to spherical or ellipsoidal particles. MPI-based or other multi-GPU and multi-node CPU parallelism are not supported in this version. The code requires nvcc+ version 24.9 or later; backward compatibility is not guaranteed, but upward compatibility is ensured.
Introduction
The Discrete Element Method (DEM), originally proposed by Cundall and Strack [1], is a versatile tool for predicting the behavior of particle-based systems. Specifically, in DEM, individual particles are modeled as discrete entities with their own physical and mechanical properties, such as size, shape, mass, Young’s modulus, and Poisson ratio. Particle motion, both translational and rotational, arises from interactions with other particles or with the environment [1]. Interparticle forces, including cohesive, bridging, and contact forces, are described using models such as spring-dashpot, friction, and collision models. The system evolves over time by integrating Newton’s second law of motion while accounting for external effects, including gravity and interstitial fluids.
Since its advent, DEM has found wide application in studying granular materials, powders, soil mechanics, and particle-fluid interactions [2, 3]. In the latter case, DEM is coupled with a computational fluid dynamics (CFD) solver [4, 5, 6, 7], forming a CFD-DEM framework. These simulations are computationally demanding, which has motivated the use of general-purpose GPU acceleration. Because GPUs excel at data-parallel tasks, GPU implementations can significantly speed up operations like particle tracking, collision detection, and fluid flow calculations. As a result, GPU-accelerated DEM and CFD-DEM simulations can significantly reduce runtime compared with modern multi-core CPU implementations, with reported GPU speedups often reaching one to two orders of magnitude depending on the problem and hardware [8, 9, 10].
Different coding strategies can be used to efficiently run CFD-DEM simulations on GPUs. These include hardware-specific low-level languages (CUDA [11], HIP [12]), directive-based programming models (OpenACC [13], OpenMP [14]), and dedicated libraries or frameworks (OpenCL [15], SYCL [16]). In the present work, we rely on the availability of a set of well-defined algorithmic primitives in many of the high-level framework, allowing the design of platform-agnostic approach through the decomposition of complex problems into such primitives. First formalized by Blelloch [17] for vector and PRAM models, a parallel paradigm driven by a decomposition into primitives such as map, scan, and reduce has gained renewed attention with massively parallel architectures, owing to the possibility to express complex kernels as compositions of a few generic building blocks [18, 19, 20, 21]. Similar ideas have emerged in sparse linear algebra, where graph and matrix operations are unified under common algebraic primitives [22]. This decomposition-based approach produces self-documenting, portable programs across different hardware, and reduces concurrency-related errors by avoiding manual synchronization. Notably, several mature frameworks provide efficient implementations of algorithmic primitives for heterogeneous platforms, including Thrust [23], Kokkos [24], and AMD’s HIPSTDPAR/rocThrust [25]. Modern C++ standards (C++17/C++20) also offer portable parallel algorithms [26, 27], which we adopt as the primary framework, falling back to specific algorithms of the Thrust library selectively when limitations of the C++ standard impacts the performance distinctly. This combination allows the code to run efficiently across different hardware with minimal changes, typically requiring only a compiler or flag adjustment.
The authors have previously developed algorithm-based implementations of fluid solvers using the Lattice Boltzmann Method (LBM). In an earlier work, the fundamental operations of the LBM algorithm were reformulated entirely in terms of algorithmic primitives such as map and reduce [28]. This approach was subsequently extended to multi-GPU systems, where a large-scale LBM framework was ported using standard parallel algorithms combined with domain decomposition [29], as well as to LBM simulations on non-uniform static or dynamic grids [30, 31, 32], and even high-order finite-difference schemes [33]. It has also been extended to multi-physics configurations, including conjugate heat transfer in porous media, where two LBM solvers are coupled to resolve both the fluid flow and heat conduction within the solid phase [34]. In the present work, we go one step further by addressing the complex case of LBM-DEM coupling within the same algorithmic paradigm, while restricting ourselves to uniform grids only.
In particle-resolved simulations, coupling a particulate phase to a fluid solver requires accurately representing the fluid-solid interface without resorting to costly remeshing. Two popular diffuse-interface strategies in the context of LBM-DEM coupling are (i) immersed boundary methods (IBM), which enforce the no-slip condition through localized forcing terms that spread particle surface forces over nearby lattice nodes, and (ii) partially saturated cell (PSC/PSM) approaches, where each lattice cell stores a fractional solid occupancy used to modulate the collision operator. Both techniques avoid dynamic mesh generation and naturally fit the Eulerian structure of LBM. IBM-based LBM-DEM coupling has been widely explored, including applications to dense suspensions and rigid spherical particles [35, 36, 37], whereas PSM-like penalization methods have become a standard choice for large particulate systems due to their simplicity and robustness [38, 39]. In the present work, we adopt the PSM approach for its high accuracy and algorithmic compatibility with parallel primitives.
This coupling strategy introduces challenges arising not only from the individual solvers but, more importantly, from their interaction. Fluid-particle interactions in densely packed systems are particularly demanding, as the computation of forces exchanged between solid and fluid phases can dominate simulation time. We describe in detail how each component of the coupling algorithm—the identification of fluid-particle interface cells, the computation of per-cell forces, and the subsequent force reduction—is implemented as a sequence of parallel primitives, primarily map, sort, and prefix-sum operations.
Hereafter, we present the new open-source, algorithm-based framework LEDDS (LBM-Enhanced Device-independent DEM Solver), which integrates an LBM solver for the fluid phase, a DEM solver for the particle phase, and a coupling algorithm based on the Partially Saturated Method (PSM) which is a penalization-like approach [38, 40]. While the LBM part of LEDDS heavily relies on template codes provided in the STLBM framework, its DEM part is conceptually inspired by the open-source DEM framework LIGGGHTS [41]. This was done to make sure the DEM component of LEDDS is consistent with well-established numerical best practices in the field. However, LEDDS does not reuse LIGGGHTS code, and instead, its DEM module has been developed from scratch as a fully independent, algorithm-based module. To achieve this, we isolated each relevant feature of LIGGGHTS, reformulated it in terms of algorithmic primitives, and verified it against the original code. This ensured that all methodological details necessary for a complete DEM implementation were incorporated in LEDDS, e.g., tracking lateral displacements during collisions to compute tangential forces accurately.
On top of this, LEDDS stands out by the uniform implementation of its DEM and LBM solvers as both are developed using modern C++. Hence, they benefit from a common software design and are parallelized using C++ parallel algorithms. In case of GPU execution, both the DEM and the LBM run on the GPU and benefit from an efficient in-memory data exchange. To further emphasize the tight interaction between these two aspects of the code, the DEM solver relies on uniform Cartesian background mesh for the detection of particle collision pairs, which naturally relates to the Cartesian grid of the LBM solver (see Figure 1). Finally, after thorough performance evaluation, we replaced selected STL algorithm calls with their Thrust counterparts, for example in particle force computations. This decision was driven by the availability of specialized implementations in Thrust, such as radix sort, which outperformed the STL’s comparison-based sorting in our use case. In both approaches, whether using the C++ Standard Library or Thrust, the implementation relies on unified memory, eliminating the need for explicit data transfers or device-specific programming constructs.

Figure 1. 2D schematic of the coupled particle-fluid simulation. Bold lines indicate the DEM (cell-linked list) grid used for collision detection, thin lines show the LBM fluid grid, and semi-transparent circles show the particles. The relative 1:3 spacing between the grids is for illustration; in practice, the ratio depends on the largest particle size. The algorithm requires the grid spacing to be larger than the particle diameter to guarantee an overlap of a particle with at most 8 cells (in 3D).
Unlike large production frameworks, LEDDS is designed as a flexible platform to explore whether a complete LBM-DEM solver can be expressed using high-level algorithmic primitives while maintaining competitive performance on heterogeneous hardware. This is investigated in the remainder of the paper that is organized as follows. Section 2 reviews the theoretical background of DEM and LBM, including their coupling via the LB-specific penalization-like scheme (PSM). Section 3 presents the implementation strategy, highlighting how algorithmic primitives, primarily from the C++ Standard Library with selective use of Thrust, are used to build an efficient device-independent CFD-DEM solver. Section 4 validates the accuracy and robustness of LEDDS through particle-only and fluid-particle benchmark tests, followed by an in-depth performance analysis in Section 5. Finally, discussions and conclusions are provided in Section 6. Appendices summarize DEM-material parameter relations (Appendix A) and cover single-precision arithmetic (Appendix B), parameter sensitivity (Appendix § 9), and GPU memory usage (Appendix § 10).
Theoretical approach
Discrete Element Method
The Discrete Element Method (DEM) models the interactions between suspended particles inside the fluid. In LEDDS, particles are represented either as spheres or ellipsoids, and their evolution is based on contact mechanics. When particles collide, they interact through localized contact regions, where forces are exchanged. The motion of a particle thus only depends on two quantities, which are the forces and torques it experiences during contact with other particles:
There is an interaction between two particles and only if they are in contact/overlap. With spheres, this happens when the distance between the centers of the spheres is smaller than the sum of their radii, denoted as and :
For ellipsoids, the same collision procedure is applied by locally approximating the surface near the contact point with an equivalent sphere. The associated radius of curvature is computed from an approximation of the mean curvature:
with the radii of the ellipsoid in local frame. This approximation is preferred over the Gaussian curvature radius, as it avoids high-order operations and is therefore better suited for single-precision computations. Using this local spherical approximation, the contact detection consists in iteratively computing a pair of closest points (one on each ellipsoid) such that the line connecting them defines a common normal direction [42]. Starting from an initial guess on each surface, the points are updated by intersecting the ellipsoids with the line defined by the centers of the associated sphere approximations. The iterations proceed until convergence to a common normal configuration is achieved, subject to a prescribed tolerance and a maximum number of iterations.
Once the contact points and the associated normal direction are determined, the interaction force between two particles can be evaluated. The force exerted by a particle on a particle is decomposed into two components: a component normal to the contact surface and a component parallel to it, where, for brevity, the notation is used in place of . These forces will be calculated using a spring-dashpot model [43]. In this model, the normal and tangential forces can be represented as a system consisting of a spring and a damper. The formula of the forces for this model is:
where and are the normal and tangential elastic coefficients, and the corresponding damping coefficients, and the normal and tangential overlap vectors, and and the normal and tangential relative velocity vectors (see Figure 2).

Figure 2. Intersection plane which characterizes a collision between two spheres. The tangential force is integrated in time along the direction .
The elastic and damping coefficients () are obtained from the material properties of the solids and the details are given in Appendix A.
As shown in Eq. (4), the normal and tangential forces, and , each contain two contributions: an elastic term opposing displacement and a damping term opposing relative velocity. Depending on the collision phase (particles approaching or separating), these contributions may act in different directions. Additionally, the tangential overlap is limited by a Coulomb friction condition:
Let us detail how the normal force, , is computed. First, we define the unit vector normal to the contact plane (it goes from to direction), while the scalar represents the value of the overlap. The normal overlap is given by the difference between the sum of the radii of the spheres and the distance between their centers:
If , the spheres are in contact. The unit vector for the normal direction is given by the normalized vector from center to center:
Then, is defined as:
The normal relative velocity is given by the following projection:
where “” is the scalar product, and (resp. ) is the velocity of the center of mass of (resp. ). Therefore, the normal force can be rewritten as:
When it comes to the tangential force, , we first compute the relative tangential velocity . To do so, we need to compute the relative velocity between the surfaces of the spheres:
where and are the the angular velocities of the spheres and , and “” is the cross product. To get the tangential component of the relative velocity, we then remove the normal component:
After this, the tangential relative displacement, , needs to be computed. Since it depends on the relative movement of the surfaces over time, we need to integrate the tangential relative velocity over the duration of the collision:
where is the time at which the collision starts. Consequently, the “displacement history” must be tracked across iterations, as a single collision may span multiple DEM steps within one fluid time step [44].
Eventually, the torque exerted by on is obtained from the force through:
where denotes the vector from the center of the particle to the contact point. In the spherical case, this formula can be simplified to:
It is then convenient to store the torque normalized by the radius, , since this quantity is identical for both particles in contact and avoids redundant computations. However, this simplification does not hold for an ellipsoid, as the normal at the contact point is not guaranteed to pass by the solid’s center.
For the interaction of particles with walls, walls are treated like a static particle of a given shape (for example an infinitely extending plane) and a given surface velocity. Further details for the handling of walls are provided in [44].
The integration scheme
The motion of a particle is governed by Newton’s equations of translation and rotation:
where is the particle mass, the linear acceleration, the inertia tensor, the angular acceleration, and a body-fixed vector defining the particle’s orientation. The term collects all external forces, and in this work, it reduces to gravity only:
These equations are integrated in time using the second-order accurate, time-reversible Verlet algorithm [45, 46]. The translational degrees of freedom are updated as
The implicit nature of the scheme, due to the anticipated acceleration term , is resolved through a single predictor-corrector step. The rotational degrees of freedom evolve according to
Here, is the rotation matrix that maps vectors from the body-fixed frame at time to the reference frame aligned with the coordinate axes at . Since the particle’s orientation is fully described by the orthonormal triad , the -th row of is given by the components of .
Assuming a homogeneous particle of mass , the inertia tensor is defined as
where is the constant mass density, is the position vector relative to the particle’s center of mass, and is the identity matrix. For the particle types implemented here, and assuming axis alignment, the inertia tensors reduce to
where is the radius of the sphere, and are the semi-axes of the ellipsoid.
The lattice Boltzmann method
Contrary to standard CFD solvers which explicitly work with macroscopic quantities, LBM focuses on the evolution of velocity distribution functions with . This mesoscopic quantity represents the number of fictitious particles at a point with a given velocity . In that context, macroscopic quantities are byproducts of , and they can
be computed as its statistical moments. As an example, density and momentum are computed as
The choice of discrete velocities is particularly important for the LBM as it dictates its macroscopic behavior [47]. To simulate the physics of isothermal and weakly compressible flows with an LBM, it is sufficient to consider the D3Q19 lattice which is a 3D model composed of 19 discrete velocities [40]. D3Q19-LBMs then solve a coupled set of 19 lattice Boltzmann equations: ,
In practice, solving each equation involves two successive steps [48]:
where represents the post-collision populations, and is the collision model in its general form. These steps collectively form the “Collide and Stream” numerical scheme, well-recognized for its efficiency [28, 49] and accuracy [50, 51].
The most common collision model considered in the LB literature is the BGK operator, which was named after its authors Bhatnagar, Gross, and Krook [52]. It corresponds to a simple relaxation process that makes tend towards an equilibrium state . It reads
where the relaxation time is
is the kinematic viscosity of the fluid, and is the speed of sound of the D3Q19 lattice [40]. The equilibrium state is an Hermite polynomial expansion of the Maxwellian, that is commonly truncated to second-order for isothermal and weakly compressible LBM [47]:
are the weights of the D3Q19 lattice that can be found in any textbook [40].
Since the advent of LBM, a large number of collision terms have been proposed [53]. Some of them target an improved stability in the low viscosity regime [54], while others are dedicated to increasing the accuracy of LBMs for under-resolved low-Reynolds simulations [55, 56]. The latter approach, commonly referred to as two-relaxation-time (TRT) LBM, introduces a set of two parameters to independently control the kinematic viscosity and the accuracy of the model. In practice, this is achieved by considering two temporary distributions where
is defined as the opposite direction of so that . A temporary pair of equilibrium states following the same rules (36) is also introduced. One eventually ends up with the TRT collision model:
In this work, controls the kinematic viscosity whereas is fixed to
for an improved accuracy in under-resolved and low Reynolds conditions [57].
In LEDDS, both BGK and TRT collision models have been implemented, but additional models can be easily included through code copy/paste from the STLBM framework. For the purposes of this study, we focus exclusively on the TRT model, as it is particularly well suited for low-Reynolds-number simulations of fluid-particle interactions.
Eventually, Dirichlet boundary conditions are enforced using the standard and velocity-modified bounce-back schemes [58]. The initial macroscopic fields are prescribed by setting the distribution functions to their corresponding equilibrium values (35).
In most simulations, the physical setup is specified through a characteristic length in physical unit , the Reynold number , and a reference velocity in physical unit . The numerical parameters are fixed by the choice of the characteristic length in lattice unit and the velocity in lattice unit , from which LEDDS computes the space discretization and discrete timestep . The relaxation time is computed from the kinematic viscosity according to Eq. (34).
In practice, the time step of the fluid is chosen to match stability and accuracy constraints, through the choice of . The DEM solver can run with the same time, which sometimes however leads to numerical instability. In this case, we resort to substepping, where the particle time step is smaller than , and the ratio is integer.
Fluid-solid coupling through partially saturated cells
To model fluid-solid interactions, we adopt a two-way coupling approach, where the fluid influences the solids and the solids in turn influence the fluid. Our study focuses on the (fully) resolved case, in which particle diameters are several times larger than the fluid cell size.
In this work, no explicit tagging of cells as fluid or solid is performed. Instead, for cells intersected by a particle, we compute the fraction of solid volume contained within each fluid cell. This quantity provides a continuous measure of the local fluid–solid overlap and allows for a smooth representation of the interface on the Cartesian grid. This approach avoids a sharp classification of cells and instead relies on a local solid fraction to account for partially occupied cells, offering a good compromise between accuracy, computational efficiency, and ease of implementation. This method, known in the LB community as the partially saturated method (PSM), is closely related to the volume penalization method widely used in CFD [61].
In the PSM, each lattice cell may be partially occupied by one or more solids, and the local collision operator is adjusted according to the cell’s solid fraction. This effectively creates a penalization-like blending between fluid and solid behavior, by enforcing the no-slip condition through a smooth transition across the interface [38]. The inclusion of partially saturated cells modifies the standard LBM collision step as follows
where
for a fluid collision operator based on the TRT approach. Here, is the solid collision operator that accounts for the influence of solid particle on the local fluid cell. The weighting factor depends on the volume fraction of the cell occupied by solid , while represents the cumulative contribution of all solids present in the cell.
The solid collision operator is expressed as [38, 62]:
where is the surface velocity of the solid [38, 62] and is given by:
is the center of mass of the solid , while is the angular velocity of the sphere, and is the position of the fluid cell . The coefficient is given by:
where is the the so-called solid fraction (which is the proportion of the cell occupied by the solid ).
When multiple solids collide, they may overlap within the same fluid cell, potentially resulting in a total value of greater than one. In our framework, we address this challenge by employing a straightforward solution, which is truncating this sum to one:
Interestingly, when a fluid cell contains no solid, equals zero, and the PSM formulation reverts to the standard “Collide and Stream” algorithm.
Ensuring a smooth evolution of the solid fraction is crucial to avoid accuracy and stability issues. In practice, there are different ways to compute that quantity (see Ref.[63] for more details). In LEDDS, two of these methods are implemented. The first method, called the subdivision method, involves picking a certain number of points spread evenly inside the cell. We then calculate the ratio between the total number of points in the cell and the number of points in the solid:
where are points taken at known locations and uniformly distributed inside a fluid cell , is the position of the sphere , is the total number of points, and is the radius of the solid sphere. This method ensures a smooth evolution of the solid fraction as the solid moves from one fluid cell to another. However, it can also be computationally intensive if the number of points inside a fluid cell is too high. Imposing 10 points per direction () was found to be a good tradeoff, but the user can adjust this parameter if needed. The second approach, referred to as the distance-based method, relies solely on a geometric distance computation [64]. The solid fraction contribution of particle to cell is estimated as
where is the centered position of the fluid cell . The calculation of is actually just the overlap distance between the solid sphere and a sphere of volume centered at the cell’s center. Alternative distance-based formulations exist. For example, Jones et al. [63] proposed a fast approximation based on a purely linear solid-fraction model using precomputed coefficients that depend only on the particle radius, demonstrating good accuracy for spherical particles. However, such an approach is not considered here because extending it to ellipsoidal particles requires significant modifications of the original, as the distance relation must account for particle orientation. In this work, the subdivision method is therefore used for physical validation (§ 4), while the distance-based approach serves primarily for performance comparison with the state-of-the-art solver waLBerla, which relies on the same strategy [64] (see § 5).
It is interesting to note that other approaches for the distance computation exist, such as Ref. [63], which showcases good results by approximating the solid fraction for spheres purely linearly, relying on a precomputed term dependent on the radius of the spheres. This methodology is however not explored in the present work, as it would be less straightforward to extend to ellipsoidal particles, where the distance also depends on the orientation of the particles. Instead, this work showcases the subdivision method for physical validation, whereas the distance-based approach is used for performance comparison with state-of-the-art solver waLBerla which also relies on this approach.
Finally, the forces and torques exerted by the fluid on a solid can also be linked to the coefficient [38, 65]:
where is the index of the cells (partially) contained inside the solid, and are the velocities of the lattice. is the th component of the solid collision term corresponding to the th solid inside the cell .
Portable implementation strategy
The main methodological novelty of this work lies in expressing all stages of a coupled LBM–DEM simulation (neighbor search, contact detection, DEM force evaluation, etc) exclusively as compositions of high-level algorithmic primitives. This formulation enables GPU acceleration without relying on device-specific kernels, while remaining applicable to non-trivial particle geometries such as ellipsoids, where contact detection and interaction forces depend on particle orientation and gap size. Below, we describe the general strategy behind LEDDS, while emphasizing both portability and performance. We first present the high-level design principles, including the use of algorithmic primitives and the structure-of-arrays memory layout, which together ensure clear, maintainable, and fully data-parallel code. We then describe the implementation details that realize these principles in practice for DEM and LBM–DEM coupling.
General principles
Algorithmic primitives
The design of LEDDS is based on a simple programming philosophy: all parallelism is expressed exclusively through high-level algorithmic primitives provided by the programming framework. Hence, the user code clearly expresses the intent of the computation, while the compiler and runtime determine how these operations are executed in parallel on the available hardware. This approach keeps the code simple, and maintainable, while avoiding manual handling of concurrency and synchronization. As a result, undefined behavior due to race conditions can be excluded by construction, as explained below.
The concept of algorithmic primitives originates from Blelloch [17], who demonstrated that a small set of data-parallel operations, such as map, reduce, and scan, forms a complete basis for expressing parallel algorithms. Originally developed for vector and PRAM architectures, this abstraction has regained importance with the rise of GPUs and heterogeneous platforms, where explicit data parallelism maps naturally to hardware. A central goal of modern high-performance computing is therefore to express algorithms entirely in terms of such primitives, enabling performance portability across current GPUs and future accelerators. According to Blelloch’s taxonomy, three fundamental categories of primitives can be defined: (1) element-wise operations such as map, gather, and scatter; (2) reductions and scans such as reduce, scan, and segmented_scan; and (3) permutation and filtering operations such as sort and partition. For simplicity, we also treat unique as a primitive, even though it can be expressed as a combination of map, scan, and scatter.
| Task / Step | Primitive | Size | Further details |
| Particle advancement | Section 3.2.1 | ||
| Integrate (Velocity-Verlet) | Map | Eq. (18a) to (24) | |
| Update cell-linked list | Map | ||
| Compute number of neighbors | Map | ||
| Collision list construction | Section 3.2.1 | ||
| Compute offset of collision pairs | Prefix Sum | ||
| Create list of collision pairs | Map | See Figure 5(b) | |
| Make collisions adjacent | Sort | See Figure 5(c) | |
| Remove duplicates | Unique | ||
| Compute DEM forces | Section 3.2.2 | ||
| Compute force and torque | Map | Eqs. (4) and (14) | |
| Update tangential displacement | Map | Eq. (13) | |
| Compute particle range | Map | See Figure 6(left) | |
| Reduce DEM forces | Section 3.2.2 | ||
| Left pair-member: reduce force | Map | See Figure 6(left) | |
| Right pair-member: sort | Sort | See Figure 6(right) | |
| Right pair-member: compute range | Map | See Figure 6(right) | |
| Right pair-member: reduce force | Map | See Figure 6(right) | |
| PSM fluid-particle coupling | Section 3.2.3 | ||
| Compute solid fractions | Map | Eq. (44) | |
| Compute hydrodynamic forces | Map | Eq. (48) | |
| LBM collision-streaming | Map | As in Ref. [28], with Eqs. (39) and (40) | |
| Force reduction | Section 3.2.3 | ||
| Sort forces per particle | Sort | ||
| Reduce forces | Reduce_by_key |
Table 1. Computational workflow of LEDDS, expressed entirely in terms of algorithmic primitives. In practice, LEDDS calls either C++ Standard Library (STL) algorithms or Thrust algorithms for any of these primitives. This unified representation covers both LBM and DEM stages without requiring any explicit synchronization or device-specific code. The table also reports the relevant problem size for each step, denoted by (number of particles), (number of grid cells for the cell-linked list), (number of inter-particle collisions), and (number of fluid cells).

Figure 6. The particle-particle forces are computed during a parallel map operation applied to all particles. For the force acting on a given particle, a non-parallel sum over all precomputed pairwise forces in which the particle appears as the left pair-member (left panel) or the right pair-member (right panel) is carried out. The former case is easily computed, as the collision pairs and forces are already sorted according to the first pair-member, as shown in Figure 5(c). For the latter case, the collision pairs must be sorted anew upon their second pair-member, coupled to their collision-force index, which no longer corresponds to the index of the collision pair. A similar approach applies for the torques.
In LEDDS, the LBM-DEM coupling algorithm is fully formulated in terms of four such primitives: map, reduce_by_key, unique, and sort. Table 1 summarizes the entire workflow, showing how these primitives are applied across the main computational steps—from particle advancement and collision detection to DEM force computation and fluid-solid coupling. Most operations rely on standard C++ parallel algorithms, while a few performance-critical ones make use of the Thrust library, which provides optimized GPU implementations. This mixed use of C++ and Thrust strikes a balance between portability and efficiency: C++ primitives ensure maintainable and hardware-agnostic code, while Thrust provides the necessary high-performance support for segmented reductions and sorting.
In this context, each primitive serves a specific role:
Map (C++ STL) Applies a user-defined operation independently to each element of an array, producing one output element per input element. Implemented using
std::transformfrom the C++ Standard Template Library (STL).
Reduce-by-key (Thrust) Combines consecutive values sharing the same key using an associative operator, producing one output per unique key. Standard C++ parallel algorithms do not yet support segmented reduction, so LEDDS uses
thrust::reduce_by_keywith logarithmic parallel span.
Unique (C++ STL) Removes consecutive duplicate elements from a sequence, leaving only the first occurrence of each distinct element. Although it can be constructed from lower-level primitives (prefix-sum & scatter), it is treated as a standalone operation for clarity. Implemented using
std::unique.
Sort (Thrust) Reorders elements by key so that elements with equal keys are contiguous. LEDDS uses a radix-based GPU implementation (
thrust::sort_by_key) for linear work and logarithmic span on fixed-width integer or floating-point keys, which is more efficient than the comparison-basedstd::sortwith higher asymptotic cost. To exclude the possibility of race conditions during the application of the map primitive, two common cases must be con-
sidered. First, non-local overlapping reads and writes must be avoided; this is typically ensured by separating input and output arrays, as in the LBM two-population collision-streaming kernel. Second, when reductions or data reordering occur at the write location, a PULL-style parallel processing approach is employed to ensure thread safety, for example when constructing particle and collision lists (Figure 5 a-Figure 5b)

Figure 5. Three-step procedure for the global computation of all unique collision pairs, illustrated with an example (solids are represented by letters instead of numbers for better readability). Step (a) and (b) are local per-cell operations. Step (c) sorts the collision pairs and makes them unique in a global operation.
From a performance standpoint, primitives with logarithmic parallel span, such as reduce_by_key, unique, and sort, are effectively limited by memory bandwidth rather than computation. On GPUs, global memory access dominates runtime, so the theoretical span does not introduce significant latency. This is particularly true for radix-based sorting, whose observed scaling is essentially linear for data size of typical LEDDS simulation runs; a comparison-based implementation would make the logarithmic factor more apparent. Consequently, all primitives in LEDDS are fully data-parallel and warrant portability of performance across platforms.
Data structure
LEDDS primarily employs a structure-of-arrays (SoA) data layout, where each field is stored in a separate array. The only exception concerns vector quantities (e.g., position, velocity, and force) whose three components are kept contiguous in memory. The SoA layout promotes coalesced memory access and often yields significantly higher performance, for instance when applied to the memory alignment of LBM populations for efficient collision-streaming execution [28]. It also integrates naturally with the use of algorithmic primitives, since operations such as sorting or reduction can easily exchange one key field for another without restructuring the associated value arrays.
The persistent arrays representing the state of the system at each time step are:
LB populations: Two 19-element fields in SoA layout representing the D3Q19 “two-population” model for the fluid (used by LBM primitives in Table 1).
Particles: Two three-component arrays storing particle positions and velocities.
Collision pairs: Each pair identifies a potential contact between two particles. Stored explicitly to index per-contact quantities such as tangential displacements.
Lateral displacements: Track tangential offsets over the lifetime of each contact and are used to compute frictional forces in the DEM model.
In addition to these persistent arrays, LEDDS relies on temporary buffers during specific algorithmic steps:
Grid for cell-linked list: The simulation domain is discretized into a uniform grid. Each cell stores a fixed-size list accommodating the maximum number of particles overlapping the cell.
DEM forces computation: Temporary arrays store pairwise contact forces and torques before reduction to per-particle values using map and
reduce_by_keyprimitives.Fluid forces computation: Hydrodynamic forces are evaluated on fluid-solid interface cells and subsequently reduced to per-particle values using the
reduce_by_keyprimitive.
Throughout the workflow (Table 1), the following symbols denote the relevant problem sizes:
Number of cells in the fluid grid.
Number of cells in the cell-linked-list grid.
Instantaneous number of particles.
Instantaneous number of inter-particle collisions.
Although per-cell operations over the fluid and grid cells share the same asymptotic complexity, , the number of grid cells is typically one to two orders of magnitude smaller than . Consequently, operations scaling with , such as those on LBM populations, tend to dominate the overall computational cost.
Implementation of DEM and LBM-DEM coupling
Hereafter, we focus on the different steps required by LEDDS to update the position of particles, compute the DEM forces, and the fluid-particle interactions. The implementation of the fluid update is detailed in Ref. [28] and is not repeated here. As shown in Figure 3, the algorithm for particle progression and interaction requires three main stages: (1) Update of the particles and of the uniform grid / cell-linked list which is used to identify potential collisions; (2) The computation of particle-particle interactions and update of corresponding tangential displacements of particles; (3) The LB update of fluid cells through the PSM, and the computation of particle-fluid interactions. In case of pure DEM simulations, the algorithm reduces to steps (1) and (2). While in principle some stages could be executed asynchronously for fluid cells far from any particle, in practice the strict data dependencies between particle updates, collision detection, and fluid-particle coupling require a serialized timestep-level execution to ensure good GPU performance. The different algorithms composing these steps are summarized in Table 1 and are explained below. For additional details regarding the memory overhead associated with the coupling, the reader is referred to Appendix § 10, where a detailed analysis is provided.

Figure 3. Overview of the LEDDS workflow from the particle perspective. Colored groups highlight the three main stages: particle/grid updates (blue), DEM force computation (red), and fluid-particle coupling (green), corresponding to § 3.2.1, § 3.2.2, and § 3.2.3, respectively. In case of pure DEM, the third stage disappears from the workflow.
Uniform grid and cell-linked list
During the first stage, particles are advanced in time using a second-order, time-reversible, explicit Velocity-Verlet scheme, following Eqs. (18a) through (19d). To avoid a neighbor search, a uniform grid is set up as a spatial acceleration structure for collision detection, commonly referred to as a cell-linked list, in which each grid cell stores the identifiers of nearby particles. In our implementation, each cell uses a fixed-size array rather than a dynamic list, since the number of particles per cell is small and bounded. Particles are associated with a grid cell when their axis-aligned bounding box (AABB) intersects the corresponding spatial region (see Figure 4).

Figure 4. Illustration of the 2D uniform grid. Spheres and overlap the cells and . Sphere is considered present in cells and due to the use of the AABB.
The update of the cell-linked list is simplified by the assumption that particles move no more than one grid-cell width per time step. In this case, the updated particles assigned to a grid cell are computed by parsing all particles currently linked to neighboring grid cells. This allows updating the cell-linked list through a unique traversal of all grid cells. At each cell, data is gathered from neighboring cell and the local neighbor information can be updated through a thread-safe PULL-operation. During the same memory traversal, the number of collision pairs for each grid cell is computed to prepare the data structures for the subsequent collision-detection stage. These steps are summarized in the following extract of Table 1:
| Particle Advancement Steps | Primitive | Size |
| Integrate (Velocity-Verlet) | Map | |
| Update grid | Map | |
| Compute number of neighbors | Map |
Table 1 (PDF p. 10).
After the grid update, potential collision pairs are identified through a sequence of parallel algorithm calls. Through a parallel map over the grid, a list of particle pairs is first generated at every cell by scanning, as before, the neighboring cells in a PULL-operation. To avoid redundant entries, only pairs satisfying are considered. The resulting collision list is dynamically allocated on the fly, providing the flexibility to handle highly dynamic simulations without additional parametrization.
The storage offsets of the collision pairs are precomputed as the result of a prefix-sum algorithm call applied to the precomputed number of neighbors, which ensures contiguous and efficient memory allocation. Since the same pair may be detected from multiple neighboring cells, the resulting list can contain duplicates. A global lexicographic sort followed by a unique operation removes these duplicates, yielding a compact and ordered list of unique collision pairs. These steps are summarized in the following extract of Table 1:
| Collision List Construction Steps | Primitive | Size |
| Compute offset of collision pairs | Prefix Sum | |
| Create list of collision pairs | Map | |
| Make collisions adjacent | Sort | |
| Make collisions unique | Unique |
Table 1 (PDF p. 10, second).
DEM forces computation
The second stage corresponds to the computation of contact forces and torques [1] between colliding particles. These terms are first computed for all identified collision pairs before being reduced to per-particle quantities. Note that forces are stored only once for every interaction pairs, as they are equal for two interacting particles owing to the action-reaction principle, while the torque is stored separately for each of the two particles of a collision pair.
Per-collision normal and tangential forces [4], and their associated torque [14], are determined by the chosen contact model and are evaluated using per-element operations over the collision list. The tangential displacement history [13] required by frictional models is updated in the same process. To preserve the tangential displacement history, the displacement from the previous time step is retrieved and updated, when available. This operation is performed in parallel for each collision pair.
The lookup is accelerated by exploiting the fact that the collision list from the previous time step is sorted. A sequential binary search is therefore used to efficiently identify the matching pair and copy the associated tangential displacement. The net force and torque acting on each particle is obtained from the per-collision forces and torques through sorting and range-based accumulation as summarized below:
| Compute DEM Forces Steps | Primitive | Size |
| Compute force and torque | Map | |
| Update tangential displacement | Map |
Table 3.
It is noted that the accumulation for the total force and torque acting on a given particle accounts for both pairs in which appears as first member, and pairs in which appears as second member. The former case is easily computed as the collision pairs are already sorted in a way in which first pair-members are memory adjacent. After computing the range of pairs for each particle, these forces are computed through a range-based accumulation (reduce_by_key). For the latter case, the pairs are first sorted again according to their second pair-member. A renewed computation of per-particle ranges, followed by a range-based accumulation, concludes the computation of DEM forces. All these steps are summarized below:
| Reduce DEM Forces Steps | Primitive | Size |
| Compute particle range | Map | |
| Left pair-member – reduce force | Map | |
| Right pair-member – sort | Sort | |
| Right pair-member – compute range | Map | |
| Right pair-member – reduce force | Map |
Table 4 (PDF p. 11).
Fluid-particle coupling
The third stage of the numerical scheme runs the LB fluid solver with partially saturated cells (40) and integrates the fluid forces acting on the particles. Given the updated particle positions, the contribution (43) of individual particles on every fluid cell is computed through an element-wise operation on all fluid cells. To enhance efficiency, an upper bound is defined for the maximal number of solids that can cut a fluid cell at the same time. In the context of fully resolved particles presented in this article, a value of is chosen. The number of solids per cell is in practice limited to two for fully resolved LBM-DEM, as the particle diameters largely exceed the size of fluid cells. The overall solid fraction (44) is computed from the contributions , and the resulting interaction forces (48) and torques (49) are computed directly for all affected cells. As a next step, the LBM collision-streaming kernel is executed as a single element-wise operation on all fluid cells, following the “two-population” LBM implementation described in the STLBM library [28], here supplemented with the PSM Eqs. (39) and (40). While this choice doubles the memory footprint, it greatly simplifies the implementation of the coupling steps without noticeably affecting the performance of the LBM module (see Figure 9.7 of Ref. [66] for a detailed comparison). These coupling steps are summarized as:
| PSM Fluid-Particle Coupling Steps | Primitive | Size |
| Compute solid fractions | Map | |
| Compute forces | Map | |
| LBM collision-streaming with PSM | Map |
Table 5 (PDF p. 11, second).
After this, the computed forces are reduced to per-particle quantities through sorting and key-based reduction operations. This is achieved applying a sort primitive over all fluid cells, and a range-based accumulation to recover per-particle forces.
| Force Reduction Steps | Primitive | Size |
| Sort forces per particle | Sort | |
| Reduce forces | Reduce_by_key |
Table 6 (PDF p. 11, third).
With these updated forces, the new acceleration and velocity of particles can now be computed which concludes the current time step.
Validation
In this section, we systematically assess the accuracy and physical consistency of the LEDDS framework across a hierarchy of test cases. We begin by validating the DEM module in collision scenarios of increasing complexity:
Head-on, perfectly elastic collision between two spheres, verifying energy and momentum conservation;
Eccentric collision between two ellipsoids, confirming correct handling of non-spherical particle dynamics;
Sphere-wall impact in the presence of friction, used to benchmark tangential force modeling; and
Many-body simulation of identical spheres confined in a box, reproducing the statistical equilibrium predicted by kinetic theory for a gas of perfectly elastic particles;
Many-body simulation of ellipsoids forming a sand pile, used to evaluate the angle of repose as a function of aspect ratio;
followed by two coupled LBM-DEM benchmarks designed to further evaluate the accuracy and robustness of the fluid-particle coupling through PSM:
Settling of a single particle in a viscous fluid, used to evaluate the dynamic behavior of an isolated particle as it interacts with the surrounding flow field and approaches its terminal settling velocity;
Rotational dynamics of a single ellipsoidal particle in a viscous shear flow, used to validate the computation of hydrodynamic torques and the accurate reproduction of Jeffery orbit dynamics of a non-spherical particle; and
Shear-flow simulation of a densely packed particle suspension, aimed at validating the effective rheology predicted by the coupled model in many-body configurations under steady shear.
Unless otherwise stated, quantities are either dimensionless, like to Reynold number, or expressed in physical units (shown in square brackets). Equivalent values in lattice units are reported when relevant to the numerical setup. All simulations are performed using single-precision arithmetic, except for the Jeffery orbit case, which uses double precision due to the very small time step required to keep the relaxation time below 2—a good practice to ensure accurate LBM results in the Stokes flow regime [40].
Head-on perfectly elastic collision of spheres (DEM)
To validate the integration of the LIGGGHTS DEM module into our framework, we first verify the conservation of momentum and energy during a head-on collision between two identical, frictionless spheres () under perfectly elastic conditions (restitution coefficient ). Energy conservation is assessed by tracking the time evolution of the total mechanical energy, defined as the sum of kinetic and elastic potential energies, throughout the collision.
As seen in Figure 7, the simulated system consists of two identical spheres, labeled and , with radii [mm], masses [Kg], Poisson’s ratios , and Young’s moduli [MPa]. Initially, the spheres are separated by 10 [m] and move toward each other with equal and opposite velocities: [m/s]. The simulation uses a time step of [s] and runs for 300 iterations. Single and double precision produce compatible results (see Appendix B), so only single-precision results are shown.
Figure 7. Initial configuration of the head-on collision between two identical spheres. Sphere (blue) and sphere (red), both of radius [mm], are initially separated by a gap of 10 [m] and move toward each other with equal and opposite velocities and .
Based on the above initial conditions, the total momentum of the system at is then . As the simulation progresses, the magnitudes of the particle velocities decrease during the collision until they reach zero (see Figure 8). Their velocities then reverse direction and increase in magnitude as the spheres move apart in opposite directions. Clearly, the total momentum remains unaffected by the collision and re-
Figure 8. Head-on elastic collision: Time evolution of the spheres’ longitudinal velocities ( and ) starting from opposite initial velocities.
tains its value of zero throughout the entire simulation.
We can also verify whether the total energy of the system, , is conserved throughout the collision. Here, is the translational kinetic energy of the spheres, and is their normal elastic potential energy. The translational kinetic energy is given by
while the normal elastic potential energy is computed from the normal elastic force, which is defined as (10). Explicitly, this force is
where and are defined in Appendix A. Integrating this force over the overlap yields the normal elastic potential energy:
As illustrated in Figure 9, the total energy of the system remains constant during the collision. Initially, all the energy in the system is of translational kinetic nature. During the initial phase of the collision, as the spheres approach each other, kinetic energy gradually transforms into elastic potential energy until particle velocities reach zero. At this point, the entire energy of the system is stored as elastic potential energy. Subsequently, the spheres start to separate under the influence of the elastic normal force, and the energy in the form of elastic potential energy transitions back into kinetic energy. This process continues until the spheres are no longer in contact, and the system’s energy is solely kinetic as they move away from each other.
Figure 9. Head-on elastic collision: Time evolution of total , translational kinetic , and elastic potential energy of two identical spheres ( and ) with opposite initial velocities.
Eccentric perfectly elastic collision of ellipsoids (DEM)
To further validate the extension of the DEM framework to non-spherical particles, we perform a second test case involving the collision of two ellipsoids. This simulation serves two main purposes: (1) to verify that the model correctly captures the rotational motion resulting from an eccentric (off-center) collision, and (2) to confirm the conservation of total mechanical energy in the absence of dissipative effects.
The physical and numerical parameters are identical to those used in the spherical benchmark, except for the particle shapes, masses, and initial positions. The two ellipsoids, labeled and , have semi-axes
and identical masses . Initially, their centers of mass are positioned at
and their initial translational velocities are along the -axis (see Figure 10). This slight vertical offset introduces an eccentricity in the impact geometry, causing the line of action of the contact force to no longer pass through the centers of mass. As a result, both ellipsoids experience a torque during contact, which initiates rotation. This behavior was absent from the previous simulations because it cannot occur in a perfectly frictionless sphere-sphere collision.
Figure 10. Initial configuration of the eccentric ellipsoid collision. Ellipsoid (blue, semi-axes [mm], [mm]) and ellipsoid (red, semi-axes [mm], [mm]) approach each other with equal and opposite velocities and . The lateral offset [mm] between centers induces an eccentric impact.
The simulation is carried out with a time step of over 10000 iterations, which corresponds to a physical time of 100 [ms].
The onset of rotational motion is quantified through the rotational kinetic energy:
where and denote the inertia tensors of the two ellipsoids, and , are their respective angular velocity vectors. The total mechanical energy of the system then reads:
where is the normal elastic potential energy, the translational kinetic energy, and the rotational kinetic energy.
As shown in Figure 11, the initial translational kinetic energy of the ellipsoids is partially converted into rotational kinetic energy through the elastic potential energy developed during the collision. When the particles first make contact, part of the translational energy is stored elastically while a smaller portion initiates rotational motion. As the ellipsoids separate, the potential energy component is progressively reconverted into translational and rotational kinetic energy components. The total energy of the system remains nearly constant throughout the simulation, confirming good energy conservation properties of the collision model for non-spherical particles.
Figure 11. Eccentric collision of two identical ellipsoids: time evolution of the total (), translational kinetic (), rotational kinetic (), and elastic potential () energies.
Quantitatively, the relative variation between the initial and final total energies is . This small deviation originates from geometric and numerical approximations inherent to the discrete-element formulation for ellipsoidal particles, particularly those related to the estimation of local curvature and contact orientation (see § 2.1). This interpretation is further supported by repeating
the same simulations using single precision (see Appendix B). The resulting total-energy deviation is 0.143%, compared with 0.141% obtained in double precision, indicating that numerical precision plays only a minor role relative to the geometric and algorithmic approximations discussed above. To the best of our knowledge, the conservation of energy during collisions of non-spherical particles is rarely addressed in the DEM literature, making this work one of the few to explicitly tackle it.
Overall, the agreement with the theoretical expectations remains very strong, thereby confirming the validity and robustness of the extended collision model for ellipsoids.
Sphere-wall collision with friction (DEM)
As a next step, we now target the validation of collisions between a single sphere and a rigid wall in the presence of friction. To this end, we simulate a sphere approaching a flat wall with a prescribed velocity and incident angle, initially without angular velocity (see Figure 12). Introducing a finite friction coefficient causes part of the translational kinetic energy to be converted into rotational motion. Since the collision is perfectly elastic (), the total energy remains constant, and the translational and rotational energies are simply redistributed by the frictional interaction.

Figure 12. Setup of the simulation for the test. Before the collision, the sphere is not rotating. The angles and are taken relatively to the normal direction of the wall.
In practice, we can quantify the effect of friction on energy transfer by tracking the sphere’s angular and tangential velocity components at the point of contact with the wall. By definition, this tangential velocity reads as
where denotes the angular velocity around the -axis and is the sphere radius. We then introduce the pre- and post-collision velocity ratios relative to the incident vertical velocity :
where for perfectly elastic collisions . These ratios are used to measure of how friction redistributes translational motion into rotational motion during the collision.
Given the complexity of tangential interactions, we focus on the regime where the sphere slides along the wall for the entire duration of contact. In this sliding regime, the tangential
force is continuously limited by Coulomb’s friction law [5]. Under this assumption, the post-collision velocity ratio can be expressed analytically as a function of and the friction coefficient [67, 68, 69]:
A critical value of exists beyond which the sliding regime is reached:
while for smaller , the collision exhibits a mixture of sticking and sliding, which is more difficult to analyze. The interested reader can refer to the detailed theoretical and experimental discussions of these regimes that are provided in Refs. [69, 70].
We conducted simulations with friction coefficients, , ranging from 0 to 0.5, and incident angles from to (corresponding to to 1.44 in radians). The post-collision ratio is plotted in Figure 13 using for direct comparison with the incident angles. Globally speaking, the results exhibit excellent agreement with theoretical predictions in the sliding regime and accurately capture the rotational motion induced by friction. Notably, the post-collision angle matches perfectly the incident angle in case of a frictionless wall, i.e., for . As expected for and , systematic deviations occur which reflects the transition to the mixed sticking/sliding regime that is also accurately captured by LEDDS.
Figure 13. Evolution of the post-collision velocity angle () as a function of the incident angle () for different friction coefficients . DEM simulations (dots) are compared with the analytical prediction for the sliding regime (dashed lines), . The vertical dotted line marks the transition threshold beyond which the full sliding hypothesis is correct.
Many-body validation: Gas-like behavior in a box (DEM)
As a first many-particle validation of the DEM part of LEDDS, we reproduce the collective dynamics of a large ensemble of colliding particles as predicted by the kinetic theory of gases. In this test, we simulate a system of 10000 identical spheres confined within a rigid, four-wall box. All particles are initially assigned the same translational velocity vector, [mm/s], and no rotational motion. Collisions are modeled as perfectly elastic () and frictionless (), which ensures that the total mechanical energy of the system remains constant and predominantly resides in translational kinetic energy. Even if there is a brief conversion of kinetic energy into elastic potential energy during contact, it is negligible in practice because the collision timescale is extremely short. As a result, energy is solely exchanged between translational degrees of freedom, much like in a monatomic gas.
When the number of particles is sufficiently large, the frequent collisions lead to momentum exchange among particles, driving the system toward a statistical equilibrium that closely resembles an ideal gas confined in a closed container. Consequently, the velocity components of the particles are expected to follow a Maxwell-Boltzmann distribution centered around zero, reflecting the fact that, in the absence of external energy input, the gas as a whole has no net motion.
Focusing on the -component of the particle velocity, the probability density function at equilibrium is given by [71, 72]:
where is the particle mass, the Boltzmann constant, and the absolute temperature of the system. This distribution is Gaussian, with zero mean and variance . From the equipartition theorem, the mean kinetic energy per particle relates to the macroscopic temperature of the gas as:
which leads to

Figure 14. Pseudo-3D sketch of the many-body DEM validation. An initially ordered system of identical spheres with initial velocity [mm/s] evolves through perfectly elastic, frictionless impacts towards a Maxwell-Boltzmann equilibrium.
Given the initial condition [mm/s], the expected standard deviation of a single velocity component is therefore [mm/s].
The simulation was run for 400000 iterations with particles of radius [mm]. Figure 15 shows the time evolution of the velocity distribution along the -axis. The initially monokinetic state progressively broadens as collisions redistribute momentum among all degrees of freedom. The distribution ultimately converges to a Gaussian centered at zero with the predicted variance, confirming that the DEM formulation correctly captures the many-body statistical behavior expected for an idealized gas of perfectly elastic spheres.
Figure 15. Many-body validation: time evolution of the particle velocity distribution ( component) for 10000 elastic and frictionless spheres confined in a box. The distribution converges toward the theoretical Maxwell-Boltzmann form with a mean velocity of zero and a standard deviation of 1 [mm/s].
Many-body validation: Angle of repose (DEM)
As a second validation of collective particle behavior, we investigate the formation of sandpiles composed of ellipsoidal particles. The test case consists in injecting particles into a confined domain, where they accumulate against a wall to form a stable granular pile (see Figure 16). The angle of repose, , defined as the maximum angle between the (top) free surface and the horizontal plane at which the pile remains stable, is used as the primary validation metric. Its accurate prediction provides a robust assessment of the contact detection algorithm and, more broadly, of the overall accuracy of LEDDS in modeling interactions between non-spherical particles.

Figure 16. Pseudo-3D sketch of the confined injection setup used to form a sand pile. Particles are introduced at through the injection box and accumulate against the lateral and bottom wall, which creates the pile shown in orange. The repose angle is measured along the (top) free surface of the pile.
In practice, sand piles can be generated through several procedure, including particle injection, tilting, and discharge [73, 74]. The injection method is adopted for its simplicity and its reduced dependence on rolling resistance effects [75]. It also avoids the need for moving or partially defined boundaries, making it particularly well suited to the current capabilities of LEDDS. Here, new particles are inserted by batches of 10, within a box of size every 1000 time steps. The injection box is positioned one maximum particle radius above the highest particle in order to (1) avoid overlapping with previously injected particles, and (2) limit the falling distance and associated kinetic energy. Since the angle of repose is symmetric in practice, only half of the sand pile is simulated by injecting particles adjacent to the left wall.
In our simulations, the computational domain has dimensions , where the equivalent particle diameter is fixed to [mm] for all aspect ratios (), and here, . Here, denotes the diameter of a sphere having the same volume as the particle. For a general ellipsoid with semi-axes , , and , its volume reads . Defining as the diameter of a sphere with the same volume, , which leads to . In this study, we restrict ourselves to axisymmetric particles with . All particles have a density [kg/m], and a small restitution coefficient to improve robustness while speeding up convergence to a steady state. Two friction coefficients, , are considered in order to assess the sensitivity of the angle of repose to interparticle friction. Near-rigid behavior is enforced using a Poisson ratio and a Young’s modulus [Pa]. The bottom and side walls are assigned identical properties to ensure consistent particle–wall interactions. Gravity is set to [m/s], and the time step is fixed to [s] (highest value for which the simulation), and simulations are performed up to time steps.
The angle of repose is evaluated after particle injection has ceased and the system has reached a stable equilibrium. The measurement procedure follows the methodology proposed by Chen et al. [76]. Notably, the contour of the sand pile is extracted using the OpenCV library where the apex and toe regions are excluded to mitigate boundary effects (upper and lower 10% height of the pile). The remaining surface profile is then partitioned into three equal segments, and a linear fit is applied to each segment to determine the local slope (see Figure 17), from which the corresponding local angles of repose are obtained. These local values are finally averaged to define the global angle of repose of the sand pile.

Figure 17. 3D Paraview illustration of the angle of repose measurement procedure for an aspect ratio . The extracted free surface is restricted to an intermediate region of the pile, excluding the apex and toe (lower and upper 10% of the pile), and partitioned into segments over which local slopes are obtained by linear fitting. Particles are colored according to their deposition order, ranging from blue (earliest deposited particles) to red (most recently deposited), highlighting the successive buildup of the granular pile.
The resulting angle of repose is reported in Figure 18 over a wide range of ellipsoid aspect ratios and for two friction coefficients. Overall, very good agreement is observed with the reference data of Chen et al. [76], with both datasets exhibiting a clear non-monotonic dependence on aspect ratio. In particular, the minimum angle of repose is obtained for spherical particles (), while larger values are observed for both oblate () and prolate () shapes. Furthermore, increasing the friction coefficient from 0.5 to 1.0 leads to a systematic increase in the angle of repose across all aspect ratios, as expected from the enhanced friction between particles. Quantitatively, deviations from the reference data remain below 5°. These discrepancies can be attributed to differences in the initial and loading conditions, which are known to influence the pile structure and angle of repose [77]. Furthermore, the procedure used to estimate this angle is subject to non-negligible uncertainties (on the order of several degrees), so the level of agreement obtained here can be considered very satisfactory.
Figure 18. Sand pile validation: angle of repose as a function of ellipsoid aspect ratio . Comparison between LEDDS results and the reference data of Chen et al. [76]. Shaded regions indicate the variability range (minimum to maximum) obtained from the local slope measurements.
Overall, these results demonstrate that LEDDS accurately captures the influence of particle shape and friction on the angle of repose, and reliably reproduces the collective behavior of both spherical and non-spherical granular assemblies.
Single-particle settling (LBM-DEM)
As a first validation of the fluid-particle coupling, we simulate the settling of a single sphere in a quiescent fluid, following the experimental benchmark of a nylon particle descending through silicone oil [78]. In this experiment, a sphere of diameter [mm] and density [kg/m] is released inside a rectangular container filled with oil, with dimensions [mm] in cross-section and [mm] in height. The sphere is initially at rest at a height of [mm] (see Figure 19 for the full setup).

Figure 19. Pseudo-3D sketch of the single-particle settling benchmark. A spherical particle is released at a height in a quiescent fluid contained in a rectangular domain and settles under gravity . The transient settling velocity and the interaction with the bottom wall are used to assess the accuracy of the fluid-particle coupling.
This benchmark is chosen not only to assess the particle’s terminal (maximum) settling velocity, but also to evaluate the full transient dynamics of the descent. An accurate fluid-particle coupling must capture reasonably well the initial acceleration, the subsequent approach to steady state, and the hydrodynamic interaction with solid boundaries during the final stage of motion. Furthermore, this benchmark allows us to identify the key parameters governing the accuracy and efficiency of the coupling strategy (PSM), so that we can propose a practical trade-off between particle resolution (i.e., the number of fluid cells per particle diameter) and the fluid time-step, which in LBM is controlled through the lattice velocity . This balance ensures reliable accuracy while keeping the computational cost to a minimum.
In our numerical setup, gravity is applied only to the particle, not to the fluid. Consequently, the buoyancy force is not generated ated through the fluid’s hydrostatic pressure field but is instead imposed analytically as
where is the fluid density, and is the particle volume. This buoyancy force is then subtracted from the gravitational force acting on the particle. This approach notably reduces the amplitude of spurious pressure waves generated at initialization, and which are inherent to any compressible fluid solver such as LBM.
We will evaluate the settling behaviour under four different fluid conditions, varying both density and viscosity as listed in Table 2. For each case, the Reynolds number is computed using the particle’s settling velocity .
| Case | [kg/m] | [Pa·s] | [m/s] | Reynolds |
| 1 | 970 | 0.373 | 0.036 | 1.5 |
| 2 | 965 | 0.212 | 0.057 | 4.1 |
| 3 | 962 | 0.113 | 0.087 | 11.6 |
| 4 | 960 | 0.058 | 0.122 | 31.9 |
Table 2. Fluid parameters considered for the particle settling setups.
Figure 20 shows the time evolution of the settling velocity for all four cases. The particle is resolved with fluid cells and simulated using a characteristic LB velocity of . The numerical results (solid lines) are compared directly with the experimental measurements of ten Cate et al. [78] (symbols).

Figure 20. Single particle settling simulation: Time evolution of the settling velocity for a particle resolved with fluid cells and an LB velocity of . Simulation results (solid lines) are compared against the reference data (dots) of ten Cate et al. [78] for several Reynolds numbers.
Globally speaking, the simulation results show very good agreement with the experimental measurements across all
Reynolds numbers for all the different stages. More quantitatively, a slight delay is observed for initial stage of the motion with the highest Reynolds number. This behavior is likely due to the initial pressure disturbance generated when the particle begins to move: at higher Reynolds numbers, viscous damping is weaker, allowing this pressure wave to persist longer and momentarily affect the early acceleration. Conversely, for the lowest Reynolds numbers, the particle tends to decelerate slightly earlier as it approaches the bottom wall. This is expected in high-viscosity regimes, where lubrication effects become more pronounced; since no explicit lubrication model is used here, the near-wall resistance is slightly over-predicted, causing an earlier reduction in velocity (see Section 3.2 of Ref. [39] for more details). In all cases, the particle terminal velocity is very well predicted by LEDDS.
Interestingly, these trends are consistent across a wide range of numerical parameters, including particle resolutions from to 30 and lattice velocities from 0.002 to 0.08, as detailed in Appendix § 9. For most Reynolds numbers, the simulation runtime can be reduced by more than a factor of 10 by lowering the particle resolution to and increasing the lattice velocity to , while still maintaining good accuracy.
Ellipsoid evolution in a shear flow (LBM-DEM)
As an additional validation of the fluid-solid coupling, we consider the rotational dynamics of an ellipsoid immersed in a viscous shear flow. This configuration is a classical benchmark for non-spherical particle dynamics, as the motion of an ellipsoid at vanishing Reynolds number is governed by the analytical solution derived by Jeffery [79]. These so-called Jeffery orbits describe the periodic rotational motion of the particle and provide an excellent test of a numerical method’s ability to accurately capture hydrodynamic torques on anisotropic solids.
As illustrated in Figure 21, the setup consists of a prolate ellipsoid with semi-axes [m] and [m] (aspect ratio of 2) placed at the center of a rectangular domain of size , with [m]. Such a large domain notably prevents the influence of the no-slip (bounce-back) boundaries on the ellipsoid dynamics. The domain is periodic in the -direction, allowing the ellipsoid to rotate freely. The rotation of the prolate is controlled by imposing a steady Couette flow through a constant velocity difference between two parallel walls: the upper wall moves at , with a shear rate [s], while the bottom wall remains stationary. The fluid velocity is initialized with a linear profile, while the ellipsoid is left in a rest state.

Figure 21. Schematic representation of the Jeffery orbit simulation. A prolate ellipsoid is placed at the center of a plane channel of height , bounded by two horizontal walls. A steady Couette flow is imposed by moving the upper wall with velocity while the lower wall remains at rest, generating a linear shear profile that drives the periodic rotation of the ellipsoid.
In this Stokes regime, the rotation of a prolate ellipsoid induced by a shear flow is governed by Jeffery’s theory, which provides analytical expressions for the orientation angle about the -axis and its angular velocity [79]:
The characteristic time scale of the rotational motion is the Jeffery period, , which corresponds here to the time required for the ellipsoid to complete one -orientation cycle (“Jeffery orbit”). It is derived by defining the inner phase , and noting that the tangent function in the solution is periodic. Requiring that the ellipsoid completes a rotation corresponding to a change in , i.e.,
To remain in the Stokes regime, the fluid and particle densities are set equal to eliminate buoyancy effects, and the simulation parameters are chosen such that the Reynolds number is fixed at , thereby minimizing inertial contributions. As with the settling benchmark, such low Reynolds numbers require small time steps to maintain accuracy. We therefore set , ensuring for all tested resolutions of the ellipsoid. Each simulation is run for a non-dimensional time of 8, corresponding to 5000 [s] of physical time (about 1.27 for the chosen parameters).
Figure 22. Simulation of Jeffery orbit at Reynolds Re = 0.08 for various fluid cell resolutions of the minor axis . Top: Time evolution of the angular velocity . Bottom: Angular velocity as a function of its orientation .
Figure 22 presents the angular velocity of the prolate ellipsoid as a function of its orientation angle, together with the corresponding time evolution of the particle rotation, for several spatial resolutions (defined as the number of fluid cells across the particle’s minor axis ). At low resolutions, the rotation rate is slightly overestimated, which results in a Jeffery period shorter than the theoretical prediction. This behavior stems from insufficiently resolved hydrodynamic torques acting on the particle. Once the resolution reaches 12 cells per minor axis (or higher) the agreement with the analytical Jeffery solution becomes significantly more accurate: both the angular-velocity profile and the oscillation period closely match the theoretical curves. Further refinement provides only minor improvements, indicating that the essential hydrodynamic torques governing the particle motion are already well captured at these moderate resolutions.
These results are fully consistent with the trends observed for spherical particles in the settling benchmark (§ 4.6 and Appendix § 9), where resolutions of approximately 10 fluid cells per particle diameter provided a good balance between accuracy and computational cost. Here, despite the added geometric complexity of an anisotropic particle, similar resolution levels for the minor axis remain sufficient to reproduce the correct rotational dynamics. This benchmark therefore confirms the accuracy of the solid-fluid coupling (PSM) for non-spherical particles and demonstrates that the framework accurately captures torque-driven motion of single particles in viscous flows.
Effective viscosity in a shear flow (LBM-DEM)
The final validation test considered in this work combines the many-body contact dynamics of § 4.4 with the fluid-particle interaction encountered in § 4.6. More precisely, we examine the effective rheology of a suspension of rigid spheres subjected to a steady shear flow [80, 81]. This configuration is widely used to assess the ability of a coupled LBM-DEM solver to reproduce bulk suspension properties, such as the increase in apparent viscosity with particle volume fraction [82, 83, 84, 85].
The suspension consists of neutrally buoyant spheres of radius [m] placed in a Newtonian fluid of dynamic viscosity [Pa·s]. The particles then have density equal to that of the fluid, and gravity is omitted. The solids are modeled with Poisson ratio , Young’s modulus [MPa], friction coefficient , and restitution coefficient . While these parameters are chosen to mimick non-elastic aspects of real particle collisions, the result shows little sensitivity to specific collision parameters in practice, as the interaction with the viscous fluids dominates energy dissipation. The fluid domain is discretized by lattice nodes, ensuring that each particle diameter is resolved by 10 fluid cells. The LBM time step is [s], and the DEM solver performs 10 substeps per fluid iteration to warrant the stability of the DEM solver.
To characterize the macroscopic rheology of the suspension, we compute the effective viscosity from the mean wall shear stress, which is directly related to the time-averaged streamwise velocity profile . In a Newtonian fluid, the wall shear stress is defined as
However, directly approximating the gradient from the first off-wall node introduces large discretization errors because of the moderate resolution. To obtain a more accurate estimate of the wall shear rate, we then fit a second-order polynomial through the wall node and the first two interior fluid nodes and evaluate

Figure 23. 3D Paraview shear flow illustrations: Instantaneous particle configurations for two volume fractions and . Only the solid phase is shown; particles are colored by their streamwise velocity component , with red indicating faster motion and blue indicating nearly quiescent regions.
its derivative analytically at the wall location,
where is the linear coefficient of the fitted polynomial. The effective suspension viscosity is finally obtained from
where is the measured wall shear rate. For validation, this quantity is compared to the classical Krieger-Dougherty model [86],
where is the particle volume fraction, being the maximum packing fraction for monodisperse spheres, and the intrinsic dynamic viscosity.
While generating initial particle configurations by random placement (with velocities matched to the Couette profile) is straightforward in the dilute regime, this approach becomes increasingly inefficient for volume fractions above due to geometric crowding. To robustly initialize dense suspensions, we successively rely on two pre-processing steps. First, a sedimentation procedure ensures all particles are accommodated within the domain without overlap. Second, a brief relaxation stage with small random velocities is applied to remove any residual spatial bias in the particle arrangement before the shear flow is imposed. Once both particles and fluid have been properly initialized, all simulations are performed over approximately convective times, which corresponds to 3 million LBM time iterations. Although this duration is insufficient to obtain statistically converged data, the statistical homogeneity of the flow in the - and -directions allows us to average over these directions to get sufficient statistical convergence for analysis.
The resulting numerical measurements are compared with the theoretical prediction in Figure 24 for volume fractions ranging from to . Perfect match is obtained for (dilute regime), and very good agreement with the Krieger-Dougherty model is obtained for higher volume fractions.
Figure 24. Evolution of the apparent viscosity as a function of the volume fraction . Simulation data (symbols) are compared to the Krieger-Dougherty semi-analytical law [81].
Performance analysis
To assess the computational efficiency and scalability of LEDDS, we perform a series of performance benchmarks using a fluidized-bed configuration across a range of solid volume fractions [64]. The analysis is conducted on both a 128-core AMD EPYC 7742 CPU and an NVIDIA A100-SXM4 (80GB) GPU. The CPU benchmarks provide a reference baseline for speedup evaluation and highlight the effects of floating-point precision on performance. The GPU benchmarks, in turn, quantify the benefits of hardware acceleration and Thrust’s GPU-optimized primitives, evaluate precision-dependent performance, and provide a direct comparison with waLBerla to gauge the efficiency of LEDDS’s high-level, portable parallelization strategy relative to a highly optimized CUDA implementation. Eventually, the A100 GPU card was chosen notably because it provides native double precision (FP64) support with a peak throughput approximately half that of single precision (FP32), which allows for meaningful comparisons between FP64 and FP32 performance. Additional results on the GPU memory footprint of LEDDS are provided in Appendix § 10.
Numerical setup for performance comparisons
The performance benchmarks are based on a fluidized-bed configuration at various solid volume fractions, following closely the setup of the performance study conducted with waLBerla [64]. This reference is particularly well suited for comparison, since both LEDDS and waLBerla rely on (1) the PSM-based fluid-particle coupling strategy, and (2) the distance-based solid-fraction computation (46). These methodological similarities then enable us to propose a meaningful GPU performance comparison against a state-of-the-art CUDA implementation.
The simulation parameters are chosen to match the reference study, i.e., the particle diameter is resolved by 20 fluid cells, and the DEM solver performs 10 sub-steps per LBM iteration. The computational domain spans fluid cells (about 80 millions) and is periodic in the - and -directions. No-slip bounce-back walls are imposed in the -direction. The Reynolds number is set to , gravity acts in the negative -direction with magnitude [m/s], and the particle-to-fluid density ratio is fixed at 1.1.
The benchmark covers particle volume fractions from up to , corresponding to a maximum of 8494 spheres. This range provides insight into the computational scaling of the PSM coupling strategy from dilute to highly packed regimes. For volume fractions above , we use the same sedimentation-relaxation strategy to get the initial positioning of particles as for the shear flow simulations conducted in § 4.8.
Performance metric and compilation flags
To ensure consistent and reproducible performance measurements, all benchmarks are executed after an initial warm-up phase designed to remove startup overheads such as memory allocation, or data transfers. On the GPU, 500 warm-up iterations are performed before measurements begin, followed by 2500 iterations used to compute the reported execution times. On the CPU, 100 warm-up iterations are used, and performance is evaluated over 500 iterations. These iteration counts are sufficiently large to average out short-term fluctuations while keeping the benchmarking stage computationally affordable.
Performance results are reported in lattice updates per second (LUPS), the standard metric in the LBM community that measures how many fluid cells are updated per second. Because this number is typically very large, especially on modern GPUs, it is common to express it in millions (MLUPS). Although the MLUPS performance metric does not directly quantify the cost of the DEM component, it reflects the overall performance of the coupled LBM-DEM simulation because it is computed from the total wall-clock time per iteration. As a result, all operations (LBM, DEM, and PSM-based coupling) are implicitly included in the metric. This makes MLUPS suitable for comparing different configurations and for assessing the overhead introduced by particle-fluid interactions as the solid volume fraction increases.
All simulations are compiled with the nvc++ compiler (version 24.9). It should be noted that the current version of LEDDS is incompatible with older versions of nvc++, while upward compatibility is currently guaranteed. LEDDS is built for the GPU using the stdpar=gpu backend and for multicore CPU execution with the stdpar=multicore backend. Both builds use the following optimization flags fast and O3 which enables a collection of aggressive optimizations. The only hardware-specific flag is -march=native, which is applied on CPUs to allow generation of machine code optimized for the host architecture. These settings provide a consistent and performance-oriented compilation strategy across platforms while preserving C++ standard compliance. CPU benchmarks are performed on a single AMD EPYC 7742 node at 2.25 [GHz], using 128 cores. This choice of CPU hardware provides a node-level comparison with the server-class A100-SXM4 (80GB) GPU.
CPU performance
The first analysis investigates how the solid volume fraction affects overall CPU performance and assesses the potential benefits of using reduced floating-point precision.
Figure 25 shows that increasing has a substantial impact on performance whatever the precision: compared to pure-fluid simulations (), the MLUPS rate decreases by a factor of 5 to 10 at . This degradation reflects the growing cost of DEM updates and particle–fluid coupling, which progressively overshadow the memory-bound LBM module as the suspension becomes more packed.
Figure 25. CPU performance comparison of single (FP32) and double (FP64) precision for STL implementation. MLUPS are shown as a function of the solid volume fraction .
In dilute regimes, where the LBM dominates and performance is mainly limited by memory bandwidth, switching from FP64 to FP32 leads to a speedup of about 1.5. As the solid volume fraction increases, however, this advantage gradually diminishes. In dense configurations, the runtime becomes increasingly dominated by DEM-related operations such as collision-list construction, contact handling, and force reduction. Since these operations rely heavily on integer keys, sorting, binary searches, and irregular memory accesses, their cost does not scale as favorably with floating-point precision as the LBM update does. As a result, FP32 brings a clear performance benefit in dilute flows, while in densely packed suspensions it is mainly advantageous because it reduces memory consumption and thus increases the number of particles and fluid cells that can be handled on a given architecture.
GPU performance
Continuing with the GPU performance analysis of LEDDS, we first compare the use of STL algorithms and Thrust primitives for the DEM and PSM-based coupling modules.
Figure 26 compares both approaches using double precision (FP64). While both implementations achieve very high and nearly identical performance for pure fluid simulations (), performance rapidly decreases as the solid volume fraction increases, and significant differences emerge. The results demonstrate that the STL/Thrust version, relying on the reduce_by_key algorithm and a radix-sort based version of sort from the Thrust library, consistently outperforms the pure STL version on GPU, achieving an almost constant speedup of about 2. Based on this observation, Thrust algorithms are used for particle/force sorting and reduction operations in all subsequent GPU benchmarks, while the remaining parts of LEDDS continue to rely on STL algorithms, as summarized in Table 1.
Figure 26. GPU performance comparison of a pure C++ STL implementation against an accelerated version, using the reduce_by_key algorithm and a radix-sort based version of sort from the Thrust library. Both implementations use double precision (FP64).
Building on the previous analysis of algorithm selection, we next investigate the impact of floating-point precision on GPU performance of the STL/Thrust version of LEDDS. Figure 27 presents MLUPS results for LEDDS using single (FP32) and double precision (FP64). In dilute configurations (), where the solver is predominantly memory-bound due to the LBM updates, switching from FP64 to FP32 yields a significant performance gain similar to the one obtained on CPU. As the solid volume fraction increases, however, the relative advantage

Figure 27. GPU performance of the STL/Thrust version of LEDDS in single (FP32) and double precision (FP64) across varying solid volume fractions. Both set of data were generated with the STL/Thrust version of LEDDS.
of FP32 gradually decreases because the computational cost becomes dominated by particle-fluid coupling and DEM updates. For high volume fractions (), the performance improvement decreases to approximately 30%. Consequently, as observed for CPU simulations, FP32 offers significant speedups in dilute regimes, while in dense suspensions it primarily reduces memory usage without substantially impacting overall performance.
GPU speedup
The primary motivation for employing algorithmic primitives (from STL and Thrust) is to develop a simulation framework capable of achieving substantial, nearly out-of-the-box GPU acceleration. Figure 28 reports the speedup obtained for the optimal CPU (STL-only) and GPU (STL/Thrust) implementations of LEDDS. Results are presented for both FP32 and FP64 precision, across solid volume fractions ranging from dilute suspensions () to densely packed ones ().
Figure 28. GPU speedup relative to the CPU baseline. Results are shown for FP32 and FP64 precision and for dilute () and dense () suspensions.
These benchmarks indicate that running LEDDS on an A100 GPU provides performance comparable to 11-18 CPUs with 128 cores for FP64, with speedups reaching up to 23 for FP32 in the dense case ().
In practical terms, the measured speedups correspond to node-level performance gains of approximately 10-20× when comparing the A100-SXM4 (80GB) to a 128-core CPU node. While the CPU version of LEDDS could likely benefit from additional optimization, it nevertheless provides a consistent reference for comparison. The overall conclusion remains that the use of algorithmic primitives enables substantial runtime acceleration on GPUs, without sacrificing portability or ease of implementation.
Comparison with CUDA-based solver
To have a more global view on the GPU performance of LEDDS, we compared it against the state-of-the-art implementation in waLBerla [64], which employs a hybrid GPU-CPU approach: the LBM and coupling modules are highly optimized on the GPU using CUDA, while an optimized DEM module runs on the CPU, with communication overhead effectively hidden (see their Figure 7).
To ensure correct usage of waLBerla, we first reproduced the performance results reported in Ref. [64] on the same hardware: A100-SXM4 (40GB) GPU. Following this verification, we measured waLBerla’s performance on an A100-SXM4 (80GB) GPU paired with an AMD EPYC-7742 CPU (64 cores), with LEDDS performance solely evaluated on the GPU.
For a broader performance comparison, simulations were conducted at solid volume fractions ranging from to by adjusting the particle generation spacing, while remaining within the range studied in previous studies. It should be noted that in waLBerla, only the particle spacing can be specified. The number of particles, and hence the actual solid volume fraction, is automatically determined by the framework. This makes a one-to-one comparison at exactly the same challenging.
The results, shown in Figure 29, indicate that LEDDS generally achieves performance within a factor of two of waLBerla using FP64. Importantly, the gap between the two solvers systematically decreases as the suspension becomes denser, and at the upper end of the tested regime their performance becomes comparable. This demonstrates that a design based on high-level algorithmic primitives can achieve GPU performance in the same order of magnitude as a hand-optimized CUDA-CPU hybrid framework, while offering good levels of portability and maintainability.
Figure 29. GPU performance comparison between LEDDS and waLBerla framework [64]. MLUPS are reported as a function of the solid volume fraction for FP64 precision.
Summary and outlook
This work presented LEDDS, a portable and fully algorithmic LBM-DEM framework for GPU-accelerated simulations of fluid-particle systems. The central philosophy underlying LEDDS is the systematic decomposition of all computational stages into high-level algorithmic primitives. By expressing neighbor search, collision detection, DEM force evaluation, fluid-solid coupling and lattice Boltzmann updates solely in terms of map, sort, scan and reduction operations, the entire codebase becomes free of device-specific kernels and relies only on compiler-supported parallelism. This design demonstrates that complex multiphysics solvers can be implemented in a manner that is simultaneously computationally efficient, readable and portable, and thus establishes a practical methodology that can be adopted by other scientific computing codes targeting heterogeneous platforms. The performance analysis confirms that the primitive-based implementation reaches throughput comparable to optimized CUDA solvers for both D3Q19-LBM and DEM components, even in demanding fluid-particle benchmarks. These results suggest that high-level algorithmic programming, when supported by efficient backend runtimes, is sufficient to deliver very good performance without resorting to handcrafted GPU kernels.
Regarding future HPC-related developments, we will focus on extending the framework toward distributed-memory, multi-GPU execution using MPI-enabled domain decomposition, which is essential for simulations involving large particle counts and high-resolution fluid grids. The algorithmic character of the implementation is expected to ease this transition, as most data-dependent operations are already structured as bulk-synchronous primitives. The memory footprint of the fluid module could be further reduced by adopting one of the “single-population” LBM implementations [66], so that even larger systems can be simulated on a single GPU. Another important direction is to extend the investigation of performance portability across GPU architectures from multiple vendors. Although the current implementation targets NVIDIA GPUs, the use of high-level algorithmic primitives makes the code largely platform-agnostic. Portability on AMD and Intel GPUs will first be assessed for pure LBM, before extending the analysis to coupled LBM-DEM simulations.
From a physical modeling standpoint, the representation of complex geometries through sphere packings and composite solids is of particular interest, as the current PSM formulation and uniform-grid DEM structures naturally accommodate such extensions. This would broaden the applicability of LEDDS to more realistic particulate systems commonly encountered in geophysics, process engineering, and biomechanics. Preliminary demonstrator simulations in a simplified arterial geometry, using both axisymmetric ellipsoids and a first approximation of biconcave particles mimicking red blood cells, illustrate the potential of the framework notably for biofluidic applications (see Figure 30). A final line of research aims at exploring the behavior of the solver for high density ratios, as it was previously done with an IBM-based approach [87].

Figure 30. 3D Paraview visualizations from demonstrator biofluidic flow simulations in a simplified arterial geometry using LEDDS. Left: Flow with axisymmetric ellipsoids colored by their velocity magnitude, ranging from low values near the vessel walls (blue) to higher values in the central region (red). Right: Preliminary simulation of more realistic red blood cell flows using biconcave particle shapes.
Overall, LEDDS provides evidence that algorithmic primitives constitute a robust foundation for portable, high-performance multiphysics computing. The approach outlined in this work may therefore serve as a blueprint for future scientific software targeting heterogeneous architectures and emerging accelerator technologies.
Acknowledgements
The authors would like to thank the reviewers for their careful reading of the manuscript and their constructive comments, which helped improve the quality and clarity of this work. This research was supported by the SNSF grant 200020_197223, “Numerical simulation of melt extraction in magmatic crystal mush”. C.C. further acknowledges financial support from the BNBU Research Fund n°UICR0700094-24 entitled “GPU-accelerated multidisciplinary methods for industrial computational fluid dynamics”.
Material properties
The elastic and damping coefficients (, , , ) can be obtained from the material properties of the solids which are the Young’s modulus and the Poisson’s ratio. Heinrich Hertz was the first to derive formulas that link the deformation to the force exerted between two spheres [88]. The following formulas allow obtaining the damping and spring coefficients from the physical properties of the materials of the spheres and [89, 90]3:
where [91]:
Figure B.31. FP64 vs FP32 comparison for a head-on sphere collision. Top: Time evolution of energy components (, , ). Bottom: Relative error normalized by the total energy, .
Figure B.32. FP64 vs FP32 comparison for an eccentric ellipsoid collision. Top: Time evolution of energy components (, , , ). Bottom: Relative error normalized by the total energy, .
where is the distance between the surfaces of the particles given by equation (6), and are the Young’s modulus of the particles, and are the Poisson’s ratio, and are the masses, and and are the radii. As specified in Eq. (3), for ellipsoid the radius is locally approximated by the radius of an equivalent sphere. The parameter is the coefficient of restitution of the collision. Formally, it corresponds to the ratio of relative velocity of the spheres after the collision to the relative velocity of the particles before the collision. The coefficient of restitution is within the interval . When it equals 1, the collision is perfectly elastic, and the damping terms and are zero. When epsilon is smaller than 1, energy loss occurs during the collision, and it is no longer perfectly elastic. If the particles and have different coefficient of restitution, we simply use the arithmetic mean of the two coefficients [89]:
Single precision
As the present version of LEDDS relies on single-GPU executions, it is essential to exploit GPU capabilities as efficiently as possible. In this context, the use of single precision is highly desirable, as it not only accelerates simulations but also significantly reduces memory consumption, thereby allowing larger numbers of particles and fluid cells to be simulated on the same hardware. However, its impact on the accuracy of DEM collision modeling must be carefully assessed, since contact resolution directly affects energy conservation and overall physical fidelity.
To this end, we compare the evolution of kinetic and elastic energies during representative collision events for both single precision (FP32) and double precision (FP64). Two benchmark cases are considered: a head-on collision between spheres (§ 4.1) and an eccentric collision between ellipsoids (§ 4.2). For each case, we report both the energy evolution and a normalized error measure defined as , which avoids spurious amplification when individual energy components approach zero. The spherical and ellipsoidal cases are shown in Figures B.31 and B.32, respectively.
For both benchmark cases, the energy evolutions obtained with FP32 and FP64 are nearly indistinguishable, indicating a very close agreement between the two precision levels. The normalized error remains small throughout the simulations. In the ellipsoidal case, deviations are observed during the collision because the transition between contact regimes is more sensitive to numerical precision, in particular due to orientation matrix manipulations and non-spherical contact mechanics. A similar, though less pronounced, behavior is observed for spherical particles, with deviations two orders of magnitude smaller than for the non-spherical case. This is primarily attributed to the simpler and more robust numerical treatment of spherical contact mechanics.
Importantly, the error remains bounded in time and vanishes after contact, indicating that no long-term drift or energy bias is introduced by single precision. These results demonstrate that FP32 provides sufficient accuracy for DEM collision modeling in the present framework, while enabling improved performance and larger simulation sizes on GPU architectures. For pure fluid (LBM) simulations, the use of FP32 is also standard practice, and we have shown in previous works that LBM solvers based on modern C++ can accurately simulate a wide range of flow regimes with FP32, from aerodynamics to aeroacoustics and even supersonic flows [29, 30, 31, 32, 92]. Additional studies conducted on single-particle settling (§ 4.6) further extend these observations to coupled LBM-DEM simulations, although the corresponding results are not shown here for brevity.
Figure C.33. Single particle settling simulations: Impact of the LB velocity (time step) on the accuracy of LEDDS simulations for a fixed resolution of the particle ().
Figure C.34. Single particle settling simulations: Impact of the particle resolution, , on the accuracy of LEDDS simulations for a fixed LB velocity ().
Parametric study for particle settling
Here, we present parametric studies to identify an optimal balance between accuracy, robustness, and computational efficiency for the particle settling benchmark, but which can more generally be used for simulations with fluid-particle interactions. Such studies are particularly critical in LBM-DEM coupling, where the LBM velocity () controls the time step, which must be sufficiently small to accurately capture the Stokes regime, yet excessively small steps also reduce computational efficiency. Similarly, the ratio of particle diameter to fluid cell size () strongly affects the accuracy of the fluid–particle coupling, while higher resolution increases computational cost and memory usage. Thus, practical simulations require a careful trade-off between accuracy and efficiency.
We first quantify the impact of the LB velocity on the particle settling simulations for a fixed particle resolution of (Figure C.33). Reducing improves accuracy in the low Reynolds number regime (), where viscous forces dominate. In higher Reynolds number regimes, where inertial effects are more significant, the results are relatively insensitive to the choice of . Overall, a value of to represents a good compromise between accuracy and computational efficiency, consistent with the main text observations regarding transient dynamics and the early-stage pressure-wave effects.
Next, we investigate the effect of particle resolution at a fixed lattice velocity (Figure C.34). For resolutions of at least 10 fluid cells per particle diameter, the terminal settling velocity is essentially unchanged, while small deviations appear near the no-slip boundary at the bottom wall. Resolutions between and 20 provide a good compromise for capturing both transient dynamics and near-wall effects in the Stokes regime, while keeping the memory footprint reasonable. Indeed, reducing the resolution from 20 to 10 reduces the fluid memory requirement by a factor of approximately 8, with minimal loss of accuracy, as also highlighted in the main text.
In summary, if the primary objective is to solely recover the correct terminal settling velocity, simulations can be accelerated by several orders of magnitude when moving from the most demanding parameter set to a more relaxed configuration such as . As a general best practice for LBM-DEM simulations with PSM coupling, we recommend using together with –, which provides a good balance between accuracy and efficiency. Lower resolutions such as should be restricted for large scale particle laden flows –typically involving hundreds/thousands of particles– where a resolution of would exceed the memory capacity of a single GPU, and the corresponding runtime would be prohibitively long.
GPU memory usage
Memory usage is a key constraint for GPU-based simulations. Since LEDDS currently targets single-GPU executions, this metric directly determines the class of GPUs required to run a given simulation and, consequently, the range of feasible configurations. In this work, we rely on high-end GPUs with large memory capacity (e.g., NVIDIA A100-SXM4 with 80 GB), which allows us to prioritize computational performance and simplified data structures, at the cost of a higher memory footprint. In the following, we first derive simple memory estimates for the LBM and DEM modules separately, and then combine them into a global model that provides order-of-magnitude predictions for the total memory requirement of a given simulation.
| Variable | Comp. | Mem. [B] | Usage |
| Type | 1 | 1 | DEM |
| Position | 3 | DEM | |
| Orientation | 9 | DEM | |
| Velocity | 3 | DEM | |
| Angular vel. | 3 | DEM | |
| Acceleration | 3 | DEM | |
| Angular accel. | 3 | DEM | |
| Semi-axes | 3 | DEM | |
| Inertia | 3 | DEM | |
| Mass | 1 | DEM | |
| Young mod. | 1 | DEM | |
| Poisson ratio | 1 | DEM | |
| Restitution coeff. | 1 | DEM | |
| Friction coeff. | 1 | DEM | |
| Force | 3 | DEM | |
| Torque | 3 | DEM | |
| Fluid force | 3 | Coupling | |
| Fluid torque | 3 | Coupling | |
| DEM | |||
| Coupling | |||
| Total |
Table D.3. Memory footprint per particle (DEM and coupling variables). Comp.: number of components; Mem.: memory in Bytes; : size of a floating-point value (4 Bytes for FP32, 8 Bytes for FP64).
DEM module: Theoretical models
The memory requirements of the DEM module are summarized in Table D.3. The minimal memory footprint per particle reads
where denotes the size (in Bytes) of a floating-point value (4 for FP32 and 8 for FP64). This corresponds to 189 [B] in FP32 and 377 [B] in FP64. The memory associated with fluid–particle coupling is
which represents a fraction
of the total memory per particle. This corresponds to approximately 13%, independently of the precision, which indicates that the coupling overhead remains relatively small within the DEM module. Based on these values, one gigabyte of memory can store on the order of particles in FP32 and in FP64, providing a useful rule of thumb for estimating the memory requirements of the DEM module.
Figure D.35. GPU memory usage (top) and corresponding memory efficiency, expressed as the number of spheres per [GB] (bottom), as functions of system size.
DEM module: Measurements
Building on this baseline, we now assess the effective memory usage in practical DEM simulations. The following measurements quantify the impact of dynamic data structures on the overall GPU memory footprint.
In this setup, spheres are initially axis-aligned and distributed on a uniform grid used for collision detection. A one-to-one mapping between particles and grid cells is enforced by selecting a DEM cell size (see § 3.2.1) slightly larger than the particle diameter (). This configuration limits the number of non-physical candidate pairs generated during collision detection and controls the number of particles per cell, while preserving numerical stability. In contrast to the benchmark in § 4.4, the computational domain is fully periodic and does not include walls, thereby avoiding additional dynamic memory allocations associated with particle-wall interactions. The total number of spheres is varied from approximately 8000 to a bit more than 4 millions by increasing the domain size.
Figure D.35 reports both the total GPU memory usage and the corresponding memory efficiency (number of spheres per [GB]), as functions of the number of simulated spheres. In both precisions, the memory footprint increases markedly with system size, from about 0.54 to 14.4 [GB] in FP64 and from about 0.59 to 9.70 [GB] in FP32 as the number of spheres increases. The difference between FP32 and FP64 becomes more pronounced for large systems, while for small cases both curves remain relatively close because the fixed memory overhead of the application is still significant. Focusing on the memory efficiency (number of spheres per [GB]), it increases with system size which reflects the progressive amortization of fixed overheads. However, even for the largest configurations, the achieved efficiency remains about one order of magnitude below the prediction of the particle-based memory model.
| Var. | Comp. | Mem. [B] | Use |
| Type | 1 | 1 | LBM |
| Pop. | LBM | ||
| 1 | Coupling | ||
| ids | 1 | Coupling | |
| 1 | Coupling | ||
| 1 | 1 | Coupling | |
| Force | 3 | Coupling | |
| Torque | 3 | Coupling | |
| Force ids | 1 | Coupling | |
| #Force | 1 | 4 | Coupling |
| Buffer | 1 | Coupling | |
| LBM | |||
| Coupling | |||
| Total |
Table D.4. Memory per fluid cell. Comp.: number of components; Mem.: memory in Bytes; Use: usage (LBM: fluid solver, Coupling: fluid-particle coupling). : size of a floating-point value (4 for FP32, 8 for FP64); : number of lattice populations; : maximal number of solids per fluid cell; : solid fraction per solid; : total solid fraction.
A more quantitative view is obtained by examining the effective memory cost per sphere. For the largest simulation, the measured memory usage corresponds to about spheres per [GB] in FP64 versus spheres per [GB] in FP32, i.e., about [kB] and [kB] per sphere, respectively. This is roughly one order of magnitude larger than the ideal per-particle estimate derived from the static memory model (D.1). The discrepancy is mainly explained by the storage of collision pairs: even in the present simplified configuration, up to candidate pairs are generated and stored, accounting for approximately 80% of the total memory usage.
Overall, these results indicate that omitting dynamic memory contributions, notably those arising from collision-pair storage and contact handling, results in a substantial underestimation of the actual memory footprint. In practice, this discrepancy can be compensated for by applying a simple correction factor of about 10 to the particle-based model (D.1), leading to more realistic estimates of DEM memory requirements.
LBM and LBM-DEM: Theoretical models
As detailed in Table D.4, the memory footprint per fluid cell is given by the general expression
where is the number of lattice populations, is the number of solids that overlap the fluid cell, and denotes the size (in Bytes) of a floating-point value (4 for FP32 and 8 for FP64).
Introducing , , and as the fractions of fluid cells containing zero, one, and two solids (the latter being the hard implementation limit in LEDDS), the average memory per fluid cell can be expressed as
In fully resolved LBM-DEM simulations, particles are typically much larger than fluid cells, so most cells contain at most one solid (), and , where is the solid volume fraction. This leads to the practical approximation
and, for the D3Q19-LBM used in LEDDS, this gives:
(fluid cell): 162/318 [B] for FP32/FP64,
(typical case): 202/386 [B] for FP32/FP64,
(upper bound): 242/454 [B] for FP32/FP64.
Focusing only on the memory associated with fluid-particle coupling per fluid cell, it reads
which represents about 35% of the total memory per cell in the worst case (), and about 20-25% for the more realistic case . Since most cells contain at most one or no solid, the effective coupling overhead is expected to remain moderate in practice, typically below 30% of the pure fluid memory footprint when accounting for a safety margin.
Based on the above estimates and assuming an average coupling overhead of about 30%, the effective memory per fluid cell is on the order of 200 [B] in FP32 and 400 [B] in FP64 for a D3Q19-LBM. This corresponds to a storage capacity of approximately fluid cells per [GB] in FP32 and in FP64. These values provide a useful rule of thumb for estimating the memory requirements of LBM-DEM simulations performed with LEDDS.
Combining both contributions, the total memory usage of the LBM-DEM model can be approximated as
where and denote the number of fluid cells and particles, respectively. Since and are of the same order of magnitude, the relative contribution of each module is primarily governed by the ratio . In practice, fully resolved LBM-DEM simulations typically require at least fluid cells per particle diameter, so that . As a result, the total memory usage is dominated by the LBM module in most practical configurations.
LBM-DEM: Measurements
Using the fluidized-bed configuration introduced in § 5, which involves up to 80 million fluid cells and approximately 8600 particles, we now evaluate the total memory usage of LEDDS under realistic conditions. In contrast to the idealized estimates above, these measurements also include the dynamic data structures required in practice for collision handling and coupling, such as the cell-linked list, collision pair lists, tangential displacement history, and temporary buffers used during sorting and reduction operations.
Figure D.36 shows the normalized GPU memory usage as a function of the solid volume fraction for different resolutions. The presence of particles leads to an increase in memory usage compared with the pure-fluid case. More precisely, increasing the solid volume fraction up to raises the total memory footprint by roughly 20-30%, depending on the resolution and precision. The dependence on the resolution is not strictly monotonic, which indicates that this increase is not governed by particle storage itself, but rather by the additional dynamic memory required for collision handling and fluid-particle coupling.
The reduction in memory footprint when switching from FP64 to FP32 remains significant, although it is smaller than a factor of two. For the pure-fluid reference case (), the memory usage decreases from 1.02, 3.94, and 27.76 GB in FP64 to 0.83, 2.41, and 15.22 GB in FP32 for , 10, and 20, respectively (see Figure D.36). This behavior is expected, as part of the memory footprint is associated with data structures (e.g., indices and buffers) that are independent of floating-point precision.
Figure D.36. Normalized GPU memory usage as a function of the solid volume fraction for different resolutions (). Top: Double precision. Bottom: Single precision
A more detailed interpretation of this increase can be obtained by separating the respective contributions of the fluid and particle data. Using the actual particle counts of the fluidized-bed configuration, namely , 2866, 5731, and 8596 for , 0.15, 0.30, and 0.45, respectively, the minimal DEM storage remains negligible compared with the fluid contribution. In FP32, the DEM memory associated with 8596 particles is only about 1.62 [MB] (and 3.24 [MB] in FP64), which does not significantly affect the total memory estimate. This confirms that the observed 20-30% increase with cannot be attributed to particle storage itself, but is primarily due to additional dynamic data structures associated with collision detection and fluid-particle coupling. A more detailed comparison can then be made against the theoretical estimate of the fluid contribution alone. Using an effective memory footprint of about 200-260 [B] per fluid cell in FP32 (including coupling and a safety margin), the expected memory usage for , 10, and 20 is approximately 0.23, 1.81, and 14.5 [GB], to be compared with the measured 0.83, 2.41, and 15.22 [GB]. The agreement improves with increasing resolution, confirming that the model captures the dominant fluid-storage contribution, while the remaining discrepancy reflects the influence of auxiliary GPU data structures.
In summary, these results confirm that the present minimal memory model provides a reliable lower bound for fully resolved LBM-DEM simulations. In practice, applying a safety factor of approximately 30-40% to the pure-fluid prediction allows for a robust prediction of the actual memory requirements for LBM-DEM simulations. This offers a simple and practical guideline to anticipate GPU memory needs and to select appropriate hardware for a given simulation.
References
- [1]P. A. Cundall, O. D. L. Strack, A discrete numerical model for granular assemblies, Géotechnique 29 (1) (1979) 47–65.DOI
- [2]H. Zhu, Z. Zhou, R. Yang, A. Yu, Discrete particle simulation of particulate systems: Theoretical developments, Chem. Eng. Sci. 62 (13) (2007) 3378–3396.DOI
- [3]H. Zhu, Z. Zhou, R. Yang, A. Yu, Discrete particle simulation of particulate systems: A review of major applications and findings, Chem. Eng. Sci. 63 (23) (2008) 5728–5770.DOI
- [4]J. Zhao, T. Shan, Coupled CFD-DEM simulation of fluid-particle interaction in geomechanics, Powder Technol. 239 (2013) 248–258.DOI
- [5]W. Zhong, A. Yu, X. Liu, Z. Tong, H. Zhang, DEM/CFD-DEM Modelling of Non-spherical Particulate Systems: Theoretical Developments and Applications, Powder Technol. 302 (2016) 108–152.DOI
- [6]P. Kieckhefen, S. Pietsch, M. Dosta, S. Heinrich, Possibilities and limits of computational fluid dynamics – discrete element method simulations in process engineering: A review of recent advancements and future trends, Annual Review of Chemical and Biomolecular Engineering 11 (1) (2020) 397–422, pMID: 32169000.
- [7]H. Ma, L. Zhou, Z. Liu, M. Chen, X. Xia, Y. Zhao, A review of recent development for the CFD-DEM investigations of non-spherical particles, Powder Technol. 412 (2022) 117972.DOI
- [8]J. Gan, Z. Zhou, A. Yu, A GPU-based DEM approach for modelling of particulate systems, Powder Technology 301 (2016) 1172–1182.DOI
- [9]L. Lu, GPU accelerated MFiX-DEM simulations of granular and multiphase flows, Particuology 62 (2022) 14–24.
- [10]S. Fu, L. Wang, GPU-based unresolved LBM-DEM for fast simulation of gas–solid flows, Chemical Engineering Journal 465 (2023) 142898.DOI
- [11]J. Nickolls, I. Buck, M. Garland, K. Skadron, Scalable parallel programming with CUDA: Is CUDA the parallel programming model that application developers have been waiting for?, Queue 6 (2) (2008) 40–53.DOI
- [12]HIP (C++ Heterogeneous-Compute Interface for Portability): https://rocm.docs.amd.com/projects/HIP/en/latest/. [link].
- [13]S. Wienke, P. Springer, C. Terboven, D. an Mey, OpenACC – First Experiences with Real-World Applications, in: Euro-Par 2012 Parallel Processing, Springer Berlin Heidelberg, Berlin, Heidelberg, 2012, pp. 859–870.DOI
- [14]R. Chandra, R. Menon, L. Dagum, D. Kohr, D. Maydan, J. McDonald, Parallel Programming in OpenMP,, 1st Edition, Elsevier, 2018.
- [15]A. Munshi, The OpenCL specification,, in: IEEE Hot Chips 21 Symposium (HCS), 2009, pp. 1–314.DOI
- [16]A. Alpay, V. Heuveline, SYCL beyond OpenCL: The architecture, current state and future direction of hipSYCL, in: Proceedings of the International Workshop on OpenCL, IWOCL ’20, Association for Computing Machinery, New York, NY, USA, 2020.DOI
- [17]G. E. Blelloch, Prefix sums and their applications, Tech. Rep. CMU-CS-90-190, Carnegie Mellon University, Pittsburgh, PA, technical Report, School of Computer Science (1990).
- [18]S. Sengupta, M. Harris, Y. Zhang, J. D. Owens, Scan primitives for GPU computing, in: Proceedings of the 22nd ACM SIGGRAPH/EUROGRAPHICS Symposium on Graphics Hardware, Eurographics Association, Aire-la-Ville, Switzerland, 2007, pp. 97–106.DOI
- [19]J. Shun, G. E. Blelloch, A simple and practical linear-work parallel algorithm for connectivity, in: Proceedings of the 26th ACM Symposium on Parallelism in Algorithms and Architectures (SPAA ’14), ACM, New York, NY, USA, 2014, pp. 143–153.DOI
- [20]D. Dhulipala, G. E. Blelloch, J. Shun, Theoretically efficient parallel graph algorithms can be fast and scalable, ACM Transactions on Parallel Computing 5 (1) (2018) 1–49.DOI
- [21]A. Brahmakshatriya, A. Brahmakshatriya, P. Sadayappan, J. D. Owens, Compilation techniques for graph algorithms on gpus, arXiv preprint arXiv:2012.07990 (2020).
- [22]A. Buluç, Linear algebraic primitives for parallel computing on large graphs, Ph.D. thesis, University of California, Santa Barbara (2010).
- [23]N. Bell, J. Hoberock, Thrust: A Productivity-Oriented Library for CUDA, in: W. mei W. Hwu (Ed.), GPU Computing Gems Jade Edition, Applications of GPU Computing Series, Morgan Kaufmann, Boston, 2012, pp. 359–371.
- [24]H. Carter Edwards, C. R. Trott, D. Sunderland, Kokkos: Enabling manycore performance portability through polymorphic memory access patterns, Journal of Parallel and Distributed Computing 74 (12) (2014) 3202–3216, domain-Specific Languages and High-Level Frameworks for High-Performance Computing.DOI
- [25]AMD, Hipstdpar: Standard c++ parallel algorithms on amd gpus, https://gpuopen.com/learn/hipstdpar/, accessed: 2025-10-14 (2023).
- [26]J. Larkin, No More Porting: Coding for GPUs with Standard C++, Fortran, and Python, in: NVIDIA GTC (spring), 2022.
- [27]J. Larkin, A. Stulova, No more porting: Accelerated computing with Standard C++, Fortran, and Python, in: NVIDIA GTC (spring), 2024.
- [28]J. Latt, C. Coreixas, J. Beny, Cross-platform programming model for many-core lattice Boltzmann simulations, PLoS One 16 (4) (2021) 1–29.DOI
- [29]J. Latt and C. Coreixas, Multi-GPU acceleration of PALABOS fluid solver using C++ standard parallelism, Computer Physics Communications 323 (2026) 110094.arxiv.org/abs/2506.09242
- [30]C. Coreixas, J. Latt, GPU-based compressible lattice Boltzmann simulations on non-uniform grids using standard C++ parallelism: From best practices to aerodynamics, aeroacoustics and supersonic flow simulations, Computer Physics Communications 317 (2025) 109833.DOI
- [31]C. Coreixas, J. Latt, Democratizing scale-resolving aerodynamics with GPU-based CFD on adaptive Octrees, in: 60th 3AF International Conference on Applied Aerodynamics, 2026.DOI
- [32]C. Coreixas, J. Latt, Democratizing computational aeroacoustics using GPU-accelerated Octree CFD solvers, in: 32nd AIAA/CEAS Aeroacoustics Conference, 2026.DOI
- [33]C. Coreixas, J. Latt, X. Shan, No more porting: GPU acceleration of lattice Boltzmann and finite difference solvers via C++ parallel algorithms, in: Symposium on high fidelity LES/DNS (HiFiLeD) for industrial relevant flow configurations, 2022.
- [34]D. Schwendener, J. Noir, J. Latt, C. Coreixas, X.-Z. Kong, Natural convection in porous media: the role of porosity and conductivity ratios in the transition from laminar to inertial convection, Journal of Fluid Mechanics 1026 (2026) A21.DOI
- [35]C. Rettinger, U. Rüde, A comparative study of fluid–particle coupling methods for fully resolved lattice Boltzmann simulations, Computers & Fluids 154 (2017) 74–89.DOI
- [36]C. Rettinger, U. Rüde, A coupled lattice Boltzmann method and discrete element method for discrete particle simulations of particulate flows, Computers & Fluids 172 (2018) 706–719.arxiv.org/abs/1711.00336
- [37]Y. Thorimbert, F. Marson, A. Parmigiani, B. Chopard, J. Latt, Lattice Boltzmann simulation of dense rigid spherical particle suspensions using immersed boundary method, Computers & Fluids 166 (2018) 286–294.DOI
- [38]D. R. Noble, J. R. Torczynski, A Lattice-Boltzmann Method for Partially Saturated Computational Cells, International Journal of Modern Physics C 09 (08) (1998) 1189–1201.DOI
- [39]C. Rettinger, U. Rüde, An efficient four-way coupled lattice Boltzmann – discrete element method for fully resolved simulations of particle-laden flows, J. Comput. Phys. 453 (2022) 110942.arxiv.org/abs/2003.01490
- [40]T. Krüger, H. Kusumaatmaja, A. Kuzmin, O. Shardt, G. Silva, E. M. Viggen, The Lattice Boltzmann Method: Principles and Practice, Graduate Texts in Physics, Springer International Publishing, Cham, 2017.DOI
- [41]C. Kloss, C. Goniva, LIGGGHTS – Open source discrete element simulations of granular materials based on LAMMPS, John Wiley & Sons, Ltd, 2011, pp. 781–788.DOI
- [42]B. Lambert, L. Weynans, M. Bergmann, Methodology for numerical simulations of ellipsoidal particle-laden flows, International Journal for Numerical Methods in Fluids 92 (8) (2020) 855–873.DOI
- [43]R. D. Mindlin, Compliance of Elastic Bodies in Contact, Journal of Applied Mechanics 16 (3) (2021) 259–268.
- [44]R. Maggio-Aprile, GPU-based simulation of dense suspensions: Coupling the DEM and the LBM in standard C++, Master thesis (2017).
- [45]L. Verlet, Computer "experiments" on classical fluids. I. Thermodynamical properties of Lennard-Jones molecules, Phys. Rev. 159 (1967) 98–103.DOI
- [46]R. D. Skeel, Integration schemes for molecular dynamics and related applications, Springer Berlin Heidelberg, Berlin, Heidelberg, 1999, pp. 119–176.DOI
- [47]X. Shan, X.-F. Yuan, H. Chen, Kinetic theory representation of hydrodynamics: A way beyond the Navier–Stokes equation, J. Fluid Mech. 550 (2006) 413–441.DOI
- [48]P. J. Dellar, An interpretation and derivation of the lattice Boltzmann method using Strang splitting, Comput. Math. Appl. 65 (2) (2013) 129 – 141.
- [49]F. Schornbaum, U. Rüde, Massively parallel algorithms for the lattice Boltzmann method on nonuniform grids, SIAM J. Sci. Comput 38 (2) (2016) C96–C126.DOI
- [50]S. Marié, D. Ricot, P. Sagaut, Comparison between lattice Boltzmann method and Navier–Stokes high order schemes for computational aeroacoustics, J. Comput. Phys. 228 (4) (2009) 1056 – 1070.DOI
- [51]A. Suss, I. Mary, T. Le Garrec, S. Marié, Comprehensive comparison between the lattice Boltzmann and Navier-Stokes methods for aerodynamic and aeroacoustic applications, Comput. Fluids 257 (2023) 105881.DOI
- [52]P. L. Bhatnagar, E. P. Gross, M. Krook, A Model for Collision Processes in Gases. I. Small Amplitude Processes in Charged and Neutral One-Component Systems, Physical Review 94 (3) (1954) 511–525, publisher: American Physical Society.DOI
- [53]C. Coreixas, B. Chopard, J. Latt, Comprehensive comparison of collision models in the lattice Boltzmann framework: Theoretical investigations, Phys. Rev. E 100 (2019) 033305.DOI
- [54]C. Coreixas, G. Wissocq, B. Chopard, J. Latt, Impact of collision models on the physical properties and the stability of lattice Boltzmann methods, Phil. Trans. R. Soc. A 378 (2175) (2020) 20190397.DOI
- [55]I. Ginzburg, F. Verhaeghe, D. d’Humières, Two-relaxation-time lattice Boltzmann scheme: About parametrization, velocity, pressure and mixed boundary conditions, Commun. Comput. Phys. 3 (2) (2008) 427–478.DOI
- [56]D. d’Humières, I. Ginzburg, Viscosity independent numerical errors for lattice Boltzmann models: From recurrence equations to “magic” collision numbers, Comput. Math. Appl. 58 (5) (2009) 823 – 840.DOI
- [57]C. Pan, L.-S. Luo, C. T. Miller, An evaluation of lattice Boltzmann schemes for porous medium flow simulation, Comput. Fluids 35 (8–9) (2006) 898 – 909, proceedings of the First International Conference for Mesoscopic Methods in Engineering and Science.DOI
- [58]Numerical simulations of particulate suspensions via a discretized Boltzmann equation. Part 1. Theoretical foundation, J. Fluid Mech. 271 (1994) 285–309.DOI
- [61]G. T. Nguyen, E. L. Chan, T. Tsuji, T. Tanaka, K. Washino, Resolved cfd–dem coupling simulation using volume penalisation method, Advanced Powder Technology 32 (1) (2021) 225–236.DOI
- [62]C. Tsigginos, J. Meng, X.-J. Gu, D. R. Emerson, Coupled LBM-DEM simulations using the partially saturated method: Theoretical and computational aspects, Powder Technology 405 (2022) 117556.DOI
- [63]B. D. Jones, J. R. Williams, Fast computation of accurate sphere-cube intersection volume, Engineering Computations 34 (4) (2017) 1204–1216, publisher: Emerald Publishing Limited.DOI
- [64]S. Kemmler, C. Rettinger, U. Rüde, P. Cuéllar, H. Köstler, Efficiency and scalability of fully-resolved fluid-particle simulations on heterogeneous cpu-gpu architectures, The International Journal of High Performance Computing Applications 39 (3) (2025) 345–363.arxiv.org/abs/2303.11811
- [65]P. Seil, LBDEM coupling: Implementation, validation, and applications of a coupled open-source solver for fluid-particle systems, 2016.
- [66]M. Holzer, Code generation in a lattice Boltzmann framework for exascale computing, PhD thesis, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), 2025.
- [67]Y. C. Chung, J. Y. Ooi, Benchmark tests for verifying discrete element modelling codes at particle impact level, Granular Matter 13 (5) (2011) 643–656.DOI
- [68]C. Rettinger, U. Rüde, An efficient four-way coupled lattice Boltzmann - discrete element method for fully resolved simulations of particle-laden flows, Tech. Rep. arXiv:2003.01490, arXiv, arXiv:2003.01490 [physics] type: article (Mar. 2020).arxiv.org/abs/2003.01490
- [69]A. Domènech-Carbó, Analysis of rolling friction effects on oblique rebound by redefining tangential restitution and friction, Physics of Fluids 31 (4) (2019) 043302.DOI
- [70]S. F. Foerster, M. Y. Louge, H. Chang, K. Allia, Measurements of the collision properties of small spheres, Physics of Fluids 6 (3) (1994) 1108–1115.DOI
- [71]L. Boltzmann, Weitere studien über das wärmegleichgewicht unter gasmolekülen, Wien. Ber. 66 (1872) 275–370.
- [72]P. Bhatnagar, E. Gross, M. Krook, A model for collision processes in gases. I. Small amplitude processes in charged and neutral one-component systems, Phys. Rev. 94 (1954) 511–525.DOI
- [73]Z. Y. Zhou, R. P. Zou, D. Pinson, A. B. Yu, Angle of repose and stress distribution of sandpiles formed with ellipsoidal particles, Granular Matter 16 (5) (2014) 695–709.DOI
- [74]Y. Grasselli, H. J. Herrmann, On the angles of dry granular heaps, Physica A: Statistical Mechanics and its Applications 246 (3) (1997) 301–312.DOI
- [75]Z. Y. Zhou, R. P. Zou, D. Pinson, A. B. Yu, Dynamic simulation of the packing of ellipsoidal particles, Industrial & Engineering Chemistry Research 50 (16) (2011) 9787–9798.DOI
- [76]H. Chen, S. Zhao, X. Zhou, DEM investigation of angle of repose for super-ellipsoidal particles, Particuology 50 (2020) 53–66.DOI
- [77]J. Horabik, P. Parafiniuk, and M. Molenda, Discrete element modelling study of force distribution in a 3D pile of spherical particles, Powder Technology 312 (2017) 194–203.DOI
- [78]A. ten Cate, C. H. Nieuwstad, J. J. Derksen, H. E. A. Van den Akker, Particle imaging velocimetry experiments and lattice-Boltzmann simulations on a single sphere settling under gravity, Physics of Fluids 14 (11) (2002) 4012–4025.DOI
- [79]G. B. Jeffery, L. N. G. Filon, The motion of ellipsoidal particles immersed in a viscous fluid, Proceedings of the Royal Society of London. Series A, Containing Papers of a Mathematical and Physical Character 102 (715) (1922) 161–179.DOI
- [80]A. Einstein, Eine neue bestimmung der moleküldimensionen, Annalen der Physik 324 (2) (1906) 289–306.DOI
- [81]I. M. Krieger, T. J. Dougherty, A mechanism for non-newtonian flow in suspensions of rigid spheres, Transactions of The Society of Rheology 3 (1) (1959) 137–152.DOI
- [82]S. V. Lishchuk, I. Halliday, C. M. Care, Shear viscosity of bulk suspensions at low reynolds number with the three-dimensional lattice Boltzmann method, Phys. Rev. E 74 (2006) 017701.pubmed.ncbi.nlm.nih.gov/16907217
- [83]E. Lorenz, A. G. Hoekstra, A. Caiazzo, Lees-edwards boundary conditions for lattice Boltzmann suspension simulations, Phys. Rev. E 79 (2009) 036706.pubmed.ncbi.nlm.nih.gov/19392080
- [84]Y. Thorimbert, F. Marson, A. Parmigiani, B. Chopard, J. Lätt, Lattice Boltzmann simulation of dense rigid spherical particle suspensions using immersed boundary method, Computers & Fluids (2018) 286–294.DOI
- [85]J. Guo, Q. Zhou, R. C.-K. Wong, Effects of volume fraction and particle shape on the rheological properties of oblate spheroid suspensions, Physics of Fluids 33 (8) (2021) 081703.
- [86]I. M. Krieger, T. J. Dougherty, A Mechanism for Non-Newtonian Flow in Suspensions of Rigid Spheres, Transactions of The Society of Rheology 3 (1) (1959) 137–152.DOI
- [87]T. Bhowmick, J. Latt, Y. Wang, G. Bagheri, Palabos turret: A particle-resolved numerical framework for settling dynamics of arbitrary-shaped particles, Computers & Fluids 299 (2025) 106696.DOI
- [88]H. Hertz, Ueber die berührung fester elastischer körper., Journal für die reine und angewandte Mathematik 92 (1882) 156–171.
- [89]T. Pöschel, T. Schwager, Computational granular dynamics: Models and algorithms, Springer-Verlag, Berlin ; New York, 2005, oCLC: ocm6063128.DOI
- [90]H. R. Norouzi, R. Zarghami, R. Sotudeh-Gharebagh, N. Mostoufi, Coupled CFD–DEM modeling: Formulation, implementation and application to multiphase flows, Wiley, Chichester, West Sussex, United Kingdom, 2016.
- [91]N. V. Brilliantov, F. Spahn, J.-M. Hertzsch, T. Pöschel, Model for collisions in granular gases, Physical Review E 53 (5) (1996) 5382–5392.pubmed.ncbi.nlm.nih.gov/9964870
- [92]K. Thyagarajan, C. Coreixas, J. Latt, Exponential distribution functions for positivity-preserving lattice Boltzmann schemes: application to 2D compressible flow simulations, Physics of Fluids 35 (2023) 126115.DOI



















