betaFeedback

WorksPhysics and Astronomy

CROMA: a GPU-accelerated Fokker–Planck solver for modeling non-thermal emission in post-processing cosmological simulations

Abstract

Numerical modeling of cosmic-ray (CR) spectra and the resulting non-thermal emission is important for understanding the physical mechanisms of CR acceleration and transport in astrophysical environments. Galaxy clusters and cosmic filaments provide particularly important environments for studying these processes, where shocks, turbulence, magnetic fields, and CRs jointly shape diffuse synchrotron emission. Spectrally resolved modeling of CRs, including cooling, re-acceleration, and hadronic interactions, requires solving time-dependent Fokker–Planck (FP) equations. An increasing number of studies employ FP solvers coupled with magnetohydrodynamical (MHD) simulations to follow the evolution of CRs in dynamically evolving environments. We present a new C/C++-based FP solver with GPU acceleration for post-processing outputs from cosmological MHD simulations. The code evolves CR proton and electron spectra along Lagrangian tracer trajectories and consistently includes re-acceleration and secondary-particle production processes. With the Chang–Cooper finite-difference scheme, the FP equation for each tracer is reduced to an independent tridiagonal system, which is solved on NVIDIA GPUs using a CUDA implementation of the parallel cyclic reduction (PCR) algorithm. The code uses sub-stepping between tracer snapshots, with the timestep constrained by the relevant CR cooling or re-acceleration timescale, and a batching algorithm that groups tracers with similar sub-stepping patterns for efficient execution. We find that the GPU acceleration increases the per-node throughput by a factor of ≈\approx10 on the Leonardo Booster system operated by CINECA. The fast FP solver is useful for high-resolution simulations dedicated to comparison with next-generation radio observations and future multi-messenger applications combining radio, gamma-ray, and high-energy neutrino observations.

Introduction

Combining magnetohydrodynamical (MHD) simulations with numerical modeling of particle acceleration and transport is essential for studying the origin and evolution of cosmic rays (CRs) and their interplay with the thermal plasma. Galaxy clusters provide a particularly important environment for studying these processes (see Dolag et al., 2008; Kravtsov and Borgani, 2012; Brunetti and Jones, [7], for reviews). Radio observations have revealed diffuse synchrotron emission extending over megaparsec (Mpc) scales, indicating the presence of relativistic electrons and magnetic fields in the intra-cluster medium (ICM) (e.g., van Weeren et al., 2019). Hadronic interactions of CR protons can produce secondary particles as well as gamma-ray and high-energy neutrino emission, which can be tested by current and future observations such as the Cherenkov Telescope Array (CTA) and IceCube (e.g., Fang and Murase, 2018; Nishiwaki et al., 2021; Hussain et al., 2023). Those observations provide a probe of CR acceleration processes, magnetic-field amplification, and their connection to the dynamical evolution of the large-scale structure of the Universe.

CRs in the ICM can be accelerated through first- and second-order Fermi mechanisms, associated respectively with shocks and turbulence driven by mergers of clusters. However, turbulence and shocks in the ICM are considered too weak to efficiently accelerate particles directly from the thermal pool. Hence, current models of diffuse synchrotron emission, such as radio halos and radio relics, often invoke the re-acceleration of pre-existing relativistic particles accumulated over the lifetime of clusters through activity associated with active galactic nuclei (AGNs), galaxies, and cosmological shocks (e.g., Völk et al., 1996; Berezinsky et al., [2]; Brunetti et al., 2001; Peterson et al., 2001; Fujita et al., 2003). In those models, the observable radio emission depends not only on the instantaneous properties of shocks and turbulence, but also on the time-dependent spectral evolution of CRs under cooling, injection, re-acceleration, and secondary-particle production through hadronic interactions.

Recent observations with next-generation radio interferometers, such as LOFAR and MeerKAT, have revealed increasingly complex morphologies and spectral structures in diffuse radio emission, including spatial variations in spectral indices and correlations between radio and thermal emission. These observations highlight features that are difficult to capture with simplified models, motivating more self-consistent comparisons between observations and multi-dimensional numerical simulations (e.g., Rajpurohit et al., 2021; Botteon et al., [5]; van Weeren et al., 2024, 2026).

An increasing number of studies have coupled CR spectral-evolution models with idealized or cosmological MHD simulations. In particular, time-dependent Fokker–Planck (FP) solvers have been used to follow the evolution of CR spectra, either in post-processing or through on-the-fly implementations ([4, 1] (Winner et al. 2019; Vazza et al. 2023; Kawashima and Asano 2025; Ivleva et al. 2026; Domínguez-Fernández et al. 2026; Nishiwaki et al. 2026)). Such simulations are powerful tools for interpreting current radio observations and for making predictions for future surveys with the Square Kilometre Array (SKA) and its precursors and pathfinders.

As the mean free path of CRs is much shorter than the spatial grid in MHD simulations, sub-grid models are required to calculate the re-acceleration and injection processes due to wave-particle interaction in the FP equation (Sect. 2). This is implemented with a momentum diffusion coefficient and injection term as a function of the outputs of MHD simulations. As observational constraints improve and numerical studies of plasma turbulence advance, these sub-grid models are expected to become increasingly sophisticated. However, such simulations remain computationally demanding.

The physical timescales involved in the FP equation can be much shorter than the MHD timescale, requiring many iterations to evolve CR spectra accurately. In addition, extensive parameter surveys are often needed to explore sub-grid models of CR acceleration and transport, further increasing the computational cost.

In this work, we address this computational challenge by presenting a new C/C++-based GPU-accelerated FP solver, which we name CROMA. Our code uses tracer particle outputs from MHD simulations such as Enzo (Bryan et al., 2014) and follows the spectral evolution of CRs along the tracer trajectories, where each tracer carries an independent CR spectrum neglecting spatial diffusion. Several FP solvers have been developed for similar applications, including CREST (Winner et al., 2019), ROGER (Vazza et al., 2023), and CRESCENDO (Böss et al., [4]). On the physical side, our code consistently models spectra of both CR protons and electrons with CR re-acceleration and secondary-particle production through hadronic interactions. On the computational side, the code has a CUDA-based GPU backend, enabling parallel computation of a large number of independent FP problems. These features make the code particularly well suited for future applications in multi-messenger astronomy and for high-resolution simulations designed for comparison with next-generation radio observations.

The CUDA backend uses the parallel cyclic reduction (PCR) algorithm to solve the tridiagonal systems obtained by discretizing the FP equation in momentum space (Sect. 4). By assigning each tracer-level FP problem to a CUDA thread block and parallelizing the operations over momentum bins, the implementation makes effective use of the large number of threads available on GPUs (Appendix C). We test the code on the Leonardo supercomputing system at CINECA and evaluate its performance and scalability for large numbers of tracer particles (Sect. 5). We also discuss future applications of the code, including CR transport between tracer particles and run-time CR spectral evolution in cosmological simulations (Sect. 6). The code developed in this work is publicly available on GitHub at https://github.com/kosukenishiwaki/croma-fp.

Fokker–Planck equation

We aim to model the spectral evolution of CRs under energy-loss and acceleration processes. We consider one-dimensional spectra in momentum space for CR electrons Ne(p)\mathcal{N}_{\mathrm{e}}(p) and CR protons Np(p)\mathcal{N}_{\mathrm{p}}(p) represented by a tracer particle, where pp is the CR particle momentum. In what follows, we omit the species subscript of N\mathcal{N} unless it is explicitly required. Here, N(p) dp\mathcal{N}(p)\,\mathrm{d}p represents the number of CR particles associated with the tracer in the momentum interval [p,p+dp][p,p+\mathrm{d}p]. The total number of CR particles carried by a tracer is calculated as ∫pmin⁡pmax⁡N(p) dp\int_{p_{\min}}^{p_{\max}}\mathcal{N}(p)\,\mathrm{d}p, where pmin⁡p_{\min} and pmax⁡p_{\max} are minimum and maximum momenta, respectively. The corresponding per-volume spectrum can be obtained by dividing N(p)\mathcal{N}(p) by the effective volume of each tracer. Neglecting spatial diffusion, the time evolution of N(p)\mathcal{N}(p) is modeled with the FP equation of the following form:

∂N(p)∂t=∂∂p[Dpp∂N(p)∂p+(H(p,z(t))−2pDpp)N(p)]−NT(p)+Q(p,z(t)),(1)\frac{\partial\mathcal{N}(p)}{\partial t} = \frac{\partial}{\partial p} \left[ D_{pp}\frac{\partial\mathcal{N}(p)}{\partial p} + \left(H(p,z(t))-\frac{2}{p}D_{pp}\right)\mathcal{N}(p) \right] -\frac{\mathcal{N}}{T(p)} +Q(p,z(t)), \tag*{(1)}

where HH, DppD_{pp}, TT, and QQ denote cooling, momentum diffusion, sink, and injection coefficients, respectively. For CR electrons, the cooling term includes radiative losses due to synchrotron and inverse-Compton emission, Coulomb cooling, and adiabatic compression or expansion, while Coulomb cooling and adiabatic compression or expansion are considered for CR protons (see Nishiwaki et al., 2021, 2026, for details).

For stochastic reacceleration represented with DppD_{pp}, the code supports the sub-grid models conventionally used for diffuse radio emission in clusters; Alfvénic stochastic acceleration (ASA, Brunetti and Lazarian, [6]) and transit-time damping (TTD, Brunetti and Lazarian, [8]; Brunetti [6]). In those models, the reacceleration efficiency is expressed as a function of the turbulent Mach number, Mturb=δv/cs\mathcal{M}_{\mathrm{turb}}=\delta v/c_{\mathrm{s}}, where csc_{\mathrm{s}} is the sound speed.

The sink term is neglected for CR electrons. For CR protons, the sink term represents the loss through the hadronic pppp collision with the background baryonic matter (Sect. 2.2).

The source term QQ should be determined from the physical assumption for the CR injection. In the default setup, the code sets a power-law spectrum at the beginning of the simulation and neglects additional sources by setting Q=0Q=0. When CR protons are included, the code follows the continuous injection of secondary electrons through the hadronic interaction (Sect. 2.2). In future work, we plan to extend the source term to include the effect of diffusive shock acceleration (DSA) at shocks, following previous studies (e.g., Böss et al., [4]; Vazza et al., 2023).

Chang-Cooper finite difference scheme

We set a one-dimensional momentum grid with NbinN_{\mathrm{bin}} bins equally-spaced in logarithmic scale. The number of momentum bins must be a power of two, Nbin=2MN_{\mathrm{bin}} = 2^M, so that the PCR solver is applicable in the CUDA backend (see also Appendix § 8). We adopt the Chang–Cooper (CC) discretization for the momentum-space flux (Chang and Cooper, 1970; Park and Petrosian, 1996) and use a first-order implicit time integration,

Nji+1−NjiΔt=Fj+12i+1−Fj−12i+1Δpj−Nji+1Tj+Qj,(2)\frac{N_j^{i+1}-N_j^i}{\Delta t}=\frac{F_{j+\frac{1}{2}}^{i+1}-F_{j-\frac{1}{2}}^{i+1}}{\Delta p_j}-\frac{N_j^{i+1}}{T_j}+Q_j, \tag*{(2)}

where Δt\Delta t is the discretized time step and we use index ii and jj for the time step and the momentum bin, respectively. The continuous expression of the flux is

F(p,t)=Dpp(p,t)∂N(p,t)∂p−(H(p,t)−2Dpp(p)p)N(p,t).(3)F(p,t)=D_{pp}(p,t)\frac{\partial N(p,t)}{\partial p}-\left(H(p,t)-\frac{2D_{pp}(p)}{p}\right)N(p,t). \tag*{(3)}

In short, the CC prescription evaluates the fluxes, Fj+12F_{j+\frac{1}{2}} and Fj−12F_{j-\frac{1}{2}}, at the interface between adjacent momentum bins by interpolating the value of NjiN_j^i from the two neighboring bin centers, with the weight δj\delta_j for advection determined by the local ratio of advective drift to diffusion. The weight δj\delta_j is chosen to preserve the correct local balance between drift and diffusion in the steady state. It reduces to a centered difference when diffusion dominates and approaches an upwind discretization when the drift term dominates (see Park and Petrosian (1996) for details).

With this scheme, Eq. (2) leads to a form of a tridiagonal linear system,

−ajNj−1i+1+bjNji+1−cjNj+1i+1=Δt Qj+Nji.(4)-a_jN_{j-1}^{i+1}+b_jN_j^{i+1}-c_jN_{j+1}^{i+1}=\Delta t\,Q_j+N_j^i. \tag*{(4)}

Here, we omit the specific expressions for coefficients aja_j, bjb_j, and cjc_j. We adopt the no-flux boundary conditions

a0=0,cNbin−1=0.(5)a_0=0,\qquad c_{N_{\mathrm{bin}}-1}=0. \tag*{(5)}

With the no-flux boundary conditions, the CC discretization conserves particle number in the absence of source and sink terms and preserves the positivity of the solution.

Hadronic chain and secondary particles

The code calculates the secondary particle creation through the decay of pions generated through the hadronic pppp interaction of CR protons:

{p+p→π0,±+X,π0→2γ,π±→μ±+νμ(νˉμ),μ±→e±+νˉμ(νμ)+νe(νˉe).(6)\begin{cases} p+p\to\pi^{0,\pm}+X,\\ \pi^0\to2\gamma,\quad\pi^\pm\to\mu^\pm+\nu_\mu(\bar{\nu}_\mu),\\ \mu^\pm\to e^\pm+\bar{\nu}_\mu(\nu_\mu)+\nu_e(\bar{\nu}_e). \tag*{(6)} \end{cases}

The spectra of the pions and secondary particles are calculated with the approximate expression given in Kelner et al. (2006). The secondary electrons and positrons are added to the CR electron spectrum through the source term QQ in Eq. (1). The code uses precomputed tables for the secondary spectra as a function of the momenta of the parent CR proton and secondary particles (see Nishiwaki et al., 2021, for further details).

The energy loss of CR protons through pppp collisions is approximated with the sink term as

T(p)=[ngasσpp(p)c]−1,(7)T(p)=\left[n_{\mathrm{gas}}\sigma_{pp}(p)c\right]^{-1}, \tag*{(7)}

where ngasn_{\mathrm{gas}} is the number density of the thermal gas particles, and σpp\sigma_{pp} is the total inelastic cross section of the pppp collision (Blattnig et al., [3]; Kamae et al., 2006). The code optionally calculates the emissivity of the secondary gamma ray and neutrinos for each tracer at each snapshot.

Synchrotron emission with precomputed tables

The code calculates the synchrotron emissivity of CR electrons in each tracer particle. Under the assumption of isotropic pitch-angle distribution for CR electrons, the synchrotron emissivity per unit frequency per unit solid angle can be written as

jν(ν,B;Ne)=3e3B8πmec2∫pmin⁡pmax⁡dp Ne(p)∫0πdθ sin⁡2θ F(ννc),(8)j_\nu(\nu,B;\mathcal{N}_e)=\frac{\sqrt{3}e^3B}{8\pi m_e c^2}\int_{p_{\min}}^{p_{\max}}dp\,\mathcal{N}_e(p)\int_0^\pi d\theta\,\sin^2\theta\,F\left(\frac{\nu}{\nu_c}\right), \tag*{(8)}

where

F(x)=x∫x∞K5/3(ξ) dξ,(9)F(x)=x\int_x^\infty K_{5/3}(\xi)\,d\xi, \tag*{(9)}

is the synchrotron kernel and

νc(p,B,θ)=3eB4πmecγ2sin⁡θ,(10)\nu_c(p,B,\theta)=\frac{3eB}{4\pi m_e c}\gamma^2\sin\theta, \tag*{(10)}

with γ=1+(p/mec)2\gamma=\sqrt{1+(p/m_ec)^2}. The calculation of Eq. (8) involves three numerical integrals and can be computationally demanding. We circumvent this problem by using pre-computed numerical tables. To accelerate the calculation of Eqs. (9), we prepare a table for the synchrotron kernel F(x)F(x) on a uniform grid in log⁡x\log x. We use Nx=2750N_x=2750 logarithmic bins between log⁡10xmin⁡=−27.5\log_{10}x_{\min}=-27.5 and log⁡10xmax⁡=2.5\log_{10}x_{\max}=2.5.

The integration over the pitch angle θ\theta in Eq. (8) can also be accelerated using fixed binning for θ\theta and the frequency ν\nu as θk\theta_k and νl\nu_l, respectively. To this end, we tabulate the characteristic frequency νc\nu_c at unit magnetic field,

νc,jkl(0)≡νc(pj,B0,θk),(11)\nu_{c,jkl}^{(0)}\equiv\nu_c(p_j,B_0,\theta_k), \tag*{(11)}

and define

yjkl≡log⁡10(νlνc,jkl(0)),(12)y_{jkl}\equiv\log_{10}\left(\frac{\nu_l}{\nu_{c,jkl}^{(0)}}\right), \tag*{(12)}

where B0=10−6 GB_0=10^{-6}\,\mathrm{G} is the reference field. The binning for ν\nu and θ\theta can be tuned through parameters. By default, we use Nν=128N_\nu=128 bins for 105.5 Hz<ν<1010.5 Hz10^{5.5}\,\mathrm{Hz}<\nu<10^{10.5}\,\mathrm{Hz} and Nθ=512N_\theta=512 bins for 0<θ<π0<\theta<\pi. For a local magnetic field BB, the parameter x=ν/νcx=\nu/\nu_c for the synchrotron kernel K(x)K(x) is obtained from

log⁡10xjkl(B)=yjkl−log⁡10(BB0).(13)\log_{10}x_{jkl}(B)=y_{jkl}-\log_{10}\left(\frac{B}{B_0}\right). \tag*{(13)}

Thus, the magnetic-field dependence can be taken into account as a shift in log⁡x\log x, while the (pj,θk,νl)(p_j,\theta_k,\nu_l) dependence is stored in the table of yjkly_{jkl}. The integral over θ\theta in Eq. (8) is now a function of the magnetic field BB. One can assume a typical range of BB in the region of interest in astrophysical environments. Setting a grid of log⁡10B\log_{10} B in advance, we compute a table of the pitch-angle integrated kernel,

F~bjl=∑k=0Nθ−1wkF(xjkl(Bb)),(14)\widetilde{F}_{bjl} = \sum_{k=0}^{N_{\theta}-1} w_k F\left(x_{jkl}(B_b)\right), \tag*{(14)}

where wk=sin⁡2θkΔθkw_k = \sin^2 \theta_k \Delta\theta_k is the weight factor and BbB_b denotes the magnetic field in the bb-th log⁡B\log B bin. We use the range −6.0≤log⁡10(BμG)≤2.0-6.0 \le\log_{10}(B_{\mu\mathrm{G}}) \le2.0, which sufficiently covers the magnetic-field strengths expected in galaxy clusters and cosmic filaments.

Note that the above BB table is used only for the synchrotron emission, and that the radiative cooling of CR electrons in the FP equation (Eq. (1)) is calculated with the exact value of the local BB recorded in the tracer data. Finally, the synchrotron emissivity is evaluated as

jν(νl,B;Ne)=3e3B8πmec2∑jNe,jΔpjF~bjl,(15)j_{\nu}(\nu_l, B; N_e) = \frac{\sqrt{3}e^3 B}{8\pi m_e c^2} \sum_j N_{e,j} \Delta p_j \widetilde{F}_{bjl}, \tag*{(15)}

with linear interpolation of log⁡10F~\log_{10} \widetilde{F} between adjacent log⁡B\log B bins. In Appendix B, we assess the errors involved in the tabulated method explained in this Section. Compared to the direct numerical integration, the error is smaller than O(10−3)O(10^{-3}) over a wide frequency range, except at the far end of the exponential tail.

Code overview

Figure 1 shows a schematic overview of the code workflow. The code first reads tracer snapshot data and distributes the tracer workload across the CPU cores with the Message Passing Interface (MPI) (Gropp et al., 2014). Between adjacent snapshots, the FP equation is solved with internal sub-stepping, where the timestep is constrained by the shortest relevant CR cooling or reacceleration timescale (Sect. 4.2).

Schematic picture of the code workflow

Figure 1. Schematic picture of the code workflow. The CUDA backend path involves the communication to the GPU device.

The background MHD quantities are linearly interpolated between adjacent snapshots during the sub-stepping. When the number of sub-steps becomes very large, the interpolation at every sub-step can be unnecessarily expensive. In such a case, the code allows the interpolation to be evaluated on a coarser temporal grid, with the maximum number of interpolation updates between two snapshots controlled by a runtime parameter.

An optional MPI load-balancing step redistributes tracers using the estimated number of FP sub-steps as a proxy for the computational cost of each tracer (Sect. 4.3). Within each rank, tracers with the same number of sub-steps are grouped together and advanced using batched FP solves (Sect. 4.4). The code has two backends for this batched process: a CPU backend and a CUDA backend. The CPU backend solves the resulting batched tridiagonal systems with the classical Thomas algorithm because of its small work complexity. The CUDA backend uses PCR, because it can update the momentum bins of many tracers in parallel and efficiently utilize the large number of GPU threads, allowing faster execution for large batches of independent systems (Appendix § 8).

The code outputs the CR spectra and associated synchrotron emission spectra for each tracer at each snapshot. When CR protons are included in the calculation, the code can also output hadronic gamma ray and neutrino spectra.

Input data

The code is designed to post-process tracer-particle data extracted from cosmological MHD simulations. The input data are provided as Hierarchical Data Format version 5 (HDF5) files, with one file corresponding to each simulation snapshot. Each file contains the local MHD quantities, such as the gas density, temperature, and magnetic field, sampled at the position of each tracer at each snapshot. Each tracer particle should be identified by a unique ID that is preserved across snapshots, allowing its trajectory to be reconstructed in time within the code.

When turbulent reacceleration models such as ASA or TTD (Sect. 2) are adopted, the local turbulent velocity amplitude is required to calculate the reacceleration timescale. The input data structure can be easily extended for future applications, for example to include shock-related quantities for modeling the diffusive shock acceleration.

The magnetic field strength in the original cosmological simulations may be underestimated when turbulent dynamo amplification is not fully resolved at the available spatial resolution. To account for such cases, the code provides an option to replace the simulated magnetic field strength with an estimate based on the turbulent dynamo model adopted in Nishiwaki et al. (2026).

The physical model and its parameters are controlled at runtime through a parameter file. The parameter file is supplied as a command-line argument when the code is executed.

Time stepping

The computational cost of the post-processing calculation is primarily determined by the number of internal time steps used to solve the FP equation. Although the tracer data are stored only at discrete simulation snapshots, the cooling and reacceleration timescales entering the FP model can be significantly shorter than the snapshot interval. The code therefore introduces sub-stepping between adjacent tracer snapshots to accurately follow the momentum-space evolution of cosmic-ray particles.

For each tracer and each snapshot interval, the target FP time step for sub-stepping, ΔtFP\Delta t_{\mathrm{FP}}, should satisfy the condition of

ΔtFP≤Csafemin⁡(tcoolmin⁡,treacc),(16)\Delta t_{\mathrm{FP}} \le C_{\mathrm{safe}} \min\left(t_{\mathrm{cool}}^{\min}, t_{\mathrm{reacc}}\right), \tag*{(16)}

where Csafe<1C_{\mathrm{safe}} < 1 is a safety factor, tcoolmin⁡t_{\mathrm{cool}}^{\min} is the minimum cooling timescale over the momentum grid, and treacc≡p2/(4Dpp)t_{\mathrm{reacc}} \equiv p^{2}/(4D_{pp}) is the local reacceleration timescale. In the conventional hard-sphere-type reacceleration, as for ASA and TTD, treacct_{\mathrm{reacc}} does not depend on the particle momentum, i.e., Dpp∝p2D_{pp} \propto p^{2}. For CR electrons in the ICM, tcoolmin⁡t_{\mathrm{cool}}^{\min} is typically determined either by the Coulomb cooling at the minimum momentum pmin⁡p_{\min} or by the radiative cooling at the maximum momentum pmax⁡p_{\max}. Thus, the computational load depends on the choice of these values. The cooling timescale of CR protons is typically much longer than that of CR electrons over most of the momentum range considered here, and therefore usually does not set the limiting time step.

As explained in Sect. 4.4, we adopt a batched method to evolve the FP timestep by grouping tracers with the same number of sub-steps. However, assigning a completely independent value of ΔtFP\Delta t_{\mathrm{FP}} to every tracer would fragment the tracer population into many small groups and reduce the efficiency of the batched method. To avoid this fragmentation of tracer groups, the target FP time step ΔtFP\Delta t_{\mathrm{FP}} is rounded down to a logarithmic mantissa ladder. In the present implementation, this ladder has ten allowed mantissae per decade, 1.0, 1.25, 1.6, 2.0, 2.5, 3.2, 4.0, 5.0, 6.4, 8.0. Note that the rounding of ΔtFP\Delta t_{\mathrm{FP}} to a smaller value does not violate the safety condition of Eq. (16).

With the rounded ΔtFP\Delta t_{\mathrm{FP}}, we estimate the number of FP sub-steps between two adjacent tracer snapshots as

Nsub=⌈ΔtsnapΔtFP⌉,(17)N_{\mathrm{sub}} = \left\lceil\frac{\Delta t_{\mathrm{snap}}}{\Delta t_{\mathrm{FP}}}\right\rceil, \tag*{(17)}

where Δtsnap\Delta t_{\mathrm{snap}} is the time interval between the two snapshots. For each NsubN_{\mathrm{sub}} assigned to tracers, the actual FP timestep is determined as ΔtFP=Δtsnap/Nsub\Delta t_{\mathrm{FP}} = \Delta t_{\mathrm{snap}}/N_{\mathrm{sub}}. During the sub-step loop, the background MHD quantities from the tracer files are linearly interpolated in time between two snapshots.

Load balancing

The code supports MPI parallelization by distributing the global tracer list across MPI ranks. Because the FP problems for individual tracer particles are independent and require no communication between MPI tasks during the solve, the computational cost of each tracer can be estimated prior to the full numerical integration. We use the accumulated number of FP sub-steps Nsub,nN_{\mathrm{sub},n} for the nn-th particle over all snapshot intervals as a proxy for the tracer workload,

Wn=∑snapshotsNsub,n.(18)W_n = \sum_{\mathrm{snapshots}} N_{\mathrm{sub},n}. \tag*{(18)}

The computational load assigned to each MPI rank is approximated by the sum of WnW_n over the tracers owned by that rank.

In general cases, a simple uniform distribution of tracers over MPI ranks can lead to a non-uniform workload distribution on MPI tasks, as shown in Figure 2. Since the overall execution time is determined by the slowest MPI rank, such load imbalance directly degrades the parallel performance. To address this problem, the code has a load-balancing option based on the estimated tracer workload WnW_n. Before the start of the main FP solve, each MPI rank estimates WnW_n for the tracers initially assigned to it. These estimates are collected and used to identify the tracers with large WnW_n. A fraction of these computationally-expensive tracers is then reassigned from heavily loaded ranks to lightly loaded ranks, so that the total estimated workload becomes more even across MPI ranks. The distribution of the workload after the load-balancing is shown with the red bars in Figure 2.

Distribution of the total number of sub-loops among the MPI ranks

Figure 2. Distributions of the total number of sub-loops among the MPI ranks. The blue bars show the case with an equal number of tracer particles on each rank, whereas the red bars show the case with load balancing with respect to the total NsubN_{\mathrm{sub}}, or WnW_n (Eq. (18)). Shown is the case for 10610^{6} tracer particles and ten MPI ranks.

The redistribution changes only the tracer ownership of each rank. Each rank constructs an active tracer list from the reassigned tracer IDs and their source offsets. These identifiers are used to access the original input data and write the updated spectra back to their global locations. Hence, the load-balancing communication is limited to tracer IDs, estimated workloads, and assignment records, while the FP solves remain local to each MPI rank.

The load-balancing procedure introduces an additional minor overhead due to the planning and the MPI communication. In practical applications, however, this overhead is negligible compared with the cost of the main FP solve. For example, in the case of Figure 2, the load-balancing overhead is approximately 0.3 second. Before load balancing, the maximum estimated workload among MPI ranks is about 7% larger than the mean. This moderate imbalance may lead to differences in the wall-time of several tens of seconds in the GPU runs (see also Sect. 5). Thus, the cost of the load-balancing step is much smaller than the potential wall-time penalty caused by the non-uniform workload.

Batched computation and CUDA backend

The update of CR spectrum by solving the FP equation is performed for batches of tracers. Within each MPI rank, the tracers with the same sub-stepping pattern are grouped together. We refer to these groups of tracers as buckets. Using the implicit CC finite-difference scheme, the FP solve leads to a tridiagonal system in momentum space (Sect. 2.1). Thus, the main numerical operation for each bucket is a batched tridiagonal solve, consisting of many independent systems with the same momentum-grid size.

In the CPU backend, bucketed batching primarily serves to regularize the data layout and reduce per-tracer control overhead; it does not change the computational complexity of the tridiagonal solves. The systems in a bucket remain independent and are solved separately using a classical Thomas algorithm. The CPU backend supports shared-memory parallelism with OpenMP within each MPI rank. The tracers assigned to each MPI rank can be further distributed among OpenMP threads and processed independently, including the bucket construction and FP solves. However, running a scaling test of Sect. 5.1 for a fixed total number of 32 CPU cores, we find that varying the decomposition between MPI ranks and OpenMP threads results in only minor differences in the overall performance. Thus, we adopt one OpenMP thread per MPI rank as the default configuration and use this setup throughout the performance tests presented below. Further optimization of the OpenMP implementation is left for future work, as well as the exploration of OpenMP offloading to provide GPU acceleration beyond the current CUDA-specific backend.

On the other hand, in the CUDA backend, buffer arrays for CR spectra are allocated on the GPU device for all tracers owned by a MPI rank. To minimize the host–device data communication, the CR spectra of each MPI task are copied from the host to these buffer arrays only at the beginning of each snapshot interval. The tracer buckets are processed by selecting the corresponding tracer entries from these device arrays. The preparation of the FP coefficients and tridiagonal matrices, as well as the tridiagonal solves, is performed on the GPU. The CUDA backend is designed for MPI runs with one MPI task per GPU. We note that allocating more CPU cores to the job than the number of MPI tasks used in the run can sometimes be beneficial for achieving the best GPU performance on a compute node.

We adopt a PCR algorithm for the tridiagonal solve in the CUDA backend. In the PCR kernel, each tridiagonal system is assigned to one CUDA thread block, and the NbinN_{\rm bin} threads within the block are mapped to the momentum bins of that system (Appendix § 8). In the present implementation, we limit the momentum-grid size to Nbin≤512N_{\rm bin} \le512. This limit is imposed to keep the number of threads per block and the shared-memory usage within a practical range for efficient execution on NVIDIA A100.

Although PCR has a higher work complexity than the Thomas algorithm, O(Nbinlog⁡2Nbin)O(N_{\rm bin}\log_2 N_{\rm bin}) instead of O(Nbin)O(N_{\rm bin}), its fine-grained parallelism is well suited to GPUs and can outperform serial tridiagonal solvers for large batches of independent systems (Appendix § 8). After all FP sub-loops are computed for all tracer buckets, the updated spectra are copied back to host memory.

During the development of the CUDA backend, ChatGPT and Codex were used to assist with the CUDA implementation, code review, and debugging. We note that the physical processes included in the Fokker–Planck equation and the Chang–Cooper numerical scheme had already been implemented in our previous work (e.g., Nishiwaki et al., 2021). The algorithmic choices, validation tests, and performance evaluations presented in this work were performed and verified by the authors.

In Sect. Appendix D and Appendix C, we check the consistency between the CPU and CUDA backends by solving test problems. We refer the reader to Fig. D.8 for examples of the output CR and emission spectra.

Scalability tests

In this section, we evaluate the GPU acceleration implemented in our code using tracer data generated by an Enzo simulation of a massive galaxy cluster (Nishiwaki et al. in prep.). We vary the size of the tracer dataset and the number of MPI ranks, and perform full FP runs, including data loading and output writing. In the following tests, we use tracer data consisting of 100 snapshots between redshifts z=1z = 1 and z=0z = 0. Note that the computational cost changes significantly from snapshot to snapshot because the cooling and reacceleration timescales depend on redshift and on the local MHD quantities. We vary the number of tracers, instead of the number of snapshots, to control the problem size in the scalability tests. In this way, all test datasets retain the same redshift range from z=1z = 1 to z=0z = 0, while the number of independent FP problems is varied by sampling tracers from the full dataset.

It is useful to define the throughput of the code as

R≡NtrcTNranks,(19)R \equiv\frac{N_{\rm trc}}{T N_{\rm ranks}}, \tag*{(19)}

where NtrcN_{\rm trc} is the total number of tracers of the run, TT is the run time, and NranksN_{\rm ranks} is the number of MPI tasks or the number of GPUs. We vary the problem size NtrcN_{\rm trc} in the following tests to assess the scalability of the code. Datasets with different NtrcN_{\rm trc} are generated by randomly sampling tracers from the full tracer dataset produced by the Enzo simulation, which contains Ntrctot=1.5×107N_{\rm trc}^{\rm tot} = 1.5 \times10^{7} tracers.

We use a fixed physical model and parameter set throughout these tests. Specifically, We set initial populations of CR protons and electrons at the initial snapshot (z=1z = 1) with the power-law spectra N∝p−2.2N \propto p^{-2.2}. We adopt the ASA turbulent reacceleration model using the local vorticity recorded in the tracer data (Brunetti and Lazarian, [6]), and Csafe=0.5C_{\mathrm{safe}} = 0.5 for the safety factor (Eq. (16)). The minimum and maximum momenta are set to pmin⁡/mc=10−0.5p_{\min}/mc = 10^{-0.5} and pmax⁡/mc=106p_{\max}/mc = 10^6 for both CR electrons and protons.

The initial energy density ratio between CR proton and electron is set to be one. We use Nbin=128N_{\mathrm{bin}} = 128 bins for the logarithmic momentum grid for both CR electrons and protons. With the CC discretization, this resolution provides sufficient accuracy for the spectral evolution for the astrophysical applications (Appendix C).

All the tests are conducted with High Performance Computing resources on the Booster partition of the Leonardo supercomputer1 at the CINECA Italian Supercomputing centre. The Leonardo Booster comprises 3456 compute nodes, each equipped with a 32-core Intel Xeon Platinum 8358 CPU, 512 GB of DDR4 memory, and four NVIDIA Ampere A100 GPUs with 64 GB of HBM2e memory each.

CPU strong scaling

We first measure the performance of the FP solver with the CPU backend through a strong-scaling test. We use a tracer sample with Ntrc=104N_{\mathrm{trc}} = 10^4 tracers. This problem size gives a single-rank runtime of around forty minutes, which is long enough to distinguish the scalable part of the computation from fixed costs that are typically of order a few minutes. We find that the end-to-end throughput is approximately R≈4R \approx4 tracers s−1^{-1} on a single rank. We measure the speedup by increasing the number of MPI tasks from 1 to 32. With 32 MPI tasks, each rank computes the FP equations for ≈310\approx310 tracers. The run with 32 MPI tasks finishes in about three minutes, at which point the overhead associated with non-parallelized parts of the code significantly affects the total execution time.

We fit the result with Amdahl’s law and find an effective serial fraction of fs=0.049f_s = 0.049. This suggests that the maximum speedup available with this setup is approximately ≈20\approx20. In a realistic production run, the problem size can be as large as Ntrc∼107N_{\mathrm{trc}} \sim10^7 (e.g., Nishiwaki et al., 2026). The scaling test presented here shows that the speedup becomes limited when the workload falls below ∼103\sim10^3 tracers per MPI task. Thus, for such production runs, using a few hundred MPI tasks would provide a reasonable balance between the total wall-clock time and the total CPU time consumption.

GPU acceleration performance

We next evaluate the performance obtained with the CUDA/GPU backend using the PCR tridiagonal solver. We measure the single-GPU throughput as a function of the problem size. Specifically, we vary the number of tracers per rank, NtrcN_{\mathrm{trc}}, and examine the saturation of the throughput in single-MPI, one-GPU runs.

Figure 4 shows the single GPU throughput for 104≤Ntrc≤2×10610^4 \le N_{\mathrm{trc}} \le2 \times10^6. The blue line shows the end-to-end throughput, computed with the wall-clock time T=TwallT = T_{\mathrm{wall}}, which includes I/O, preparation of FP coefficients, execution of the PCR solver, calculation of synchrotron emission, and other host- and device-side overheads. The orange line shows the throughput of the tridiagonal solve alone, computed with T=TsolveT = T_{\mathrm{solve}}. In our test case with Nbin=128N_{\mathrm{bin}} = 128, the throughput of the tridiagonal solver saturates at around Rsolve≈103R_{\mathrm{solve}} \approx10^3 tracers s−1^{-1} per MPI task. We find that the efficiency of GPU acceleration increases with NtrcN_{\mathrm{trc}}, and the end-to-end throughput reaches Rwall∼400R_{\mathrm{wall}} \sim400 tracers s−1^{-1} per MPI task at Ntrc∼106N_{\mathrm{trc}} \sim10^{6}. This is almost two orders of magnitude larger than the single-rank CPU throughput (Figure 3).

Strong scaling test with the CPU backend

Figure 3. Strong scaling test with the CPU backend. The vertical axis shows the total throughput over all MPI ranks. Each MPI rank uses a single CPU thread in this benchmark. The test uses a fixed tracer sample with Ntrc=104N_{\mathrm{trc}} = 10^4. The black dashed line shows the ideal scaling expected for a constant per-rank throughput.

Single-GPU throughput as a function of number of tracers

Figure 4. Single-GPU throughput as a function of NtrcN_{\mathrm{trc}} per rank. The blue and orange lines show the throughput measured from the total wall-clock time and from the time spent in the tridiagonal solve, respectively.

Figure 4 shows that small-sized problems, such as those with Ntrc∼104N_{\mathrm{trc}} \sim10^{4} tracers per task, cannot fully utilize the capability of the GPU. A suitable use case for the CUDA/GPU backend is therefore Ntrc∼105N_{\mathrm{trc}} \sim10^{5}–10610^{6} tracers per rank in the case of Nbin=128N_{\mathrm{bin}} = 128. However, one should also take care of the increase of the device memory consumption with NtrcN_{\mathrm{trc}}. Including the arrays for the CR spectra and the FP coefficients, the total array size per tracer is approximately 32 KiB for Nbin=128N_{\mathrm{bin}} = 128. Using a single A100 GPU with 64 GB of memory on Leonardo, we could not run the case with Ntrc=2×106N_{\mathrm{trc}} = 2 \times10^{6} because of device memory overflow. We note that the preferable range of NtrcN_{\mathrm{trc}} per task depends on NbinN_{\mathrm{bin}}. As NbinN_{\mathrm{bin}} increases, the computational and memory costs per tracer increase, so a smaller number of tracers per task can be sufficient to make effective use of the GPU. In practice, values of Nbin=128N_{\mathrm{bin}} = 128, 256, or 512 are reasonable choices depending on the available computational resources and the required numerical accuracy (see also Appendix C).

In the left panel of Figure 5, we assess the weak scaling of the GPU code. We test the cases of 1, 2, 4, 8 and 16 GPUs with Ntrc=5×105N_{\mathrm{trc}} = 5 \times10^{5}, 10610^{6}, 2×1062 \times10^{6}, 4×1064 \times10^{6}, and 8×1068 \times10^{6}, respectively. The number of tracers per rank is fixed at 5×1055 \times10^{5}. Since each computational node of the Leonardo Booster system is equipped with four GPUs and 32 CPU cores, the runs with 8 and 16 GPUs are executed on two and four nodes, respectively.

GPU weak scaling test and time breakdown

Figure 5. GPU weak scaling test (left panel) and time breakdown (right panel). In the left panel, we show the total wall-clock time and component-wise execution times as a function of the number of GPUs. The thick black line shows the total wall time, while the thin lines show the time spent in each process, averaged over MPI ranks. We use a fixed number of 5×1055 \times10^{5} tracers per rank. In the right panel, we show the time breakdown for the 16-GPU run in the weak-scaling test.

We find that the normalized total wall-clock time, shown by the thick black line, varies by only several percent over the range from 1 to 16 GPUs. The right panel of Figure 5 shows the breakdown of the execution time for the 16-GPU run. We place timers in several parts of the code and decompose the execution time into seven categories: input of the tracer data and output of the CR and emission spectra (I/O); host-side operations in the snapshot loop (Host); calculation of the FP and cooling coefficients (Coeff); calculation of the hadronic secondary kernels and secondary CR electron spectra (Secondary); tridiagonal solves (Solve); calculation of synchrotron and hadronic gamma-ray emissivities using the tabulated methods (Emission); and overhead associated with host–device communication and CUDA calls (CUDA overhead). The breakdown is obtained by averaging the measured times over the 16 MPI ranks.

We find that the off-loaded tridiagonal solve is the most time-consuming part (right panel of Figure 5). The host-side operations, which mainly involve planning the tracer buckets and organizing the bucket arrays, together with the CUDA overhead, remain subdominant compared with the main computational costs associated with the coefficient calculation and tridiagonal solves. As shown in the left panel, the Coeff and Solve components remain nearly constant with increasing GPU count, indicating that the main GPU-offloaded computations scale efficiently. Overall, the GPU backend shows good weak scaling up to 16 GPUs, with only a modest increase in the total wall-clock time.

Discussion and conclusions

In this work, we present a C/C++-based parallel FP solver with a CUDA backend for GPU acceleration. The code follows the spectral evolution of CRs and computes the associated non-thermal emission in post-processing using tracer-particle data from MHD simulations. The physical model implemented in the code includes turbulent reacceleration and secondary-particle production through hadronic pppp collisions, which are plausible mechanisms for diffuse synchrotron emission in galaxy clusters. The code assumes Lagrangian tracer particles as input and neglects CR transport between tracers. Under this assumption, the FP calculations for different tracers are independent, making the problem highly scalable. The code exploits independence between tracers by performing batched FP solves for groups of tracers with the same number of FP sub-steps. In the CUDA backend, the PCR solver parallelizes the tridiagonal solve over the momentum grid using CUDA threads, accelerating the computation compared with the classical serial Thomas solver adopted in the CPU backend. The consistency between the CPU and GPU backends has been extensively tested in Appendix C and Appendix § 11. The potential bottleneck associated with the computation of synchrotron emissivity is avoided by using precomputed tables. The accuracy of this tabulated method is presented in Appendix B.

We measure the code performance on a Leonardo Booster node equipped with 32 CPU cores and four NVIDIA A100 GPUs. From the scaling test presented in Sect. 5.1, we can see that the CPU backend reaches a per-node throughput of ≈120\approx120 tracers s−1^{-1} when using 32 MPI tasks for sufficiently large problem sizes. On the other hand, the CUDA/GPU backend reaches ≈1600\approx1600 tracers s−1^{-1} when using four GPUs (Figure 4 and Figure 5). Thus, the CUDA backend increases the effective per-node throughput by a factor of ∼10\sim10.

The speedup achieved with the CUDA implementation enables larger tracer calculations than would be practical with the CPU version alone. For example, in Nishiwaki et al. (2026), we studied emission in cluster outskirts using tracer particles with a characteristic mass of ∼107M⊙\sim10^{7}M_{\odot} and produced synthetic maps with an effective spatial resolution of ≈16\approx16 kpc. The increase in per-node throughput by roughly a factor of ten obtained in this work would allow about ten times more tracers to be evolved at similar computational cost, enabling spatial sampling on scales of several kpc. Such physical resolution would enable direct comparison of synthetic images with future SKA observations of nearby clusters at angular resolutions of a few arcseconds.

The speedup is also beneficial for exploring a substantially larger parameter space of uncertain sub-grid models, such as the re-acceleration efficiency and the CR proton-to-electron ratio. The code would help generate large suites of synthetic images and provide training or validation data for machine-learning approaches for radio observations (e.g., Gheller et al., 2018; Gheller and Vazza, 2022; Santillán et al., 2025).

The code can be extended in future work by including the effect of DSA in the CR source terms, and by coupling to cosmological MHD simulations through run-time FP solves. The code could be ported to other GPU architectures, such as AMD GPUs, by implementing a portable GPU framework.

Acknowledgements

We acknowledge INAF for awarding this project access to the LEONARDO supercomputer, owned by the EuroHPC Joint Undertaking, hosted by CINECA (Italy) and the LEONARDO consortium. The computations were conducted using the resources allocated to the projects designated "INA24C6B13" and "INA24C7B01," with K.N. serving as the Principal Investigator. We acknowledge financial support from INAF grants "Theory and simulations of non-thermal phenomena in galaxy clusters and beyond," and "A new numerical view of the cycle of relativistic electrons in clusters of galaxies."

The PCR algorithm

Listing 1: Schematic CUDA implementation of the PCR solver. One thread block is assigned to each tracer, and one thread is assigned to each momentum bin.

__global__ void solve_cc_pcr_kernel(...)
{
    int tracer = blockIdx.x;
    int j      = threadIdx.x;
    int idx    = tracer * Nbin + j;

    // Initialize PCR coefficients
    a[j] = -a_batch[idx];
    b[j] = b_batch[idx];
    c[j] = -c_batch[idx];
    d[j] = dt * source[idx] + spectrum[idx];

    __syncthreads();

    for (int stride = 1; stride < Nbin; stride *= 2)
    {
        a_new[j] = 0.0;
        b_new[j] = b[j];
        c_new[j] = 0.0;
        d_new[j] = d[j];

        if (j - stride >= 0) {
            alpha = -a[j] / b[j - stride];

            a_new[j] = alpha * a[j - stride];
            b_new[j] += alpha * c[j - stride];
            d_new[j] += alpha * d[j - stride];
        }

        if (j + stride < Nbin) {
            beta = -c[j] / b[j + stride];

            b_new[j] += beta * a[j + stride];
            c_new[j] += beta * c[j + stride];
            d_new[j] += beta * d[j + stride];
        }

        __syncthreads();
        swap_buffers();
    }

    spectrum[idx] = d[j] / b[j];
}

We show a schematic CUDA kernel of the PCR solver in Listing 1. The PCR loop starts by eliminating xj−1x_{j-1} and xj+1x_{j+1} in the jj-th equation of the tridiagonal system using the (j−1)(j-1)-th and (j+1)(j+1)-th equations (Eq. (4)). The system equations after mm loops can be written as

aj−sm[m]xj−2sm+bj−sm[m]xj−sm+cj−sm[m]xj=dj−sm[m],aj[m]xj−sm+bj[m]xj+cj[m]xj+sm=dj[m],aj+sm[m]xj+bj+sm[m]xj+sm+cj+sm[m]xj+2sm=dj+sm[m].\begin{aligned} a_{j-s_m}^{[m]}x_{j-2s_m}+b_{j-s_m}^{[m]}x_{j-s_m}+c_{j-s_m}^{[m]}x_j&=d_{j-s_m}^{[m]},\\ a_j^{[m]}x_{j-s_m}+b_j^{[m]}x_j+c_j^{[m]}x_{j+s_m}&=d_j^{[m]},\\ a_{j+s_m}^{[m]}x_j+b_{j+s_m}^{[m]}x_{j+s_m}+c_{j+s_m}^{[m]}x_{j+2s_m}&=d_{j+s_m}^{[m]}. \end{aligned}

where

sm=2m(20)s_m=2^m \tag*{(20)}

is the stride of the momentum index. The superscript [m][m] denotes the coefficients after mm loops. To reconcile the conventional notation for the Chang–Cooper scheme with the PCR notation used in this section, we define the initial PCR coefficients as aj[0]=−aja_j^{[0]}=-a_j, bj[0]=bjb_j^{[0]}=b_j, cj[0]=−cjc_j^{[0]}=-c_j, and dj[0]=Δt Qj+ujid_j^{[0]}=\Delta t\,Q_j+u_j^i.

By eliminating xj−smx_{j-s_m} and xj+smx_{j+s_m} in Eqs. (A.1), (A.2), and (A.3) with

αj[m]=−aj[m]bj−sm[m](21)\alpha_j^{[m]}=-\frac{a_j^{[m]}}{b_{j-s_m}^{[m]}} \tag*{(21)}
βj[m]=−cj[m]bj+sm[m](22)\beta_j^{[m]}=-\frac{c_j^{[m]}}{b_{j+s_m}^{[m]}} \tag*{(22)}

the updated coefficients are given by

aj[m+1]=αj[m]aj−sm[m],(23)a_j^{[m+1]}=\alpha_j^{[m]}a_{j-s_m}^{[m]}, \tag*{(23)}
bj[m+1]=bj[m]+αj[m]cj−sm[m]+βj[m]aj+sm[m],(24)b_j^{[m+1]}=b_j^{[m]}+\alpha_j^{[m]}c_{j-s_m}^{[m]}+\beta_j^{[m]}a_{j+s_m}^{[m]}, \tag*{(24)}
cj[m+1]=βj[m]cj+sm[m],(25)c_j^{[m+1]}=\beta_j^{[m]}c_{j+s_m}^{[m]}, \tag*{(25)}
dj[m+1]=dj[m]+αj[m]dj−sm[m]+βj[m]dj+sm[m].(26)d_j^{[m+1]}=d_j^{[m]}+\alpha_j^{[m]}d_{j-s_m}^{[m]}+\beta_j^{[m]}d_{j+s_m}^{[m]}. \tag*{(26)}

At the boundaries, the corresponding neighboring equation may not exist. In that case, the missing contribution is set to zero. Specifically, we set αj[m]=0\alpha_j^{[m]}=0 if (j−sm)<0(j-s_m)<0, and βj[m]=0\beta_j^{[m]}=0 if (j+sm)≥Nbin(j+s_m)\ge N_{\mathrm{bin}}. The terms involving coefficients outside the range 0≤j<Nbin0\le j<N_{\mathrm{bin}} are omitted.

After M=log⁡2NbinM=\log_2N_{\mathrm{bin}} loops, the stride becomes sM=Nbins_M=N_{\mathrm{bin}}, and the system is reduced to independent equations,

bj[M]xj=dj[M].(27)b_j^{[M]}x_j=d_j^{[M]}. \tag*{(27)}

The solution of the tridiagonal system is given by

xj=dj[M]bj[M].(28)x_j=\frac{d_j^{[M]}}{b_j^{[M]}}. \tag*{(28)}

PCR has a higher work complexity than the classical Thomas algorithm, O(Nbinlog⁡2Nbin)O(N_{\mathrm{bin}}\log_2N_{\mathrm{bin}}) instead of O(Nbin)O(N_{\mathrm{bin}}), but its fine-grained parallelism can make it faster on GPUs when large batches of independent tridiagonal systems are solved concurrently (e.g., Zhang et al. (2010)).

In our CUDA implementation, each tracer is assigned to one CUDA thread block. Within each PCR solve, the NbinN_{\mathrm{bin}} momentum bins of the tridiagonal system are mapped one-to-one to the threads within the block. Each thread updates the coefficients corresponding to one momentum bin, and block-wide

synchronizations are used between successive PCR iterations as the stride is doubled. The block-wide synchronization ensures that all updated PCR coefficients are available before the next iteration begins (Listing 1).

Since one momentum bin is mapped to one CUDA thread, increasing NbinN_{\mathrm{bin}} directly increases the number of threads per block. It increases the shared-memory workspace required to store the PCR coefficients during the reduction. We adopt Nbin=512N_{\mathrm{bin}}=512 as a practical upper limit in the present implementation. For unsupported grid sizes, the CUDA backend falls back to a device implementation of the Thomas algorithm.

For the time integration of a tracer, FP sub-steps can be executed within a single fused CUDA kernel. The CR proton and electron systems are solved sequentially within the same thread block and reuse the same shared-memory workspace. Keeping the spectra resident on the device and performing successive FP updates within the kernel reduces repeated kernel-launch and host–device communication overheads. The updated spectra are transferred back to host memory only after the device-side updates required for the current tracer batch have been completed.

Accuracy of the synchrotron table

As explained in Sect. 3, we use precomputed tables of Eq. (14) to speed up the calculation of the synchrotron integral (Eq. (8)). In the left panel of Fig. B.6, we compare the spectra calculated with the log BB table interpolation and those from direct numerical integration over the pitch-angle in Eq. (8). We use Nθ=1024N_\theta=1024 for the pitch-angle bin. We use the default binning of the magnetic field with −6≤log⁡10BμG≤2-6\le\log_{10}B_{\mu\mathrm{G}}\le2 and 512 bins. We study the cases of various spectral shapes, using the CR electron spectra calculated in the test of Sect. Appendix § 11. We confirm that the error induced by the tabulated method is marginal except for the exponential tail at high frequencies. In the right panel of Fig. B.6, we show the decay of the maximum relative error with the increased resolution of the log BB bin. The table interpolation method accelerates the calculation of the synchrotron emissivity by a factor of NθN_\theta, i.e., ≈103\approx10^3.

Comparison to analytical solutions

In this section, we test the implemented CC scheme and the tridiagonal solvers by comparing the numerical and analytical solutions of hard-sphere-type problems. Here, we basically repeat the tests presented in Nishiwaki et al. (2026) but with our newly developed CUDA pipeline.

A generic form of FP equation can be written as

∂u∂t=1A(x)∂∂x[C(x)∂u∂x+B(x)u]−uT(x)+Q(x).(29)\frac{\partial u}{\partial t}=\frac{1}{A(x)}\frac{\partial}{\partial x}\left[C(x)\frac{\partial u}{\partial x}+B(x)u\right]-\frac{u}{T(x)}+Q(x). \tag*{(29)}

where xx denotes the momentum or energy coordinate, u(x,t)u(x,t) is the spectrum, C(x)C(x), B(x)B(x), T(x)T(x), and Q(x)Q(x) are diffusive, advective, sink, and source coefficients, respectively. We consider the hard-sphere-like case of C(x)=x2C(x)=x^2, T(x)=1T(x)=1, A(x)=1A(x)=1, and Q(x)=δ(x−xinj)Q(x)=\delta(x-x_{\mathrm{inj}}) in Eq. (C.1). We consider two forms of the

Accuracy tests for synchrotron emission with the three models in Table D.1

Figure B.6. Accuracy tests for synchrotron emission with the three models in Table D.1. In the left panel, we show the synchrotron spectra calculated with the tabulated method with the discretized magnetic field grid explained in Sect. 3 (colored lines) and compare them with the spectra calculated by direct numerical calculation of Eq. (8) without using the fixed-B tables (black lines). We use a fixed range for −6≤log⁡10BμG≤2-6 \le\log_{10} B_{\mu\mathrm{G}} \le2 with 512 bins. The right panel shows the relation between the number of log⁡10B\log_{10} B bins and the maximum relative error in the frequency range.

Comparison to the analytical solutions of the hard-sphere problems

Figure C.7. Comparison to the analytical solutions of the hard-sphere problems. The upper panels show analytical (black) and numerical (blue and red) spectra. The left and right panels are the cases for B(x)=−x+1B(x)=-x+1 and B(x)=−x+x2B(x)=-x+x^{2}, respectively. The solid lines with star or square markers show the results calculated with Nbin=128N_{\mathrm{bin}}=128, while the dashed lines with triangle markers show Nbin=512N_{\mathrm{bin}}=512 results. The lower panels show the absolute relative error to the analytical solutions.

momentum-advection term: B(x)=−x+1B(x)=-x+1, representing a case dominated by adiabatic expansion, and B(x)=−x+x2B(x)=-x+x^2, representing a case dominated by radiative cooling. The minimum and maximum values of xx are set to xm=10−3x_{\rm m}=10^{-3} and xM=104x_{\rm M}=10^4. The steady state solutions for these problems can be found in Park and Petrosian (1995).

Fig. C.7 shows the steady-state solutions for the above setup. We compare two cases for the resolution of the xx bin, Nx=128N_x=128 and 512. The numerical solutions match well with the analytical ones in both cases, except for the bins close to the boundary. The differences between CPU and GPU solutions are marginal compared with the discretization error inherent in the CC scheme. These tests validate the tridiagonal solvers implemented in the code.

Simulated spectra under ICM conditions

In Appendix C, we check the accuracy and consistency of the FP solver for both CPU and GPU backends using the hard-sphere type problems with known analytical solutions. In this section, we examine the spectra calculated with CPU and GPU pipelines in more realistic problems to show the qualitative effects of the physical parameters and validate the consistency of the two backends. We consider benchmark models with representative ICM parameters. We consider three representative cases: a cooling-dominated model, a re-acceleration model, and a secondary-electron model. We use the physical parameters and assumptions summarized in Table D.1. We assume a single power-law spectrum with a slope of δinj=−2.2\delta_{\rm inj}=-2.2 for the initial CR spectrum. The initial spectrum is normalized with the CR number fraction ϕ\phi to the number of thermal electrons or protons. The inverse-Compton cooling is calculated with a fixed redshift z=0.2z=0.2, for simplicity. Using the fixed background parameters, we solve the evolution for 3 Gyr. We compute the CR spectra and the synchrotron emissivity, and compare the results from the CPU and CUDA backends.

Fig. D.8 shows the CR and synchrotron spectra for the three cases. In upper panels, we show the CR electron spectra multiplied by momentum, pNe(p)pN_{\rm e}(p). We reproduce the typical bump-shaped spectrum caused by Coulomb and radiative cooling in the cooling-dominated model. In the re-acceleration model, the bump is broadened and the cut-off is shifted to higher energies, which leads to an increase in the synchrotron emissivity. In the secondary-electron model without re-acceleration, the spectrum shows a single power-law shape above ∼\simGeV energies. In each case, the relative errors in the CRe and synchrotron spectra between the CPU and GPU calculations are of order O(10−11)O(10^{-11}). These results confirm the consistency between the two backends.

Declaration of generative AI and AI-assisted technologies in the manuscript preparation process

During the preparation of this work, the authors used ChatGPT GPT-5.5 and 5.6 to assist with improving the language, clarity, and readability of the manuscript. After using this tool, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

CR electron and synchrotron spectra for the cooling, re-acceleration, and secondary models, with CPU and GPU comparisons and relative errors

Figure D.8. CR electron (upper) and synchrotron (lower) spectra for the three models in Table D.1. We also show the relative errors between solutions calculated with CPU and CUDA/GPU paths.

modelNbinN_{\mathrm{bin}}redshiftρgas\rho_{\mathrm{gas}} [g cm−3^{-3}]BB [μ\muG]δinj\delta_{\mathrm{inj}}ϕCRe\phi_{\mathrm{CRe}}ϕCRp\phi_{\mathrm{CRp}}treacct_{\mathrm{reacc}} [Myr]
cooling1280.23×10−273 \times10^{-27}3-2.210−610^{-6}0–
re-acceleration1280.23×10−273 \times10^{-27}3-2.210−610^{-6}0300
secondary1280.23×10−273 \times10^{-27}3-2.2010−710^{-7}–

Table D.1. Parameters used in the code tests in Appendix § 11.

References

  1. [1]Beduzzi, L., Vazza, F., Cuciti, V., Brunetti, G., Brüggen, M., Wittor, D., 2024. Cosmological simulations of the generation of cluster-scale radio emission from turbulent re-acceleration. A&A 690, A67. doi:10.1051/0004-6361/202450763, arXiv:2406.09859.DOI
  2. [2]Berezinsky, V.S., Blasi, P., Ptuskin, V.S., 1997. Clusters of Galaxies as Storage Room for Cosmic Rays. ApJ 487, 529–535. doi:10.1086/304622, arXiv:astro-ph/9609048.DOI
  3. [3]Blattnig, S.R., Swaminathan, S.R., Kruger, A.T., Ngom, M., Norbury, J.W., 2000. Parametrizations of inclusive cross sections for pion production in proton-proton collisions. Phys. Rev. D 62, 094030. doi:10.1103/PhysRevD.62.094030, arXiv:hep-ph/0010170.DOI
  4. [4]Böss, L.M., Steinwandel, U.P., Dolag, K., Lesch, H., 2023. CRESCENDO: an on-the-fly Kolmogorov-Planck solver for spectral cosmic rays in cosmological simulations. MNRAS 519, 548–572. doi:10.1093/mnras/stac3584, arXiv:2207.05087.DOI
  5. [5]Botteon, A., van Weeren, R.J., Brunetti, G., Vazza, F., Shimwell, T.W., Brüggen, M., Röttgering, H.J.A., de Gasperin, F., Akamatsu, H., Bonafede, A., Cassano, R., Cuciti, V., Dallacasa, D., Di Gennaro, G., Gastaldello, F., 2022. Magnetic fields and relativistic electrons fill entire galaxy cluster. Science Advances 8, eabq7623. doi:10.1126/sciadv.abq7623, arXiv:2211.01493.DOI
  6. [6]Brunetti, G., 2016. The challenge of turbulent acceleration of relativistic particles in the intra-cluster medium. Plasma Physics and Controlled Fusion 58, 014011. doi:10.1088/0741-3335/58/1/014011, arXiv:1509.03299.DOI
  7. [7]Brunetti, G., Jones, T.W., 2014. Cosmic rays in galaxy clusters and their nonthermal emission. International Journal of Modern Physics D 23, 1–76. doi:10.1142/S0218271814300079.DOI
  8. [8]Brunetti, G., Lazarian, A., 2007. Compressible turbulence in galaxy clusters: Physics and stochastic particle re-acceleration. Monthly Notices of the Royal Astronomical Society 378, 245–275. doi:10.1111/j.1365-2966.2007.11771.x.DOI

Paper details

vlnc:W_su8jjzv2
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

GPT-5.5 (bare), 5.6

View statement
Looking for related papers…
Contents