betaFeedback

WorksPhysics and Astronomy

Choose Your Diffusion: Efficient and flexible ways to accelerate the diffusion model in fast high energy physics simulation

Abstract

The diffusion model has demonstrated promising results in image generation, recently becoming mainstream and representing a notable advancement for many generative modeling tasks. Prior applications of the diffusion model for both fast event and detector simulation in high energy physics have shown exceptional performance, providing a viable solution to generate sufficient statistics within a constrained computational budget in preparation for the High Luminosity LHC. However, many of these applications suffer from slow generation with large sampling steps and face challenges in finding the optimal balance between sample quality and speed. The study focuses on the latest benchmark developments in efficient ODE/SDE-based samplers, schedulers, and fast convergence training techniques. We test on the public CaloChallenge and JetNet datasets with the designs implemented on the existing architecture, the performance of the generated classes surpass previous models, achieving significant speedup via various evaluation metrics.

Introduction

The Large Hadron Collider (LHC) [1] is one of the biggest particle accelerator human have ever made. More than billions of events are generated and recorded by experiments like ATLAS [2] and CMS [3] near the interaction points during each run. Sufficient and precise simulation for comparing the data and theoretical prediction is required to find the evidence of new physics at the energy frontier.

The Geant4 software toolkit [4] is applied for the detector simulation. The simulated primary particles would propagate through specific material and geometry, resulting vast amount of secondary particles produced in each propagated steps to achieve precision. As a consequence, the detector simulations for the modern high granularity detectors occupy the most computation resources [5]. Reconstructing the physics objects like jets with complicated substructure for different physics process also plays a crucial role in providing the enough statistics for the data-driven analysis. In the upcoming High Luminosity LHC [6], the expected integrated luminosity will scale up to ten times the current level. It is nearly impossible to perform the full simulation for all the events with constrained computational budget, manifesting the necessity of fast simulation.

To address with these challenges comprehensively, different techniques adopted from deep generative modeling, where a lot of phenomenal improvements have quickly emerged over the years, have become the popular approach as fast surrogate models in high energy physics.

One latest category of generative models, the Diffusion Model (DM) as well as its variations, has become increasingly dominant in various generative applications [7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21]. There has been a few studies using DM in both fast detector and end-to-end event simulations.

For fast detector simulation, CaloScore [22] utilizes the score-based generative model and the conditional UNet to produce almost indistinguishable generated showers from truth ones. CaloDiffusion [23], on the other hand, utilizes the denoising diffusion probabilistic model, incorporates novel geometry adaptation, and enhances the UNet with middle self-attention and cylindrical convolutional layers. This combination positions CaloDiffusion as one of the top models in fast detector simulation. Moreover, CaloClouds [24] offers an alternative way to generate geometry-independent point clouds for great modelling of physically relevant distributions.

For end-to-end event simulation [20, 21], there are a series of studies for generating the point cloud with DM [25, 26]. Notably, those studies also use architectures like transformer [25] or EPiC layer [26] that utilize the permutation invariance of jet constituents.

Among those original studies, their first versions often suffer from a long generation time, as desired results typically converge only after a large number of sampling steps. Further studies involving the fresh distillation methods like progressive distillation [27, 28], consistency model [23, 29, 30] to get the result in only few steps with the exchange of longer training time, and other ways [29, 31] to speedup the backward process.

Most of the time, different DM methods face a tradeoff between the quality and speed of generated classes. There is a lack of a comprehensive study on the training and sampling dynamics of diffusion models applied for various purposes and datasets in high energy physics. In this study, we investigate the effects of different training strategy by offering a soft way to expedite the convergence of performance. We also adopt several popular training-free ODE/SDE based samplers and noise schedulers in post-DDPM era [32, 33, 34, 35, 36]. By balancing the discretization errors and accumulated errors in the sampling process, a flexible and efficient way can be offered to make the diffusion model faster and more accurate.

We finally make our approaches portable by testing in different network and different datasets using the public fast and high-fidelity calorimeter dataset CaloChallenge [37, 38] and the jet point cloud dataset JetNet [39]. The performance surpasses the previous model by several evaluation metrics [8, 25, 37, 38, 40].

This paper is outlined as follows, § 2 covers the fundamentals of diffusion model, elucidating how data such as shower images or point clouds are specifically generated through the backward process. § 3 provides the a brief introduction about the dataset used in this study. § 4 discusses the training strategy applied. Various metrics will be presented to evaluate the performance in § 5. Lastly, § 6 gives a empirical discussion, conclusion and outlooks about the study.

Dynamics in the Diffusion Model

The diffusion model was initially introduced to sample complex data distributions by drawing inspiration from concepts in non-equilibrium thermodynamics [41]. At first, we gradually add noises to an initial data sample from x0x_0 to xTx_T until the time step TT, usually denoted as the forward process qq where the data points are degraded as TT increase. Then it follows a backward process pp where noisy data could be denoised iteratively to new observation ∼x0\sim x_0 as the time goes reversely.

Two main classes are introduced later [42, 43]: Denoising Diffusion Probabilistic Models (DDPMs) [44, 45] and Score-based Generative Models (SGMs) [46].

For DDPM, the forward process transforms input distribution into a prior distribution by

q(xt∣xt−1)=N(xt∣1−αtxt−1,αt)(1)q(x_t|x_{t-1}) = \mathcal{N}(x_t|\sqrt{1-\alpha_t}x_{t-1},\alpha_t) \tag*{(1)}

where αt\alpha_t is the noise variance in every time step in case of the noise has unit variance ϵ∼N(0,I)\epsilon\sim\mathcal{N}(0,I), the single step formulation can be rewritten as:

q(xt∣x0)=N(xt∣α~tx0,1−α~t)(2)q(x_t|x_0) = \mathcal{N}(x_t|\sqrt{\tilde{\alpha}_t}x_0,1-\tilde{\alpha}_t) \tag*{(2)}
xt=α~tx0+1−α~tϵ(3)x_t = \sqrt{\tilde{\alpha}_t}x_0+\sqrt{1-\tilde{\alpha}_t}\epsilon \tag*{(3)}

where α~t=∏i=1T(1−αi)\tilde{\alpha}_t = \prod_{i=1}^{T}(1-\alpha_i). The direct calculation from x0x_0 can allow us to random sample time steps to add noise during training. In the backward process, a probability function p(xt−1∣xt)p(x_{t-1}|x_t) is learned to denoise the noisy data at previous time step until reach the initial input x0x_0.

The SGM is formularize similarly, in the score Stochastic Differential Equations (SDEs), the forward process is defined by [46]:

dx=f(x,t)dt+g(t)wt(4)d x = f(x,t) d t + g(t) w_t \tag*{(4)}

where ff and gg are drift and diffusion coefficients chosen differently for variance preserving and variance exploding noise, ww is the standard Brownian motion.

The reverse-time SDE is defined as:

dx=[f(x,t)−g(t)2∇xlog⁡pt(x)]dt+g(t)dw~(5)d x = [f(x,t)-g(t)^2\nabla_x\log p_t(x)]d t + g(t)d\widetilde{w} \tag*{(5)}

unlike DDPM, the quantity we are solving is the score function (A vector field indicating the direction toward regions of increased data density given a certain level of noise.) ∇xlog⁡pt(x)\nabla_x\log p_t(x). The learning process of this gradient of log probability functions offers another possibility to use not only SDE but also ODE to get the solution. Then follows by the generalization from [47], we can write the reverse SDE equation as a sum of probability flow ODE and Langevin diffusion SDE:

dx=−σ˙(t)σ(t)∇xlog⁡pt(x)dt−β(t)σ(t)2∇xlog⁡pt(x)dt+2β(t)σ(t)dwt(6)d x = -\dot{\sigma}(t)\sigma(t)\nabla_x\log p_t(x)d t-\beta(t)\sigma(t)^2\nabla_x\log p_t(x)d t+\sqrt{2\beta(t)\sigma(t)}d w_t \tag*{(6)}

where σ(t)\sigma(t) is the noise schedule, β(t)=σ˙(t)/σ(t)\beta(t)=\dot{\sigma}(t)/\sigma(t), the first term is the marginally equivalent ODE, the second and third terms are added as the deterministic flow and noise injection term for Langevin diffusion, the schematic diagram as shown in Figure 1.

Schematic diagram for a noisy shower image and ODE and SDE samplers

Figure 1. Upper: The schematic diagram for a noisy shower image (energy deposited in one layer of calorimeter) xtx_t denoises along the ODE curvature with varying time steps to clean shower close to x0x_0. Lower: Arrows indicate a schematic sketch how ODE and SDE samplers reach the x0x_0 along timesteps, ODE flows along the certain direction toward x0x_0. tit_i is the ithi^{\mathrm{th}} timestep, SDE will have extra noise injection γti\gamma t_i in each time step before moving to next step. The Euler and Heun are the first and second order calculation for updating xix_i.

This determines the characteristics of the general ODE/SDE solver. ODE solvers often have smaller discretization errors than SDE solvers in smaller sampling steps [48, 49]. However, as the sampling steps become larger, the accumulated error for SDEs would be much smaller than that for ODEs, resulting in a smaller total error (better performance) in large sampling steps. The noise scheduler determines how fast those samplers should learn along different time steps. In the paper, we will cover up to seven mainstream samplers nowadays (four ODEs and three SDEs) over different schedulers.

The schedulers and samplers directly decide how the noisy data xtx_t will reach x0x_0 with proper noise levels. Our choices for schedulers and samplers are listed belows:

  • Schedulers:

    • The default cosine beta scheduler for discrete time DDPM pretrained model, this is the baseline to compare with performance for other schedulers and samplers.

      αˉ=cos⁡(π(t/T+s)2(1+s))\bar{\alpha}=\cos\left(\frac{\pi(t/T+s)}{2(1+s)}\right)

      where s=0.008s=0.008 that determines the minimum noise for last steps. We can tell by the function, the noises are added by a linearly decrease rate during intermediate steps to avoid too much noise in the last few steps.

    • Variance Preserving schedulers as a more generalized version of cosine beta schedulers used in both score matching [46] and DDPM, defined as a continuous time step tt and σ=(exp⁡(βmax⁡t22+βmin⁡t)−1)\sigma=\sqrt{\left(\exp\left(\frac{\beta_{\max}t^2}{2}+\beta_{\min}t\right)-1\right)} where βmin⁡\beta_{\min} and βmax⁡\beta_{\max} are constant to determine the lower and upper bound of noise level.

    • The schedulers proposed by Karras [47]. σi<N=(σmax⁡1/ρ+iN−1(σmin⁡1/ρ−σmax⁡1/ρ))ρ\sigma_{i<N}=\left(\sigma_{\max}^{1/\rho}+\frac{i}{N-1}(\sigma_{\min}^{1/\rho}-\sigma_{\max}^{1/\rho})\right)^\rho and σN=0\sigma_N=0. It’s the monotonically decreasing function, so the noise added in a decreasing rate, ρ\rho determines the curvature. Larger ρ\rho means more steps near σmin⁡\sigma_{\min} are shortened. Reasonable value for σmin⁡,σmax⁡,ρ\sigma_{\min},\sigma_{\max},\rho in our study should around [0.01,0.001][0.01,0.001], [20,80][20,80], [5,10][5,10] respectively. This scheduler is helpful for generating good quality of data in low sampling step region.

    • Combine Karras schedulers proposed with Lu [50] by transforming the σ\sigma to log scale. σi<N=(log⁡(σmax⁡)1/ρ+iN−1(log⁡(σmin⁡)1/ρ−log⁡(σmax⁡)1/ρ))ρ\sigma_{i<N}=\left(\log(\sigma_{\max})^{1/\rho}+\frac{i}{N-1}(\log(\sigma_{\min})^{1/\rho}-\log(\sigma_{\max})^{1/\rho})\right)^\rho. The values for σmin⁡,σmax⁡\sigma_{\min},\sigma_{\max} remains the same region, ρ=1\rho=1 to keep a healthy noise level range. This scheduler is even more decreasing than Karras schedulers, so the step size changes more rapidly in low sampling step while more slowly in last steps.

  • Samplers:

  • The same DDPM samplers are used in [23, 44]. The Euler-Maruyama method approximates the reverse diffusion process by iteratively denoising the sample while adding stochastic noise at each discrete timestep.

  • EDM Stochastic sampler with Heun’s method [47], as illustrated in Figure 1, a noise injection step is performed to current time step t^i=ti+γti\hat{t}_i = t_i + \gamma t_i where γ\gamma is determined by three parameters γ=min⁡(Schurn/N,2−1)\gamma= \min(S_{churn}/N, \sqrt{2}-1) when Smin⁡≤ti≤Smax⁡S_{\min} \leq t_i \leq S_{\max}. The Smax⁡,Smin⁡S_{\max}, S_{\min} is the range for injection, SchurnS_{churn} is the stochasticity strength which give the lower bound to start adding noise. It controls how fast the sampler converges to zero noise. The optimal sets of parameters can make the sampling faster and better. Contrarily, a bad sets of those stochastic term will make possible degradation of the generated sample. In our study, such degradation only happens when extreme values (e.g. large γ\gamma, or SchurnS_{churn} close to initial timestep) are selected. The difference between this approach and Euler Maruyama is that the denoising steps in EDM samplers happens in some intermediate state t^i\hat{t}_i instead of actual timestep state tt, the difference between two states become diminishing when N goes large. This SDE sampler, unlike other SDEs that only give satisfied result after a very large sampling steps, can converge quickly if one can manage to tune those stochastic parameters well.

  • Restart sampler proposed in [48]. This sampler introduces a way to strongly contract the accumulated error based on EDM ODE Heun sampler. A back-and-forth ODE step will repeat KK times in the predefined single or multiple intervals [tmax⁡,tmin⁡][t_{\max}, t_{\min}].

  • DPM-Solver samplers proposed in [49, 50]. A popular ODE solver that can speedup the sampling process by applying change-of-variable to noise level. We choose to use the latest multistep solver that calculate iteratively by the knowledge of current and next timestep.

  • Combined DPM-Solver and EDM SDE samplers. Instead of using Euler and Heun step, it uses the "mid-point" step by change-of-variable to intermediate state t^i\hat{t}_i.

  • LMS sampler [51]. For each iteration, the sampler will denoise the input data using the model and updates the xix_i using the linear multistep integration method with the calculated coefficients of predefined orders.

  • Uni-PC sampler introduced in [52]. A unified predictor-corrector framework that can be applied on many existing samplers. We apply it with DPM-Multistep-sampler with both varying coefficient and B(h)B(h) solver proposed in the paper.

Dataset

CaloChallenge

Fast Calorimeter Simulation Challenge 2022 [37] is the GEANT4 simulation for particle showers deposited in the calorimeter. The purpose for this challenge is to develop and benchmark the fast and high-fidelity calorimeter simulation using modern techniques from deep learning. Three datasets are provided ranging from easy to hard. The complexity of dataset is determined by the dimensionality of the calorimeter showers, including factors such as the number of layers and voxels on each layer.

In each dataset, particles are simulated to propagate along the zz-axis inside the cylindrical and concentric detector. Similar to the sampling layers in the common calorimeter, a discrete number of layers are placed with specific radial and angular bins rr and α\alpha. Two classes of quantities are stored: the incident particle energies and voxel energies stored in each shower.

By mapping the voxel location with given r\mathbf{r} and α\boldsymbol{\alpha} bins, one can reconstruct the layer deposited energy inside the shower. This offers flexibility for designing the generative modelling tasks using either low or high level features.

This study will primarily focus on testing dataset 2 which simulates a physical detector with 45 layers of absorber material (Tungsten) and active material (Silicon). A total of 100,000 electrons in each file enter the calorimeter with a direction. The incoming particle energies span over a log-uniform distribution from 1 GeV and 1 TeV. Each layer contains 16 angular and 9 radial bins, so 16×9=14416 \times9 = 144 cells, and a total number of 144×45=6480144 \times45 = 6480 voxels in each shower. A minimal cutoff around 15 keV is applied to voxel energy.1

JetNet

JetNet [39] is the point cloud like dataset which consists of 200,000 simulated events. Each event contains the corresponding jet types with transverse momenta pT∼1p_{\mathrm{T}} \sim1 TeV, originating from either gluons, top or light quarks. Additionally, a narrow kinematic window is imposed, ensuring that the relative transverse momenta of the generated parton and gauge boson ΔpT/pT≤0.1\Delta p_{\mathrm{T}}/p_{\mathrm{T}} \le0.1. The jet clustering is performed using the anti-kTk_{\mathrm{T}} algorithm [53] within a cone size of R=0.4R = 0.4.

For up to 30 particle constituents stored in one jets by descending pTp_{\mathrm{T}} order, three features are provided: the relative angular coordinate of particles w.r.t. the jets: ηrel=ηparticle−ηjet\eta^{\mathrm{rel}} = \eta^{\mathrm{particle}} - \eta^{\mathrm{jet}}, mod⁡(ϕrel=ϕparticle−ϕjet,2π)\operatorname{mod}(\phi^{\mathrm{rel}} = \phi^{\mathrm{particle}} - \phi^{\mathrm{jet}}, 2\pi) and relative transverse momentum pTrel=pTparticle/pTjetp_{\mathrm{T}}^{\mathrm{rel}} = p_{\mathrm{T}}^{\mathrm{particle}}/p_{\mathrm{T}}^{\mathrm{jet}}. There is also a fourth variable that masks whether the particles is genuine or zero-padded when there are fewer than 30 constituents stored in the jets.

We will show preliminary results for testing on gluon and top quark jets. The gluon jet can serve as a baseline test since it usually produces the dominant background for many physics processes and has a relatively simple topology. The top jet has more subtle substructure and presents challenging tasks for generative models.

Preprocessing and Training

In this section, we provide an overview for the data processing, and training details on two datasets mentioned in § 3.

CaloChallenge

We follow a common preprocessing procedure for CaloChallenge dataset.

  • First, the voxel energies viv_i are normalized by the incident particle energy.

  • Then a logit-norm transformation is applied to ensure input data has zero mean and unit variance.

ui=log⁡(x1−x),x=vi−2δvi+δui′=ui−uˉσu(7)\begin{aligned} u_i &= \log\left(\frac{x}{1-x}\right), \quad x = v_i - 2\delta v_i + \delta\\ u_i' &= \frac{u_i-\bar{u}}{\sigma_u} \tag*{(7)} \end{aligned}

Here, δ=10−6\delta= 10^{-6} is introduced to avoid the discontinuity. We choose to use the network from [23] which is composed of a conditional UNet with middle layer self-attention layer, cylindrical convolution and Geometry Latent Mapping (GLaM). The latter two techniques are found to be beneficial for generating quantities in rr and α\alpha bins. In order to demonstrate the portability of our studies, the neural network architect remains unchanged from the version in the literature [23]. The original training is tailored for discrete noise level samplers like DDPM [44]. We have integrated our training setups, including different loss function designs and data preprocessing, into the new training process. We use publicly available pre-trained models with 400 DDPM steps as the baseline to compare the performance.

JetNet

For JetNet dataset, we follow the preprocessing procedure from [25, 26]. The Equivariant Point Cloud (EPiC) layer based on the deep sets framework is chosen to be used as it shows great quality of generated classes while having lighter architecture than transformer-based models [26]. With the same training procedure changes, we further modify the architectures with similar sinusoidal and cosine positional embedding by leveraging sine and cosine functions of the noise levels.

Training the diffusion model is essential for generating higher quality showers across different noise levels. We delve into two questions: 1. How can the data with added noise be correctly weighted and learned by the model so it can be denoised more easily in the backward process? 2. How can the training converges faster to a better reliable result?

Training Details

We follow the way from [47] to rewrite the score function to a denoiser function which minimizing the L2L^2 denoising error between xx and model output D(x)D(x) at each noise level:

∇xlog⁡pσ(x)=(D(x)−x)/σ2(8)\nabla_x \log p_\sigma(x) = (D(x)-x)/\sigma^2 \tag*{(8)}

If the network is preconditioned with σ\sigma independent skip connections, DD can be written as

Dθ(x,σ)=cskip(σ)x+cout(σ)Fθ(cin(σ)x,σ,z).D_\theta(x,\sigma) = c_{\mathrm{skip}}(\sigma)x + c_{\mathrm{out}}(\sigma)F_\theta(c_{\mathrm{in}}(\sigma)x,\sigma,z).

A set of preconditioning parameters used to predict noise ϵ\epsilon with this denoiser function:

L=Eσ,ϵ[w(σ)∥Dθ(x,σ)−x0∥22]=Eσ,ϵ[w(σ)∥Fθ(cin∗xσ,σ,z)−1cout(x0−cskip(σ)∗(xσ))∥22](9)\begin{aligned} L &= \mathbb{E}_{\sigma,\epsilon}[w(\sigma)\lVert D_\theta(x,\sigma)-x_0\rVert_2^2] \\ &= \mathbb{E}_{\sigma,\epsilon}\left[w(\sigma)\left\lVert F_\theta(c_{\mathrm{in}}*x_\sigma,\sigma,z)-\frac{1}{c_{\mathrm{out}}}(x_0-c_{\mathrm{skip}}(\sigma)*(x_\sigma))\right\rVert_2^2\right] \tag*{(9)} \end{aligned}

where FθF_\theta is the preconditioned network, cin,coutc_{\mathrm{in}},c_{\mathrm{out}} maps the input and output data magnitude, and cskipc_{\mathrm{skip}} controls the skip connection of the network. The parameters cin=1/σ2+σc2c_{\mathrm{in}}=1/\sqrt{\sigma^2+\sigma_c^2} and cout=(σ∗σc)/t2+σc2c_{\mathrm{out}}=(\sigma*\sigma_c)/\sqrt{t^2+\sigma_c^2}, σ\sigma is the current noise levels, σc\sigma_c is the constant value, a reasonable value is around [0.5,1.5][0.5,1.5] to avoid large variation for gradient scale. cskip=σc2/(σ2+σc2)c_{\mathrm{skip}}=\sigma_c^2/(\sigma^2+\sigma_c^2) is important quantity to partially mask the skip connection when the noise level goes large. This can prevent the network from inaccurate prediction. The setting is important in our case given noise varies widely in most of samplers we show later. When the σ\sigma goes higher, the cskipc_{\mathrm{skip}} becomes smaller. In a way, this compels the skip connection to be deactivated, encouraging the network to prioritize learning the signal for overriding the input rather than predicting noise to correct it. We choose the continuous timestep training where log⁡t=N(Pmean,Pstd2)\log t=\mathcal{N}(P_{\mathrm{mean}},P_{\mathrm{std}}^2) where Pmean,PstdP_{\mathrm{mean}},P_{\mathrm{std}} is -1.2 and 1.2 correspondingly.

From our study, the choice of the weighting function is very important as it not only determines the speed of loss convergence but also the quality of the outputs and how the specific energy distribution tends to evolve across different time steps. To mitigate the instability of the optimization over the training, and balance the gradient magnitude over diverse noise levels, one loss weight category is often mentioned — Signal to noise ratio (SNR) weighting [32, 54, 55, 56]. We choose the min-SNR weighting as it is found to be beneficial for training high-resolutions dataset. The original min-SNR weighting proposed from [57] defined to be min⁡{SNR⁡(t),γ}\min\{\operatorname{SNR}(t),\gamma\}, where SNR⁡(t)\operatorname{SNR}(t) is defined as 1−σ2σ2\frac{1-\sigma^2}{\sigma^2}, γ\gamma is the upper limit for weighting value, usually set to be 2-5. This could avoid the model putting too much weight on the low noise level.

In stead of directly clamping a constant value, we propose a softer version of min-SNR weighting which is more flexible to find the optimal value for better training. Specifically, the weight function is

w(t)=(σ∗σc)2(σ2+σck)2(10)w(t) = \frac{(\sigma* \sigma_c)^2}{(\sigma^2 + \sigma_c^k)^2} \tag*{(10)}

that σ\sigma and σc\sigma_c are the same as the preconditioning parameters, the reasonable value of k in the range of [2, 4]. This function has one minimum value of 0 around zero t and two local maxima on positive and negative t-axes. The constant k determines the local maximum weight value, and the SNR value where the maxima start to smoothly fall down to 0. The advantages of this weighting scheme will be shown later in Sec 5 with comparison of default constant (unit) weighting strategy.

Performance

Various schemes for the backward diffusion process have been discussed in previous section. To comprehensively compare their performance and enhance understanding of choosing the appropriate samplers and schedulers for a specific purpose, we offer several methods for evaluation.

CaloChallenge

There are multiple important distributions related to the layer and voxel energies for CaloChallenge dataset. We compare the generated samples with GEANT4 as reference. The voxel energy distribution is defined as the distribution of raw generated low level voxel energies. The total energy distribution is defined as the distribution of total energy of the generated shower. The center energy of the shower is defined as xˉ=⟨xiEi⟩∑iEi\bar{x} = \frac{\langle x_i E_i\rangle}{\sum_i E_i} for cell location xix_i and energy EiE_i, and shower width is defined as x2‾−xˉ2\sqrt{\overline{x^2}-\bar{x}^2}, we will show these two quantities at the polar coordinate α\alpha and rr. The mean layer energy distribution defined as the mean of generated layer energies. The shower response EratioE_{\mathrm{ratio}} is the total shower energy divided by the incident particle energies, this quantity is extremely important for energy calibration of many particles at LHC. As it will be used to correct the deposited shower energy inside calorimeter into the real particle energy, the small difference between generated sample and GEANT4 means more consistency for particle reconstruction procedure of both fast and full simulation. We will also evaluate the separation power between generated EratioE_{\mathrm{ratio}} and reference one.

In addition, a public evaluation network [37] DNN classifier with 2 layers, each comprising 2048 nodes and a 0.2 dropout rate, is employed to assess the area under the receiver operating characteristic curve (AUC) for all high-level features, including center energies, shower width, and layer energies, of both generated samples and references. An AUC value that close to 0.5 suggests that the generated and target classes are nearly indistinguishable. Conversely, a higher AUC value indicates a more evident distinction between the quantities of the two classes.

First, a comparison plot for EratioE_{\mathrm{ratio}} with default EDM training loss weight and our proposed training weight is shown in Figure 2. The choice of the weighting function influences how the kinematics distribution evolves over time. After 200 training epochs, the model trained with the default weighting still struggles to learn the EratioE_{\mathrm{ratio}} distribution. In contrast, employing proposed min-SNR weighting results in a better alignments with reference histogram, the generated EratioE_{\mathrm{ratio}} aligning with GEANT4 sample after only 100 epochs. This observation affirms that min-SNR weighting expedites the convergence of high-level features to achieve better performance compared to constant weighting.2

Figure 2. The distribution for total shower energy divided by the incident particle energy, tale shaded region is the GEANT4 reference, the solid line is the generated distribution by 79 steps EDM samplers with different weight at 30, 50, 70, 100, 200 epochs. Purple: default denoiser training with unit weight. Pink: proposed min-SNR weighting. The default training still struggles to learn this variable after 200 epochs whereas the training with proposed weighting strategy already gives good shape alignment after 100 epochs.

Few plots for center energy in rr direction are selected in Figure 3. We choose the EDM Heun SDE and Restart samplers that have good sample quality and inference speed trade-off. Notably, the EDM samplers and Restart samplers with Karras schedulers can learn this quantities really well. The ratio difference is almost indistinguishable with naked eyes. The default scheduler for the EDM sampler is Karras. To better illustrate the impact of different noise schedulers on the same sampler, a broader exploration of scheduler choices—such as Karras, DPM++ , and Lu schedulers with varying decay rates for σ\sigma is conducted. For small (∼30\sim30) steps EDM, the differences in last three radial bins are larger meaning it underestimates the lower energy region. When the steps become larger, we see the EDM predicts well in all radial bins. The Restart sampler can achieve this with only 30 steps. Given the Restart samplers works with a predefined parameters for configuring the EDM ODE, the generation time is directly proportional with the restart iterations in the configuration, usually longer than default EDM samplers. A lack of principled way for these hyperparameters is the inherent limitation for this sampler. Later we will see the best performance is achieved around 30-70 steps as we only use specific configurations in this range.

Comparison plots for average center energy of 5000 showers in radial bins

Figure 3. Comparison plots for average center energy of 5000 showers in radial bins. left: EDM with 30 steps, middle: EDM with 79 steps, right: Restart with 30 steps. Those samplers are chosen because of their good sample quality and inference speed trade-off, while the difference almost indistinguishable.

Then the mean layer energies, total shower energies, shower response, and voxel energies for the same three cases are shown in Figure 4, 5, 6.

Comparison plots for mean layer energies of 5000 showers

Figure 4. Comparison plots for mean layer energies of 5000 showers. left: EDM with 30 steps, middle: EDM with 79 steps, right: Restart with 30 steps.

The low steps EDM sampler has good performance on low energy voxel while mediocre performance on high energy tails of voxel distributions. In Figure 5, we have seen the shape of shower response between generated and reference sample for Restart and EDM samplers are more aligned than other samplers, especially around the peak of histogram (∼0.8\sim0.8). DPM-solver sampler with Karras scheduler does very well on predicting the voxel energy around 15 steps, achieving an exceptional agreement on both low and high energy tails shown in the left plot of Figure 6. The energy thresholds also affect low voxel energies generated by different samplers. For the baseline comparison, we used the pre-trained CaloDiffusion [23] model for dataset 2 with default 400 steps DDPM has 4-10% difference in the intermediate energy level (5-1000 MeV), and less than 30% difference with GEANT4 until 0.7 MeV. A clear difference has been seen below 0.7 MeV. The same scenario happens to EDM, Restart and LMS samplers, but all three samplers show more robust performance (i.e. a smaller excess in low energy tail). Generated showers produced by EDM and Restart samplers both have much less difference with GEANT4 around the intermediate energy level. Two new ODE samplers, DPM-Solver and UniPC, hold only less than 10% difference in low energy region, nevertheless perform worse on other regions. Indeed, some of them are struggling to match the low voxel energy, the presence of the tail is probably a consequence of model itself and too low energy threshold. It is expected this deficiency should not have big impact on the downstream reconstruction of

Comparison histograms for shower energies

Figure 5. Comparison histograms for 100k shower energies. left: shower response, right: total shower energy from 400 steps DDPM (CaloDiffusion), 79 steps EDM, 30 steps Restart 36 steps LMS, 15 steps DPM-Solver ++, and 15 steps UniPC samplers

Comparison histograms for raw voxel energies

Figure 6. Comparison histograms for raw voxel energies of 100k showers: Left: 30 steps EDM, 79 steps EDM, 30 steps Restart samplers; Right: 400 steps DDPM (CaloDiffusion), 36 steps LMS, and 15 steps DPM-Solver ++ samplers. Bottom is the percentage of difference between Geant4 and generated samples. The upward arrows indicate that the differences fall beyond the window at certain voxel energy levels.

shower.

We showed the possible AUC values for high level feature classifiers with different samplers and schedulers by evaluating different samplers 20 times in Figure 7. DPM-solver sampler seems to perform poorly on predicting the low and high-level features simultaneously. The Uni-PC sampler has very similar performance with DPM-solver sampler and only gives slightly better performance over very low sampling (10-20) steps. The lowest AUC is around 57 at 50th step. The EDM and Restart samplers surpass the previous 400 steps DDPM samplers at 50, 25 steps (100, 60 function evaluations) correspondingly, combined with other plots we showed, achieving the same and better performance with a 3-8 times speedup. The LMS ODE sampler has a better performance than EDM below 45 steps, but reaches a worse convergence than other samplers. The best AUC value for Restart samplers achieve at 30-36 and 79 steps, this might because we use the predefined configuration specifically for those steps. All proposed samplers have comparable and better performance with smaller steps. Most of the samplers we use actually have more parameters to tune than the baseline DDPM. For the common ODE samplers, a closer matching noise level during sampling would often yield better results. A possible future development is to fine-tune the scheduler configuration for ODE samplers in higher steps.

Figure 7. AUC values for different samplers as a function of evaluation steps, the shaded area is possible AUC value range for 20 evaluations through independent classifier training. The effective ratio is defined as the ratio between actual AUC from generated sample with different solvers minus the best possible AUC value 0.5, divided by the AUC value from generated sample with DDPM minus 0.5 AUCgen−0.5AUCDDPM−0.5\frac{\mathrm{AUC}_{\mathrm{gen}}-0.5}{\mathrm{AUC}_{\mathrm{DDPM}}-0.5}. Lower is better, value below 1 means better performance than DDPM at certain evaluation steps.

Now we take a look at one of the most difficult and important quantities for the low level voxel diffusion – EratioE_{\mathrm{ratio}}. In Figure 8, we choose 6 cases in our study. First, much faster convergences have been seen from new introduced samplers by 5-20 times. The obtained result matches with the Table.Table 3 mentioned in Song’s [59]. For EDM sampler, the separation power, defined as the triangular discriminator [8] ∑i=1n(countdata,i−countgen,i)2∑i=1n(countdata,i+countgen,i)\frac{\sum_{i=1}^{n}(\mathrm{count}_{\mathrm{data},i}-\mathrm{count}_{\mathrm{gen},i})^2}{\sum_{i=1}^{n}(\mathrm{count}_{\mathrm{data},i}+\mathrm{count}_{\mathrm{gen},i})} which calculated from each bins of the histogram, quickly drops around 60 steps and achieves better performance than the baseline 400 steps DDPM sampler. The 20-30 steps DPM-Solver and UniPC samplers already have lower separation powers than 200 steps DDPM sampler. The 18 steps Restart sampler already could match the similar performance as the baseline. If we zoom-in the figure, at around 100-200 steps, EDM and Restart samplers have 3-5 times weaker separation power than the baseline DDPM samplers.

Separation power for $E_{\mathrm{ratio}}$ with different samplers and schedulers

Figure 8. Separation power for EratioE_{\mathrm{ratio}} with different samplers and schedulers as a function of evaluation steps, the shaded area is possible values of separation power for 20 independent samplings.

DatasetSChurnS_{\mathrm{Churn}}Smax⁡S_{\max}Smin⁡S_{\min}SnoiseS_{\mathrm{noise}}σmax⁡\sigma_{\max}σmin⁡\sigma_{\min}
CaloCha.[15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40][20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40][0.01-0.1][1.001-1.004][40-80][0.002-0.01]
JetNet[20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50][20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39][0.01-0.1][1.001-1.006][30-80][0.002-0.01]

Table 3. Summary table for Stochastic parameter and scheduler of EDM Heun Karras sampler

The preliminary Fréchet Particle Distance (FPD) [58] values for some samplers are also provided in Table 1. The FPD metric is relatively biased and unstable among subsamples we choose. So we follow the way from [23] to subtract the non-zero FPD value between two GEANT4 sample. Surprisingly, we find the EDM sampler converges even faster in this metric, surpassing the DDPM sampler around 36 steps (72 NFE). The 18-steps (∼40\sim40 NFE) Restart sampler has slightly worse performance, this is mainly because we choose the predefined configuration for smallest EratioE_{\mathrm{ratio}} separation power, a throughout fine-tune can make FPD value 20-30 percent smaller with almost same generation time. However, it still obtains the best FPD results with larger steps.

Samplers(steps)AUC ↓\downarrowSeparation power (10−210^{-2}) ↓\downarrowFPD ↓\downarrow
DDPM(79)61.45±0.0561.45 \pm0.058.10±1.058.10 \pm1.050.074±0.010.074 \pm0.01
DDPM(200)55.72±0.0355.72 \pm0.033.44±0.403.44 \pm0.400.046±0.0070.046 \pm0.007
DDPM(400)<u>54.6±0.0254.6 \pm0.02</u><u>1.55±0.301.55 \pm0.30</u><u>0.043±0.0070.043 \pm0.007</u>
EDM(36)55.3±0.0655.3 \pm0.062.56±0.722.56 \pm0.720.035±0.0070.035 \pm0.007
EDM(79)52.9±0.0452.9 \pm0.040.76±0.200.76 \pm0.200.027±0.0040.027 \pm0.004
EDM(200)52.6±0.03†52.6 \pm0.03^{\dagger}0.55±0.10†0.55 \pm0.10^{\dagger}0.023±0.004‡0.023 \pm0.004^{\ddagger}
EDM(400)52.6±0.01∗52.6 \pm0.01^{*}0.16±0.05∗0.16 \pm0.05^{*}0.021±0.003∗0.021 \pm0.003^{*}
EDM_DPM++(79)53.0±0.0453.0 \pm0.040.82±0.200.82 \pm0.200.026±0.0040.026 \pm0.004
EDM_Lu(79)52.8±0.0452.8 \pm0.040.86±0.200.86 \pm0.200.026±0.0040.026 \pm0.004
Restart(18)56.7±0.0756.7 \pm0.071.69±0.661.69 \pm0.660.059±0.0090.059 \pm0.009
Restart(36)54.0±0.0654.0 \pm0.060.73±0.260.73 \pm0.260.025±0.0030.025 \pm0.003
Restart(79)53.9±0.05‡53.9 \pm0.05^{\ddagger}0.57±0.19‡0.57 \pm0.19^{\ddagger}0.022±0.004†0.022 \pm0.004^{\dagger}
LMS(36)56.0±0.0456.0 \pm0.046.65±1.356.65 \pm1.350.095±0.010.095 \pm0.01
DPM++(25)56.8±0.0756.8 \pm0.072.52±0.512.52 \pm0.510.113±0.010.113 \pm0.01
UniPC(25)56.9±0.0756.9 \pm0.072.31±0.452.31 \pm0.450.112±0.010.112 \pm0.01

Table 1. Summary table for high-level feature classifier mean AUC, EratioE_{\mathrm{ratio}} separation power, and Frechét Particle Distance [58] of of different samplers. Underline is the 400-step DDPM sampler (CaloDiffusion) as baseline, the *, †, ‡, are the first, second, third best results. Bold is the highlighted result with either great performance or good quality-sampling time balance.

The separation power is decreased in smaller steps (≤100\le100) but increased in larger steps by changing the EDM scheduler from Karras to Lu. The choice of schedulers provides a training-free method to balance the quality and speed of the backward diffusion process. The EDM/DPM-Solver combined sampler with Karras scheduler performs pretty similarly as the EDM sampler with Lu scheduler. The best performances of Restart samplers again are achieved in 30-36 steps. This sampler is crafted for efficiently conducting fast generation with outstanding performance and can be fine-tuned with post-training configuration. The DPM-solver exhibits a rapid drop in separation power within 10-20 steps but ceases to improve beyond those steps, achieving better performance than the 100-step DDPM around the 15{sup}th step. The plot further confirms that ODE sampler does better in low sampling steps as the discretization error of them are smaller than SDE ones in this region, the contraction of accumulated errors from SDE sampling process become much stronger than ODE so that a better convergence observed in higher sampling steps than ODE ones [48]. Some ODE samplers, such as LMS and DPM-solver, perform well on some metrics but not all. There is not a clearly superior option that can achieve the best quality while having the lowest generation time.

Image not extracted

Figure 10. Comparison plots for relative angular coordinate of particle constituents inside top quark jets with 200 steps DDIM Euler Sampler, 18 steps multi-level Restart Sampler, 50 steps EDM samplers . left: Δη\Delta\eta, right: Δϕ\Delta\phi.

JetNet

We employ network training techniques with the same continuous noise level and representative samplers for the JetNet dataset to demonstrate that these methods are entirely portable across various datasets. As depicted in Figure 9 and 10, the low level feature generations with a 50-steps EDM sampler and 18-steps Restart sampler show better performance than a 200-steps DDIM [36] Euler sampler on gluon jets. For more complicated topology, EDM and Restart have advantages in simulating the tails of the distribution over large η\eta region. EDM sampler has small deficiency for underestimating the particles from top quark jets after Δϕ>0.2\Delta\phi> 0.2. This would have impact on reconstructing the jet level variables in Figure 11.

Comparison plots for relative angular coordinates of particle constituents inside gluon initiated jets

Figure 9. Comparison plots for relative angular coordinate of particle constituents inside gluon initiated jets with 200 steps DDIM Euler Sampler, 18 steps multi-level Restart Sampler, 50 steps EDM samplers . left: Δη\Delta\eta, right: Δϕ\Delta\phi.

Image not extracted

Figure 11. Comparison plots for substructure of particle constituents inside gluon/-top quark jets with different samplers and schedulers.

The same training setup in PC-JeDi [25] is used with DDPM samplers as baseline. We found the Euler samplers could give a better performance with our new proposed training. Both EDM and Restart samplers capture most of the high level gluon jet features, with only few percent difference in relative transverse momentum and mass of jets. The improvement is clearer in more challenging substructure variables such as the n-subjetiness between subleading and third leading jets - τ32\tau_{32} when comparing with default DDPM and Restart samplers. Although some ODE samplers outperform than DDPM on τ32\tau_{32} of gluon jets, they can not achieve better overall performance, except LMS samplers. The largest discrepancy for this quantity appears around 0.7-0.9 region where the baseline DDPM sampler has around 30-40% difference. The low-steps Restart sampler gives obviously better generated distribution than others with less than 10% difference over most regions.

While simulating the complex distribution in a highly boosted environment, most of the samplers fail to achieve satisfactory alignment around the two asymmetric peaks in the relative mass distribution for top jets. We then anatomise the performance of proposed samplers via the three low level features with the same DNN classifier used in previous section, the values for the Fréchet Particle distance (FPD) calculated from the ParticleNet [60] multi-classifier with N=6N = 6, and the unbinned metrics (Wasserstein distance) for reconstructed individual high level features. Performances are summarized on Table. Table 2. The Restart sampler provides the consistently best results for both top and gluon jets in all three metrics. Different scheduler variants on SDE samplers should not have too much influence on results. The 25-steps DPMSolver sampler shows competitive result as the 200-steps traditional Euler and DDPM samplers on gluon jets.

Samplers(NFE)AUC ↓\downarrowFPD ↓\downarrowWm(10−4)W_m(10^{-4})WpT(10−4)W_{p_T}(10^{-4})Wτ32(10−3)↓W_{\tau_{32}}(10^{-3})\downarrow
Top
DDPM(200)52.8±0.040.16±0.0313.5412.0416.05
Euler(200)52.6±0.040.15±0.0320.4710.0811.54
EDM(100)51.9±0.03†0.07±0.01†7.65†7.07†8.32†
EDM_DPM++(100)52.0±0.02‡0.08±0.005‡8.05‡7.17‡8.34‡
Restart(70)51.3±0.03*0.02±0.005*4.30*5.65*5.90*
DPM++(25)53.7±0.050.20±0.0425.6410.4525.76
UniPC(25)54.0±0.070.22±0.0327.8910.4926.04
Gluon
DDPM(200)52.5±0.040.10±0.035.696.0114.30
Euler(200)52.6±0.040.14±0.046.406.634.79†
EDM(100)51.9±0.03†0.07±0.01†5.03†4.65†4.92
EDM_DPM++(100)52.0±0.03‡0.08±0.01‡5.40‡4.59‡5.03‡
Restart(70)51.2±0.02*0.02±0.007*3.03*3.20*3.15*
DPM++(25)52.8±0.050.16±0.026.059.9710.09
UniPC(25)52.9±0.050.16±0.026.159.985.41

Table 2. Summary table for low-level feature classifier mean AUC, Frechét Particle Distance calculated from jet-level classifier, and high-level features unbinned (Wasserstein) distance of different samplers on gluon-initiated jets top and gluon quark jets. Underline is the 400-step DDPM sampler (PC-JeDi) as baseline, the *, †, ‡, are the first, second, third best results. Bold is the highlighted result with either great performance or good quality-sampling time balance.

In Figure 12- Figure 13, Restart sampler yields the best performance even in small number of function evaluations. Even though the boosts brought by those changes isn’t as pronounced as they are in CaloChallenge, high step EDM and Restart samplers still have much lower FPD values than other samplers.

Image not extracted

Figure 12. Comparison plots for FPD values for top jets as a function of evaluation steps with different samplers and schedulers. The shaded area is possible values of separation power for 10 independent samplings.

Image not extracted

Figure 13. Comparison plots for FPD values for gluon jet as a function of evaluation steps with different samplers and schedulers. the shaded area is possible values of separation power for 10 independent samplings.

Conclusion and outlooks

In this study, we conducted a comprehensive investigation into the utilization of various state-of-the-art samplers and schedulers across diverse datasets in collider physics. We introduced a soft version of loss weighting techniques to expedite the convergence of physics-informed quantities towards a better reliable result. Our study presents several approaches for evaluating generated samples across different scenarios, providing a range of options for accelerating diffusion model without additional training or achieving superior results with fewer enhancements. Innovative sampling methods, such as Restart, exhibit better performance with just 18-30 steps, leading to a O(10)O(10) times speeding up without a significant performance loss. We showcase a potentially new baseline for training and sampling diffusion models in this rapidly evolving field.

Future development will aim to identify the optimal sets for these samplers on each dataset to achieve the maximum possible boost. Additionally, exploration into other areas, such as score distillation sampling and model fine-tuning, will be undertaken to obtain similar performance levels with even faster simulations.

Acknowledgements

We thank Kevin Pedro and Oz Amram for fruitful discussions about details of CaloDiffusion project, and insightful comments for different evaluation metrics.

Hyperparameters of Samplers and Schedulers

We will list more details about the hyperparameter range we ended up choosing for each sampler.

For the Stochastic parameter we listed on Table 3, the choices of SChurnS_{\mathrm{Churn}} and Smin⁡S_{\min} are very important. In the CaloChallenge dataset, a too small SChurnS_{\mathrm{Churn}} would make the prediction worse over lower energy region, resulting a higher EratioE_{\mathrm{ratio}} separation power, and a high SChurnS_{\mathrm{Churn}} would make noise injection power too high over low sampling step region to converge quickly. With our training setup, we usually set a larger SChurnS_{\mathrm{Churn}} on the Jetnet dataset. That is also one reason why a smaller improvement is observed here. The SnoiseS_{\mathrm{noise}} does not affect too much on both dataset.

We also find several ways to constrain the multilevel configuration for generating satisfied results with Restart sampler by empirical evaluations.

Oftentimes different NstepsN_{\mathrm{steps}} across different level (range of time steps) yield better FPD value. Gradually increasing NstepsN_{\mathrm{steps}} toward last few steps give slightly better result than constant NstepsN_{\mathrm{steps}} among all levels. With a larger KK iteration value in 3{sup}rd and 5{sup}th, we obtain the best results in current sampling steps, but also increase the generation time by few percent.

For DPM-solver, we choose a slightly different noise level. The βmax⁡,βmin⁡\beta_{\max},\beta_{\min} for VP scheduler is 19.9, 0.002. For Karras and Lu scheduler, σmax⁡,σmin⁡\sigma_{\max},\sigma_{\min} is [15, 40], [0.001, 0.01]. For LMS, we choose [3, 4, 6]{sup}th order of calculation. The UniPC sampler is using 2{sup}nd order B(h)B(h)-2 solver with the same scheduler as DPM-solver.

levelNstepsN_{\mathrm{steps}}KKσmax⁡\sigma_{\max}σmin⁡\sigma_{\min}
0[2-5][2\text{-}5][2-15][2\text{-}15][1.09,0.30][1.09,0.30][0.14,0.06][0.14,0.06]
1[2-5][2\text{-}5][1-2][1\text{-}2][40.79][40.79][19.35][19.35]
2[3-6][3\text{-}6][1-2][1\text{-}2][1.92][1.92][1.09][1.09]
3[4-6][4\text{-}6][2-4][2\text{-}4][1.09,0.59,0.30][1.09,0.59,0.30][0.59,0.06][0.59,0.06]
4[4-6][4\text{-}6][1-4][1\text{-}4][0.59,0.30][0.59,0.30][0.30,0.06][0.30,0.06]
5[4-7][4\text{-}7][2-6][2\text{-}6][0.30][0.30][0.06][0.06]

Table 4. Summary table for good parameter of Restart Karras sampler, 0 denoted as the single level Restart.

Image not extracted

Figure 14. FPD values in CaloChallenge for 36 steps multilevel Restart samplers with 25 different configurations

References

  1. [1]L. Evans and P. Bryant, LHC Machine, Journal of Instrumentation 3(08), S08001 (2008), doi:10.1088/1748-0221/3/08/S08001.DOI
  2. [2]G. Aad et al., The ATLAS Experiment at the CERN Large Hadron Collider, JINST 3, S08003 (2008), doi:10.1088/1748-0221/3/08/S08003.DOI
  3. [3]S. Chatrchyan et al., The CMS Experiment at the CERN LHC, JINST 3, S08004 (2008), doi:10.1088/1748-0221/3/08/S08004.DOI
  4. [4]S. Agostinelli et al., GEANT4–a simulation toolkit, Nucl. Instrum. Meth. A 506, 250 (2003), doi:10.1016/S0168-9002(03)01368-8.DOI
  5. [5]S. Amoroso et al., Challenges in Monte Carlo Event Generator Software for High-Luminosity LHC, Comput. Softw. Big Sci. 5(1), 12 (2021), doi:10.1007/s41781-021-00055-1, 2004.13687.DOI
  6. [6]G. Apollinari, O. Brüning, T. Nakamoto and L. Rossi, High Luminosity Large Hadron Collider HL-LHC, doi:10.5170/CERN-2015-005.1 (2015).DOI
  7. [7]M. Paganini, L. de Oliveira and B. Nachman, CaloGAN: Simulating 3D high energy particle showers in multilayer electromagnetic calorimeters with generative adversarial networks, Physical Review D 97(1) (2018), doi:10.1103/physrevd.97.014021.DOI
  8. [8]C. Krause and D. Shih, Fast and accurate simulations of calorimeter showers with normalizing flows, Physical Review D 107(11) (2023), doi:10.1103/physrevd.107.113003.DOI
  9. [9]R. Zhang, AtlFast3, the ATLAS fast simulation tool in Run3 (2023).
  10. [10]E. Buhmann, S. Diefenbacher, E. Eren, F. Gaede, D. Hundhausen, G. Kасieczka, W. Kоrcari, K. Krüger, P. McKeown and L. Rustige, Hadrons, better, faster, stronger (2021), 2112.09709.
  11. [11]I. Pang, J. A. Raine and D. Shih, Supercalo: Calorimeter shower super-resolution (2024), 2308.11700.
  12. [12]M. F. Giannelli and R. Zhang, Caloshowergan, a generative adversarial networks model for fast calorimeter shower simulation (2023), 2309.06515.DOI
  13. [13]S. Hoque, H. Jia, A. Abhishek, M. Fadaie, J. Q. Toledo-Marín, T. Vale, R. G. Melko, M. Swiatlowski and W. T. Fedorko, Caloqvae : Simulating high-energy particle-calorimeter interactions using hybrid quantum-classical generative models (2023), 2312.03179.arxiv.org/abs/2312.03179
  14. [14]J. Birk, E. Buhmann, C. Ewen, G. Kasieczka and D. Shih, Flow matching beyond kinematics: Generating jets with particle-id and trajectory displacement information (2023), 2312.00123.DOI
  15. [15]S. Bieringer, A. Butter, S. Diefenbacher, E. Eren, F. Gaede, D. Hundhausen, G. Kasieczka, B. Nachman, T. Plehn and M. Trabs, Calomplification — the power of generative calorimeter models, Journal of Instrumentation 17(09), P09028 (2022), doi:10.1088/1748-0221/17/09/P09028.DOI
  16. [16]M. A. W. Scham, D. Krücker, B. Käch and K. Borras, Deeptregan: Fast generation of high dimensional point clouds (2023), 2311.12616.
  17. [17]K. Dohi, Variational autoencoders for jet simulation (2020), 2009.04842.arxiv.org/abs/2009.04842
  18. [18]C. Krause and D. Shih, Caloflow ii: Even faster and still accurate generation of calorimeter showers with normalizing flows (2023), 2110.11377.
  19. [19]C. Krause, B. Nachman, I. Pang, D. Shih and Y. Zhu, Anomaly detection with flow-based fast calorimeter simulators (2023), 2312.11618.DOI
  20. [20]A. Butter, T. Jezo, M. Klasen, M. Kuschick, S. Palacios Schweitzer and T. Plehn, Kicking it Off(-shell) with Direct Diffusion (2023), 2311.17175.arxiv.org/abs/2311.17175
  21. [21]A. Butter, N. Huetsch, S. Palacios Schweitzer, T. Plehn, P. Sorrenson and J. Spinner, Jet Diffusion versus JetGPT – Modern Networks for the LHC (2023), 2305.10475.
  22. [22]V. Mikuni and B. Nachman, Score-based generative models for calorimeter shower simulation, Physical Review D 106(9) (2022), doi:10.1103/physrevd.106.092009.DOI
  23. [23]O. Amram and K. Pedro, Denoising diffusion models with geometry adaptation for high fidelity calorimeter simulation (2023), 2308.03876.arxiv.org/abs/2308.03876
  24. [24]E. Buhmann, S. Diefenbacher, E. Eren, F. Gaede, G. Kasieczka, A. Korol, W. Korcari, K. Krüger and P. McKeown, Caloclouds: Fast geometry-independent highly-granular calorimeter simulation (2023), 2305.04847.DOI
  25. [25]M. Leigh, D. Sengupta, G. Quétant, J. A. Raine, K. Zoch and T. Golling, Pc-jedi: Diffusion for particle cloud generation in high energy physics (2023), 2303.05376.DOI
  26. [26]E. Buhmann, C. Ewen, D. A. Faroughy, T. Golling, G. Kasieczka, M. Leigh, G. Quétant, J. A. Raine, D. Sengupta and D. Shih, Epic-ly fast particle cloud generation with flow-matching and diffusion (2023), 2310.00049.arxiv.org/abs/2310.00049
  27. [27]V. Mikuni and B. Nachman, Caloscore v2: Single-shot calorimeter shower simulation with diffusion models (2023), 2308.03847.DOI
  28. [28]T. Salimans and J. Ho, Progressive distillation for fast sampling of diffusion models (2022), 2202.00512.arxiv.org/abs/2202.00512
  29. [29]E. Buhmann, F. Gaede, G. Kasieczka, A. Korol, W. Korcari, K. Krüger and P. McKeown, Caloclouds ii: Ultra-fast geometry-independent highly-granular calorimeter simulation (2023), 2309.05704.arxiv.org/abs/2309.05704
  30. [30]Y. Song, P. Dhariwal, M. Chen and I. Sutskever, Consistency models (2023), 2303.01469.arxiv.org/abs/2303.01469
  31. [31]M. Leigh, D. Sengupta, J. A. Raine, G. Quétant and T. Golling, Pc-droid: Faster diffusion and improved quality for particle cloud generation (2023), 2307.06836.DOI
  32. [32]R. Rombach, A. Blattmann, D. Lorenz, P. Esser and B. Ommer, High-resolution image synthesis with latent diffusion models (2022), 2112.10752.arxiv.org/abs/2112.10752
  33. [33]N. Ruiz, Y. Li, V. Jampani, Y. Pritch, M. Rubinstein and K. Aberman, Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation (2023), 2208.12242.arxiv.org/abs/2208.12242
  34. [34]L. Zhang, A. Rao and M. Agrawala, Adding conditional control to text-to-image diffusion models (2023), 2302.05543.arxiv.org/abs/2302.05543
  35. [35]S. Zhai, Y. Dong, Q. Shen, S. Pu, Y. Fang and H. Su, Text-to-image diffusion models can be easily backdoored through multimodal data poisoning (2023), 2305.04175.arxiv.org/abs/2305.04175
  36. [36]J. Song, C. Meng and S. Ermon, Denoising diffusion implicit models (2022), 2010.02502.
  37. [37]M. F. Giannelli, G. Kasieczka, B. N. Claudius Krause, D. Salamani, D. Shih and A. Zaborowska, Fast calorimeter simulation challenge 2022 (2022).
  38. [38]C. Krause, M. F. Giannelli, G. Kasieczka, B. Nachman and et al., CaloChallenge 2022: A community challenge for fast calorimeter simulation (2024), 2410.21611.
  39. [39]R. Kansal, J. Duarte, H. Su, B. Orzari, T. Tomei, M. Pierini, M. Touranakou, J.-R. Vlimant and D. Gunopulos, Particle cloud generation with message passing generative adversarial networks (2022), 2106.11535.arxiv.org/abs/2106.11535
  40. [40]S. Diefenbacher, E. Eren, G. Kasieczka, A. Korol, B. Nachman and D. Shih, Dctgran: improving the precision of generative models with reweighting, Journal of Instrumentation 15(11), P11004–P11004 (2020), doi:10.1088/1748-0221/15/11/p11004.DOI
  41. [41]J. Sohl-Dickstein, E. A. Weiss, N. Maheswaranathan and S. Ganguli, Deep unsupervised learning using nonequilibrium thermodynamics (2015), 1503.03585.
  42. [42]L. Yang, Z. Zhang, Y. Song, S. Hong, R. Xu, Y. Zhao, W. Zhang, B. Cui and M.-H. Yang, Diffusion models: A comprehensive survey of methods and applications (2023), 2209.00796.arxiv.org/abs/2209.00796
  43. [43]B. B. Moser, A. S. Shanbhag, F. Raue, S. Frolov, S. Palacio and A. Dengel, Diffusion models, image super-resolution and everything: A survey (2024), 2401.00736.DOI
  44. [44]J. Ho, A. Jain and P. Abbeel, Denoising diffusion probabilistic models (2020), 2006.11239.arxiv.org/abs/2006.11239
  45. [45]A. Nichol and P. Dhariwal, Improved denoising diffusion probabilistic models (2021), 2102.09672.arxiv.org/abs/2102.09672
  46. [46]Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon and B. Poole, Score-based generative modeling through stochastic differential equations (2021), 2011.13456.
  47. [47]T. Karras, M. Aittala, T. Aila and S. Laine, Elucidating the design space of diffusion-based generative models (2022), 2206.00364.arxiv.org/abs/2206.00364
  48. [48]Y. Xu, M. Deng, X. Cheng, Y. Tian, Z. Liu and T. Jaakkola, Restart sampling for improving generative processes (2023), 2306.14878.arxiv.org/abs/2306.14878
  49. [49]C. Lu, Y. Zhou, F. Bao, J. Chen, C. Li and J. Zhu, Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models (2023), 2211.01095.DOI
  50. [50]C. Lu, Y. Zhou, F. Bao, J. Chen, C. Li and J. Zhu, Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps (2022), 2206.00927.arxiv.org/abs/2206.00927
  51. [51]Runge–Kutta Methods, chap. 3, pp. 137–316, John Wiley & Sons, Ltd, ISBN 9780470753767, doi:https://doi.org/10.1002/9780470753767.ch3 (2008).
  52. [52]W. Zhao, L. Bai, Y. Rao, J. Zhou and J. Lu, Unipc: A unified predictor-corrector framework for fast sampling of diffusion models (2023), 2302.04867.arxiv.org/abs/2302.04867
  53. [53]M. Cacciari, G. P. Salam and G. Soyez, The anti-ktjet clustering algorithm, Journal of High Energy Physics 2008(04), 063–063 (2008), doi:10.1088/1126-6708/2008/04/063.
  54. [54]P. Dhariwal and A. Nichol, Diffusion models beat gans on image synthesis (2021), 2105.05233.
  55. [55]C. Meng, R. Rombach, R. Gao, D. P. Kingma, S. Ermon, J. Ho and T. Salimans, On distillation of guided diffusion models (2023), 2210.03142.arxiv.org/abs/2210.03142
  56. [56]J. Ho, T. Salimans, A. Gritsenko, W. Chan, M. Norouzi and D. J. Fleet, Video diffusion models (2022), 2204.03458.
  57. [57]T. Hang, S. Gu, C. Li, J. Bao, D. Chen, H. Hu, X. Geng and B. Guo, Efficient diffusion training via min-snr weighting strategy (2023), 2303.09556.arxiv.org/abs/2303.09556
  58. [58]R. Kansal, A. Li, J. Duarte, N. Chernyavskaya, M. Pierini, B. Orzari and T. Tomei, Evaluating generative models in high energy physics, Physical Review D 107(7) (2023), doi:10.1103/physrevd.107.076017.DOI
  59. [59]Y. Song and P. Dhariwal, Improved techniques for training consistency models (2023), 2310.14189.arxiv.org/abs/2310.14189
  60. [60]H. Qu and L. Gouskos, Jet tagging via particle clouds, Physical Review D 101(5) (2020), doi:10.1103/physrevd.101.056019.DOI

Paper details

vlnc:W_9rufn3k9
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents