betaFeedback

WorksPhysics and Astronomy

Cross-Geometry Transfer Learning in Fast Electromagnetic Shower Simulation

Abstract

Accurate particle shower simulation remains a critical computational bottleneck for high-energy physics. Traditional Monte Carlo methods, such as Geant4, are computationally prohibitive, while existing machine learning surrogates are tied to specific detector geometries and require complete retraining for each design change or alternative detector. We present a transfer learning methodology for generative calorimeter simulation models that enables adaptation across diverse geometries with high data efficiency. Using point cloud representations and pre-training on the International Large Detector, our approach handles new configurations without re-voxelizing showers for each geometry. On the CaloChallenge dataset, transfer learning with only 100 target-domain samples achieves a 51%51\% improvement on the geometric mean of Wasserstein distance over training from scratch. Parameter-efficient fine-tuning with bias-only adaptation achieves competitive performance while updating only 17%17\% of model parameters. Our analysis provides insight into adaptation mechanisms for particle shower development, establishing a baseline for future progress of point cloud approaches in calorimeter simulation.

Introduction

The next decade of large-scale experiments in high-energy physics (HEP) will produce experimental data at unprecedented volumes. This increase is driven by the higher collision rates expected at the High-Luminosity Large Hadron Collider (HL-LHC) and by the deployment of high-granularity detectors with an expanding number of readout channels [1]. While Geant4 [2] provides accurate physics simulation, a single HL-LHC event may require minutes of CPU time to simulate [3]. In particular, calorimeter shower development constitutes the dominant computational bottleneck in detector simulation [4, 5, 6, 7, 8]. This growing computational demand cannot be satisfied solely by hardware improvements. Single-core CPU performance has essentially plateaued: Moore’s Law scaling no longer delivers the improvements we need [9], making fundamental algorithmic innovations essential rather than relying on incremental optimisations.

The HEP community has looked to machine learning as a potential acceleration method in response to these computing limitations. These fast simulation (FastSim) techniques learn to predict the final detector response directly from incident particle attributes instead of modelling particle interactions step-by-step through detector materials, potentially leading to orders of magnitude speedups. Recently, significant progress has been made in the development of surrogate simulators based on generative modeling approaches [10, 11, 12, 13], ranging from generative adversarial networks [14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30], to variational auto-encoders [31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42], flow-based models [43, 44, 45, 46, 47, 48, 49, 50, 51, 52], diffusion models [53, 54, 55, 56, 57, 58, 59, 60] and autoregressive models [61, 62, 63, 64]. While these methods have demonstrated impressive performance on standardised benchmarks [65], most remain tied to a specific detector geometry: when detector designs evolve, or conditions change during data taking, these models typically require complete retraining with new simulation datasets. Recent efforts have begun addressing this limitation through meta-learning [35], multi-geometry pre-training [66], and geometry-parametrised architectures [52, 62], but the general problem of efficient cross-geometry adaptation remains open.

Point cloud representations have emerged to address geometry dependence [67, 68, 69, 70, 71], generating showers as 3D space points with associated energy depositions that can, in principle, project onto arbitrary detector configurations. Recent work has demonstrated that point cloud models can achieve a favourable balance between speed and accuracy for highly granular calorimeter simulation in realistic applications [72], validating this representation choice for practical deployment. While this flexibility comes with computational overhead: variable-cardinality management, sparse representations with O(104)O(10^{4}) points, and complex detector reintegration, the more fundamental challenge is that representation flexibility alone does not guarantee successful transfer. Cross-geometry generalisation requires both the geometric flexibility of point clouds and the learnt physics knowledge that generalises across detectors. This work investigates whether single-detector pre-training on point clouds can provide both representation flexibility and model transferability, treating them as complementary rather than equivalent capabilities.

The foundation model paradigm from Natural Language Processing (NLP) and computer vision [73, 74, 75] offers a natural framework for developing generalisable simulation models. Early explorations include OmniJet-α\alpha, which introduced the first general-purpose HEP model for classification and jet generation [76, 77], demonstrating the feasibility of unifying multiple tasks within a single architecture. This was later extended to showers in OmniJet-αC\alpha_C [78], though it lacked any pre-training or adaptation mechanism.

More recently, CaloDiT-2 [66] demonstrated successful pre-training on four detector geometries from the LEMURS dataset [79], achieving effective transfer through standard fine-tuning, marking a first step towards a potential FastSim foundation [80]. Our work differs from CaloDiT-2 in two key aspects. First, in data scope, we focus on single-detector pre-training to explore scenarios where only one well-characterized detector dataset is available for pre-training, rather than requiring multiple diverse detector datasets as in CaloDiT-2. This reflects practical constraints where comprehensive simulation data may exist for established detectors but not for new designs under development. Second, in representation, we employ point clouds rather than fixed grids, trading some computational overhead for geometric flexibility and a direct match to the sparse nature of calorimeter showers. When combined with Parameter-Efficient Fine-Tuning (PEFT) [81], this approach aims to simplify the adaptation pipeline and reduce computational requirements for model scaling. This paper investigates the feasibility of single-detector transfer learning for point cloud calorimeter simulation, focusing on parameter-efficient adaptation strategies and the underlying physics transformations that influence transferability.

The remainder of this paper is structured as follows: § 2 details the model architecture and transfer learning methodology. § 3 describes the datasets used for pre-training and fine-tuning. § 4 defines the evaluation metrics and presents cross-calorimeter transfer learning results across different fine-tuning techniques. § 5 concludes with discussion and future directions.

Cross-Calorimeter Transfer Learning

This section describes the model architecture and transfer learning methodology used to adapt calorimeter simulation across different detector geometries.

Model Architecture

The present work uses the CaloClouds [69, 70, 71] network architecture as the base model to simulate electromagnetic showers across different calorimeter geometries. This framework comprises two complementary generative models:

PointWise Net employs a diffusion model following the EDM (Elucidating Design Space) framework [82] to generate the spatial coordinates (x,y,z)(x, y, z) and energy depositions ee of shower hits as continuous point clouds. A detailed description is available in Ref. [70]. The final layer produces the denoised point cloud prediction, which enables the generation to be independent of specific detector voxelization, allowing projection onto arbitrary geometric configurations. While point clouds provide flexibility in the transverse plane (x,y)(x, y), longitudinal variations in detector materials fundamentally alter the physics of shower development through changes in radiation length and interaction properties, requiring retraining rather than simple geometric projection.

ShowerFlow predicts the number of points per calorimeter layer Nz,iN_{z,i} that subsequently condition PointWise Net’s generation process. The architecture uses normalising flow blocks (detailed in Appendix § 9), trained to learn the relationship between incident particle energy and layer-wise shower occupancy. For ShowerFlow training, we apply a fixed-scale normalisation strategy that differs from the original CaloClouds implementation. Rather than normalising each event’s point counts to [0, 1] independently, a constant normalisation value norm_points = 800 is applied across all events for each calorimeter layer. This choice is motivated by the hypothesis that the event-wise normalisation might compress the ranges in ways that could obscure scale information relevant for transfer learning across datasets with different energy and occupancy distributions.

Transfer Learning Framework

The approach adapts a pre-trained model, initially trained on photon-induced showers in the International Large Detector (ILD) geometry, to enable unsupervised knowledge transfer to different calorimeter configurations. This methodology eliminates the requirement for labelled data correspondence that characterises supervised approaches in similar applications [76, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92], the conceptual approach is illustrated in Figure 1.

Figure 1. The transfer learning approach presented in this work. A model pre-trained on the ILD detector is adapted to new geometries, such as CaloChallenge Dataset 3, through fine-tuning. This approach contrasts with the conventional "from scratch" paradigm, where models are initialised with random weights and must learn all physics representations directly from the target dataset. The dashed box with a question mark represents potential future applications to additional detector configurations.

We evaluate cross-geometry adaptation through two primary training strategies:

from scratch, in which models are initialised with random weights, representing the conventional training paradigm where each detector geometry requires complete model training. This serves as our baseline for quantifying the benefits of transfer learning.

fine-tuning, in which models are initialised from weights pretrained on ILD photon showers, then all parameters are updated during adaptation to the CaloChallenge electron shower task. This tests whether learned representations generalise across different detector conditions.

The transfer presents multiple simultaneous challenges. First, the detector geometry changes from planar (ILD) with rectangular cells to cylindrical (CaloChallenge) with radial-azimuthal segmentation. The transfer challenge persists at η=0\eta= 0 where both detectors have flat layers, since ILD uses rectangular cells while CaloChallenge employs curved arc-shaped voxels in (r,φ)(r,\varphi), fundamentally altering how generated point clouds project onto the readout structure. Second, the readout granularity differs: 30 layers versus 45 layers. Third, the incident energy range and distributions, where the target dataset (downstream) extends beyond the pre-training uniformly distributed in 10–90 GeV to test the extrapolation capabilities of log-uniform distributed at both low (1–10 GeV) and high (90–1000 GeV) energies, as shown in Figure 2. Fourth, the particle type changes from photons to electrons, though both produce electromagnetic cascades governed by similar quantum electrodynamics processes. These compound shifts test whether shower physics learned in one context can transfer to substantially different conditions.

Figure 2. Incident energy distributions for pre-training (ILD, red, uniform 10-90 GeV) and downstream (CaloChallenge, blue, log-uniform 1-1000 GeV) datasets. Left: Full range, with a dashed box indicating the overlap region. Right: Magnified overlap showing distributional differences that, combined with particle type and geometry shifts, constitute the compound domain shift addressed in this work.

The model autonomously adapts, guided solely by the objective function [77], from the compact ILD geometry to new geometric configurations such as the larger cylindrical configuration of CaloChallenge dataset 3, preserving fundamental particle shower physics while adjusting to changes in spatial scale and detector granularity. For this study, electromagnetic shower physics is a good testing ground for geometry and scale adaptation since it is essentially particle-agnostic beyond the first interaction stage.

PEFT strategies are investigated to enhance sustainability by updating only parameter subsets. All training strategies are evaluated across varying downstream dataset sizes (10210^2 to 10510^5 samples) to assess data efficiency when expensive Geant4 simulation limits available training data.

Datasets

This study employs two distinct electromagnetic shower datasets generated through Geant4 simulations. The first dataset comprises photon showers simulated in the ILD detector, a realistic detector design developed for potential construction at the International Linear Collider for model pre-training, while the second contains electron showers in a cylindrical calorimeter geometry for transfer learning evaluation for downstream. Figure 3 shows visually the datasets considered in this study.

Figure 3. Representative electromagnetic shower event displays illustrating the domain shift. Left: 81 GeV photon shower in the planar ILD detector. Right: 913 GeV electron shower in the cylindrical CaloChallenge detector. The cylindrical layer structure is visible in the curved distribution of energy deposits along the longitudinal axis. Data representation from Ref. [47].

Pre-training dataset

This section describes the pre-training dataset used before task-specific fine-tuning. The approach employs the electromagnetic calorimeter (ECAL) datasets from Ref. [69], utilising these pre-trained representations as the starting point. The pre-training dataset consists of 524k1 photon showers with incident energy uniformly distributed between 10 and 90 GeV, simulated in the ILD [93].

The ILD ECAL features 30 layers alternating between tungsten absorbers (2.1 mm thick for the first 20 layers, 4.2 mm for the last 10) and silicon sensors (0.5 mm thick with 5 mm ×\times 5 mm readout cells). Data representation employs two coordinate systems: a local system [X,Y,Z][X, Y, Z] centred at the photon’s impact position, and a global ILD system [X′,Y′,Z′][X^{\prime}, Y^{\prime}, Z^{\prime}], with photons originating at [X′=0,Y′=1811.3 mm,Z′=4 mm][X^{\prime}=0, Y^{\prime}=1811.3\ \mathrm{mm}, Z^{\prime}=4\ \mathrm{mm}] travelling along Y′Y^{\prime}. The energy depositions from Geant4 (so called steps) are pre-clustered by layer and projected onto a grid with 36 times higher resolution than the physical calorimeter (0.83 mm ×\times 0.83 mm cells), reducing approximately 20,000 points per shower by a factor of roughly 7. Cluster positions are normalised to [−1,1][-1, 1] within a bounding box from −200-200 mm to 200 mm in XX and YY.

Downstream dataset

For task-specific fine-tuning, this study employs Dataset 3 [94] from the Fast Calorimeter Simulation Challenge (CaloChallenge) [65], designed to facilitate deep generative model development for calorimeter simulation [95]. Dataset 3 contains electron showers with log-uniform incident energies from 1 GeV to 1 TeV, simulated using the geometry from the Par04 example of Geant4 [96].

This geometry represents an idealised cylindrical calorimeter consisting of 90 concentric cylinders alternating between absorber material (1.4 mm of tungsten (W)) and active material (0.3 mm of silicon (Si)), contrasting with the planar ILD geometry. The calorimeter has an inner radius of 800 mm and a depth of 153 mm, with perpendicular showers positioned in the central η=0\eta= 0 section. In the frame of reference considered in this study, each voxel along the yy-axis corresponds to two physical layers (W-Si-W-Si) with a length of Δz=3.4\Delta z = 3.4 mm (equivalent to 0.8X00.8X_0 of the absorber), resulting in 45 readout layers compared to 30 in the pre-training dataset. Showers are segmented into 18 radial and 50 azimuthal bins, yielding 900 voxels per layer and 40,500 voxels per shower. This segmentation, combined with the broader energy range, produces point clouds that can exceed three times the size of pre-training data at the highest energies.

To enable effective transfer learning, the CaloChallenge dataset undergoes preprocessing to align with the pre-training format (detailed in Appendix § 8). Key steps include cylindrical smearing to convert voxelized deposits into continuous point clouds, sampling-fraction reversal to recover raw energy depositions, and point-based ordering for batch assembly efficiency.

The combination of geometric transformation from planar to cylindrical layout, energy distribution change from uniform (10–90 GeV) to log-uniform (1–1000 GeV), and differences in detector granularity creates a challenging transfer learning scenario that tests whether representations learned from ILD photon showers generalize to fundamentally different downstream conditions. The dataset is split into 100,000 samples for training with 10,000 samples reserved for validation and testing.

Experiments

To assess the transfer learning capabilities for cross-geometry shower generation, this study examines how pre-trained representations influence downstream performance across different detector configurations. The experimental design isolates the contribution of learned physics knowledge by evaluating different training strategies and fine-tuning approaches across varying training dataset sizes. Random training examples are sampled from the full training set of CaloChallenge.

The training methodology is adapted according to computational requirements and model complexity. For the PointWise point cloud diffusion model generator [70], which represents the most computationally expensive component, all training strategies are evaluated to assess the trade-off between adaptation effectiveness and computational cost. Detailed training hyperparameter specifications are provided in Appendix § 9.

For the ShowerFlow model, which determines the total number of points for point cloud post-diffusion calibration, only full fine-tuning is employed due to its relatively modest computational requirements during training. This approach leverages the complete learned representations while maintaining training efficiency for this less computationally complex architectural component.

Evaluation Metrics

Generative models for calorimeter simulation must accurately reproduce statistical distributions of the training data. This evaluation employs hit-level and shower-level observables to assess model fidelity, comparing distributions between ground truth and generated samples using physically meaningful metrics, shown in Table 1.

ObservableDescription
Voxel Energy SpectrumDistribution of energy depositions across all voxels
Energy ratioTotal measured energy summed over all voxels divided by incident energy
Visible EnergyTotal energy deposition per shower
OccupancyFraction of active voxels in a shower
Longitudinal ProfileEnergy-weighted distribution along calorimeter layers
Radial ProfileEnergy-weighted distribution of distances from the incident point

Table 1. Observables for evaluating generated calorimeter shower fidelity.

Two complementary statistical metrics quantify agreement between generated samples and Geant4 reference data:

Kullback-Leibler divergence provides a robust distributional comparison across the entire observable range:

KL⁡(P∥Q)=∑iPilog⁡(PiQi)(1)\operatorname{KL}(P\|Q) = \sum_i P_i \log\left(\frac{P_i}{Q_i}\right) \tag*{(1)}

where PiP_i and QiQ_i represent the probabilities of reference and generated samples in the ii-th bin. Bins are defined by reference distribution quantiles rather than fixed widths, ensuring uniform sensitivity across the observable range, and preventing dominance by high-density regions while capturing tail behaviour. The KL divergence is computed using scipy.stats.entropy [97].

Wasserstein-1 distance offers a symmetric measure of distributional similarity based on optimal transport theory:

W1(P,Q)=min⁡π∈Π(P,Q)∑i,j∣xi−xj∣π(xi,xj)(2)W_1(P,Q) = \min_{\pi\in\Pi(P,Q)} \sum_{i,j} |x_i-x_j|\pi(x_i,x_j) \tag*{(2)}

Here, Π(P,Q)\Pi(P,Q) denotes the joint coupling distributions with marginals PP and QQ. This metric quantifies the minimal cost of transforming one distribution into another, providing geometrically interpretable measures to small shifts from calorimeter resolution effects. The Wasserstein-1 distance is computed using scipy.stats.wasserstein_distance [97].

These complementary metrics offer a comprehensive assessment of generation quality: quantile KL divergence provides uniform sensitivity across the full observable range, while Wasserstein distance measures overall distributional similarity.

ShowerFlow Transfer Learning & Post-Diffusion Calibration

ShowerFlow predicts the point counts per layer Nz,iN_{z,i}, i.e. the number of energy deposits in layer ii, that condition PointWise Net’s point cloud generation. This model is trained exclusively for occupancy prediction and subsequent occupancy-based calibration, rather than energy per layer calibration as in Ref. [70]. To correct systematic biases in generated occupancy, we apply an energy-dependent calibration to the predicted point counts2 that matches the relationship between total point count and occupancy fraction (active voxels) in generated versus reference showers. We fit cubic polynomials pdata(O)p_{\mathrm{data}}(O) and pgen(O)p_{\mathrm{gen}}(O) relating occupancy to point counts for reference and generated data respectively, then apply the transformation Ncal=pgen−1(pdata(Ngen))N_{\mathrm{cal}}=p_{\mathrm{gen}}^{-1}(p_{\mathrm{data}}(N_{\mathrm{gen}})) to map generated counts through the reference occupancy relationship. Unlike the original manual approach, this automatically adapts to new datasets, with the calibrated counts NcalN_{\mathrm{cal}} and counts per layer Nz,i,calN_{z,i,\mathrm{cal}} subsequently conditioning the diffusion sampling.

The pretrained ILD model has 30 layers while CaloChallenge has 45 layers, creating a dimensional mismatch for the normalising flow architecture that cannot dynamically expand. To bridge this gap, we model the additional 15 layers using log-normal distributions with parameters (μ,σ)(\mu, \sigma) estimated from 100 randomly sampled CaloChallenge showers, corresponding to the smallest dataset size we evaluate. During fine-tuning, the model predicts counts for the original 30 layers using pretrained weights, while the extra 15 layers are initialised from these log-normal distributions and then learned.

Formally, the total predicted count is Ngen=∑i=130Nz,iILD+∑i=3145Nz,iadaptedN_{\mathrm{gen}} = \sum_{i=1}^{30} N_{z,i}^{\mathrm{ILD}} + \sum_{i=31}^{45} N_{z,i}^{\mathrm{adapted}}, where the first term uses ILD pretrained backbone representations and the second term adapts to the new geometry.

Figure 4 shows that fine-tuning consistently outperforms training from scratch across all dataset sizes (see Appendix § 10 for detailed per layer histograms and convergence analysis, as well as the KL metric evaluation). The benefit is clear in low-data regimes (<103< 10^3 samples) where pretrained representations provide essential inductive bias, reducing overfitting despite the architectural workaround for layer mismatch.

Figure 4. ShowerFlow transfer performance measured by normalised Wasserstein distance between generated and reference point-count distributions, averaged across all 45 calorimeter layers. Each point represents the median performance across five independent training runs with different random seeds. Error bands show the standard deviation across seeds. Evaluation is performed on the full 10,000-sample validation set. Fine-Tuning from ILD-pretrained weights substantially outperforms training From Scratch in low-data regimes.

This layer extension strategy generalises to arbitrary detector configurations with different numbers of calorimeter layers. For a target detector with LtargetL_{\mathrm{target}} layers and a pre-trained model with LpreL_{\mathrm{pre}} layers, the approach requires modifications only in the ShowerFlow component: when Ltarget>LpreL_{\mathrm{target}} > L_{\mathrm{pre}}, additional layers are initialised from parametric distributions estimated from a small target domain sample, as demonstrated here. When Ltarget<LpreL_{\mathrm{target}} < L_{\mathrm{pre}}, the excess output dimensions can be discarded or merged. The PointWise Net diffusion model requires no architectural changes, as it operates on individual points conditioned on layer indices rather than fixed-dimensional outputs, making it inherently flexible to varying layer counts.

Cross-Calorimeter Performance

All results in this section employ the complete generation pipeline: ShowerFlow predicts point counts Nz,iN_{z,i} per layer, which then condition PointWise Net’s diffusion-based point cloud generation. For the comparison between from scratch and full fine-tuned models (subsection 4.3.1), both ShowerFlow and PointWise Net are trained with the same strategy. For parameter-efficient methods (subsection 4.3.2), ShowerFlow is always fully fine-tuned due to its modest computational cost, while PointWise Net employs various PEFT techniques.

Adapting a pre-trained model to a new detector geometry requires careful consideration of inference time constraints. Since the target application requires fast and scalable point cloud generation, we focus exclusively on fine-tuning techniques that preserve the original inference speeds; methods such as adapters [81] are then excluded. Only methods that retain the original inference graph are considered: partial fine-tuning, BitFit, and Low-Rank Adaptation (LoRA). This constraint ensures practical deployment in latency-critical applications while demonstrating that LoRA and BitFit extend effectively beyond language models to point cloud diffusion tasks.

All performance metrics represent Wasserstein distances computed for six physics observables (see § 4.1). We aggregate these using the geometric mean to ensure balanced evaluation across observables with different scales:

yˉjk=(∏i=16yijk)1/6,(3)\bar{y}_{jk} = \left(\prod_{i=1}^{6} y_{ijk}\right)^{1/6}, \tag*{(3)}

where each training method jj across the six physical observables ii is calculated for different training shower sizes kk. This prevents any single metric from dominating the evaluation while maintaining sensitivity to performance variations. While this aggregation provides useful guidance and quantitative benchmarks, we emphasise examining individual observables directly, as aggregated metrics can obscure important physics specific performance patterns. The geometric mean serves primarily to guide overall assessment, while detailed observable analysis reveals the true model behaviour. Further consideration on Equation (3) and its error propagation is detailed in Appendix F.

From scratch vs Full fine-tuning

We now compare the two training strategies introduced in § 2.2. In this comparison, both ShowerFlow and PointWise Net are either trained from scratch with random initialisation, or full fine-tuned from ILD pretrained weights. We use the term full fine-tuning to distinguish this from the parameter-efficient methods examined in § 4.3.2.

Figure 5 shows distinct performance patterns across training dataset sizes. The voxel energy spectrum reveals minimal differences between training strategies and dataset sizes, with both apapproaches yielding similar distributions, regardless of whether pre-training is used. In addition, both approaches generate excessively high-energy voxel deposits (>100 MeV>100\ \mathrm{MeV}) compared to Geant4, likely due to the point cloud projection occasionally concentrating multiple hits into a single voxel. Despite this limitation, the observable appears to be learned effectively even without transfer learning, suggesting that the point cloud representation naturally captures the energy deposition patterns independent of the source detector. This contrasts with geometric observables, like longitudinal and radial profiles, where pre-training provides clear advantages. Occupancy is underestimated at high values due to information loss during the projection from point clouds to regular cell geometry. The Full fine-tuned training shows superior performance in longitudinal and radial profiles, particularly at low data regimes, demonstrating better adaptation of shower structure to the new geometry. Additionally, the improved visible energy performance in the fine-tuned model indicates more stable energy ratio modelling, especially crucial when training data is limited.

Geant4 vs generated showers at training sizes $D$

Figure 5. (a) Distributions: cell energy spectrum (left), total deposited energy over incident energy (centre), visible energy (right). (b) Distributions: occupancy (left), longitudinal (centre), radial profile (right). Figure 5: Geant4 vs generated showers at training sizes DD. Top rows: FROM SCRATCH; bottom rows: FULL FINE-TUNED. All histograms from 10, 000 events with energy logarithmically distributed from 1 − 1000 GeV. Bottom panels show GEANT4 ratios. The error band corresponds to the statistical uncertainty in each bin.

Figure 6 quantifies the transfer learning advantage. With only 10210^{2} training samples, full fine-tuned model achieves a geometric mean Wasserstein distance of 0.064 ± 0.006 compared to 0.132 ± 0.018 for from scratch training. Despite the large variance in the baseline, the ∼ 51% reduction in mean WD demonstrates statistically significant transfer learning benefits in data constrained scenarios. This benefit diminishes with the increase in training data.

Figure 6. Wasserstein evaluation metrics for shower generation for the six physical observables across different dataset sizes. The resulting bands represent averages over five independent seeds with RMS uncertainty bands. A residual at 10410^4 samples remains visible in some individual observables.

Individual observables show differential sensitivity to transfer learning. Longitudinal and radial profiles benefit most, as geometric features learned from ILD transfer effectively despite detector differences. The voxel energy spectrum shows minimal improvement, likely because point clouds inherently provide dense sampling for this observable regardless of training set size.

At 10410^{4} training samples, a residual increase in Wasserstein distance remains visible in some individual observables, most notably the longitudinal profile and visible energy. After retraining with non-overlapping data partitions to ensure fully independent runs, this effect persists, indicating a genuine model behaviour rather than a statistical artefact. This effect is confined to individual metrics and does not impact the aggregate geometric mean. The overall trend continues to demonstrate a clear transfer learning advantage in the low-data regime (<103<10^{3} samples).

Parameter-Efficient Fine-Tuning Strategies

Beyond full fine-tuning, we evaluate adaptation methods that update only a subset of parameters in PointWise Net while preserving the original inference architecture. These techniques may offer crucial advantages for multi-detector deployment and computational efficiency. As pretrained models scale and become more general-purpose, the computational cost of retraining all parameters for each detector configuration becomes increasingly impractical, particularly when considering deployment across multiple experimental setups. The study presented in this section is the first application of PEFT methods to a pretrained model in the context of fast particle shower simulations.

BitFit [98] represents the most parameter-efficient approach, training only bias terms while freezing all weights. This method modifies 17% of the model parameters by recalibrating activation thresholds throughout the network, thereby adjusting response patterns for the target detector geometry. Top2 fine-tuning freezes the feature extraction layers and updates only the final two layers3 as well as the time-step layer. This approach tests whether the earlier layers contain reusable representations that enable accurate generation with minimal adaptation. This configuration was selected through systematic ablation studies that examined various combinations of layers, revealing that updating the top two and the time-embedding layers provides the optimal balance between expressivity and efficiency.

LoRA [99] introduces low-rank decomposition matrices that adapt pretrained representations through additive updates. We employ rank 10610^{6}, selected based on the comprehensive analysis in Appendix § 12, where we demonstrate that CaloChallenge requires higher ranks than typical NLP applications due to the high-dimensional nature of particle shower transformations.

Figure 7. Parameter-efficient fine-tuning performance across training data volumes. Wasserstein distances evaluated on generated showers. Uncertainties represent standard error across three independent training runs with different random seeds.

Table 2 presents quantitative comparisons across methods and dataset sizes. BitFit achieves performance within 5% of full fine-tuning on the geometric mean Wasserstein distance while updating only 17% of trainable parameters. Top2 fine-tuning with 44% of parameters shows comparable results, suggesting that adaptation primarily occurs in higher layers while lower layers remain largely transferable. LoRA exhibits degraded performance despite utilising 52% of parameters, with consistently poor results at intermediate data scales and particular degradation in the voxel energy spectrum, though showing comparable performance for energy ratio and occupancy observables.

MethodParams (%)Training Dataset SizeMean
From scratch100%10210^{2}10310^{3}10410^{4}10510^{5}
From scratch100%13.2±1.813.2 \pm1.87.1±0.47.1 \pm0.46.4±0.66.4 \pm0.64.5±0.34.5 \pm0.37.8±0.57.8 \pm0.5
Full fine-tuned100%6.4±0.66.4 \pm0.66.2±0.26.2 \pm0.26.1±0.46.1 \pm0.44.7±0.34.7 \pm0.35.9±0.25.9 \pm0.2
BitFit17%6.9±0.46.9 \pm0.46.1±0.26.1 \pm0.26.2±0.56.2 \pm0.55.4±0.35.4 \pm0.36.2±0.26.2 \pm0.2
Top244%7.6±0.67.6 \pm0.66.2±0.26.2 \pm0.26.5±0.46.5 \pm0.47.1±0.87.1 \pm0.86.8±0.36.8 \pm0.3
LoRA R10652%8.4±0.98.4 \pm0.99.4±0.69.4 \pm0.67.8±0.47.8 \pm0.48.0±0.68.0 \pm0.68.4±0.38.4 \pm0.3

Table 2. Performance comparison across training strategies. Geometric mean Wasserstein distance values ×10−2\times10^{-2} for readability. Uncertainties show standard error over five seeds. At D=104D = 10^{4}, training sets are constructed from non-overlapping data partitions to ensure independence across runs.

The results reveal several important patterns. At small data scales (10210^{2}), pre-training provides clear benefits across all methods, with transfer learning reducing Wasserstein distance by 51% compared to training from scratch. The intermediate data regime (10310^{3}–10410^{4}) shows more complex behaviour, with minor variations in relative performance that may reflect sampling effects and the interplay between pre-training bias and target domain adaptation. At the largest scale (10510^{5}), the performance gap narrows as sufficient data allows even from scratch training to converge effectively.

LoRA’s consistent underperformance warrants specific discussion. Unlike its success in NLP tasks, LoRA struggles with calorimeter simulation even at rank 106. Our analysis in Appendix § 12.§ 2 reveals that weight updates in shower physics exhibit high intrinsic dimensionality across network layers, with some requiring ranks exceeding 200 for accurate reconstruction. This fundamental mismatch between LoRA’s low-rank assumption and the complexity of physics transformations explains its limited effectiveness.

The success of BitFit and Top2 methods suggests that effective adaptation for CaloChallenge operates through two mechanisms: recalibrating activation patterns via bias adjustments and refining high-level feature combinations in final layers. Both approaches preserve the learned representations while allowing targeted modifications for detector-specific characteristics.

Table 3 quantifies the computational cost of each training strategy. BitFit reduces per-step training time by 21% and peak memory by 23% compared to full fine-tuning, while Top2 achieves intermediate reductions of 12% and 10% respectively. LoRA incurs a 40% increase in training time despite fewer trainable parameters, due to the additional low-rank matrix operations. Full fine-tuning converges in approximately 100k steps compared to 500k for training from scratch, making pre-training beneficial both in final performance and in total training cost. Combining per-step cost with convergence, BitFit offers the most efficient total training time (~200k ×\times 49 ms ≈\approx 2.7 h) compared to full fine-tuning (~100k ×\times 62 ms ≈\approx 1.7 h) and from scratch (~500k ×\times 62 ms ≈\approx 8.6 h), while using 23% less peak memory. Since GPU power draw is approximately constant across all configurations, per-step energy consumption scales proportionally with training time. These findings have relevant implications for deploying generative models across diverse detector configurations. The reduced memory requirements of parameter-efficient methods may enable multi-geometry adaptation without proportional increases in storage. By freezing most parameters, these techniques accelerate convergence and mitigate catastrophic forgetting [100], essential properties for continual learning across evolving detector designs. As calorimeter models scale up and become more general, our results indicate that successful adaptation strategies might respect the high-dimensional nature of physics data, favouring threshold recalibration and selective layer updates over aggressive low-rank compression. These empirical findings challenge the universal applicability of low-rank adaptation methods and motivate the development of physics-aware parameter-efficient techniques.

MethodParams (%)ms/stepPeak VRAM (GB)Conv. steps (×103\times10^{3})
From scratch100%62.3±0.362.3 \pm0.316.4∼500\sim500
Full fine-tuning100%62.3±0.362.3 \pm0.316.4∼100\sim100
BitFit17%49.0 ± 0.112.6∼200\sim200
Top244%54.5±0.454.5 \pm0.414.8∼250\sim250
LoRA R10652%†52\%^{\dagger}86.9±0.486.9 \pm0.413.7∼50\sim50

Table 3. Computational cost for each training method, measured on a single NVIDIA A100 80GB PCIe GPU with batch size 64. Per-step timings averaged over 3 epochs after 1 warmup epoch. Average GPU power draw is approximately constant across configurations (∼278\sim278 W).

† Relative to original model parameters (504,920). LoRA introduces additional parameters, bringing the total to 777,128.

‡ Approximate convergence steps, defined as the first step within 5% of each method’s best geometric mean Wasserstein distance. Values vary across dataset sizes, metrics, and seeds; reported here as indicative estimates.

Conclusions and Outlook

This study explores single-detector pre-training on point cloud representations as a path for generalisable cross-geometry transfer learning in calorimeter simulation. Our findings demonstrate that meaningful transfer learning is achievable even from single-geometry pre-training.

The main findings are that, in low-data regimes (10210^{2} samples), pre-training on ILD photon showers enables adaptation to the CaloChallenge electron shower task, yielding a statistically significant 51% performance improvement over training from scratch. Among parameter-efficient methods, BitFit achieves performance within 5% of full fine-tuning using only 17% of parameters, while LoRA shows limited effectiveness even at rank 106. Our post-hoc singular value analysis provides theoretical insight into why LoRA struggles, suggesting that particle shower transformations may have higher intrinsic dimensionality than typical NLP tasks.

Several limitations constrain our conclusions. A residual increase in Wasserstein distance at D=104D = 10^{4} persists in some individual observables across independent runs, suggesting a genuine model behaviour at this intermediate data scale rather than a statistical artefact. This effect does not impact the aggregate geometric mean. Computational benchmarking demonstrates that parameter efficiency translates into practical cost reductions: BitFit reduces per-step training time by 21% and peak memory by 23%, while full fine-tuning converges in approximately 100k steps compared to 500k for training from scratch. Most importantly, without direct comparison to multi-detector pre-training approaches, we cannot claim relative performance against existing foundation model approaches.

Despite these limitations, this work contributes to understanding transfer learning in calorimeter simulations. The success of BitFit and selective layer fine-tuning suggests that adaptation primarily involves targeted recalibration rather than fundamental representation changes. In scenarios where multi-detector datasets are unavailable or computational resources are limited, single-detector pre-training offers an adequate starting point for rapid prototyping.

Future work should pursue several directions. First, developing more generalizable pre-training strategies that leverage point cloud representations across different calorimeter geometries and energy ranges would strengthen the foundation model approach. Second, systematic comparisons with multi-detector pre-training approaches would establish relative performance benchmarks. Finally, extending this framework to hadronic showers and mixed particle types would test the generalizability of transfer learning in more complex scenarios.

Code Availability

The code for this study can be found under https://github.com/FLC-QU-hep/CaloTransfer.

Acknowledgments

We thank Katja Krüger for valuable comments on the manuscript. This research was supported in part by the Maxwell computational resources operated at Deutsches Elektronen-Synchrotron DESY, Hamburg, Germany. This project has received funding from the European Union’s Horizon 2020 Research and Innovation programme under Grant Agreement No 101004761. We acknowledge support by the Deutsche Forschungsgemeinschaft under Germany’s Excellence Strategy – EXC 2121 Quantum Universe – 390833306 and via the KISS consortium (05D23GU4, 13D22CH5) funded by the German Federal Ministry of Research, Technology, and Space (BMFTR) in the ErUM-Data action plan.

Pre-processing

To enhance transfer learning from the pre-trained model, the CaloChallenge dataset is aligned with the pre-training format via three preprocessing steps:

Cylindrical smearing: Voxelized energy depositions are converted into point clouds. Each energy deposit, originally localised at the voxel centre, is spatially redistributed by sampling uniformly within the cylindrical boundaries of its host voxel. This process applies Gaussian noise to the radial (rr) and azimuthal (ϕ\phi) coordinates while preserving the longitudinal (zz) position, ensuring energy conservation within individual detector cells. The smearing maintains the detector’s cylindrical geometry while generating continuous spatial distributions that facilitate the training of diffusion models. Figure 8 illustrates this transformation for a representative shower with incident energy of 500.3 GeV, showing the transition from discrete voxelized deposits to spatially smeared point clouds in the transverse plane of the calorimeter.

Figure 8. Cylindrical smearing transformation applied to electromagnetic shower data from CaloChallenge. The left panel shows the original voxelized energy depositions concentrated at voxel centres. The right panel displays the result after cylindrical smearing, where energy deposits are spatially redistributed within their respective voxel boundaries using Gaussian noise in cylindrical coordinates. The colour scale represents energy deposition values, and the concentric circles indicate the detector’s cylindrical segmentation.

Sampling-fraction reversal: The normalisation applied to account for sampling fractions is inverted to recover raw energy depositions in the active material, matching the silicon-layer energy scale used during pre-training.

Point-based ordering: Showers are sorted by point count to assemble mini-batches of similar complexity, replicating the efficiency gains observed in the original pre-training data version.

These procedures maintain the physical integrity of CaloChallenge showers while standardizing geometry, energy scale, and batch complexity, thereby facilitating effective cross-architecture knowledge transfer.

Hyperparameters used in experiments

Table 4 shows the baseline configuration we used across all experiments, while Table 5 details how we adapted learning rates for different training methods and dataset sizes. We report the median performance over 5 random seeds, with results taken from the best-performing epoch for each run. To maintain training stability while optimizing memory usage, we also implemented adaptive batch sizing following the approach of Keskar et al. [101].

CategoryConfiguration
Training SetupBatch Size: 64; Optimizer: RAdam; LR Schedule: Linear (100K warmup →\to 300K decay); Maximum Gradient Steps: 1.1M; Weight Decay: 0.01; Device: NVIDIA® A100
EDM ConfigurationKL Weight (β\beta): 10−310^{-3}; KLD Min: 1.0; Noise Schedule: Quadratic; EMA: Inverse (power=0.6667, max=0.9999)
Samplingσdata\sigma_{\mathrm{data}}: 0.5; σ\sigma Distribution: LogNormal(μ=−1.2\mu= -1.2, σ=1.2\sigma= 1.2); ODE Solver: Heun; Sampling Steps: 32; σmin⁡\sigma_{\min} / σmax⁡\sigma_{\max}: 0.002 / 80.0; ρ\rho / schurns_{\mathrm{churn}} / snoises_{\mathrm{noise}}: 7.0 / 0.0 / 1.0

Table 4. PointWise Net settings and sampling parameters used across all training methods.

MethodParameter10210^{2}10310^{3}10410^{4}10510^{5}
From scratchLR Start / End2e-4 / 1e-4
From scratch# Gradient Steps250,0001,000,000500,000750,000
Full Fine-tunedLR Start / End5e-4/5e-51e-4/1e-52.5e-5/2.5e-65e-6/5e-7
Full Fine-tuned# Gradient Steps100,00050,000100,000250,000
Top2 Fine-tunedLR Start / End5e-4/5e-51e-4/1e-52.5e-5/2.5e-65e-6/5e-7
Top2 Fine-tuned# Gradient Steps1,000,000500,000750,000750,000
BitFitLR Start / End2e-3/2e-44e-4/4e-51e-4/1e-52e-5/2e-6
BitFit# Gradient Steps1,000,000750,000500,000500,000
LoRA R8LR Start / End1e-3/1e-42e-4/2e-55e-5/5e-61e-5/1e-6
LoRA R8# Gradient Steps250,00010,000100,000200,000
LoRA R8LoRA α\alpha / rr8 / 8
LoRA R106LR Start / End1e-3/1e-42e-4/2e-55e-5/5e-61e-5/1e-6
LoRA R106# Gradient Steps100,000100,00010,00050,000
LoRA R106LoRA α\alpha / rr106 / 106

Table 5. PointWise Net Learning rate schedules and method-specific parameters adapted to different dataset sizes.

Table 6 presents the configuration for the ShowerFlow model, which uses a different architecture and thus required its own optimization strategy. The batch sizes were scaled with dataset size to balance training efficiency and stability.

CategoryHyperparameterShowerFlow
DataPin MemoryTrue
DataWorkers4
DataShuffleTrue
ArchitectureNum Blocks2
ArchitectureNum Inputs45
ArchitectureConditioning Inputs1 (Energy)
ArchitectureCoupling Hidden Dims[920, 920]
ArchitectureSpline Hidden Dims[368, 368]
ArchitectureSpline Bins8
TrainingDeviceNVIDIA® V100
TrainingOptimizerAdam
TrainingSchedulerNone
TrainingLearning Rate1×10−41 \times10^{-4}
TrainingBatch Size†[64, 2048]
TrainingMaximum Epochs1000
TrainingGradient Clipping*104→5×10510^{4} \to5 \times10^{5}

Table 6. ShowerFlow model architecture and training configuration with dataset-dependent batch sizing.

ShowerFlow Transfer

Figure 9 and Figure 10 show the distribution of points per layer generated by ShowerFlow. The shower development peaks between layers 10 and 25, where the electromagnetic cascade is most active, and the highest number of voxels are triggered. Accurate modeling of these distributions is crucial since the points per layer serve as conditioning input for generating the full EM showers and calibrating the shower structure. The fine-tuned model clearly outperforms the from scratch version in low data regimes, demonstrating successful knowledge transfer from the pre-training phase. This advantage becomes less pronounced as training data increases, since sufficient data allows the model to learn the distributions directly.

Histograms of points per layer for Geant4 reference and ShowerFlow trained from scratch, shown for layers 1, 6, 12, 18, 24, 30, 36, and 42

Figure 9. Histograms of points per layer for CaloChallenge: Geant4 reference (gray) versus ShowerFlow trained from scratch with varying dataset sizes. All distributions computed from 10, 000 showers with logarithmic energy sampling between 1 and 1000 GeV.

Histograms of points per layer for Geant4 reference and ShowerFlow finetuned from ILD pre-training, shown for layers 1, 6, 12, 18, 24, 30, 36, and 42

Figure 10. Histograms of points per layer for CaloChallenge: Geant4 reference (gray) versus ShowerFlow finetuned from ILD pre-training with varying dataset sizes. All distributions computed from 10, 000 showers with logarithmic energy sampling between 1 and 1000 GeV.

To select the optimal epoch for each configuration, we tracked the Wasserstein distance and KL divergence across training, as shown in Figure 11 and Figure 12. Given the training instability and overfitting risk in low data regimes, we selected epochs based on the minimum averaged metric across validation samples rather than single point validation loss minima.

Figure 11. ShowerFlow convergence curves using Wasserstein distance. Each configuration averaged over 5 random seeds. Epoch 0 represents pretrained weights (finetuned version only).

Figure 12. ShowerFlow convergence curves using KL divergence. Each configuration averaged over 5 random seeds. Epoch 0 represents pretrained weights (finetuned version only).

Figure 13 presents the final performance when epochs are selected using KL divergence, while Figure 4 shows selection based on WD. Both metrics reveal that transfer learning provides substantial benefits primarily in low data regimes (<5×103< 5 \times10^{3} samples), with comparable or slightly reduced performance at larger dataset sizes, where the model has sufficient data to learn from scratch.

Figure 13. ShowerFlow transfer learning performance measured by KL divergence averaged across all calorimeter layers. Fine-tuning significantly outperforms training from scratch in low data regimes. Results averaged over five random seeds.

Further plots

This section of the appendix presents comprehensive evaluation metrics that complement the main results. Section D.1 provides detailed histogram comparisons for all parameter-efficient fine-tuning methods, while Section D.2 presents Kullback-Leibler divergence analysis as an alternative metric to validate the Wasserstein distance findings.

PEFT histograms

Figure 14 and Figure 15 present detailed distribution comparisons between Geant4 reference data and generated showers for all PEFT methods at various training dataset sizes. These histograms reveal method-specific strengths and weaknesses: BitFit maintains stable energy spectrum reconstruction across all scales, Top2 fine-tuning shows particularly good longitudinal profile modeling, while LoRA variants exhibit systematic biases in occupancy and radial distributions that persist even with increased training data.

Geant4 vs generated showers for parameter-efficient methods at training sizes $D$

Figure 14. Geant4 vs generated showers for parameter-efficient methods at training sizes DD. A comprehensive distribution analysis of generated showers for Top2 fine-tuned (first two rows), and BitFit (last two rows). All histograms from 10,000 events with energies logarithmically distributed from 1 to 1000 GeV. Bottom panels show Geant4 ratios with statistical uncertainties. The error band represents the statistical uncertainty in each bin.

Geant4 versus generated showers for LoRA R106 across voxel energy spectrum, energy ratio, visible energy, occupancy, mean energy by layer, and mean energy by radius

Figure 15. Geant4 vs generated showers for parameter-efficient methods at training sizes DD. A comprehensive distribution analysis of generated showers for LoRA R8 (first two rows), and LoRA R106 (last two rows). All histograms from 10, 000 events with energies logarithmically distributed from 1 to 1000 GeV. Bottom panels show Geant4 ratios with statistical uncertainties. The error band represents the statistical uncertainty in each bin.

KL evaluation

To verify that our conclusions are robust to metric choice, we repeat all evaluations using the Kullback-Leibler divergence. Figure 16 presents these results across all training strategies and observables.

Figure 16. Kullback-Leibler divergence evaluation across training strategies. Top: From-scratch versus full fine-tuning comparison. Bottom: Complete PEFT comparison including BitFit (green), Top2 (purple), and LoRA R106 (brown). The energy ratio anomaly at 10410^{4} samples is visible in both panels. Error bands represent the standard error across five random seeds.

The KL metric confirms our main findings while revealing additional insights. Transfer learning provides consistent benefits in low-data regimes, with KL divergence reducing by 35-50% compared to training from scratch. The energy ratio anomaly at 10410^{4} samples appears in both evaluation panels, confirming this is specific to the fine-tuning pathway rather than a metric artifact.

Among PEFT methods, BitFit remains most effective, closely tracking full fine-tuning performance across most observables. The exception is the voxel energy spectrum, where all PEFT methods show degradation, a pattern amplified by KL’s sensitivity to distribution tails. The logarithmic scale variations across observables reflect their different intrinsic complexities, with KL showing larger relative differences between methods than the Wasserstein distance.

Additional Experiments on Low-Rank Matrices

We present additional results from our investigation into the low-rank update matrices. The results presented in this appendix show significant variability across different rank choices and dataset sizes, without clear monotonic trends. This instability likely reflects the fundamental mismatch between LoRA’s low-rank assumption and the high-dimensional nature of shower physics events. We include these results for completeness and to inform future investigations, while acknowledging that no clear optimal rank emerges from this analysis.

Effect of the rank rr on the downstream task

Using the CaloChallenge as an example, we report the WD and KL metrics achieved by different choices of the rank rr after training the best number of steps.

We present our results in Table 7 and Table 8. The optimal rank for CaloClouds is between 48 and 106, depending on the metric used. Note that the relationship between model size and the optimal rank for adaptation is still an open question. The lack of clear trends supports our main finding that LoRA is poorly suited for this application. The optimal rank appears to vary unpredictably with dataset size, suggesting that the weight updates required for shower physics adaptation do not naturally decompose into low-rank structures.

Method# Trainable ParametersTraining Dataset SizeTraining Dataset SizeTraining Dataset SizeTraining Dataset SizeMean
Method# Trainable Parameters10210^{2}10310^{3}10410^{4}10510^{5}Mean
LoRA R12.67K0.2000.1600.1200.1480.157
LoRA R25.14K0.1850.1730.1050.1530.154
LoRA R410.27K0.1780.1480.1030.1420.143
LoRA R820.54K0.1320.1700.0970.1390.135
LoRA R1641.10K0.1780.1680.1220.1480.154
LoRA R3282.18K0.1530.1580.2610.1480.180
LoRA R48123.26K0.1450.1540.1180.1310.137
LoRA R64164.35K0.1490.1970.1340.1520.158
LoRA R106272.21K0.1020.1480.1040.1230.119
LoRA R204523.87K0.1100.1280.1090.1460.123

Table 7. Study of the rr parameter with WD evaluation metric.

Method# Trainable ParametersTraining Dataset SizeTraining Dataset SizeTraining Dataset SizeTraining Dataset SizeMean
Method# Trainable Parameters10210^{2}10310^{3}10410^{4}10510^{5}Mean
LoRA 12.67K0.2160.1670.2260.1750.196
LoRA 25.14K0.2920.2770.2410.2290.260
LoRA 410.27K0.3590.1890.1140.1590.205
LoRA 820.54K0.2460.2150.1400.1930.199
LoRA 1641.10K0.2750.3080.1580.1980.235
LoRA 3282.18K0.2170.1970.2870.1780.220
LoRA 48123.26K0.2140.1640.1900.1930.190
LoRA 64164.35K0.3150.2770.2200.2160.257
LoRA 106272.21K0.2090.2410.1880.2240.215
LoRA 204523.87K0.1610.2230.1970.1980.195

Table 8. Study of the rr parameter with KL evaluation metric.

Understanding LoRA Limitations through Post-Hoc Weight Analysis

To investigate why LoRA underperforms in the point cloud generation task, we conduct a post-hoc analysis of weight differences from successful full fine-tuning.6 This inverse LoRA decomposition analyses the actual weight updates from full fine-tuning to determine whether these transformations are inherently high rank, providing theoretical grounding for LoRA’s limited effectiveness.

Given a pre-trained model with weights Wpre∈Rm×nW_{\mathrm{pre}} \in\mathbb{R}^{m \times n} and a fully fine-tuned model with weights WftW_{\mathrm{ft}}, we compute the weight update as:

ΔW=Wft−Wpre∈Rm×n.(4)\Delta W = W_{\mathrm{ft}} - W_{\mathrm{pre}} \in\mathbb{R}^{m \times n}. \tag*{(4)}

The Singular Value Decomposition (SVD) factorises this matrix as

ΔW=UΣVT=∑i=1ρσiuiviT,(5)\Delta W = U\Sigma V^{\mathsf{T}} = \sum_{i=1}^{\rho} \sigma_i u_i v_i^{\mathsf{T}}, \tag*{(5)}

where ρ=min⁡(m,n)\rho= \min(m,n) is the maximum possible rank of ΔW\Delta W. For a matrix of dimension m×nm \times n, the rank cannot exceed the smaller dimension; for instance, a 512 × 256 matrix has at most rank 256. According to the Eckart-Young-Mirsky theorem [103], the optimal rank rr approximation minimizing Frobenius norm error is:

ΔWr=∑i=1rσiuiviT.(6)\Delta W_r = \sum_{i=1}^{r} \sigma_i u_i v_i^{\mathsf{T}}. \tag*{(6)}

The reconstruction error ϵr\epsilon_r is defined as the relative Frobenius norm:

ϵr=∥ΔW−ΔWr∥F∥ΔW∥F.(7)\epsilon_r = \frac{\lVert\Delta W - \Delta W_r \rVert_F}{\lVert\Delta W \rVert_F}. \tag*{(7)}

Using the orthogonality properties of SVD, this can be expressed in terms of singular values. Since ∥ΔW∥F2=∑i=1ρσi2\lVert\Delta W \rVert_F^2 = \sum_{i=1}^{\rho} \sigma_i^2 and the residual ΔW−ΔWr\Delta W - \Delta W_r contains only the truncated singular values, we have ∥ΔW−ΔWr∥F2=∑i=r+1ρσi2\lVert\Delta W - \Delta W_r \rVert_F^2 = \sum_{i=r+1}^{\rho} \sigma_i^2; therefore

ϵr=(∑i=r+1ρσi2∑i=1ρσi2)1/2.(8)\epsilon_r = \left(\frac{\sum_{i=r+1}^{\rho} \sigma_i^2}{\sum_{i=1}^{\rho} \sigma_i^2}\right)^{1/2}. \tag*{(8)}

Table 9 presents the layer wise theoretical minimum reconstruction errors. The results immediately reveal a striking pattern: boundary layers (0 and 5) achieve perfect reconstruction at their maximum rank of 4, while internal layers exhibit severe approximation errors even at rank 106. This heterogeneity poses a fundamental challenge for uniform rank allocation strategies.

LayerShapeMax RankRank 8Rank 10695% Energy Rank
Layerϵ8\epsilon_8 (%)Qualityϵ106\epsilon_{106} (%)Quality
Layer 0128×4128 \times44<0.01∗< 0.01^{*}Saturated<0.01∗< 0.01^{*}Saturated4
Layer 1256×128256 \times12812865.1Poor3.5Acceptable47
Layer 2512×256512 \times25625674.8Poor19.9Poor97
Layer 3256×512256 \times51225674.9Poor22.3Poor106
Layer 4128×256128 \times25612869.4Poor4.7Acceptable57
Layer 54×1284 \times1284<0.01∗< 0.01^{*}Saturated<0.01∗< 0.01^{*}Saturated4

Table 9. Layer-wise theoretical minimum reconstruction errors for LoRA approximations of full fine-tuning updates. Analysis performed independently per layer without inter-layer coupling.

Figure 17 illustrates how reconstruction error varies dramatically across layers as rank increases. Layers 2 and 3, which encode the most complex transformations, show particularly slow error reduction, remaining above 20% error even at rank 106. The singular value spectrum in Figure 18 provides deeper insight into this phenomenon. The normalized singular values reveal that layers 2 and 3 maintain significant magnitude even at high indices, with values staying above 1% of the maximum past index 250. This slow decay indicates these transformations span nearly the full parameter space rather than concentrating in a low dimensional subspace.

Figure 17. Per-layer theoretical minimum reconstruction error ϵr\epsilon_r for LoRA approximations. Analysis performed on individual layers without inter-layer coupling. Actual LoRA training would yield higher errors due to joint optimisation constraints and gradient coupling across layers.

Figure 18. Normalized singular value spectrum (σ~i=σi/σ1\widetilde{\sigma}_i = \sigma_i/\sigma_1) of weight updates from full fine-tuning. The slow decay in layers 2 and 3 indicates high intrinsic dimensionality incompatible with low rank approximation, while the sharp drops in layers 0 and 5 reflect their rank 4 constraint.

The singular value analysis reveals critical limitations for physics applications. Achieving 95% energy capture requires ranks of 47, 97, 106, and 57 for layers 1 through 4 respectively. The compression ratios of only 2.4 to 2.6 times for critical layers contrast sharply with the 10 to 100 times compression achieved in NLP tasks where LoRA succeeds [99].

These theoretical bounds suggest that successful adaptation requires higher ranks than commonly used in NLP applications. While this analysis doesn’t capture full training dynamics, it provides useful insight into why LoRA underperforms in our experiments and may guide future development of physics-specific PEFT methods. These findings suggest that the low-rank assumption underlying LoRA may be less suitable for physics transformations than for language tasks. While not conclusive, this analysis provides a starting point for understanding PEFT limitations in scientific applications and motivates exploration of alternative approaches that can accommodate heterogeneous complexity across network layers.

Geometric Mean and Error Propagation

For aggregating performance metrics across different observables with disparate scales, we employ a weighted geometric mean computed in logarithmic space. Given nn metrics with values {yi}i=1n\{y_i\}_{i=1}^{n}, standard deviations {σi}i=1n\{\sigma_i\}_{i=1}^{n}, and weights {wi}i=1n\{w_i\}_{i=1}^{n} (where ∑iwi=1\sum_i w_i = 1), the geometric mean and its uncertainty are computed as follows.

Geometric Mean Calculation

The weighted geometric mean is defined as:

yˉgeom=∏i=1nyiwi=exp⁡(∑i=1nwiln⁡yi).(9)\bar{y}_{\mathrm{geom}} = \prod_{i=1}^{n} y_i^{w_i} = \exp\left(\sum_{i=1}^{n} w_i \ln y_i\right). \tag*{(9)}

In practice, we compute this in base-10 logarithm for numerical stability:

yˉgeom=10Lˉ,(10)\bar{y}_{\mathrm{geom}} = 10^{\bar{L}}, \tag*{(10)}

where the mean in log-space is:

Lˉ=∑i=1nwilog⁡10(yi+ϵ),(11)\bar{L} = \sum_{i=1}^{n} w_i \log_{10}(y_i+\epsilon), \tag*{(11)}

with ϵ=10−10\epsilon= 10^{-10} added to avoid numerical issues with zero values.

Error Propagation

The uncertainty propagation through the logarithmic transformation follows from the delta method. For a value yiy_i with standard deviation σi\sigma_i, the uncertainty in log-space is:

σlog⁡,i=σi(yi+ϵ)ln⁡(10).(12)\sigma_{\log,i} = \frac{\sigma_i}{(y_i+\epsilon)\ln(10)}. \tag*{(12)}

The weighted variance in log-space becomes:

σLˉ2=∑i=1nwi2σlog⁡,i2.(13)\sigma_{\bar{L}}^2 = \sum_{i=1}^{n} w_i^2 \sigma_{\log,i}^2. \tag*{(13)}

Finally, the standard deviation of the geometric mean is obtained by transforming back from log-space:

σyˉgeom=yˉgeom⋅ln⁡(10)⋅σLˉ.(14)\sigma_{\bar{y}_{\mathrm{geom}}} = \bar{y}_{\mathrm{geom}} \cdot\ln(10) \cdot\sigma_{\bar{L}}. \tag*{(14)}

This approach ensures proper handling of metrics spanning multiple orders of magnitude while maintaining mathematically consistent error propagation.

References

  1. [1]ATLAS collaboration, ATLAS Software and Computing HL-LHC Roadmap, Tech. Rep. CERN-LHCC-2022-005, LHCC-G-182, CERN, Geneva (2022).
  2. [2]GEANT4 collaboration, GEANT4 - A Simulation Toolkit, Nucl. Instrum. Meth. A 506 (2003) 250.
  3. [3]J. Gavranović and B.P. Kerševan, Systematic evaluation of generative machine learning capability to simulate distributions of observables at the Large Hadron Collider, Eur. Phys. J. C 84 (2024) 911 [2310.08994].arxiv.org/abs/2310.08994
  4. [4]CMS collaboration, CMS Phase-2 Computing Model: Update Document, Tech. Rep. CMS-NOTE-2022-008, CERN-CMS-NOTE-2022-008, CERN, Geneva (2022).
  5. [5]ATLAS collaboration, The simulation principle and performance of the ATLAS fast calorimeter simulation FastCaloSim, Tech. Rep. ATL-PHYS-PUB-2010-013, CERN, Geneva (2010).
  6. [6]ATLAS collaboration, Performance of the Fast ATLAS Tracking Simulation (FATRAS) and the ATLAS Fast Calorimeter Simulation (FastCaloSim) with single particles, Tech. Rep. ATL-SOFT-PUB-2014-001, CERN, Geneva (2014).
  7. [7]CMS collaboration, The fast simulation of the CMS detector at LHC, J. Phys. Conf. Ser. 331 (2011) 032049.
  8. [8]M. Hildreth, V.N. Ivanchenkо, D.J. Lange and for the CMS Collaboration, Upgrades for the cms simulation, Journal of Physics: Conference Series 898 (2017) 042040.
  9. [9]HEP Software Foundation collaboration, A Roadmap for HEP Software and Computing R&D for the 2020s, Comput. Softw. Big Sci. 3 (2019) 7 [1712.06982].
  10. [10]B. Hashemi and C. Krause, Deep generative models for detector signature simulation: A taxonomic review, Rev. Phys. 12 (2024) 100092 [2312.09597].DOI
  11. [11]F.Y. Ahmad, V. Venkataswamy and G. Fox, A Comprehensive Evaluation of Generative Models in Calorimeter Shower Simulation, 2406.12898.arxiv.org/abs/2406.12898
  12. [12]ATLAS collaboration, AtlFast3: The Next Generation of Fast Simulation in ATLAS, Comput. Softw. Big Sci. 6 (2022) 7 [2109.02551].DOI
  13. [13]M. Barbetti, Lamarr: LHCb ultra-fast simulation based on machine learning models deployed within Gauss, in 21th International Workshop on Advanced Computing and Analysis Techniques in Physics Research: AI meets Reality, 3, 2023 [2303.11428].arxiv.org/abs/2303.11428
  14. [14]M. Paganini, L. de Oliveira and B. Nachman, CaloGAN : Simulating 3D high energy particle showers in multilayer electromagnetic calorimeters with generative adversarial networks, Phys. Rev. D 97 (2018) 014021 [1712.10321].DOI
  15. [15]ATLAS collaboration, Fast simulation of the ATLAS calorimeter system with Generative Adversarial Networks, Tech. Rep. ATL-SOFT-PUB-2020-006, CERN, Geneva (2020).
  16. [16]G.R. Khattak, S. Vallecorsa, F. Carminati and G.M. Khan, Fast simulation of a high granularity calorimeter by generative adversarial networks, Eur. Phys. J. C 82 (2022) 386 [2109.07388].
  17. [17]M. Erdmann, J. Glombitza and T. Quast, Precise simulation of electromagnetic calorimeter showers using a Wasserstein Generative Adversarial Network, Comput. Softw. Big Sci. 3 (2019) 4 [1807.01954].DOI
  18. [18]P. Musella and F. Pandolfi, Fast and Accurate Simulation of Particle Detectors Using Generative Adversarial Networks, Comput. Softw. Big Sci. 2 (2018) 8 [1805.08850].DOI
  19. [19]L. de Oliveira, M. Paganini and B. Nachman, Learning Particle Physics by Example: Location-Aware Generative Adversarial Networks for Physics Synthesis, Comput. Softw. Big Sci. 1 (2017) 4 [1701.05927].DOI
  20. [20]L. de Oliveira, M. Paganini and B. Nachman, Controlling Physical Attributes in GAN-Accelerated Simulation of Electromagnetic Calorimeters, J. Phys. Conf. Ser. 1085 (2018) 042017 [1711.08813].DOI
  21. [21]M. Paganini, L. de Oliveira and B. Nachman, Accelerating Science with Generative Adversarial Networks: An Application to 3D Particle Showers in Multilayer Calorimeters, Phys. Rev. Lett. 120 (2018) 042003 [1705.02355].DOI
  22. [22]G.r. Khattak, S. Vallecorsa and F. Carminati, Three dimensional energy parametrized generative adversarial networks for electromagnetic shower simulation, in 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 3913–3917, 2018, DOI.DOI
  23. [23]S. Vallecorsa, F. Carminati and G. Khattak, 3D convolutional GAN for fast simulation, EPJ Web Conf. 214 (2019) 02010.
  24. [24]D. Belayneh et al., Calorimetry with deep learning: particle simulation and reconstruction for collider physics, Eur. Phys. J. C 80 (2020) 688 [1912.06794].DOI
  25. [25]V. Chekalina, E. Orlova, F. Ratnikov, D. Ulyanov, A. Ustyuzhanin and E. Zakharov, Generative Models for Fast Calorimeter Simulation: the LHCb case, EPJ Web Conf. 214 (2019) 02034 [1812.01319].
  26. [26]S. Diefenbacher, E. Eren, G. Kasieczka, A. Korol, B. Nachman and D. Shih, DCTR GAN: Improving the Precision of Generative Models with Reweighting, JINST 15 (2020) P11004 [2009.03796].
  27. [27]K. Jaruskova and S. Vallecorsa, Ensemble Models for Calorimeter Simulations, J. Phys. Conf. Ser. 2438 (2023) 012080.DOI
  28. [28]M. Faucci Giannelli and R. Zhang, CaloShowerGAN, a generative adversarial network model for fast calorimeter shower simulation, Eur. Phys. J. Plus 139 (2024) 597 [2309.06515].DOI
  29. [29]F. Carminati, A. Gheata, G. Khattak, P. Mendez Lorenzo, S. Sharan and S. Vallecorsa, Three dimensional Generative Adversarial Networks for fast simulation, J. Phys. Conf. Ser. 1085 (2018) 032016.DOI
  30. [30]J. Erdmann, A. van der Graaf, F. Mausolf and O. Nackenhorst, SR-GAN for SR-gamma: super resolution of photon calorimeter images at collider experiments, Eur. Phys. J. C 83 (2023) 1001 [2308.09025].
  31. [31]ATLAS collaboration, Deep generative models for fast shower simulation in ATLAS, Tech. Rep. ATL-SOFT-PUB-2018-001, CERN, Geneva (2018).
  32. [32]E. Buhmann, S. Diefenbacher, E. Eren, F. Gaede, G. Kasieczka, A. Korol et al., Getting High: High Fidelity Simulation of High Granularity Calorimeters with High Speed, Comput. Softw. Big Sci. 5 (2021) 13 [2005.05334].DOI
  33. [33]J.C. Cresswell, B.L. Ross, G. Loaiza-Ganem, H. Reyes-Gonzalez, M. Letizia and A.L. Caterini, CaloMan: Fast generation of calorimeter showers with density estimation on learned manifolds, in 36th Conference on Neural Information Processing Systems: Workshop on Machine Learning and the Physical Sciences, 11, 2022 [2211.15380].arxiv.org/abs/2211.15380
  34. [34]S. Diefenbacher, E. Eren, F. Gaede, G. Kasieczka, A. Korol, K. Krüger et al., New angles on fast calorimeter shower simulation, Mach. Learn. Sci. Tech. 4 (2023) 035044 [2303.18150].
  35. [35]D. Salamani, A. Zaborowska and W. Pokorski, MetaHEP: Meta learning for fast shower simulation of high energy physics experiments, Phys. Lett. B 844 (2023) 138079.DOI
  36. [36]P. Raikwar, R. Cardoso, N. Chernyavskaya, K. Jaruskova, W. Pokorski, D. Salamani et al., Transformers for Generalized Fast Shower Simulation, EPJ Web Conf. 295 (2024) 09039.DOI
  37. [37]Q. Liu, C. Shimmin, X. Liu, E. Shlizerman, S. Li and S.-C. Hsu, Calo-VQ: Vector-Quantized Two-Stage Generative Model in Calorimeter Simulation, 2405.06605.arxiv.org/abs/2405.06605
  38. [38]K. Deja, J. Dubiński, P. Nowak, S. Wenzel and T. Trzciński, End-to-end sinkhorn autoencoder with noise generator, 2006.06704.
  39. [39]E. Buhmann, S. Diefenbacher, E. Eren, F. Gaede, G. Kasieczka, A. Korol et al., Decoding Photons: Physics in the Latent Space of a BIB-AE Generative Network, EPJ Web Conf. 251 (2021) 03003 [2102.12491].DOI
  40. [40]E. Buhmann, S. Diefenbacher, D. Hundhausen, G. Kasieczka, W. Korcari, E. Eren et al., Hadrons, better, faster, stronger, Mach. Learn. Sci. Tech. 3 (2022) 025014 [2112.09709].
  41. [41]A. Hariri, D. Dyachkova and S. Gleyzer, Graph Generative Models for Fast Detector Simulations in High Energy Physics, 2104.01725.
  42. [42]A. Abhishek, E. Drechsler, W. Fedorko and B. Stelzer, CaloDVAE : Discrete Variational Autoencoders for Fast Calorimeter Shower Simulation, in arXiv, 10, 2022 [2210.07430].arxiv.org/abs/2210.07430
  43. [43]C. Krause and D. Shih, Fast and accurate simulations of calorimeter showers with normalizing flows, Phys. Rev. D 107 (2023) 113003 [2106.05285].DOI
  44. [44]C. Krause and D. Shih, Accelerating accurate simulations of calorimeter showers with normalizing flows and probability density distillation, Phys. Rev. D 107 (2023) 113004 [2110.11377].DOI
  45. [45]S. Diefenbacher, E. Eren, F. Gaede, G. Kasieczka, C. Krause, I. Shekhzadeh et al., L2LFlows: generating high-fidelity 3D calorimeter images, JINST 18 (2023) P10017 [2302.11594].DOI
  46. [46]F. Ernst, L. Favaro, C. Krause, T. Plehn and D. Shih, Normalizing Flows for High-Dimensional Detector Simulations, SciPost Phys. 18 (2025) 081 [2312.09290].DOI
  47. [47]T. Buss, F. Gaede, G. Kasieczka, C. Krause and D. Shih, Convolutional L2LFlows: generating accurate showers in highly granular calorimeters using convolutional normalizing flows, JINST 19 (2024) P09003 [2405.20407].DOI
  48. [48]C. Krause, I. Pang and D. Shih, CaloFlow for CaloChallenge dataset 1, SciPost Phys. 16 (2024) 126 [2210.14245].DOI
  49. [49]I. Pang, D. Shih and J. A. Raine, Calorimeter shower superresolution, Phys. Rev. D 109 (2024) 092009 [2308.11700].DOI
  50. [50]E. Dreyer, E. Gross, D. Kobylianskii, V. Mikuni, B. Nachman and N. Soybelman, Automated Approach to Accurate, Precise, and Fast Detector Simulation and Reconstruction, Phys. Rev. Lett. 133 (2024) 211902 [2406.01620].DOI
  51. [51]J. Erdmann, J. Kann, F. MausoIf and P. Wissmann, Paraflow: fast calorimeter simulations parameterized in upstream material configurations, Eur. Phys. J. C 85 (2025) 857 [2503.21461].DOI
  52. [52]D. Smith, A. Ghosh, J. Liu, P. Baldi and D. Whiteson, Fast multi-geometry calorimeter simulation with conditional self-attention variational autoencoders, 2411.05996.arxiv.org/abs/2411.05996
  53. [53]V. Mikuni and B. Nachman, Score-based generative models for calorimeter shower simulation, Phys. Rev. D 106 (2022) 092009 [2206.11898].DOI
  54. [54]O. Amram and K. Pedro, Denoising diffusion models with geometry adaptation for high fidelity calorimeter simulation, Phys. Rev. D 108 (2023) 072014 [2308.03876].arxiv.org/abs/2308.03876
  55. [55]V. Mikuni and B. Nachman, CaloScore v2: single-shot calorimeter shower simulation with diffusion models, JINST 19 (2024) P02001 [2308.03847].DOI
  56. [56]L. Favaro, A. Ore, S.P. Schweitzer and T. Plehn, CaloDREAM – Detector Response Emulation via Attentive flow Matching, SciPost Phys. 18 (2025) 088 [2405.09629].DOI
  57. [57]S. Diefenbacher, V. Mikuni and B. Nachman, Refining fast calorimeter simulations with a Schrödinger Bridge, JINST 20 (2025) P08007 [2308.12339].arxiv.org/abs/2308.12339
  58. [58]F.T. Acosta, V. Mikuni, B. Nachman, M. Arratia, B. Karki, R. Milton et al., Comparison of point cloud and image-based models for calorimeter fast simulation, JINST 19 (2024) P05003 [2307.04780].arxiv.org/abs/2307.04780
  59. [59]D. Kobylianskii, N. Soybelman, E. Dreyer and E. Gross, Graph-based diffusion model for fast shower generation in calorimeters with irregular geometry, Phys. Rev. D 110 (2024) 072003 [2402.11575].DOI
  60. [60]D. Kobylianskii, N. Soybelman, N. Kakati, E. Dreyer, B. Nachman and E. Gross, Advancing set-conditional set generation: Diffusion models for fast simulation of reconstructed particles, Phys. Rev. D 110 (2024) 092013 [2405.10106].DOI
  61. [61]Y. Lu, J. Collado, D. Whiteson and P. Baldi, Sparse autoregressive models for scalable generation of sparse images in particle physics, Phys. Rev. D 103 (2021) 036012 [2009.14017].DOI
  62. [62]J. Liu, A. Ghosh, D. Smith, P. Baldi and D. Whiteson, Geometry-aware Autoregressive Models for Calorimeter Shower Simulations, in 36th Conference on Neural Information Processing Systems: Workshop on Machine Learning and the Physical Sciences, 12, 2022 [2212.08233].arxiv.org/abs/2212.08233
  63. [63]J. Liu, A. Ghosh, D. Smith, P. Baldi and D. Whiteson, Generalizing to new geometries with Geometry-Aware Autoregressive Models (GAAMs) for fast calorimeter simulation, JINST 18 (2023) P11003 [2305.11531].DOI
  64. [64]M.R. Buckley, C. Krause, I. Pang and D. Shih, Inductive simulation of calorimeter showers with normalizing flows, Phys. Rev. D 109 (2024) 033006 [2305.11934].DOI
  65. [65]M. Faucci Giannelli, G. Kasieczka, B. Nachman, D. Salamani, D. Shih and A. Zaborowska, “Fast calorimeter simulation challenge 2022 github page.” https://github.com/CaloChallenge/homepage, 2022.
  66. [66]P. Raikwar, A. Zaborowska, P. McKeown, R. Cardoso, M. Pi­orczynski and K. Yeo, A Generalisable Generative Model for Multi-Detector Calorimeter Simulation, 2509.07700.arxiv.org/abs/2509.07700
  67. [67]T. Buss, F. Gaede, G. Kasieczka, A. Korol, K. Krüger, P. McKeown et al., CaloHadronic: a diffusion model for the generation of hadronic showers, 2506.21720.DOI
  68. [68]S. Schnake, D. Krücker and K. Borras, CaloPointFlow II Generating Calorimeter Showers as Point Clouds, 2403.15782.arxiv.org/abs/2403.15782
  69. [69]E. Buhmann, S. Diefenbacher, E. Eren, F. Gaede, G. Kasieczka, A. Korol et al., CaloClouds: fast geometry-independent highly-granular calorimeter simulation, JINST 18 (2023) P11025 [2305.04847].DOI
  70. [70]E. Buhmann, F. Gaede, G. Kasieczka, A. Korol, W. Korkarci, K. Krüger et al., CaloClouds II: ultra-fast geometry-independent highly-granular calorimeter simulation, JINST 19 (2024) P04020 [2309.05704].arxiv.org/abs/2309.05704
  71. [71]T. Buss, H. Day-Hall, F. Gaede, G. Kasieczka, K. Krüger, A. Korol et al., CaloClouds3: Ultra-Fast Geometry-Independent Highly-Granular Calorimeter Simulation, arXiv (2025) [2511.01460].DOI
  72. [72]T. Buss, H. Day-Hall, F. Gaede, G. Kasieczka, K. Krüger, A. Korol et al., A First Full Physics Benchmark for Highly Granular Calorimeter Surrogates, 2511.17293.arxiv.org/abs/2511.17293
  73. [73]R. Bommasani, D.A. Hudson, E. Adeli, R. Altman and S.A. et al., On the opportunities and risks of foundation models, 2108.07258.arxiv.org/abs/2108.07258
  74. [74]S. Reed, K. Zolna, E. Parisotto, S.G. Colmenarejo, A. Novikov, G. Barth-Maron et al., A generalist agent, 2205.06175.arxiv.org/abs/2205.06175
  75. [75]T.B. Brown et al., Language Models are Few-Shot Learners, Adv. Neural Inf. Process. Syst. 33 (2020) 1901 [2005.14165].arxiv.org/abs/2005.14165
  76. [76]J. Birk, A. Hallin and G. Kasieczka, OmniJet-α: the first cross-task foundation model for particle physics, Mach. Learn. Sci. Tech. 5 (2024) 035031 [2403.05618].DOI
  77. [77]O. Amram, L. Anzalone, J. Birk, D.A. Faroughy, A. Hallin, G. Kasieczka et al., Aspen Open Jets: Unlocking LHC Data for Foundation Models in Particle Physics, 2412.10504.
  78. [78]J. Birk, F. Gaede, A. Hallin, G. Kasieczka, M. Mozzanica and H. Rose, OmniJet-αC: Learning point cloud calorimeter simulations using generative transformers, 2501.05534.
  79. [79]P. McKeown, P. Raikwar and A. Zaborowska, LEMURS dataset: Large-scale multi-detector ElectroMagnetic Universal Representation of Showers, 2509.05108.arxiv.org/abs/2509.05108
  80. [80]A. Radford and K. Narasimhan, Improving language understanding by generative pre-training, in OpenAI technical report, 2018, https://api.semanticscholar.org/CorpusID:49313245.
  81. [81]N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo et al., Parameter-efficient transfer learning for NLP, in Proceedings of the 36th International Conference on Machine Learning, K. Chaudhuri and R. Salakhutdinov, eds., vol. 97 of Proceedings of Machine Learning Research, pp. 2790–2799, PMLR, 09–15 Jun, 2019, https://proceedings.mlr.press/v97/houlsby19a.html.arxiv.org/abs/1902.00751
  82. [82]T. Karras, M. Aittala, T. Aila and S. Laine, Elucidating the design space of diffusion-based generative models, in Advances in Neural Information Processing Systems, S. Koyejо, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho and A. Oh, eds., vol. 35, pp. 26565–26577, Curran Associates, Inc., 2022, https://proceedings.neurips.cc/paper_files/paper/2022/file/a98846e9d9cc01cfb87eb694d946ce6b-Paper-Conference.pdf.arxiv.org/abs/2206.00364
  83. [83]F. Mokhtar, J. Pata, D. Garcia, E. Wulff, M. Zhang, M. Kagan et al., Fine-tuning machine-learned particle-flow reconstruction for new detector geometries in future colliders, Phys. Rev. D 111 (2025) 092015 [2503.00131].DOI
  84. [84]A. Chappell and L.H. Whitehead, Application of transfer learning to neutrino interaction classification, Eur. Phys. J. C 82 (2022) 1099 [2207.03139].DOI
  85. [85]T. Golling, L. Heinrich, M. Kagan, S. Klein, M. Leigh, M. Osadchy et al., Masked particle modeling on sets: towards self-supervised high energy physics foundation models, Mach. Learn. Sci. Tech. 5 (2024) 035074 [2401.13537].DOI
  86. [86]C. Li et al., Accelerating Resonance Searches via Signature-Oriented Pre-training, 2405.12972.arxiv.org/abs/2405.12972
  87. [87]V. Mikuni and B. Nachman, Solving key challenges in collider physics with foundation models, Phys. Rev. D 111 (2025) L051504 [2404.16091].arxiv.org/abs/2404.16091
  88. [88]M.P. Kuchera, R. Ramanujan, J.Z. Taylor, R.R. Strauss, D. Bazin, J. Bradt et al., Machine Learning Methods for Track Classification in the AT-TPC, Nucl. Instrum. Meth. A 940 (2019) 156 [1810.10350].DOI
  89. [89]R. Tombs and C.G. Lester, A method to challenge symmetries in data with self-supervised learning, JINST 17 (2022) P08024 [2111.05442].
  90. [90]F.A. Dreyer, R. Grabarczyk and P.F. Monni, Leveraging universality of jet taggers through transfer learning, Eur. Phys. J. C 82 (2022) 564 [2203.06210].DOI
  91. [91]H. Beauchesne, Z.-E. Chen and C.-W. Chiang, Improving the performance of weak supervision searches using transfer and meta-learning, JHEP 02 (2024) 138 [2312.06152].arxiv.org/abs/2312.06152
  92. [92]W. Bhimji, C. Harris, V. Mikuni and B. Nachman, OmniLearned: A Foundation Model Framework for All Tasks Involving Jet Physics, arXiv (2025) [2510.24066].DOI
  93. [93]ILD Concept Group collaboration, International Large Detector: Interim Design Report, 2003.01116.
  94. [94]M. Faucci Giannelli, G. Kasieczka, C. Krause, B. Nachman, D. Salamani, D. Shih et al., “Fast Calorimeter Simulation Challenge 2022 - Dataset 3.” https://doi.org/10.5281/zenodo.6366324, Mar., 2022.
  95. [95]O. Amram et al., CaloChallenge 2022: A Community Challenge for Fast Calorimeter Simulation, 2410.21611.
  96. [96]Geant4 Collaboration, “Par04 Example.” https://gitlab.cern.ch/geant4/geant4/-/tree/master/examples/extended/parameterisations/Par04, 2025.
  97. [97]P. Virtanen, R. Gommers, T.E. Oliphant et al., SciPy 1.0: fundamental algorithms for scientific computing in Python, Nature Methods 17 (2020) 261 [1907.10121].
  98. [98]E. Ben Zaken, Y. Goldberg and S. Ravfogel, BitFit: Simple parameter-efficient fine-tuning for transformer-based masked language-models, in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), S. Muresan, P. Nakov and A. Villavicencio, eds., (Dublin, Ireland), pp. 1–9, Association for Computational Linguistics, May, 2022, DOI.
  99. [99]E.J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang et al., LoRA: Low-Rank Adaptation of Large Language Models, 2106.09685.arxiv.org/abs/2106.09685
  100. [100]M. McCloskey and N.J. Cohen, Catastrophic interference in connectionist networks: The sequential learning problem, Academic Press 24 (1989) 109.DOI
  101. [101]N.S. Keskar, D. Mudigere, J. Nocedal, M. Smelyansky and P.T.P. Tang, On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima, in Proceedings of the 5th International Conference on Learning Representations (ICLR), 9, 2016 [1609.04836].arxiv.org/abs/1609.04836
  102. [102]A. Aghajanyan, S. Gupta and L. Zettlemoyer, Intrinsic dimensionality explains the effectiveness of language model fine-tuning, in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), C. Zong, F. Xia, W. Li and R. Navigli, eds., (Online), pp. 7319–7328, Association for Computational Linguistics, Aug., 2021, DOI.arxiv.org/abs/2012.13255
  103. [103]C. Eckart and G. Young, The approximation of one matrix by another of lower rank, Psychometrika 1 (1936) 211.DOI

Paper details

vlnc:W_xm7xfwyd
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents