betaFeedback

WorksPhysics and Astronomy

Particle Identification Studies using a Highly Granular Crystal Electromagnetic Calorimeter

Abstract

A proposal for the Circular Electron Positron Collider electromagnetic calorimeter is a highly granular scintillating crystal design that uses silicon photomultipliers to measure physics signals from photons. The potential to develop particle identification methods using deep and traditional machine learning is explored. A contrastive machine learning technique was found to enable the separation of showering and 'punch-through' pions without explicit labels. A supervised particle identification method was then trained using the results, using both a graph neural network and a boosted decision tree to distinguish electron, showering hadron, and minimum-ionizing-particle-like events. The classifier achieved an area-under-the-receiver operating characteristic curve of 0.996 for both cases. A Shapley additive explanations analysis showed the boosted decision tree classifier matched expectations for each event type. The particle identification capability is then demonstrated on experimental data taken from the CERN Proton Synchrotron T9 beamline.

Introduction

Future Higgs factories such as the Circular Electron-Positron Collider (CEPC) [1] and Future Circular Electron-Positron Collider (FCC-ee) [2] will precisely measure the properties of the Higgs boson, weak vector bosons, and the top quark, while also performing critical validations of the Standard Model. A typical figure of merit for the required performance of Particle Flow is to reconstruct the mass of the Higgs boson with 3 % to 4 % resolution. This approach necessitates accurate tracking of charged particles within a jet, sophisticated event reconstruction techniques, and highly granular sampling calorimeters. Furthermore, the electromagnetic calorimeter (ECAL) must possess a resolution sufficient to reconstruct the energy of photons, while also being designed with enough granularity to resolve individual particles with excellent energy and time resolution.

Particle identification (PID) is possible using the highly granular energy density information. This information can be used to enhance event reconstruction. Examples of relevant physics measurements that rely on PID are jet tagging (e.g., Higgs reconstruction from hadronic decays (e.g. H→qqˉH \rightarrow q\bar{q}), from photon decays (e.g., H→γγH \rightarrow\gamma\gamma) and π0\pi^{0} reconstruction (B0Bˉ0→π0π0B^{0}\bar{B}^{0} \rightarrow\pi^{0}\pi^{0}, relevant for CP-violation studies [3]). Jet energy resolution can also be improved through superior charged particle and track association by Particle Flow Algorithms (PFAs).

Hadrons may interact with the calorimeter either by ionizing and then showering, or by ionizing the medium only. These events are called showering hadrons and ’punch-through’ hadrons, respectively. Punch-through hadrons behave generally indistinguishably from muons or other MIPs at the momenta expected for future Higgs factories. A standard method of identifying MIP-like events is to build an ad hoc selection criterion for a MIP-like event by assuming a cut criterion. However, this method is limited as the final classification will only be as accurate as the initial definition and may introduce bias. For instance, challenging examples to define in a cut-based analysis could be when the ionizing track through the calorimeter is split between cells or when there is partial showering by either hadrons or muons. This motivates research into novel ways of defining subsets from data with at least two classes (e.g., showering and MIP-like pion), for which a pure sample of one class is available (muon/antimuon).

Two studies are performed. First, an unsupervised contrastive deep machine learning method is used to distinguish MIP-like pions from showering pions. Then, using the learned representation of MIP-like and showering pions, particle identification (PID) models are developed using deep machine learning and boosted decision tree classifiers (BDT), which are then compared.

Methods and Tools

Highly-Granular Crystal ECAL Prototype

The High-Granularity Crystal Calorimeter (HGCCAL) is the primary choice for the CEPC ECAL system [1]. A prototype has been developed to test the performance. Each layer of the crystal ECAL prototype consists of 14 scintillating crystal bars per layer, each of 12×12×2 cm312 \times12 \times2\ \mathrm{cm}^{3} volume. Each subsequent layer is rotated orthogonally with respect to the previous one. This ’cross-hatch’ structure enables reconstruction of the energy density of deposits in the crystal by combining information from two orthogonal and adjacent layers and transverse granularity while minimizing dead volume and the number of readout channels compared to a design with single crystals. Scintillation light from the crystal is read out from each end by two SiPMs. There are 14 layers of the calorimeter. The first 12 layers are composed of beryllium germanium oxide (BGO). The last 2 layers are composed of bismuth silicate (BSO), which acts as a tail-catcher. The total depth of the calorimeter is 24.9X024.9 X_{0}.

During 2025, the calorimeter was used in a test beam campaign at the Proton Synchrotron (PS) and Super Proton Synchrotron (SPS) at CERN in Geneva, Switzerland. The entire system was configured in a low-energy beamline setup with all upstream beam instrumentation removed to minimize particle scattering and momentum spread. A total of 20.8 million events were collected across an energy range of 0.1 GeV to 10 GeV. However, low beam purity for certain energies meant that samples of electrons were heavily contaminated by (anti)muons and hadrons in the range 6 GeV to 10 GeV. This motivated the development of a PID algorithm in order to purify the data.

A photograph of the calorimeter at the test beam is shown in Figure 2.

Photograph of the HGCCAL at the test beam at PS in 2025

Figure 2. Photograph of the HGCCAL at the test beam at PS in 2025.

Datasets

Simulation A Geant4 simulation [4] of electron, pion, and anti-muon events was used in the study, in the energy range from 0.1 GeV to 10 GeV. The raw simulation was digitized to model experimental and detector effects. Firstly, the digitization accounts for scintillation effects: the Poisson fluctuations of the number of scintillation photons, the attenuation of the light reaching the SiPMs due to self-absorption in the crystal, and the photon detection efficiency (PDE) of the sensors. The effective light yield has been determined as 200 p.e./MIP (200 p.e./8.9 MeV). Secondly, SiPM effects such as saturation and cross-talk are considered. Lastly, the effect of the readout electronics is considered, which affects the conversion of the SiPM charge to analog-to-digital (ADC) channels. Three gain levels are used to obtain a broad dynamic range of the sensors. The threshold between low and medium/high gain is 8×1038 \times10^{3} p.e. The signal is also convoluted with electronic noise as a normal distribution centered at 0 p.e. with an experimentally obtained scale of 1.56 p.e.. The simulation was then calibrated to units of GeV. A table of statistics is available in Appendix Table 2. Example events from the simulation are visualized in Figure 3.

3D event displays of interacting particles from the simulation dataset

Figure 3. Event displays of interacting particles from the simulation dataset. The crystal bars are indicated by the cuboidal bars. The color (blue to red) and opacity (clear to opaque) indicate higher vs. lower energy deposits in the bar. The xx and yy axes indicate the odd and even layers, and the zz-axis indicates the depth. The top left and right plots show an electron and anti-muon event, and the bottom left and right events show a showering pion and punch-through pion, respectively. The energy sums are shown in the legend.

Experimental Data During 2025, experimental data were taken at the CERN PS-T9 beamline using the crystal ECAL prototype. During this test beam, upstream instrumentation, such as SiFi trackers and Čerenkov detectors, was removed to minimize the beam momentum spread and improve resolution measurements. Energies from 5 GeV to 10 GeV of the campaign are studied henceforth and are used only to demonstrate the performance of the PID applied to data, as these were the most heavily contaminated of the samples available. The 5 GeV sample was expected to be a relatively pure sample of electrons. By contrast, in the range 6 GeV to 10 GeV, considerable contamination by (anti)muons and hadrons was expected.

The data were reconstructed using a standard analysis procedure for calorimeters with SiPM-based readout. The SiPM response is measured in “analog-to-digital” counts (ADC counts). Two ASIC operation modes were measured simultaneously during the test beam: high-gain and low-gain ADC values. Measurements of the pedestal, ADCpedADC_{\mathrm{ped}} and MIP-scale ADCMPVADC_{\mathrm{MPV}}, are required for energy reconstruction. The pedestal was measured for each readout channel using a double-sided Crystal Ball distribution (DSCB) fit of the SiPM pedestal for both high- and low-gain modes.

Diagram illustrating the design of the HGCCAL calorimeter

Figure 1. Diagram illustrating the design of the HGCCAL calorimeter.

The DSCB is introduced in Eq. 14a in Appendix Sec. A. The mode of the fit yields the pedestal value, ADCped\mathrm{ADC}_{\mathrm{ped}}. A normal distribution was also found to be consistent. The MIP-scale was measured using μ+\mu^{+} events. Ionizing energy deposits were selected by rejecting pedestal energy deposits based on the fitted distribution of the pedestal, L(DSCB(ADC∣θped))<−10\mathcal{L}(\mathrm{DSCB}(\mathrm{ADC} \mid\theta_{\mathrm{ped}})) < -10 and ADC>ADCped\mathrm{ADC} > \mathrm{ADC}_{\mathrm{ped}}, where θped\theta_{\mathrm{ped}} are the parameters from the pedestal fit. All subsequent layers were required to have a single energy ±1\pm1 bar index (lateral/horizontal), to ensure a well-defined, straight track per event. A Landau-Gaussian convolution was fitted to each channel to measure the MPV for the high-gain mode, ADCMPV,HG\mathrm{ADC}_{\mathrm{MPV,HG}}.

The high- and low-gain modes are non-linear and must be corrected. Events are selected from a high-purity 5 GeV dataset containing predominantly e−e^{-} events, and the joint distribution of the high- and low-gain ADC values is examined. Firstly, a locally weighted scatterplot smoothing (LOWESS) spline [5, 6] was fitted to the joint distribution to extract the overall profile of the non-linearity. Then, a reference linear relationship between high- and low-gain modes was reconstructed by selecting the spline region with the least curvature and requiring the line to pass through the pedestals of both modes. The low-gain MPV value, ADCMPV,LG\mathrm{ADC}_{\mathrm{MPV,LG}}, is then estimated from the line based on the value of ADCMPV,HG\mathrm{ADC}_{\mathrm{MPV,HG}}. Finally, the correction for the low-gain mode is obtained using a translation with a spline, Ccorr(ADCHG,ichannel)C_{\mathrm{corr}}(\mathrm{ADC}_{\mathrm{HG}}, i_{\mathrm{channel}}), which maps the spline to the reference line. An additive constant is used, as it does not affect the variance of the low-gain ADC value around the new line.

The energy is then reconstructed according to Eq. (1)

EMIP={ADCHG−ADCPed,HGADCMPV,HGif ADCHG<7900 countsADCLG+Ccorr(ADCHG,ichannel)−ADCPed,LGADCMPV,LGotherwise(1)E_{\mathrm{MIP}} = \begin{cases} \frac{\mathrm{ADC}_{\mathrm{HG}}-\mathrm{ADC}_{\mathrm{Ped,HG}}}{\mathrm{ADC}_{\mathrm{MPV,HG}}} & \text{if }\mathrm{ADC}_{\mathrm{HG}}<7900\text{ counts} \\ \frac{\mathrm{ADC}_{\mathrm{LG}}+C_{\mathrm{corr}}(\mathrm{ADC}_{\mathrm{HG}},i_{\mathrm{channel}})-\mathrm{ADC}_{\mathrm{Ped,LG}}}{\mathrm{ADC}_{\mathrm{MPV,LG}}} & \text{otherwise} \end{cases} \tag*{(1)}

A table of statistics is available in Appendix Table 3.

Classifier Frameworks

GravNet

GravNet is a graph neural network architecture introduced in Ref. [7] for learning on point clouds. It was explicitly designed for Particle Flow event reconstruction in highly granular calorimeters and is therefore used in this study. Point clouds are defined as points representing some information in Cartesian space. In the context of calorimeters, these ’features’ are the energy deposited or its timestamp, while the position of the sensor measuring the energy defines the graph. This model was chosen as it has demonstrated superior performance compared to applicable alternatives such as DGCNN and PointNet in Particle Flow reconstruction in highly granular calorimeters.

GravNet operates by clustering using the kk-nearest neighbors of each point in a projected subspace, ss, where proximity in the space represents similarity in the context defined by the optimization. The Gaussian function is then used to calculate weights for each neighbour cluster, wij=exp⁡(−γ⋅d(si,sj)2)w_{ij}=\exp(-\gamma\cdot d(s_i,s_j)^2), d(u,v)d(u,v) is the L2-norm between two vectors uu and vv, d(u,v)=∥u−v∥2d(u,v)=\lVert u-v\rVert_2, and sis_i, sjs_j. An additional learned feature space FLRF_{\mathrm{LR}} is obtained, representing learned properties about each point in the data. FLRF_{\mathrm{LR}} is then weighted according to all the weights obtained for each cluster (i.e., each point is scaled by kk weights from the subspace) and aggregated over the cluster dimension. In this study, the maximum, the mean, and the second moment, representing unnormalized variance, are used as aggregation functions. A final output layer projects the output to a new representation of the graph. The concept is illustrated in Figure 4.

Flow chart diagram illustrating the architecture of GravNet

Figure 4. Flow chart diagram illustrating the architecture of GravNet. VV are some input vertices indicated by the Utah teapot. f(V;θ)f(V;\theta) indicates a convolutional layer with learnable parameters θ\theta. kk-NN indicates the kk nearest neighbours.

GravNet operations are typically applied recursively to obtain deeper representations of the underlying distributions of the data. The PyTorch Geometric [8] and PyTorch [9] software are used to implement this framework in the study.

Boosted Decision Trees

Boosted decision trees (BDTs) are a commonly used machine learning model for classification. They utilize an ensemble of many weak decision trees. A decision tree is a learning model that splits data using recursively applied, learnable

thresholds and is typically shallow. Ensembles are used because they are less sensitive to outliers than a single strong classifier. The trees are aggregated to produce a prediction.

’Gradient boosting’ refers to optimizing using the gradient of the loss with respect to the ensemble’s predictions. This is achieved by fitting a new tree using the gradients and adding it to the model during training. It has been demonstrated that this is a more accurate learning method and generalizes better than a single decision tree [10]. They are often applied in particle physics as classifier models as they are typically interpretable and robust.

The xGBoost framework [11] is used to implement boosted decision trees in the study.

Methods

Unsupervised PID for Label Extraction for Pions

Hadrons interacting with a calorimeter are generally expected to either shower or act as minimum ionizing particles and punch through. The set of events recorded by the calorimeter can be defined as π∈M∪M‾\pi\in M \cup\overline{M}, where MM is the set of MIP-like hadrons and M‾\overline{M} is the set of showering hadrons or other types of events, such as partially showering hadrons or other particles. By contrast, muons are drawn nearly always from MM, with a very small probability of showering.

A discriminative representation of MM can be learned by requiring two criteria. Firstly, any two samples drawn from MM should be ’close’ in a chosen metric. Secondly, M∪M‾M \cup\overline{M} should be ’far’ from MM in that metric. Since punch-through hadron events, represented in the study by MIP-like pions in simulation, are drawn from the subset of MM within M∪M‾M \cup\overline{M} and are indistinguishable and constrained to remain close to the muon events drawn from MM, optimizing both conditions necessarily pushes elements of M‾\overline{M}, which are showering pions in simulation, further from MM. In summary, if a pure set of one class can be obtained, then it can be extracted from a mixed set without prior knowledge of the difference. This concept is referred to as contrastive learning.

A method known as a Siamese neural network is typically used for contrastive learning. This is a configuration of a machine learning model that takes multiple inputs during training with the same weights and outputs a latent vector for each input. The network is optimized to draw examples of the same class closer and push other classes further apart by comparing the outputs of the neural network between similar and dissimilar samples of inputs.

The loss function is called ’triplet loss’ [12]. An ’anchor’ vector, a∈Ma \in M, is used as a reference point for comparisons. Then, two further samples, a positive and a negative sample (p∈Mp \in M, n∈M∪M‾n \in M \cup\overline{M}), are selected. The loss is then defined according to Eq. (2):

L(a,p,n)=max⁡{d(a,p)−d(a,n)+margin⁡,0},(2)\mathcal{L}(a,p,n)=\max\{d(a,p)-d(a,n)+\operatorname{margin},0\}, \tag*{(2)}

where L\mathcal{L} is the loss, d(u,v)d(u,v) is the L2-norm between two vectors uu and vv, d(u,v)=∥u−v∥2d(u,v)=\lVert u-v\rVert_{2}. The maximum is taken over one mini-batch used for training. The fundamental idea is that vectors of the positive class are drawn towards the anchor class, while those from the negative class are pushed away from it. The maximum function selects the most challenging case in the batch to speed up convergence and is called ’mining’. For this reason, the anchor and positive samples were randomly sampled from the MM dataset and the negative from the M‾\overline{M} dataset for every triplet.

It is noted that the neural network may learn a trivial solution to the contrastive problem by increasing the magnitude of the latent vector to become arbitrarily large. For instance, the network may enter a failed convergence state where the vectors of one class are much longer than the other, which means the loss does not improve in a meaningful way. The vector is normalised using the L2-norm to avoid this problem, which constrains the distance to d(u,v)=21−cos⁡θd(u,v)=2\sqrt{1-\cos\theta}, d(u,v)∈[0,2)d(u,v)\in[0,2).

Inference is achieved by finding the latent vectors from the training dataset of the target class and taking a normalized average. This average vector is called the ’prototype’. The cosine similarity between the positive class and the prototype then represents the intra-class variation. This can be used to determine a cutoff criterion to classify events as either positive or negative in the mixed dataset.

Supervised PID for Performance Studies

The previous section provides a method for obtaining labels for the MIP-like pion sample. After this is achieved, the goal of the study is to assess the particle identification performance of the ECAL prototype. The model should classify three groups of events: ’electron’, ’MIP-like’, and ’showering hadron-like’. The showering hadron fraction is determined using the results of the Siamese neural network method presented in § 2.3.1, the results of which are discussed in § 3.1.

Diagram illustrating the concept of contrastive triplet loss training

Figure 5. Diagram illustrating the concept of contrastive triplet loss training. Red vectors indicate the positive (target) class (M)(M) and the blue vectors represent the negative class (M‾)(\overline{M}). The deep red line indicates the anchor. The red lines indicate other μ\mu events. The blue lines indicate showering hadrons.

PID was performed using both the GravNet and BDT models, using cross-entropy loss:

L(y,p)=−1N∑i=1N∑j=1Cyijlog⁡(pij)(3)\mathcal{L}(y,p)=-\frac{1}{N}\sum_{i=1}^{N}\sum_{j=1}^{C}y_{ij}\log(p_{ij}) \tag*{(3)}

where L\mathcal{L} is the loss, NN is the number of events (either over a mini-batch for the neural network or the entire dataset for the BDT), CC is the number of classes (3), with indices ii and jj, respectively. yijy_{ij} is the ground-truth label (yij=1y_{ij}=1 if event ii belongs to class jj) and pijp_{ij} is the predicted probability of that class by the model. Each model predicts logits (zz), and the probability is reconstructed using the Softmax operator σ(z)\sigma(z):

σ(zj)=exp⁡(zj)∑jexp⁡(zj)(4)\sigma(z_j)=\frac{\exp(z_j)}{\sum_j \exp(z_j)} \tag*{(4)}

Training

Unsupervised PID

A Siamese neural network consisting of five successive GravNet layers, using 8 spatial dimensions for the clustering space and 32 dimensions for the feature space. 16 neighbors are used for clustering. The input to the neural network is the natural logarithm of one plus the energy deposited in a bar, which is performed to constrain the highly skewed energy distribution while remaining in the domain Ehit∈[0,∞)E_{\mathrm{hit}}\in[0,\infty), and the graph nodes are defined by the crystal bar index and layer. All intermediate layer outputs are concatenated to preserve multi-scale features, which are then aggregated via mean and second-moment pooling across the event. These pooled features pass through a two-layer multi-layer perceptron (MLP) with 512 and 256 hidden units, incorporating 20 % dropout for regularization. The outputs are L2-normalized 128-dimensional embeddings where same-class particles cluster and different classes separate in the latent space. In all cases, the leaky ReLU activation with a default negative-value gradient of 0.01 is used.

Two models were trained: one to distinguish e−e^{-} from μ+\mu^{+} as a control method, and another to distinguish π−\pi^{-} from μ+\mu^{+}. A validation sample of 1000 events of each particle type at every particle energy was used. The model learning rate was optimized using the method of Ref. [13], using the torch_lr_finder software library [14]. The ADAM optimizer [15] and an exponential learning rate decay were used to optimize.

After training converged, a global prototype, pp, for both models was determined by taking the normalized mean of a random subset of 1×1051 \times10^{5} embedding vectors from the training dataset on the target class, μ+\mu^{+}. The validation set containing each particle was then projected onto the prototype, cos⁡θ=u⋅p\cos\theta= \mathbf{u} \cdot\mathbf{p}. This maps the vectors to a cosine similarity in the domain [−1,1)[-1, 1). A cutoff criterion was determined from the intra-class distribution of μ+\mu^{+}, using the right-tailed confidence interval P(cos⁡θ≥X)=99P(\cos\theta\ge X) = 99 %, performed for each classifier separately.

Firstly, the discrimination power of the model was tested using the e−e^{-} vs. μ+\mu^{+} model. This trivial separation task illustrates that the model functions as intended. The confusion matrix and Receiver Operating Characteristic (ROC) curve show the true positive rate as a function of the false positive rate of the classifier:

TPR=TPTP+FN(5)\mathrm{TPR} = \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}} \tag*{(5)}
FPR=FPFP+TN(6)\mathrm{FPR} = \frac{\mathrm{FP}}{\mathrm{FP}+\mathrm{TN}} \tag*{(6)}

Where TPR and FPR are the true and false positive rates, TP and FP are the numbers of true and false positives, and TN and FN are the numbers of true and false negatives. The area under the curve (AUC) measures performance, AUC ∈[0.5,1)\in[0.5, 1), where 0.5 is a random classifier, and 1.0 is perfect.

The same study was then performed for the π−\pi^{-} vs. μ+\mu^{+} model, with the expectation that some of the π−\pi^{-} events cannot be distinguished from the μ+\mu^{+} events. These ’misidentified’ events are then compared to MIP-like μ+\mu^{+} events. As a ground-truth label is not available in this case, agreement between MIP-like π−\pi^{-} events and μ+\mu^{+} events is measured in several ways. Fits of the Landau-Gaussian distribution to the energy deposited per layer are used to measure the agreement with the expectation for ionizing energy loss. If the cut effectively selects MIP-like events, the agreement of the Landau-Gaussian distributions with the layer-wise energy distributions should be good; the most probable value for the energy loss per layer should be equal across layers and approximately the same for the ’misidentified’ π−\pi^{-} events and the μ+\mu^{+} sample. It was noted that below around 0.5 GeV, the μ+\mu^{+} sample did not achieve full penetration of the calorimeter. The 0.4 GeV sample is nonetheless presented to illustrate this effect. To assess the showering part of the sample, shower shape fits were used to compare the measured value of the pion nuclear interaction length for BGO with the literature. Finally, the fraction of events predicted to ’punch through’ the calorimeter can be used to assess the agreement with expectations for the MIP-like behavior of the π−\pi^{-} sample:

PS=e−Lλπ(7)P_{\mathrm{S}} = e^{-\frac{L}{\lambda_{\pi}}} \tag*{(7)}

Where LL is the longitudinal depth of the calorimeter, L=28L = 28 cm, and λπ\lambda_{\pi} is the pion interaction length of BGO, λπ=26.72\lambda_{\pi} = 26.72 cm [16, 17].

Supervised PID

The classifiers are designed to separate 3 classes of events: ’electron’, ’showering hadron’, and ’MIP-like’ events. From the raw dataset, e−e^{-} and μ+\mu^{+} are labeled ’electron’ and ’MIP-like’ by default. The π−\pi^{-} particles are labeled using the Siamese network developed in Sec. 2.4.1. The 99 % right-tailed percentile of the prototype projection distribution is used as a cut criterion. Events with a projected value below the criterion are labeled as ’showering hadron-like’, and above the criterion they are labeled as ’MIP-like’. Justification for this is given in the results presented in Sec. 3.1. A separate validation sample of 1000 events for every event class and available particle energy was selected, totaling 84 000 events.

Two classifier models were trained to test the performance and validity of different classifier methods. The first was a GravNet network with the same structure as described in Sec. 2.4.1, with separate weights, used as the classifier model. The only difference is in the final layer, which produces 3 logits for each event class. All training and hyperparameter optimization details were otherwise the same as for that network. As the computational resources required for hyperparameter tuning are considerably less for a BDT than a deep neural network, the hyperparameters (learning rate, maximum tree depth, fraction of the training data to randomly sample without replacement before growing each tree, and fraction of variables to randomly select for each tree) were tuned using the Optuna hyperparameter tuning library [18]. The second was a standard BDT that uses a simple set of standard global variables for showers as input, shown in Table 1. The distributions of each parameter are shown for the 0.2 GeV, 1 GeV and 5 GeV samples in Appendix Figure 18, Appendix Figure 19 and Appendix Figure 20, respectively. The moments of the central log ratio (CLR) of the bar energies in a shower used in the classifier are the standard deviation (σclr\sigma_{\mathrm{clr}}), skewness (γclr\gamma_{\mathrm{clr}}), and excess kurtosis (κclr\kappa_{\mathrm{clr}}). The CLR is defined as:

Variable NameSymbolDefinition
CLR Varianceσclr\sigma_{\mathrm{clr}}m2(CLR⁡(Ebar))\sqrt{m_{2}(\operatorname{CLR}(E_{\mathrm{bar}}))}
CLR Skewnessγclr\gamma_{\mathrm{clr}}m3(CLR⁡(Ebar))m2(CLR⁡(Ebar))1.5\frac{m_{3}(\operatorname{CLR}(E_{\mathrm{bar}}))}{m_{2}(\operatorname{CLR}(E_{\mathrm{bar}}))^{1.5}}
CLR Excess Kurtosisκclr\kappa_{\mathrm{clr}}m4(CLR⁡(Ebar))m2(CLR⁡(Ebar))2\frac{m_{4}(\operatorname{CLR}(E_{\mathrm{bar}}))}{m_{2}(\operatorname{CLR}(E_{\mathrm{bar}}))^{2}}
Center of Gravity, xxCoGX\mathrm{CoG}_{X}∑i=2,even6Ebar,i⋅iEsum,even\frac{\sum_{i=2,\mathrm{even}}^{6}E_{\mathrm{bar},i}\cdot i}{E_{\mathrm{sum,even}}}
Center of Gravity, yyCoGY\mathrm{CoG}_{Y}∑i=1,odd14Ebar,i⋅iEsum,odd\frac{\sum_{i=1,\mathrm{odd}}^{14}E_{\mathrm{bar},i}\cdot i}{E_{\mathrm{sum,odd}}}
Center of Gravity, zzCoGZ\mathrm{CoG}_{Z}∑l=114Ebar,l⋅lEsum\frac{\sum_{l=1}^{14}E_{\mathrm{bar},l}\cdot l}{E_{\mathrm{sum}}}
Mean Distance from Center of Gravity, xx⟨∣X−CoGX∣⟩\langle|X-\mathrm{CoG}_{X}|\rangle∑i=2,even6∣i−CoGX∣Nhit,even\frac{\sum_{i=2,\mathrm{even}}^{6}|i-\mathrm{CoG}_{X}|}{N_{\mathrm{hit,even}}}
Mean Distance from Center of Gravity, yy⟨∣Y−CoGY∣⟩\langle|Y-\mathrm{CoG}_{Y}|\rangle∑i=1,odd14∣i−CoGY∣Nhit,odd\frac{\sum_{i=1,\mathrm{odd}}^{14}|i-\mathrm{CoG}_{Y}|}{N_{\mathrm{hit,odd}}}
Fraction of Shower Energy In Layer llflayer,l−1f_{\mathrm{layer},l-1}Ebar,lEsum\frac{E_{\mathrm{bar},l}}{E_{\mathrm{sum}}}

Table 1. Event-level aggregated parameters used as inputs to the BDT. The table lists the variable name, symbol, and definition for each. Here, ii is the bar index (left→\toright, up→\todown) running from 1 to 6, ll is the layer index from 1 to 14, EsumE_{\mathrm{sum}} is the total deposited energy in the event, NhitsN_{\mathrm{hits}} is the number of bars above threshold, and “even”/“odd” refer to even/odd bar indices (for transverse) or layers (for longitudinal). flayer,l−1f_{\mathrm{layer},l-1} is defined such that flayer,0f_{\mathrm{layer},0} corresponds to the first layer. mim_i refers to the ithi^{\mathrm{th}} statistical moment.

Distributions of event-level variables used in the BDT classifier for the 0.2 GeV simulation sample

Figure 18. Distributions of event-level variables used in the BDT classifier for the 0.2 GeV simulation sample. The black, red, and green distributions represent electron, MIP-like, and showering hadron-like events. The symbols shown correspond to Table 1.

Distributions of event-level variables used in the BDT classifier for the 1 GeV simulation sample

Figure 19. Distributions of event-level variables used in the BDT classifier for the 1 GeV simulation sample. Else, as in Figure 18.

Distributions of event-level variables used in the BDT classifier for the 5 GeV simulation sample

Figure 20. Distributions of event-level variables used in the BDT classifier for the 5 GeV simulation sample. Else, as in Figure 18.

CLR⁡(Ebar)=log⁡(Ebar)−⟨log⁡(Ebar)⟩,Ebar>0(8)\operatorname{CLR}(E_{\mathrm{bar}})=\log(E_{\mathrm{bar}})-\langle\log(E_{\mathrm{bar}})\rangle,\qquad E_{\mathrm{bar}}>0 \tag*{(8)}

CLR is chosen for representing shower fluctuations, as the transformation is invariant under additive shifts in log space and is therefore independent of the total energy deposited in an event. This has two key advantages: no energy scale is required for the classifier, which means that the model is never exposed to the limits of the energy spectrum; it provides good agreement when comparing simulation to data, for which MIP-energy scaling may be slightly different.

Results

Unsupervised PID

Firstly, the results of the e−e^{-} vs. μ+\mu^{+} model are presented to demonstrate the classification capacity of the model using ground truth labels. Then, the results of the e−e^{-} vs. π−\pi^{-} model are presented and analysed.

e- vs. +

The distribution of the projected value onto the prototype for the training and validation positive class (μ+\mu^{+}), and the validation negative class (e−e^{-}), the selected cut value, confusion matrix, and ROC curve are presented for the π−\pi^{-} vs. e−e^{-} model in Figure 6.

Figure 6. Performance metrics of the unsupervised e−e^{-} vs. μ+\mu^{+} model. Figure 6a shows the distribution of the projections onto the prototype for the positive training and validation datasets (μ+\mu^{+}), shown in light shaded blue and blue. The vertical purple line indicates the 99 % right-tailed percentile (‘cut’) of the positive training dataset (shaded blue region). The negative validation sample less than or equal to the cut value (showering hadron) is shown in green, and the negative validation sample greater than the cut value (MIP-like) is shown in red. Figure 6b shows the confusion matrix of the classifier using the particle labels. White to blue indicates fewer to more events. Figure 6c shows the ROC curve of the classifier using the particle labels. The blue line shows the ROC curve, and the purple dashed line shows the expectation for a random classifier. The AUC is shown in the legend.

Figure 6a shows the distribution of the μ+\mu^{+} and π−\pi^{-} projections on the prototype in the hyperspherical latent space of the Siamese network. A mostly bimodal distribution is observed with clear peaks at cos⁡θ≃−1\cos\theta\simeq-1 (not MIP-like) and cos⁡θ≃1\cos\theta\simeq1 (MIP-like). This indicates the two classes are positioned at antipodal poles in the latent space, with each pole corresponding to one of the two classes. A small third peak is observed for the π−\pi^{-} sample at around cos⁡θ≃0.5\cos\theta\simeq0.5. This third peak is observed most strongly for the 0.1 GeV sample and is practically negligible at higher energies. The peak corresponds to electron events with roughly the same number of active bars in the event as a μ+\mu^{+} event of the same energy, and indicates that these ’low activity’ events are closer to MIP-like events than normal showering electrons. However, it is noted that this third peak does not considerably overlap with the μ+\mu^{+} projection distribution, and therefore does not considerably affect the effectiveness of the discrimination cut at cos⁡θ=0.954\cos\theta= 0.954.

Figure 6b and Figure 6b shows the confusion matrix and ROC for e−e^{-} and μ+\mu^{+} classification, respectively. The muon classifier has a muon purity of 98.1 % and rejects 99.81 % of electrons. The AUC for the classifier is greater than 99.9 %. This provides clear evidence that the model works effectively as intended as a model for a basic particle classification task.

π−\pi^{-} vs. μ+\mu^{+}

The same results are first presented for the π−\pi^{-} vs. μ+\mu^{+} in Figure 7, as for the e−e^{-} vs. μ+\mu^{+} in Figure 6. Figure 7a shows a similar result as for the e−e^{-} vs. μ+\mu^{+} case in Figure 6a, indicating antipodal poles for each class. However, a considerable fraction of the total sample of events matches the distribution of μ+\mu^{+}, indicated by the red histogram where cos⁡θ>0.485\cos\theta> 0.485. Again, a third peak is observed at around cos⁡θ≃0.5\cos\theta\simeq0.5, and was again found to correspond to ’low activity’ π−\pi^{-} events with similar numbers of active bars to the μ+\mu^{+} sample.

Projection distributions for the $\mu^{+}$ vs. $\pi^{-}$ model

(a)

Confusion matrix for the $\mu^{+}$ vs. $\pi^{-}$ validation set

(b)

ROC curve for the $\mu^{+}$ vs. $\pi^{-}$ validation set

(c)

Figure 7. Performance metrics of the unsupervised π−\pi^{-} vs. μ+\mu^{+} model. Otherwise, figures correspond to descriptions of Figure 6.

Comparison with the confusion matrix in Figure 7c reveals a μ+\mu^{+} efficiency of 98.6 % but a purity of only 71.2 %. The large fraction indicates irreducible background, which can be interpreted as MIP-like hadrons, of which a total of 40 % are π+\pi^{+} passing the cut across all particle energies. This latter fraction can be interpreted as punch-through pions, which are presently studied.

Energy deposited by ionization is expected to follow the Landau-Gaussian distribution, defined according to Eq. (9)

LandauGaus⁡(x;μ,σ,MPV⁡,ξ)=∫−∞∞L(x′;MPV⁡,ξ)⋅N(x−x′;μ=0,σ) dx′(9)\operatorname{LandauGaus}(x;\mu,\sigma,\operatorname{MPV},\xi) = \int_{-\infty}^{\infty} L(x';\operatorname{MPV},\xi) \cdot\mathcal{N}(x-x';\mu=0,\sigma)\,\mathrm{d}x' \tag*{(9)}

where LL and N\mathcal{N} are the Landau and Gaussian distributions, μ\mu and σ\sigma are the Gaussian mean and width parameters, and MPV⁡\operatorname{MPV} and ξ\xi are the most probable value and scale parameter of the Landau distributions. The Landau distribution is calculated via approximation according to Ref. [19]. The convoluted density function is determined numerically.

The overall distribution of deposited energy in each bar and the corresponding distribution per layer were found for each of the ’real’ μ+\mu^{+} events and the π−\pi^{-} events passing or failing the cut, defined as ’MIP-like’ or ’showering hadron-like’ π−\pi^{-} events, respectively. Events were selected with particle energies greater than 1 GeV, below which MIP-like events were observed to range out in the calorimeter due to reaching the Bragg region of the Bethe-Bloch equation. The distributions were then fitted using the Landau-Gaussian distribution with the MIGRAD algorithm of iminuit using extended binned log-likelihood [20].

The total distribution of energies and the fitted MPV per layer for both classes are shown in Figure 8. Example fits per particle energy are shown in Appendix Fig. 21. The results show that μ+\mu^{+} and MIP-like π−\pi^{-} events follow a Landau-Gaussian distribution with good agreement, with reduced χ2\chi^{2} values of 5.39 and 6.99 in both cases. When fitted layer by layer, the agreement improves to reduced χ2\chi^{2} values of 1.85 ±\pm 0.64 and 2.00 ±\pm 0.78, respectively. The energy deposited per layer in both cases is nearly a constant value of 2.0 MIP, corresponding to the sum of two SiPMs independently calibrated to 1 MIP for each bar. The MPV values differ by only 1.53 ±\pm 0.26 % over the layers. This result can be explained by the marginally greater MPV of μ+\mu^{+} than π−\pi^{-} owing to its smaller mass. This is illustrated in Appendix Fig. 22, which shows the differential energy loss per layer for μ+\mu^{+} and π−\pi^{-}, for which the loss is greater for μ+\mu^{+} for energies above around 0.5 GeV. It is noted that this MPV value is averaged across several energies on which there is a small expected dependence, which is not shown.

Landau-Gaussian fits and per-layer MPV values

Figure 8. Left: Landau-Gaussian fits to the energy distribution for all bars with energies greater than 1 GeV. Solid and dashed lines indicate simulation and fits, respectively. Right: Per-layer MPV values of fits with particle energies. Shaded areas indicate the standard deviation of the MPV across particle energies from 1 GeV to 10 GeV. The bottom subplot shows the ratio between the μ+\mu^{+} and π−\pi^{-} MPV, indicated by the red dots. Blue and red lines and shaded regions in both plots represent the μ+\mu^{+} and π−\pi^{-} respectively. Fit values and the mean and standard deviation of the reduced-χ2\chi^{2} values are shown in the legend.

The events failing the cut were expected to be showering π−\pi^{-} events. By fitting the expected longitudinal hadronic shower shape event by event, the shower start position, zssz_{\mathrm{ss}}, can be extracted and studied. The model for the shower shape is described as a mixture of a uniform distribution to describe the MIP track prior to the shower, and a Gamma distribution for the showering particle, which is often used to parameterize the longitudinal development of particle showers [21, 22, 23, 24, 25]

dEdz=EMIP⋅Uniform⁡(z;0,zss)+Eshower⋅Gamma⁡(z;αz,βz,Δz=zss),(10)\frac{\mathrm{d}E}{\mathrm{d}z} = E_{\mathrm{MIP}} \cdot\operatorname{Uniform}(z;0,z_{\mathrm{ss}}) + E_{\mathrm{shower}} \cdot\operatorname{Gamma}(z;\alpha_z,\beta_z,\Delta_z=z_{\mathrm{ss}}), \tag*{(10)}
Gamma⁡(z;αz,βz,Δz)=βzαzΓ(αz)(z−Δz)αz−1e−βz(z−Δz)Θ(z−Δz),(11)\operatorname{Gamma}(z;\alpha_z,\beta_z,\Delta_z) = \frac{\beta_z^{\alpha_z}}{\Gamma(\alpha_z)}(z-\Delta_z)^{\alpha_z-1}e^{-\beta_z(z-\Delta_z)}\Theta(z-\Delta_z), \tag*{(11)}
Uniform⁡(z;a,b)=1b−aΘ(z−a)Θ(b−z).(12)\operatorname{Uniform}(z;a,b) = \frac{1}{b-a}\Theta(z-a)\Theta(b-z). \tag*{(12)}

where β\beta is a shower rate parameter, β=(α−1)/Tz\beta=(\alpha-1)/T_z, TzT_z is the shower’s maximum position, αz\alpha_z is a shape parameter that characterizes the shower’s symmetry, and Δz\Delta_z is an offset. EMIPE_{\mathrm{MIP}} and EshowerE_{\mathrm{shower}} are the energies deposited by the ionizing track and shower, respectively, with total energy Esum=EMIP+EshowerE_{\mathrm{sum}}=E_{\mathrm{MIP}}+E_{\mathrm{shower}}. Two physical bounds are imposed: the MIP plateau height is restricted to 0.3≤EMIP/zss≤80.3 \le E_{\mathrm{MIP}}/z_{\mathrm{ss}} \le8 MIP per layer, which constrains the total energy of the MIP track prior to the shower to default-distributed energies. The shower rate parameter is the reciprocal of the scale in zz, and is therefore constrained such that the shower scale is physical, βz≥13\beta_z \ge\frac{1}{3} layer−1^{-1}. In addition, the shower start is required to precede the shower maximum, zss<Tzz_{\mathrm{ss}} < T_z. The distributions were fitted using the MIGRAD algorithm

of iminuit with an extended binned log-likelihood, under the simplifying assumption that the bin fluctuations are Poisson-distributed. Weighting methods such as the Bohm-Zech transformation, which uses an effective Poisson count neff=(∑Ei)2/∑Ei2n_{\mathrm{eff}} = (\sum E_i)^2/\sum E_i^2, were considered. However, in low-occupancy bins where only a couple of high-energy hits dominate, neffn_{\mathrm{eff}} becomes small, and the fit becomes unstable. Low-energy bins must nonetheless be included due to the limited number of bins available. However, the unweighted Poisson assumption is incorrect and underestimates the uncertainty in each bin, resulting in cases where reduced-χ2\chi^{2} can be less than 1. Therefore, to interpret the results, the Wasserstein-1 (W1W_1) statistic is used to assess the fit between the empirical cumulative distribution function (ECDF) of the data and the binned CDF. The Wasserstein-1 statistic, or ’Earth-Mover’s Distance’, is used to describe the fit quality. The statistic indicates the average ’distance’ required to transport the probability mass from one distribution to the other. It is defined as the integral over the absolute distance in probability between two CDFs:

W1=∫−∞∞∣F(x)−G(x)∣(13)W_1 = \int_{-\infty}^{\infty} \lvert F(x) - G(x) \rvert \tag*{(13)}

where FF and GG are two CDFs.

The results are shown in Figure 9. These show that the events failing the cut are generally well described as showering particles with ionizing tracks, corresponding to hadron showers. It is noted that the worst-fitting example shown includes multiple subshowers, which could be fitted with additional Gamma terms.

Examples of the best, median and worst fits for $\pi^-$ events failing the cut with energy greater than 1 GeV

Figure 9. Examples of the best, median and worst fits for π−\pi^- events failing the cut with energy greater than 1 GeV. The left subfigure shows the empirical cumulative distribution function (ECDF) vs. the Wasserstein-1 values, W1W_1, and the percentiles shown on the right in grey text. The green, blue, and red text indicate the best, typical (median), and worst case for the fits up to the 99 % percentile. The green energy-weighted histograms are shown on the right, with each row corresponding to the colored axis label. The yy-axis of the histograms shows the energy deposited per bar index or layer. The red line indicates the fitted profile for that event.

The pion interaction length, λπ\lambda_{\pi}, was then fitted using the zssz_{\mathrm{ss}} value under the assumption of an exponential distribution. The truncated exponential distribution was fitted using Eq. (14):

f(zss;λπ)=e−(zss−zmin⁡)λπλπ(1−e−(zmax⁡−zmin⁡)λπ),zmin⁡≤zss≤zmax⁡(14)f(z_{\mathrm{ss}};\lambda_{\pi}) = \frac{e^{-\frac{(z_{\mathrm{ss}}-z_{\min})}{\lambda_{\pi}}}}{\lambda_{\pi}\left(1-e^{-\frac{(z_{\max}-z_{\min})}{\lambda_{\pi}}}\right)}, \quad z_{\min} \le z_{\mathrm{ss}} \le z_{\max} \tag*{(14)}

where zmin⁡=2z_{\min}=2 layer and zmax⁡=10z_{\max}=10 layer.

The reason for the truncation and restricted fit range was that a skewed subset of pathological fits was observed outside this range. Failed fits were observed in the range zss<2z_{\mathrm{ss}}<2, where the MIP and shower components become nearly degenerate. In these cases, the Gamma shower profile absorbed the MIP track. Additionally, cases with multiple subshowers were also observed, as shown in Figure 9. In the range zss>10z_{\mathrm{ss}}>10, only three layers remain, which were observed to be insufficient to fit the showering component.

The results are shown in Figure 10. The fitted pion interaction length was found to be 26.8±1.926.8 \pm1.9 cm. This matches the value quoted by the PDG of 26.72 cm from Refs. [16, 17]. It may therefore be argued that the showering particles are well described according to expectations of showering π−\pi^- events.

Fit of the pion interaction length from the fitted shower start positions of $\pi^{-}$ events that fail the cut

Figure 10. Fit of the pion interaction length from the fitted shower start positions of π−\pi^{-} events that fail the cut. The green and grey shaded areas represent the fitted and excluded regions, respectively. The red line indicates the truncated exponential fit. Fit values and reduced χ2\chi^{2} are shown in the legend.

SHapley Additive exPlanations (SHAP) [26] is a model-agnostic technique for interpreting machine learning models, rooted in game-theoretic Shapley values [27]. This method quantifies the significance of each input variable to the model’s decision-making process by measuring the average marginal effect of including that variable across all subsets of the other features. This can be achieved in practice by replacing the variable with random values from a background distribution, which approximates ’removing’ that feature from the model and is similar in spirit to the ’leave-one-out’ method of jackknife re-sampling. and studying the deviation of the output of the model compared to its average output.

The graph neural network classifier output, given the μ+\mu^+ prototype, can be studied using the Captum library with the ShapleyValueSampling method [28, 29]. The result is a ’SHAP value’: a positive value promotes the selection of that class, and a negative value discourages it. This can be studied to show how the model makes a decision based on the input.

Event displays and their corresponding SHAP values per bar are shown in Figure 11. It can be observed that negative inference bars typically have higher energies than those bars with positive inference, without an observably strong correlation to the spatial position. It was predicted that the optimized cut worked primarily to classify using the bar-by-bar energy distribution rather than its distribution in space.

Example two-dimensional event displays of $\mu^{+}$ and $\pi^{-}$ with corresponding SHAP values

Figure 11. Example two-dimensional event displays of μ+\mu^{+} and π−\pi^{-} with corresponding SHAP values. The top/second row (μ+\mu^{+}) and the third/bottom row (π−\pi^{-}) show the example events at the extremes of the cut (top and bottom row) and the closest examples to the cut boundary (second and third row). The left column shows the event displays, with blue to red color indicating low to high energy deposited in each bar on a log scale. The projection value on the prototype and the energy sum are shown in the title. The right column shows the corresponding SHAP values for each active bar, with blue to red color indicating negative to positive promotion of the μ+\mu^{+} class by that bar. In both columns, the xx and yy correspond to the lateral index and longitudinal layer, respectively.

To assess this hypothesis, the joint distribution of the average SHAP value per bar was plotted against the expectation value of the log-likelihood of the energy coming from the Landau-Gaussian distribution fitted in Figure 8, with fitted parameters for the real μ+\mu^{+} data, E(ln⁡P(Ebar∣LandauGaus(x;μ,σ,MPV,ξ)))\mathbb{E}(\ln P(E_{\mathrm{bar}} \mid\mathrm{LandauGaus}(x;\mu,\sigma,\mathrm{MPV},\xi))). The SHAP value and mean log probability are calculated for each event.

The mean SHAP value is a measure of the total vote for the MIP-like class given every bar in the event. If the measured probability can act as a sufficiently accurate classifier that performs similarly to the unsupervised cut, it may be inferred that the neural network has learned this property as the main discriminating feature between the two classes. Therefore, a 1D linear discriminant analysis (i.e a single threshold value) was fitted to maximally separate the classes using the expected log-likelihood statistic. The AUC of the ROC curve obtained from this classifier indicates its similarity to the cut that was optimized by the Siamese neural network.

The results are shown in Figure 12. The probability density clusters of events passing and failing the cut are clearly separated in both the expectation value of the SHAP value and the expectation value of the log-likelihood of the energy deposited given the fitted Landau-Gaussian distribution. The linear discriminant classifier using the expected value of the log-likelihood per event achieves an AUC between MIP-like and showering hadron-like, as defined by the unsupervised classifier, of 0.927. This study illustrates that the tag can generally be understood from physical expectations that on average, the

Joint distribution of average SHAP value and average log-likelihood for bar energy

Figure 12. Joint distribution of the average SHAP value of a bar per event and the average log-likelihood of the bar energy coming from a Landau-Gaussian distribution. The green and red kernel density estimates (KDE) show the smoothed probability distributions of the π−\pi^{-} events passing and failing the cuts, respectively. The blue line contours indicate the distribution of the μ+\mu^{+} events passing the cut. The dashed line indicates the 1D linear discriminator between the two classes of π−\pi^{-} events. The purple region indicates the decision acceptance boundary for MIP-like events.

Finally, the fraction of events accepted in the simulation by the cut is shown in Figure 13. For the π−\pi^{-} sample, the fraction is compared to the punch-through fraction, defined as the median number of layers per event for which the median log-likelihood of the energy deposited per layer being drawn from the Landau distribution is greater than -15. The plot shows that the fraction passing the cut is close to 1.0 for μ+\mu^{+} and at energies above around 2 GeV, close to the expected punch-through probability of 0.35. Below 1 GeV, the fraction starts to increase. The increase is observed most strongly where π−\pi^{-} are observed to begin to range out in the calorimeter. Appendix Figure 22 compares the Bethe-Bloch energy loss for π−\pi^{-} and μ+\mu^{+} for different initial particle energies. It shows that for π−\pi^{-}, the Bragg region occurs at greater incident momentum than for μ+\mu^{+} due to the latter’s higher mass.

Punch-through fraction and median MIP-like layers per event

Figure 13. Punch-through fraction and median MIP-like layers per event with the log-likelihood of the energy being greater than -15. The blue and red lines show the fractions of the μ+\mu^{+} and π−\pi^{-} data passing the cut. The dashed red line indicates the expected average punch-through pion fraction. The dashed black and green lines indicate the number of MIP-like layers for μ+\mu^{+} and π−\pi^{-}, respectively.

Estimated ionizing energy loss in BGO as a function of average final momentum

Figure 22. Estimated ionizing energy loss in BGO as a function of average final momentum, estimated from average total reconstructed energy, for μ+\mu^{+} and misidentified π−\pi^{-} particles, indicated by blue dashed lines and circle markers, and the misclassified π−\pi^{-} as red dashed lines and triangle markers. Values for the ZA\frac{Z}{A} ratio, density, and ionization potential for BGO were taken from Ref. [31]. The density effect was not included.

In summary, the unsupervised method is demonstrated to be able to perform successfully as a classifier for μ+\mu^{+} and e−e^{-} events without labels, and similarly distinguish punch-through from showering π−\pi^{-} by comparison with μ+\mu^{+}. The method allows the re-derivation of an approximately equivalent cut using the expectation values of the log-likelihood of

the Landau-Gauss distribution over all hits. This cut was identified through SHAP analysis. The method assumed no prior knowledge of the events before classification.

Supervised PID

The confusion matrices, ROC curves, and AUCs as a function of particle energy for the supervised models to separate ’electron’, ’showering hadron’, and ’MIP-like’ event classes as defined earlier are presented for GravNet and BDT models in Figure 14 and Figure 15, respectively.

Figure 14. Figure 14a shows the confusion matrix for the BDT classifier. Figure 14b shows the ROC curve, where the black, green and red lines correspond to the ROC curve for the electron, showering hadron and MIP-like classification vs. all other classes. The red dashed line indicates the micro-average (overall ROC curve for all classes against all other classes). The legend shows the AUC for each case. Figure 14c shows the performance of the classifier in terms of the AUC of one class vs all others (one vs. rest) and micro-average as a function of the particle energy, with the same colour scheme as Figure 14c.

GravNet Confusion Matrix (Validation Set): true labels Electron, Showering hadron-like, MIP-like; predicted labels Electron, Showering hadron-like, MIP-like; matrix values 27184, 681, 135; 2029, 24967, 171; 69, 307, 28457; colorbar labelled Counts

(a)

GravNet Multi-class ROC Curves (One-vs-Rest): axes True Positive Rate and False Positive Rate; legend Electron (AUC = 0.994), Showering hadron-like (AUC = 0.993), MIP-like (AUC = 0.999), Micro-average (AUC = 0.996)

(b)

GravNet AUC vs Beam Energy: axes AUC (One-vs-Rest) and Beam Energy; legend Electron, Showering hadron-like, MIP-like, Micro-average

(c)

Figure 15. Performance metrics of the classification of electron, showering hadron and MIP-like events for the GravNet model. Otherwise, the figures correspond to the descriptions in Figure 14.

In both cases, a total micro-average AUC, defined as the AUC comparing the performance across all classes, was found to be 0.996. This indicates highly effective clustering for both models. The fact that both models achieved this result independently suggests that the calorimeter captures sufficient energy density information that the event classes can be trivially identified with high accuracy using the crystal ECAL prototype.

The comparison of Figure 14a and Figure 15a reveals that the classification performance varies between model types. The BDT model is more accurate at determining electrons but is less accurate at distinguishing MIP-like particles from showering hadrons. The GravNet model is more accurate at identifying MIP-like particles but less accurate at distinguishing showering hadrons from electrons. In summary, distinguishing showering hadrons is the most challenging task for either model. This is also shown in Figure 14a, where the one vs. all AUC scores for electrons, showering hadrons, and MIP-like particles are [0.999, 0.994, 0.997] for the BDT model and [0.994, 0.993, 0.999] for the GravNet model, respectively. This result can be understood as a consequence of the model design. Graph neural networks are effective at describing sparsity between nodes, aiding identification of the regular structure of MIP-like events. By contrast, a BDT uses aggregated event information, which is more informative for events where there are many individual energy deposits such as electrons. It is noted that an ensemble using both methods would likely improve the performance overall, though this is left to a future study.

The AUC of the classifier as a function of particle energy is shown in Figure 14c and Figure 15c. For particle energies below 2 GeV, both models degrade in performance, and above 2 GeV they saturate at close to 1.0. For the BDT model, performance sharply degrades as the model approaches 0.1 GeV, whereas for the GravNet model, the performance in this region is worse overall but more consistent in the very low energy regime.

The boosted decision tree enables a detailed exploration of how the classifier performs and indicates what information is useful for the model in a particular context. The model’s performance can be related back to the physics of particle showers.

A SHAP analysis was performed using the TreeExplainer in the eponymous SHAP software library [30], which can compute exact SHAP values efficiently for tree-based models. A ’beeswarm’ plot of the BDT SHAP-values with ranked variables produced from the TreeExplainer is shown for each class in Figure 16. The plot can be interpreted as follows:

BDT SHAP Summary

Figure 16. SHAP ’beeswarm’ plot for the BDT model, showing the relationship between the input variable value and the influence of the variable on the model prediction. Each plot, left to right, shows the electron, showering hadron, and MIP-like event classes. In each case, the variables are ranked from top to bottom according to their influence on the model’s predictions. A negative to positive SHAP value indicates negative to positive influence of the variable on the model’s final decision. The blue-to-pink color scheme indicates the scale of the parameter, from the 5th to the 95th percentile (small-to-large).

  • Electrons are selected using a small longitudinal center of gravity and average distance from the lateral center of gravity, little energy deposited in the final layer of the calorimeter, and a broad skewness and spread of the central log ratio of the bar energies within the particle shower. The model therefore selects energy-compact, shallow particle showers that shower immediately to be electrons;

  • Showering hadron-like events are selected using a large average distance from the center of gravity, a small amount of energy deposited in the first and second calorimeter layer, a broad skewness and spread of the central log ratio of the bar energies within the particle shower, and a heavy-tailed CLR of the shower energies when compared to MIP-like particles from the kurtosis. The model therefore selects energy-sparse events that do not always shower immediately after entering the calorimeter;

  • MIP-like events are selected using a small variance, skewness and kurtosis of the central-log ratio, a small distance to the layer centre of gravity and a comparatively large fraction of total energy deposited in the last layer, and small fractions of energy deposited in every other layer. The model therefore selects energies that are self-similar and tightly laterally distributed, with a relatively small amount of energy deposited per layer.

It is shown that the variable correlations match physical expectations: electrons are selected as longitudinally short, laterally energy-dense showers starting immediately as they enter the calorimeter; hadrons are selected as longitudinally longer, laterally broad, energy-sparse showers expected to shower after some distance traveling through the calorimeter, and MIP-like events are very tightly distributed, reach the back of the calorimeter, and have a longitudinal center-of-gravity near the center of the calorimeter. These are all in line with physical expectations.

The BDT PID was superior to the graph neural network at selecting electrons. It was therefore applied to test beam data for the 5 GeV to 10 GeV samples, for which contamination was most severe during the test beam. The reconstructed energy by the calorimeter is shown in Figure 17. The energy distributions of electrons are expected and match the expected

2025 June Testbeam Energy Distributions

Figure 17. Energy sum distributions for the 2025 Testbeam data, separated by the PID. The 5 GeV to 7 GeV and 8 GeV to 10 GeV are shown on the top and bottom rows, respectively. The black, green, and red lines show the electron, showering hadron-like, and MIP-like events, respectively. The fraction of events in the dataset is shown by the color-coded pie chart in the top left corner. The expected electron particle energies are shown as the purple dashed lines.

skewed Gaussian core with leakage. The sharp MIP peak shown in green indicates MIP-like events deposit roughly the same amount of energy per event. The broad continuum of energies measured for the red showering hadron-like events can be understood as a result of the partial containment and random shower start depth, which smears out the energy. The pie charts show the fraction of electrons. At beam energies greater than 6 GeV, the fraction of electrons is only 3 % to 6 % and decreases with energy, illustrating the importance of PID for the analysis of this experimental data.

In summary, the supervised PID method has learned a highly effective event discrimination technique that aligns with physical expectations of interactions observed by the calorimeter.

Conclusion

Two machine learning methods were developed for PID in a highly granular crystal ECAL prototype to assess the performance of the detector in classifying basic types of events.

Firstly, one unsupervised, contrastive method was developed for separating MIP-like punch-through π−\pi^{-} events from showering pions using a graph neural network model with triplet loss. The classification capacity was found to be nearly perfect when classifying μ+\mu^{+} from e−e^{-}, and when applied to μ+\mu^{+} from π−\pi^{-}, it was demonstrated to select events that were in good agreement with the expectations of a MIP-like particle and showering hadron-like events. A SHAP study was used to summarize that the learned cut could be approximated using the expectation value of the log-likelihood of a Landau-Gaussian distribution over the hits in the calorimeter. This method therefore allowed the development of a bias-free cut that could be readily interpreted.

Secondly, a supervised classifier using a graph neural network and a boosted decision tree was implemented to classify electrons, showering hadrons, and MIP-like events. Both classifiers achieved high accuracy using independent methods. Misclassification of showering hadrons was the most common of all the classes. The performance in both cases was comparable. The boosted decision tree was studied using the SHAP method and found to agree with expectations on the behavior of each event type. The BDT was applied to data and recovered the expected distributions of energy of each event type.

In summary, this study illustrates the potential for future calorimeters to achieve excellent PID performance even when the granularity is relatively coarse. It motivates further research into broader PID, such as whether or not a wider range of particles (e.g., photons, kaons, pions, protons/neutrons) can be identified by their energy deposits in a highly granular HCAL + ECAL system. It is noted as a caveat that photons were not included in this study, as its primary purpose is to purify experimental test-beam data, in which photons are unlikely to be observed. Furthermore, the effects of PID on Particle Flow reconstruction could be explored, with the goal of improving track-cluster association and therefore jet energy resolution in future Higgs factory experiments. Additionally, the unsupervised method presented can be applied beyond calorimetry to the discrimination of signal and background.

Acknowledgements

We gratefully acknowledge the CERN management for its support and hospitality and the accelerator staff for reliable and efficient operation of the test beam. This research was supported by the National Key Research and Development Program of China (Grant Number 2023YFA1606300).

Non-Standard Probability Density Functions

The double-sided Crystal Ball function (DSCB) is a piecewise distribution described by Eq. 14a:

DSCB⁡(x;μ,σ,αL,nL,αR,nR)={AL⋅(BL−x−μσ)−nL,x−μσ<−αL,exp⁡(−(x−μ)22σ2),−αL≤x−μσ≤αR,AR⋅(BR−x−μσ)−nR,x−μσ>αR.(15)\operatorname{DSCB}(x;\mu,\sigma,\alpha_L,n_L,\alpha_R,n_R)= \begin{cases} A_L \cdot\left(B_L-\frac{x-\mu}{\sigma}\right)^{-n_L}, & \frac{x-\mu}{\sigma}<-\alpha_L,\\ \exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right), & -\alpha_L\le\frac{x-\mu}{\sigma}\le\alpha_R,\\ A_R \cdot\left(B_R-\frac{x-\mu}{\sigma}\right)^{-n_R}, & \frac{x-\mu}{\sigma}>\alpha_R. \end{cases} \tag*{(15)}
AL/R=(nL/R∣αL/R∣)nL/R⋅exp⁡(−∣αL/R∣22)(16)A_{L/R}=\left(\frac{n_{L/R}}{|\alpha_{L/R}|}\right)^{n_{L/R}}\cdot\exp\left(-\frac{|\alpha_{L/R}|^2}{2}\right) \tag*{(16)}
BL/R=nL/R∣αL/R∣−∣αL/R∣(17)B_{L/R}=\frac{n_{L/R}}{|\alpha_{L/R}|}-|\alpha_{L/R}| \tag*{(17)}

where μ\mu and σ\sigma represent a normally-distributed core region bounded by αL/R\alpha_{L/R} and with power law tails defined by nL/Rn_{L/R}. This distribution is used to fit unknown distributions with a core fraction that is normally distributed with heavy or asymmetrical tails. It is used in this study to fit heavy-tailed distributions with a predominantly Gaussian core without requiring selection cuts, and to estimate the valid normal regions of those distributions.

Supplementary Figures and Tables

0.1 GeV to 0.8 GeV
EbeamE_{\mathrm{beam}} [GeV]ParticleEvent Class0.10.20.30.40.50.60.70.8
μ+\mu^{+}MIP-like62193107287118085123768125756127980129136130083
e−e^{-}Electron2717736169393284055241760423324278643090
π−\pi^{-}MIP-like56768227128711921420674211292128220397
Shr. Hadron995524769242122018620329203462108721777
0.9 GeV to 4 GeV
EbeamE_{\mathrm{beam}} [GeV]ParticleEvent Class0.911.522.533.54
μ+\mu^{+}MIP-like130215131409132617133560134033134449134781134662
e−e^{-}Electron4343743682440664462644458447654466544971
π−\pi^{-}MIP-like2022419633198781942918735181741747016850
Shr. Hadron2248223007235682416125047259242692927326
4.5 GeV to 8 GeV
EbeamE_{\mathrm{beam}} [GeV]ParticleEvent Class4.555.566.577.58
μ+\mu^{+}MIP-like134683134628135395135345135869135421135527135240
e−e^{-}Electron4506744920450374493944994448384497044943
π−\pi^{-}MIP-like1638416100161631569115896154621543815732
Shr. Hadron2787228073282732865228455288052891728593
8.5 GeV to 10 GeV
EbeamE_{\mathrm{beam}} [GeV]ParticleEvent Class8.599.510
μ+\mu^{+}MIP-like135841135378135819135382
e−e^{-}Electron45033452664475744986
π−\pi^{-}MIP-like15773158411563215890
Shr. Hadron28715290192922828853

Table 2. Counts of unique simulated events by particle, event class, and beam energy in simulation.

EbeamE_{\mathrm{beam}} [GeV]5678910
Electron270,092161,45536,53727,28221,04816,334
Shr. Hadron354,33022,120275,401280,530285,193287,792
MIP-like248,6778,873160,673164,181164,086165,511

Table 3. Event-class breakdown of the 2025 June Testbeam data in the range of electron energies from 5 GeV to 10 GeV

Example fits of the Landau-Gaussian distribution to the hit energy distribution

Figure 21. Example fits of the Landau-Gaussian distribution to the hit energy distribution in the first layer of the calorimeter. The light blue and red lines indicate the distributions of the total energy deposited in a bar in the first calorimeter layer, for μ+\mu^{+} and misidentified π−\pi^{-}, respectively. The dark blue and dark red lines indicate fits of the Landau-Gaussian distribution from Eq. (9), with the same color coding. Figure 21a–Figure 21b show the 0.4 GeV, 1 GeV, 5 GeV and 10 GeV datasets.

References

  1. [1]The CEPC Study Group. CEPC Technical Design Report – Reference Detector. arXiv:2510.05260 [hep-ex]. Oct. 2025. DOI: 10.48550/arXiv.2510.05260. URL: http://arxiv.org/abs/2510.05260 (visited on 09/02/2026).
  2. [2]M. Benedikt et al. “Future Circular Collider Feasibility Study Report: Volume 2 Accelerators, Technical Infrastructure and Safety”. en. In: The European Physical Journal Special Topics 234.19 (Dec. 2025), pp. 5713–6197. ISSN: 1951-6355, 1951-6401. DOI: 10.1140/epjs/s11734-025-01967-4. URL: https://link.springer.com/10.1140/epjs/s11734-025-01967-4 (visited on 11/03/2026).DOI
  3. [3]Yuexin Wang et al. “Prospects for B⁰ₛ → π⁰π⁰ and B⁰ₛ → ηη modes and corresponding CP asymmetries at Tera-Z”. en. In: Journal of High Energy Physics 2022.12 (Dec. 2022), p. 135. ISSN: 1029-8479. DOI: 10.1007/JHEP12(2022)135. URL: https://link.springer.com/10.1007/JHEP12(2022)135 (visited on 05/08/2026).DOI
  4. [4]S. Agostinelli et al. “Geant4—a simulation toolkit”. In: Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 506.3 (July 2003), pp. 250–303. ISSN: 0168-9002. DOI: 10.1016/S0168-9002(03)01368-8. URL: https://www.sciencedirect.com/science/article/pii/S0168900203013688 (visited on 09/02/2026).
  5. [5]William S. Cleveland and Catherine Loader. “Smoothing by Local Regression: Principles and Methods”. In: Statistical Theory and Computational Aspects of Smoothing. Ed. by Wolfgang Härdle and Michael G. Schimek. Heidelberg: Physica-Verlag HD, 1996, pp. 10–49. ISBN: 978-3-642-48425-4.DOI
  6. [6]Josef Perktold et al. statsmodels/statsmodels: Release 0.14.2. Apr. 2024. DOI: 10.5281/ZENODO.10984387. URL: https://zenodo.org/doi/10.5281/zenodo.10984387 (visited on 11/05/2026).
  7. [7]Shah Rukh Qasim et al. “Learning representations of irregular particle-detector geometry with distance-weighted graph networks”. en. In: The European Physical Journal C 79.7 (July 2019), p. 608. ISSN: 1434-6052. DOI: 10.1140/epjc/s10052-019-7113-9. URL: https://doi.org/10.1140/epjc/s10052-019-7113-9 (visited on 01/23/2026).
  8. [8]Matthias Fey and Jan Eric Lenssen. Fast Graph Representation Learning with PyTorch Geometric. Version Number: 3. 2019. DOI: 10.48550/ARXIV.1903.02428. URL: https://arxiv.org/abs/1903.02428 (visited on 03/07/2026).
  9. [9]Adam Paszke et al. PyTorch: An Imperative Style, High-Performance Deep Learning Library. arXiv:1912.01703 [cs]. Dec. 2019. DOI: 10.48550/arXiv.1912.01703. URL: http://arxiv.org/abs/1912.01703 (visited on 03/07/2026).
  10. [10]Jerome H. Friedman. “Greedy function approximation: A gradient boosting machine.” In: The Annals of Statistics 29.5 (Oct. 2001), pp. 1189–1232. ISSN: 0090-5364, 2168-8966. DOI: 10.1214/aos/1013203451. URL: https://projecteuclid.org/journals/annals-of-statistics/volume-29/issue-5/Greedy-function-approximation-A-gradient-boosting-machine/10.1214/aos/1013203451.full (visited on 01/23/2026).DOI
  11. [11]Tianqi Chen and Carlos Guestrin. “XGBoost: A Scalable Tree Boosting System”. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. arXiv:1603.02754 [cs]. Aug. 2016, pp. 785–794. DOI: 10.1145/2939672.2939785. URL: http://arxiv.org/abs/1603.02754 (visited on 03/07/2026).
  12. [12]Vassilios Balntas et al. “Learning local feature descriptors with triplets and shallow convolutional neural networks”. en. In: Proceedings of the British Machine Vision Conference 2016. York, UK: British Machine Vision Association, 2016, pp. 119.1–119.11. ISBN: 978-1-901725-59-9. DOI: 10.5244/C.30.119. URL: http://www.bmva.org/bmvc/2016/papers/paper119/index.html (visited on 01/16/2026).DOI
  13. [13]Leslie N. Smith. “Cyclical Learning Rates for Training Neural Networks”. In: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV). Mar. 2017, pp. 464–472. DOI: 10.1109/WACV.2017.58. URL: https://ieeexplore.ieee.org/document/7926641 (visited on 02/24/2026).DOI
  14. [14]torch-lr-finder: Pytorch implementation of the learning rate range test. URL: https://github.com/davidtvs/pytorch-lr-finder (visited on 02/24/2026).
  15. [15]Diederik P. Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. arXiv:1412.6980 [cs.LG]. Jan. 2017. DOI: 10.48550/arXiv.1412.6980. URL: http://arxiv.org/abs/1412.6980 (visited on 11/16/2026).
  16. [16]F. Takahashi et al. “Review of Particle Physics”. In: Int. J. Mod. Phys. A 41 (2026), p. 2630011. DOI: 10.1142/S0217751X26300115.DOI
  17. [17]Atomic and nuclear properties of bismuth germanate (BGO) [(Bi2O3)2(GeO2)3]. URL: https://pdg.lbl.gov/2026/AtomicNuclearProperties/HTML/bismuth_germanate_BGO.html (visited on 07/07/2026).
  18. [18]Takuya Akiba et al. Optuna: A Next-generation Hyperparameter Optimization Framework. arXiv:1907.10902 [cs]. July 2019. DOI: 10.48550/arXiv.1907.10902. URL: http://arxiv.org/abs/1907.10902 (visited on 03/07/2026).
  19. [19]Takuma Yoshimura. Numerical Evaluation and High Precision Approximation Formula for Landau Distribution. July 2024. DOI: 10.36227/techrxiv.171822215.53612870/v2. URL: https://www.techrxiv.org/users/661998/articles/1085065-numerical-evaluation-and-high-precision-approximation-formula-for-landau-distribution?commit=177a98d2f93eea43ff2e16176ed9c3d5c48ed244 (visited on 03/05/2026).DOI
  20. [20]Hans Dembinski et al. scikit-hep/iminuit. Nov. 2025. DOI: 10.5281/zenodo.17565861. URL: https://zenodo.org/records/17565861 (visited on 03/02/2026).
  21. [21]R.K. Bock, T. Hansl-Kozanecka, and T.P. Shah. “Parametrization of the longitudinal development of hadronic showers in sampling calorimeters”. en. In: Nuclear Instruments and Methods in Physics Research 186.3 (July 1981), pp. 533–539. ISSN: 01675087. DOI: 10.1016/0029-554X(81)90232-9. URL: https://linkinghub.elsevier.com/retrieve/pii/0029554X81902329 (visited on 06/12/2026).DOI
  22. [22]Egidio Longo and Ignazio Sestili. “Monte Carlo calculation of photon-initiated electromagnetic showers in lead glass”. In: Nuclear Instruments and Methods 128.2 (Oct. 1975), pp. 283–307. ISSN: 0029-554X. DOI: 10.1016/0029-554X(75)90679-5. URL: https://www.sciencedirect.com/science/article/pii/0029554X75906795 (visited on 09/01/2026).DOI
  23. [23]M Chadeeva. “Parametrisation of hadron shower profiles in the CALICE Sc-Fe AHCAL”. In: (2015). URL: https://cds.cern.ch/record/2000272.
  24. [24]G. Grindhammer and S. Peters. The Parameterized Simulation of Electromagnetic Showers in Homogeneous and Sampling Calorimeters. Version Number: 1. 2000. DOI: 10.48550/ARXIV.HEP-EX/0001020. URL: https://arxiv.org/abs/hep-ex/0001020 (visited on 05/22/2026).
  25. [25]Olin Lyod Pinto. “Shower Shapes in a Highly Granular SiPM-on-Tile Analog Hadron Calorimeter”. en. Accepted: 2022-11-03T12:50:01Z. doctoralThesis. Staats- und Universitätsbibliothek Hamburg Carl von Ossietzky, 2022. URL: https://ediss.sub.uni-hamburg.de/handle/ediss/9855 (visited on 06/12/2026).DOI
  26. [26]Scott M. Lundberg et al. “From local explanations to global understanding with explainable AI for trees”. en. In: Nature Machine Intelligence 2.1 (Jan. 2020), pp. 56–67. ISSN: 2522-5839. DOI: 10.1038/s42256-019-0138-9. URL: https://www.nature.com/articles/s42256-019-0138-9 (visited on 03/07/2026).DOI
  27. [27]L.S. Shapley. Notes on the N-Person Game — II: The Value of an N-Person Game. en. ASTIA Document No. ATI 210720. Rand Corporation, Aug. 1951. URL: https://www.rand.org/pubs/research_memoranda/RM0670.html (visited on 03/07/2026).
  28. [28]Narine Kokhlikyan et al. Captum: A unified and generic model interpretability library for PyTorch. arXiv:2009.07896 [cs.LG]. Sept. 2020. DOI: 10.48550/arXiv.2009.07896. URL: http://arxiv.org/abs/2009.07896 (visited on 10/25/2026).
  29. [29]meta-pytorch/captum. original-date: 2019-08-27T15:34:41Z. Oct. 2026. URL: https://github.com/meta-pytorch/captum (visited on 10/25/2026).
  30. [30]shap/shap. original-date: 2016-11-22T19:17:08Z. Mar. 2026. URL: https://github.com/shap/shap (visited on 03/07/2026).
  31. [31]J. A. Bakken et al. “Ionization loss in BGO”. In: Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 270.2 (July 1988), pp. 397–402. ISSN: 0168-9002. DOI: 10.1016/0168-9002(88)90706-1. URL: https://www.sciencedirect.com/science/article/pii/0168900288907061 (visited on 03/06/2026).DOI

Paper details

vlnc:W_6s5tbr2r
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 5

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents