betaFeedback

WorksPhysics and Astronomy

Deep Generative Models for Ultra-High Granularity Particle Physics Detector Simulation: A Voyage From Emulation to Extrapolation

Abstract

Simulating ultra-high-granularity detector responses in Particle Physics represents a critical yet computationally demanding task. This thesis aims to overcome this challenge for the Pixel Vertex Detector (PXD) at the Belle II experiment, which features over 7.5M pixel channels-the highest spatial resolution detector simulation dataset ever analysed with generative models. This thesis starts off by a comprehensive and taxonomic review on generative models for simulating detector signatures. Then, it presents the Intra-Event Aware Generative Adversarial Network (IEA-GAN), a new geometry-aware generative model that introduces a relational attentive reasoning and Self-Supervised Learning to approximate an "event" in the detector. This study underscores the importance of intra-event correlation for downstream physics analyses. Building upon this, the work drifts towards a more generic approach and presents YonedaVAE, a Category Theory-inspired generative model that tackles the open problem of Out-of-Distribution (OOD) simulation. YonedaVAE introduces a learnable Yoneda embedding to capture the entirety of an event based on its sensor relationships, formulating a Category theoretical language for intra-event relational reasoning. This is complemented by introducing a Self-Supervised learnable prior for VAEs and an Adaptive Top-q sampling mechanism, enabling the model to sample point clouds with variable intra-category cardinality in a zero-shot manner. Variable Intra-event cardinality has not been approached before and is vital for simulating irregular detector geometries. Trained on an early experiment data, YonedaVAE can reach a reasonable OOD simulation precision of a later experiment with almost double luminosity. This study introduces, for the first time, the results of using deep generative models for ultra-high granularity detector simulation in Particle Physics.

München 2023

Deep Generative Models for Ultra-High Granularity Particle Physics Detector Simulation: A Voyage From Emulation to Extrapolation / Tiefe Generative Modelle für die Simulation von Teilchenphysik-Detektoren mit ultrahoher Granularität: Eine Reise von der Emulation zur Extrapolation

Baran (Hosein) Hashemi

Dissertation

der Fakultät für Physik

der Ludwig-Maximilians-Universität München

vorgelegt von

Baran (Hosein) Hashemi

aus Rasht, Iran

Erstgutachter: Prof. Dr. Thomas Kuhr Zweitgutachter: Prof. Dr. Lukas Heinrich Tag der mündlichen Prüfung: 22.11.2023

Zusammenfassung

Die Simulation von Detektorantworten mit ultrahoher Granularität in der Teilchenphysik stellt eine kritische, jedoch rechenintensive Aufgabe dar. Diese Arbeit zielt darauf ab, diese Herausforderungen zu bewältigen, indem sie sich auf den Pixel-Vertex-Detektor (PXD) im Belle II-Experiment konzentriert, der über 7,5 Millionen Pixelkanäle verfügt – den höchstaufgelösten Detektorsimulationsdatensatz, der jemals mit tiefen generativen Modellen analysiert wurde. Als erste Beitrag führe ich das Intra-Event-Aware Generative Adversarial Network (IEA-GAN) ein, ein Modell, das relationales Denken und selbstüberwachtes Lernen integriert, um ein „Ereignis“ im Detektor zu simulieren. Diese Studie etabliert PXD-Daten als feinkörnigen Datensatz und unterstreicht die Bedeutung der intra-eventuellen Korrelation für nachgelagerte physikalische Analysen. IEA-GAN emuliert PXD-Signaturen auf eine geometriebewusste und korrelierte Weise.

Aufbauend darauf führt diese Arbeit YonedaVAE ein, ein fortschrittliches generatives Modell, das das offene Problem der Out-of-Distribution (OOD) Simulation von realen Daten angeht. Inspiriert durch die Kategorientheorie verwendet YonedaVAE ein lernbares Yoneda-Embedding, um die Gesamtheit eines Ereignisses anhand seiner Sensorbeziehungen zu erfassen, und formuliert eine formale Sprache für intra-eventuelles relationales Denken. Dies wird ergänzt durch einen selbstdestillierten Mengengenerator und einen adaptiven Top-q Abtastmechanismus, der es dem Modell ermöglicht, Punktwolken mit variablen intra- und inter-eventuellen Kardinalitäten weit über die Kardinalität der Trainingsdaten hinaus zu sampeln. Eine variable intra-event Kardinalität wurde zuvor noch nicht erreicht und ist von entscheidender Bedeutung, wenn man sich mit unregelmäßigen Detektorgeometrien und Treffermustern durch den Detektor beschäftigt.

Diese Studie präsentiert die ersten Ergebnisse für ein generatives Modell, das auf realen Daten in der Teilchenphysik mit ultrahoher Granularität trainiert wurde. Sie zeigt, dass das YonedaVAE-Modell, trainiert auf frühzeitigen Hintergrunddaten eines Experiments, eine vernünftige Simulation eines späteren Experiments mit fast doppelter Leuchtkraft erreichen kann, während es gleichzeitig eine signifikante Speicherentlastung bietet. Bemerkenswert ist, dass YonedaVAE diese Extrapolation ohne vorherige Exposition gegenüber Hochleuchtkraftdaten erreicht, was seine Robustheit und Generalisierbarkeit unterstreicht. Insgesamt reduzieren diese Modelle nicht nur erheblich den Rechenaufwand, sondern erreichen auch eine noch nie dagewesene Präzision in Detektorsimulationen mit ultrahoher Granularität. Dies eröffnet neue Wege sowohl für die Recheneffizienz als auch für die Genauigkeit in der Teilchenphysik.

PXD Background Generation: Real Data

Introduction

Real PXD data and OOD Simulation: Challenges and Motivations

Set Generation:

Length Extrapolation

Context Extrapolation:

Prologue: Category Theory and Yoneda Perspective

What is Category Theory?

Basic Definitions

Functors and Natural Transformations

The Yoneda Perspective

Yoneda Embedding and Yoneda Lemma
Presheaf

YonedaVAE

Problem setup

YonedaVAE: Yoneda Encoder

Eventformer and Attention Gating
Yoneda-Pooling
Yoneda-Pooling and Category Theory

YonedaVAE: Set Generation

Self-Distillation
Adaptive Top-q sampling

YonedaVAE: Decoder

Evaluation Results

Marginal Distributions

FID, KID, and Vendi Score

Topological and Geometrical Clustering for PXD data

Topological Data Analysis
Persistent Homology and filtration
Topological Data Analysis of PXD Background
Geometrical Clustering

Physics Analysis

Ablation Studies

Conclusion

Introduction

The current consensus to model the origin of our universe centers on an event known as the Big Bang [1, 2], which occurred 13.8 billion years ago. This explosive instant was characterized by extreme heat and dense energy, giving rise to elementary particles and their corresponding anti-particles. Mere fractions of a second later, cosmic inflation dramatically expanded the universe. What transpired next remains a fundamental enigma, serving as one of the motivating forces behind the creation of the Belle II experiment [3]. This crucial and enigmatic process, termed “Baryogenesis,” violated baryon number conservation, resulting in the observed predominance of baryons over anti-baryons.

In mainstream cosmology, the Universe underwent an initial inflationary phase, swiftly transitioning to a period of radiation domination. This was followed by an era of matter domination, ultimately giving way to the dark energy-dominated epoch we find ourselves in today [4]. While we can somewhat simulate baryon formation in particle colliders, the puzzle of the universe’s “missing” antimatter remains unsolved. To generate matter and antimatter at differing rates, a baryon-generating interaction must meet three critical conditions [5]: violation of the baryon number, breaking of C-symmetry and CP-symmetry, and a deviation from thermal equilibrium.

The Belle II experiment focuses particularly on CP-symmetry violation, which results in an imbalance between the number of left-handed and right-handed baryons and anti-baryons. This adds another layer of complexity to the baryogenesis conundrum. Current explanations within the Standard Model fail to account for the observed matter-antimatter disparity, signaling the need to delve into unexplored areas of physics. To navigate this uncharted territory, researchers employ two primary approaches. The first, known as the “high energy frontier,” is utilized by the Large Hadron Collider (LHC) and aims to directly create and analyze new particles through high-energy collisions. SuperKEKB (Belle II) takes a different tack and operates on the “intensity frontier,” focusing on high-precision experiments to identify deviations from the Standard Model and search for new, weakly-coupled mediators in the dark sector. Thus, the quest endures, the mysteries dating back to the universe’s earliest moments to today’s cutting-edge experiments, each discovery bringing us closer to understanding the nature of our existence.

The key to high-precision measurements at SuperKEKB is the collider’s ability to produce “clean” events. This enables precise measurements, particularly for events where particles like neutrinos escape undetected. High-precision measurement at Belle II is achieved by recording a large number of collisions to offer a statistically robust sample for analyses. Additionally, the excellent reconstruction of particle trajectories, particularly their decay vertices, is crucial. SuperKEKB has attained an unprecedented instantaneous luminosity of 4.7×1034 cm−2s−14.7 \times10^{34}\ \mathrm{cm}^{-2}\mathrm{s}^{-1} and aims for an even higher rate of 8×1035 cm−2s−18 \times10^{35}\ \mathrm{cm}^{-2}\mathrm{s}^{-1}. Luminosity is a measure of how many particles pass through a given area in a specific time period. It’s essentially a measure of the “brightness” of the collider, and higher luminosity implies that more particle collisions are likely to occur, increasing the odds of observing rare processes.

Discoveries of New Physics at Belle II are heavily dependent on the precise reconstruction of particle decay vertices. To achieve this level of precision, the Belle II detector is equipped with a state-of-the-art PiXel vertex Detector (PXD) located very close to the interaction point. This serves as the innermost sub-detector of Belle II. The PXD, with its “ultra-high granular” matrix of sensors, records the passage of charged particles with a total readout of 7.68×1067.68 \times10^{6} channels per event. However, its position as the innermost sub-detector results in a high level of “background” noise. “background” refers to any detector hit that is not of interest but cannot be fully eliminated from the data. These are usually events that look similar to the desired signal but are caused by different processes. These backgrounds, as artifacts in the PXD, directly influence the precision of particle trajectory finding and reconstruction.

To validate our understanding of theoretical models in physical processes, Belle II, like other particle physics experiments, heavily relies on simulation for various tasks. These include data selection, statistical inference, and design optimization for new experiments. Consequently, emulating the PXD background responses is critical for studying experimental conditions. However, this emulation is both storage-intensive and computationally demanding. The current landscape for PXD background simulation includes three primary approaches: First is the somewhat accurate but storage-consuming and computationally intense “Geant4 simulation.” Second is the highly accurate but storage-intensive “random trigger” method. Lastly, there’s the “surrogate model,” known for its fast and efficient simulation capabilities.

In this thesis, my primary objective is to develop specialized Deep Generative Models (DGM) as surrogates to enhance computational efficiency and data storage for Belle II PXD detector simulation. To address the challenges presented by the ultra-high resolution of PXD data, I introduce a fresh perspective through “event-based relational reasoning.” In an event-based relational reasoning approach, each sensor in an event is studied in relation to the other sensors. The central premise is that understanding each PXD sensor solely based on its intrinsic features is insufficient for accurate simulation. Instead, how each sensor interacts with other sensors within the event provides a contextualized representation, very much like like transitioning from “syntax” to “semantics” in linguistics. Through this thesis, I demonstrate that incorporating this relational inductive bias leads to the creation of more accurate surrogate models for detector simulation. Within the ensuing chapter, Chapter 2, I introduce the Belle II detector and its software framework. Then, there will be an in-depth discussion on the PXD background and various ways of simulating its detector signatures.

Chapter 3 lays the foundation for deep generative models, relational reasoning, and self-supervised learning, the three Machine Learning (ML) pillars of this thesis, by providing an overview of the prerequisite technologies and methodologies.

In Chapter 4, I present an exhaustive and taxonomically organized review of the use of deep generative models in Experimental Particle Physics, with a special focus on detector simulation. By the end of this chapter, I construct a holistic view to demonstrate that current state-of-the-art deep generative models are incomplete for tackling ultra-high granular PXD background generation.

Moving on to Chapter 5, I delve into the specifics of PXD background generation and offer a task-categorization perspective for it. I also introduce two new models that I have developed: the Prior Embedding GAN [6], founded on Contrastive Learning, and the Intra-Event Aware GAN (IEA-GAN) [7], which relies on Self-Supervised Learning and relational reasoning. These models are trained and tested on Geant4-simulated PXD background data. During this process, novel technologies in deep generative models tailored for fine-grained images will be introduced. For evaluation, I initially motivate the significance of “intra-event correlation” for the first time in the fast simulation domain via a study focused on Helix Parameter Resolution. Subsequently, I conduct a comprehensive evaluation of various figures of merit. As a result of this analysis, IEA-GAN has successfully integrated into the basf2 software [8] suite as a surrogate module for emulating PXD background on the fly for analysis.

In Chapter § 3, I engage with the “real” PXD detector background data coming from the random trigger. This chapter explores the challenges and motivations behind the Out-Of-Distribution (OOD) simulation of PXD background data and extrapolation beyond current experimental limits, particularly concerning luminosity. I then seek to unify relational reasoning concepts through the lens of Category Theory, the abstract study of mathematical objects and their interrelations. As a result, I introduce YonedaVAE, a zero-shot point cloud deep generative model capable of generating PXD background hits with an unprecedented cardinality of 100 700 hits per event, despite being trained only on 7600 PXD hits per event from Experiment 12. Remarkably, YonedaVAE performs robustly when tested on data from Experiment 26, which has nearly double the luminosity of Experiment 12. The chapter presents results in two primary tasks: “length extrapolation,” where the model has access to an individual sensor condition, and “context extrapolation,” where it has access only to a global event-level condition during inference and has to solve an Inverse Problem. For evaluation, after an in-depth evaluation of low-level marginal distributions and NN-based metrics, I introduce a new diversity measure for detector simulation, the Vendi Score, previously used in Ecology and Protein Design. Then, I examine the geometrical and topological properties of the generated PXD background data via clustering analyses and Topological Data Analysis (TDA). Finally, I study the tracking analysis of the generated PXD background in comparison to the real PXD detector background data. Consequently,

I demonstrate the efficacy of YonedaVAE, which excels not only in the OOD simulation of PXD background but also effectively manages to condition its output based on both sensor locations and background levels.

In the final chapter, Chapter 7, a summary of the study’s key findings will be presented, along with a thoughtful discussion regarding the limitations. These limitations offer crucial insights for potential refinements in the current methodologies and lay the groundwork for the next phase of investigations in this domain.

Belle II experiment: The PXD Saga

The Belle II experiment, a precision marvel of modern particle physics located at the SuperKEKB accelerator in Tsukuba, Japan, initiated data collection [9] from electron-positron collisions in March 2019. It has achieved an unprecedented instantaneous luminosity, reaching up to 4.7×1034 cm−2s−14.7 \times10^{34}\ \mathrm{cm}^{-2}\mathrm{s}^{-1}, and accumulated data equivalent to 424 fb−1424\ \mathrm{fb}^{-1} [10]. A pivotal component of the Belle II apparatus is its Vertex Detector (VXD), which is instrumental for the precise reconstruction of both primary and secondary decay vertex locations. The VXD is comprised of an outer section with four layers of Silicon Vertex Detector (SVD). The inner section features a two-layer PiXel Detector (PXD), the protagonist of this thesis.

In the ensuing sections of this chapter, I will guide you through the experimental setup of the Belle II experiment and the PXD sub-detector. As we traverse this journey, we will delve into the nuances of the responses recorded by the PXD and their origin. Eventually, I will discuss the simulation of PXD data signatures and its challenges. By the end of the chapter, I discuss the plan for this thesis in the following chapters.

Belle II Experiment

The Belle II experiment [3] represents the next chapter in a rich legacy of collider experiments focused on B meson physics, collectively known as B factories. Situated at KEK in Tsukuba, Japan, Belle II is the successor to the original Belle experiment and operates in conjunction with the SuperKEKB accelerator. This state-of-the-art facility is an asymmetric electron-positron collider featuring a High Energy Ring (HER) for electrons at 7 GeV and a Low Energy Ring (LER) for positrons at 4 GeV. Designed to achieve a peak luminosity of 6.5×1035 cm−2s−16.5 \times10^{35}\ \mathrm{cm}^{-2}\mathrm{s}^{-1}, it aims to outperform its predecessor, KEKB, by a factor of 40.

The Belle II detector employs a Cartesian coordinate system that is right-handed for its spatial description (see Figure 2.1). The origin of this coordinate framework is situated at the point where particle interactions are expected to occur, commonly known as the nominal interaction point (IP). At this point, both beams are crossing at an angle of 11 mrad and collisions occur. The orientation of the axes is established as follows:

The Belle II coordinate system

Figure 2.1. The Belle II coordinate system.

  • zz-axis: This axis is aligned with the magnetic field generated by the solenoid, and

parallel to the e+e^{+} beam. It essentially provides a longitudinal perspective, running parallel to the primary axis of the detector.

  • yy-axis: Oriented in the vertical direction, this axis points towards the ceiling of the detector hall. It serves as the upward vertical reference for the system.

  • xx-axis: This axis extends radially, pointing outward in a direction perpendicular to the plane formed by the zz and yy axes. It is oriented towards the exterior of the accelerator ring.

In this coordinate system, the positive zz-axis is directed towards the “forward” orientation while the negative zz-axis indicates the “backward” direction. Unless stated otherwise, all projections of the Belle II detector onto the xyxy-plane are designed to be viewed from the forward end to the backward end of the detector.

The collision dynamics are carefully controlled to yield a center-of-mass energy of ECMS=4EHERELER≈10.58 GeVE_{\mathrm{CMS}} = \sqrt{4E_{\mathrm{HER}}E_{\mathrm{LER}}} \approx10.58\ \mathrm{GeV}, aligning closely with the mass of Υ(4S)\Upsilon(4S) resonance. In this setup, B and Bˉ\bar{\mathrm{B}} mesons are produced and almost immediately decay, initiating complex chains of subsequent decays. These chains are critical to the study of B meson physics, as they often yield a final set of stable particles whose properties can be thoroughly analyzed.

The asymmetry along the zz-axis in Belle II’s detector design is a response to the unique collision dynamics, where the center-of-mass frame and the flight trajectories of produced B mesons are both directed towards the forward part of the detector. This forward focus introduces a Lorentz boost to the B mesons, effectively increasing their flight length within the detector. Thus, longer flight length yields a better resolution on lifetimes, enabling more precise measurements. This leads to a division of the angular acceptance into three separate regions in terms of polar angles: the Forward, Barrel, and Backward regions. The polar angle θ\theta is the angle in the (x,z)(x,z)-plane with respect to the zz-axis. Specifically, the Forward region encompasses angles 17∘<θ<30∘17^{\circ} < \theta< 30^{\circ}, the Barrel region is defined by 30∘<θ<125∘30^{\circ} < \theta< 125^{\circ}, and the Backward region spans 125∘<θ<150∘125^{\circ} < \theta< 150^{\circ}.

All but the outermost component of the Belle II detector operates within a uniform solenoidal magnetic field of magnitude 1.5 T1.5\ \mathrm{T}, oriented parallel to the detector’s main axis.

A recorded collision, namely an “event”, involves mostly e+e−→Υ(S)→BˉBe^{+}e^{-} \to\Upsilon(S) \to\bar{B}B reaction that typically yields around 12 tracks. The particles most commonly detected in such events are electrons, positrons, photons, muons, π±\pi^{\pm} mesons, K±K^{\pm} mesons, protons, and deuterons.

Belle II is engineered to focus on capturing decay products of B mesons due to their extremely short lifetimes. It features a cylindrical, multi-layered structure that envelopes the Interaction Point (IP), where the particle collisions take place. The sub-components of the Belle II detector are specialized for measuring various particle properties. Among these are the tracking sub-detectors, responsible for gauging particle momentum and locating decay vertices. Additionally, there are particle identification sub-detectors to classify the type of particle and a calorimeter tasked with reconstructing the energies of photons and electrons. The detector consists of a beam pipe and the following sub-detectors from inside out, as shown in Figure 2.2 and § 5.1, the Vertex Detector (VXD) (the Pixel Vertex Detector (PXD) and Silicon Vertex Detector (SVD)), the Central Drift Chamber (CDC), The Time of Propagation Counter (TOP), The Aerogel Rich Detector (ARICH), The Electromagnetic calorimeter (ECL), and the KLong and Muon detector (KLM). Each plays a specific role, and together, they provide a comprehensive dataset essential for the reconstruction of B meson decays.

The Belle II detector with a zoomed-in view of the inner part

Figure 2.2. The Belle II detector with a zoomed-in view of the inner part adopted from [11]

The original Belle experiment and its contemporary BaBar set remarkable milestones, including the groundbreaking discovery of CP violation in the B meson system [12]. This discovery led to the validation of the Kobayashi-Maskawa model and earned a Nobel Prize in 2008 [13, 14]. By combining advancements in technology and insights from previous experiments like Belle and BaBar, the Belle II experiment aims to push the frontiers of our understanding of particle physics in the luminosity front.

PurposeNameComponentConfigurationReadout channels
Beam pipeBerylliumCylindrical, inner radius 10 mm, 10 µm Au, 0.6 mm Be, 1 mm paraffin, 0.4 mm Be-
TrackingPXDSilicon Pixel (DEPFET)Sensor size: 15×(L1 136, L2 170) mm², Pixel size: 50×(L1a 50, L1b 60, L2a 75, L2b 85) µm²10M
SVDSilicon StripRectangular and trapezoidal, strip pitch: 50(p)/160(n) - 75(p)/240(n) µm245k
CDCDrift Chamber14336 wires in 56 layers, inner radius of 160 mm outer radius of 1130 mm14k
Particle IDTOPRICH with quartz radiator16 segments in ϕ\phi at r∼120r \sim120 cm, 275 cm long, 2 cm thick quartz bars8k
ARICHRICH with aerogel radiator2×2 cm thick focusing radiators, HAPD photodetectors78k
CalorimetryECLCsI(Tl)Barrel: r=125−162r = 125 - 162 cm, end-cap: z=−102−+196z = -102 - +196 cm6624 (Barrel), 1152 (FWD), 960 (BWD)
Muon IDKLM barrelRPCs and scintillator strips2 layers with scintillator strips and 12 layers with 2 RPCsθ\theta 16k, ϕ\phi 16k
KLM end-capscintillator strips12 layers of (7-10)×40 mm² strips17k

Table 2.1. Summary of Belle II detector components, adopted from [15].

Belle II Software Framework

Alongside hardware improvements, the experiment’s analysis software has also been revamped to enhance both performance and user experience, culminating in a new software, basf2 [8, 16]. The methods described in this thesis are also implemented in the Belle II software, basf2. In the following, I briefly skim through the bird-eye view of the underlying processes in Belle II data-taking and the software pipeline based on [10]. Later, we go through the most relevant parts: simulation and reconstruction.

  1. Data Collection and Event Triggers: In the Belle II experiment, data acquisition occurs in specialized sessions characterized by a constant flux of electron-positron collisions. These collisions happen at an elevated frequency, and the detector system continuously scans the collision outcomes in real time. The focus is on identifying unique markers that signify the creation of BBˉ\mathrm{B}\bar{\mathrm{B}} meson pairs. An autonomous component within the Belle II online system (shown in Figure 2.3), known as the “trigger,” oversees this detection process. The online system consists of the Data Acquisition (DAQ), Level 1 Trigger (L1), and the High-Level Trigger (HLT). When the trigger discerns that BBˉ\mathrm{B}\bar{\mathrm{B}} mesons have been produced, it activates a comprehensive readout of the entire detector, converting this information into a format suitable for downstream analysis and reducing the amount of data as much as possible before they reach the first storage. The DAQ system is there to make sure that all trigger signals are synchronously delivered to all sub-detectors, and provides the high-speed data links to read out the full detector data for each event and forward it to the HLT system. The Belle II trigger, responsible for starting the data readout of the whole detector for interesting events, is engineered to handle collision rates up to 30 kHz at full SuperKEKB design luminosity.

A schematic and simplified diagram of the Belle II data flow

Figure 2.3. A schematic and simplified diagram of the Belle II data flow, taken from [15]

  1. Event Classification and Data Segmentation: In particle physics, the term “event” represents a specific, time-bound physical process, commonly a particle decay. Within the context of Belle II, a “signal event” refers to, e.g., the unique occurrence tied to BBˉ\mathrm{B}\bar{\mathrm{B}} meson pair and the subsequent decay products generated by each meson. The organized data captured by the full detector readout, instigated by the trigger, exemplifies one such signal event. Collections of these signal events are then grouped into datasets that serve as the primary material for scientific discovery. It is crucial

to distinguish these signal events from “background noise,” which includes extraneous physical processes that can interfere with the experimental results. They will be discussed in detail during the next section. As the luminosity of SuperKEKB escalates through a reduction of transverse beam size and a doubling of the beam currents, compared to its predecessor KEKB, the background noise at the Belle II detector is substantially raised. Generally, this background interference originates from two primary sources: beam-induced processes and luminosity backgrounds. Beam-induced backgrounds are a consequence of beam particles interacting with elements like residual gas inside the beam pipe, bending magnets, or other particles within the same bunch. On the other hand, luminosity backgrounds result from beam collisions that yield inconsequential physics phenomena, such as Bhabha scattering or two-photon interactions. Identifying and discarding these background processes is of paramount importance, especially for low-multiplicity analyses.

3. Event generation and Simulation: The ability to compare the observed data from the detector with theoretical expectations is paramount for the validation of results in particle physics. To fulfill this requirement, simulated events are generated that mimic the behavior of real detector events as closely as possible. This simulation process relies heavily on the Monte Carlo (MC) method, which involves the repeated sampling of pseudo-random numbers. Simulated data enables the estimation of the statistical uncertainties associated with the recorded data and the estimation of background noise. It is also used to validate the detector and its operation, estimate the detector’s efficiency, and perform calibration purposes. Simulation of data involves two steps, namely, event generation and the simulation of the detector response of each generated event. Event generation consists of simulating the physics interactions under study. Based on the initial conditions set for colliding electrons and positrons, a range of particles are generated in accordance with the specific physics model being examined—be it advanced theories. These generated events are often categorized into different samples for specific analyses: “signal MC” pertains to the decay process being studied, whereas “generic MC” refers to basic standard model processes. Occasionally, additional “background MC” samples may be generated for processes that need to be excluded from the analysis. For Belle II, this phase is efficient and tailored for various study objectives.

  1. Detector Simulation: Upon generating the essential 4-momentum vectors and taking the detector geometry and the magnetic field into account, the next task is to simulate their interactions with the detector material, capturing processes such as ionization, scintillation, Bremsstrahlung, pair production, and Cherenkov radiation, among others. The simulation of the detector response is called “digitization.” This part of simulation has been the subject of extensive research and development, culminating in the simulation software Geant4 [17]. Geant4 takes the generated particles and replicates their interactions within a virtual Belle II environment. Subsequently, the energy deposition and particle interactions within each sub-detector are computed. Geant4 performs the simulation by tracking the particles, one at a time, through the geometry. It takes into account the effect of the magnetic field on the particle, the energy loss, and multiple scattering the particle experiences while traversing material and various other electromagnetic and hadronic effects. If a particle decays, the decay products are added to the list of particles. The initial four-momentum vectors given to Geant4 are called primary particles, while all particles created from interactions or decay during the simulation are called secondary particles. Additional specialized software modules then convert these Geant4 outputs into realistic detector signals. The digitization is the last step in the processing chain used solely for Monte Carlo data. Post-digitization, the workflow for both Monte Carlo and real-world data aligns in terms of data processing steps. For instance, the pixel detector software translates the energy deposits into pixel activations, PXD digits. This part of the simulation is the computational bottleneck both from the time and space (storage) complexity perspective.

For example, the PXD background data has the highest storage consumption among all sub-detectors, with an utterly infeasible hypothetical online simulation time. A solution to alleviate the detector simulation problem is to use surrogate models under the topic of “Fast Simulation,” the main focus of this thesis.

  1. Reconstruction Procedures: The term “reconstruction” in this setting refers to the act of methodically determining the attributes of a decaying BB meson and its decay products. These reconstructed signal events serve as the foundational data for the downstream physical analyses. The reconstruction workflow adopts a bottom-up strategy, starting with the final state particles and their detector responses and moving backward to recreate each decay event until the originating BBˉB\bar{B} pair (or one BB) is reached. However, it’s never possible to uniquely identify all the particles in the interaction because of not only hadronic interactions and background processes but also because we a priori don’t know which decay products correspond to which mother particles. So, all we can do is look at the detector response, find a set of most likely

particles, and then leave it to the analyses to do a proper statistical analysis of the events. The first stages of reconstruction consist of two main procedures:

  • Clustering: One of the first steps in reconstruction is “clustering”, where one needs to modulate the detector responses in each sub-detector if they are related. This is done by means of a clusteriser, which groups together adjacent detector signals (nearest neighbor) into clusters, performed for both real data and Monte Carlo data in the same way. One can then calculate geometric properties of these clusters like size, shape or center. For PXD, either a center of gravity or an analog head-tail algorithm (for cluster multiplicities larger than two hit pixels), is used. These algorithms aggregate and interpret data from pixelated detectors such as PXD that results in “PXD Clusters.” For real detector data, the simulation stage is skipped, and the input is the fired pixels, “PXD Digits.” The “Center of Gravity” (cog) method is widely used for localizing the point where a particle has interacted with a detector, especially in situations where the interaction affects more than one pixel. The idea is to take a weighted average of the positions of all the pixels in a cluster, with the weights being the ionization strengths (e.g., charge, energy deposition, etc.) in those pixels, as:

cog⁡=∑i=1Nwi⋅xi∑i=1Nwi\operatorname{cog} = \frac{\sum_{i=1}^{N} w_i \cdot x_i}{\sum_{i=1}^{N} w_i}

Where wiw_i is the weight (often the energy deposition) of the ithi^{th} pixel, and xix_i is the position of the ithi^{th} pixel. NN is the total number of pixels in the cluster. The “Analog Head-Tail” algorithm is used primarily when the cluster multiplicity is larger than two hit pixels. The algorithm aims to identify the ’head’ and ’tail’ of a cluster, which correspond to the entry and exit points of the particle in the detector plane. This is particularly useful for distinguishing between particles that might have similar total charge deposition but different directions. The clustering of the PXD pixels is done in a row-wise manner, with increasing values for the column index and the row index. Starting with the pixel in the upper left corner. The clusteriser of basf2 checks each pixel to make sure that the ratio of its charge over a common noise level is above a given threshold. If it is, the left neighbor in the same row and the direct neighbors in the previous row are investigated. If one or more clusters have already been found in those neighboring pixels, the clusters are merged and the pixel is assigned to this cluster. Otherwise, a new cluster is created, and the pixel becomes its first member. The clusteriser proceeds with the pixel to the right of the current pixel or, if the pixel is the last pixel in the current row, with the first pixel of the next row. The procedure is repeated until the last pixel in the last row has been processed. This clustering scheme investigates each pixel only once and requires only the current and the previous pixel row to be stored in memory, making it an efficient and memory-saving pixel clustering method. After having grouped all pixels into clusters, the position of each cluster is determined: if the size of the cluster, defined by the number of pixels belonging to the cluster, exceeds a given threshold, the head-tail algorithm is used to calculate its position. The default threshold is set to 3 pixels. The Analog Head-Tail algorithm calculates the position of the cluster by using the outermost pixels of a cluster (head and tail).

  • Track Reconstruction: The Tracking (Track Reconstruction) of Belle II is designed to capture the spatial location of charged particles as they move through the detector. Reconstruction involves the inference of the trajectories of particles through the tracking detectors, called tracks. Multiple sensor layers work in tandem to collect this positional data, enabling the accurate reconstruction of particle paths, known as “tracking.” Belle II incorporates two primary tracking steps to ensure precise trajectory identification and thereby contribute to accurate decay vertex estimates, the track finding and the track fitting step. During the track finding step, the clusters from the PXD/SVD and the fired wires in the CDC that belong to the same track are identified. In other words, it finds patterns in the hits or hit clusters in the tracking detectors. Then, the identified clusters and wires are passed to the track fitting, which estimates the optimal track parameters. Here, the goal is to determine the best estimate of the kinematic variables describing the particle trajectories corresponding to each found hit/cluster pattern to obtain the particle position and momentum close to the interaction region as precisely as possible.

Pixel Vertex Detector (PXD)

The Pixel Vertex Detector (PXD) [11] serves as the innermost layer of the Belle II detector complex, shown in Figure 2.4 and Figure 2.6. This semiconductor-based apparatus employs DePFET (Depleted P-channel Field Effect Transistor) technology, a semi-conductor sensor that combines the detection of the passage of charged particles and the amplification of their deposited energy within one device. The PXD is the first full-scale detector employing the DePFET technology in High Energy Physics. The primary function of the PXD is to determine the positions of charged particles that emerge from collision events. A key objective is the highly accurate determination of decay vertices, a task for which the PXD is strategically located in close proximity to the interaction point (IP). The device spans a comprehensive polar angle range, specifically 17∘<θ<150∘17^\circ< \theta< 150^\circ, within the Belle II setup. The PXD incorporates 40 pixel sensors, depicted in Figure 2.6b, which are organized into two concentric circles (annulus) around the IP, termed the inner and outer layers. These pixel sensors are paired into structures known as “ladders,” aligned along the zz-axis. The geometrical arrangement of these ladders in the xx-yy plane resembles an octagon for the inner layer and a dodecagon for the outer layer.

The first picture of the full PXD during the commissioning phase in DESY Hamburg

Figure 2.4. The first picture of the full PXD during the commissioning phase in DESY Hamburg.

Figure 2.6. Pixel detector layout The pixel detector (PXD) is the inner-most sub-detector of the Belle II experiment (a) and is configured in a two-layered overlapping sensor structure (b). Geant4 Simulated PXD Background image examples (c) for sensors 7 (top) and 25 (down).

Each of these pixel sensors integrates a 250×768250 \times768 matrix of silicon pixels that are engineered to interact with charged particles, as shown in Figure 2.5. The activation. of a pixel is closely linked with the ionization of underlying material within the silicon medium and producing electron-hole pairs. Data relating to the spatial coordinates of the particle-pixel interaction and the number of ionized electrons (accumulated charge) is then collected as “hits” (see Figure 2.6c).

PXD specifications showing the inner and outer segments, sensor characteristics, and pixel dimensions
LayerSensorsLaddersRadius [mm]Matrix area [mm2^{2}]Pixel area [µm2^{2}]Segment
Inner1681412.5 × 44.8050 × 60Outer
50 × 55Inner
50 × 70Inner
Outer24122212.5 × 61.4450 × 85Outer

Figure 2.5. PXD specifications and detailed information adopted from [18]

The output from the pixel sensor is essentially a map detailing the hits and their corresponding levels of charge displacement, measured during the readout interval. This raw data undergoes noise filtering and digitization, converting it into 8-bit integers that range from 0 to 255. Given the technical constraints of silicon pixel technology, physical necessities, and the need for rapid readout, the pixel sensors are calibrated for optimal spatial resolution, thus ensuring adequate tracking precision [18].

Remarkably, the PXD can generate an “ultra-high granular” amount of raw data, owing to its intricate pixel configuration comprising a total of 7.68×1067.68 \times10^{6} pixel channels. The data production rate can reach up to 20 GB/s, outpacing the combined data rates of all other components in the detector by over a factor of ten. The PXD necessitates a parallel readout time of 20 μs20\ \mu\mathrm{s} from the individual pixel sensors which is considerably longer than the approximately 10 μs10\ \mu\mathrm{s} required for a beam particle to complete one circuit of the collider.

PXD Background: Types and Levels

Due to its proximity to the IP and the incoming particle beams, the PXD is susceptible to a huge amount of radiation and background levels. PXD hits generated by background shower particles deteriorate the detector’s physics performance. The rates of these background processes are correlated with multiple factors e.g. beam size, beam current, luminosity, accelerator status, and vacuum conditions [15]. Generally, the processes contributing to background in the detector can be classified into five main categories:

  • Touschek Background: At Belle II, Touschek scattering is a predominant source of background. This phenomenon involves Coulomb interactions between two particles within the same beam bunch. A “bunch” refers to a group of particles that are packed together and travel around the accelerator ring almost as a single unit. These bunches are created to maximize the chances of interaction between particles when two such bunches cross paths in the detector. As a result of such interactions, the energy levels of the two participating particles diverge from the nominal beam energy; one particle experiences an energy increase while the other undergoes an energy loss. The rate of Touschek scattering is directly related to the square of the beam current and inversely related to both the number of bunches in the accelerator ring and the size of the beam [19].

  • Beam-Gas Background: Scattering between the beam and residual gas molecules within the beam tube stands as another significant source of background noise at Belle II. Two types of beam-gas scatterings are prevalent: the elastic Coulomb scattering, which alters the trajectory of the beam particles, and the inelastic Bremsstrahlung, which diminishes their energy. The rate of such scatterings is directly linked to the residual gas pressure inside the tube and the current of the beam.

  • Luminosity-dependent Background: Background arising from beam interactions at the Interaction Point (IP) is referred to as luminosity-dependent background, and its intensity is directly proportional to the luminosity, as shown in Figure 2.7.1 In the case of SuperKEKB, the target luminosity is approximately 40 times greater than KEKB’s peak luminosity, making the luminosity-dependent background a significant concern. A critical form of this background stems from radiative Bhabha scattering (e+e−→e+e−γe^{+}e^{-} \to e^{+}e^{-}\gamma). In this process, beam particles emit photons and deviate from their

PXD occupancy as a function of luminosity

Figure 2.7. PXD occupancy (layer 1 here) increases with beam currents and luminosity, coming from raw data. The occupancy of PXD is defined as the ratio of the number of hits to the total number of PXD pixels. The Experiment numbering corresponds to the Belle II run periods.

nominal paths. At a high luminosity regime, this background dominates over other

Belle II backgrounds. Two-photon processes (e+e−→e+e−e+e−e^+e^- \to e^+e^-e^+e^-) represent another source of background. The emitting low-momentum electron-positron pairs cause the beam particles to lose energy, similar to the radiative Bhabha process. Moreover, when possessing low transverse momentum, the emitted electron and positron curl in the Belle II solenoid field, making multiple hits on the PXD.

  • Background from Synchrotron Radiation: Another background component affecting the inner detectors of Belle II arises from Synchrotron Radiation. The intensity of this radiation is governed by the square of both the beam energy and the magnetic field strength. Thus, the HER electron beam predominantly contributes to the Synchrotron Radiation background. The energy of Synchrotron Radiation photons impacting the PXD and the SVD ranges from a few kiloelectronvolts (keV) to multiple tens of keV.

  • Background Due to Injection: The beam lifetime in SuperKEKB is notably brief, necessitating frequent top-up injections through a betatron injection scheme [20]. When the overall beam current drops below 99% of the nominal current, additional charge is injected into low bunch-current buckets at varying frequencies (between

1–25 Hz). These new bunches oscillate horizontally around the main beam, causing temporary spikes in the background levels of the Belle II detector. These spikes last for milliseconds each time the newly-injected bunch crosses the interaction point.

Thus, these traversing background processes can deposit considerable energy, thereby reducing the lifespan of the detector and creating malfunctioning pixels or zones of reduced efficiency. Moreover, due to its proximity to the IP, PXD is also vulnerable to back-scattered low-energy Synchrotron Radiation photons. This huge amount of background manifests itself primarily in a sensor-level observable called the “occupancy.” The occupancy of a pixel sensor, defined as the ratio of the number of hits to the total pixel count in the matrix, is directly influenced by the level and type of background noise. Currently, the average PXD occupancy is below 0.3%.

In reality, there exists a constraint due to bandwidth limitations. For a 30 kHz trigger rate, data loss starts to become a factor when the average occupancy of the inner PXD layer exceeds 3%. Beyond this occupancy threshold, offline performance experiences a marked decline due to issues like cluster merging and a higher likelihood of incorrectly associating hits to particle tracks. It should be noted that performance degradation can begin even below this specified level.

PXD Background Simulation: Ideas and Challenges

To accurately replicate experimental conditions, emulating background processes is essential. Two methods are currently used, as depicted in Figure 2.9, for implementing background simulation: “background simulation” and “random trigger” Both techniques have their pros and cons, and the choice between them depends on the specific requirements of the simulation and the resources available.

Three PXD background simulation options: background overlay with Geant4 simulation, background overlay with random trigger, and background overlay with surrogate model

Figure 2.9. PXD Background simulation options.

Background Simulation

In the “background simulation” approach, a collection of synthetically emulated background events serves as a resource. These backgrounds are simulated with the software framework Strategic Accelerator Design (SAD) [21]. SAD, initialized with beam optics parameters and detector elements, tracks scattered particles through the sequence of detector elements. The tracking simulation initiates by establishing uniformly spaced scattering zones around each ring, generating particle bunches in these regions. These particles are randomly created within a three-dimensional Gaussian distribution. Their momentum and statistical weight are calculated using established scattering theories, specifically Coulomb, Bremsstrahlung, and Touschek scattering. These background events are essentially the particle outcomes of specific background processes. Using Geant4 for detector response simulation, the PXD digits are then stored in a dedicated file. This file is then utilized to overlay background particles onto those from a signal or physics event, as shown in Figure 2.9a. This method improves the quality of digitization by combining energy deposits to generate sensor responses. However, it neither fully reconciles the discrepancy between simulated and real background data nor is it computationally efficient. The reason is that it has to be pre-generated and stored, thus creating a high “space-complexity” from the computational point of view, as shown in Figure 2.8. On the other hand, it is extremely inefficient to run this on the fly and creates a high “time-complexity” issue.

Figure 2.8. Geant4 simulated Background data volume comparison of PXD data per event compared to other sub-detectors at Belle II. The PXD background data is nearly as much as the combined background data for all other sub-detectors.

Random Trigger

In the “random trigger” technique, an alternative to direct simulation, real experimental sensor responses to background noise are directly utilized randomly. This approach operates on the premise that each sensor’s response can be associated with a unique background event. The procedure begins with a focus on simulating the energy deposits of only the signal particles. Subsequently, sensor-specific responses from actual recorded background events—coming from random trigger events—are overlaid during the digitization stage, as shown in Figure 2.9b. The random trigger activates data collection separate from the event signature. Three kinds of random triggers exist [18]: a periodic one synced with SuperKEKB’s bunch crossing signal, a pseudo-random trigger based on an independent local clock, and a delayed Bhabha trigger that fires shortly after a specific bunch passes. This results in a more faithful simulation of real PXD conditions than the former technique. This method even allows for the incorporation of noise from detector electronics. However, it also comes with challenges, such as the potential loss of certain background properties due to threshold effects, the lack of real detector background response for higher (undetected) luminosity, and as always, the requirement for storage and transfer of a large volume of extra data. Thus, this also creates a high “space-complexity” issue from the computational point of view.

PXD Background Simulation: Surrogate models

A strong solution to the above challenges and issues for PXD simulation is “Surrogate models”. Surrogate models for fast detector simulation are simplified, computationally efficient approximations that emulate the behavior of more complex, detailed simulations of particle detectors on the fly of the analysis pipeline. Traditional detector simulation methods, as discussed above, are very computationally intensive and storage costly. A surrogate model aims to replicate the essential features of a full detector simulation but at a fraction of the computational cost. These models are constructed using Deep Generative Models (DGM), where the surrogate model is trained on a dataset either generated by the Geant4 simulation or the real detector data. Once trained, the model can generate samples that are statistically similar to their training data and even generalize to the data beyond the training data. There are some key requirements that the model should meet to be effective and efficient:

  1. Low time-complexity: The surrogate model must be computationally efficient in order to facilitate fast simulations. Computational speed is crucial when performing large-scale simulations or when needing to iterate the model many times for optimization or fine-tuning. Thus, the model should take advantage of parallel processing capabilities and have to be optimized for the hardware it is expected to run on, whether it’s a CPU or GPU.

  1. Low space-complexity: It should come with a minimal storage cost. Hence, the underlying compression technique has to reduce the storage footprint without sacrificing too much in terms of the downstream physics analysis.

  1. Realistic and Diverse: It has to generate samples as diverse as possible from the downstream physics analysis point of view. Thus, the sampling techniques should be capable of employing the nuanced behaviors and symmetries of the PXD to ensure that the model captures the diversity inherent in the real data.

  1. Extrapolation: It has to be able to extrapolate to background levels beyond the current beam parameters and luminosities in order to analyze the PXD operation at higher luminosity and to do physics analysis beyond the current experimental limits. Therefore, the model should be robust against overfitting and incorporate a measure of control when extrapolating to give a range of plausible outcomes.

In this thesis, I propose the surrogate modeling solution as an amortized simulator for the PXD background simulation as a fast and efficient simulation method, as shown in Figure 2.9c. First, a DGM is developed to emulate Geant4 simulated background data as an approximation of the real data with much lower data complexity and implement this approach into the Belle II software, basf2. Then, for the final challenge of real PXD detector data, I introduce a much more efficient DGM as a solution that successfully satisfies all the above conditions.

PXD Background Simulation: Figure of Merits

For evaluation, we have three main categories of metrics: Low-level Metrics, Neural Network-based Metrics, and Physics-Level Metrics. These metrics evaluate different aspects of the generated PXD background. Through the next chapters, I elaborate on and incorporate them to analyze and compare the results with each other. Some of them are being introduced for the first time in the Fast Detector Simulation domain. In the following, I briefly introduce them.

Low-Level Metrics

For Low-Level metrics, we are interested in analyzing the histogram projections of the data. Therefore, the Low-Level observables as marginal distributions include charge distribution and average charge values per sensor, occupancy distribution and average occupancy per sensor, and the correlation between the average occupancies.

Neural Network-based Metrics

The necessity for incorporating Neural Network (NN)-based metrics in the evaluation process of PXD background simulation arises from two main factors:

  • Complexity of Data: PXD background data is intrinsically complex and is influenced by numerous variables originating from the underlying Physics processes. Moreover, they exist in a high-dimensional space, often making evaluating their quality and diversity challenging using traditional metrics. Thus, simple 1D and 2D histogram-based Low-Level observables might not fully capture this complexity. NN-based metrics can distill this high dimensionality into a more manageable form by focusing on relevant feature spaces. This is particularly useful for understanding what aspects of the data are being captured (or missed) by the DGM, such as the data modality (mode collapse).

  • Quantitative Assessment: While Low-Level metrics provide a direct evaluation of specific properties, they might lack the capacity for an overall quantitative assessment of data similarity. NN-based metrics offer a single quantitative score that can be compared across different models or iterations.

Thus, NN-based metrics offer a multifaceted evaluation approach that complements the Low-Level metrics. They enable a more comprehensive, nuanced, and rigorous analysis of the performance and reliability of PXD background simulations. The NN-based metrics that I incorporate in this thesis are the Frechet Inception Distance (FID) (see Chapter 5) [22], Kernel Inception Distance (KID) (see Chapter 5) [23], and Vendi Score (see Chapter § 3) [24] and show how they stand out as particularly insightful and interpretable evaluation tools. FID and KID are used to measure the similarity between the distributions of generated data and real data in the feature space of an NN model that is pre-trained on the PXD background dataset with a very high precision. Vendi score, on the other hand, is to measure mode collapse (see Chapter 3) and the diversity evaluation problem (see Chapter § 3) of the generated data.

Clustering and Topological Data Analysis

In PXD background simulation, a fundamental question arises concerning the spatial organization of PXD hits. Specifically, how are points in a PXD point cloud clustered? What is the complexity of these clusters, and what topological and geometrical features are discernible? Or, as we adjust our observation window, what shifts can be observed in the geometric representation of the PXD hits?

Topological Data Analysis (TDA) serves as an instrumental tool for investigating these questions. It allows us to delve into the intrinsic shape of the data by formalizing the notions of of proximity and continuity of data. TDA focuses on identifying topological features in data sets, such as connected components, loops, and higher-dimensional cycles (see Chapter § 3). These features offer invaluable insights into the structural properties of PXD hits across multiple scales, which is especially useful for identifying underlying patterns and anomalies for both the real PXD data and the generated ones. This is the very first time that detector signatures are being analyzed through the lens of TDA.

Crucially, the reconstruction process in Belle II relies on clusters of PXD hits to perform its analyses. As such, it becomes imperative to accurately reproduce the background on this cluster level. To address this, I further complement the TDA perspective with clustering analysis implemented in the Belle II software. Integrating these two approaches aims to capture a more complete understanding of the structural, geometrical, and topological properties inherent in the PXD background data. Clustering analysis in Belle II offers a different set of evaluative tools that focus on grouping data points based on similarity measures such as distance or density. The complementary nature of these methods allows for a more robust validation mechanism, ensuring that the background is faithfully represented at the cluster level. This integrated approach provides an interpretable framework for the TDA and the clustering analysis, fulfilling the requirements for a precise and comprehensive background simulation.

Physics-Level Metrics

Eventually, within this study, I also analyze the effect of generated and simulated PXD backgrounds on high-precision charged track reconstruction and compare them. This is indeed crucial for both decay vertex measurement, quantifying and correcting systematic uncertainties, and any subsequent physics analyses. The Helix parameters quantify the relative distance to the Point of Closest Approach (POCA), a reference point on the particle track closest to the center of the coordinate system. The POCA is determined by extrapolating a particle track to the global detector zz-axis. In the Belle II experiment, the trajectory of charged particles in a uniform magnetic field can be encapsulated by five helix parameters {d0,z0,ϕ0,ω,tan⁡λ}\{d_0, z_0, \phi_0, \omega, \tan\lambda\} relative to POCA, depicted in Figure 2.10.

The perigee parametrization of the track helix

Figure 2.10. The perigee parametrization of the track helix adopted from [25]

  1. d0d_0: Refers to the impact parameter, indicating the shortest perpendicular distance from POCA. This provides insight into the spatial offset of the helix trajectory from the coordinate system’s center.

  1. z0z_0: Denotes the zz-coordinate at the point where the helix is closest to the POCA, revealing the helix’s vertical positioning relative to the central zz-axis.

  1. ϕ0\phi_0: Represents the azimuthal angle of the particle’s momentum at its nearest proximity to the POCA, specifying the helix’s initial rotational direction on the xyxy-plane.

  1. ω\omega: Characterizes the curvature of the helix, which is inversely related to the radius of curvature RR. The curvature provides a sense of the helix’s tightness and is influenced by particle momentum and magnetic field intensity.

  1. tan⁡λ\tan\lambda: Corresponds to the tangent of the dip angle λ\lambda, determining the helix’s inclination with respect to the xyxy-plane. A flat helix exhibits tan⁡λ=0\tan\lambda= 0, while a steeper helix is indicated by a larger tan⁡λ\tan\lambda.

These helix parameters collectively offer a detailed geometric and kinematic description of a charged particle’s trajectory. The resolution of these helix parameters serves as a compelling physics-level metric and comparative assessment for studying the effects of PXD background on charged track reconstruction (see Chapter 5).

On the Shoulder of Giants: The Modern Machine Learning Tools

Introduction

The arena of Artificial Intelligence has been evolving at a relentless pace, fueled by an incessant pursuit to replicate and augment human cognitive abilities. In this quest, the development of advanced machine learning techniques has been instrumental, providing the bedrock for sophisticated AI systems. This chapter embarks on a journey to explore these techniques, focusing on three pivotal advancements that have revolutionized the landscape: Transformers and Attention Mechanism, Deep Generative Models, and Self-Supervised Learning.

The first section, “Transformers and Attention Mechanism”, dives deep into understanding these models that have redefined the field of natural language processing. With their ability to handle long-range dependencies in sequences, they form a key pillar in this study, providing us with the tools to extend and adapt these mechanisms to the context of detector simulation.

In the “Deep Generative Models” section, I venture into the realm of generative models, the vanguard of modern machine learning that strives to capture and mimic reality. By providing a deep dive into the two main latent variable models, Generative Adversarial Networks and Variational Auto Encoders, I aim to elucidate how these models serve as a launching pad for the innovative techniques and methods that I will introduce in the following chapters.

Finally, the “Self-Supervised Learning” section leads us to the frontier of machine learning research, the Dark Matter of Artificial Intelligence. By harnessing the power of SSL and enabling models to generate their own supervision, it has the potential to significantly improve the efficiency of learning algorithms. This concept will play a central role as I build upon it to devise novel learning strategies.

This chapter is designed to provide a comprehensive understanding of these three pillars of modern machine learning. By dissecting their inner workings, highlighting their strengths, and understanding their limitations, I lay the groundwork for the novel mechanisms and strategies I will introduce in the coming chapters. As we navigate this journey, we are indeed standing on the shoulders of giants, harnessing their wisdom and insights to forge new paths in the exciting world of AI research.

Attention Mechanism and Transformers

The concept of attention in neural networks is a powerful mechanism that allows a model to enhance its predictive ability by selectively focusing on specific subsets of data. This idea, inspired by the human cognitive function, assigns learned weights to quantify the degree of attention, thereby forming the output as a weighted average. In essence, humans do not process all incoming information simultaneously; instead, attention is selectively allocated to relevant information when necessary. Initially implemented in the realm of computer vision to alleviate the computational load of image processing, attention was designed to concentrate on specific regions of images rather than the entire picture, effectively mimicking human perceptual tendencies. However, the inception of the attention mechanisms we recognize today is primarily traced back to the field of natural language processing. Bahdanau et al. [26]. employed attention in a machine translation model to rectify the structural issues inherent in recurrent neural networks, subsequently highlighting the benefits of attention. This endorsement paved the way for the refinement and popularization of attention techniques across a multitude of tasks.

Attention Mechanism and Self-Attention

Self-attention represents a unique form of attention mechanism that allows a model to make inferences about a specific section of a data sample by leveraging information from other portions of the same sample in a permutation-invariant way. This concept echoes the principles of non-local means [27], an image denoising technique, where each pixel in the output image is a function of all pixels in the input image.

Formally, an attention mechanism, is the data of (K,Q,V,s,A)(\mathbf{K},\mathbf{Q},\mathbf{V},s,\mathbf{A}). The vector spaces K∈RN×dk\mathbf{K} \in\mathbb{R}^{N \times d_k}, Q∈RN×dk\mathbf{Q} \in\mathbb{R}^{N \times d_k} and V∈RN×dv\mathbf{V} \in\mathbb{R}^{N \times d_v} are the set of Keys, Queries, and Values. The bilinear map s:K×Q→RN×N\mathbf{s} : \mathbf{K} \times\mathbf{Q} \to\mathbb{R}^{N \times N} is a score function between the key and the query where the codomain is an attention score. The query serves as a request for information, and the corresponding attention score quantifies the relevance of the data encapsulated in the key vector with respect to the query. In the case of self-attention, the keys, queries, and values all come from the same sequence, allowing each element to attend to all others in that sequence. Different score functions are the additive [26], simple Dot-Product [28], Scaled Multiplicative [29], General [30], Activated General [31], and similarity-based [32]. Then, the attention, Att\mathbf{Att}, is defined as,

Att(K,Q,V):=Align⁡(s(K,Q))V,\mathbf{Att}(\mathbf{K},\mathbf{Q},\mathbf{V}) := \operatorname{Align}(s(\mathbf{K},\mathbf{Q}))\mathbf{V},

Transformer

where dkd_k and dvd_v are the dimensions of the corresponding vector spaces. The goal of the attention module is to produce a weighted average of the value vectors. Thus, the scores are redistributed via an alignment operation Align⁡(⋅):Rd→[a,b]\operatorname{Align}(\cdot):\mathbb{R}^{d}\to[a,b] that maps the non-compact distribution to a compact one between [a,b][a,b]. Different alignment functions are the soft or global alignment, like the Softmax function [28], that also introduces a probabilistic interpretation to the input vectors. Another variant, the hard alignment [33], forces the attention model to focus on exactly one feature vector in a deterministic sense.

For instance, the attention mapping used in the vanilla Transformer [29] adopts the scaled dot-product as the bilinear map between keys and queries as

Att⁡(K,Q,V):=softmax⁡(KQTdk)V.(1)\operatorname{Att}(\mathbf{K},\mathbf{Q},\mathbf{V}) := \operatorname{softmax}\left(\frac{\mathbf{K}\mathbf{Q}^{\mathrm{T}}}{\sqrt{d_k}}\right)\mathbf{V}. \tag*{(1)}

As the dimensionality of the key, represented by dkd_k, increases, the dot product of the query and key can potentially escalate in magnitude. If these large values are then passed through the softmax function during the alignment stage, the resulting gradient can diminish significantly, impeding the model’s ability to converge effectively. The incorporation of the normalization factor 1dk\frac{1}{\sqrt{d_k}} serves as a countermeasure to this issue, effectively preventing the occurrence of vanishing gradients even when dealing with large inputs.

Rather than simply computing the attention once, the multi-head mechanism runs through the scaled dot-product attention of linearly transformed versions of keys, queries, and values multiple times in parallel via learnable maps WikW_i^k, WiqW_i^q and WivW_i^v. The independent attention outputs over hh number of heads are then aggregated and projected back into the desired number of dimensions via WpW^p,

References

  1. [1]M. Srebre, P. Schmolz, H. Hashemi, M. Ritter, and T. Kuhr 2020. Generation of Belle II Pixel Detector Background Data with a GAN, EPJ Web of Conferences, 245, 02010. 10.1051/epjconf/202024502010.DOI
  2. [2]H. Hashemi, N. Hartmann, T. Kuhr, M. Ritter, and M. Srebre 2021. PE-GAN: Prior Embedding GAN for PXD images at Belle II, EPJ Web of Conferences, 251, 03031. 10.1051/epjconf/202125103031.
  3. [3]H. Hashemi, N. Hartmann, S. Sharifzadeh, J. Kahn, and T. Kuhr 2022. Intra-Event Aware Imitation Game for Fast Detector Simulation, NeurIPS 2022 workshop paper (Machine learning and the physical sciences). NeurIPS_ML4PS_2022_97.
  4. [4]H. Hashemi, N. Hartmann, S. Sharifzadeh, J. Kahn, and T. Kuhr 2023. Ultra-High-Resolution Detector Simulation with Intra-Event Aware GAN and Self-Supervised Relational Reasoning, Under Peer Review by Nature Communications. 10.48550/arXiv.2303.08046
  5. [5]B. Hashemi, C. Krause, 2023. Deep Generative Models for Detector Signature Simulation: A Taxonomic Review, Under Peer Review by Reviews in Physics. 10.48550/arXiv.2312.09597DOI
  6. [6]B. Hashemi, 2023. YonedaVAE: Self-Supervised Out-of-Distribution Multi-Set Generation for Amortized Simulations, Under Submission to Machine Learning: Science and Technology.
  7. [7]B. Hashemi, B. Rieck, 2023. Topological Data Analysis for Particle Physics Detector signatures, Under Submission to Machine Learning: Science and Technology.

Paper details

vlnc:W_yhwayct6
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents