betaFeedback

WorksPhysics and Astronomy

TomOpt: Differential optimisation for task- and constraint-aware design of particle detectors in the context of muon tomography

Abstract

We describe a software package, TomOpt, developed to optimise the geometrical layout and specifications of detectors designed for tomography by scattering of cosmic-ray muons. The software exploits differentiable programming for the modeling of muon interactions with detectors and scanned volumes, the inference of volume properties, and the optimisation cycle performing the loss minimisation. In doing so, we provide the first demonstration of end-to-end-differentiable and inference-aware optimisation of particle physics instruments. We study the performance of the software on a relevant benchmark scenario and discuss its potential applications. Our code is available on Github.

Introduction

Differentiable programming for detector design optimisation

Over the past two decades, the availability of high-performance computing and the development of neural networks of larger capacity have conspired to fuel a revolution in the way we think at the optimisation of complex systems. When the dimensionality of the space of relevant design parameters exceeds a few units, and brute-force scans cease to be a viable option for its exploration. We nowadays, have the option of letting automated systems find their way to configurations that correspond to advantageous extrema of carefully specified objective functions. The engine under the hood of these optimisation searches is automatic differentiation, which allows computer programs to keep track of the gradient of the objective function, through the chain rule of differential calculus, as computer code performs arbitrarily complex successions of operations to model the behaviour of the system.

Crucial to a successful optimisation of the system is the inclusion in the model of all relevant effects that have an impact on the precision of the inference that the data generated by the system may produce. An incomplete description of the inference itself, or a mock up of the reconstruction techniques performing the dimensionality reduction step which translates raw data into high-level features informing the objective function, are likely to prevent the identification of designs that maximise the true objective, as they introduce a misalignment. Instead, it is possible to introduce a substantial margin of flexibility for the components of the model responsible for explaining factors that influence the absolute magnitude of the objective function but do not affect its gradient direction within the design space. The model, in other words, can be approximate as long as it correctly captures the interdependence of the various components, as the latter is what matters in determining the gradients of the objective function.

Particle detectors are among the most complex interconnected systems humans have ever designed and built: the large experiments at the CERN LHC, for example, include tens of millions of detection elements reading out highly interdependent physical phenomena produced when protons collide in the core of the detectors, at rates of 40 million times a second. For most of present-day experimental situations, we do not yet possess the capability of producing a fully differentiable, high-fidelity end-to-end model of the whole procedure that translates raw detector readouts into final measurements of physical quantities; this prevents the application of gradient descent methods to optimise the detector design. Because of the high complexity of that goal, we approach the problem by turning, for the time being, to easier modelling tasks that nonetheless share the same basic ingredients and structure with it. The successful optimisation of particle detectors of low complexity may allow us to teach ourselves how to tackle progressively harder problems, while accumulating working solutions to sub-tasks which are liable to be reassembled into larger modeling problems as needed.

Particle detectors used in muon tomography applications (see next subsection) appear especially fit to an end-to-end modelling. They involve physical phenomena of low complexity as the data-generating process usually involves the reconstruction of only one particle at a time, and they are liable to a successful parametric modeling of all the relevant effects. They thus constitute an ideal entry point in a long-term study of the end-to-end optimisation of full-fledged experiments. In this document we describe the studies we undertook to develop a software package for the optimisation of muon tomography apparatuses. In its present implementation, the code only considers a subset of the possible three-dimensional geometries of tomography systems, and is meant to provide only a proof of principle of the accessibility of a full, end-to-end optimisation of the problems these instruments are tasked to solve – most notably, imaging of inaccessible volumes, and hypothesis testing on their composition. It is our intention to develop a future version of the software which will provide a more complete solution of the concrete problems in tomography applications.

Muon scattering tomography

Muons, elementary particles related to the electrons but about 200 times heavier, are constantly and naturally produced by cosmic-ray interactions in the atmosphere. Their flux at sea level is of the order of 100 Hz m−2^{-2}, and their energy spectrum is very broad, peaking at a few GeV and extending up to the TeV scale. Muons are not subject to the strong nuclear interaction, and in the energy range 1 GeV to 100 GeV their electromagnetic interactions with matter are relatively mild: they mostly loose energy by ionisation, at a rate of about 200 MeV per meter of water. This makes them the most penetrating charged elementary particles.

The first known use of these cosmogenic muons for a practical application was the measurement of the overburden of a tunnel in an Australian mine, reported in 1955 [2]. A decade later, muons were famously used for searching for hidden chambers in Chephren’s pyramid in Egypt [3]. While that pioneering attempt did not find any new chambers, it convincingly established their absence from the bulk of the volume of that pyramid. Half a century later, the ScanPyramids collaboration was more lucky and was able to report the actual identification of unexpected features in Khufu’s Great Pyramid [4, 5]. These results have been based on the absorption of muons, i.e. on measuring the reduction of the muon flux due to their passage through matter. The same method has been applied successfully to several other use cases in archaeology and geophysics (see [6] for reports on several recent examples). However, since 2003 another process of muon-matter interaction has been used for a variety of applications: multiple scattering, where the muon undergoes several elastic electromagnetic interactions with the nuclei of the traversed material. As the strength of each collision depends on the charge of the nucleus, the deflection of a muon trajectory has a known dependence on the atomic number, ZZ [7, 8]

The first proposal to use muon scattering for elemental classification [9] focused on the search for high-ZZ materials hidden in containers. Searches of illicit materials at border controls are still one of the main drivers for research in muon tomography, see Ref. [10] for a recent review. However, in the last two decades this method has been explored for a large variety of other applications, including material identification in nuclear waste drums [11], detection of nuclear warheads [12], and various industrial applications to process monitoring and preventive maintenance [13]. While most applications of the method are focusing on the identification of high-ZZ elements, some teams are attacking the much more challenging task of identifying relatively light materials [14, 15, 16, 17], which demands excellent angular resolution given the aforementioned dependence on ZZ of the scattering angle.

Muon imaging, considering both absorption- and scattering-based methods, is a booming research direction with tens of new publications per year at a steadily growing rate [18]. The interested reader can find more information in some recent reviews [19, 13].

Inferring the atomic number of a passive volume irradiated by a flux of muons from cosmic rays requires modelling the muon flux at the Earth surface, and the multiple scattering muons undergo in matter due to multiple interactions with the Coulomb field of medium nuclei and electrons.

A formula for the flux of cosmic muons at sea level was first given by Gaisser in 1990 [20], neglecting Earth’s curvature and assuming muons do not decay while they traverse the atmosphere; the formula is therefore approximately valid only for muons arriving at sea level with a zenith angle θ<70∘\theta< 70^\circ and with an energy Eμ>100/cos⁡θE_\mu> 100/\cos\theta GeV. An improved formula was given by Guan et al. [21], accounting for the curvature of the Earth and providing a modified term that improves the agreement of the formula with experimental data at lower muon energies. Other alternative formulas have been proposed in the literature, e.g. the model by Shukla et al. [22] provides a closed form for the zenith angle distribution which gives an improved description of the data at all angles, including at θ>70∘\theta> 70^\circ. TomoOpt, the software package we developed and describe in § 2, enables the user to choose between the model by Guan et al. and that of Shukla et al.

For what concerns the scattering of muons in a material, a simple model is recommended by the Particle Data Group (PDG) [23] and used in TomOpt as the default scattering model (which we refer to as the PDG scattering model). It assumes a small-angle Gaussian scattering, and does not take into account non-Gaussian tails from rare large-angle scatterings. The model consists of picking two random scattering angles independently in orthogonal directions, as well as correlated transverse displacements. The initial model was proposed to predict a homogeneous material propagation, but then used by several groups in a step-by-step algorithm to model propagation in a volume discretised in voxels (voxelised) according to the radiation length X0X_0 of the voxels the muons traverse.

The PDG scattering model begins by computing the width of the angular distribution θ0\theta_0. For high energy muons with momentum pp travelling a distance dd the PDG model assumes:

θ0=0.0136 GeVpnX0×(1+0.038ln⁡nX0) ,(1)\theta_0 = \frac{0.0136\ \mathrm{GeV}}{p}\sqrt{n_{X_0}} \times(1 + 0.038 \ln n_{X_0}) \,, \tag*{(1)}

where nX0=d/X0n_{X_0} = d/X_0, dd is the distance traversed by the muon in the material in meters and X0X_0 is the material radiation length in meters. The logarithmic term is added in the model to account for possible low ZZ materials and to be able to reproduce Molière’s theory [24, 25]. The angular scattering is sampled from a zero-centred Gaussian distribution of this width with random numbers z1z_1 and z2z_2 taken from a normal distribution of mean 0 and variance 1:

δθ=θ0×z2.(2)\delta\theta= \theta_0 \times z_2. \tag*{(2)}

The lateral displacement due to scattering is computed as:

δx=θ0(z1d12+z2d2).(3)\delta x = \theta_0 \left(z_1 \frac{d}{\sqrt{12}} + z_2 \frac{d}{2}\right). \tag*{(3)}

Overview of this document

The present document is organised as follows: in Sec. 2 we describe the software package we developed, which was used to obtain all results we discuss in the following sections; Section 3 describes our benchmark of choice, involving the estimation of the fill level at a metal refinery; Finally, in Sec. 4 we summarise the results and discuss the future perspectives of this work.

TomOpt

The simulation and optimisation routines used for this study are structured into a software package referred to as TomOpt. The code for TomOpt is available open-source on Github [1]. In this section we will attempt to provide the reader with a sufficient understanding of the program.

TomOpt is a highly modular Python-based package that provides the full suite of tools and resources required for the investigation of the general problem of optimization of a scattering tomography detector. Its modular design allows each stage of the process to be performed by inheriting classes written to be most suitable for the particular task being studied. The key feature of TomOpt is the fully differentiable nature of the detector modelling and inference pipeline, implemented using the automatic differentiation framework PyTorch [26]. In this Section we describe the capabilities and workflow of TomOpt.

The TomOpt project began back in April 2021 with the aim of providing a practical demonstration of the differentiable pipeline proposed by the MODE Collaboration in Ref. [27] (and later published in Ref. [28]). Initially worked at by two of us, the project quickly gained contributors, helped by a proof-of-concept demonstration at the 1st MODE workshop on differentiable programming [29] in September that year. Over 2022, progress focussed on improving the capabilities, stability, and physical accuracy of TomOpt, as well as explorations into advanced inference using maximum likelihood methods, and deep-learning approaches. The primary contributors met twice for extended in-person development and knowledge sharing (both times kindly hosted by CP3, Belgium). Additionally, TomOpt was used as the basis for a data-challenge preceding the 2nd MODE workshop on differentiable programming [30], where further progress on the project was presented [31]. Progress in 2023 concerned the development of benchmarks for this publication. It was slowed slightly by one of the primary contributors leaving academia for industry and only able to offer evenings and weekends; but anyway, here we are at last!

Cosmic muon generation

In the context of detector optimisation, it is essential to have a realistic and flexible cosmic muon source that can adapt to the various muon scattering tomography applications and geographical locations. With that in mind, TomOpt samples muons from a horizontal generation surface SgenS_{\mathrm{gen}} with customisable energy and angular distributions. The user is free to provide their own parameterisation of the differential cosmic muon flux, expressed as:

Φz(E,θ)=dNdE⋅dθ⋅dϕ⋅dSz m−2⋅sr−1⋅GeV−1,(4)\Phi_z(E,\theta)=\frac{dN}{dE\cdot d\theta\cdot d\phi\cdot dS_z}\ \mathrm{m}^{-2}\cdot\mathrm{sr}^{-1}\cdot\mathrm{GeV}^{-1}, \tag*{(4)}

where dNdN is number of muon crossing a horizontal surface element dSzdS_z, with energy between EE and E+dEE+dE, zenith angle between θ\theta and θ+dθ\theta+d\theta, and azimuthal angle between ϕ\phi and ϕ+dϕ\phi+d\phi. Most of cosmic muon flux parametrisation are given as:

Φ(E,θ)=dNdE⋅dSn⋅dΩ m−2⋅sr−1⋅GeV−1,(5)\Phi(E,\theta)=\frac{dN}{dE\cdot dS_n\cdot d\Omega}\ \mathrm{m}^{-2}\cdot\mathrm{sr}^{-1}\cdot\mathrm{GeV}^{-1}, \tag*{(5)}

where dNdN is number of muon crossing a surface element dSndS_n perpendicular to the muon direction, with energy between EE and E+dEE+dE, and solid angle between Ω\Omega and Ω+dΩ\Omega+d\Omega. Because dΩ=sin⁡θ dθ dϕd\Omega=\sin\theta\,d\theta\,d\phi and dSz=dSncos⁡θdS_z=dS_n\cos\theta, Φz\Phi_z can be obtained from Φ\Phi as:

Φz=Φ⋅sin⁡θ⋅cos⁡θ.(6)\Phi_z=\Phi\cdot\sin\theta\cdot\cos\theta. \tag*{(6)}

The present version of TomOpt embeds two default differential muon flux parameterisations: Guan et al. [21] and Shukla et al. [22]. After providing the desired generation surface span dxdx, dydy, energy range ErangeE_{\mathrm{range}}, zenith angle range θrange\theta_{\mathrm{range}}, and number of muons NmuonN_{\mathrm{muon}}, their position is uniformly distributed on SgenS_{\mathrm{gen}}.

Figure 1. Representation of the generation surface in TomOpt (left), muon zenith angle (middle) and energy (right) distribution sampled from the Guan et al. [21] parameterisation.

For computing efficiency purposes, it might be relevant to choose θrange\theta_{\mathrm{range}} upper bound <70∘<70^\circ since horizontal muons are unlikely to reach both the passive volume and the detector layers. Once muons are generated, they will be propagated along negative zz direction. The simulation of their propagation through passive and active materials is described in 2.2 and 2.3 respectively.

Passive volumes and multiple scattering

Once muons are generated, TomOpt models their passage the volume of interest (passive volume) and their interaction with matter. The passive volumes of interest are modelled in 3D using layers of voxels, and one can assign to each voxel a material (specified by its radiation length X0X_0). TomOpt has it own database of X0X_0 values for various pure elements computed assuming nominal material density at 20 °C. Alternatively, users can easily provide their custom materials radiation lengths and densities, which are then plugged into Eq. (1). They may also manually specify the layout of materials inside the volumes of interest, or instead provide a customised generator class to provide randomly generated volumes that are representative of the current task.

For computational efficiency via vectorisation, batches of many muons traverse the volume simultaneously. Each muon is defined using its momentum, (x,y,z)(x,y,z) position, and (θ,ϕ)(\theta, \phi) trajectory, where 0≤θ<π/20 \le\theta< \pi/21 is the deviation from the zz axis, and 0≤ϕ<2π0 \le\phi< 2\pi is the azimuthal angle in the xyxy plane.

Example of voxelised volume of interest

Figure 2. Example of voxelised volume of interest.

The PDG scattering model described in Eq. (1) is the default scattering model implemented in TomOpt. We show in § 2.8 that, although embedding such model in a step-by-step algorithm affects the precision of the Coulomb scattering simulation compared to, for instance, highly accurate but slow models such as those provided in Geant 4 [32, 33, 34], the optimisation-focused nature of the procedure we present in this paper requires prioritising computational speed over precision.

As described in § 1.2, we sample the angular scattering according to Eq. (2) and subsequently compute the displacement due to the scattering according to Eq. (3): this procedure is performed twice per scattering call, to compute scatterings in perpendicular directions (xμx_\mu and yμy_\mu) in the muon reference frame. The muon-frame scatterings must then be converted into the volume frame using 3D rotations matrices. As mentioned earlier in this section, muons are propagated by steps of distance drdr along their flight path. The default value of drdr is set to 1 cm, but can be changed to accommodate for the voxel size, the volume of interest size, and the remaining distance to the bottom of the voxel. The optimal choice of drdr is discussed in § 2.8. The algorithm 1 summarises TomOpt muon propagation implementation.

Detector modelling and hit recording

The detectors modelled in TomOpt must have a flexible enough parameterisation to adapt to the use case, whilst not being constrained by a priori assumptions about the optimal configuration. The basic detector unit used is referred to as a panel. In xyxy-space, each panel has a spatial resolution σmax⁡\sigma_{\max} and detection efficiency ϵmax⁡\epsilon_{\max}, along with a fixed cost per unit area. In this way, the user may simulate detectors that are either constructable in-house, or otherwise commercially available. Panels are free to be positioned in xyzxyz space within pre-specified regions, referred to as detector layers. In optimisation, the aim is to learn the optimal position in xyzxyz and width in xyxy (xyxy-span) per panel; five learnable parameters per panel. Currently, the number of panels per detector layer is fixed throughout the optimisation process, due the difficulties in computing gradients for integer variables.

Algorithm 1 Muon propagation in TomOpt

for each voxel layer do &nbsp;&nbsp;&nbsp;&nbsp;for each muon do &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;while muon inside layer do &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Compute θ0(dr,X0)\theta_0(dr, X_0) (1), δθμ\delta\theta_\mu (2), (δxμ,δyμ)(\delta x_\mu, \delta y_\mu) (3) ▹\triangleright μ\mu denotes variables computed in the muon reference frame &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Compute dθvold\theta_{\mathrm{vol}}, dϕvold\phi_{\mathrm{vol}}, δxvol\delta x_{\mathrm{vol}}, δyvol\delta y_{\mathrm{vol}}, δzvol\delta z_{\mathrm{vol}} ▹\triangleright vol denotes variables computed in the volume reference frame &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;x←x+δxvolx \leftarrow x + \delta x_{\mathrm{vol}} ▹\triangleright Update muon position &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;y←y+δyvoly \leftarrow y + \delta y_{\mathrm{vol}} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;z←z+δzvolz \leftarrow z + \delta z_{\mathrm{vol}} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;θ←x+δθvol\theta\leftarrow x + \delta\theta_{\mathrm{vol}} ▹\triangleright Update muon direction &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ϕ←y+δϕvol\phi\leftarrow y + \delta\phi_{\mathrm{vol}} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;if zμ<zbottomz_\mu< z_{\mathrm{bottom}} then &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;zμ=zbottomz_\mu= z_{\mathrm{bottom}} ▹\triangleright Muons that get propagated outside of the layer are moved (up) to the bottom of the layer, xx, yy and zz positions are updated according to θ\theta and ϕ\phi &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;end if &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;end while &nbsp;&nbsp;&nbsp;&nbsp;end for end for

Muon positions (referred to as hits) are recorded in xyzxyz by the panels. The effect of spatial resolution on the hits will depend on the detector technology used. For simplicity, the panels implemented in TomOpt assume that the recorded position of a hit can be sampled from a Gaussian distribution G(μ=xytrue,σ=σxy,eff)\mathcal{G}\left(\mu= xy_{\mathrm{true}}, \sigma= \sigma_{xy,\mathrm{eff}}\right) centered over the true position of the muon xytruexy_{\mathrm{true}}, the scale of which depends on the effective spatial resolution σxy,eff\sigma_{xy,\mathrm{eff}}, as dictated by σmax⁡\sigma_{\max} of the detection panel, and the position of the muon relative to the panel centre.

In reality, a muon passing through a panel would either record a hit, or not, depending on the efficiency of the panel. For differentiability reasons, TomOpt instead takes a probabilistic interpretation: hits are always recorded, but with an efficiency factor ϵhit\epsilon_{\mathrm{hit}}. The total efficiency of the muon ϵμ\epsilon_\mu is then computed given its individual hits as the probability of at least two hits before and after the passive volume.2 Both the hit and overall efficiency enter various parts of the inference and optimisation process, and will be discussed in later sections.

Optimisation mode Optimisation of the panel parameters requires that the muon trajectories be differentiable with respect to them. Trajectory reconstruction (discussed next) will naturally be differentiable with respect to the zz position of the hits, however derivatives for the xyxy-position and xyxy-span are non-trivial: these affect only whether a muon hits a panel or not, and hits are then recorded with a constant resolution and efficiency, i.e. the optimisation process would be insensitive to muons which could have been recorded but were not due to the position and size of the panel.

Instead, during optimisation mode, TomOpt takes an unphysical approach in which “panels” will always record a hit for every muon at its current zz position, even if these hits are located outside the panel. The effective resolutions σxy,eff\sigma_{xy,\mathrm{eff}} and efficiencies ϵeff\epsilon_{\mathrm{eff}} of hits, however decreases the further away the hits are from the centre of the panel, at a rate that scales with the span of the panel. They are computed by scaling the maximum panel resolution σmax⁡\sigma_{\max} and efficiency ϵmax⁡\epsilon_{\max}, according to where the muon passes the panel’s z-position:

σxy,eff=σmax⁡/w(xytrue),ϵeff=ϵmax⁡⋅w(xytrue),(7)\sigma_{xy,\mathrm{eff}} = \sigma_{\max}/w(xy_{\mathrm{true}}), \quad\epsilon_{\mathrm{eff}} = \epsilon_{\max} \cdot w(xy_{\mathrm{true}}), \tag*{(7)}

where w(xytrue)w(xy_{\mathrm{true}}) is the panel model evaluated at the true muon-position, as presented in Figure 3. This scales between zero and one, reaching a maximum when the muon hits at the centre of the panel. The current model used in TomOpt is a pair of double-sigmoid functions in xx and yy. The values of this pair are used to provide resolutions in xyxy, and their product provides the hit efficiency. These are then scaled such that a hit at the centre receives the full resolution and efficiency of the panel. Effectively, this can provide detection performance close to the actual detector for hits within the panel, and a smooth transition to zero efficiency and resolution outside the panel. In this way, each hit and muon trajectory will be differentiable with respect to all parameters of the panel. When derivatives are not required, e.g. for validating the detector, the panels can of course be reverted to a more physical interpretation. The algorithm 2 presented below summarises TomOpt hit recording implementation.

Detector modeling

Figure 3. Example of detector panel modeling with sigmoid function used during optimisation (blue) and with rectangle function used during validation (red).

The smoothness of the sigmoid model (how slowly the hit performance transitions at the panel edge) can be adjusted, and even evolved during optimisation. Inference with smoother panels will be more sensitive to muons that are far from the panel, and so are more suitable for optimising the xyxy position and span of the panels: since the inference will account for the efficiency on the muons, muons far from the panel are able to affect the inference. Panels with a sharper transition have close to full hit performance within the majority of the area of the panel, and so inference is more sensitive to the zz position of the panels: muons far from the panel receive diminished weights, and muons inside the panel receive a more uniform efficiency and resolution therefore the only remaining parameter to have non-negligible gradient is the zz-position. Again, the user is free to implemented their own panel model ww to reach the desired detector configuration. Examples could be different scalings in xx and yy, or a periodic function to represent detection wires with a given density.

Algorithm 2Hit recording in TomOpt
for each muon do
&nbsp;&nbsp;&nbsp;&nbsp;for each detector panel do
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Propagate muon to the next detector panel at zjz_j
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Record true hit: (xtrue,ytrue,zj)(x_{\mathrm{true}}, y_{\mathrm{true}}, z_j)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Compute effective hit resolutions: σx,eff=σmax⁡/w(xtrue)\sigma_{x,\mathrm{eff}} = \sigma_{\max}/w(x_{\mathrm{true}}), σy,eff=σmax⁡/w(ytrue)\sigma_{y,\mathrm{eff}} = \sigma_{\max}/w(y_{\mathrm{true}})
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Compute reconstructed hit: (xtrue+δx,ytrue+δy,zj)(x_{\mathrm{true}}+\delta_x, y_{\mathrm{true}}+\delta_y, z_j) ▹\triangleright δx,y\delta_{x,y} sampled from G(0,σx,y,eff)\mathcal{G}(0,\sigma_{x,y,\mathrm{eff}})
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Compute hit efficiency ϵeff=ϵmax⁡⋅w(xtrue)⋅w(ytrue)\epsilon_{\mathrm{eff}} = \epsilon_{\max} \cdot w(x_{\mathrm{true}}) \cdot w(y_{\mathrm{true}})
&nbsp;&nbsp;&nbsp;&nbsp;end for
end for

Table 2.

Trajectory fitting and Point-of-Closest-Approach finding

The aim of trajectory fitting is to determine the paths of each muon as they enter and exit the passive volume using the hits recorded by the two sets of detector panels. Each trajectory is computed via analytic minimisation of a likelihood function that takes the xyzxyz hits as an input. When computing the fit, a weight is associated to each of the hit positions, that depends on the effective xyxy resolution of the hits σxy,eff\sigma_{xy,\mathrm{eff}}, and their efficiencies ϵeff\epsilon_{\mathrm{eff}}, i.e. low-resolution, or low efficiency, hits have less of an impact on the trajectory fit than higher resolution, or more probable hits. No uncertainty in zz is considered. From an optimisation perspective, the true trajectory of the muon is most likely to be recovered when the detector panels are positioned such that all hits for the muon are recorded with high efficiency and resolution. Potentially this would come at the cost of a poor reconstruction of other muon tracks, and so a global compromise must be found, as determined by the loss function of the muon tomography task being performed.

Once the incoming and outgoing trajectories of the muons are computed, scattering variables such as the incoming and outgoing θ\theta and ϕ\phi angles, and the total amount of angular scattering may be differentiably computed. By exploiting the differentiable computation of these variables, their associated uncertainties can also be easily computed using the product of their partial derivatives with respect to the recorded hits and the xyxy uncertainties on the hits.

Once muon tracking is performed, TomOpt proceeds to a first inference phase in order to associate a region of space with the measured muon deflection. It is done using an extended version of the Point-of-Closest-Approach algorithm, which is a standard in muon scattering tomography. In this approach, the entirety of the muon scattering is assumed to have occurred at a single point in the passive volume, located by extrapolating the inferred trajectories, as shown in Figure 4. Because the PoCA points are computed from the incoming and outgoing reconstructed tracks, they also bear an uncertainty in xx, yy and zz. The PoCA points may then be used for a variety of purposes, such as: directly producing a 3D image for human interpretation; or computing hit densities in voxels, perhaps scaling their weights by their associated angular scattering. Exactly which variables are required, and the way in which they are used depends on the task-specific inference approach used. We will show an example of this in § 3.3.1.

Example of PoCA point finding

Figure 4. Example of PoCA point finding. When the scattering angle is large, the PoCA point is well defined and is inside the passive volume (left). For almost parallel incoming and outgoing tracks, PoCA point might be reconstructed outside of the volume of interest.

Volume inference

The purpose of “volume inference” is to convert the muon-trajectory variables into a (set of) prediction(s) for the volume. The form of these predictions and the inference method used will depend on the muon tomography task being performed. A variety of methods may be used here, provided they allow the predictions to be differentiable with respect to the muon-trajectory variables, and a range of these will be explored in a second

Algorithm 3 Track fitting and PoCA finding in TomOpt

  • for each muon do

    • for incoming and outgoing tracks do

      • Fit track vector

    • end for

    • Compute PoCA point from the track vectors

    • Compute PoCA point uncertainty via automatic differentiation

  • end for

publication.

Our implementation of PoCA in TomOpt aims to compute estimates of the voxel X0X_0 values by inverting the PDG scattering model. The basic approach for a given batch of muons with their PoCA located in a given voxel of height δz\delta z is:

θ0rms=θtotrms2,(8)\theta_{0}^{\mathrm{rms}} = \frac{\theta_{\mathrm{tot}}^{\mathrm{rms}}}{\sqrt{2}}, \tag*{(8)}
θ0rms=0.0136 GeVprmsnX0rms,(9)\theta_{0}^{\mathrm{rms}} = \frac{0.0136\ \mathrm{GeV}}{p^{\mathrm{rms}}} \sqrt{n_{X_0}^{\mathrm{rms}}}, \tag*{(9)}
nX0rms=δzX0cos⁡(θˉrms),n_{X_0}^{\mathrm{rms}} = \frac{\delta z}{X_0 \cos\left(\bar{\theta}^{\mathrm{rms}}\right)},
θˉrms=θinrms+θoutrms2,\bar{\theta}^{\mathrm{rms}} = \frac{\theta_{\mathrm{in}}^{\mathrm{rms}}+\theta_{\mathrm{out}}^{\mathrm{rms}}}{2},
X0=(0.0136 GeVprms)2δzcos⁡(θˉrms)2θtotrms 2.(10)X_0 = \left(\frac{0.0136\ \mathrm{GeV}}{p^{\mathrm{rms}}}\right)^2 \frac{\delta z}{\cos\left(\bar{\theta}^{\mathrm{rms}}\right)} \frac{2}{\theta_{\mathrm{tot}}^{\mathrm{rms}\,2}}. \tag*{(10)}

Where θtotrms\theta_{\mathrm{tot}}^{\mathrm{rms}} is the root-mean-square (RMS) of the measured scattering angles. In Eq. (9), θinrms\theta_{\mathrm{in}}^{\mathrm{rms}} and θoutrms\theta_{\mathrm{out}}^{\mathrm{rms}} respectively refer to the incoming and outgoing muon zenith angle. Note that the natural log term present in Eq. (1) has been ignored to simplify the inversion. The effects of efficiency and resolution may be further included in Eq. (10) by instead aggregating over the muons with a weighted RMS, where muon-wise weights are computed as the efficiency divided by the variance of the squared value of the variable being aggregated.

In the above inference, we only considered muons whose PoCA was located inside the voxel in question, however we have an uncertainty on the PoCA position, and some muons may have a PoCA located outside the passive volume due to mis-reconstruction. An extension is then to consider that muons can contribute to inference in multiple voxels by considering that the PoCA could have occurred in voxels surrounding its nominal location. The PoCA uncertainty PoCAunc\mathrm{PoCA}_{\mathrm{unc}} in xx, yy and zz is computed via automatic differentiation. Then the probability density function (PDF) for a PoCA to be located within a voxel is modelled using 3 independent Gaussian distributions centered over the nominal PoCA location and scaled in the three directions according to PoCAunc\mathrm{PoCA}_{\mathrm{unc}}. A “scattering probability” is then computed per voxel for each muon, by integrating the PoCA PDFs over ever voxel in the passive volume. An illustration of this procedure is presented on Figure 5. This probability then enters into the weighted RMS as a multiplicative coefficient.

Illustration of PoCA point being extended to other neighbouring voxels with a Gaussian model

Figure 5. Illustration of PoCA point being extended to other neighbouring voxels with a Gaussian model. For the sake of readability, only the extension along the yy dimension is shown.

Algorithm 4Volume inference in TomOpt
for each PoCA point do
Compute scattering probability per voxel
Compute muon weights per voxel as muon efficiency times scattering probability in each voxel
end for
for each voxel do
Compute weighted θ0rms\theta_{0}^{\mathrm{rms}} from Eq. (8)
Compute X0X_{0} from Eq. (10)
end for

Table 4.

This method of using PoCA points, however, is biased to underestimate X0X_{0}, since the entirety of the scattering is assumed to happen in one scattering event. The muons of course undergo many scattering events as they traverse through the volume. Additionally, it converts muon-wise data into voxel-wise predictions, however the muon tomography task may not require voxel-wise predictions, but instead perhaps predictions concerning the volume as a whole. Nonetheless, it can serve as a useful and generic first-stage of inference to serve as input to a second, more task-specific, stage of inference; indeed both of the benchmarks presented in this paper take this approach. TomOpt implements more advanced methods such as an Expectation-Maximisation algorithm, where the strong (and often unrealistic) assumption of a single scattering centre is dropped and all most likely paths of the muon are taken into account with a proper weight [35].

Alternatively, deep-learning based methods can be applied to muon-level data. Here, latent-space representations of the relevant muons in each voxel can be learnt using graph-neural-networks that are able to learn their own graphs in a latent-coordinate-space, such as those proposed in Ref. [36]. These representations can then be refined per voxel, based on the surround voxels through a learnable higher-level graph. An initial investigation of this is presented in Ref. [37]: the representation in each voxel is then refined based on the surrounding voxels. More simply, deep-learning methods can be used as a second stage to X0X_0 inference, to conveniently convert the dimensionality of voxel-level data to volume-wise predictions. Convolutional neural networks are appropriate for this task, as demonstrated during the Second MODE Workshop data-challenge [30].

Loss functions and optimisation

Similar to the volume inference, the most suitable loss function for the optimisation will depend on the muon tomography task being performed, e.g. cross-entropy for classification tasks. The total loss function, however, should not only consider the performance of the detector, but also its budget, for example the cost in currency units that the detector would take to build given the surface area of its detectors. The cost could enter directly into the loss via a scaling coefficient, however it is more likely that the user already has in mind a particular target budget, and wants to design the best detector that gets close to this budget.

TomOpt implements two approaches: fixed-budget, where the user specifies the maximum budget allowed and the optimisation is constrained to not surpassing that value; and budget-penalisation, where the user specifies a loose target budget.

In the budget-penalisation case, a cost component of the loss added according to a function that is low up to the target budget, and then begins to rapidly increase if the detector exceeds the target budget. In this approach, the detector design can go over budget, but only if the increase in performance vastly surpasses the budget excess, as exemplified in Figure 6. This mode introduces hyper-parameters for the weight of the new component in the total loss, and the functional form of the penalisation given the cost and budget.

Example of the cost component of the loss as a function of detector cost

Figure 6. Example of the cost component of the loss as a function of detector cost, for a target budget of 10 A.U. A sigmoid component below the budget provides a slowly increasing gradient as the cost approaches the target budget, beyond which the loss increases linearly. The function is fully differentiable.

Alternatively, in “fixed-budget optimisation mode”, the user still specifies a total and fixed cost for the detector, and a fractional assignment of this cost to each panel is learnt, starting from a uniform assignment. The panels still learn a separate xyxy span, however the the area of the panel is then rescaled such that the cost of the panel is equal to its assigned fraction of the cost of the detector. This mode of optimisation has two main advantages: the user is able to ensure that they meet their budget exactly; there is no balancing coefficients required in the loss function between performance and cost, the loss is purely based on performance.

Once the value of the loss function has been computed, the analytic effects of each detector parameter can be computed through back-propagation [38, 39, 40] of the the derivative of the loss through the inference and hit-recording steps. The parameters may then be updated via stepping in the direction of steepest descent of the loss surface. For this process, TomOpt leverages the standard optimisers built into PyTorch (SGD [41], Adam [42], etc.).

Training is supported by a callback system, which allows aspects of the modules and the training procedure to be adjusted without requiring new modules to be written.

Update cycle

When computing a value for the loss function, it is important to ensure that it is computed under conditions that are generally representative of those that can be encountered during actual operation, in terms of e.g. number of muons and passive volumes. In effect, this means that optimisation should be performed using a range of passive volumes (whether generated on demand, or supplied by the user as a dataset), and that these should each be scanned using a number of muons that would be reasonable to expect given the position and exposure time of the proposed detector. Additionally, rather than computing the loss using just a single passive volume, it should instead be averaged over many passive volumes in order to provide a more generalising update to the detector parameters.

The update cycle in TomOpt is performed thus in the following way:

  1. Figure 7c: Scanning of a given passive volume involves passing a batch of muons through the volume and inferring their tracks. For computational reasons, the total number of muons may be split into smaller batches. Once all muons have been processed, the volume predictions can be made and the loss computed;

Figure 7. Breakdown of the fitting procedure of detectors in TomOpt

  1. Figure 7b: This process is repeated for a batch of volumes, and the sum (or average) of their losses can be used to update the detector parameters. This is referred to as an epoch, in keeping with Machine Learning terminology3;

  1. Figure 7a: The fit runs for a specified number of epochs before finishing. It is also possible to run epochs in validation mode, where the detector is not updated: the detector panels can be therefore considered, more realistically, as discrete detector panels rather than with an associated distributed resolution and efficiency, as discussed in § 2.3.

The appendix in § 5 shows an overview of each of the steps presented above.

Current limitations

Muon propagation through matter

As described in § 2.2, the implementation of multiple Coulomb scattering is rather straight-forward since the computation efficiency is prioritised over precision. Scattering angle and displacement distributions are assumed to be a 100 % Gaussian, where in reality only 98% of the bulk is Gaussian and the tails follow a 1/θ41/\theta^{4} law corresponding to large scattering events. In addition, The PDG model used is not designed for a step-by-step algorithm: when it is applied to such an algorithm, it results in underestimating the scattering and displacement amplitudes more and more with each step, as shown in Figure 8.

Comparison of standard deviations of projected scattering angles and displacement distributions between TomOpt and Geant 4 for various number of propagation steps

Figure 8. Comparison of standard deviations of projected scattering angles and displacement distributions between TomOpt and Geant 4 and for various number of propagation steps.

It becomes clear that the choice of step length drdr affects the precision of the multiple coulomb scattering, and thus has to be chosen carefully. In this context, it is advised to choose a step length drdr which represent about 1% of the passive volume size.

In addition to multiple Coulomb scattering, muons lose energy through ionisation thus sub-GeV muons might decay in the passive volume. Energy loss has not been implemented in TomOpt yet, which results in a overestimation of the detected muon flux. Both of these limitations could be overcome by building a parametric model of Geant 4. Creating a data-base of muon energy loss, scattering and displacement for various material and at several energy scales from Geant 4 simulations, TomOpt could simulate muon transport more accurately and effortlessly.

Detector and cost modeling

TomOpt’s ambition is to guide the user through the choice of detector technology and not to provide accurately simulation of particle detection, signal generation or light transport. Thus, the difference between various detector technologies will be implemented at the level of cost modelling, meaning how the cost scales with the active detection surface and number of readout channel. Such features have not been implemented yet, but are planned to be included in future updates.

Regarding detector modelling, TomOpt is still limited to horizontal panels with continuous spatial resolution and uniform granularity.This limits the phase space of possible detector geometry.

Volume inference

For now, TomOpt only provides a basic implementation of radiation length inference based on the inversion of the PDG scattering model utilising the Point of Closest Approach. The inversion of the scattering model Eq. (10) assumes muon momentum to be known with a 100% precision, which is quite unrealistic in the context of a muon scattering tomography experiment. If a momentum estimation is available, the user can still modify the inversion model and replace the nominal muon momentum by a the estimated momentum. If not, it can be replaced by the RMS computed over the whole cosmic muon momentum spectrum.

Benchmark: Fill-level estimation at metal refinery

The following section describes a benchmark involving the estimation of the fill level at a metal refinery, and aims at demonstrating a typical inference and optimisation chain using TomOpt. After evaluating the performance of an initial sub-optimal detector configuration, the latter will be optimised, and its performance will be compared to two human baseline detector configurations.

Description

Furnace ladles are structures used to transport melted metal produced in an industrial plant, as shown in Figure 9. A description of a generic industrial furnace ladle model used to simulate muography measurements can be found in [43]. A typical example of the part of the manufacturing process which involves the use of the ladle could be described in a general way as follows: firstly the metal is melted in a furnace, then it is transported with a furnace ladle, and finally it is poured into moulds to create metal parts with a defined shape. This parts can later be post-processed, in order to get the final product.

Example of a furnace ladle used in foundry industry

Figure 9. Example of a furnace ladle used in foundry industry. Image by Boris Bukovský, via Pixabay.

In this process, the amount of melted metal inside the furnace is a key parameter. The lack of metal in the ladle can result in moulds that are not completely filled. Therefore, they may not be used to produce the final product correctly. On the other hand, an excessive charge of metal in the ladle leaves remnants and scraps, causing problems and losses in the manufacturing process. It has to be mentioned that usually, in the top of the melted metal inside the ladle, there is a lighter layer of slag produced in the melting process itself. This layer, prevents the use of optical methods to determine the liquid metal level. The resolution needed by the industry to optimise the manufacturing process is of the order of a few centimetres, although it depends on the distinctive features of each case.

Detector and volume

Active layers, i.e., the detector panels, are located above and below the measurement target, in an usual scattering muography configuration. At least two panels at different zz locations providing x−yx-y positions of muons are required per tracking detector (upper, and lower detectors). As specified in § 2.3, they are modelled considering several parameters, such as position resolution σmax⁡\sigma_{\max}, detection efficiency ϵmax⁡\epsilon_{\max}, and cost per unit area.

Within the volume of interest between the detectors, a rectangular furnace ladle filled with liquid steel has been simulated (passive volume). In the Figure 10 a sketch of the furnace ladle is shown. The grey parts correspond to ladle walls, made of solid steel, the molten steel is represented in yellow, the slag layer on top of it in orange, and the remaining void volume over them in blue. The definition of materials is detailed in Table 1.

Top and front views of the rectangular furnace ladle

Figure 10. Top view (left), and front view (right) of the rectangular furnace ladle simulated. It is composed of walls (grey), liquid steel (yellow), slag (orange) and the air volume (blue) over the liquid steel and slag.

MaterialDensity, dd [g cm−2^{-2}]Radiation length, X0X_0 [cm]Composition
Solid steel7.8181.782FeFe (99%), CC (1%)
Liquid steel7.0001.991FeFe (99%), CC (1%)
Slag2.7008.211CaOCaO (53.5%), Al2O3Al_2O_3 (33.5%)
Air0.00120530390N2N_2 (78%), O2O_2 (20%)

Table 1. Density, radiation length and composition of the simulated furnace ladle materials.

Inference method and loss function

The aim of this task is to design a detector that can provide an accurate measurement for the fill-heights of furnaces. This is essentially a single-variable regression task, and a natural choice for the loss function could be the mean squared-error (MSE) between the predicted and true fill-heights. However, as will be discussed below, MSE is not necessarily the most suitable loss for optimisation of this task.

Inference

In the spirit of an end-to-end optimisation pipeline, the inference must output the fill-height prediction and receive optimisation loss gradients based on this. In developing a suitable inference method, we noticed that the mean value of the zz-positions of PoCAs (z‾p\overline{z}_p) was monotonic with the fill-height, due to hard scatterings occurring inside the liquid-metal part of the furnace volume, rather than the air and slag regions. This meant that it could serve as an indicator of the mass density distribution centre in the vertical ZZ axis (see Figure 11). As a benefit, this inference approach does not require computing X0X_0 predictions for the voxels, and thus is more computationally efficient.

Muography of a simulation ladle furnace filled with steel

Figure 11. Muography of a simulation ladle furnace filled with steel up to 0.4 m0.4\,\mathrm{m}. The bottom wall of the furnace goes from 0.2 m0.2\,\mathrm{m} to 0.3 m0.3\,\mathrm{m} in zz axis. On the left, the front view of PoCA reconstruction. On the right, PoCA distribution in vertical axis (zz) and filling level inference example: true steel level (blue), zˉp\bar{z}_p (red), and hpredh_{\mathrm{pred}} (green) for the particular case of a bias correction on zpredz_{\mathrm{pred}}, with a=2a=2 and b=−0.2b=-0.2 (see § 3.4 for details).

This can be adapted to an inference method suitable for detector optimisation by computing a weighted average of the PoCA zz-positions, according to the muon-trajectory properties. The basis for this weight is the muon-trajectory efficiency (ϵp\epsilon_p), divided by the squared uncertainty of its PoCA zz-position ((Δzp)2(\Delta z_p)^2).

However, we also observed that PoCAs located in the furnace walls could bias the prediction, and reduce precision. Additionally, we know we have uncertainties on the xyxy positions of the PoCAs. In a similar manner to the X0X_0 inference described in § 2.5, we define Gaussian distributions in xyxy centred on the PoCAs and scaling with their xyxy uncertainties, and integrate them over each voxel in the transverse area of the furnace. We then use a pair of double-sigmoid-based functions in xyxy to ascribe a weight to each voxel, that smoothly decreases to zero at the furnace walls, and maximises to one at the centre. This is used to multiply the integral in each voxel per PoCA. The xyxy weight of each PoCA (wxy,pw_{xy,p}) is defined as sum of its weighted voxel integrals, and this enters as a multiplicative factor of the overall PoCA weight:

wp=ϵp(Δzp)2wxy,p.(11)w_p = \frac{\epsilon_p}{(\Delta z_p)^2}w_{xy,p}. \tag*{(11)}

This weight has the effect of emphasising the zz positions of PoCAs that are well measured in zz, located near the centre of the furnace, and have a high trajectory efficiency.

The predicted height is then taken as the weighted average of PoCA zz positions:

zpred.=∑pwpzp∑pwp.(12)z_{\mathrm{pred.}} = \frac{\sum_p w_p z_p}{\sum_p w_p}. \tag*{(12)}

Whilst the mean of PoCA zz positions is actually more likely to correspond to the centre of the fill-material, this redefinition to the fill-height can be accomplished by the linear transformation described in the next section.

Bias correction

The method described above, does not perfectly predict the fill-height: as mentioned, the mean of PoCA zz positions is likely to correspond to the centre of the fill material, but with some finite precision; the PoCA method is inherently biased through its ascription of the entire muon scattering to a single point; additionally, the bias on the prediction is detector-configuration-dependent.

Provided that the biased prediction increases with the true fill-height, bias correction can be performed for a given detector configuration using many volumes of different fill-heights and adjusting the predictions (e.g via a continuous function, or bin-based corrections) such that on average, the predictions match the targets. Knowing this, what we really want the inference and detectors to produce is a prediction that has low in standard deviation when repeated multiple times for the same volume, whilst still increasing in mean value with the true fill-height.

Eventually, though, we will need the detector to be used for actual measurements of fill-heights. At this point, the predictions from the inference must be de-biased as well as possible, in order to correspond to the true targets. The de-biasing can take the form of a parametric function of the predictions, and for simplicity we opt for a linear correction of the form:

hpred.=a×zpred.+b,(13)h_{\mathrm{pred.}} = a \times z_{\mathrm{pred.}} + b, \tag*{(13)}

where parameters aa and bb are computed based on the predictions and targets from a large set of known example volumes at various fill-heights. The effect of this de-biassing can be seen in Figure 14.

Figure 14. Performance of the initial detector before and after the de-biasing process.

In order to improve monitoring capabilities during the optimisation, we refine the values of aa and bb after each update to the detector and display the resulting MSE of de-biased predictions. In order to avoid having to re-scan a large number of volumes, this refinement is performed using predictions and targets collected during the current batch of volumes.

Loss function

Even with the de-biasing, the detectors can still produce predicted heights that contain a residual bias, and optimisation under MSE risks being sensitive to these residual biases, resulting in unstable optimisation. As mentioned, the aim of the detector optimisation is to eventually allow the inference to produce precise predictions that increase with the true fill-height. A loss function that is suitable for this is one that captures the distribution of predictions for each target fill-height, i.e.:

L=1Nl∑lσ(pl),(14)\mathcal{L} = \frac{1}{N_l} \sum_l \sigma(p_l), \tag*{(14)}

where σ(pl)\sigma(p_l) is the standard deviation of predictions for fill-height index ll. If the spread of predictions becomes smaller, the loss will decrease, and vice versa, thus the loss encourages precise measurements of the fill-heights.

However, there is nothing to ensure that the predictions increase in value with the fill-height: given sufficient flexibility in the detector (and potentially inference), it may be possible that the system outputs the same value for all predictions, thus proving zero spread, and so a perfect loss value. In order to penalise such a collapse, we need to include a factor that encourages a large distribution in predictions for different fill-heights, thus the final loss function becomes:

L=1Nl∑lσ(pl)σ(pl‾),(15)\mathcal{L} = \frac{1}{N_l} \frac{\sum_l \sigma(p_l)}{\sigma(\overline{p_l})}, \tag*{(15)}

where σ(pl‾)\sigma(\overline{p_l}) is the standard deviation of the mean prediction for each fill-height.

If the system were to output the same value for every volume (thus minimising the spread of predictions within each fill-height), the spread of mean predictions for each fill-height would also collapse to zero and act to increase the loss value. The loss function is therefore minimised when the distributions of predictions in each fill-height is small, and when the mean predictions between fill-heights is as large as possible. Thus the predictions become easier to de-bias: if the predicted height for a given volume is within a certain range, we can be more sure it corresponds to a particular true fill-height. We refer to Eq. (15) as the “spread over range” loss function.

In the Figure 12, the evolution of the proposed loss function with relation to the span of detection panels is shown. The centre of the panels in the horizontal axes X and Y, is the same that the one of the furnace ladle. A sharp loss decrease is noticed when detection panel surface is increased within the limits of the furnace ladle interior, where the liquid steel is located, i.e., the muography target. However, if the panel span exceeds the width of the ladle interior, the loss reduction starts to be less noticeable. We found that this effect is due to the smaller proportion of muons that cross the target when detectors are larger than the furnace interior width, and also because of the contribution of furnace walls to muon deflection. Their presence modifies the density distribution in zz axis, distorting our predictions, although this effect is smoothed by the weighted PoCA inference (see § 2.5). Either way, the “spread over range” loss is continuously and endlessly reduced when increasing the detector surface, the expected behaviour regarding the performance of a muography, since a larger surface allows to collect a higher number of meaningful muons, even if they traverse the target in a transversal direction. This generally improves the precision of inference algorithms. Note that in the “spread over range” part of the loss function is not included the loss component of detection surface cost (see § 2.6).

“Spread over range” loss function tested for different detection surfaces

Figure 12. “Spread over range” loss function tested for different detection surfaces, and using the hpredh_{\mathrm{pred}} inference method with unweighted zpredz_{\mathrm{pred}} and a linear de-biasing of a=2a = 2 and b=−0.2b = -0.2.

Optimisation process and results

Passive-volume data

Using 10 cm voxels, we generate volumes at eight different fill-heights to simulate from ladles that are almost empty to ones that are almost full. In every case, we add a 10 cm layer of slag on top. When optimising the detector, each of these volumes will be evaluated five times (40 volumes per volume batch) – whilst the volumes will always be the same, stochasticity in prediction will still be present due to the muon generation. When evaluating the performance of the detectors, each volume is evaluated 12 times. 1000 muons will be used per evaluation.

Initial configuration

In order to test the optimisation capabilities of TomOpt, we initially set the detector in a knowingly sub-optimal configuration: one in which the detector panels are off-set from the passive volume in x−yx-y, thus lowering the flux muons that are well detected, and interact with the passive volume; and closely space in zz, thus increasing the uncertainty in PoCA reconstruction.

The detector itself (illustrated in Figure 13) consists of four equally sized square panel above the passive volume, and another four panels below. These panels begin centred in x−yx-y over the corner of the passive volume, and with an (unrealistic) separation in zz of 1 cm between each panel. There is a 5 cm separation between the passive volume to the nearest panels. The total cost of the detector is fixed at six arbitrary units, and panels cost one arbitrary unit per m2^2. The available budget is initially distributed equally to all panels, thus each panel measures 86.6 cm in both xx and yy length. Panels are defined to have nominal efficiencies of 90 %, and x−yx-y resolutions of 0.1 mm.

Initial layout of the detector panels above and below the passive volume

Figure 13. Initial layout of the detector panels above and below the passive volume.

The initial performance for the detector is shown in Figure 14. The “raw” predictions are the result of the inference with no de-biasing. We can observe a slight slope to their values w.r.t. the fill-height, however the large spread in predictions for the same target, means that it is difficult to distinguish different fills. The de-biasing process via a linear correction is able slightly reduce the MSE, but cannot do much more that shift the predictions. Averaging across the range of fill-heights considered the initial detector has a mean MSE of 0.037 m2^2, and an average absolute error on fill-level prediction of 16 cm.

Our intuitive expectation is that from this position, the panels should move to cover the centre of the passive volume, and increase in zz-separation. There is, however, a trade-off in increasing the zz-separation: whilst it generally increases the PoCA resolution, it also decreases the hit-reconstruction for muons entering or leaving the passive volume at large angles. We therefore expect the optimisation to find a natural point at which the improved resolution is balanced by the decreased high-angle-muon flux. Additionally, we have no confident intuition over how the panels should be resized to best improve performance given a constant budget.

Optimisation callbacks

As mentioned in § 2.6, TomOpt has a callback system that allows the user to augment the basic optimisation loop to include different functionality, and to help stabilise the optimisation process. For this task, the following callbacks are used:

  • MuonResampler – When muons are generated, they may not ever interact with the passive volume, and so form a source of background, which would ideally be removed by some filtering process. In the interests of simplicity, we are not considering background processes here4. The MuonResampler callback acts to regenerate muons until all muons in the batch will interact with the passive volume at some point. This therefore improves the generation efficiency.

  • LinearCorrection – The de-biasing process mentioned in § 3.4 is implemented as a callback that can modify in-place the outputs of the inference module, collect raw predictions and targets during the volume batch, and then re-fit its correction after each detector update.

  • SpreadRangeLoss – By default, TomOpt computes the loss per volume and tracks a running sum of losses over the volume batch. However the “spread over range” loss can only be computed for a population of volume predictions. This callback collects the predictions over the course of the volume batch and then sets the value of the loss prior to back-propagation.

  • NoMoreNaNs – Optimisations can take several hours, and whilst TomOpt is fairly resilient, given the number of muons being propagated and the complexity of the reconstruction and inference process, it is inevitable that invalid (NaNNaN) gradients are

occasionally produced. These would otherwise kill the optimisation process, however after back-propagation but before the parameter updates, this callback goes through the parameters and sets any NaNNaN gradients to be zero.

  • OptConfig – The learning rate of optimisers in gradient descent is one of their most important parameters, however it can be difficult to set suitably with limited a priori knowledge. Whilst techniques such as the “learning rate range finder test” from Ref. [44] have been developed, they are designed for DNNs, where the learnable parameters lack a physical meaning. Here, our parameters do have physical meaning (e.g. the detector position is a learnable parameter measured in metres). We are thus able to do something better: we can specify the desired change per update of parameters in their physical units, and over the course of a warm-up phase (in which all parameters are fixed) compute the average gradient received by each parameter. From these averages, we can then compute the learning rate that is likely to produce the desired update rate. As an example we can say that we want the panel xyxy positions to update at a rate of 1 cm per iteration. We can then monitor the median gradients for a few iterations and then set an appropriate learning rate that results in a change of about 1 cm per iteration according to γ=1 cm/∇‾xy-pos..L\gamma= 1\,\mathrm{cm}/\overline{\nabla}_{xy\text{-pos.}}.\mathcal{L}.

  • PanelCentreing – In order to help stabilise the optimisation in TomOpt, we use this callback: after panel xyxy positions have been updated, it goes through and for panels above and below the passive volume separately, it computes the mean position in x−yx-y of panels and then sets panels to that position. Panels below the passive volume can still have different positions to those above, but all panels above and below each have a common x−yx-y position. Whilst this does limit the originality of detector configuration that TomOpt can explore, it is currently necessary to avoid instabilities and divergences in the optimisation.

  • PanelMetricLogger – This provides real-time feedback to the user, and displays the current layout of the detector, the history of the loss values, and other diagnostic information.

  • EpochSave – This automatically saves the state of the detector after every update, allowing the user to manually pick the best performing detector in case of performance divergence, or to recover in case of errors

  • OneCycle – This is a learning rate scheduler that adapts the learning rate (and optionally momentum/β1\beta_1 coefficients) or the optimisers according to the 1-cycle schedule described in Refs. [45, 46]. This is mainly used to allow the learning rate to gradually decrease over the course of the optimisation, allowing the detector configuration to better converge, whilst allowing larger updates near the beginning to training to move quickly from sub-optimal to near-optimal positions.

  • SigmoidPanelSmoothnessSchedule – as mentioned in § 2.3, the physical panels are approximated differentiably during the training epochs using a sigmoid-based model. It was also suggested that different values of the smoothness of the model allows different parameters of the detector panels more or less of an effect on the loss. This callback allows the smoothness to be decreased across the optimisation process, meaning that close to the end of the training, the panels better approximate physical detectors, and the loss is more sensitive to the exact zz position of the panels.

Optimisation process

Five parameter families are considered: xyxy positions of the detector panels and the zz positions of the panels are considered separately for panels above and below the volume (four parameter families); and the budget assignment per panel (one parameter family). Over the course of five epochs, we monitor the gradients received by the trainable parameters in these families and compute nominal learning rates that correspond to position updates at a rate of 1 cm per epoch, and budget weights at a rate of 0.1 (dimensionless).

Stage one Over the course of 10 epochs, the detector is updated, using 1-cycle learning rate annealing to scale the nominal learning rates over the course of the optimisation: initially they increase to allow large updates of the detector, and towards the end they decrease to much smaller values, allowing the panels to converge and stabilise. Additionally, the sigmoid detector model is annealed from an initially smooth configuration to a sharper, more physical distribution of efficiency and resolution. On a 2018 Macbook Pro with an Intel i7, this takes about 30 minutes to run (no GPU is used).

Figure 15 illustrates the detector configuration before and after this stage of the optimisation process. From this we can see that the panels have indeed moved to be more central in x−yx-y, and have expanded in zz.

Comparison of detector configurations before and after stage one optimisation process

Figure 15. (a) Initial detector configuration. (b) Detector configuration after stage one optimisation process. Comparison of detector configurations before and after stage one of the optimisation process. The coloured lines/squares indicate the positions and sizes of the panels, and the hatched area indicates the position and size of the passive volume. The top rows indicate panels above the passive volumes, and the bottom rows indicate panels below the passive volume.

Figure 16 compares the performance of the optimised detector to the initial detector, and indicates a significant improvement in both the precision and bias: now predictions are clearly distinguishable by true fill-height. The optimised detector has a mean MSE of 0.0014 m2^2, and an average absolute error on fill-level prediction of 2.8 cm.

Figure 16. Performance of the optimised detector after stage 1 compared to the initial detector (both shown after the de-biasing process). Heights shown include the space below the passive volume.

Stage two The previous optimisation process showed that TomOpt can take a poorly designed detector and quickly shift it into a configuration that provides much better performance. In this second stage we will demonstrate that TomOpt can be used to refine detectors that are already thought to be well performing, by altering only subsets of the total available parameters. To do so, we will use our detector from stage one.

During stage one, we saw that the panels moved to become central in x−yx-y. We will take this experimental evidence, along with our domain knowledge of the problem, to assume that the ideal position for the detectors is directly above and below the passive volume. We will manually shift the panel to the centre and fix them there, allowing only the zz and budget assignments to slowly change.

Once again, we compute suitable nominal learning rates for our three remaining parameter families by monitoring the gradients they receive for five epochs. Over another 10 epochs we allow the detector to further be refined, this time with smaller update steps. Figure 17 illustrates the detector after this refinement stage, and Figure 18 its performance. Mainly we can see that the panels below the passive volume have moved to cover a larger zz range. While this could come at the expense of missing some high-angle muons leaving the passive volume, the panels have slightly increased in size by decreasing the intermediate panels above the passive volume. The refined detector has a mean MSE of 0.0011 m2m^2, and an average absolute error on fill-level prediction of 2.5 cm.

Detector configuration after stage one optimisation process

(a) Detector configuration after stage one optimisation process.

Detector configuration after stage two optimisation process

(b) Detector configuration after stage two optimisation process.

Figure 17. Comparison of detector configurations after stage one (left) and stage two (right) of the optimisation process.

Figure 18. Performance of the optimised detector after stage 2 compared to the initial detector (both shown after the de-biasing process). Heights shown include the space below the passive volume.

Comparison to human baselines

Having demonstrated that TomOpt can successfully optimise detectors, we now want to see whether the resulting detector is good compared to a human-designed detector. For this we will take two baseline configurations:

  • Baseline 1: Detector panels are placed in pairs with a 5 cm separation between panels, and a 25 cm separate between pairs above and below the passive volume. This configuration aims to maximise the precision of the muon trajectory reconstruction by allowing precise measurements of the muon positions over a long baseline.

  • Baseline 2: Detector panels are placed with an even 10 cm spacing. Baseline 1 potentially misses some high-angle muons due to the large distance to the second pair panels. This baseline attempt to strike a balance between precision and exposure.

These are illustrated in Figure 19.

Baseline detector 1

(a) Baseline detector 1.

Baseline detector 2

(b) Baseline detector 2.

Figure 19. Configurations for the human-designed baseline detectors. The coloured lines/squares indicate the positions and sizes of the panels, and the hatched area indicates the position and size of the passive volume. The top rows indicate panels above the passive volumes, and the bottom rows indicate panels below the passive volume

Figure 20 compares the performances of the stage-two optimised detector against the two baseline detectors. Whilst all three detectors offer similar levels of performance, our optimised detector is able to not only match human performance, but slightly and consistently outperforms it.

Figure 20. Performance of the optimised detector after stage 2 compared to human-designed baseline detectors (both shown after the de-biasing process). Heights shown include the space below the passive volume.

Conclusions

In this document, we have described TomOpt; a system that allows muon-tomography detector designers to specify their end goal numerically as a task-specific loss function, and through the use of automatic differentiation optimise their designs to provide the best possible performance. Since the updates to the detector are made in the presence of both the inference algorithm and end-goal, the designers can be sure that changes made have a genuine impact on the final performance, rather than focussing on optimising task-agnostic proxy metrics (such as track resolution), and hoping that these eventually translate to better actual performance for the task at hand.

We demonstrated the capabilities of TomOpt in a simulation of one of the typical industrial use-cases – that of estimating the fill-level of furnace ladles at a metal refinery. Using task-specific inference and losses, we were able to optimise an initial, poorly designed detector into one that was able to not-only match, but slightly outperform two human-designed baseline detectors in terms of fill-height prediction precision. In this example we performed the optimisation such that the detector always conformed to a fixed cost, however alternatively the optimisation could be performed to minimise the detector cost and not exceed a specified budget.

We believe that TomOpt provides the first demonstration of end-to-end-differentiable and inference-aware optimisation of particle physics instruments, and represents an important step in changing the way we design experimental apparatus. Given the number of free parameters considered here, it is worth mentioning that non-differentiable, black-box optimisers such as Bayesian optimisation (BO) could be well applicable. However given the parameter spaces of larger detectors, such as those at the CERN LHC, where perhaps more than 1000 parameters might be available for control, we believe that differentiable optimisation will be required. With that in mind, our primary of this paper was to demonstrate the viability of measurement-aware detector design in a differentiable pipeline.

In practical application on larger detectors, BO could potentially be used on a reduced parameter set to act as an initial global search to find a decent initialisation, after which full differentiable optimisation can be used on the larger parameter space to refine the detector. Given BO’s capability to more easily optimise categorical parameters, this initial search may be additionally beneficial in fixing such parameters prior to differentiable optimisation.

Looking to the immediate future of this work, there are many possible extensions. So far, we have relied on “classical” inference algorithms, but inference via deep-neural networks are naturally compatible with TomOpt’s requirement on differentiable processes. We will be exploring their use in future publications, as these can help to ease the burden of designing task-specific differentiable inference algorithms. Additionally, as a quick simulator of the physics, TomOpt can be exploited for developing and testing advanced (non-differentiable) inference algorithms for muon-tomography in general. The available degrees of freedom of detectors, in terms of placement, technology, and modelling needs to be expanded, to allow for e.g. detectors to be placed on the side of the passive volume, or otherwise rotated. Finally, we will be looking to expand the repertoire of benchmark scenarios used for demonstration and testing.

Appendix

SymbolDefinitionUnit
θ\thetaMuon zenith anglerad, °
ϕ\phiMuon azimuthal anglerad, °
X0X_0Radiation lengthm
SgenS_{\mathrm{gen}}Muon generation surface
dxdx, dydyGeneration surface span in xx, yym
ErangeE_{\mathrm{range}}Muon generation energy rangeGeV
θrange\theta_{\mathrm{range}}Muon generation zenith angle rangeGeV
NmuonsN_{\mathrm{muons}}Number of generated muons
ϵmax⁡\epsilon_{\max}Panel maximum efficiency
σmax⁡\sigma_{\max}Panel maximum spatial resolution in xx and yym
xspan,jx_{\mathrm{span},j}, yspan,jy_{\mathrm{span},j}Panel jj span in xx and yy as a learnable parameterm
xjx_j, yjy_j, zjz_jPanel jj position in xx, yy, zz as a learnable parameterm
ϵeff\epsilon_{\mathrm{eff}}Reconstructed hit effective uncertainty
xrecx_{\mathrm{rec}}, yrecy_{\mathrm{rec}}Reconstructed muon hitm
ϵμ\epsilon_\muPer-muon efficiency
ϵbatch\epsilon_{\mathrm{batch}}Muon batch efficiency, sum of the per-muon efficiencies
L\mathcal{L}Total loss function
Lcost\mathcal{L}_{\mathrm{cost}}Detector cost loss function
Lerror\mathcal{L}_{\mathrm{error}}Error loss function
β\betaLoss function cost weight
Lvolume\mathcal{L}_{\mathrm{volume}}Volume loss function
XkX^kSet of all learnable parameters
{…,(xj,yj,zj,xspan,j,yspan,j),…}\{\ldots, (x_j, y_j, z_j, x_{\mathrm{span},j}, y_{\mathrm{span},j}), \ldots\} at iteration kk
∇X\nabla_XLoss function gradients w.r.t learnable parameters XX

Table 2 (PDF p. 30). Description of variables and learnable parameters from Figure 21, Figure 22 and Figure 23.

Scan loop for a batch of passive volumes (left), scan loop for muons over a single passive volume (right) and breakdown of the fitting procedure of detectors in TomOpt (below).

Figure 21. Scan loop for a batch of passive volumes (left), scan loop for muons over a single passive volume (right) and breakdown of the fitting procedure of detectors in TomOpt (below).

Simplified representation of the various computation steps from the fit epoch and scan volume methods

Figure 22. Simplified representation of the various computation steps from the fit epoch and scan volume methods, from Figure 21.

Figure 23. Simplified representation of the various computation steps from the fit epoch and scan volume methods, from Figure 21.

Acknowledgements

This work was partially supported by the EU Horizon 2020 Research and Innovation Programme under grant agreement No. 101021812 (“SilentBorder”), and by the Fonds de la Recherche Scientifique - FNRS under Grants No. T.0099.19 and J.0070.21. The collaboration of Aitor Orio was co-funded by the Spanish Ministry of Science and Innovation through the program “Ayudas para contratos para la formación de doctores en empresas (Doctorados Industriales) 2018” (Grant reference: DIN2018-009886). Pietro Vischia’s work was partially supported by the FNRS under Grant No. 40000963 and by the Ramón y Cajal program under the Project No. RYC2021-033305-I funded by MCIN/AEI/10.13039/501100011033 and by the European Union NextGenerationEU/PRTR. Max Lamparth was supported by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s Excellence Strategy – EXC-2094 – 390783311. We are grateful to Pablo Canteli Morales for the useful comments he provided us.

References

  1. [1]G. C. Strong et al., TomOpt: Differential Muon Tomography Optimisation, Feb., 2024. https://doi.org/10.5281/zenodo.10673885. https://github.com/GilesStrong/tomopt.
  2. [2]E. P. George, Cosmic Rays Measure Overburden of Tunnel, Commonwealth Engineer July 1 (1955) 455.
  3. [3]L. W. Alvarez et al., Search for Hidden Chambers in the Pyramids, Science 167 (1970) 832.pubmed.ncbi.nlm.nih.gov/17742609
  4. [4]K. Morishima et al., Discovery of a big void in Khufu’s Pyramid by observation of cosmic-ray muons, Nature 552 no. 7685, (2017) 386, arXiv:1711.01576 [physics.ins-det].DOI
  5. [5]S. Procureur et al., Precise characterization of a corridor-shaped structure in Khufu’s Pyramid by observation of cosmic-ray muons, Nature Communications 14 no. 1, (2023) 1144.DOI
  6. [6]L. Oláh, H. K. Tanaka, and D. Varga, eds., Muography: Exploring Earth’s Subsurface with Elementary Particles. John Wiley & Sons, 2022.DOI
  7. [7]E. Rutherford, The scattering of α and β particles by matter and the structure of the atom, Lond. Edinb. Dubl. Phil. Mag. 21 no. 125, (1911) 669.
  8. [8]G. R. Lynch and O. I. Dahl, Approximations to multiple Coulomb scattering, Nucl. Inst. Meth. B 58 (1991) 6.DOI
  9. [9]K. N. Borozdin et al., Radiographic imaging with cosmic-ray muons, Nature 422 (2003) 277.pubmed.ncbi.nlm.nih.gov/12646911
  10. [10]S. Barnes, A. Georgadze, A. Giammanco, M. Kiisk, V. A. Kudryavtsev, M. Lagrange, and O. L. Pinto, Cosmic-Ray Tomography for Border Security, Instruments 7 no. 1, (2023) 13.DOI
  11. [11]M. Weekes, A. Alrhel, D. Barker, D. Kikola, A. K. Kopp, M. Mhaidra, J. Stowell, L. Thompson, and J. J. Velthuis, Material identification in nuclear waste drums using muon scattering tomography and multivariate analysis, Journal of Instrumentation 16 no. 05, (2021) P05007.DOI
  12. [12]C. L. Morris, J. Bacon, K. Borozdin, J. Fabritius, H. Miyadera, J. Perry, and T. Sugita, Horizontal cosmic ray muon radiography for imaging nuclear threats, Nucl. Instr. Meth B 330 (2014)42–46.DOI
  13. [13]International Atomic Energy Agency, Muon imaging: Present Status and Emerging Applications, IAEA TECDOC 2012, IAEA, Vienna, 2022. https://www.iaea.org/publications/15182/muon-imaging.
  14. [14]D. Mrdja, I. Bikit, K. Bikit, J. Slivka, J. Hansman, L. Oláh, and D. Varga, First cosmic-ray images of bone and soft tissue, EPL (Europhysics Letters) 116 no. 4, (nov, 2016) 48003. https://doi.org/10.1209/0295-5075/116/48003.
  15. [15]I. Bikit, D. Mrdja, K. Bikit, J. Slivka, N. Jovancevic, L. Oláh, G. Hamar, and D. Varga, Novel approach to imaging by cosmic-ray muons, EPL (Europhysics Letters) 113 no. 5, (mar, 2016) 58001. 10.1209/0295-5075/113/58001.DOI
  16. [16]Z. Yifan, Z. Zhi, Z. Ming, W. Xuewu, and Z. Ziran, Discrimination of drugs and explosives in cargo inspections by applying machine learning in muon tomography, High Power Laser and Particle Beams 30 no. 30086002, (2018) 086002.DOI
  17. [17]J. Xuan-Tao, L. Si-Yuan, H. Yu-He, Z. Kun, Z. Jin, P. Xiao-Yu, X. Min, and W. Xiao-Dong, A novel 4D resolution imaging method for low and medium atomic number objects at the centimeter scale by coincidence detection technique of cosmic-ray muon and its secondary particles, Nuclear Science and Techniques 33 no. 2, (2022)2210–3147.
  18. [18]M. Holma, J. Joutsenvaara, and P. Kuusiniemi, Trends in Publishing Muography Related Research: The Situation at the End of 2020, Journal of Advanced Instrumentation in Science 2022 (2022) 292. http://journals.andromedapublisher.com/index.php/JAIS/article/view/292.DOI
  19. [19]L. Bonechi, R. D’Alessandro, and A. Giammanco, Atmospheric muons as an imaging tool, Rev. Phys. 5 (2020) 100038, arXiv:1906.03934 [physics.ins-det].DOI
  20. [20]T. K. Gaisser, Cosmic rays and particle physics. Cambridge University Press, 1990.
  21. [21]M. Guan, M.-C. Chu, J. Cao, K.-B. Luk, and C. Yang, A parametrization of the cosmic-ray muon flux at sea-level, 2015.arxiv.org/abs/1509.06176
  22. [22]P. Shukla and S. Sankrith, Energy and angular distributions of atmospheric muons at the Earth, Int. J. Mod. Phys. A 33 no. 30, (2018) 1850175, arXiv:1606.06907 [hep-ph].arxiv.org/abs/1606.06907
  23. [23]Particle Data Group Collaboration, R. L. Workman and Others, Review of Particle Physics, PTEP 2022 (2022) 083C01.
  24. [26]A. Paszke et al., PyTorch: An Imperative Style, High-Performance Deep Learning Library, in Advances in Neural Information Processing Systems 32, H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, eds., pp. 8024–8035. Curran Associates, Inc., 2019. http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf.arxiv.org/abs/1912.01703
  25. [27]A. G. Baydin, K. Crammer, P. de Castro Manzano, C. Delaere, D. Derkach, J. Donini, T. Dorigo, A. Giammanco, J. Kieseler, L. Layer, G. Louppe, F. Ratnikov, G. C. Strong, M. Tosi, A. Ustyuzhanin, P. Vischia, and H. Yarar, Toward Machine Learning Optimization of Experimental Design, Nuclear Physics News 31 no. 1, (2021)25–28, https://doi.org/10.1080/10619127.2021.1881364. https://doi.org/10.1080/10619127.2021.1881364.
  26. [28]T. Dorigo, A. Giammanco, P. Vischia, M. Aehle, M. Bawaj, A. Boldyrev, P. de Castro Manzano, D. Derkach, J. Donini, A. Edelen, F. Fanzago, N. R. Gauger, C. Glaser, A. G. Baydin, L. Heinrich, R. Keidel, J. Kieseler, C. Krause, M. Lagrange, M. Lamparth, L. Layer, G. Maier, F. Nardi, H. E. Pettersen, A. Ramos, F. Ratnikov, D. Röhrich, R. R. de Austri, P. M. R. del Árbol, O. Savchenko, N. Simpson, G. C. Strong, A. Taliercio, M. Tosi, A. Ustyu zhanin, and H. Zaraket, Toward the end-to-end optimization of particle physics instruments with differentiable programming, Reviews in Physics 10 (2023) 100085. https://www.sciencedirect.com/science/article/pii/S2405428323000047.
  27. [29]G. Strong and T. Dorigo, TOMOPT: Differential Muon Tomography Optimisation, 1st MODE Workshop on Differentiable Programming (2021). https://indico.cern.ch/event/1022938/timetable/?view=standard#7-tomopt-pytorch-based-differe. Accessed 2023/09/23.
  28. [30]G. Strong et al., Data Challenge for the 2nd MODE Workshop on Differentiable Programming, 2nd MODE Workshop on Differentiable Programming (2022). https://indico.cern.ch/event/1145124/contributions/4806648/. Repository https://github.com/GilesStrong/mode_diffprog_22_challenge.DOI
  29. [31]G. Strong, T. Dorigo, A. Giammanco, M. L. Pietro Vischia, Jan Kieseler, F. Nardi, H. Zaraket, M. Lamparth, F. Fanzago, O. Savchenko, and A. Bordignon, TomOpt: Differentiable Optimisation of Muon-Tomography Detectors, 2nd MODE Workshop on Differentiable Programming (2022). https://indico.cern.ch/event/1145124/contributions/4983513/. Accessed 2023/09/23.
  30. [32]S. Agostinelli et al., Geant4—a simulation toolkit, Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 506 no. 3, (2003)250–303.DOI
  31. [33]J. Allison et al., Geant4 developments and applications, IEEE Transactions on Nuclear Science 53 no. 1, (2006)270–278.
  32. [34]J. Allison et al., Recent developments in Geant4, Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 835 (2016)186–225.
  33. [35]L. J. Schultz, G. S. Blanpied, K. N. Borozdin, A. M. Fraser, N. W. Hengartner, A. V. Klimenko, C. L. Morris, C. Orum, and M. J. Sossong, Statistical Reconstruction for Cosmic Ray Muon Tomography, IEEE Transactions on Image Processing 16 no. 8, (2007)1985–1993.pubmed.ncbi.nlm.nih.gov/17688203
  34. [36]S. R. Qasim, J. Kieseler, Y. Iiyama, and M. Pierini, Learning representations of irregular particle-detector geometry with distance-weighted graph networks, Eur. Phys. J. C 79 no. 7, (2019) 608, arXiv:1902.07987 [physics.data-an].DOI
  35. [37]G. Strong, 2-Level Graphs for Muon-Tomography, 5th CERN Inter-experiment Machine Learning Workshop (2022). https://indico.cern.ch/event/1078970/timetable/?view=standard#42-two-level-graphs-for-muon-t. Accessed 2023/09/23.
  36. [38]S. Linnainmaa, The representation of the cumulative rounding error of an algorithm as a Taylor expansion of the local rounding errors, Master’s thesis, Univ. Helsinki, 1970.
  37. [39]P. J. Werbos, Applications of Advances in Nonlinear Sensitivity Analysis, in Proceedings of the 10th IFIP Conference, 3.18 - 4.9, NYC, pp. 762–770. 1981.
  38. [40]D. E. Rumelhart, G. E. Hinton, and R. J. Williams, Learning representations by back-propagating errors, Nature 323 (Oct., 1986)533–536.
  39. [41]J. Hadamard, Mémoire sur le problème d’analyse relatif à l’équilibre des plaques élastiques encastrées. Mémoires présentés par divers savants à l’Académie des sciences de l’Institut de France: Éxtrait. Imprimerie nationale, 1908. http://books.google.com.au/books?id=BTEPAAAAIAAJ.
  40. [42]D. P. Kingma and J. Ba, Adam: A Method for Stochastic Optimization, in 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, Y. Bengio and Y. LeCun, eds. 2015. http://arxiv.org/abs/1412.6980.
  41. [43]P. Martínez, C. Díez, P. Gómez, and A. Orio, Process monitoring: measurement of the metal-slag interface in furnace ladles, IAEA TECDOC SERIES. Muon Imaging. Present Status and Emerging Applications 2012 (2022)34–35. https://www.iaea.org/publications/15182/muon-imaging.
  42. [44]L. N. Smith, Cyclical Learning Rates for Training Neural Networks, in 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 464–472. 2017. arXiv:1506.01186. http://arxiv.org/abs/1506.01186.
  43. [45]L. N. Smith and N. Topin, Super-convergence: very fast training of neural networks using large learning rates, in Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications, T. Pham, ed., vol. 11006, pp. 369 – 386, International Society for Optics and Photonics. SPIE, 2019. arXiv:1708.07120. https://doi.org/10.1117/12.2520589.
  44. [46]L. N. Smith, A disciplined approach to neural network hyper-parameters: Part 1 - learning rate, batch size, momentum, and weight decay, CoRR abs/1803.09820 (2018), arXiv:1803.09820. http://arxiv.org/abs/1803.09820.

Paper details

vlnc:W_5z2nspy6
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents