WorksPhysics and Astronomy
Ensemble Reservoir Computing for Dynamical Systems: Prediction of Phase-Space Stable Region for Hadron Storage Rings
Abstract
We investigate the ability of an ensemble reservoir computing approach to predict the long-term behaviour of the phase-space region in which the motion of charged particles in hadron storage rings is bounded, the so-called dynamic aperture. Currently, the calculation of the phase-space stability region of hadron storage rings is performed through direct computer simulations, which are resource- and time-intensive processes. Echo State Networks (ESN) are a class of recurrent neural networks that are computationally effective, since they avoid backpropagation and require only cross-validation. Furthermore, they have been proven to be universal approximants of dynamical systems. In this paper, we present the performance reached by ESN based on an ensemble approach for the prediction of the phase-space stability region and compare it with analytical scaling laws based on the stability-time estimate of the Nekhoroshev theorem for Hamiltonian systems. We observe that the proposed ESN approach is capable of effectively predicting the time evolution of the extent of the dynamic aperture, improving the predictions by analytical scaling laws, thus providing an efficient surrogate model.
Introduction
The advent of superconducting, high-energy hadron storage rings and colliders elevated non-linear beam dynamics to the forefront of accelerator design and operation. When studying phenomena in the field of single-particle beam dynamics, the concept of dynamic aperture (DA), that is, the extent of the phase space region where bounded motion occurs, has been a key observable to guide the design of several past (see, e.g. [1, 2, 3, 4, 5, 6]), present, e.g. the CERN Large Hadron Collider (LHC) [7], and future hadron machines (see e.g. [8, 9, 10, 11, 12, 13, 14, 15]).
DA prediction involves many challenging aspects, including understanding the mechanisms that determine its behaviour and addressing several computational problems. An important issue is the possibility of modelling the evolution of DA as a function of the number of turns, which has been studied since the end of the 90s [16, 17]. Indeed, determining how to describe and efficiently predict the value of the DA might solve some fundamental problems in accelerator physics, linked to performance optimisation of storage rings and colliders. The high computational cost of direct numerical simulations would be significantly reduced if a reliable model for the time evolution of the DA were available. In fact, the numerical simulations required to assess the performance of a circular accelerator cannot cover a time span comparable with operational intervals. For the LHC case, simulations up to turns are at the limit of the CPU-time capabilities, although this represents only about 89 s of storage time, knowing that a typical fill time is of the order of several hours. Eventually, a model for the evolution of DA over time would also open the possibility of studying observables that are more directly related to machine performance, such as beam losses and lifetime [18] and luminosity evolution in colliders [19, 20].
A successful solution to this problem has been found by building models for DA scaling with time based on fundamental results of dynamical system theory, such as the Nekhoroshev theorem [21, 22, 23]. In fact, models with two or three parameters can be derived that can be fitted to numerical data that represent the evolution of the DA and used to predict the DA value for times beyond the current computational capabilities [24].
In the last decade, the use of neural networks has increased significantly in a large number of diverse research areas, and this observation has suggested their application to the prediction of the evolution of DA. For example, neural networks are used for speech recognition [25] or to forecast wind power [26]. Among neural network techniques, the most common architectures are feedforward [27], convolutional [28], and recurrent [29] neural networks. Feedforward neural networks are made up of neurons connected to other neurons, only. They provide only input-output relationships and can approximate very large classes of functions. On the other hand, recurrent neural networks are made up of neurons connected to themselves and other neurons. They preserve an internal state that is a non-linear transformation of the input signal and can therefore be considered as dynamical systems.
Echo State Networks (ESN) are one of the classes of recurrent neural networks that use the reservoir computing approach [30]. This approach has the main advantage of significantly reducing the computational time required by the training process, which is performed to find the optimal parameters (called weights) of a neural network. In fact, the peculiarity of the ESN is that training is performed, usually using linear regression [31], to calculate the weights used to project the reservoir state onto the output state. Therefore, no backpropagation is needed. Backpropagation [32] refers to the numerical procedure, usually based on the stochastic gradient method, used for the training of feedforward networks, which is responsible for a large share of its computational cost. ESN have also been proven to be universal approximants of dynamical systems [33]. Thus, ESN seem to be natural candidates for performing the prediction of DA for a large number of turns, and hence challenge the performance of the deterministic models developed so far.
This paper is organised as follows: In § 2, we introduce the concept of DA and the approach used to provide numerical estimates of its value. Analytical scaling laws, based on the Nekhoroshev theorem and used to predict the time evolution of DA, are also presented. § 4 introduces the continuous-time leaky ESN framework that is used for the prediction of DA. The Echo State Property (ESP), and a sufficient condition that can be applied in practice to satisfy it, are discussed in the Appendix § 9. Section 4 describes the ensemble procedure used in the cross-validation of the ESN and in the prediction of DA. The results are presented and discussed in § 6, while conclusions are drawn in § 7.
Dynamic Aperture
Generalities
We consider a Hamiltonian system in , with a stable fixed point at the origin, whose dynamics is generated by a polynomial map , and such that the linear part of is described by the direct product of rotations. Under these conditions, the DA of the system under consideration is the extent of the region of phase space in which bounded motion occurs.
Ensemble Reservoir Computing for Dynamical Systems
Following [34] and restricting the analysis to the case of Hamiltonian systems in , which are relevant for accelerator physics, we consider the phase space volume of the initial conditions that are bounded after iterations, namely
where is the characteristic function defined as equal to one if the orbit starting at is bounded and zero if it is not.
To exclude the disconnected part of the stability domain in the integral (1), we have to choose a suitable coordinate transformation. As linear motion is given by the direct product of constant rotations, the natural choice is to use the polar variables , where and are the linear invariants of dynamics. The non-linear part of the equations of motion adds a coupling between the two planes, the perturbative parameter being the distance from the origin. Therefore, it is natural to replace and by the polar variables and , respectively:
Substituting in Eq. (1) we obtain
where represents the volume element
Having fixed and , let be the last value of whose orbit is bounded after iterations. Then, the volume of a connected domain in which the motion is bounded is given by
where
In this way, we exclude stable islands that are not connected to the main stable domain. Note that, in principle, this method might also lead to excluding connected parts. We then define the DA as the radius of the hypersphere that has the same volume as the stability domain
When Eq. (5) is implemented in a computer code, one considers steps in the angle and steps in the angles , and the dynamic aperture reads
where .
The numerical error is given by the discretization in angles , , and radius , which gives a relative error proportional to , , and , respectively. This numerical error can be optimised by choosing integration steps that produce comparable errors, i.e. . In this way, neglecting the constants in front of the error estimates, one can obtain a relative error of by evaluating the orbits, i.e. iterates. The fourth power in the number of orbits comes from the dimensionality of phase space and makes a precise estimate of the dynamic aperture very CPU time consuming.
It is possible to reduce the size of the scanning procedure, and hence the CPU time needed, by setting the angles to a constant value, e.g. zero, thus performing only a 2D scan over and . This is what is generally done in SixTrack simulations [35, 36]. In this case, the transformation (2) reads
and the original integral is transformed to
Having fixed , let be the last value of whose orbit is bounded after iterations. Then, the volume of a connected stability domain is given by
We define the dynamic aperture as the radius of the sphere that has the same volume as the stability domain1
When Eq. (10) is implemented in a computer code, one considers steps in the angle , and the dynamic aperture reads
so that the numerical error is given by discretising the angle and the radius , which yields a relative error proportional to and , respectively. In this case, the integration steps should also be selected to produce comparable errors, i.e. . In this way, neglecting the constants which are in front of the error estimates, one can obtain a relative error of by evaluating orbits, i.e. iterates.2 Note that Eq. (10) can be evaluated using higher-order numerical integration rules as implemented in the post-processing tools linked with SixTrack [36].
It is worth noting that, in some applications, the simplified formula
which corresponds to computing the average of over the angle , could be used [17].
DA Scaling Law
All the definitions of DA estimates presented in the previous section are functions of , the turn number used to estimate the orbit stability from the results of numerical simulations. It is evident that the definition of DA itself implies that it is a non-increasing function of . The key point is whether it is possible to find the functional form of this time dependence, and several studies have shown that this is indeed the case [17, 24]. In fact, such a functional form can be built by considering the estimate of the stability time provided by the Nekhoroshev theorem [21, 22, 23], which is a key and very general theorem in the theory of Hamiltonian dynamical systems.
The first models were described in [17] and then reviewed in [24] and the two that we retained after the review read
Model 2 ,
where the free parameters are , but it is customary to set , and
Model 4
where the free parameters are , , and possibly , unless it is fixed to the value of according to the analytic Nekhoroshev estimate. stands for the negative branch of the Lambert- function, a multi-valued special function (see, e.g. [37] for a review of the properties and applications of the Lambert function). Note that stands for or , depending on the numerical approach used to estimate the DA. The nomenclature of the models presented in Eqs. (14) and (15) reflects the historical development of these models and the nomenclature used in [24].
An example of the numerical calculation of the DA for a realistic model of the luminosity upgrade of the CERN LHC, HL-LHC [13], and the corresponding fitted scaling law using all available DA data are shown in Figure 1, where the excellent agreement between the numerical data and the fit model is clearly visible. We denote by SL-ALL the fitting of Model 2 using all available DA data.
Figure 1. Example of DA numerical computation for a realistic model of the HL-LHC with the corresponding fitted scaling law. The excellent agreement between the numerical data and the fit model is clearly visible.
Note that in the rest of the paper Model 2 is the only scaling law model used.
DA data organisation
In this section, we present the data sets used to test the predictive model introduced in Section 4. The first data set is obtained from a realistic model of the HL-LHC, whereas the second one is obtained from the 4D Hénon map.
The HL-LHC case
The HL-LHC data set, presented in Figure 2, is composed of 60 realisations (also called seeds due to the underlying random generator used for the generation of the realisations) of the magnetic field errors of the magnetic lattice of the HL-LHC, for the collision optics with cm and proton energy of 7 TeV. The 60 realisations are supposed to accurately represent the actual lattice of the HL-LHC; for this reason, the DA computation is customarily performed using the complete set of realisations to provide an accurate estimate of the DA of the actual accelerator. Magnetic field errors are assigned to all magnets that make up the ring. Initial conditions (also called particles) are distributed in physical space to probe the orbit stability and thus determine the DA. Different amplitudes and angles in the plane are used to sample the phase space. In the cases considered here, 11 angles, uniformly distributed in the interval , are used, while the amplitudes are uniformly distributed in the interval , with 30 initial conditions defined in each interval. Note that 30 particles are evenly distributed in each amplitude interval of , and represents the root mean square (rms) beam size, which is used as a natural unit in these studies. All initial conditions are tracked for turns. The numerical estimates of DA as a function of are calculated according to Eq. (10) and are shown in Figure 2 (left).
Figure 2. Left: Evolution of DA as a function of time for the 60 realisations of the HL-LHC magnetic lattice. Right: Splitting of the HL-LHC data set into training, validation, and test sets.
We build piecewise constant functions so that each DA estimate now contains $10^{3}$ data points, with the aim of obtaining DA estimates in constant time steps. These data points are then divided into training set, validation set, and test set. The first data are used for training, the next $k_{\mathrm{val}}=50】【。 data for validation, and the remaining data for testing. Note that the end of the training and validation sets corresponds to , and the end of the testing to turns. A graph of the 60 piecewise constant functions split into training set, validation set and test set is shown in Figure 2 (right). Note that each of the 60 realisations corresponds to a different DA on which we will train, validate, and test our ESN model.
The 4D Hénon map case
The 4D Hénon map is a well-known dynamical system that displays a rich dynamical behaviour as presented in, e.g. [38]. The model used to generate DA estimates is defined as:
where the subscript denotes the discrete time and is a matrix given by the direct product of two rotation matrices R:
where the linear frequencies vary with the discrete time according to
where denotes the amplitude of the frequency modulation and and are fixed parameters, which are taken from previous studies [24].3
The 4D Hénon map is a simplified model of a circular accelerator. In particular, it describes the effects of a sextupole and octupole magnet on the transverse particle motion through the quadratic, due to the sextupole, and cubic, due to the octupole, non-linear terms. Being a simplified accelerator model, it allows one to track particles up to a much larger number of turns, and for more amplitudes and angles, namely 100 amplitudes and angles uniformly distributed in the interval and respectively. The 4D Hénon map data set is composed of 60 cases, for 20 different values of uniformly distributed in the interval and , covering up to turns. Similarly to the HL-LHC data set, we build piecewise-constant functions so that each case yields 1000 data points. The first data are used for training, the next data for validation, and the last data for testing. Note that we used the same number of training, validation, and test data for the HL-LHC case. The 60 piecewise constant functions divided into training set, validation set, and test set are shown in Figure 3.

Figure 3. Splitting of the 4D Hénon map data set into training, validation, and test sets. The sudden drop in DA visible for occurs when .
Note that because of the larger number of amplitudes and angles considered, the DA data are smoother than those of the HL-LHC case. Furthermore, each of the 60 cases generated in this data set corresponds to a different dynamics for which we will train, validate, and test our ESN model.
Echo State Networks
In this section, we present some general concepts about ESN. More specifically, we introduce the mathematical framework of continuous-time leaky ESN applied to supervised learning tasks.
Shallow ESN
Shallow ESN are a class of Recurrent Neural Networks using the Reservoir Computing approach [30]. In this type of neural network, the data input is fed into a single, random, and non-trainable network, called the reservoir. The reservoir is eventually connected by trainable weights to the ESN output. The use of ESN for time series prediction has become widespread due to its inexpensive training process and its remarkable performance in the modelling of dynamical systems [39].
Contrary to feedforward neural networks, ESN do not suffer from vanishing or divergent gradients (caused by the fact that the parameters of neural networks remain almost constant or lead to numerical instabilities), which induces poor performance of the training algorithm [40].
ESN can be defined for discrete- or continuous-time systems. The reservoir dynamics can be defined with or without the leaking rate parameter, which can be considered as the speed of the reservoir update dynamics. We introduce the definition of a shallow leaky ESN in continuous time as in [41]. We consider the case of networks with continuous-time , inputs, reservoir neurons, and outputs. Note that we will use small letters to indicate vectors and capital letters to indicate matrices. We define by the input data and the training data that we want to learn with the ESN model. The ESN output is denoted by , while the internal reservoir activation state is given by . Furthermore, we define the input weight matrix , the reservoir weight matrix , and the output weight matrix . The discretised (by the Euler method) time dynamics of a leaky ESN is given by:
where with the size of the Euler discretization step and a global time constant, the leaking rate, a sigmoid function, the output activation function, denotes vector concatenation, the update of the reservoir activation state at discrete time and the ESN output at the same time .
In the case of a linear readout, i.e. when is the identity function, we can rewrite Eq. (21) in matrix notation as:
where contains the ESN outputs at every time step and where contains the concatenation of the input and the activation state of the reservoir at every , namely
where denotes the Burn-In data, i.e. the number of input data we want to discard at the beginning of the training phase.
The optimal output weight matrix can be found by solving the following minimisation problem:
where denotes the cost function we want to minimise and is the Euclidean norm of the th row of .
The solution of the minimisation problem stated in Eq. (24) can be found efficiently using linear regression with Tikhonov (Ridge) regularisation [42]:
where the superscript denotes the transpose, is the identity matrix, and is the training data matrix, which contains the training data at time step .
The learning phase is carried out on the so-called training set, which contains the training data . A sketch of the training phase of the ESN is provided in Figure 4.

Figure 4. Sketch of the training procedure for a shallow leaky ESN. The size of the matrices has been arbitrarily selected. denotes the square of the Euclidean norm error between the ESN output and the training data , .
After training, the ESN hyperparameters, defined in Section 4, are tuned using validation data. Finally, the ESN is tested using the data to check the ability of the ESN to predict new data. The validation and test procedures are detailed in Section 4. As stated in Eq. (24), only the output weight matrix is trained, while the input and reservoir matrices and are randomly generated, as explained in detail in Section 4.
Deep ESN
A deep ESN is an ESN composed of stacked reservoirs, as shown in the sketch of the deep ESN training phase in Figure 5.

Figure 5. Sketch of the training procedure for deep ESN with reservoirs.
In this case, denotes the th reservoir weight matrix, the th input weight matrix, the local internal reservoir state vector, and the global internal reservoir state vector. Equations (20) and (21) for a shallow ESN read now
where is the concatenation of all .
ESN predictive model for DA evolution
In the previous section, we have introduced the definition of a shallow leaky ESN and its extension as a deep ESN. In Eqs. (20) and (26) we can already identify some parameters (called hyperparameters) of the ESN predictive model. These are the leaking rate , the number of stacked reservoirs , the dimension of the reservoir matrix and the activation function usually set as the hyperbolic tangent function . In Appendix § 9, we give a sufficient condition on the spectral radius of the reservoir matrix , which can also be considered as a hyperparameter, that guarantees the Echo State Property (ESP).
Other hyperparameters are often introduced in the implementation of ESN equations. Specifically, the sparsity ratio of the reservoir matrix , i.e. the fraction of 0 elements in the reservoir matrix and (as in [43]), which corresponds to the number of time steps of the input data that are discarded. Furthermore, the regularisation parameter in Eq. (25) also needs to be optimised and is also considered a hyperparameter of the ESN model. Setting large values for is generally used to avoid overfitting and may improve prediction in the test set. To complete the definition of the predictive model of the ESN, we must assign a value to all hyperparameters, knowing that the performance of the model strongly depends on the choice of their values.
It is a common procedure in ESN training to perform an optimisation of these hyperparameters, which is usually done by grid search methods [44], in the validation set. The validation procedure considered here is based on an ensemble approach to deal with the randomness of the reservoirs. Eventually, once the predictive model has been trained and validated, we can test it in the test set with unseen data.
ESN ensemble validation approach
The ensemble validation approach used in our studies is based on the principle of minimising the average of the Relative Root Mean Square Error (RRMSE) of dynamics predicted (i.e, 60 seeds for the HL-LHC dataset and 60 cases for the 4D Henon map) for different randomly generated reservoirs and various hyperparameters values on the validation set. Note that for each of the dynamics, we predict a mean over the reservoirs. Additionally, each of the dynamics contains different input/training/validation/test data, so that each prediction is performed independently of the others. We define this RRMSE on the validation set as:
where is the number of validation data, is the mean over the reservoirs for the ith dynamics at time , and is the validation data at the same time for the same ith dynamics.
This procedure aims to build a robust predictive model in which all hyperparameters are fixed. The search of the hyperparameters values minimizing the is done over a domain . Each of the hyperparameters is updated one by one using the value in , which minimises . Furthermore, as mentioned above, this ensemble validation method requires the generation of different random matrices and . This is done by sampling their elements from a uniform pseudorandom distribution in and scaling them to the interval so that they also have negative elements. The procedure for generating and is detailed in Algorithm 1, while a pseudocode of the general ensemble validation procedure is presented in Algorithm 2.
Note that the functions Training() and Prediction() implement the equations presented in § 4.
Algorithm 1 Generation of the random matrix and .
Require: spectral radius we want to set to , input size, reservoir size
Ensure: input weight matrix, reservoir weight matrix
Random initialisation of , , , ,
Compute spectral radius of
Scale
Algorithm 2 Validation.
Require: domain of search of an hyperparameter , random different pairs of , dynamics we want to validate with the associated inputs data , training data and validation data
Ensure: set of all the fixed hyperparameters which minimise in average for the dynamics and reservoirs.
for do
for do
for to do
for to do
end for
end for
end for
Set and update
end for
ESN ensemble test approach
Once the parameters and hyperparameters of the ESN predictive model have been tuned using training set and validation set, we can test our ESN model for the prediction of not previously used data, i.e. DA values at a larger time. We denote by the number of data in the test set we try to predict.
The algorithm 3 describes the test procedure for a single dynamics, i.e. a single realisation of the HL-LHC magnetic lattice or a single case for the 4D Hénon map data set. We can loop the procedure to perform the prediction in
Algorithm 3 Test
Require: number of different pairs of , test data of size , have been tuned using the data in the validation set with the corresponding output weight matrix .
Ensure: mean of the predicted outputs over the reservoirs, between and
for to do
end for
the test set for the dynamics. Note that, contrary to the validation, here the prediction is performed in the test set for data not previously used.
Results and Discussion
In this section, we present the DA predictions obtained with our ESN-based predictive model. In particular, we compare these predictions with those of the fitted scaling law presented in Eq. (14) and used in [24]. We recall that the ESN output is the mean prediction over random reservoirs. The validation and testing methods are those introduced in Section 4. We tested the proposed approaches with the HL-LHC data sets and the 4D Hénon map presented in § 2.
DA Predictions for the HL-LHC data set
Validation of the ESN
In this stage, we search for the set of hyperparameters that minimises, on average over the seeds and randomly generated reservoirs, the RRMSE in the validation set. Here, the number of predicted dynamics is equal to the number of seeds. We also recall that the number of validation data is and the definition of is presented in Algorithm 2. The optimal hyperparameters are determined one by one by a grid search over a wide range of possible parameter values, and the search domains of the hyperparameters are listed in Table 1.
Table 1. Search domains of the various hyperparameters . Note that the hyperparameters are tuned one by one while the others are kept constant.
Figure 6 shows as a function of the various hyperparameters in . The values of the hyperparameters are updated one-by-one with those that minimise . As we can see, a shallow ESN with a small number of neurons provides the best results. Stacking more reservoirs does not improve the predictions. In fact, adding reservoirs or increasing the number of neurons makes the model overfit, so it cannot predict correctly in the validation set. This can be explained by the small number of features that the ESN must learn and by the characteristics of the DA data, which are not enough.

Figure 6. RRMSE as a function of the various hyperparameters in . Values of constant hyperparameters: , , , , . Values of constant hyperparameters: , , , , . Values of constant hyperparameters: , , , , . Values of constant hyperparameters: , , , , . Values of constant hyperparameters: , , , , . Values of constant hyperparameters: , , , , . ESN (left graph) and SL (right graph). The scaling law fit is performed using the first DA data. Note that ESN and SL share the same test set. Figure 9 shows the distribution of the RRMSE values defined in Algorithm 3, for both the ESN model and SL.
Figure 9. Distribution of for seeds for ESN and SL.
Regarding the other hyperparameters, the optimum spectral radius value initially set to is updated to and satisfies the ESP. Furthermore, since the optimal value of is smaller than , it can be considered small, which justifies setting the sparsity ratio so that all elements of are non-zero. Then, we decided to choose the activation function , since it is the most used in ESN, and the leaking rate to simplify the equations described in (26). Eventually, the values of and initially set to and are updated to and respectively. The values of the hyperparameters updated after validation and used for the prediction stage in the test set are summarised in Table 2.
| 20 | 0.99 | 1 | 0 | 1 | 0 |
Table 2. Set of the hyperparameters tuned after validation using HL-LHC DA data.
The ESN model
Once the ESN has been trained and validated, we can test it with the test set for data not previously used using the hyperparameters reported in Table 2. We recall that the number of test data is , i.e. half of the total number of data used. In Figure 7, we show the mean prediction in the test set together with the envelope (i.e. minimum and maximum) of the predictions that are associated with the randomly generated reservoirs for an arbitrary seed (number 1). We also plot the distribution of the prediction of DA at turns (end of the test set).
Figure 7. Left: Numerical DA data, prediction of DA , average, minimum, and maximum over the randomly generated reservoirs as a function of time. Right: distribution for the randomly generated reservoirs at . The seed used for both plots is number 1.
As mentioned above, we will denote by the ESN mean prediction and only plot this mean value to avoid overloading the graphs with the values generated by random reservoirs. To have a complete view, Figure 8 shows the predictions of seeds in the train set, validation set and test set. Vertical dashed lines indicate the end of the train set and validation set for
Figure 8. DA predictions for ESN (left) and SL (right) for seeds.
We report in Table 3 the mean, maximum, minimum, and standard deviation of for the predictions of ESN and SL over seeds. The ESN model and SL generate predictions whose distributions have essentially the same mean and minimum values. However, some outliers appear in the SL distribution, which affect the maximum and standard deviation values. This contributes to the generation of more stable predictions by ESN, i.e. without outliers, and significantly lower values of the standard deviation and maxima.
| Mean | Max | Min | Std | |
| ESN | 0.37 | 0.94 | 0.06 | 0.20 |
| SL | 0.42 | 1.78 | 0.07 | 0.35 |
Table 3. Mean, maximum, minimum, and standard deviation of the distribution.
The SL-ESN model
In this section, we consider whether ESN predictions can possibly be used to replace the tracking simulations that generated the data in the test set. In this sense, we fit the SL to the data plus the ESN predictions in the test set. We denote this fit procedure by SL-ESN and compare it with the results of SL-ALL, which represents the best results that can be achieved with the SL approach4. The idea is to check the quality of the approximation of SL-ESN in the test set, in view of further prediction beyond this set. The predictions provided by SL-ESN and SL-ALL for the seeds can be seen in Figure 10 and the distribution of is shown in Figure 11, while the mean, maximum, minimum, and standard deviation of in Table 4.

Figure 10. Predictions for SL-ESN (left) and SL-ALL (right) for seeds.

Figure 11. Distribution of RRMSE for seeds for SL-ESN and SL-ALL.
| Mean | Max | Min | Std | |
| SL-ESN | 0.33 | 1.01 | 0.10 | 0.21 |
| SL-ALL | 0.21 | 0.64 | 0.06 | 0.13 |
Table 4. Mean, maximum, minimum, and standard deviation of the RRMSE distribution.
As it might be expected, all indicators of the distribution of for SL-ESN are significantly larger than those for SL-ALL, as the first approach fits the prediction of ESN, not the real DA data. In fact, SL-ESN is essentially equivalent to ESN alone and hence more stable than SL alone as far as outliers are concerned. In other words, the SL-ESN seems to be an effective surrogate model that improves the predictions given by the SL only.
After having evaluated the accuracy of the SL-ESN model in the test set, we can check if it can replace the tracking simulations in this set. To do so, we compute predictions beyond the test set and up to turns. Since we do not have real DA data in this time interval, we cannot compute any metrics, and we use the envelope, i.e. minimum and maximum, of the predictions given by SL-ESN and SL-ALL to check whether SL-ESN approximates well the predictions given by SL-ALL beyond the test set. We plot the envelope of the predictions given by SL-ESN and SL-ALL beyond the test set in Figure 12 (left), and we also show the relative error defined as where is max or min (right).

Figure 12. Left: Envelope, i.e. minimum and maximum values of the SL-ESN and SL-ALL predictions extrapolated beyond the test set. Right: Relative error of the minimum and maximum DA predictions up to turns.
The two envelopes almost overlap until turns, with and that are below 1%. From this observation we conclude that we may only need to perform the tracking simulation until the end of the validation so that the tracking in the test set could be spared. In fact, the predictions provided by SL-ESN are very similar to those of SL-ALL. In this way, we could use the ESN predictions to replace the tracking in the test set. This result is in line with what was found in [45], i.e. that the addition of synthetic points obtained by using Gaussian Processes improved the quality of the fitted SL model.
Running the SixTrack code [35, 36] and the ESN model on the same CPU architecture, we have a speed-up of a factor 20 by replacing the tracking simulations on turns, representing the test set, with the prediction of the DA values by ESN. This evaluation of CPU time reduction can be easily improved by a trivial parallelisation of the ESN over the 100 reservoirs. Of course, the actual gain depends on several details, such as the model under consideration and the definition of the times that define the validation and test sets. It is worth stressing that whenever an actual accelerator lattice is used for the numerical DA computations, the CPU time needed depends not only on the number of turns used for the tracking, but also on the size of the accelerator, which corresponds approximately to the number of magnets comprised in the lattice, and on the characteristics of the magnetic field errors included in the accelerator model. In this respect, the computational gain implied by the proposed approach is even more relevant for the case of large future colliders, such as the Future Circular Hadron Collider (FCC-hh) under study at CERN [46, 47].
DA Predictions for the Hénon map data set
To check the robustness of the current strategy, we apply it to a new system, which is the 4D Hénon map introduced in § 2.
The ESN model
Hyperparameters have been determined using the same approach as for the HL-LHC data and are reported in Table 5. In this case, we also use , but we have to stress that the various dynamics differ between them much more than the dynamics of the HL-LHC case. In fact, changes in the values of and lead to radically different dynamical behaviours, whereas the HL-LHC realisations are much closer to each other, representing minor variations of the same dynamical behaviour.
| 20 | 0.99 | 1 | 0 | 1 | 0.004 | tanh | 0 |
Table 5. Set of the hyperparameters tuned after validation using Hénon map DA data.
Only the values of and are different from those of the HL-LHC case. Note that the value of found is much lower than that of HL-LHC. This means that the model is less overfitting than with the HL-LHC data, especially because the Hénon DA data are much smoother.
In Figure 13, we plot the DA predictions given by ESN and SL. For ESN, we recall that we used and data, and for SL we used the data. Furthermore, test set is the same for both ESN and SL. As we can see, the SL predictions in the test set do not perform well, whereas those provided by the ESN fit the training/validation/test data much better.

Figure 13. DA predictions for ESN (left) and SL (right) for seeds.
In Figure 14 we compare the distributions of for ESN and SL, and the first is clearly much narrower and closer to zero than the latter. This behaviour is easily explained by considering the fact that the scaling law is an asymptotic law that aims to describe the long-term behaviour of the DA (using very few model parameters). Therefore, it is not effective in reproducing the detailed behaviour of the DA for low numbers of turns. Our ESN model is able to fit both the short-term and long-term behaviour simultaneously, thus explaining the observed better performance.

Figure 14. Distribution of for seeds for ESN and SL.
The mean, maximum, minimum, and standard deviation of for the two approaches are reported in Table 6.
| Mean | Max | Min | Std | |
| ESN | 1.13 | 1.89 | 0.59 | 0.28 |
| SL | 3.17 | 5.85 | 1.25 | 1.18 |
Table 6. Mean, maximum, minimum, and standard deviation of the distribution.
The table shows, in a quantitative way, the differences observed in the histogram of the distributions. In fact, the RRMSE of the ESN is on average about 3 times lower than that of the SL, which is a significant improvement compared to the case of HL-LHC. Several reasons can explain this behaviour. First, the DA data for the Hénon map are much smoother than those of the HL-LHC data set, which improves training and limits overfitting of the ESN. Second, as already mentioned, the behaviour of the dynamics is very diverse, and the SL, with only two free parameters, is clearly disadvantaged with respect to the ESN. Moreover, since the SL is an asymptotic law, its performance has been downgraded by including low-turn DA data.
The SL-ESN model
We repeat the procedure to check if the ESN predictions can replace the tracking simulation in the test set. As previously, we compare SL-ESN with SL-ALL. The predictions given by SL-ESN and SL-ALL for the 60 cases can be seen in Figure 15, the distribution of is shown in Figure 16, and the mean, maximum, minimum, and standard deviation of are reported in Table 7.

Figure 15. Predictions for SL-ESN (left) and SL-ALL (right) for seeds.

Figure 16. Distribution of RRMSE for seeds for SL-ESN and SL-ALL.
| Mean | Max | Min | Std | |
| SL-ESN | 0.71 | 1.57 | 0.26 | 0.33 |
| SL-ALL | 0.72 | 1.29 | 0.35 | 0.24 |
Table 7. Mean, maximum, minimum, and standard deviation of the RRMSE distribution.
In this case, the SL-ESN performs equally well as the SL-ALL. In fact, the mean of is the same. Furthermore, fitting the SL to the predictions of the ESN allows us to improve the accuracy of the ESN and the SL. Taking into account the average, SL-ESN is almost 2 times and 4 times more accurate than ESN and SL, respectively. Similarly to the HL-LHC case, the standard deviation and maximum RRMSE of SL-ESN are much lower than those of SL, which shows a certain robustness of the conclusions that SL-ESN helps improve SL.
To further check whether the ESN predictions can replace the tracking simulation in the test set, we perform the prediction beyond the test set up to turns. As previously, we do not have the real DA data in this range, so we cannot compute any metrics. We plot the envelope of the predictions given by SL-ESN and SL-ALL in Figure 17.

Figure 17. Left: Envelope, i.e. minimum and maximum values of the SL-ESN and SL-ALL predictions extrapolated beyond the test set. Right: Relative error of the minimum and maximum DA predictions up to turns.
The two envelopes of the predictions almost overlap until , and the relative errors and are below 1.5%, as for the case HL-LHC. This indicates, once again, that the tracking simulation in the test set could be replaced by the ESN predictions.
Conclusions
In this article, we have presented the results obtained with an ensemble approach to ESN reservoir computing for the prediction of the dynamic aperture of a circular hadron accelerator. In particular, we have compared the performance of ESN with that of a scaling law based on the Nekhoroshev theorem to predict the evolution of the dynamic aperture over time. This analysis has been carried out on two data sets that have been generated using numerical simulations performed on realistic models of the transverse beam dynamics in the HL-LHC and on a modulated 4D Hénon map with quadratic and cubic non-linearities.
We have shown that the average accuracy in the test set of the scaling law used to fit the ESN predictions was better than that of the scaling law alone. In particular, we have observed that the standard deviation of the RRMSE of the scaling law combined with the ESN is much lower than that of the scaling law alone. This leads to more reliable predictions. The fact that this observation is confirmed for both data sets gives us confidence that the combination of the scaling law and the ESN is the best approach.
A consequence of this result is that the tracking performed in the test set can be avoided by replacing it with the predictions of the ESN. In fact, for both the HL-LHC and Hénon map data sets, the predictions of the scaling law combined with the ESN and of the scaling law fitted to the entire data set are close to the percent level, even for numbers of turns three orders of magnitude beyond that of the test set. The gain in CPU time depends on the size of the accelerator and the complexity of its model. However, it is clear that the proposed approach is particularly appealing for hadron colliders of the post-LHC era that are currently being studied.
The study presented here represents only the beginning of a research area that could be further developed in the future given the promising results obtained. The partition of available data into training, validation, and test data sets should be studied in more detail to assess whether such a partition could be obtained using an appropriate algorithm. The established link between dynamic aperture and models for the evolution of intensity in hadron rings and the evolution of luminosity in hadron colliders could be further developed by using the promising results discussed in this paper. Investigations on the possibility of using ESN to improve the modelling of beam lifetime and luminosity evolution should be seriously considered and pursued. Finally, the predictive power of ESN could be applied to indicators of chaos, which are dynamical observables computed over the orbit of an initial condition to establish whether the motion is regular or chaotic, to improve their performance.
This would be another important topic that could bring important insight to the field of non-linear beam dynamics.
Acknowledgements
We would like to express our gratitude to C.E. Montanari for providing the software to perform dynamic aperture simulations for the 4D Hénon map.
The Echo State Property
An important prerequisite for the output-only training is the so-called Echo State Property (ESP), which guarantees that initial conditions have an effect that vanishes over time. We use the results presented in [48] to recall the definition of ESP and a new sufficient condition that can be used in practice. In fact, satisfying the ESP allows us to guarantee that the reservoir activation state is uniquely determined by any left-infinite input sequence .
To define ESP, we require the compactness condition, that is, we assume that the states and inputs belong to compact sets , and that and , . In practice, the ESN inputs will always be bounded, so the compactness of is guaranteed. Furthermore, for bounded sigmoid functions , such as tanh, the state space is also compact. We define and , which are the sets of infinite left input and reservoir activation state sequences.
Definition 3.1 (ESP). A network with the compactness condition has the ESP with respect to if for any left input sequence and any two state sequences compatible with (i.e. , ), then for all , , where denotes a small value.
Definition A is not easily applicable in practice. Thus, we introduce the following Theorem A that should be used in practise as it provides a sufficient condition to satisfy the ESP in the case of a leaky ESN:
Theorem 3.1 (Sufficient condition of the ESP). If the spectral radius of the matrix
is smaller than 1, then the leaky ESN with satisfies the ESP for all inputs. However, this condition is only sufficient, but not necessary. In other words, setting does not necessarily lead to poor performance of leaky ESN.
References
References
- [41]Jaeger, H., Lukoševičius, M., Popovici, D., Siewert, U.: Optimization and applications of echo state networks with leaky-integrator neurons. Neural Networks 20(3), 335–352 (2007)pubmed.ncbi.nlm.nih.gov/17517495
- [42]Johansen, T.A.: On Tikhonov regularization, bias and variance in nonlinear system identification. Automatica 33, 441–446 (1997)DOI
- [43]Kawai, Y., Park, J., Asada, M.: A small-world topology enhances the echo state property and signal propagation in reservoir computing. Neural Networks 112, 15–23 (2019)DOI
- [44]Bergstra, J., Bardenet, R., Bengio, Y., Kégl, B.: Algorithms for hyper-parameter optimization. In: Shawe-Taylor, J., Zemel, R., Bartlett, P., Pereira, F., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 24. Curran Associates, Inc., ??? (2011)
- [45]Giovannozzi, M., Maclean, E., Montanari, C.E., Valentini, G., Van der Veken, F.F.: Machine learning applied to the analysis of nonlinear beam dynamics simulations for the cern large hadron collider and its luminosity upgrade. Information 12(2) (2021). https://doi.org/10.3390/info12020533
- [46]Abada, A., Abbrescia, M., AbdusSalam, S.S., Abdyukhanov, I., Abelleira Fernandez, J., Abramov, A., Aburaia, M., Acar, A.O., Adzic, P.R., P., A., et al.: FCC–hh: The Hadron Collider: Future Circular Collider Conceptual Design Report Volume 3. Future Circular Collider Eur. Phys. J. Spec. Top. 228, 755–1107 (2019). https://doi.org/10.1140/epjst/e2019-900087-0
- [47]Benedikt, M., Chance, A., Dalena, B., Denisov, D., Giovannozzi, M., Gutleber, J., Losito, R., Mangano, M.L., Raubenheimer, T., Riegler, W., Risselada, T., Schulte, D., Zimmermann, F.: Status and challenges of the Future Circular Hadron Collider FCC-hh. In: Proceedings of 41st International Conference on High Energy Physics — PoS(ICHEP2022), vol. 414, p. 058 (2022). https://doi.org/10.22323/1.414.0058
- [48]Yildiz, B.I., Jaeger, H., Kiebel, S.J.: Re-visiting the echo state property. Neural Networks 35, 1–9 (2012)DOI