WorksPhysics and Astronomy
Beyond global density in a stochastic sandpile: the role of spatial signatures in the predictability of extremes
Abstract
In self-organized critical systems the global load is a strong precursor of extreme events. In the two-dimensional Manna sandpile, grain density alone predicts extreme avalanches with an AUC of about 0.96. We ask whether density fully determines the system's readiness for an extreme event, that is, whether microstates with the same number of grains are equivalent. Because density is such a strong predictor, a method that sees the full microstate can score well just by counting grains. We therefore compare only states with an identical number of grains. Even then, both a parameter-free statistic and a convolutional network distinguish states that precede an extreme avalanche from typical ones, while shuffled controls do not. The signal is a load contrast between the center and the dissipative boundaries, carried by the two lowest symmetric cosine modes. Their normalized energy is a single variable that needs no training. Together with density it recovers most of the gain that a network trained on the full microstate achieves over density alone. Readiness for an extreme avalanche therefore depends both on the total load and on how that load is distributed relative to the boundaries.
Introduction
Self-organized criticality (SOC) describes slowly driven systems that reach a critical state without any tuning of parameters and relax through avalanches with a power-law size distribution [1, 2, 3]. The concept has been applied to earthquakes [4], solar flares [5] and neuronal activity [6], and in each of these settings the impact is dominated by extreme events. The absence of a characteristic scale suggested that a large event has no precursor [7, 8]. Between large events, however, an SOC system is slowly reloaded, so the state just before an extreme event may differ from a typical one [9, 10]. Precursors of extreme avalanches in sandpile models have been sought in time series of avalanche sizes and activity [10, 11, 12, 13] and in features of the system state [9, 14, 15]. In an experiment on a pile of beads they were also sought in changes of its internal structure [16].
Here we study the Manna model [17] in its Abelian variant [18], which is regarded as the representative of a universality class distinct from that of the deterministic model of Bak, Tang and Wiesenfeld (BTW) [3, 19, 20]. For this model the question of a precursor has a partial answer. The mean height , which is the global load of the system, predicts extreme avalanches very well [9]. This fits the picture of SOC as an absorbing-state phase transition in which density is the control parameter [21, 22]. Following Ref. [9], we call a state with elevated supercritical. If told the whole story, microstates with the same number of grains would be equally ready for an extreme event. Earlier studies indicated that they are not. Before the largest avalanches the grains were seen to gather into clusters, and a precursor built from these clusters could forecast large avalanches [9, 14, 15]. It remained open whether this component carries information independent of density or merely reflects it [15]. To our knowledge it was also not known what form this information takes or whether it can be expressed without tuned parameters.
The question is whether the number of grains alone is sufficient. Our focus is not on improving the forecast, but on identifying the macroscopic quantities that describe the system’s readiness for an extreme event. A direct comparison of spatial arrangement is difficult because density is such a strong predictor that a method using the full microstate, including a neural network, can perform well simply by counting grains. This creates the possibility of shortcut learning [23] and of attributing the prediction to the wrong physical mechanism [24]. We therefore remove density by thinning each configuration at random to a common number of grains and use shuffled configurations to define the null level. We then examine the remaining spatial information with a model-free statistic and with a convolutional neural network. The relevant scale is tested by destroying structure below a prescribed length and by comparing different system sizes. Finally, we expand the microstate in cosine modes, for which density is exactly the zero mode, and use this representation to identify a small set of macroscopic variables. The network is used only as a reference. Related work has already applied neural networks to avalanche prediction from sandpile configurations [25].
Methods
Model and prediction task
The Manna model is a stochastic sandpile on a square lattice of side . At each time step one grain is added to a randomly chosen site. A site that holds at least two grains becomes unstable and passes two of them on, each to an independently drawn neighbor (von Neumann neighborhood). Toppling continues as long as any site is unstable. Grains that topple across the edge of the lattice leave the system, so the boundary is dissipative. Between avalanches every site is either empty or occupied by a single grain, for . The microstate is thus a binary image, and its global load is the mean height, equal to the number of grains divided by the number of sites,
We keep the index for quantities followed along the run, such as , and drop it when we refer to a single, fixed image.
The avalanche size is the number of topplings triggered by the grain added at step to the microstate . We call an avalanche extreme when , where is the scaling exponent of avalanche sizes [9, 10] and is the threshold for which Ref. [9] reports the most efficient prediction at . We choose this threshold so that the prediction from density alone can be compared directly with the results of that work. We use it at every system size. A microstate is labeled if the avalanche that follows it is extreme and otherwise. The label thus refers to an avalanche that has not happened yet, and no image shows the avalanche it is labeled with. The threshold scales with the system size in the same way as the cutoff of the avalanche distribution, so at every we ask about roughly the same kind of event. Still, the frequency of extremes falls with faster than scaling predicts. An avalanche distribution with exponent [9] implies that doubling reduces the number of extremes in a run of the same length to of its previous value. In our data it falls to 0.33 between and 64 and to 0.49 between and 128 (Table I, positives from both simulation phases combined). This means that the smallest system still has clear finite-size corrections. Extremes are relatively more frequent there, and a comparison with the larger systems does not concern exactly the same kind of event.
Data and evaluation
For each system size we warm up the simulation for steps and then run two phases of steps each, a training phase and a test phase that continues the same run. We call the sequence of all consecutive states of a phase a stream. The positives are all the extremes that occurred. The negatives are quiet states taken every 50 000 steps, with no condition on density or on history (Table I). For every example we store a window of fifty consecutive configurations, the last of which is the microstate that precedes the labeled avalanche. A single configuration is called a frame or an image. All predictors use only the last frame, because the dynamics is Markovian and the earlier frames carry no information about the avalanche size beyond what the last one contains. The data generation and collection process is presented on Figure 1. The full windows are used only to set the thinning level (§ 2 C). The test set is used only for the final evaluation. Every constant and every mode later in the paper comes from the training set or from a validation part split off from it (20%, stratified by label and drawn separately for each of the five random seeds).
Figure 1. Data generation for each system size . After a warm-up of steps, one run continues through a training and a test phase of steps each. Each example stores 50 frames ending in . Predictors use only , and the full window only sets the thinning level. All choices come from the training set, with a validation split stratified by label and drawn per seed (5 times).
| training: total | training: positives | test: total | test: positives | negatives per phase | |
| 32 | 18 114 | 8 114 | 18 108 | 8 108 | 10 000 |
| 64 | 12 674 | 2 674 | 12 741 | 2 741 | 10 000 |
| 128 | 11 339 | 1 339 | 11 309 | 1 309 | 10 000 |
Table 1. Collected examples for each system size .
AUC: For a predictor mapping each state to a scalar score ,
where and are the scores of a randomly drawn positive state and an independently drawn negative state [26, 27]. This quantity does not depend on the class proportions, and a score that returns the same number for every state produces only ties, which gives .
Prediction error: To compare with earlier work [9, 10] we also use the error [28, 29], the sum of the fraction of missed extremes and the false-alarm rate , i.e., the fraction of quiet states on which an alarm is raised, at a single threshold. The threshold is chosen by minimizing on the training set and is then applied to the test set. Computed on the full test stream, this error is directly comparable with the results of Ref. [9]. Computed on the sets of Table I, it is used to compare predictors. Neither metric depends on the class proportions, which is why we keep all of the rare positives and subsample the abundant negatives.
Removing density: thinning and conditioning on the number of grains
Because is a good predictor, no statement about space that rests on raw images can separate arrangement from density. We therefore remove density from the data.
Thinning: From every image we remove grains at random, without looking at the label, until exactly remain. Parameter is a single number per system size, shared by both classes, all examples and all predictors. It equals the smallest number of grains found in any frame of the training set. This level is not tuned. It is forced on us, because grains may be removed but must not be added. Adding a grain would mean inventing its position, which is exactly the quantity we are asking about. From the last frames, on which the predictors operate, thinning removes on average between about 1% and 8% of the grains, depending on the system size (Table II). Test examples whose last frame has fewer than grains are discarded. After thinning every image has density , the same in both classes, so a predictor that uses density alone returns a constant and, by the tie term of Eq. (2), has an AUC of exactly 1/2. A binary image with a fixed number of ones remains, and two such images can differ only in where the ones are. During training the thinning is drawn anew every time an example is read. During evaluation each example is always thinned in the same way, which means that all models are evaluated on identical images.
| frames | removed: worst frame | removed: mean | discarded test examples | ||
| 32 | 905 700 | 659 | 14.4% | 7.2% | 0 of 18 108 |
| 64 | 633 700 | 2 790 | 6.1% | 2.9% | 0 of 12 741 |
| 128 | 566 950 | 11 464 | 2.6% | 1.1% | 1 of 11 309 |
Table 2. The forced thinning level and the fraction of grains that thinning removes, computed over all frames of all training windows. In the last frames, on which the predictors operate, this fraction is slightly larger. The last column gives the number of test examples discarded because their last frame has fewer than grains.
Null level: We repeat every run on shuffled images, in which the grains are first scattered uniformly over the lattice and only then thinned to the same . The architecture, the training schedule and the seeds of the random number generator stay the same. The result of such a run is the null level, the amount of signal that the training and evaluation procedure can produce by itself.
Conditioning on the number of grains: Thinning equalizes the number of grains, but it removes more grains from a frame that had more of them. This leaves a weak trace of the original number of grains , and the null level does not capture this trace. For this reason we also compare only states with an identical original number of grains . Within such a group exactly the same number of grains is removed from every image, so the procedure is statistically identical for both classes and cannot separate them by itself. Also, is constant within a group, so its conditional AUC is exactly 1/2. We report the conditional AUC computed jointly over all pairs of a positive and a negative with the same number of grains,
where and are the positives and the negatives with grains, is the score of example , and equals 1 when the condition holds and 0 otherwise. This control does not require thinning, so we also apply it to raw images.
Spatial observables
After the number of grains is fixed, the remaining information in an image is the spatial arrangement of the grains. We examine this arrangement in two ways. The first is a quantity computed directly from the image, without machine learning. The second progressively removes structure below a given scale and tests which scales contribute to the network prediction.
Excess block-density variance
We divide the lattice into non-overlapping blocks of side and compute the density of each, . The variance of these densities measures how uneven the arrangement is at scale [30],
where is the density of the image on which is computed. By varying from to we probe the structure at successive scales. The blocks must tile the lattice exactly, so the scan has a resolution of one octave. The raw difference in between the classes cannot be compared across scales or across system sizes, because the block-density variance depends on the block size even when the grains are placed completely at random. This is why we express the difference between the classes in units of the variance that a random placement of the same number of grains would give at that scale,
where and are the means over positives and over negatives, and is the expected value of for the same number of grains scattered over the lattice completely at random (derived in Appendix A). The random level is used here as a unit and is not meant as a model of the system. In models of the Manna class the density fluctuations of critical states are suppressed relative to a random arrangement [31]. The scan is computed on thinned images, so and a single applies to both classes. Thinning weakens the excess by an amount that depends on the system size but not on the scale. For this reason we divide each curve by its own maximum and compare, between sizes, the shape of and the position of its maximum.
Destruction scan
The block-density variance tells us at which scale the classes differ in the data, but not which scale the network uses. To put the same question to the network on the same axis, we divide the lattice of each thinned test image into blocks of side and scatter the grains of every block at random over the sites of that block. This keeps the number of grains in each block, and with it all structure at scales larger than , and removes only the arrangement of the grains inside the blocks. It is a counterpart of the surrogate-data method [32]. The network trained on thinned images is evaluated on the modified images without retraining, and we measure the fraction of its advantage over the random level that is lost,
where is the result on intact images and the result on images destroyed below scale . The curve is cumulative, so the scale of the signal shows up in its increments. The difference is the fraction of the network’s advantage that comes from structure with sizes between and . The normalization relates the loss to the advantage of the given model, which makes different sizes comparable. The receptive field of the network grows in proportion to (§ 2 F), so the scan shows which scales are used by a network that was allowed to use them. As evidence about scale, the scan is not independent of this design choice.
Modal decomposition
The tools of the previous section show at which scale the spatial information lies, but they do not say what shape it has, and they rely on thinning. The modal decomposition fills both gaps. We write the microstate in a basis in which density is one coordinate and each of the others describes a spatial pattern with a definite scale and orientation. Let be the microstate stored in the last frame of a window, without thinning, and its value at site . We expand the microstate in an orthonormal cosine basis (the two-dimensional DCT-II) [33],
with amplitudes , where
and the indices label the modes. The function changes sign times across the system, so small indices correspond to long-wavelength modes and large indices to fine-scale structure. We choose this basis because its zero mode is the density, , while every mode describes only the arrangement of the grains. This separation is algebraic and not statistical, which is why we also assess the predictive power of the modes within groups of states with an identical number of grains (§ 2 C). We also do not claim that this basis diagonalizes the dynamics. It diagonalizes the lattice Laplacian with a reflecting boundary [33], whereas the boundary of the model is dissipative. The contribution of a pattern to the microstate is measured by the mode energy . The total energy of the nonzero modes equals , which follows from Parseval’s theorem and from . This total is fixed by alone and carries no new information. Only the way this energy is distributed among the modes is new, so we work with the normalized mode energies
that is, with the share of a given mode in the total inhomogeneity of the microstate. Each mode is treated separately as a one-dimensional predictor and assessed by . The mode with the largest AUC is selected on the training set only, and the degenerate modes , which map onto each other under a rotation by , are summed. The resulting quantity is invariant under rotations and reflections of the lattice, the same symmetries that the dynamics of the model has.
Predictors
The block-density variance and the normalized mode energies are used directly as scores and have no parameters fitted to the data. The scale and the pair of modes are chosen on the training set. We combine density and the selected mode energy by logistic regression without regularization, on features standardized with the statistics of the training set,
where the logit is the logarithm of the odds , is the sum of the normalized energies of the degenerate pair picked by the selection rule of § 2 E, and the tilde denotes standardization. The fitted coefficients and give the change in the log-odds of an extreme per one standard deviation of and of , and is the intercept. The model is fitted on the training set and applied unchanged to the test set.
The reference is a convolutional neural network [34]. It consists of convolutional blocks with stride 2 (convolution, batch normalization [35], ReLU), a head that computes the mean and the standard deviation over positions for every channel of the last feature map, and a linear layer. The number of blocks grows with the system size (3, 4, 5 for ), so that the receptive field stays a fixed fraction of the system side instead of a fixed number of sites. The head averages over positions, so by construction the network is only weakly sensitive to where in the system a given structure lies. We regard it as a typical general-purpose model. It gives a lower estimate of what can be extracted from the microstate, not an upper bound. We train five models from independent initializations and average their metrics, never their scores. Training details and learning curves are given in Appendix B.
Statistics
Confidence intervals come from a bootstrap [36] over test examples (95% percentile intervals from 2000 resamples, paired when predictors are compared). In comparisons with the network the bootstrap is run with the five training seeds held fixed, so it does not include the spread between seeds. We report that spread separately as a standard deviation (). The values come from a one-sided permutation test with 2000 label permutations, for which the smallest attainable value is 0.0005 [37]. The bootstrap assumes that the examples are independent. For the positives this is supported by the coefficient of variation of the intervals between extremes, 0.98–1.05, which is consistent with a Poisson process. For the negatives the autocorrelation of at the sampling interval (50 000 steps) is indistinguishable from zero (its absolute value does not exceed 0.02 at any size).
Results
We first establish density as the baseline predictor. We then characterize the spatial signal by its scale and shape and reduce the dominant pattern to the single variable . The final comparison asks how well the pair reproduces the performance of a network trained on the full microstate. Unless stated otherwise, the reported values come from the test set.
Baseline: density as a predictor of extremes
We begin by checking that our simulations reproduce the known result that the density alone predicts extreme avalanches well. The avalanche-size distributions for the three system sizes collapse after rescaling (Figure 2a), as described for the Manna model [10], and density separates states before an extreme from quiet states with an AUC of 0.95 to 0.97 (Figure 2b, Table III). Reference [9] reports, for and the same threshold , the values AUC = 0.946, and , obtained from 131 test events.
| AUC() | at | |||||
| 32 | 0.9542 | 1.330 | 2.071 | 0.7002 | 0.228 | 0.228 |
| 64 | 0.9674 | 1.298 | 2.190 | 0.7058 | 0.192 | 0.195 |
| 128 | 0.9725 | 1.293 | 2.222 | 0.7099 | 0.172 | 0.174 |
Table III. Baseline: prediction from density alone on the full test stream. The presented quantities are described as follows: (i) is the exponent of the avalanche-size distribution (linear regression on a log-log scale, 49 logarithmic bins, range ), (ii) is the absolute slope of the linear dependence of the error on the rescaled extreme threshold [9] (linear regression for , with the density threshold chosen separately for each ), (iii) is the density threshold that minimizes the error in the training phase (an alarm is raised when ), and (iv) is the smallest error attainable by a threshold rule on density alone. Its threshold is chosen on the test stream itself, so it is a lower bound and not a predictor. The last column is the prediction error at the threshold .
Figure 2. (a) Probability densities of avalanche sizes after both axes are rescaled with the system size, using and [9]. The fitted power-law exponents are given in the legend. (b) ROC curves for the density alone, which separates states before an extreme from quiet states with AUC -0.97. The figures are outcomes of simulation of the algorithm presented by Shapoval et al. [9], and prepared by the authors of this manuscript.
Our results at (AUC = 0.973, , , 1309 events) agree with these. The rate of extremes agrees as well (131 in steps against 1309 in ), and so does the slope of the linear decrease of the error with the threshold ( in Ref. [9] for larger systems, while our values grow with up to 2.22, Table III). This confirms that the simulation and the data collection are correct, and it sets the baseline for the comparisons that follow.
The strength of this predictor also makes the question about the arrangement harder to answer. A convolutional network trained on the collected examples (Table I) without thinning reaches an AUC of 0.990 to 0.996, which is more than density alone. On raw images, however, this surplus cannot be cleanly isolated, because by far the largest part of the score can be obtained just by counting grains [23]. In the next step we therefore remove density from the data.
Spatial information at a fixed number of grains
Thinning (§ 2 C) brings every image to the same number of grains, so density is no longer a variable and its AUC is 1/2 by construction. If the classes can still be told apart, only the arrangement of the grains can be responsible. We test this with two tools, the block-density variance at , computed directly from the image, and a convolutional network (Table IV). We fix the scale before looking at the test set: is where the excess of Section III C peaks on the training set.
| , raw | , raw | , thinned | , null level | network, thinned | network, null level | |
| 32 | 0.954 | 0.691 | 0.680 | 0.500 0.003 | 0.8042 0.0012 | 0.4999 () |
| 64 | 0.967 | 0.744 | 0.731 | 0.499 0.006 | 0.8567 0.0024 | 0.4960 () |
| 128 | 0.974 | 0.782 | 0.779 | 0.498 0.011 | 0.8658 0.0050 | 0.5017 () |
Table 4. Spatial signal at a fixed number of grains (AUC on the sets of Table I). The quantity is the block-density variance at , and the null level is measured on shuffled images. The spread refers to five random shuffles (block variance) or five training seeds (network). The values come from the permutation test.
Both tools distinguish the classes at a fixed number of grains, the block variance with an AUC of 0.68 to 0.78 and the network with 0.80 to 0.87. The block variance loses almost nothing through thinning, and the null level coincides with the random level in both cases. The signal is therefore not produced by the training and evaluation procedure itself. Nor is it the trace of the original density that thinning leaves behind. Within groups of states with an identical number of grains the AUC (Eq. (3)) does not drop. It rises to 0.72-0.85 for the block variance and to 0.89-0.93 for the network. The groups that contain both classes cover from 78% (at ) to 95% (at ) of the positives, so the conditioning covers most of the data.
At an identical number of grains, then, the arrangement alone separates states before an extreme from quiet states. The number of grains does not fully describe the state. Much of the signal is already carried by a statistic that involves no training, so the effect is not an artifact of a flexible model.
Scale of the signal: a fixed fraction of the system size
We now ask at what spatial scale the information that separates the classes is found. We use both tools of § 2 D, the excess block variance , which describes the data, and the destruction scan , which describes the network (Figure 3).
Figure 3. (a) Shape of the excess block variance, divided by its own maximum. Error bars are 95% bootstrap intervals. At every size the maximum falls at (dashed line). (b) Lost fraction of the network’s advantage after the structure below scale is destroyed. The curve is cumulative, so the scale of the signal is given by its largest increment, between and . In both panels the scale decreases to the right, from to (at to ).
The maximum of falls at for all three sizes, both on the training set, from which we read it off, and on the test set. This corresponds to blocks of to sites, a fixed fraction of the system size instead of a fixed number of sites. Near the maximum the shapes of the curves agree. The largest discrepancy between sizes, at , falls within the confidence intervals when the scan is repeated at a fixed original number of grains, which marks it as a trace of thinning and not a property of the arrangement (Appendix A). The position of the maximum is known only to within an octave, because the scan covers only scales that differ by a factor of two.
The destruction scan leads to the same conclusion. Destroying the structure below takes away less than half of the network’s advantage. More than half (0.55 to 0.59) disappears only when the destruction is extended to , so it comes from structure with sizes between and .
The model-free scan thus places the signal at a scale proportional to , and the destruction scan shows that a network whose receptive field scales with the system uses the same band. That the scale of the precursor is proportional to had been conjectured earlier [15]. The scan alone does not tell us whether this proportionality reflects the self-similarity of the critical state or a load profile imposed by the dissipative boundary, because the distance from the boundary also scales with . A similar doubt was raised for the BTW model [12]. We return to this point in the discussion.
Shape of the signal: a contrast between the center and the boundaries
The scale does not yet tell us what shape the structure behind the signal has. To find out, we expand the microstates in cosine modes (§ 2 E) and check for each mode how well its normalized energy distinguishes the classes (Figure 4).

Figure 4. AUC of the normalized energy of every mode on the training set for (a) , (b) and (c) . Each panel shows the full spectrum, and the inset enlarges the modes with , the region framed in the upper left corner. The numbers next to the selected pair are its AUC values. White marks a mode that carries no information (AUC ) and darker red a mode with higher energy before an extreme. No mode falls below AUC , so values under are also drawn in white. The mode corresponds to density and is not assessed here. At every the same pair, and , stands out.
At every size the two strongest modes are the same pair, and , with an AUC of 0.74 to 0.77 on the training set, while no other mode exceeds 0.59. The pair was identified at and then kept fixed. This means that at and the pair was not selected on the data on which it is assessed, and the corresponding panels of Figure 4 are a confirmation and not a discovery.
The pattern has a simple geometric interpretation. The function is positive near both boundaries and negative in the middle, and it changes sign at a distance from each boundary. The amplitude of the mode thus measures the load contrast between the central strip and the boundary strips, and the mode measures the same along the other axis. In the great majority of states of both classes the center is heavier. This is expected, because grains leave the system through the boundary and the stationary density is lower there. The classes differ in the strength of this overload. The mean signed amplitude, normalized by and averaged over both orientations, is larger in absolute value before an extreme than in quiet states, by a factor of at falling to at (Appendix C).
A single spatial precursor
The two modes that stand out describe the same pattern in two orientations and map onto each other under a rotation of the lattice by . Following the rule of § 2 E, we sum their normalized energies,
is a single number per microstate, computed from a single frame and without training. It measures the strength of the contrast between the center and the boundaries regardless of its sign and orientation.
The quantity is separated from density algebraically but not statistically. Its AUC therefore does not by itself show whether the variable contains information beyond the total number of grains. Conditioning on the original number of grains provides this test (§ 2 C). The AUC of remains high within groups with the same , reaching -, whereas density has an AUC of exactly (Table V). Thus the spatial signal persists without thinning and without a network. The geometry of also accounts for the two scale scans in Section III C. Blocks of side coincide with the strips in which the mode of index 2 has a constant sign and separate the central strip from the boundary strips. A block of side contains equal parts of the two regions, so rearranging grains within it cancels the contrast (Appendix C).
| 32 | 0.83 | 0.92 | 0.500 |
| 64 | 0.85 | 0.95 | 0.500 |
| 128 | 0.86 | 0.97 | 0.500 |
Table V. The precursor with density removed by conditioning instead of thinning. The value of is computed based on Eq. (3) within groups of states with an identical number of grains (Section II C).
Two macroscopic variables against the full microstate
We next consider prediction from the raw images, where density is available. The logistic model of Eq. (10) combines density with . Its performance can be compared with the network of § 2 F, which is trained on the full microstate and therefore has access to all sites (Table VI).
| [95% CI] | (network) | ||||||
| 32 | 0.954 | 0.826 | 0.985 | +0.032 [0.029, 0.034] | 0.228 | 0.118 | 0.097 |
| 64 | 0.967 | 0.852 | 0.993 | +0.026 [0.023, 0.028] | 0.191 | 0.075 | 0.066 |
| 128 | 0.974 | 0.856 | 0.996 | +0.023 [0.019, 0.026] | 0.170 | 0.059 | 0.061 |
Table VI. Two macroscopic variables against a convolutional network on the full microstate. The pair denotes the logistic model of Eq. (10). All values are computed on the sets of Table I, so and may differ from the values in Table III, which are computed on the full stream. Here the negatives are a sample of 10 000 states and the threshold is chosen on the collected training set (which is why at and 128 comes out slightly below the bound determined on the stream). The confidence interval of comes from a paired bootstrap over test examples. The difference is computed before rounding, so it may deviate by 0.001 from the difference of the columns.
Adding to density lowers the error by about 0.11 at every size, which amounts to 84% to 102% of the reduction achieved by the network on the full microstate (95% confidence intervals from 80-88% at to 94-110% at ). At and 64 the network remains better. At the two variables match it within the spread between training seeds (0.059 against 0.061 0.008). Figure 5 shows why density alone is not enough. The clouds of the two classes are shifted relative to each other along both axes, so the decision boundary is oblique.
Figure 5. Test states in the plane of the two macroscopic variables, the density and the mode energy , for (a) , (b) and (c) (at most 2500 randomly chosen states per class). The values of are given in units of , and the range of the vertical axis differs between panels. Quiet states are gray, and states before an extreme have the color of their system size. The dashed line is the decision boundary of the logistic model, on which the model gives equal odds to both classes. Its coefficients were fitted on the training set and then frozen, so the line is shown on data the model has not seen. The alarm threshold used for in Table VI is a parallel line shifted toward lower load. The clouds are displaced both horizontally (higher load) and vertically (stronger contrast between the center and the boundaries), which is why two variables separate them better than either one alone.
The pair is not a complete description. On thinned images, where density is switched off, alone reaches an AUC of 0.75-0.83, compared with 0.80-0.87 for the network. Neither of these scores contains the other, however. Within groups of similar (20 quantiles) following the same (3), the network still distinguishes the classes with an AUC of 0.70-0.75, so it also uses structure that does not describe. Within groups of similar network score, still gives 0.54-0.64. The network is also overfitted at the larger sizes (Appendix B), which in the comparison with the two-variable model works in favor of that model.
Discussion and Conclusions
The main result is that the global density does not completely describe the readiness of the Manna model for an extreme avalanche. Density is a strong predictor, consistent with earlier work [9], but states with the same number of grains remain distinguishable. After thinning to a fixed grain count, both a model-free statistic and a convolutional network perform above the null level. Conditioning on the original number of grains gives the same result and rules out a residual dependence on the thinning procedure. The remaining signal is macroscopic: it appears on a scale proportional to the system size and is associated with a load contrast between the center and the dissipative boundaries, represented by the degenerate cosine modes and . The corresponding scalar separates states with an identical grain count. When density is restored, the pair recovers most of the error reduction obtained by the network over density alone. Thus the number of grains is not a complete description of the system’s state with respect to extreme-avalanche prediction.
This result adds to the picture of SOC as an absorbing-state phase transition in which density is the control parameter [21, 22]. In a system with open boundaries the load is not uniform, so besides its mean value it also matters how the load is distributed relative to the boundaries. A natural interpretation, which we treat as a hypothesis and not as a measured result, is that an avalanche developing in an overloaded interior has a longer way to the dissipative boundary and can grow larger because of this. The hypothesis has some support. The variant of the Manna model studied here is Abelian [18], and in Abelian sandpile models the balance of grains in the stationary state relates the mean number of topplings to the Green’s function of the Laplacian with a dissipative boundary [38]. This function is largest for grains added at the center of the system. If the precursor is tied to the boundary, the fact that the maximum falls at the same for all sizes cannot decide between the self-similarity of the critical state and a load profile imposed by the finite system. The distance from the boundary scales with as well. We therefore read this agreement as evidence against an artifact with a fixed size in lattice sites. It does not prove self-similarity. Within the range studied, the role of spatial organization does not weaken with system size. The conditional increases, although the relative difference in contrast between the classes decreases.
That extreme avalanches are preceded by spatial clustering has been reported previously [9, 14, 15]. Here the spatial contribution is isolated by construction, quantified against the null level, and localized in scale. The modal decomposition then identifies the dominant pattern and reduces it to the scalar . Comparing with the network trained on the full microstate shows that this variable captures a large part of the information available beyond density. The network is therefore useful mainly as a reference for the information content of the microstate, while the modal analysis gives a direct physical description of the dominant spatial component.
Several limitations remain. The pair does not exhaust the spatial information: on thinned images the network still uses structure that does not describe. Whether density and this residual structure can be replaced by a single quantity derived from the dynamics, as suggested by the boundary hypothesis, remains to be established and is the subject of a separate paper. The present simulations also cover only a fourfold range of system sizes, –. Because larger systems require more computation and contain fewer extreme events, these data do not determine the large- limit or separate a boundary effect from a property of the critical state. A model with bulk dissipation and periodic boundaries, without a distinguished center, would provide a useful test. Finally, it remains unclear whether the same spatial description applies to other sandpile models, including the deterministic BTW model, which belongs to a different universality class [19] and has different finite-size predictability [10]. Within the range studied here, however, the predictive state depends on both the number of grains and their spatial arrangement, and most of the network’s additional information is captured by the single variable .
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Data Availability
The code that supports the findings of this study is available from the corresponding author upon request.
Acknowledgements
J.S. is financially supported by the Foundation for Polish Science (FNP) grant ‘Centre for Credible AI’ No. FENG.02.01-IP.05-0058/24.
Declaration of generative AI and AI-assisted technologies in the manuscript preparation process
During the preparation of this work the authors used Claude (Anthropic) and GPT (OpenAI) in order to assist with checking the reported numbers against the data and with language editing of the manuscript. After using these tools, the authors reviewed and edited the content as needed and take full responsibility for the content of the published article.
Scale scans: random level and the effect of thinning
Random level. The quantity of Eq. (5) is not postulated but derived. Consider grains placed uniformly at random on sites, with at most one grain per site. The number of grains in a block of sites then follows a hypergeometric distribution with variance
The density of a block is and the mean density of every block is , so the expected value of equals , which is the expression in Eq. (5). For a block is a single site and for every configuration, so by construction. At we therefore omit the point .
Effect of thinning. Removing grains at random from an image with grains multiplies its excess variance over the random level by , equally at every scale . This is why we compare the shape of the curves between sizes. Positives, however, have more grains than negatives, so their excess is weakened more, which adds a contribution to that has nothing to do with the arrangement. This is the same trace of the original number of grains that is discussed in § 2 C, and we remove it in the same way. We repeat the scan at a fixed original number of grains. The difference is computed within groups of identical and averaged with weights . In the repeated scan, normalized as in Figure 3a, the maximum stays at , and the relative excess at is 0.23-0.28 compared with 0.38-0.58 in the basic scan, with widely overlapping confidence intervals.
Neural network: training, learning curves and generalization gap
Architecture and training. The head of the network of § 2 F computes, for each of the 32 channels of the last feature map, the mean and the standard deviation over positions, which gives 64 numbers per image. The convolutions have kernels, and the blocks have between 16 and at most 64 channels. The last feature map has size at every , and the network has between 14 and 79 thousand parameters. During training we apply random elements of the symmetry group (rotations by multiples of and reflections) as augmentation. The dynamics of the model is invariant under these operations. We use the Adam optimizer [39] (learning rate , weight decay ), a batch size of 64, at most 100 epochs with a patience of 10, and early stopping based on the validation loss instead of the AUC. The loss function is the binary cross-entropy.
Learning curves. Figure B.1 shows the AUC on the validation part of the training set after each epoch, separately for each of the five seeds. A dot marks the epoch with the smallest validation loss, from which the weights used in the paper are taken.
None of the 45 training runs (thinned, shuffled and raw images, three system sizes, five seeds) reached the limit of 100 epochs. The stopping criterion ended them after 11-48 epochs, on the flat part of the curve. The mean AUC of the networks shown in the figure on the test set, which takes no part in training, differs from the validation AUC by at most 0.018. The network trained on shuffled images stays at the random level throughout training (validation AUC between 0.46 and 0.54).
Generalization gap. At and 128 the network is overfitted. On thinned images its AUC on the examples it was trained on exceeds the test AUC by about 0.01 and 0.04 (the difference is negligible at ), which lowers the result in Table IV and does not inflate it. On raw images its error at rises from 0.034 on the examples it was trained on to 0.057 on its own validation part, which contributes no gradient steps (in both cases with the threshold chosen on the same part). On the test set, with the threshold taken from the training set, it is 0.061 (Table VI). In the comparison with the two-variable model this works in favor of that model.
Spatial pattern: direction of the contrast and properties of
Direction of the contrast. The sign of the normalized amplitude tells which part of the system is heavier. A negative value means a heavier center. In each of the two orientations separately, the amplitude is negative in at least 87% of quiet states and in at least 97% of states before an extreme.
Figure B.1. Learning curves of the convolutional network. Values of AUC on the validation part after successive training epochs (a)–(c) on thinned images and (d)–(f) on raw images, for , and from left to right. Each line is one of the five seeds, and a dot marks the epoch from which the saved weights are taken. The dashed line is the AUC of these weights on the test set, averaged over seeds.
and density. Within each class the Spearman rank correlation between and is negative, from to in quiet states and from to before an extreme. Density thus partly masks the signal of , and conditioning on the number of grains lifts this mask. This explains the increase of in Table V.
and the scale of the signal. After the grains are rearranged inside blocks of side , still distinguishes the classes (the AUC drops from – to –). After rearrangement inside blocks of side it no longer does (–), in line with the geometric argument of Section III E. The same argument explains the maximum of at and the largest increment of between and .
References
- [1]P. Bak, C. Tang, K. Wiesenfeld, Self-organized criticality: An explanation of the 1/f noise, Phys. Rev. Lett. 59 (1987) 381–384. doi:10.1103/PhysRevLett.59.381.DOI
- [2]H. J. Jensen, Self-Organized Criticality: Emergent Complex Behavior in Physical and Biological Systems, Cambridge University Press, Cambridge, 1998. doi:10.1017/CBO9780511622717.DOI
- [3]G. Pruessner, Self-Organised Criticality: Theory, Models and Characterisation, Cambridge University Press, Cambridge, 2012. doi:10.1017/CBO9780511977671.DOI
- [4]P. Bak, C. Tang, Earthquakes as a self-organized critical phenomenon, J. Geophys. Res. 94 (1989) 15635–15637. doi: 10.1029/JB094iB11p15635.DOI
- [5]T. E. Lu, R. J. Hamilton, Avalanches and the distribution of solar flares, Astrophys. J. Lett. 380 (1991) L89. doi: 10.1086/186180.
- [6]J. M. Beggs, D. Plenz, Neuronal avalanches in neocortical circuits, J. Neurosci. 23 (2003) 11167–11177. doi:10.1523/JNEUROSCI.23-35-11167.2003.DOI
- [7]P. Bak, M. Paczuski, Complexity, contingency, and criticality, Proc. Natl. Acad. Sci. U.S.A. 92 (1995) 6689–6696. doi: 10.1073/pnas.92.15.6689.DOI
- [8]R. J. Geller, D. D. Jackson, Y. Y. Kagan, F. Mulargia, Earthquakes cannot be predicted, Science 275 (1997) 1616–1616. doi:10.1126/science.275.5306.1616.DOI
- [9]A. Shapoval, D. Savostianova, M. Shnirman, Universal predictability of large avalanches in the manna sandpile model, Chaos 32 (2022) 083130. doi:10.1063/5.0102019.
- [10]D. Sapozhnikov, A. Shapoval, M. Shnirman, Comparing prediction efficiency in the btw and manna sandpiles, Sci Rep 14 (2024) 29259. doi:10.1038/s41598-024-80621-w.DOI
- [11]S. L. Pepke, J. M. Carlson, Predictability of self-organizing systems, Phys. Rev. E 50 (1994) 236–242. doi:10.1103/PhysRevE.50.236.DOI
- [12]A. Garber, S. Hallerberg, H. Kantz, Predicting extreme avalanches in self-organized critical sandpiles, Phys. Rev. E 80 (2009) 026124. doi:10.1103/PhysRevE.80.026124.DOI
- [13]A. Deluca, N. R. Moloney, Á. Corral, Data-driven prediction of thresholded time series of rainfall and self-organized criticality models, Phys. Rev. E 91 (2015) 052808. doi:10.1103/PhysRevE.91.052808.DOI
- [14]A. B. Shapoval, M. G. Shnirman, Strong events in the sand-pile model, Int. J. Mod. Phys. C 15 (2004) 279–288. doi:10.1142/S012918310400570X.DOI
- [15]A. B. Shapoval, M. G. Shnirman, How size of target avalanches influences prediction efficiency, Int. J. Mod. Phys. C 17 (2006) 1777–1790. doi:10.1142/S0129183106010212.DOI
- [16]O. Ramos, E. Altshuler, K. J. Måløy, Avalanche prediction in a self-organized pile of beads, Phys. Rev. Lett. 102 (2009) 078701. doi:10.1103/PhysRevLett.102.078701.DOI
- [17]S. S. Manna, Two-state model of self-organized criticality, J. Phys. A: Math. Gen. 24 (1991) L363. doi:10.1088/0305-4470/24/7/009.DOI
- [18]D. Dhar, Some results and a conjecture for manna’s stochastic sandpile model, Physica A 270 (1999) 69–81. doi:10.1016/S0378-4371(99)00149-1.DOI
- [19]A. Ben-Hur, O. Biham, Universality in sandpile models, Phys. Rev. E 53 (1996) R1317–R1320. doi:10.1103/PhysRevE.53.R1317.
- [20]S. Lübeck, Universal scaling behavior of non-equilibrium phase transitions, Int. J. Mod. Phys. B 18 (2004) 3977–4118. doi:10.1142/S0217979204027748.DOI
- [21]R. Dickman, A. Vespignani, S. Zapperi, Self-organized criticality as an absorbing-state phase transition, Phys. Rev. E 57 (1998) 5095–5105. doi:10.1103/PhysRevE.57.5095.DOI
- [22]R. Dickman, M. A. Muñoz, A. Vespignani, S. Zapperi, Paths to self-organized criticality, Braz. J. Phys. 30 (2000) 27–41. doi:10.1590/S0103-97332000000100004.DOI
- [23]R. Geirhos, J.-H. Jacobsen, C. Michaelis, R. Zemel, W. Brendel, M. Bethge, F. A. Wichmann, Shortcut learning in deep neural networks, Nat Mach Intell 2 (2020) 665–673. doi:10.1038/s42256-020-00257-z.DOI
- [24]G. Carleo, I. Cirac, K. Cranmer, L. Daudet, M. Schuld, N. Tishby, L. Vogt-Maranto, L. Zdeborová, Machine learning and the physical sciences, Rev. Mod. Phys. 91 (2019) 045002. doi:10.1103/RevModPhys.91.045002.DOI
- [25]A. Alvez, C. Furtlehner, F. Landes, Learning and extrapolating scale-invariant processes, Phys. Rev. E 114 (2026) 025301. doi:10.1103/nntc-spjp.DOI
- [26]T. Fawcett, An introduction to roc analysis, Pattern Recognit. Lett. 27 (2006) 861–874. doi:10.1016/j.patrec.2005.10.010.DOI
- [27]J. A. Hanley, B. J. McNeil, The meaning and use of the area under a receiver operating characteristic (ROC) curve, Radiology 143 (1982) 29–36. doi:10.1148/radiology.143.1.7063747.DOI
- [28]G. M. Molchan, Earthquake prediction as a decision-making problem, PAGEOPH 149 (1997) 233–247. doi:10.1007/BF00945169.DOI
- [29]V. I. Keilis-Borok, A. A. Soloviev (Eds.), Nonlinear Dynamics of the Lithosphere and Earthquake Prediction, Springer Series in Synergetics, Springer, 2003. doi:10.1007/978-3-662-05298-3.
- [30]S. Torquato, F. H. Stillinger, Local density fluctuations, hyperuniformity, and order metrics, Phys. Rev. E 68 (2003) 041113. doi:10.1103/PhysRevE.68.041113.DOI
- [31]D. Hexner, D. Levine, Hyperuniformity of critical absorbing states, Phys. Rev. Lett. 114 (2015) 110602. doi:10.1103/PhysRevLett.114.110602.DOI
- [32]J. Theiler, S. Eubank, A. Longtin, B. Galdrikian, J. D. Farmer, Testing for nonlinearity in time series: the method of surrogate data, Physica D 58 (1992) 77–94. doi:10.1016/0167-2789(92)90102-S.DOI
- [33]G. Strang, The discrete cosine transform, SIAM Review 41 (1999) 135–147. doi:10.1137/S0036144598336745.DOI
- [34]Y. LeCun, Y. Bengio, G. Hinton, Deep learning, Nature 521 (2015) 436–444. doi:10.1038/nature14539.DOI
- [35]S. Ioffe, C. Szegedy, Batch normalization: Accelerating deep network training by reducing internal covariate shift, in: Proceedings of the 32nd International Conference on Machine Learning, 2015, pp. 448–456. URL https://proceedings.mlr.press/v37/ioffe15.html
- [36]B. Efron, R. J. Tibshirani, An Introduction to the Bootstrap, Chapman & Hall, 1994. doi:10.1201/9780429246593.DOI
- [37]B. Phipson, G. K. Smyth, Permutation p-values should never be zero: Calculating exact p-values when permutations are randomly drawn, Statistical Applications in Genetics and Molecular Biology 9 (2010). doi:10.2202/1544-6115.1585.DOI
- [38]D. Dhar, Self-organized critical state of sandpile automaton models, Phys. Rev. Lett. 64 (1990) 1613–1616. doi:10.1103/PhysRevLett.64.1613.DOI
- [39]D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, in: 3rd International Conference on Learning Representations, 2014. URL https://arxiv.org/abs/1412.6980




