betaFeedback

WorksComputer Science

Illusion of Explainability: A Sanity Check for Spectral Band Attribution in Satellite Time-Series Crop Classification

Abstract

Attribution methods are commonly validated through convergence across methods and consistency with domain knowledge, yet neither criterion establishes whether identified features are necessary for a task. This limitation is particularly relevant in multispectral remote sensing, where overlapping spectral information makes individual bands potentially substitutable. We investigate this gap in crop-type classification using GradientSHAP, Shapley sampling, and Occlusion on a U-TAE segmentation model trained on Sentinel-2 time series from the PASTIS benchmark. All three methods identify vegetation-sensitive spectral regions consistent with domain knowledge, while Shapley and Occlusion exhibit strong ordering sensitivity in deletion and insertion tests. However, these properties do not reliably predict retraining performance. Aggregate attribution consensus masks substantial per-class disagreement, with pairwise mean Spearman correlations ranging from 0.09 to 0.79. Models retrained from scratch using the five lowest-ranked bands perform comparably to or better than those using the five highest-ranked bands. Furthermore, attribution rankings from an independently trained model with identical configuration and seed identify a different top-five subset that yields better retraining performance. Analysis of all 252 five-band subsets reveals consistently low effective dimensionality (1.35 to 2.62), suggesting substantial spectral redundancy. These findings demonstrate that plausible and apparently faithful attribution rankings do not necessarily identify task-relevant bands for improved retrained performance, highlighting the limitations of band-level feature attribution in spectrally redundant input spaces.

Introduction

Crop-type classification is formulated as a semantic segmentation problem with the objective of assigning a crop label to each pixel of a satellite image time series. Many crops are spectrally indistinguishable on any single date and separable only by their phenological trajectories over the season, making multi-temporal, multispectral input essential. At scale, we rely on deep neural networks to achieve this; however, this comes at a cost to interpretability, since these outputs feed subsidy verification, yield forecasting, and decisions about which bands future sensors should carry.

The standard way to recover interpretability is through post hoc attribution. SHAP unifies a family of additive attribution methods under a Shapley-value framework [15]; GradientSHAP approximates these values via expected gradients over a baseline distribution, building on Integrated Gradients [6, 25]; and occlusion-based attribution instead measures the change in output when a feature is masked [27], closely related to perturbation-based faithfulness metrics such as insertion and deletion curves [18, 21]. In remote sensing, these scores are aggregated into per-band rankings and checked against domain knowledge, with the red, red-edge, and near-infrared regions well established as the most discriminative for vegetation and crop mapping [12, 26]. Recovering these bands, especially across several methods, is commonly taken as validation.

Neither check supports that conclusion. Agreement with a domain prior indicates that an attribution is plausible, not faithful, and masking-based faithfulness describes a frozen model’s response to counterfactual inputs, not a band’s information content for the task. Hooker et al. [11] made this concrete with ROAR, showing that in large-scale image classification, many pixel-attribution methods are no better than random ranking when accuracy is measured by retraining on data with the top-ranked features removed rather than by masking a frozen model. Sanity checks in a similar spirit have shown that saliency maps can be insensitive to model and label randomisation [2], and that ROAR’s own retraining protocol leaks information through the masking pattern [19].

Our protocol adapts ROAR to spectral bands rather than pixels, and to multi-temporal multi-class segmentation rather than single-label classification. The gap between plausibility and necessity is especially sharp in multispectral data, where adjacent bands are correlated by construction. Shapley-value attributions may behave unreliably under feature dependence: the coalition semantics become ambiguous [24], off-manifold baselines assign mass to correlated features that carry no independent signal [14, 22], and accurate estimation requires modelling the dependence structure explicitly [1]. A model may therefore attribute heavily to one band while the same discriminative signal remains recoverable from a correlated subset.

Therefore, the ranking describes the solution this model happened to find, and does not by itself establish which bands are uniquely required by the task. To quantify how much independent signal a band subset actually carries, we use the participation ratio, a rotation-invariant summary of a covariance spectrum, which is used in neuroscience to measure the effective dimensionality of correlated population signals [8]. We test this on PASTIS [20] using a U-TAE segmentation model, evaluating GradientSHAP, Shapley sampling, and Occlusion at three levels of evidence: plausibility (agreement with domain knowledge and across methods), faithfulness (the frozen model’s response to masking in ranked order), and selection utility (performance when the model is retrained on the selected bands).

In the retraining experiment, models retrained from scratch on the five lowest-ranked bands perform as well as or better than those trained on the five highest-ranked bands, whereas the ranking of a second, equally accurate training run with identical configuration and seed selects a top five that retrains better (Figure 1). Thus, the rankings pass the first two evaluation criteria while failing to distinguish the subsets under retraining, and the redundancy analysis provides a plausible explanation. Our contribution is diagnostic: we propose no new attribution method, and we do not claim the aggregate vegetation-sensitive band finding is wrong. We show instead that the criteria used to validate an attribution can fail to distinguish an explanation that identifies uniquely important features from one that selects an arbitrary member of a redundant set. In correlated sensor settings, this distinction should therefore be tested rather than assumed.

Figure 1. Two runs of one training configuration (same code and seed, equal accuracy) give Shapley rankings that differ by a single band swap (outlined). Both show order-sensitive frozen-model behaviour under the evaluated checks, yet retraining on the top-5 is significantly worse than on the bottom-5 for Run A and significantly better for Run B (mean and 95% CI over five seeds). Orange: top-5; blue: bottom-5.

Related effects are known elsewhere: equally accurate models can rely on different features [4, 5, 7], explanation methods often disagree [13], and top-kk truncation of an importance ranking ignores redundancy [9, 17]. We show how they combine in spectral band selection, where retraining exposes them.

Experimental setup

Dataset. We use PASTIS [20], a panoptic agricultural satellite image time-series benchmark comprising 2,433 patches of 128×128128 \times128 pixels across four regions of France, each a Sentinel-2 optical time series with 10 spectral bands (B02, B03, B04, B05, B06, B07, B08, B8A, B11, B12; listed in the supplementary material) and between 38 and 61 acquisitions. The dataset defines 18 crop classes plus a background class, which we ignore (we call this the 18-class setting), with a heavy long-tailed class distribution—background and meadow alone cover roughly 68% of labeled pixels. Unless stated otherwise we use a fixed split of PASTIS’s official folds: folds 1–3 for training (1,455 patches), fold 4 for validation (482) and fold 5 as the held-out test set (496). For the fold-rotation experiment, fold ff is the test set, fold f−1f-1 (cyclically) validation and the remaining three folds training, so that f=5f=5 recovers the fixed split. Cloud filtering is performed using a Sentinel-2 Scene Classification Layer (SCL)-based protocol; full details are given in the supplementary material.

Model. We use U-TAE [20] for crop classification in the PASTIS dataset. It is an architecture consisting of a U-Net encoder with a temporal self-attention bottleneck for satellite image time series. For the highly imbalanced classes we used focal loss instead of cross-entropy, which down-weights well-classified pixels, AdamW instead of Adam, a linear warmup plus cosine learning-rate schedule instead of a flat rate, and gradient-norm clipping at 0.5. The attribution model is trained on all ten bands for 91 epochs; every retrained model in § 3 is trained for 50 epochs with the same schedule, so arms are compared at a matched training budget. We train the attribution model twice, as Run A and Run B, with identical code, configuration, random seed (42) and software environment. cuDNN is set to deterministic mode, but other GPU operations are not deterministic, so the two runs diverge during training and reach different solutions of equal accuracy (test mIoU 0.9131 and 0.9137). They are two equally accurate solutions under nominally identical settings, used as a case study rather than as seed replicates; independently seeded models are analysed separately. GradientSHAP was computed for Run B only, so attributions, cross-method agreement and faithfulness use Run B; Run A’s Shapley and Occlusion rankings are also evaluated for faithfulness, and Run A’s Shapley ranking defines the main retraining arms. Comparing the two runs measures ranking stability and retraining utility. Background, and in the reduced label space every non-selected crop class, is mapped to an ignore label that is excluded from the loss and from all metrics, so metrics are computed over crop classes only.

Attribution methods. We transform the model’s class-specific spatio-temporal predictions into a spectral importance vector containing one score for each of the ten input bands for each of the seven classes being predicted. We compute this band-level importance with three methods, each producing one score per class and band on the held-out test set: GradientSHAP sums attributions over timesteps, while Occlusion and Shapley mask each band across all timesteps.

  • GradientSHAP. We implement GradientSHAP following the expected-gradients formulation of [6], using baselines drawn from a background data distribution rather than a single fixed reference. References are real per-pixel time series from the training folds, and absolute attributions, averaged over 50 Monte Carlo samples per patch and class, are summed over timesteps (settings in the supplementary material).

  • Shapley. Shapley evaluates the importance of bands by calculating their average marginal contribution across different subsets (referred to as coalitions) of all the available bands. For each input band, the contribution is determined by the change in model output when the band is added to a subset compared to the same subset without the said band. We estimate these average marginal contributions with the Monte-Carlo permutation estimator [23] using antithetic pairs [16]: 30 band orderings per patch (15 and their reverses), adding bands one at a time to an all-zero input and recording each band’s change in the class probability, so the values are sampled rather than exact (details in the supplementary material). Bands are ranked by their signed Shapley value averaged over patches and classes.

  • Occlusion. Occlusion is a leave-one-out perturbation: each band in turn is set to zero across all timesteps, and the drop in the class probability over the class’s ground-truth pixels is averaged across patches, giving one score per class and band.

Cross-method agreement. For each class, we compare the resulting band-ranking vectors across attribution methods using Spearman rank correlation.

Faithfulness protocol. We evaluate each ranking with deletion and insertion curves [18, 21]: for deletion, we start from all 10 bands and remove them one at a time in ranked order, plotting class probability at each step (a faithful ranking should show a steep early drop); for insertion, we start from zero bands and add them in ranked order (a faithful ranking should show a steep early rise). We summarize each curve by its area under the curve (AUC), and compare against a random-order control. Bands are masked by replacing their normalised reflectance values with zero at every timestep and every pixel of the patch, so a masked band is absent from the entire time series. Because zero-filling is not a natural observation and may measure the model’s reaction to an off-manifold input rather than band importance [10], we repeat the evaluation with two further baselines: the per-band dataset mean, and a marginal baseline that substitutes the same band from a randomly drawn other patch, preserving its distribution and temporal profile while breaking its dependence on the retained bands.

Retraining and identifiability evaluation. We assess operational band-selection utility by testing whether retraining on the top-ranked subset consistently outperforms lower-ranked and random subsets under the same architecture, dataset and training protocol. Following ROAR [11], we additionally retrain U-TAE from scratch on band subsets selected by rank, using the full temporal sequence but only the selected spectral channels, rather than only masking a frozen model, which avoids the mask-pattern leakage of pixel-level ROAR [19]; every compared top- and bottom-5 holds two 10 m bands, so spatial resolution does not separate them. We use Shapley rankings for this step because Shapley showed the strongest agreement with Occlusion in the cross-method analysis; the main arms use Run A’s ranking (top-5 B04 B07 B11 B08 B12). We retrain the top-, bottom- and random five-band subsets, top- and bottom-kk subsets for k∈{1,2,3}k \in\{1, 2, 3\}, and the full ten bands on the fixed split, with five training seeds per five-band arm and three for the smaller subsets. Because seed and fold variance are different noise sources [3], we also rotate the test fold, and we retrain twenty further random five-band subsets to obtain the distribution of performance across subsets. Rankings are computed on the test fold, which also evaluates fixed-split retraining; this can only favour the top-ranked subsets. Test folds 1–4 of the fold rotation share no patches with the ranking data, and recomputing rankings on the validation fold for Run B and the two seeded models gives near-identical orders (ρ=0.89\rho= 0.89–0.990.99) with Run B’s top- and bottom-5 unchanged, so its retraining comparison involves no test-set reuse. The comparison is run in both label spaces, using the full 18-class model’s own Shapley ranking for the 18-class setting (three seeds per arm), and with the top- and bottom-5 of Run B’s ranking (five seeds per arm).

Statistical analysis. Seed-level comparisons use Welch’s tt-test and an exact permutation test, and fold-level comparisons are paired by fold (paired tt-test and exact sign-flip test). We test equivalence with two one-sided tests (TOST) against a margin of twice the pooled seed SD (0.008 mIoU), the run-to-run noise floor. pp-values are Holm-corrected over the six pairwise seed comparisons; the Run B comparison is a separate follow-up test.

Spectral redundancy analysis. For a kk-band subset, we compute the participation ratio PR=(∑iλi)2/∑iλi2\mathrm{PR} = (\sum_i \lambda_i)^2/\sum_i \lambda_i^2 over the eigenvalues λi\lambda_i of the k×kk \times k Pearson correlation matrix of the selected bands [8], and a variance coverage VC=∑jvjRj2\mathrm{VC} = \sum_j v_j R_j^2, where Rj2R_j^2 measures how well the subset reconstructs the jj-th principal component of all 10 bands and vjv_j is its variance share. We compute both for every subset of every size k=2,…,9k = 2, \ldots, 9, on pixels from cloud-free timesteps. Since both measures are linear, we add two nonlinear ones: a nonlinear VC, in which the principal components are reconstructed from the subset with gradient-boosted trees on held-out pixels, and a task probe, a gradient-boosted pixel classifier trained on each subset’s cloud-free temporal profiles and scored by macro-F1 on the test fold.

Results and discussion

Baseline segmentation performance and class selection

We initially evaluate U-TAE using all 18 crop classes of the PASTIS dataset. The results showed substantially different predictive performance across classes, motivating a reduced-class experiment suite for the subsequent explainability experiments. For diagnostic stability, we selected seven well-classified crop classes of the full-class model, to avoid poorly supported classes in the attribution analysis; on the validation fold they are seven of the eight classes with the highest F1 (the eighth is Beet). The classes shortlisted were: Winter rapeseed, Corn, Soft winter wheat, Meadow, Soy, Grapevine and Winter barley. All remaining classes were relabeled to background and a new model was trained using this modified label space. Because background is excluded from the loss and from all metrics, the reduced task is seven-way discrimination among crop pixels. Because selecting well-classified classes favours well-separated ones, we treat the seven-class and full 18-class tasks as two settings: the seven-class setting is used for the attribution diagnostics, and the retraining comparison is run in both.

The 7-class model’s performance is reported in Table 1; all subsequent results use the 7-class setting unless stated otherwise, and per-class results for both settings are in the supplementary material.

ModelmIoUMacro F1Weighted F1
Garnot & Landrieu (5-fold)‡.631––
Ours (18-cl., 5-fold).680 ± .010–†.897 ± .004
Ours (18-cl., fixed split).674 ± .012.789 ± .009.890 ± .006
Ours (7-class).913.954.967

Table 1. Performance of our U-TAE reproduction on PASTIS. The 18-class setting uses all 18 crop classes and all 10 spectral bands. The 7-class setting uses the selected seven crop classes with all remaining classes relabelled as background. Metrics are computed over crop classes. †Not reported; see the fixed-split row. ‡Includes background in mIoU, so not directly comparable.

Aggregate attribution and domain knowledge

At the aggregate level, all three methods assign substantial importance to the Near-Infrared (NIR, B08), Red Edge (B05, B06, B07), and Short-Wave Infrared (SWIR, B11) spectral regions. The top-ranked bands for each attribution method are summarized in Table 2. This alignment with established remote sensing literature looks, in isolation, like validated, method-agnostic evidence for band importance. However, this aggregate view can obscure substantial, setting-dependent disagreement at the per-class level.

Method1st2nd3rd4th5th
GradientSHAPB06B04B05B11B07
OcclusionB07B11B05B08B06
Shapley samplingB07B11B08B04B05

Table 2. Top-5 bands per attribution method for the 7-class model (Run B), ranked by descending importance.

Per-class attribution disagreement

At the per-class level, however, the bands identified as important vary substantially across crop classes and between methods (Figure 2), so aggregation masks disagreement. Similar method-dependent disagreement persists in the full 18-class setting, indicating that the phenomenon is not specific to the reduced label space (heatmaps in the supplementary material). The global ranking is itself sensitive to how per-class scores are combined: averaging signed Shapley values over classes gives B07 as the top band, averaging their magnitudes gives B08, per-class normalisation gives B04, and pooling over all patches gives B08.

Figure 2. Per-class band importance for the 7-class model from GradientSHAP (left), Occlusion (centre), and Shapley sampling values (right), each row normalised to its largest magnitude; all three are computed on Run B. The methods exhibit different class-specific importance patterns despite their aggregate agreement. A star marks the top band for each crop class.

Cross-method agreement. To quantify this disagreement, we computed the pairwise Spearman rank correlation (ρ\rho) on the spectral importance feature vectors across classes. Rank correlation compares orderings without requiring comparable magnitudes. The resulting mean rank correlations and top-band agreement rates are reported in Table 3; per-class values are in the supplementary material.

InputMethod pairMean ρ\rhoTop match
RawGradientSHAP vs. Occlusion0.161/7
RawShapley vs. Occlusion0.784/7
RawGradientSHAP vs. Shapley0.091/7
PCA-10GradientSHAP vs. Occlusion0.614/7
PCA-10Shapley vs. Occlusion0.796/7
PCA-10GradientSHAP vs. Shapley0.573/7

Table 3. Mean per-class Spearman rank correlation between attribution-method band rankings, and fraction of classes where compared methods agree on the single top band, for the 7-class setting. All rankings from Run B (for PCA-10, Run B of the PCA model).

Agreement is neither uniformly high nor method-invariant: mean ρ\rho ranges from 0.09 to 0.79, and Shapley agrees with Occlusion far more than GradientSHAP does (0.78 vs. 0.16 on raw bands). Agreement is also setting-dependent: decorrelating the bands with PCA raises GradientSHAP’s agreement with occlusion from 0.16 to 0.61 and with Shapley from 0.09 to 0.57, while Shapley–occlusion agreement is unchanged (0.78 to 0.79). Band correlation, rather than the methods themselves, accounts for most of the disagreement. Aggregate, class-averaged agreement statistics should therefore be reported with this instability made explicit.

Faithfulness under perturbation. The resulting AUC values are reported in Table 4, while the corresponding insertion and deletion curves are shown in Figure 3. Shapley and Occlusion outperform the random ordering on both deletion and insertion, while GradientSHAP does not outperform its random control under zero-filling. Run A’s Shapley and Occlusion rankings, which define the main retraining arms, pass the same checks (Table 4). These results show that the rankings can be faithful and order-sensitive on a frozen model without establishing that the identified bands are uniquely or reliably necessary after retraining.

RankingZero DelZero InsMean DelMean InsMarginal DelMarginal Ins
Run B
Shapley.074.641.225.472.250.386
Occlusion.079.556.226.458.242.411
GradientSHAP.260.176.299.330.286.324
Random.245.246.343.352.324.326
Run A (ranking used for the main retraining arms)
Shapley.076.635.233.481.253.397
Occlusion.078.566.252.447.256.389
Random.259.259.341.346.326.327

Table 4. Deletion (↓\downarrow) and insertion (↑\uparrow) AUC for the 7-class model under three removal baselines: zero-filling, the per-band mean, and a marginal sample (the band taken from another patch). Each run is compared with its own random control.

Figure 3. Deletion and insertion curves for the 7-class model (Run B), in the ranked order (left) and with the order reversed so the least important band is removed or inserted first (right). The reversed curves invert relative to the standard ordering, confirming order-sensitivity on the frozen model.

Removal baseline. Zero-filling creates spectral vectors the model never saw during training, so the curves may partly measure its reaction to off-manifold input [10]. Shapley and Occlusion themselves use zero-filling, so zero-fill curves partly favour them; the per-band mean and a marginal sample are stricter tests. Shapley and Occlusion still outperform the random ordering under both (Table 4), but Shapley’s deletion-AUC gain over random falls from 0.171 to 0.118 and 0.074, and under the marginal baseline

Occlusion overtakes Shapley. The ordering remains informative under these baselines, although its advantage varies. Reversing the band order (least-important first) makes deletion AUC markedly higher and insertion AUC markedly lower (Figure 3, right), so the faithfulness result depends on the attribution-derived ordering rather than on noise.

Retraining and identifiability. Having evaluated plausibility and faithfulness, we next test selection utility after retraining. Across five random seeds per arm, the bottom-5 and random-5 subsets achieve similar retrained performance, and both outperform the top-5 subset (Table 5, Figure 4a), despite their substantially different Shapley rankings. The top-5 deficit against bottom-5 is significant after Holm correction (Δ=−0.0155\Delta= -0.0155 mIoU, exact permutation p=0.008p = 0.008, Holm-adjusted p=0.048p = 0.048); against random-5 it is borderline (p=0.024p = 0.024, Holm-adjusted 0.0710.071). Bottom-5 and random-5 do not differ significantly (p=0.45p = 0.45), although the experiment does not establish their equivalence either (TOST p=0.053p = 0.053; 90% CI of the difference −0.0027-0.0027 to 0.00810.0081 against a ±0.008\pm0.008 margin). This indicates that, in this setting, the attribution ranking does not identify a uniquely necessary set of five bands, and if anything points away from the best subset. Importantly, redundancy does not make every subset equally informative: a single band loses about 0.11 mIoU, while several distinct five-band subsets retain comparable performance.

Band subsetmIoUMacro F1Weighted F1
Full (all 10all\ 10).9106 ± .0024.9528 ± .0013.9662 ± .0016
Top-5.8893 ± .0044.9407 ± .0026.9591 ± .0018
Bottom-5.9047 ± .0027.9496 ± .0015.9634 ± .0013
Random-5.9021 ± .0056.9481 ± .0031.9626 ± .0020
Top-5*.9014 ± .0023.9477 ± .0013.9627 ± .0010
Bottom-5*.8952 ± .0032.9443 ± .0018.9603 ± .0014
B03 only.8034 ± .0044.8874 ± .0029.9300 ± .0016
PCA (10 comp.10\ comp.).9144.9550.9675

Table 5. Retraining U-TAE on 7-class band subsets. Top-5 and Bottom-5 are the five highest and lowest ranked bands of Run A’s Shapley ranking; B03 is the single lowest-ranked band; Random-5 is an arbitrary five-band subset. Results are mean ± standard deviation across five random seeds (50 epochs). PCA: a single model trained for 91 epochs, as the attribution model (0.9131 on raw bands). *Run A’s values ranked by magnitude, three seeds.

Figure 4. Retraining U-TAE on ranked band subsets (7-class unless noted). (a) Test mIoU over five seeds per arm (dots; hollow for Run B), with mean (tick) and 95% CI (bar), for subsets chosen by Run A’s and Run B’s Shapley rankings. (b) Top-kk minus bottom-kk mIoU with 95% CI; rows use Run A’s ranking except the 18-class row (that model’s own) and the last (Run B’s). (c) Twenty random five-band subsets and the three named subsets at seed 42, ranked; the grey band is ±1\pm1 seed SD around the median.

Distribution over five-band subsets. We also retrained twenty more subsets drawn uniformly at random (Figure 4c). At the matched seed, Run A’s Shapley top-5 is the lowest of all 23 subsets (0.8847; next lowest 0.8903), and its five-seed mean confirms that this is not a single unlucky run. The spread across subsets is only modestly larger than seed-to-seed noise (SD 0.0058 vs. 0.0040), so most five-band subsets perform alike. Run A’s class-averaged Shapley values are all positive, so its bottom-5 holds the weakest, not harmful, bands. Ranking by magnitude instead swaps only B06

for B08, yet reverses the direction: that top-5 retrains 0.006 above its bottom-5 (three seeds, p=0.06p = 0.06; Table 5), so the verdict depends on the aggregation as well as the training run. Subsets with and without B12 differ by less than 0.001 on average (B05: 0.003, p=0.18p = 0.18).

Fold-level variation. Since fold variation is a separate noise source, we also rotated the test fold. Across folds, the all-band model varies with an SD of 0.0055, about twice its seed-to-seed variation. Bottom-5 outperforms top-5 on four of five folds and ties on the fifth (mean gap 0.0088, paired tt-test p=0.057p = 0.057; Figure 4b), including three of the four test folds that share no patches with the ranking data. The direction of the top-5 deficit is thus consistent across test folds.

The 18-class setting. To check that the result is not specific to seven well-separated classes, we repeated the comparison with all 18 classes, using the 18-class model’s own ranking. Every five-band subset now loses 0.026–0.032 mIoU against all ten bands, and top-5 retrains slightly above bottom-5 and random-5 (by 0.005 and 0.006), although neither difference is significant (Welch p=0.064,0.098p = 0.064, 0.098). The seven-class reversal therefore does not carry over, but neither does a clear benefit from following the ranking.

Subset size. Finally, we retrained the top- and bottom-kk bands for k=1,2,3k = 1, 2, 3. At each size the two show no significant difference (all ∣Δ∣≤0.005|\Delta| \le0.005, p>0.4p > 0.4), and random subsets perform alike (Figure 4b). Across the tested subset sizes, folds and label spaces, Run A’s ranking thus shows no consistent advantage for its top-ranked bands.

Stability across equally accurate models. For band selection to be reliable, its practical recommendations should remain stable across comparably accurate models, so we compared the rankings of Run A and Run B. The global orders agree loosely (Shapley ρ=0.92\rho= 0.92, Occlusion ρ=0.77\rho= 0.77), but the parts practitioners act on do not (Figure 5): the top Shapley band moves from B04 to B07, and B12 and B05 exchange places across the top-5 boundary. Two further all-band models trained with different seeds behave the same way: across all four models, only B04, B07 and B08 remain in the Shapley top-5 (supplementary material). A ranking thus partly reflects the solution a training run found and

Figure 5. Global Shapley rank of each band for Run A and Run B, which share configuration and seed and have the same accuracy. Highlighted bands cross the top-5 boundary.

does not, by itself, establish which bands are most useful after retraining.

Ranking of an equally accurate model. Since Run A and Run B rank the bands differently, we also retrained on the top-5 and bottom-5 of Run B’s ranking, which differ from Run A’s only by exchanging B12 and B05. The outcome reverses (Figure 1): Run B’s top-5 retrains significantly better than its bottom-5 (0.9053 ±\pm 0.0036 vs. 0.8982 ±\pm 0.0030 over five seeds; Welch p=0.010p = 0.010), and is higher on every seed. The two top-5 sets share four bands, so the reversal turns on one choice: B05 rather than B12 adds 0.016 mIoU here and 0.007 in the bottom-5 pair, versus average presence effects of 0.003 (B05) and under 0.001 (B12) across random subsets. Bootstrapping test patches, which also absorbs per-patch Monte Carlo noise, gives 95% CIs of B12−B05 that exclude zero with opposite signs in the two runs, suggesting that the swap reflects differences between the learned solutions rather than sampling noise in the attributions. Whether following the ranking helps or hurts thus depends on which of two equally accurate training runs was explained.

Spectral redundancy analysis. If all 5 bands are perfectly correlated, we would get PR = 1, and if completely uncorrelated, PR = 5. Computed on cloud-free timesteps and crop pixels from the training folds, the Shapley-selected top-5 subset has PR ≈2.39\approx2.39 and the bottom-5 subset PR ≈1.81\approx1.81, both far below the five independent signals the bands could carry. Both subsets reconstruct most of the spectral variance of the full 10-band space (VC = 0.99 and 0.93, respectively). Across all (105)=252\binom{10}{5} = 252 five-band subsets, PR ranges from 1.35 to 2.62, and across every subset size it stays well below 5 (Figure 6a). Measured on unfiltered pixels, PR is lower still (1.08–1.45), because clouds make all bands co-vary.

Figure 6. (a) Participation ratio of every kk-band subset (median and range) on cloud-free crop pixels and on unfiltered pixels; the dashed line is the ceiling for fully independent bands. (b) Macro-F1 of a pixel-level gradient-boosted classifier over all subsets of each size, with Run A’s Shapley top-5 and bottom-5 marked. (c) Rank of the 23 retrained five-band subsets under the pixel classifier and after U-TAE retraining.

Reprojecting all 10 bands into their principal components and retraining U-TAE on this space shows the same picture: in the 7-class setting, the PCA-10 model (0.9144) performs on par with the raw-band model trained the same way (0.9131). The leading three principal components explain 98.4% of spectral variance (loadings in the supplementary material), consistent with the ten bands spanning far fewer independent axes, a plausible explanation for why the top-5 versus bottom-5 distinction brings no retraining advantage.

Nonlinear and task-aware redundancy. PR and VC are linear and may miss nonlinear dependence. Reconstructing the principal components with gradient-boosted trees raises five-band coverage from 0.74–0.99 to 0.85–0.99, so the bands are, if anything, more redundant than the linear measures suggest. To measure task information directly, we trained a pixel classifier on each subset’s cloud-free temporal profiles; it saturates after about four bands (Figure 6b) and, unlike U-TAE, it favours the Shapley top-5 (92nd percentile of 252 subsets; bottom-5 at the 19th). Across the 23 retrained subsets, none of these measures predicts U-TAE’s accuracy (all ∣ρ∣≤0.30|\rho| \le0.30, p≥0.16p \ge0.16; Figure 6c). Thus, band necessity depends on the learner: bands that matter to a pixel classifier can be dispensable to a model that also exploits spatial context.

Implications for geospatial XAI. We do not claim that attribution methods are useless: the aggregate vegetation-sensitive band finding is not spurious. Rather, in redundant multispectral settings, masking-based faithfulness and agreement with domain priors are useful but insufficient validation criteria. Retraining on a few band subsets is a cheap additional check, but should use rankings from more than one trained model: as Run A and Run B show, the conclusion from one model’s ranking can reverse for an equally accurate second model. Effective dimensionality is a useful warning rather than a verdict: it signals that many band subsets are likely to suffice, but not which one retrains best, which retraining directly evaluates. In practice, report rankings under two aggregations and two training runs, use a non-zero masking baseline, and retrain a few subsets before selecting bands.

Conclusion. Taken together, the results show that attribution rankings can agree with domain knowledge and provide faithful, order-sensitive explanations of a frozen model while failing to distinguish bands that are uniquely important from redundant substitutes after retraining. With seven classes, Run A’s top-ranked bands retrain worse than its lowest-ranked; elsewhere the ranking shows no consistent advantage, and a second run’s ranking or a magnitude aggregation reverses the result, so a retraining-based verdict depends on the model and on analysis choices. Spectral redundancy helps explain why many distinct five-band subsets perform alike. Attribution rankings should therefore not be used alone to justify spectral-band selection; in applications where the choice of bands matters, it is better to retrain and report ranking stability across runs.

Limitations

We evaluate a single backbone (U-TAE) on a single dataset (PASTIS), so how far the findings depend on this architecture or on the spectral correlation of Sentinel-2 over French farmland is unknown. Retraining uses subsets chosen from the 10-band Shapley ranking, which need not transfer to a 5-band retrain. The seven-class setting favours well-separated classes (the 18-class retraining checks this), and Run A’s ranking comes only from the test fold, which also evaluates its fixed-split retraining. Arms use at most five seeds (three for 18 classes), only faithfulness uses non-zero baselines, and temporal or spatial attribution is not studied.

References

  1. [1]Kjersti Aas, Martin Jullum, and Anders Løland. Explaining individual predictions when features are dependent: More accurate approximations to Shapley values. Artificial Intelligence, 298:103502, 2021. 2arxiv.org/abs/1903.10464
  2. [2]Julius Adebayo, Justin Gilmer, Michael Muely, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. In Advances in Neural Information Processing Systems (NeurIPS), 2018. 2arxiv.org/abs/1810.03292
  3. [3]Xavier Bouthillier, Pierre Delaunay, Mirko Bronzi, Asya Trofimov, Brennan Nichyporuk, Justin Szeto, Naz Sepah, Edward Raff, Kanika Madan, Vikram Voleti, Samira Ebrahimi Kahou, Vincent Michalski, Dmitriy Serdyuk, Tal Arbel, Chris Pal, Gaël Varoquaux, and Pascal Vincent. Accounting for variance in machine learning benchmarks. In Proceedings of Machine Learning and Systems (MLSys), 2021. 3arxiv.org/abs/2103.03098
  4. [4]Alexander D’Amour, Katherine Heller, Dan Moldovan, Ben Adam, Babak Alipanahi, Alex Beutel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew D. Hoffman, et al. Underspecification presents challenges for credibility in modern machine learning. Journal of Machine Learning Research, 23(226):1–61, 2022. 2
  5. [5]Jiayun Dong and Cynthia Rudin. Exploring the cloud of variable importance for the set of all good models. Nature Machine Intelligence, 2(12):810–824, 2020. 2DOI
  6. [6]Gabriel Erion, Joseph D. Janizek, Pascal Sturmfels, Scott M. Lundberg, and Su-In Lee. Improving performance of deep learning models with axiomatic attribution priors and expected gradients. Nature Machine Intelligence, 3:620–631, 2021. 2, 3arxiv.org/abs/1906.10670
  7. [7]Aaron Fisher, Cynthia Rudin, and Francesca Dominici. All models are wrong, but many are useful: Learning a variable’s importance by studying an entire class of prediction models simultaneously. Journal of Machine Learning Research, 20(177):1–81, 2019. 2arxiv.org/abs/1801.01489
  8. [8]Peiran Gao and Surya Ganguli. On simplicity and complexity in the brave new world of large-scale neuroscience. Current Opinion in Neurobiology, 32:148–155, 2015. 2, 4
  9. [9]Isabelle Guyon and André Elisseeff. An introduction to variable and feature selection. Journal of Machine Learning Research, 3:1157–1182, 2003. 2
  10. [10]Giles Hooker, Lucas Mentch, and Siyu Zhou. Unrestricted permutation forces extrapolation: Variable importance requires at least one more model, or there is no free variable importance. Statistics and Computing, 31(6):82, 2021. 3, 5arxiv.org/abs/1905.03151
  11. [11]Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, and Been Kim. A benchmark for interpretability methods in deep neural networks. In Advances in Neural Information Processing Systems (NeurIPS), 2019. 2, 3arxiv.org/abs/1806.10758
  12. [12]Markus Immitzer, Francesco Vuolo, and Clement Atzberger. First experience with Sentinel-2 data for crop and tree species classifications in central Europe. Remote Sensing, 8(3):166, 2016. 2DOI
  13. [13]Satyapriya Krishna, Tessa Han, Alex Gu, Steven Wu, Shahin Jabbari, and Himabindu Lakkaraju. The disagreement problem in explainable machine learning: A practitioner’s perspective. Transactions on Machine Learning Research, 2024. 2arxiv.org/abs/2202.01602
  14. [14]I. Elizabeth Kumar, Suresh Venkatasubramanian, Carlos Scheidegger, and Sorelle A. Friedler. Problems with Shapley-value-based explanations as feature importance measures. In International Conference on Machine Learning (ICML), pages 5491–5500, 2020. 2arxiv.org/abs/2002.11097
  15. [15]Scott M. Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems (NeurIPS), 2017. 1arxiv.org/abs/1705.07874
  16. [16]Rory Mitchell, Joshua Cooper, Eibe Frank, and Geoffrey Holmes. Sampling permutations for Shapley value estimation. Journal of Machine Learning Research, 23(43):1–46, 2022. 3arxiv.org/abs/2104.12199
  17. [17]Hanchuan Peng, Fuhui Long, and Chris Ding. Feature selection based on mutual information: Criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(8):1226–1238, 2005. 2pubmed.ncbi.nlm.nih.gov/16119262
  18. [18]Vitali Petsiuk, Abir Das, and Kate Saenko. RISE: Randomized input sampling for explanation of black-box models. In British Machine Vision Conference (BMVC), 2018. 2, 3arxiv.org/abs/1806.07421
  19. [19]Yao Rong, Tobias Leemann, Vadim Borisov, Gjergji Kasneci, and Enkeljeda Kasneci. A consistent and efficient evaluation strategy for attribution methods. In International Conference on Machine Learning (ICML), pages 18770–18795, 2022. 2, 3arxiv.org/abs/2202.00449
  20. [20]Vivien Sainte Fare Garnot and Loic Landrieu. Panoptic segmentation of satellite image time series with convolutional temporal attention networks. In IEEE/CVF International Conference on Computer Vision (ICCV), 2021. 2arxiv.org/abs/2107.07933
  21. [21]Wojciech Samek, Alexander Binder, Grégoire Montavon, Sebastian Lapuschkin, and Klaus-Robert Müller. Evaluating the visualization of what a deep neural network has learned. IEEE Transactions on Neural Networks and Learning Systems, 28(11):2660–2673, 2017. 2, 3DOI
  22. [22]Carolin Strobl, Anne-Laure Boulesteix, Thomas Kneib, Thomas Augustin, and Achim Zeileis. Conditional variable importance for random forests. BMC Bioinformatics, 9:307, 2008. 2
  23. [23]Erik Strumbelj and Igor Kononenko. An efficient explanation of individual classifications using game theory. Journal of Machine Learning Research, 11:1–18, 2010. 3DOI
  24. [24]Mukund Sundararajan and Naresh Jaini. The many Shapley values for model explanation. In International Conference on Machine Learning (ICML), pages 9269–9278, 2020. 2arxiv.org/abs/1908.08474
  25. [25]Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International Conference on Machine Learning (ICML), 2017. 2arxiv.org/abs/1703.01365
  26. [26]Francesco Vuolo, Martin Neuwirth, Markus Immitzer, Clement Atzberger, and Wai-Tim Ng. How much does multi-temporal Sentinel-2 data improve crop type classification? International Journal of Applied Earth Observation and Geoinformation, 72:122–130, 2018. 2DOI
  27. [27]Matthew D. Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In European Conference on Computer Vision (ECCV), 2014. 2arxiv.org/abs/1311.2901

Paper details

vlnc:W_gxt9y6tj
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents