betaFeedback

Reading v1View latest version

WorksMathematics

Beyond Global Importance: A Heterogeneity Index for Sensitivity Analysis

Abstract

Global sensitivity analysis quantifies how strongly inputs influence a model output, however, input importance often varies across the model space. We introduce a heterogeneity index H to quantify this variation. The index compares normalized regional sensitivity profiles using mean pairwise total-variation distance and its values range from zero to one. We test H on nine benchmark functions and examine its robustness to the number of regions, sample size, and sampling strategy. We further demonstrate it using models of wooden-pallet carbon footprint, flood risk, and steel-structure reliability. Across these cases, H reveals changes in sensitivity that global indices obscure, including inputs whose importance changes substantially across the model space and differences between rare behavioral regimes. The index complements global sensitivity analysis by indicating when a single global sensitivity profile is insufficient. An open-source implementation is provided in the simdec package.

Introduction

Environmental and energy models commonly represent complex systems shaped by nonlinear processes, interacting drivers, and interdependent uncertain conditions [16, 5]. Consider a power-system model combining variable renewable generation, electricity demand, storage, and market or network constraints. A global sensitivity analysis may identify electricity demand as the dominant source of output variation overall, while under particular combinations of highly intermittent generation, high demand, or constrained import, other factors such as storage capacity or demand response flexibility may become substantially more influential. Such heterogeneity need not correspond to an explicit threshold or a regime known in advance; it may emerge only when the model is examined across different parts of its input or output space. This matters for decision-making because recommendations based on global importance can overlook the factors that become decisive under particular operating conditions. The term heterogeneity is used in different ways across disciplines. In this study, we define heterogeneity as variation in the overall composition of input importance across the model space.

Global sensitivity analysis (GSA) provides a rigorous framework for quantifying how uncertainty in model inputs contributes to uncertainty in model outputs and has become an established tool for model diagnosis, uncertainty assessment, and decision support [22]. Most commonly used global sensitivity measures, however, summarize importance over the full distributions of the inputs and outputs. This aggregation can conceal variation in sensitivity within the model space. Wei, Lu, Wu and Zhou [24] noted that standard GSA measures input importance over its full range but does not identify how particular ranges contribute to output uncertainty, while Wei, Lu and Song [23] subsequently developed regional extensions of Sobol’ indices to examine how main and total effects change when input ranges are restricted. The same phenomenon has been demonstrated from a temporal perspective. Time-varying sensitivity analysis has shown that dominant model parameters can change substantially with environmental conditions and that these changes may be difficult to anticipate from the model formulation alone [9]. Related work has therefore developed visualization approaches specifically for exploring changing sensitivity patterns across time and evaluation scales [15]. Collectively, these studies show that sensitivity itself can be heterogeneous: the factors that matter, and their relative importance, may change across the domain of a model.

Several branches of sensitivity analysis provide ways to investigate such variation. Regional sensitivity analysis partitions simulations into behavioral and non-behavioral subsets and examines differences between them [21], while later regional approaches investigate contributions from particular portions of the input domain [24, 23]. Target and conditional sensitivity analysis focus respectively on the factors driving entry into a critical output domain and on sensitivity within that domain [14]. For temporal or spatial outputs, cluster-based GSA identifies distinct model behaviors and the inputs associated with transitions between them [18]. Graphical methods such as simulation decomposition (SimDec) provide another route by visualizing how conditional distributions and interactions change across combinations of model inputs, revealing heterogeneous effects that conventional global sensitivity indices may conceal [12]. These approaches differ substantially in their objectives and formulations, but share the recognition that a single global importance measure may provide an incomplete description of model behavior.

What remains difficult to express is the magnitude of heterogeneity itself. Existing approaches can identify sensitivity within particular regions, trace changing indices across time or input ranges, distinguish critical and non-critical domains, or reveal heterogeneous patterns visually. They do not, however, provide a general scalar measure of how much the complete sensitivity profile changes across a partition of the model space. Consequently, there is no direct counterpart to a global sensitivity index for answering a simple diagnostic question: How heterogeneous is the sensitivity structure of this model? Such a measure could complement conventional GSA by indicating when a global sensitivity profile is sufficiently representative and when, instead, more detailed regional investigation is warranted.

In this paper, we propose the heterogeneity index HH to quantify changes in sensitivity profiles across regions of a model. The general index HZH_Z is defined from pairwise distances between normalized regional sensitivity profiles, where ZZ denotes the variable used to partition the model space. Regions may be defined by the model output, yielding HYH_Y; by an input, yielding HXiH_{X_i}; or by an externally specified categorical partition representing application-specific states or behaviours. We first examine the index’s properties using controlled models in which we can systematically introduce heterogeneity. An existing workbench of test functions for sensitivity analysis [3] is then used to investigate the influence of regionalisation and sample size, and to examine the relationship between global input importance and heterogeneity. Finally, three practical cases test whether HH can compactly identify forms of heterogeneous model behaviour previously revealed only through detailed visualisation or regional sensitivity analysis. The method is implemented in the free and open-source simdec package [19, 20] to facilitate its use alongside conventional sensitivity analysis.

Proposed heterogeneity index

Let Y=f(X)Y = f(\mathbf{X}) be a model with inputs X=(X1,…,XK)\mathbf{X} = (X_1, \ldots, X_K), and let PP denote a generic variable used to define regions of the model space. In a standard analysis, PP is either the output YY, one of the model inputs XjX_j, or an external application-specific partitioning variable ZZ. The proposed heterogeneity index HPH_P measures how much the model’s sensitivity structure changes across regions defined by PP.

Two cases arise naturally in a standard analysis. When P=YP = Y, the resulting index HYH_Y measures changes in sensitivity structure across the output distribution. When P=XjP = X_j, the index HXjH_{X_j} measures changes in sensitivity structure across the range of input XjX_j. An external partitioning variable ZZ extends the same analysis to application-specific conditions or regimes, such as safe/failure states, operating modes, or policy scenarios, yielding HZH_Z.

For a continuous partitioning variable PP, its range is divided into qq equal-frequency regions R1,…,RqR_1, \ldots, R_q. A categorical PP is divided according to its observed states.

Regional sensitivity profiles

Within each region RrR_r, we compute a vector of combined sensitivity indices [11],

Sr=(S1,r,…,SK,r)T.(1)\mathbf{S}_r = (S_{1,r}, \ldots, S_{K,r})^{\mathsf{T}}. \tag*{(1)}

The combined indices allocate estimated second-order interaction effects between the participating inputs according to the Shapley allocation principle.2

Si,rcomb=Si,r(1)+12∑j≠iSij,r(2).(2)S_{i,r}^{\mathrm{comb}} = S_{i,r}^{(1)} + \frac{1}{2}\sum_{j\ne i} S_{ij,r}^{(2)}. \tag*{(2)}

Unlike first-order indices alone, which omit interactions, and total-effect indices, which count interactions repeatedly, the combined indices distribute second-order interaction effects across the participating inputs. Their sum therefore approaches one when the full output variance is explained [8], provided that interactions above second order can be neglected [11].

The set of inputs included in a regional sensitivity profile depends on the partitioning variable. If a categorical input XjX_j is used as PP, it is constant within each region and is therefore omitted from the regional sensitivity calculation. A continuous input used as PP remains in the analysis because it still varies within each region. When an external partitioning variable ZZ is used, it serves only to assign observations to regions and is not introduced among the inputs subjected to sensitivity analysis. For example, ZZ may distinguish behavioral and non-behavioral model realizations in the terminology of Spear and Hornberger [21].

Regional sensitivity analysis requires the output to retain variation within each region, Var⁡(Y∣Rr)>0\operatorname{Var}(Y \mid R_r) > 0. Consequently, HYH_Y is undefined for a categorical output because partitioning the output according to its categories makes YY constant within each region and regional sensitivity indices cannot be estimated. Such a model can nevertheless be analyzed using HXjH_{X_j} or, where relevant, an external partition HZH_Z.

Normalization and heterogeneity

The sum of regional sensitivity indices may differ from the global estimate. This is particularly pronounced when conditioning on the output. Multiple compensating input configurations may generate the same output region, producing regional equifinality and a loss of regional explainability [8].

To make the heterogeneity index respond to changes in the composition of input importance rather than to changes in the total regional sensitivity sum, each regional profile is normalized:

S~i,r=Si,r∑j∈IPSj,r,∑i∈IPS~i,r=1,(3)\widetilde{S}_{i,r} = \frac{S_{i,r}}{\sum_{j\in I_P} S_{j,r}}, \qquad\sum_{i\in I_P} \widetilde{S}_{i,r} = 1, \tag*{(3)}

where IPI_P denotes the set of inputs retained in the regional sensitivity calculation for partitioning variable PP.

Non-normalized regional sensitivity indices remain useful diagnostics and are retained for interpretation and visualization, whereas the normalized profiles are used to calculate HPH_P. The consequences of this distinction, including the removal of output-conditioned equifinality from the heterogeneity measure, are illustrated in § 4.1.

For two regions RrR_r and RsR_s, the difference between their normalized sensitivity profiles is measured by the total variation distance,

Drs=12∑i∈IP∣S~i,r−S~i,s∣.(4)D_{rs} = \frac{1}{2}\sum_{i\in I_P}\left|\widetilde{S}_{i,r}-\widetilde{S}_{i,s}\right|. \tag*{(4)}

Because the normalized profiles are nonnegative and sum to one, Drs∈[0,1]D_{rs}\in[0,1]. The factor 1/21/2 avoids double counting sensitivity that moves from one input to another: a decrease in one component must be accompanied by an increase elsewhere in the profile.

Si(1)=Var⁡[E(Y∣Xi)]Var⁡(Y),S_i^{(1)} = \frac{\operatorname{Var}[\mathbb{E}(Y\mid X_i)]}{\operatorname{Var}(Y)},

while the second-order interaction effect between XiX_i and XjX_j is

Sij(2)=Var⁡[E(Y∣Xi,Xj)]Var⁡(Y)−Si(1)−Sj(1).S_{ij}^{(2)} = \frac{\operatorname{Var}[\mathbb{E}(Y\mid X_i,X_j)]}{\operatorname{Var}(Y)} - S_i^{(1)} - S_j^{(1)}.

For qq regions, the heterogeneity index is the mean distance over all unique pairs of regional sensitivity profiles,

HP=1(q2)∑r<sDrs=1(q2)∑r<s12∑i∈IP∣S~i,r−S~i,s∣.(5)H_P = \frac{1}{\binom{q}{2}} \sum_{r<s} D_{rs} = \frac{1}{\binom{q}{2}} \sum_{r<s} \frac{1}{2} \sum_{i \in\mathcal{I}_P} \left\lvert\widetilde{S}_{i,r} - \widetilde{S}_{i,s} \right\rvert. \tag*{(5)}

Thus HP∈[0,1]H_P \in[0, 1]. A value of zero means that all regions have the same relative sensitivity profile. Larger values indicate progressively stronger redistribution of importance across the regions. Appending an irrelevant input with zero sensitivity in every region does not affect HPH_P, because it contributes zero to every pairwise distance. The complete procedure is summarized in Algorithm 1.

Algorithm 1 Heterogeneity analysis

  1. Obtain an input–output dataset (X,Y)(\mathbf{X}, Y)

  2. Set the partitioning variables to P={Y,X1,…,XK}\mathcal{P} = \{Y, X_1, \ldots, X_K\} for a standard analysis, or P={Z}\mathcal{P} = \{Z\} for an external application-specific partition for each partitioning variable P∈PP \in\mathcal{P} do

  3. Define regions R1,…,RqR_1, \ldots, R_q from PP for each region RrR_r do

  4. Compute the combined sensitivity profile Sr\mathbf{S}_r using Eq. (2)

  5. Normalize the profile using Eq. (3) end for

  6. Compute pairwise profile distances using Eq. (4)

  7. Compute HPH_P using Eq. (5)

  8. Optionally compute Ci,PC_{i,P} using Eq. (6) end for

  9. Return HPH_P for each analyzed partitioning variable P∈PP \in\mathcal{P}

Individual contributions to heterogeneity

Although HPH_P compares complete sensitivity profiles, its value can be decomposed into contributions from individual inputs. For input XiX_i, define

Ci,P=1(q2)∑r<s12∣S~i,r−S~i,s∣.(6)C_{i,P} = \frac{1}{\binom{q}{2}} \sum_{r<s} \frac{1}{2} \left\lvert\widetilde{S}_{i,r} - \widetilde{S}_{i,s} \right\rvert. \tag*{(6)}

Because the order of the finite sums can be interchanged,

HP=∑i∈IPCi,P.(7)H_P = \sum_{i \in\mathcal{I}_P} C_{i,P}. \tag*{(7)}

Thus, Ci,PC_{i,P} is the contribution of input XiX_i to the total profile difference summarized by HPH_P. Individual contributions are optional diagnostics and are useful when a numerical attribution of heterogeneity to particular inputs is desired. Otherwise, the regional sensitivity profiles themselves show which components of the sensitivity structure change across regions.

Simple two-variable example. To build intuition for the meaning of the heterogeneity index, consider a model with two inputs, X1X_1 and X2X_2, and a partitioning variable PP that defines two regions, R1R_1 and R2R_2. Their normalized sensitivity profiles are

S~=R1R2X10.80.2X20.20.8(8)\widetilde{\mathbf{S}} = \begin{array}{c|cc} & R_1 & R_2 \\ X_1 & 0.8 & 0.2 \\ X_2 & 0.2 & 0.8 \end{array} \tag*{(8)}

Because there is only one pair of regions,

HP=12(∣0.8−0.2∣+∣0.2−0.8∣)=0.6.(9)H_P = \frac{1}{2} \left( \lvert0.8 - 0.2 \rvert+ \lvert0.2 - 0.8 \rvert\right) = 0.6. \tag*{(9)}

The corresponding individual contributions are

C1,P=C2,P=0.3,(10)C_{1,P}=C_{2,P}=0.3, \tag*{(10)}

so that

HP=C1,P+C2,P=0.6.H_P=C_{1,P}+C_{2,P}=0.6.

The interpretation is straightforward: 60% of the sensitivity mass must be reallocated to transform the sensitivity profile in one region into that in the other. Adding a third input with zero sensitivity in both regions leaves HP=0.6H_P=0.6, illustrating that the index is invariant to the addition of irrelevant dimensions.

Implementation

The heterogeneity analysis is implemented in the open-source simdec Python package. The package can be installed from PyPI using pip install simdec and imported in Python as follows:

import simdec

For a standard analysis, the user provides the model inputs and output:

H = simdec.heterogeneity_indices(
    output=y,
    inputs=X
)

By default, continuous partitioning variables are divided into five equal-frequency regions. The function calculates heterogeneity with respect to the output and all model inputs simultaneously,

HY, HX1,…,HXK.H_Y,\ H_{X_1},\ldots,H_{X_K}.

The number of regions can be changed using n_regions:

H = simdec.heterogeneity_indices(
    output=y,
    inputs=X,
    n_regions=10
)

An external application-specific partitioning variable ZZ can be supplied directly:

H = simdec.heterogeneity_indices(
    output=y,
    inputs=X,
    custom_partition=Z
)

In this case, only HZH_Z is calculated and the partitioning variable is not introduced among the model inputs. If custom_partition is continuous, it is divided into n_regions equal-frequency regions. If it is categorical, its natural categories are used and n_regions is ignored.

The resulting heterogeneity indices are available through

H.indices

while detailed results for each partition are stored in H.details. For example,

H.details["X1"].raw_profiles
H.details["X1"].normalized_profiles
H.details["X1"].regional_sums
H.details["X1"].region_counts
H.details["X1"].individual_contributions

The raw regional profiles contain the regional combined sensitivity indices, whereas the normalized profiles are those used in the calculation of HH. Regional sensitivity-index sums and observation counts are retained as diagnostics. Individual contributions Ci,PC_{i,P} are also returned for each analyzed partitioning variable PP and sum to the corresponding HPH_P.

Regional sensitivity profiles can be visualized directly from the stored results without recalculating the analysis:

H.plot()                    # all partitions
H.plot("X1")                # one partition
H.plot(["Y", "X1", "X2"])   # selected partitions

The plots display the raw regional combined sensitivity indices, while the heterogeneity index itself is calculated from normalized profiles.

Categorical variables are handled automatically. When a categorical input XiX_i defines the regions for HXiH_{X_i}, that input is excluded from its own regional sensitivity profiles because it is constant within each region. If the model output is categorical, HYH_Y is not computed because conditioning on the output categories makes the output constant within each region. The function issues an explanatory warning and continues to calculate HXiH_{X_i} for the model inputs.

Results: Heterogeneity versus global sensitivity

Having defined the heterogeneity index, we first examine whether it captures information distinct from conventional global sensitivity analysis. We evaluate the global combined sensitivity indices SiS_i, input-conditioned heterogeneity HXiH_{X_i}, and output-conditioned heterogeneity HYH_Y across the benchmark functions summarized in Table 1. With the exception of the Binary model, which was constructed specifically for this study to provide a controlled case of regime-dependent sensitivity, the test functions are drawn from the benchmark collection of Azzini and Rosati [3]. Figure 1 provides the central comparison.

FunctionFormulationInputs and parameters
Controlled binaryY=X1+(1+aX2)X3Y = X_1 + (1 + aX_2)X_3X1,X3∼U(0,1)X_1, X_3 \sim U(0,1); X2∼Bernoulli⁡(0.5)X_2 \sim\operatorname{Bernoulli}(0.5); a=4a = 4
F1f(x)=10x1+0.5x32f(\mathbf{x}) = 10x_1 + 0.5x_3^2x1,x2∼U(−5,5)x_1, x_2 \sim U(-5,5)
F2f(x)=2x1−x22f(\mathbf{x}) = 2x_1 - x_2^2x1,x2∼U(−5,5)x_1, x_2 \sim U(-5,5)
F3f(x)=x12+x24+x1x2+x2x34f(\mathbf{x}) = x_1^2 + x_2^4 + x_1x_2 + x_2x_3^4x1,x2,x3∼U(−4,4)x_1, x_2, x_3 \sim U(-4,4)
F4f(x)=0.2ex1−3+2.2∣x2∣+1.3x62−2x22−0.5x42−0.5x42+2.5x12+0.7x13+3(8x1−2)2+(5x2−3)2+1+sin⁡(5x1)cos⁡(3x23)f(\mathbf{x}) = 0.2e^{x_1-3+2.2\lvert x_2\rvert} + 1.3x_6^2 - 2x_2^2 - 0.5x_4^2 - 0.5x_4^2 + 2.5x_1^2 + 0.7x_1^3 + \frac{3}{(8x_1-2)^2+(5x_2-3)^2+1} + \sin(5x_1)\cos(3x_2^3)x1,x2∼U(−1,1)x_1, x_2 \sim U(-1,1)
Ishigamif(x)=sin⁡x1+7sin⁡2x2+0.1x34sin⁡x1f(\mathbf{x}) = \sin x_1 + 7\sin^2 x_2 + 0.1x_3^4\sin x_1x1,x2,x3∼U(−π,π)x_1, x_2, x_3 \sim U(-\pi,\pi)
Hartmann 6-Df(x)=−∑i=14αiexp⁡[−∑k=16Aik(xk−Pik)2]f(\mathbf{x}) = -\sum_{i=1}^{4}\alpha_i\exp\left[-\sum_{k=1}^{6}A_{ik}(x_k-P_{ik})^2\right]xk∼U(0,1)x_k \sim U(0,1); standard Hartmann 6-D parameters α\boldsymbol{\alpha}, AA, PP
Krigingf(x)=1+e−2[(x1−1)2+x22]−0.5(x32+x42)+e−2[x12+(x2−1)2]−0.5(x32+x42)f(\mathbf{x}) = 1 + e^{-2[(x_1-1)^2+x_2^2]}-0.5(x_3^2+x_4^2) + e^{-2[x_1^2+(x_2-1)^2]}-0.5(x_3^2+x_4^2)x1,…,x4∼U(0,1)x_1,\ldots,x_4 \sim U(0,1)
Owenf(x)=∏j=16[μj+τj12(xj−0.5)]f(\mathbf{x}) = \prod_{j=1}^{6}\left[\mu_j+\tau_j\sqrt{12}(x_j-0.5)\right]xj∼U(0,1)x_j \sim U(0,1); μj=1\mu_j=1; τ=(1,1,0.5,0.5,0.25,0.25)\boldsymbol{\tau}=(1,1,0.5,0.5,0.25,0.25)

Table 1. Test functions used to examine the sensitivity of HH to the number of regions.

Relationship between global sensitivity and heterogeneity across the benchmark functions

Figure 1. Relationship between global sensitivity and heterogeneity across the benchmark functions: (a) Global combined sensitivity index SiS_i versus input-conditioned heterogeneity HXiH_{X_i}; (b) output-conditioned heterogeneity HYH_Y.

Figure 1(a) shows that global sensitivity and input-conditioned heterogeneity describe distinctly different properties of model behavior. Inputs occur throughout the resulting SiS_i–HXiH_{X_i} space: high global sensitivity may coincide with either high or low heterogeneity, while inputs with modest global sensitivity can exhibit substantial changes in the sensitivity structure across their ranges.

The contrast is particularly clear for F1 and F2. In both functions, X1X_1 has substantial global importance but essentially zero input-conditioned heterogeneity. Its contribution to output variance is therefore important, but the sensitivity structure remains stable as X1X_1 changes. Global importance alone does not imply heterogeneous model behavior.

The opposite pattern is visible in the Hartmann 6-D function. Five of its six inputs lie in the region of relatively low global sensitivity but above-median heterogeneity. Individually, these variables account for only modest portions of global output variance, yet conditioning on their values reveals pronounced changes in the model’s sensitivity profile. This low-SiS_i, high-HXiH_{X_i} combination is particularly relevant for sensitivity analysis because such variables may receive little attention in a conventional global ranking despite being associated with substantially different local model behavior.

The Binary model provides the most extreme input-conditioned heterogeneity in the benchmark set. Its binary variable X2X_2 was deliberately constructed to switch the model between different sensitivity structures, and consequently produces the largest observed HXiH_{X_i}. At the other end of the spectrum, the Owen function provides a multi-input example in which input-conditioned heterogeneity remains uniformly low: despite having six inputs with different levels of global importance, none produces a large change in the sensitivity profile across its range.

More generally, there is no common pattern in how heterogeneity is distributed among the inputs of a function. In some models, such as the Binary and Ishigami functions, substantial heterogeneity is concentrated in one particular input. In Hartmann 6-D it is distributed across most inputs, whereas in Owen it remains low throughout. Heterogeneity is therefore not simply a model-wide characteristic inherited equally by its inputs, but can itself have a highly heterogeneous structure.

Taken together, these results establish a distinctive property of the proposed heterogeneity index that sets it apart from conventional sensitivity indices. The global sensitivity index SiS_i measures how much an input contributes to output variation overall. In contrast, HPH_P measures how much the sensitivity structure changes across regions defined by the partitioning variable PP. Consequently, high and low values of SiS_i can coexist with either high or low heterogeneity. Sensitivity and heterogeneity therefore describe different dimensions of model behavior: SiS_i identifies what matters, whereas HH identifies how much what matters changes.

Validation and robustness

We evaluate the proposed index in three steps. First, controlled models are used to verify that HH responds to deliberately introduced heterogeneity and to illustrate the effect of normalizing the regional sensitivity profiles. Second, we test sensitivity to the number of regions. Third, we evaluate sample-size stability and repeatability under simple random and quasi-Monte Carlo sampling.

Raw versus normalized sensitivity indices

To test whether the heterogeneity index behaves as intended, we use two simple controlled models in which heterogeneity is introduced through a single parameter aa. At a=0a = 0, both models reduce to the additive form A+BA + B, with a stable sensitivity structure across the model space. Increasing aa progressively introduces variation in the relative importance of the inputs, providing a controlled setting in which the response of HH can be examined.

All experiments use a scrambled Sobol’ sample of N=217=131,072N = 2^{17} = 131{,}072 observations. The parameter is varied over a∈{0,0.25,0.5,0.75,1,2,4}a \in\{0, 0.25, 0.5, 0.75, 1, 2, 4\}.

The first model contains a binary regime variable,

Y=A+(1+aK)B,A,B∼U(0,1),K∼Bernoulli⁡(0.5).(11)Y = A + (1 + aK)B,\qquad A, B \sim U(0,1),\qquad K \sim\operatorname{Bernoulli}(0.5). \tag*{(11)}

At a=0a = 0, the model reduces to Y=A+BY = A + B and the state of KK does not alter the sensitivity structure. As aa increases, the contribution of BB differs increasingly between the two states of KK. We compute HKH_K directly across the two states of KK, and HYH_Y across five equal-frequency regions of the output.

For illustration, Figure 2 shows heterogeneity calculated from both the raw regional sensitivity profiles and the normalized regional sensitivity profiles used in the proposed index. For input conditioning, the raw and normalized distances match when the raw regional sums are already close to one. For output conditioning, the distinction is more consequential. Even at a=0a = 0, output conditioning produces different raw regional sums because of equifinality ([8]), so the raw profile distance exceeds 0 even though the underlying sensitivity composition is unchanged. Normalization removes this component and makes HYH_Y reflect only changes in relative importance.

Controlled binary model showing heterogeneity indices and regional sensitivity profiles

Figure 2. Controlled binary model: (a) heterogeneity indices for KK and YY, computed using both raw and normalized regional sensitivity indices; (b) YY-conditioned regional raw sensitivity profiles, revealing a decline in regional explainability; and (c) YY-conditioned regional normalized sensitivity profiles, isolating changes in the relative sensitivity structure.

The behavior of HYH_Y is not expected to increase smoothly with the heterogeneity control parameter aa. As aa increases, HKH_K tracks the regime difference deliberately introduced through the binary input KK, and therefore changes monotonically with the strength of that mechanism. By contrast, HYH_Y measures heterogeneity after those input regimes have been mapped into the output space. Its value therefore depends not only on how different the K=0K = 0 and K=1K = 1 regimes are, but also on how their resulting output distributions overlap, separate, and mix within the chosen output regions. Consequently, stronger input-space heterogeneity does not necessarily translate into a proportionally larger HYH_Y. At some values of aa, different input regimes may still populate the same output regions, while at others they become more clearly separated. Output-conditioned heterogeneity therefore reflects the combined effect of regime difference and the geometry of the input-to-output mapping.

The second model replaces the binary regime with a continuous one,

Y=A+(1+aC)B,A,B,C∼U(0,1).(12)Y = A + (1 + aC)B, \qquad A, B, C \sim U(0,1). \tag*{(12)}

The variable CC and the output YY are each divided into five equal-frequency regions. At a=0a = 0, the model again reduces to Y=A+BY = A + B. Increasing aa progressively changes the contribution of BB across the range of CC. Figure 3 repeats the raw-versus-normalized comparison for the continuous model. The experiment verifies that the index can capture progressively changing, rather than only discrete, regimes. It also reinforces the distinction between input-space heterogeneity, HCH_C, and output-space heterogeneity, HYH_Y: the former measures the imposed structural change directly across CC, whereas the latter measures how that change appears after projection into the output space.

Controlled continuous model

Figure 3. Controlled continuous model: (a) heterogeneity indices for CC and YY, computed using both raw and normalized regional sensitivity indices; (b) YY-conditioned regional raw sensitivity profiles, showing changes in both sensitivity composition and regional explainability; and (c) YY-conditioned regional normalized sensitivity profiles, isolating changes in relative sensitivity structure.

Sensitivity to the number of regions

The second experiment examines the dependence of HH on the number of regions used to partition a continuous conditioning variable. We use the controlled binary model in Eq. (11) at a=4a = 4 and eight benchmark models [3]: F1, F2, F3, F4, Ishigami, Hartmann 6-D, Kriging, and Owen (Table 1).

A single scrambled Sobol’ sample of N=217=131,072N = 2^{17} = 131{,}072 observations is generated for each model and reused across all partitions. For continuous conditioning variables, the number of equal-frequency regions is varied from q=2q = 2 to q=15q = 15; the binary variable retains its two natural states. The analysis is repeated for the output and for every model input.

Figure 4 shows that the estimated heterogeneity index can be sensitive to the number of regions, particularly when qq is very small. Moving from q=2q = 2 to q=3q = 3 often produces pronounced jumps, whereas from approximately q=4q = 4 onward the trajectories are more frequently smooth. In many cases, further increasing qq gradually reduces the estimated heterogeneity, suggesting that finer partitioning can dilute broad differences in sensitivity structure across regions.

Sensitivity of the heterogeneity index $H$ to the number of regions across the nine test models

Figure 4. Sensitivity of the heterogeneity index HH to the number of regions across the nine test models. Each panel shows HH obtained by conditioning on the output YY and on each model input.

Several functions nevertheless show persistent non-monotonic behavior. F3 is the clearest example, with a pronounced odd-even pattern in qq. This behavior can be understood from its regional sensitivity profiles: a distinct sensitivity structure near the center of the output space is captured within a single region when qq is odd, but split across two neighboring regions when qq is even, reducing its contribution to the pairwise profile differences and hence to HH. Oscillatory functions, such as Ishigami, likewise show some non-monotonic dependence of HH on qq, reflecting the interaction between the imposed regional partition and repeated local changes in model behavior. Appendix 1 presents the regional sensitivity profiles for all nine functions across several values of qq.

Overall, the experiment suggests that q=5q = 5 is a reasonable starting point for estimating HH: it avoids the coarse-partition effects frequently observed for q=2q = 2 or 3, while retaining sufficiently broad regions for stable regional sensitivity estimation. However, for black-box models, oscillatory responses, or models suspected to contain localized regime switches, heterogeneity should be examined across several values of qq rather than relying on a single partition.

Sample-size stability

We next fix the regional resolution at q=5q = 5 and investigate sensitivity to sample size and sampling strategy. The same nine models are evaluated using both simple random sampling (MC) and independently scrambled Sobol’ quasi-Monte Carlo sampling (QMC). For MC, the sample sizes are N=100N = 100, 1,000, 10,000, and 100,000. QMC uses the nearest powers of two, N=128N = 128, 1,024, 8,192, and 131,072. Each model–method–sample-size combination is repeated ten times with an independently generated sample. Within each repetition, the same dataset is reused for all conditioning variables, and the quantile boundaries are re-estimated from that dataset.

The resulting distributions of HH reveal both the location and repeatability of the estimates as NN increases. Figure 5 shows the Kriging model as a representative example, with the MC and QMC results displayed side by side. Equivalent convergence plots for all nine models are provided in Appendix 2.

Sample-size stability of $H$ for the Kriging function under Monte Carlo (MC) and Sobol’ quasi-Monte Carlo (QMC) sampling

Figure 5. Sample-size stability of HH for the Kriging function under Monte Carlo (MC) (left) and Sobol’ quasi-Monte Carlo (QMC) sampling (right).

Figure 5 illustrates the effect of sample size for the Kriging function. At N=102N = 10^{2}, estimates of HH are clearly unreliable, with substantial dispersion across repetitions and pronounced bias relative to the larger-sample results. With q=5q = 5, this sample size leaves only about 20 observations per region for estimating regional sensitivity indices, including second-order effects, so this instability is unsurprising. At N=103N = 10^{3}, the estimates improve considerably, but noticeable bias and variability remain. From approximately N=104N = 10^{4} onward, the estimates become both stable and mutually consistent across sampling schemes. Small samples do not bias HH in a single direction: for the Kriging model they underestimate heterogeneity for YY, X1X_{1}, and X2X_{2}, while overestimating it for X3X_{3} and X4X_{4}. There is no striking difference between simple random sampling and scrambled Sobol’ QMC, except that the N≈103N \approx10^{3} estimates show a somewhat larger displacement between the two schemes.

To summarize repeatability across all models, Figure 6 reports the sample standard deviation of HH across the ten repetitions. The vertical axis therefore represents estimator variability. Actual sample sizes are used on the horizontal axis, and MC and QMC are shown separately as solid versus dashed line. At the smallest sample sizes, regional estimation can become degenerate when the internal sensitivity-estimation bins approach singleton occupancy.

Sample size and strategy effects on the heterogeneity index $H$ across all nine test functions

Figure 6. Sample size and strategy effects on the heterogeneity index HH across all nine test functions.

Across all nine functions, the dispersion of HH across repetitions generally decreases rapidly with sample size (Figure 6). At the smallest usable samples, standard deviations are often on the order of 0.02–0.07, whereas by N≈104N \approx10^4 they are typically below 0.01 and approach zero at N≈105N \approx10^5. The pattern is broadly consistent across output- and input-conditioned estimates and across MC and QMC sampling, with neither sampling scheme showing a systematic precision advantage. Some functions, notably F3, Ishigami, Hartmann 6-D, and Kriging, retain somewhat greater variability for particular conditioning variables at intermediate sample sizes, reflecting more complex regional sensitivity structures. Overall, the results support N≈104N \approx10^4 as a practical starting point for stable estimation in these benchmark models, while emphasizing that required sample size depends on both model structure and the number of observations retained within each region.

Environmental case studies

The practical contributions of the heterogeneity index are validated using three previously studied models in which heterogeneous behavior has already been identified visually. The first case revisits the carbon-footprint model studied by Kozlova et al. ([12]), where heterogeneous effects were previously established through simulation decomposition. The second considers a flood model examined extensively using regional sensitivity analysis ([8]), allowing us to assess how well the heterogeneity index summarizes known variation in regional sensitivity profiles. The third revisits a steel structure reliability model exhibiting complex nested heterogeneous effects previously revealed through simulation decomposition ([1]).

Carbon footprint of wooden pallets

The first case concerns the life-cycle carbon footprint of wooden transportation pallets. The model was originally introduced for simulation decomposition by Deviatkin et al. ([7]) and subsequently used by Kozlova et al. ([12]) as an example of heterogeneous effects in environmental models. Total life-cycle emissions are calculated as

Etotal=(Eproduction+Erepair+Eend−of−life)FN+EtransportF,E_{\mathrm{total}} = \left(E_{\mathrm{production}} + E_{\mathrm{repair}} + E_{\mathrm{end-of-life}}\right)\frac{F}{N} + E_{\mathrm{transport}}F,

where FF is the number of customer trips and NN is the number of uses of an individual pallet. Eight uncertain inputs describe the number of uses, transportation distance, materials and consumption, truck type, and end-of-life treatment. End of life is represented by two alternatives, incineration and landfilling (Table 2). Further details of the model can be found in Kozlova et al. ((2024)).

InputDistributionParametersUnit
TimberLognormalmean = 21.9, sd = 6.9kg
NailsLognormalmean = 0.35, sd = 0.11kg
ElectricityLognormalmean = 0.69, sd = 0.73kWh
Thermal energyLognormalmean = 4.1, sd = 2.5MJ
Number of usesUniform1 to 83uses
Transportation distanceUniform100 to 200km
Truck typeDiscrete uniformbig / medium / small, each p=1/3p = 1/3–
End of lifeDiscrete uniformincineration / landfilling, each p=1/2p = 1/2–

Table 2. Model inputs and uncertainty assumptions for the wooden-pallet carbon-footprint case.

The previous analysis was based on 1,000 Monte Carlo simulations. Global sensitivity analysis identified end of life and the number of uses as the two dominant inputs with a substantial interaction between them. Simulation decomposition then revealed a particular heterogeneous effect: the direction of the effect of the number of uses depends on the end-of-life treatment, Figure 7. Under landfilling, increasing the number of uses reduces life-cycle emissions, as the production burden is distributed over a greater number of trips. Under incineration, the relationship reverses because the accounting convention assigns negative emissions to burning wood as a substitute for fossil fuels. Thus, the same change in NN can move the model output in opposite directions depending on the end-of-life state.

Stacked histograms and scatter plot illustrating the reversal in the direction of the Number-of-uses effect on marginal CO$_2$ emissions

Figure 7. Illustration of the reversal in the direction of the Number-of-uses effect on marginal CO2_2 emissions depending on the End-of-life scenario: incineration (yellow) or landfilling (blue).

Here, the model is re-simulated with 10510^{5} Monte Carlo runs to obtain more stable estimates of both global sensitivity and heterogeneity, both are reported in Figure 8.

Combined sensitivity and heterogeneity indices for the wooden-pallet carbon-footprint model

Figure 8. Combined sensitivity and heterogeneity indices for the wooden-pallet carbon-footprint model.

Figure 8 reveals a markedly different picture when global importance and heterogeneity are considered jointly. End of life is the globally dominant variable (Si=0.44S_i = 0.44), but its input-conditioned heterogeneity is very small (HXi=0.03H_{X_i} = 0.03). Its regional profile in Figure 9 (right) confirms the lack of variability in sensitivity between the two values. Hence, a reversal in the direction of the effect, as expected, escapes the heterogeneity index, since variance-based sensitivity indices measure the magnitude of an effect rather than its sign. The heterogeneity index should therefore be interpreted as a screening measure for changes in sensitivity structure, rather than as a replacement for inspecting the underlying regional relationships.

Selected raw regional sensitivity profiles for the wooden-pallet carbon-footprint model

Figure 9. Selected raw regional sensitivity profiles for the wooden-pallet carbon-footprint model. Profiles are shown for the output YY and for inputs having either HXi≥0.10H_{X_i} \ge0.10 or Si≥0.30S_i \ge0.30. The values of HH are calculated from normalized regional sensitivity profiles, while raw profiles are displayed here to retain information on regional explainability.

Nevertheless, the heterogeneity manifests elsewhere in the model, which was not detected in the previous analysis. The number of uses combines high global importance (Si=0.30S_i = 0.30) with substantially greater heterogeneity (HXi=0.20H_{X_i} = 0.20). Its regional profile, Figure 9 (middle), shows the increasing effect of timber in higher regions, while the number of uses itself prevails only in the lowest region.

Most strikingly, the output-conditioned heterogeneity is HY=0.54H_Y = 0.54, which indicates substantial changes in the composition of model sensitivity across the output distribution. Across the output space (Figure 9, left), the number of uses dominates sensitivity in the extreme regions of the emissions distribution, while timber becomes influential in the intermediate regions despite its negligible global sensitivity index. End of life becomes the dominant contributor in the central region, precisely where the two end-of-life regimes meet and the effect of the number of uses reverses direction. Thus, although HYH_Y does not measure the sign of an effect directly, its large value captures the substantial redistribution of sensitivity associated with this reversal.

In practical terms, HH therefore provides a compact indication of when the global ranking is likely to be insufficient and when examining the regional sensitivity profiles is worthwhile.

Flood model

The second case revisits the canonical flood model examined extensively in the literature [17, 13, 6, 10, 4], including a recent study on regional variance-based sensitivity analysis Helo et al. [8]. The model describes water depth HWH_W, overflow margin SS, and total cost CC as functions of eight uncertain hydraulic and geometric inputs. In compact form,

α=Zm−ZvL,HW=(QKsBα)3/5,S=Zv+HW−Zb−Hd,(13)\alpha= \frac{Z_m - Z_v}{L}, \qquad H_W = \left(\frac{Q}{K_s B \sqrt{\alpha}}\right)^{3/5}, \qquad S = Z_v + H_W - Z_b - H_d, \tag*{(13)}

where flooding occurs when S>0S > 0. The cost CC combines dyke construction cost and flood-related cost, with the latter switching to a fixed flood cost when S>0S > 0. The full model formulation is given in Helo et al. [8]. The input assumptions used here are summarized in Table 3.

Table 3. Uncertain inputs of the flood model.

SymbolInputProbability distributionUnit
QQFlow rateGumbel(γ=1013\gamma= 1013, β=558\beta= 558), Q>0Q > 0m3^{3}/s
KsK_sStrickler coefficientN(30,7.52)\mathcal{N}(30, 7.5^{2}), Ks>0K_s > 0m1/3^{1/3}/s
ZvZ_vDownstream bed elevationU(49,51)\mathcal{U}(49, 51)m
ZmZ_mUpstream bed elevationU(54,56)\mathcal{U}(54, 56)m
BBRiver widthTriangular(295, 300, 305)m
LLRiver reach lengthTriangular(4990, 5000, 5010)m
ZbZ_bBank elevationTriangular(55, 55.5, 56)m
HdH_dDyke heightU(2,9)\mathcal{U}(2, 9)m

caption:

:::

Standard heterogeneity analysis. The model was first evaluated using N=100,000N = 100{,}000 simulations and the standard heterogeneity analysis with q=5q = 5 equal-frequency regions. For each output, global combined sensitivity indices SiS_i, input-conditioned heterogeneity indices HXiH_{X_i}, and output-conditioned heterogeneity HYH_Y were calculated (Figure 10).

Global combined sensitivity and heterogeneity indices for water depth, overflow margin, and cost

Figure 10. Global combined sensitivity indices SiS_i and heterogeneity indices for the flood model with (a) water depth HWH_W, (b) overflow margin SS, and (c) cost CC as outputs.

Overall, the standard analysis indicates only modest heterogeneity. Output-conditioned heterogeneity is HY=0.19H_Y = 0.19 for water depth, 0.23 for overflow, and 0.20 for cost. Most input-conditioned indices are below 0.1, with the clearest exception being flow rate for water depth (HQ=0.26H_Q = 0.26).

These aggregate results are consistent with, but substantially more compact than, the earlier regional analysis that involved visual examination of regional sensitivity profiles [8]. That analysis showed that water depth is dominated by QQ at low values and by KsK_s at high values; overflow changes from contributions by QQ, ZvZ_v, and HdH_d in the lower tail to dominance by HdH_d in intermediate regions and KsK_s in the upper tail; and cost follows a broadly similar pattern, with QQ becoming particularly important in its upper region. The present HYH_Y values quantify the overall magnitude of these previously observed redistributions. Similarly, the earlier input-conditioned analysis found the sensitivity profiles of HWH_W and SS to be nearly invariant across HdH_d, but identified greater variation for CC. The corresponding heterogeneity indices here, approximately HHd=0.01H_{H_d} = 0.01, 0.01, and 0.08, respectively, provide a concise quantitative confirmation of that result.

The modest values of HYH_Y, however, also expose a limitation of equal-frequency partitioning for rare-event models. Flood events occupy only a very small fraction of the simulation sample and are therefore diluted within a much larger upper quantile. A behaviorally important regime can consequently differ strongly from the remainder of the system while producing only moderate heterogeneity across equal-frequency regions.

Flood versus OK regimes. To isolate this behavior, an application-specific partition was defined as

Zflood={OK,S≤0,Flood,S>0.(14)Z_{\mathrm{flood}} = \begin{cases} \mathrm{OK}, & S \le0, \\ \mathrm{Flood}, & S > 0. \end{cases} \tag*{(14)}

A larger sample of 3×1063 \times10^{6} simulations was generated to provide sufficient observations in the rare Flood regime, which accounted for roughly 900–1,000 observations or about 0.03% of the sample. The same partition ZfloodZ_{\mathrm{flood}} was applied separately to all three outputs. The analysis was repeated 10 times to assess the stability of the resulting heterogeneity estimates under rare-event sampling. Table 4 reports the results of this experiment, and Figure 11 shows the associated regional sensitivity profiles for one of the ten experiments.

OutputMean HfloodH_{\mathrm{flood}}SDMin–max HfloodH_{\mathrm{flood}}Min–max ∑Si\sum S_i in Flood
Water depth HWH_W0.6410.0380.608–0.7400.306–0.968
Overflow margin SS0.6760.0290.635–0.7310.305–0.909
Cost CC0.3670.0330.296–0.4060.481–0.741

Table 4. Heterogeneity between the OK and Flood regimes.

Regional combined sensitivity profiles for water depth, overflow margin, and cost

Figure 11. Regional combined sensitivity profiles for the application-specific partition Zflood={OK,Flood}Z_{\mathrm{flood}} = \{\mathrm{OK}, \mathrm{Flood}\}, shown for (a) water depth HWH_W, (b) overflow margin SS, and (c) cost CC.

The resulting heterogeneity is considerably stronger than under equal-frequency partitioning. The Flood/OK distinction produces very high heterogeneity for water depth and overflow, with approximately two-thirds of the normalized sensitivity profile redistributed between the two regimes. Importantly, HfloodH_{\mathrm{flood}} is high not only for overflow SS, whose threshold defines the partition, but also for water depth HWH_W. The result therefore identifies a distinct behavioral region of the system in which the drivers of another continuous output change substantially.

The profiles in Figure 11 clarify the source of this large heterogeneity. In the OK regime, water-depth variance is overwhelmingly governed by QQ, whereas within the Flood regime the influence shifts strongly toward KsK_s. A similar redistribution occurs for overflow: HdH_d, which is the dominant global source of variance, becomes much less important once flooding has already occurred, while KsK_s becomes the largest contributor to variation within the Flood regime. Thus, a variable may be influential in determining whether the system enters a regime without remaining influential in explaining variation within that regime. The result extends the earlier regional study, which considered the Flood/No-flood partition only for SS. It is also consistent with its quantile analysis, where KsK_s became dominant in the highest overflow region [8].

The raw sensitivity profiles also reveal an estimation issue in the rare Flood region. In the previous regional analysis, N=105N = 10^5 produced only 23 flood observations and a sensitivity-index sum above 8; increasing the sample to 10610^6 produced 316 flood observations and a sum close to one, which was interpreted as evidence of improved regional estimation [8]. The larger repeated experiments conducted here show that closeness of this sum to one is not, by itself, a robust convergence diagnostic for such an extreme region. Across repetitions, the raw sensitivity-index sum within Flood ranged from 0.306 to 0.968 for HWH_W (mean 0.654, SD 0.222), from 0.305 to 0.909 for SS (mean 0.612, SD 0.204), and from 0.481 to 0.741 for CC (mean 0.552, SD 0.082), see the right column of Table 4. These deviations can reflect a combination of sampling variability in the rare and strongly skewed Flood region, dependencies induced by conditioning on the event, and variance associated with effects beyond the second order represented by the current combined indices.

In contrast, HfloodH_{\mathrm{flood}} is substantially more stable across the same repetitions, particularly for HWH_W and SS (Table 4). This provides an empirical illustration of the role of normalization in the proposed index: HZH_Z measures redistribution in the composition of sensitivity rather than changes in the absolute regional sensitivity sum. The cost result should nevertheless be interpreted cautiously. Within the Flood regime, CC has very little remaining variance because the flood-related cost is fixed once S>0S > 0.

Overall, the case adds two insights to the previous regional analysis. First, the standard HYH_Y and HXiH_{X_i} indices reduce a large collection of regional profiles to interpretable measures of the magnitude of heterogeneity while reproducing the qualitative conclusions of the earlier study. Second, the application-specific HfloodH_{\mathrm{flood}} exposes strong behavioral heterogeneity that is largely diluted by equal-frequency regions. In particular, the transition from the overwhelmingly QQ-driven OK regime to a KsK_s-dominated Flood regime is summarized by a single, high, and comparatively stable heterogeneity measure.

Steel structure reliability model

The third case revisits a structural reliability model for fatigue assessment of welded steel components that had previously been analyzed using global sensitivity analysis and Simulation Decomposition [11, 1]. The model is based on the 4R method, which combines material strength, welding residual stress, applied stress ratio, and weld quality to estimate the effective stress used in fatigue assessment [2]. In simplified form, the output is the mean-stress-corrected reference effective notch stress range,

Δσk,ref=Δσk1−Rlocal,(15)\Delta\sigma_{k,\mathrm{ref}} = \frac{\Delta\sigma_k}{\sqrt{1-R_{\mathrm{local}}}}, \tag*{(15)}

where the local stress state depends nonlinearly on the four model inputs. Thus, the model can be written schematically as

Δσk,ref=f(σres,R,Rp0.2,Kf).(16)\Delta\sigma_{k,\mathrm{ref}} = f\left(\sigma_{\mathrm{res}}, R, R_{p0.2}, K_f\right). \tag*{(16)}

which also illustrates the input-output set-up for sensitivity analysis: residual stress σres\sigma_{\mathrm{res}}, applied stress ratio RR, steel grade Rp0.2R_{p0.2}, and the fatigue-effective stress factor KfK_f are used as inputs. The original analysis used 10410^4 simulations and identified substantial nonlinear and heterogeneous relationships among the three most influential inputs [11, 1].

Figure 12 shows that global importance and heterogeneity convey distinctly different information. Residual stress is both the globally dominant input (Si=0.52S_i = 0.52) and strongly heterogeneous (Hσres=0.41H_{\sigma_{\mathrm{res}}} = 0.41), indicating that its importance is accompanied by substantial changes in the surrounding sensitivity structure across its range. Stress ratio shows a similar, although somewhat weaker, combination of global importance and heterogeneity (Si=0.35S_i = 0.35, HR=0.31H_R = 0.31).

Global combined sensitivity and heterogeneity indices for the steel structure reliability model

Figure 12. Global combined sensitivity indices SiS_i and heterogeneity indices for the steel structure reliability model. Input-conditioned heterogeneity HXiH_{X_i} is shown alongside the corresponding global sensitivity index, while HYH_Y summarizes changes in sensitivity structure across the output distribution.

A different pattern is observed for steel grade. Its global sensitivity is relatively modest (Si=0.10S_i = 0.10), yet its heterogeneity is considerably larger (HRp0.2=0.24H_{R_{p0.2}} = 0.24). Thus, a ranking based only on global sensitivity would suggest a secondary role, whereas the heterogeneity index indicates that different values of steel grade are associated with substantially different sensitivity structures. This result is consistent with the earlier SimDec analysis, which showed that the effects of stress ratio and residual stress depend strongly on material-strength conditions [11, 1]. The output-conditioned heterogeneity is also substantial, HY=0.38H_Y = 0.38, motivating inspection of the regional sensitivity profiles.

Figure 13 reveals a third type of information that is not apparent from either global SiS_i or input-conditioned HXiH_{X_i}. The fatigue-effective stress factor KfK_f has both negligible global sensitivity (Si=0.04S_i = 0.04) and low input-conditioned heterogeneity (HKf=0.04H_{K_f} = 0.04). Nevertheless, its contribution changes markedly across the output distribution and it becomes the largest sensitivity contributor in the highest output region. Hence, an input that appears unimportant both globally and across its own range can still become locally important in a particular region of the model response.

Selected regional sensitivity profiles for the steel structure reliability model

Figure 13. Selected regional sensitivity profiles for the steel structure reliability model. Raw regional combined sensitivity indices are shown for the output and for inputs with HXi≥0.10H_{X_i} \ge0.10 or Si≥0.30S_i \ge0.30. Heterogeneity indices are calculated from the corresponding normalized regional sensitivity profiles.

The case therefore illustrates three complementary situations: (i) residual stress is both globally important and heterogeneous; (ii) steel grade has modest global importance but comparatively high heterogeneity; and (iii) KfK_f has low global and input-conditioned importance but becomes influential in an extreme output region. Together, these patterns demonstrate why SiS_i, HXiH_{X_i}, and HYH_Y each provide complementary views of model behavior.

Conclusions

This paper introduced the heterogeneity index, HH, to quantify changes in the sensitivity profile across regions of a model. The index is based on the mean pairwise total-variation distance between normalized regional sensitivity profiles and is bounded between zero and one. Its general formulation, HPH_P, allows the regions to be defined by the output, yielding HYH_Y; by an input, yielding HXiH_{X_i}; or by an application-specific partition variable, ZZ, representing meaningful behavioral or operational regimes.

Tests on controlled and benchmark models confirmed that the index behaves as intended: it approaches zero when regional sensitivity profiles remain stable, increases as deliberately introduced heterogeneity becomes stronger, and captures changes in sensitivity composition that are not apparent from global sensitivity indices alone. The experiments also showed that the number of regions and sample size affect estimation accuracy, but that the index becomes stable with sufficiently populated regions. Normalizing regional sensitivity profiles proved important for separating changes in the composition of sensitivity from changes in the absolute amount of regionally explained variance.

The three practical cases illustrate complementary ways in which this additional information matters. First, substantial output-conditioned heterogeneity can signal that a single global sensitivity profile conceals important changes in model behavior, including changes associated with reversals in the direction of effects. Second, an input with relatively modest global importance may nevertheless be associated with strong changes in the overall sensitivity structure. Third, application-specific partitions can reveal pronounced differences between behavioral regimes that are diluted by default equal-frequency regionalization, if the regime of interest is rare. Together, these cases demonstrate that global importance and heterogeneity answer different questions: SiS_i indicates how much an input matters overall, whereas HH indicates how much what matters changes.

Our findings indicate that heterogeneity analysis should become a standard component of computational model assessment in conjunction with conventional sensitivity analysis. Its use is particularly important in high-stakes applications such as public policy, medicine, defence, aerospace, energy, and critical infrastructure, where decisions based on a single global sensitivity profile may overlook regime-dependent changes in what is driving the model outcomes. The proposed approach possesses an important limitation. The current implementation constructs regional sensitivity profiles from combined first- and second-order effects. Consequently, variance associated with third- and higher-order interactions is not represented. Future research can extend the framework toward higher-order effects while controlling the additional computational cost. Further work may also examine adaptive regionalization and uncertainty measures for HH in sparsely populated or rare-event regimes.

The heterogeneity-index function is implemented in the free and open-source simdec package [20] to facilitate straightforward application alongside existing sensitivity-analysis workflows and support its wider dissemination.

Software and data availability

The heterogeneity index is implemented in the open-source Python package simdec, version 1.5.2. The source code is publicly available at https://github.com/SimulatIon-Decomposition/simdec-python, and the package can be installed from PyPI using pip install simdec.

All computational experiments, case-study analyses, and figures reported in this paper can be reproduced using the Jupyter notebooks available at https://github.com/gnopik/Heterogeneity-index.

Acknowledgements

The authors gratefully acknowledge financial support from the Natural Sciences and Engineering Research Council of Canada (NSERC), Grant No. OGP0155871.

References

Regional sensitivity profiles across different numbers of regions

Regional sensitivity profiles for the controlled binary model

Figure 14. Regional sensitivity profiles for the controlled binary model.

Regional sensitivity profiles for F1 function

Figure 15. Regional sensitivity profiles for F1 function.

Regional sensitivity profiles for F2 function

Figure 16. Regional sensitivity profiles for F2 function.

Regional sensitivity profiles for F3 function

Figure 17. Regional sensitivity profiles for F3 function.

Regional sensitivity profiles for F4 function

Figure 18. Regional sensitivity profiles for F4 function.

Regional sensitivity profiles for the Ishigami function

Figure 19. Regional sensitivity profiles for the Ishigami function.

Regional sensitivity profiles for the Hartmann 6-D function

Figure 20. Regional sensitivity profiles for the Hartmann 6-D function.

Regional sensitivity profiles for the Kriging function

Figure 21. Regional sensitivity profiles for the Kriging function.

Regional sensitivity profiles for the Owen function

Figure 22. Regional sensitivity profiles for the Owen function.

Sample-size stability of the heterogeneity index

Sample-size stability of the heterogeneity index for the controlled binary model

Figure 23. Sample-size stability of the heterogeneity index HH for the controlled binary model under simple random sampling and scrambled Sobol’ QMC sampling. The dotted reference line for X2=KX_2 = K indicates the analytical value HK=6/13≈0.462H_K = 6/13 \approx0.462.

Sample-size stability of the heterogeneity index for the F1 function

Figure 24. Sample-size stability of the heterogeneity index HH for F1 function under simple random sampling and scrambled Sobol’ QMC sampling.

Sample-size stability of the heterogeneity index for the F2 function

Figure 25.

Sample-size stability of the heterogeneity index $H$ for F2 function under simple random sampling and scrambled Sobol’ QMC sampling.

Figure 25 (PDF p. 28). Sample-size stability of the heterogeneity index HH for F2 function under simple random sampling and scrambled Sobol’ QMC sampling.

Sample-size stability of the heterogeneity index $H$ for F3 function under simple random sampling and scrambled Sobol’ QMC sampling.

Figure 26. Sample-size stability of the heterogeneity index HH for F3 function under simple random sampling and scrambled Sobol’ QMC sampling.

Sample-size stability of the heterogeneity index $H$ for F4 function under simple random sampling and scrambled Sobol’ QMC sampling.

Figure 27. Sample-size stability of the heterogeneity index HH for F4 function under simple random sampling and scrambled Sobol’ QMC sampling.

Sample-size stability of the heterogeneity index $H$ for the Ishigami function under simple random sampling and scrambled Sobol’ QMC sampling.

Figure 28. Sample-size stability of the heterogeneity index HH for the Ishigami function under simple random sampling and scrambled Sobol’ QMC sampling.

Sample-size stability of the heterogeneity index $H$ for the Hartmann 6-D function under simple random sampling and scrambled Sobol’ QMC sampling.

Figure 29. Sample-size stability of the heterogeneity index HH for the Hartmann 6-D function under simple random sampling and scrambled Sobol’ QMC sampling.

Sample-size stability of the heterogeneity index $H$ for the Kriging function under simple random sampling and scrambled Sobol’ QMC sampling.

Figure 30. Sample-size stability of the heterogeneity index HH for the Kriging function under simple random sampling and scrambled Sobol’ QMC sampling.

Image not extracted

Figure 31. Sample-size stability of the heterogeneity index HH for the Owen function under simple random sampling and scrambled Sobol’ QMC sampling.

References

  1. [1]Ahola, A., Kozlova, M., Yeomans, J.S., 2024. Capturing multi-dimensional nonlinear behaviour of a steel structure reliability model–global sensitivity analysis; in: Kozlova, M., Yeomans, J.S. (Eds.), Sensitivity Analysis for Business, Technology, and Policymaking Made Easy with Simulation Decomposition. Routledge. chapter 11, pp. 257–282. doi:10.4324/9781003453789-15.DOI
  2. [2]Ahola, A., Muikku, A., Braun, M., Björk, T., 2021. Fatigue strength assessment of ground fillet-welded joints using 4r method. International journal of fatigue 142, 105916.DOI
  3. [3]Azzini, I., Rosati, R., 2022. A function dataset for benchmarking in sensitivity analysis. Data in Brief 42, 108071. doi:10.1016/j.dib.2022.108071.DOI
  4. [4]Baudin, M., Dutfoy, A., Iooss, B., Popelin, A.L., 2016. OpenTURNS: An Industrial Software for Uncertainty Quantification in Simulation. Springer International Publishing, Cham. pp. 1–38. URL: https://doi.org/10.1007/978-3-319-11259-6_64-1, doi:10.1007/978-3-319-11259-6_64-1.
  5. [5]Beven, K., Freer, J., 2001. Equifinality, data assimilation, and uncertainty estimation in mechanistic modelling of complex environmental systems using the glue methodology. Journal of hydrology 249, 11–29.DOI
  6. [6]De Rocquigny, E., 2012. Modelling under risk and uncertainty: an introduction to statistical, phenomenological and computational methods. John Wiley & Sons.
  7. [7]Deviatkin, I., Kozlova, M., Yeomans, J.S., 2021. Simulation decomposition for environmental sustainability: Enhanced decision-making in carbon footprint analysis. Socio-Economic Planning Sciences 75, 100837.DOI
  8. [8]Helo, J., Kozlova, M., Roy, P., Yeomans, J.S., 2026. Regional variance-based sensitivity analysis and a study of regional equifinality. Risk Analysis 46, e70346.DOI
  9. [9]Herman, J., Reed, P., Wagener, T., 2013. Time-varying sensitivity analysis clarifies the effects of watershed model formulation on model behavior. Water Resources Research 49, 1400–1414.DOI
  10. [10]Iooss, B., Lemaître, P., 2015. A review on global sensitivity analysis methods, in: Delling, G., Meloni, C. (Eds.), Uncertainty Management in Simulation-Optimization of Complex Systems. Springer, Boston, MA, pp. 101–122. doi:10.1007/978-1-4899-7547-8_5.DOI
  11. [11]Kozlova, M., Ahola, A., Roy, P., Yeomans, J.S., 2025. Simple binning algorithm and simdec visualization for comprehensive sensitivity analysis of complex computational models. Journal of Environmental Informatics Letters 13, 38–56.
  12. [12]Kozlova, M., Moss, R.J., Yeomans, J.S., Caers, J., 2024. Uncovering heterogeneous effects in computational models for sustainable decision-making. Environmental Modelling & Software 171, 105898. doi:10.1016/j.envsoft.2023.105898.DOI
  13. [13]Limbourg, P., De Rocquigny, E., 2010. Uncertainty analysis using evidence theory—controlling level-1 and level-2 approaches with data availability and computational constraints. Reliability Engineering & System Safety 95, 550–564.
  14. [14]Marrel, A., Chabridon, V., 2021. Statistical developments for target and conditional sensitivity analysis: Application on safety studies for nuclear reactor. Reliability Engineering & System Safety 214, 107711.
  15. [15]Massmann, C., Wagener, T., Holzmann, H., 2014. A new approach to visualizing time-varying sensitivity indices for environmental model diagnostics across evaluation time-scales. Environmental Modelling & Software 51, 190–194.
  16. [16]Pianosi, F., Beven, K., Freer, J., Hall, J.W., Rougier, J., Stephenson, D.B., Wagener, T., 2016. Sensitivity analysis of environmental models: A systematic review with practical workflow. Environmental Modelling & Software 79, 214–232.
  17. [17]de Rocquigny, É., 2006. La maîtrise des incertitudes dans un contexte industriel. Ire partie: une approche méthodologique globale basée sur des exemples. Journal de la Société française de statistique 147, 33–71.
  18. [18]Roux, S., Buis, S., Lafolie, F., Lamboni, M., 2021. Cluster-based gsa: Global sensitivity analysis of models with temporal or spatial outputs using clustering. Environmental Modelling & Software 140, 105046.
  19. [19]Roy, P.T., Kozlova, M., 2024. Simulation decomposition in python. Journal of Open Source Software 9, 6713.DOI
  20. [20]Simulation-Decomposition, 2026. Simulation-decomposition: Sensitivity analysis using simulation decomposition. https://github.com/Simulation-Decomposition. Accessed: 2026-04-19.
  21. [21]Spear, R.C., Hornberger, G.M., 1980. Eutrophication in peel inlet—ii. identification of critical uncertainties via generalized sensitivity analysis. Water research 14, 43–49.
  22. [22]Tarantola, S., Ferretti, F., Lo Piano, S., Kozlova, M., Lachi, A., Rosati, R., Puy, A., Roy, P., Vannucci, G., Kuc-Czarn ecka, M., Saltelli, A., 2024. An annotated timeline of sensitivity analysis. Environmental Modelling & Software 174, 105977.DOI
  23. [23]Wei, P., Lu, Z., Song, J., 2015. Regional and parametric sensitivity analysis of sobol’ indices. Reliability Engineering & System Safety 137, 87–100.DOI
  24. [24]Wei, P., Lu, Z., Wu, D., Zhou, C., 2013. Moment-independent regional sensitivity analysis: application to an environmental model. Environmental modelling & software 47, 55–63.DOI

Paper details

vlnc:W_dvnpf3ms
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…
Contents