betaFeedback

WorksMathematics

Wasserstein Target Diagram for Model Evaluation

Abstract

The target diagram is often used in geosciences, which provides a visualization tool for model evaluation and shows the temporal or spatial matching of data. However, its construction requires paired data of equal sample sizes, limiting its application to model comparisons involving unpaired samples or probability distributions. In this study, we propose the Wasserstein target diagram based on the 2-Wasserstein distance from optimal transport. Unlike the original target diagram, the Wasserstein target diagram does not require paired data of equal sizes and evaluates models from the probability distribution perspective. We further establish that the two target diagrams coincide when the given paired data are comonotone, which is common in climate science, such as cumulative precipitation and solar radiation. Through two simulation studies and an application to wind speed distribution fitting, we demonstrate that the proposed diagram can detect the probability distribution discrepancies that the original target diagram may not distinguish. The Wasserstein target diagram therefore lifts the model evaluation to the probability level and provides a complementary approach to estimating the agreement in probability distributions.

Introduction

Evaluating the agreement between observations and model predictions is an important task in statistical modeling and scientific applications. Common evaluation measures often use statistical metrics to quantify the model prediction performance. For example, the bias of mean values measures systematic overestimation or underestimation, while the root mean squared error (RMSE) summarizes the overall magnitude of prediction errors. In the last two decades, the Taylor diagram [9] and the target diagram [8], which both visualize the evaluation metrics in a single diagram, have been widely used in model evaluation. The Taylor diagram uses the law of cosines to visualize the evaluation metrics, including correlation coefficient, centered root mean squared error (CRMSE), and variances of observed and predicted data. However, the Taylor diagram ignores the mean values. To solve this problem, the authors in [8] introduced the target diagram by decomposing RMSE into bias of mean values and CRMSE, which satisfy the Pythagorean theorem. The mathematical construction of the target diagram is as follows.

Given predicted data {xi}i=1N\{x_i\}_{i=1}^{N} and observed data {yi}i=1N\{y_i\}_{i=1}^{N}, the RMSE is given as

RMSE⁡2=1N∑i=1N(xi−yi)2=1N∑i=1N(xˉ−yˉ+(xi−xˉ)−(yi−yˉ))2\operatorname{RMSE}^{2} = \frac{1}{N}\sum_{i=1}^{N}(x_i-y_i)^2 = \frac{1}{N}\sum_{i=1}^{N}\left(\bar{x}-\bar{y}+(x_i-\bar{x})-(y_i-\bar{y})\right)^2

where xˉ\bar{x} and yˉ\bar{y} are the means of {xi}i=1N\{x_i\}_{i=1}^{N} and {yi}i=1N\{y_i\}_{i=1}^{N}, respectively. Let xi′=xi−xˉx_i^\prime=x_i-\bar{x} and yi′=yi−yˉy_i^\prime=y_i-\bar{y} be the centered data of {xi}i=1N\{x_i\}_{i=1}^{N} and {yi}i=1N\{y_i\}_{i=1}^{N}. Then

RMSE⁡2=1N∑i=1N(xˉ−yˉ)2+2N∑i=1N(xˉ−yˉ)(xi′−yi′)+1N∑i=1N(xi′−yi′)2=Bias⁡2+CRMSE⁡2,\begin{aligned} \operatorname{RMSE}^{2} = \frac{1}{N}\sum_{i=1}^{N}(\bar{x}-\bar{y})^2 + \frac{2}{N}\sum_{i=1}^{N}(\bar{x}-\bar{y})(x_i^\prime-y_i^\prime) + \frac{1}{N}\sum_{i=1}^{N}(x_i^\prime-y_i^\prime)^2 \\ &= \operatorname{Bias}^{2} + \operatorname{CRMSE}^{2}, \end{aligned}

where Bias =xˉ−yˉ=\bar{x}-\bar{y} and CRMSE is the RMSE of centered data given by

CRMSE2=1N∑i=1N(xi′−yi′)2,\mathrm{CRMSE}^{2}=\frac{1}{N}\sum_{i=1}^{N}(x_i^{\prime}-y_i^{\prime})^{2},

and the cross term vanishes since

∑i=1N(xi′−yi′)=∑i=1N(xi−xˉ)−∑i=1N(yi−yˉ)=0.\sum_{i=1}^{N}(x_i^{\prime}-y_i^{\prime})=\sum_{i=1}^{N}(x_i-\bar{x})-\sum_{i=1}^{N}(y_i-\bar{y})=0.

Given a,b∈Ra,b\in\mathbb{R}, define

sign⁡(a−b)={1,a>b,0,a=b,−1,a<b.\operatorname{sign}(a-b)= \begin{cases} 1, & a>b,\\ 0, & a=b,\\ -1, & a<b. \end{cases}

Let σx\sigma_x and σy\sigma_y be the sample standard deviations of {xi}i=1N\{x_i\}_{i=1}^{N} and {yi}i=1N\{y_i\}_{i=1}^{N}, respectively. Then a point in the target diagram with respect to {xi}i=1N\{x_i\}_{i=1}^{N} and {yi}i=1N\{y_i\}_{i=1}^{N} has coordinates (s,t)(s,t) where

s=sign⁡(σx−σy)⋅CRMSEs=\operatorname{sign}(\sigma_x-\sigma_y)\cdot\mathrm{CRMSE} and t=Bias=xˉ−yˉt=\mathrm{Bias}=\bar{x}-\bar{y}.

When σx≠σy\sigma_x\ne\sigma_y, sign⁡(σx−σy)=±1\operatorname{sign}(\sigma_x-\sigma_y)=\pm1, and we have the following Pythagorean theorem

RMSE2=CRMSE2+Bias2=s2+t2.(1)\mathrm{RMSE}^{2}=\mathrm{CRMSE}^{2}+\mathrm{Bias}^{2}=s^{2}+t^{2}. \tag*{(1)}

Therefore, the radial distance from the point (s,t)(s,t) to the origin in the target diagram represents RMSE. The sign of the horizontal coordinate indicates whether the predictions have a larger or smaller standard deviation than the observations, while the sign of the vertical coordinate shows whether the predictions have a larger or smaller mean value than the observations. Therefore, the target diagram represents the spatial or temporal matching of predicted data to observations, and allows multiple models to be compared within a common coordinate system. The closer a model is to the origin in the target diagram, the smaller its RMSE and the better its performance. An illustration of the target diagram is presented in Figure 1 using six prediction samples of equal size generated from different distributions.

Figure 1. Target Diagram Example. The reference sample consists of n=1,000n = 1{,}000 observations generated from a normal distribution N(5,2.25)\mathcal{N}(5, 2.25), with mean 4.940654 and standard deviation 1.501181. Predicted data are generated from six different distributions. All samples have equal size (n=1,000n = 1{,}000) and are paired with the reference. The horizontal coordinate represents the signed CRMSE, the vertical coordinate represents bias, and radial distance represents RMSE. Positive (negative) horizontal coordinates indicate that predicted data have larger (smaller) standard deviations than the reference. Positive (negative) vertical coordinates indicate that predicted data have larger (smaller) means than the reference.

The target diagram requires paired observed and predicted data of equal length, since RMSE and CRMSE are based on pointwise differences. Consequently, it cannot be directly applied when the two samples have different sizes or only observed data exist. Furthermore, in many applications, the objective is to compare probability distributions rather than paired observations, for example, how to choose the best model when observed data are fitted to different probability distributions. In such settings, RMSE is not directly applicable without introducing an artificial pairing. Motivated by this insight, we lift the target diagram into the probability distribution level and propose the Wasserstein target diagram to evaluate agreement in probability distributions. Relevant studies on the Taylor and Wasserstein Taylor diagrams can be found in [1]. The Wasserstein distance, originating from the optimal transport theory, provides a natural approach to comparing probability distributions [2]. In particular, for two probability measures μ\mu and ν\nu on R\mathbb{R} with finite second moments, the 2-Wasserstein distance (W2)(W_2) satisfies the decomposition

W22(μ,ν)=(mμ−mν)2+W22(μ~,ν~),(2)W_2^2(\mu,\nu)=(m_\mu-m_\nu)^2+W_2^2(\tilde{\mu},\tilde{\nu}), \tag*{(2)}

where mμm_\mu and mνm_\nu are the distribution means, and μ~\tilde{\mu} and ν~\tilde{\nu} denote the corresponding centered distributions. Similar to the role of Equation (1) for the target diagram, Equation (2) supports the construction of the Wasserstein target diagram.

The main contributions of this work are as follows. First, we develop the Wasserstein target diagram as a visual representation of the decomposition in Equation (2) and evaluate models from the probability distribution perspective. It can compare unpaired samples and samples of different sizes, and detect distributional discrepancies beyond the bias. Second, in Theorem 3.1, we show that the target diagram and the Wasserstein target diagram coincide if and only if the given data are comonotone, where the data are taken as equally weighted empirical distributions in the Wasserstein target diagram. Finally, we demonstrate the proposed approach through two simulation studies with equal and unequal sample sizes and an application to the distribution fitting of wind speed.

The rest of the paper is organized as follows. § 2 introduces the preliminaries of optimal transport and the Wasserstein distance. § 3 presents the construction and theoretical properties of the Wasserstein target diagram. § 4 illustrates the proposed diagram through simulation studies and a real-world data application in wind speed. Finally, § 5 provides the discussion and conclusion.

Preliminaries

Let P(R)\mathcal{P}(\mathbb{R}) denote the set of Borel probability measures on the real line R\mathbb{R}, and let P2(R)⊂P(R)\mathcal{P}_2(\mathbb{R}) \subset\mathcal{P}(\mathbb{R}) denote the set of Borel probability measures with finite second moments, i.e.,

P2(R)={μ∈P(R):∫R∣x∣2 dμ(x)<∞}.\mathcal{P}_2(\mathbb{R}) = \left\{ \mu\in\mathcal{P}(\mathbb{R}) : \int_{\mathbb{R}} |x|^2 \, d\mu(x) < \infty\right\}.

Given μ∈P(R)\mu\in\mathcal{P}(\mathbb{R}) and a Borel measurable map T:R→RT:\mathbb{R}\to\mathbb{R}, the pushforward of μ\mu by TT, denoted by T#μT_{\#}\mu, is defined by

T#μ(B):=μ(T−1(B))T_{\#}\mu(B):=\mu\left(T^{-1}(B)\right)

for every Borel set B⊆RB\subseteq\mathbb{R}. For μ∈P2(R)\mu\in\mathcal{P}_{2}(\mathbb{R}), its mean mμm_{\mu} and standard deviation σμ\sigma_{\mu} are given as

mμ:=∫Rx dμ(x)andσμ:=[∫R(x−mμ)2 dμ(x)]1/2.(3)m_{\mu}:=\int_{\mathbb{R}}x\,d\mu(x)\quad\text{and}\quad\sigma_{\mu}:=\left[\int_{\mathbb{R}}(x-m_{\mu})^{2}\,d\mu(x)\right]^{1/2}. \tag*{(3)}

The centered measure of μ\mu is defined as μ~:=(τμ)#μ\widetilde{\mu}:=(\tau_{\mu})_{\#}\mu where τμ(x)=x−mμ\tau_{\mu}(x)=x-m_{\mu}.

It is known that the 2-Wasserstein distance (W2W_{2}) from optimal transport is a metric on P2(R)\mathcal{P}_{2}(\mathbb{R}) and is often used to quantify the similarities and discrepancies between probability distributions. See [2] for more details on the Wasserstein distance. Given μ,ν∈P2(R)\mu,\nu\in\mathcal{P}_{2}(\mathbb{R}), the 2-Wasserstein distance between μ\mu and ν\nu is given as

W2(μ,ν):=(inf⁡γ∈Γ(μ,ν)∫R×R∣x−y∣2 dγ(x,y))1/2,(4)W_{2}(\mu,\nu):=\left(\inf_{\gamma\in\Gamma(\mu,\nu)}\int_{\mathbb{R}\times\mathbb{R}}|x-y|^{2}\,d\gamma(x,y)\right)^{1/2}, \tag*{(4)}

where Γ(μ,ν)\Gamma(\mu,\nu) is the set of transport couplings with marginals μ\mu and ν\nu defined as

Γ(μ,ν):={γ∈P(R×R):πx#γ=μ, πy#γ=ν},\Gamma(\mu,\nu):=\left\{\gamma\in\mathcal{P}(\mathbb{R}\times\mathbb{R}):\pi_{x\#}\gamma=\mu,\ \pi_{y\#}\gamma=\nu\right\},

and πx\pi_{x} and πy\pi_{y} are projections: for (x,y)∈R2(x,y)\in\mathbb{R}^{2}, πx(x,y)=x\pi_{x}(x,y)=x and πy(x,y)=y\pi_{y}(x,y)=y. Note that the 2-Wasserstein distance on R\mathbb{R} has a quantile representation and is often used in numerical studies. Given μ,ν∈P2(R)\mu,\nu\in\mathcal{P}_{2}(\mathbb{R}), let FF and GG denote the associated cumulative distribution functions:

F(x)=μ((−∞,x])andG(x)=ν((−∞,x]).F(x)=\mu((-\infty,x])\quad\text{and}\quad G(x)=\nu((-\infty,x]).

The generalized inverses of FF and GG, also called the quantile functions, are defined by

F−1(t)=inf⁡{x∈R:F(x)≥t},0<t<1,F^{-1}(t)=\inf\{x\in\mathbb{R}:F(x)\geq t\},\quad0<t<1,

and

G−1(t)=inf⁡{x∈R:G(x)≥t},0<t<1.G^{-1}(t)=\inf\{x\in\mathbb{R}:G(x)\geq t\},\quad0<t<1.

Then the 2-Wasserstein distance between μ\mu and ν\nu satisfies

W2(μ,ν)=(∫01∣F−1(t)−G−1(t)∣2 dt)1/2.(5)W_{2}(\mu,\nu)=\left(\int_{0}^{1}\lvert F^{-1}(t)-G^{-1}(t)\rvert^{2}\,dt\right)^{1/2}. \tag*{(5)}

In particular, given data {xi}i=1M\{x_{i}\}_{i=1}^{M} and {yi}i=1N\{y_{i}\}_{i=1}^{N} where MM may not equal NN, let μM\mu_{M} and νN\nu_{N} be the corresponding empirical measures, i.e., μM:=1M∑i=1Mδxi\mu_{M}:=\frac{1}{M}\sum_{i=1}^{M}\delta_{x_{i}} and νN:=1N∑i=1Nδyi\nu_{N}:=\frac{1}{N}\sum_{i=1}^{N}\delta_{y_{i}}. Then the associated quantile functions F−1F^{-1} and G−1G^{-1} are stepwise functions and have the form

F−1(t)={x(1),0<t≤1/Mx(2),1/M<t≤2/M⋮⋮x(M),(M−1)/M<t≤1andG−1(t)={y(1),0<t≤1/Ny(2),1/N<t≤2/N⋮⋮y(N),(N−1)/N<t≤1F^{-1}(t)= \begin{cases} x_{(1)}, & 0<t\leq1/M\\ x_{(2)}, & 1/M<t\leq2/M\\ \vdots& \vdots\\ x_{(M)}, & (M-1)/M<t\leq1 \end{cases} \quad\text{and}\quad G^{-1}(t)= \begin{cases} y_{(1)}, & 0<t\leq1/N\\ y_{(2)}, & 1/N<t\leq2/N\\ \vdots& \vdots\\ y_{(N)}, & (N-1)/N<t\leq1 \end{cases}

where x(1)≤x(2)≤⋯≤x(M)x_{(1)}\leq x_{(2)}\leq\cdots\leq x_{(M)} and y(1)≤y(2)≤⋯≤y(N)y_{(1)}\leq y_{(2)}\leq\cdots\leq y_{(N)} are order statistics.

Let Ii=(i−1M,iM]I_{i}=\left(\frac{i-1}{M},\frac{i}{M}\right] and Jj=(j−1N,jN]J_{j}=\left(\frac{j-1}{N},\frac{j}{N}\right], for i=1,⋯ ,Mi=1,\cdots,M and j=1,⋯ ,Nj=1,\cdots,N. The length of the intersection Ii∩JjI_{i}\cap J_{j} is

wij:=∣Ii∩Jj∣=[min⁡{iM,jN}−max⁡{i−1M,j−1N}]+,w_{ij}:=\lvert I_{i}\cap J_{j}\rvert=\left[\min\left\{\frac{i}{M},\frac{j}{N}\right\}-\max\left\{\frac{i-1}{M},\frac{j-1}{N}\right\}\right]_{+},

where [x]+:=max⁡{x,0}[x]_{+}:=\max\{x,0\}. Therefore, Equation (5) implies

W22(μM,νN)=∑i=1M∑j=1N∫Ii∩Jj∣x(i)−y(j)∣2 dt=∑i=1M∑j=1Nwij∣x(i)−y(j)∣2.(6)W_{2}^{2}(\mu_{M},\nu_{N})=\sum_{i=1}^{M}\sum_{j=1}^{N}\int_{I_{i}\cap J_{j}}|x_{(i)}-y_{(j)}|^{2}\,\mathrm{d}t=\sum_{i=1}^{M}\sum_{j=1}^{N}w_{ij}|x_{(i)}-y_{(j)}|^{2}. \tag*{(6)}

Especially, when M=NM=N, the 2-Wasserstein distance between μN\mu_{N} and νN\nu_{N} satisfies

W22(μN,νN)=1N∑i=1N∣x(i)−y(i)∣2.(7)W_{2}^{2}(\mu_{N},\nu_{N})=\frac{1}{N}\sum_{i=1}^{N}|x_{(i)}-y_{(i)}|^{2}. \tag*{(7)}

Similar to the decomposition of squared RMSE, the squared 2-Wasserstein distance admits the following decomposition

W22(μ,ν)=(mμ−mν)2+W22(μ~,ν~),(8)W_{2}^{2}(\mu,\nu)=(m_{\mu}-m_{\nu})^{2}+W_{2}^{2}(\widetilde{\mu},\widetilde{\nu}), \tag*{(8)}

where μ~\widetilde{\mu} and ν~\widetilde{\nu} are the centered measures of μ\mu and ν\nu, respectively. Furthermore, [3] showed that the distribution differences in terms of squared 2-Wasserstein distance can be decomposed into three parts, including bias (or location), size (or variability), and shape. That is,

W22(μ,ν)=(mμ−mν)2+(σμ−σν)2+2σμσν(1−ρq)(9)W_{2}^{2}(\mu,\nu)=(m_{\mu}-m_{\nu})^{2}+(\sigma_{\mu}-\sigma_{\nu})^{2}+2\sigma_{\mu}\sigma_{\nu}(1-\rho_{q}) \tag*{(9)}

where ρq\rho_{q} is the quantile correlation coefficient between μ\mu and ν\nu defined by

ρq=∫01(F−1(t)−mμ)(G−1(t)−mν) dtσμσν.\rho_{q}=\frac{\int_{0}^{1}(F^{-1}(t)-m_{\mu})(G^{-1}(t)-m_{\nu})\,\mathrm{d}t}{\sigma_{\mu}\sigma_{\nu}}.

See [1, 4, 5] for more applications of such decomposition.

Wasserstein Target Diagram

Given predictions {xi}i=1N\{x_{i}\}_{i=1}^{N} and observations {yi}i=1N\{y_{i}\}_{i=1}^{N}, suppose xˉ\bar{x} and yˉ\bar{y} are corresponding sample means, σx\sigma_{x} and σy\sigma_{y} are associated sample standard deviations, and {xi′=xi−xˉ}i=1N\{x_{i}^{\prime}=x_{i}-\bar{x}\}_{i=1}^{N} and {yi′=yi−yˉ}i=1N\{y_{i}^{\prime}=y_{i}-\bar{y}\}_{i=1}^{N} are their centered data. Then we have the following decomposition

RMSE2=Bias2+CRMSE2,\mathrm{RMSE}^{2}=\mathrm{Bias}^{2}+\mathrm{CRMSE}^{2},

where Bias=xˉ−yˉ\mathrm{Bias}=\bar{x}-\bar{y}, CRMSE\mathrm{CRMSE} is the centered RMSE, with

RMSE2=1N∑i=1N(xi−yi)2andCRMSE2=1N∑i=1N(xi′−yi′)2.\mathrm{RMSE}^{2}=\frac{1}{N}\sum_{i=1}^{N}(x_{i}-y_{i})^{2}\quad\text{and}\quad\mathrm{CRMSE}^{2}=\frac{1}{N}\sum_{i=1}^{N}(x_{i}^{\prime}-y_{i}^{\prime})^{2}.

Then the point in the target diagram for {xi}i=1N\{x_{i}\}_{i=1}^{N} and {yi}i=1N\{y_{i}\}_{i=1}^{N} has coordinates (s,t)(s,t) where

s=sign⁡(σx−σy)⋅CRMSEandt=Bias,s=\operatorname{sign}(\sigma_{x}-\sigma_{y})\cdot\mathrm{CRMSE}\quad\text{and}\quad t=\mathrm{Bias},

so that when σx≠σy\sigma_{x}\ne\sigma_{y}, RMSE2=s2+t2\mathrm{RMSE}^{2}=s^{2}+t^{2}. Therefore, the radial distance from the point (s,t)(s,t) to the origin in the target diagram represents the associated RMSE.

The target diagram requires paired observed and predicted data of equal length, and cannot be directly applied when the two samples have different sizes. For example, only observed data exist in the wind speed distribution fitting [10], and the target diagram does not work. In contrast, the Wasserstein distance compares probability distributions directly and does not require paired data or equal sample sizes. This motivates us to lift the target diagram into the probability level via the Wasserstein distance and construct an analogous Wasserstein target diagram. The mathematical details are as follows.

Let μ,ν∈P2(R)\mu,\nu\in\mathcal{P}_{2}(\mathbb{R}) with means mμ,mνm_{\mu},m_{\nu} and standard deviations σμ,σν\sigma_{\mu},\sigma_{\nu}, respectively. Let μ~\widetilde{\mu} and ν~\widetilde{\nu} denote their centered measures. Equation (8) shows that

W22(μ,ν)=(mμ−mν)2+W22(μ~,ν~).(10)W_{2}^{2}(\mu,\nu)=(m_{\mu}-m_{\nu})^{2}+W_{2}^{2}(\widetilde{\mu},\widetilde{\nu}). \tag*{(10)}

Then the coordinates of a point (s,t)(s,t) in the Wasserstein target diagram with respect to μ\mu and ν\nu are given as

s=sign⁡(σμ−σν)⋅W2(μ~,ν~) and t=mμ−mν,(11)s=\operatorname{sign}(\sigma_\mu-\sigma_\nu)\cdot W_2(\tilde{\mu},\tilde{\nu})\text{ and }t=m_\mu-m_\nu, \tag*{(11)}

where ss is the signed centered 2-Wasserstein distance (W2W_2) between μ\mu and ν\nu, and tt is the bias. When σμ≠σν\sigma_\mu\ne\sigma_\nu, the coordinates satisfy

s2+t2=W22(μ~,ν~)+(mμ−mν)2=W22(μ,ν),(12)s^2+t^2=W_2^2(\tilde{\mu},\tilde{\nu})+(m_\mu-m_\nu)^2=W_2^2(\mu,\nu), \tag*{(12)}

which shows that the radial distance from point (s,t)(s,t) to the origin in the Wasserstein target diagram represents the 2-Wasserstein distance W2(μ,ν)W_2(\mu,\nu).

Both the target diagram and Wasserstein target diagram use the bias as the vertical coordinate. For the horizontal coordinate and radial distance to the origin, the target diagram uses signed CRMSE and RMSE, while the Wasserstein target diagram uses signed centered 2-Wasserstein distance and 2-Wasserstein distance. Note that in target and Wasserstein target diagrams, the sign of horizontal coordinate indicates whether the predictions have a larger or smaller standard deviation than the observations, while the sign of the vertical coordinate indicates whether the predictions have a larger or smaller mean than the observations. In particular, the magnitude of the horizontal coordinate in the Wasserstein target diagram reflects discrepancies between the centered distributions, including distribution differences in size and shape (see Equations (8) and (9)). Figure 2 shows the comparison of target diagram and Wasserstein target diagram.

Figure 2. Comparison of the Target Diagram and Wasserstein Target Diagram. The target diagram evaluates models based on the temporal or spatial matching between predicted and observed data, while the Wasserstein target diagram is from the probability distribution perspective. Panel (a): Given data {xi}i=1N\{x_i\}_{i=1}^{N} and {yi}i=1N\{y_i\}_{i=1}^{N} with sample means xˉ\bar{x} and yˉ\bar{y} and different standard deviations σx\sigma_x and σy\sigma_y, the target diagram is constructed by the decomposition RMSE2=Bias2+CRMSE2\mathrm{RMSE}^2=\mathrm{Bias}^2+\mathrm{CRMSE}^2, where the horizontal and vertical coordinates are sign⁡(σx−σy)⋅CRMSE\operatorname{sign}(\sigma_x-\sigma_y)\cdot\mathrm{CRMSE} and the bias xˉ−yˉ\bar{x}-\bar{y}, respectively, where CRMSE is the centered RMSE. The radial distance to the origin is RMSE. Panel (b): Given probability distributions μ,ν∈P2(R)\mu,\nu\in\mathcal{P}_2(\mathbb{R}) with means mμm_\mu and mνm_\nu and different standard deviations σμ\sigma_\mu and σν\sigma_\nu, the Wasserstein target diagram is constructed by the decomposition W22(μ,ν)=Bias2+W22(μ~,ν~)W_2^2(\mu,\nu)=\mathrm{Bias}^2+W_2^2(\tilde{\mu},\tilde{\nu}), where the horizontal and vertical coordinates are sign⁡(σμ−σν)⋅W2(μ~,ν~)\operatorname{sign}(\sigma_\mu-\sigma_\nu)\cdot W_2(\tilde{\mu},\tilde{\nu}) and the bias mμ−mνm_\mu-m_\nu, respectively. The radial distance is the 2-Wasserstein distance W2(μ,ν)W_2(\mu,\nu).

When evaluating models, the target diagram uses paired data of equal length and shows the spatial or temporal matching between predictions and observations, while the Wasserstein target diagram uses the 2-Wasserstein distance and compares the distribution similarities of used models to the reference. Therefore, the Wasserstein target diagram allows multiple probability distributions to be compared with a reference within a common coordinate system, and the closer a distribution is to the origin in the diagram, the smaller its 2-Wasserstein distance and the better its performance. Unlike the target diagram, the Wasserstein target diagram does not require paired observations or equal sample size and can be directly applied to evaluate the probability distribution fitting of given data. Furthermore, Theorem 3.1 shows that the target diagram and the Wasserstein target diagram coincide if and only if the given data {xi}i=1N\{x_i\}_{i=1}^{N} and {yi}i=1N\{y_i\}_{i=1}^{N} are comonotone, where the data are taken as equally weighted empirical measures in the Wasserstein target diagram. The proof is based on the rearrangement inequality. Recall that given {xi}i=1N\{x_i\}_{i=1}^{N} and {yi}i=1N\{y_i\}_{i=1}^{N} with order statistics {x(i)}i=1N\{x_{(i)}\}_{i=1}^{N} and {y(i)}i=1N\{y_{(i)}\}_{i=1}^{N}, that is,

x(1)≤x(2)≤⋯≤x(N)andy(1)≤y(2)≤⋯≤y(N),x_{(1)} \le x_{(2)} \le\cdots\le x_{(N)} \quad\text{and} \quad y_{(1)} \le y_{(2)} \le\cdots\le y_{(N)},

the rearrangement inequality claims

∑i=1Nxiyi≤∑i=1Nx(i)y(i),\sum_{i=1}^{N} x_i y_i \le\sum_{i=1}^{N} x_{(i)} y_{(i)},

where the equality holds if and only if {xi}i=1N\{x_i\}_{i=1}^{N} and {yi}i=1N\{y_i\}_{i=1}^{N} are comonotone, i.e.,

(xi−xj)(yi−yj)≥0,for any i,j.(x_i-x_j)(y_i-y_j) \ge0,\quad\text{for any } i,j.

Then we have the following main result in this paper.

Theorem 3.1. Given data {xi}i=1N\{x_i\}_{i=1}^{N} and {yi}i=1N\{y_i\}_{i=1}^{N} with different standard deviations, let

μN:=1N∑i=1NδxiandνN:=1N∑i=1Nδyi.\mu_N := \frac{1}{N}\sum_{i=1}^{N}\delta_{x_i}\quad\text{and}\quad\nu_N := \frac{1}{N}\sum_{i=1}^{N}\delta_{y_i}.

The Wasserstein target diagram for μN\mu_N and νN\nu_N coincides with the target diagram for {xi}i=1N\{x_i\}_{i=1}^{N} and {yi}i=1N\{y_i\}_{i=1}^{N} if and only if {xi}i=1N\{x_i\}_{i=1}^{N} and {yi}i=1N\{y_i\}_{i=1}^{N} are comonotone, i.e.,

(xi−xj)(yi−yj)≥0,for any i,j.(x_i-x_j)(y_i-y_j) \ge0,\quad\text{for any } i,j.

Proof. By Equation (7), the 2-Wasserstein distance between μN\mu_N and νN\nu_N satisfies

W22(μN,νN)=1N∑i=1N∣x(i)−y(i)∣2=1N∑i=1Nxi2+1N∑i=1Nyi2−2N∑i=1Nx(i)y(i),W_2^2(\mu_N,\nu_N)=\frac{1}{N}\sum_{i=1}^{N}\left|x_{(i)}-y_{(i)}\right|^2=\frac{1}{N}\sum_{i=1}^{N}x_i^2+\frac{1}{N}\sum_{i=1}^{N}y_i^2-\frac{2}{N}\sum_{i=1}^{N}x_{(i)}y_{(i)},

where x(1)≤x(2)≤⋯≤x(N)x_{(1)} \le x_{(2)} \le\cdots\le x_{(N)} and y(1)≤y(2)≤⋯≤y(N)y_{(1)} \le y_{(2)} \le\cdots\le y_{(N)}. Then by the rearrangement inequality,

W22(μN,νN)≤1N∑i=1Nxi2+1N∑i=1Nyi2−2N∑i=1Nxiyi=1N∑i=1N(xi−yi)2=RMSE2,\begin{aligned} W_2^2(\mu_N,\nu_N) \le\frac{1}{N}\sum_{i=1}^{N}x_i^2+\frac{1}{N}\sum_{i=1}^{N}y_i^2-\frac{2}{N}\sum_{i=1}^{N}x_i y_i \\ &= \frac{1}{N}\sum_{i=1}^{N}(x_i-y_i)^2=\mathrm{RMSE}^2, \end{aligned}

and the equality holds if and only if {xi}i=1N\{x_i\}_{i=1}^{N} and {yi}i=1N\{y_i\}_{i=1}^{N} are comonotone, i.e.,

(xi−xj)(yi−yj)≥0,for any i,j.(x_i-x_j)(y_i-y_j) \ge0,\quad\text{for any } i,j.

Suppose xˉ\bar{x} and yˉ\bar{y} are the means of {xi}i=1N\{x_i\}_{i=1}^{N} and {yi}i=1N\{y_i\}_{i=1}^{N}, and {xi′=xi−xˉ}i=1N\{x'_i=x_i-\bar{x}\}_{i=1}^{N} and {yi′=yi−yˉ}i=1N\{y'_i=y_i-\bar{y}\}_{i=1}^{N} are the associated centered data. Let μ~N\tilde{\mu}_N and ν~N\tilde{\nu}_N denote the centered measures of μN\mu_N and νN\nu_N, i.e.,

μ~N:=1N∑i=1Nδxi′andν~N:=1N∑i=1Nδyi′.\tilde{\mu}_N := \frac{1}{N}\sum_{i=1}^{N}\delta_{x'_i} \quad\text{and}\quad\tilde{\nu}_N := \frac{1}{N}\sum_{i=1}^{N}\delta_{y'_i}.

Then applying the same argument to μ~N\tilde{\mu}_N and ν~N\tilde{\nu}_N gives

W22(μ~N,ν~N)≤1N∑i=1N(xi′−yi′)2=CRMSE2,W_2^2(\tilde{\mu}_N,\tilde{\nu}_N) \leq\frac{1}{N}\sum_{i=1}^{N}(x'_i-y'_i)^2 = \mathrm{CRMSE}^2,

and the equality holds if and only if

(xi′−xj′)(yi′−yj′)=(xi−xj)(yi−yj)≥0,for any i,j,(x'_i-x'_j)(y'_i-y'_j)=(x_i-x_j)(y_i-y_j)\geq0,\quad\text{for any }i,j,

which means the equality holds if and only if {xi}i=1N\{x_i\}_{i=1}^{N} and {yi}i=1N\{y_i\}_{i=1}^{N} are comonotone. Therefore, under the comonotone condition,

W2(μN,νN)=RMSEandW2(μ~N,ν~N)=CRMSE.W_2(\mu_N,\nu_N)=\mathrm{RMSE}\quad\text{and}\quad W_2(\tilde{\mu}_N,\tilde{\nu}_N)=\mathrm{CRMSE}.

Moreover,

mμN=xˉ,mνN=yˉ,σμN=σx,andσνN=σy,m_{\mu_N}=\bar{x},\quad m_{\nu_N}=\bar{y},\quad\sigma_{\mu_N}=\sigma_x,\quad\text{and}\quad\sigma_{\nu_N}=\sigma_y,

where σx\sigma_x and σy\sigma_y are sample standard deviations of {xi}i=1N\{x_i\}_{i=1}^{N} and {yi}i=1N\{y_i\}_{i=1}^{N}, respectively. Then the target diagram and Wasserstein target diagram have the same vertical coordinate, horizontal magnitude, and horizontal sign. Hence, they coincide.

Conversely, if the target diagram and Wasserstein target diagram coincide, then the corresponding horizontal coordinates are the same, i.e.,

sign⁡(σx−σy)⋅CRMSE=sign⁡(σμN−σνN)⋅W2(μ~N,ν~N).\operatorname{sign}(\sigma_x-\sigma_y)\cdot\mathrm{CRMSE} =\operatorname{sign}(\sigma_{\mu_N}-\sigma_{\nu_N})\cdot W_2(\tilde{\mu}_N,\tilde{\nu}_N).

Note that σμN=σx\sigma_{\mu_N}=\sigma_x, σνN=σy\sigma_{\nu_N}=\sigma_y, and σx≠σy\sigma_x\ne\sigma_y. Then sign⁡(σx−σy)=sign⁡(σμN−σνN)=±1\operatorname{sign}(\sigma_x-\sigma_y)=\operatorname{sign}(\sigma_{\mu_N}-\sigma_{\nu_N})=\pm1 and thus W2(μ~N,ν~N)=CRMSEW_2(\tilde{\mu}_N,\tilde{\nu}_N)=\mathrm{CRMSE}, which implies the equality condition in the rearrangement inequality:

(xi−xj)(yi−yj)≥0,for any i,j.(x_i-x_j)(y_i-y_j)\geq0,\quad\text{for any }i,j.

□\square

In climate science, the cumulative climate variables indexed by time are nondecreasing and provide natural examples of comonotone structures, such as cumulative precipitation, accumulated runoff, and cumulative solar radiation. Therefore, for such cumulative variables, the Wasserstein target diagram just becomes the original target diagram. Note that when the given data {xi}i=1N\{x_i\}_{i=1}^{N} and {yi}i=1N\{y_i\}_{i=1}^{N} have the same sample standard deviations, i.e., σx=σy\sigma_x=\sigma_y, the horizontal coordinate is zero by the definition of the sign function, regardless of the value of CRMSE\mathrm{CRMSE} or W2(μ~,ν~)W_2(\tilde{\mu},\tilde{\nu}). Consequently, the point on the vertical axis may represent equal sample standard deviations, zero CRMSE\mathrm{CRMSE}, or zero W2(μ~,ν~)W_2(\tilde{\mu},\tilde{\nu}), and the radial distance does not necessarily equal RMSE\mathrm{RMSE} or W2(μ,ν)W_2(\mu,\nu). This special case should be particularly considered when it appears in both target and Wasserstein target diagrams.

Applications of the Wasserstein Target Diagram

In this section, we illustrate the Wasserstein target diagram through two simulations and a wind speed application. The first simulation compares the target and Wasserstein target diagrams using observed and predicted samples of equal size. The second simulation considers unequal sample sizes, for which the original target diagram does not apply directly. Finally, we apply the Wasserstein target diagram to wind speed data to compare fitted probability distributions with observed data. Throughout this section, NN denotes a normal distribution parameterized by mean and variance, LogN⁡\operatorname{LogN} denotes a lognormal distribution parameterized by log-scale mean and variance, and GAM⁡\operatorname{GAM} and WW denote gamma and Weibull distributions parameterized by shape and scale, respectively. The parameters of these distributions in the simulation studies are also compared with the reference distribution. In addition, we use W2W_{2} for the 2-Wasserstein distance.

Simulation with Equal Sample Sizes: Comparison of Target Diagrams

To illustrate the Wasserstein target diagram and compare it with the original target diagram, we generate an observed sample of size n=1,000n = 1{,}000 from a normal distribution N(4,4)N(4, 4), with theoretical mean 4 and standard deviation 2. The observed sample has a mean of 3.920872 and standard deviation of 2.001575. We independently generate nine prediction samples from the following distributions, each with the same sample size n=1,000n = 1{,}000:

  • N(4,4)N(4, 4): the same underlying normal distribution as the observed sample.

  • N(3,2.25)N(3, 2.25): smaller mean 3 and standard deviation 1.5.

  • N(5,6.25)N(5, 6.25): larger mean 5 and standard deviation 2.5.

  • N(5,2.25)N(5, 2.25): larger mean 5 but smaller standard deviation 1.5.

  • N(3,6.25)N(3, 6.25): smaller mean 3 but larger standard deviation 2.5.

  • GAM(5,1)GAM(5, 1): a gamma distribution with shape parameter 5 and scale parameter 1, having larger mean 5 and larger standard deviation 5≈2.236\sqrt{5} \approx2.236.

  • W(6,4)W(6, 4): a Weibull distribution with shape parameter 6 and scale parameter 4, having smaller mean ≈3.711\approx3.711 and standard deviation ≈0.719\approx0.719.

  • LogN(1.5,0.2025)LogN(1.5, 0.2025): a lognormal distribution with log-scale mean 1.5 and log-scale standard deviation 0.45, having larger mean ≈4.959\approx4.959 and standard deviation ≈2.350\approx2.350.

  • BimodalBimodal: an equally weighted mixture 12N(3,0.25)+12N(6,0.25)\frac{1}{2}N(3, 0.25) + \frac{1}{2}N(6, 0.25), having larger mean 4.5 but smaller standard deviation 2.5≈1.581\sqrt{2.5} \approx1.581.

As the sample generation suggests, the N(4,4)N(4, 4) predicted samples are generated from the same distribution as the observed samples, which means that the corresponding 2-Wasserstein distance (W2W_{2}) is small, but their RMSE may remain large because it depends on the pointwise pairing. In this case, the Wasserstein target diagram can detect the distributional similarities between predicted and observed data, but the target diagram cannot, as shown in Table 1 and Figure 3. Table 1 shows the biases, signed centered 2-Wasserstein distances, and 2-Wasserstein distances between predicted and observed data, where the 2-Wasserstein distance is computed by Equation (7). Figure 3 further shows the associated target diagram and Wasserstein target diagram. It can be seen from the Wasserstein target diagram that the N(4,4)N(4, 4) predicted samples are closest to the origin, with a 2-Wasserstein distance of 0.173417. In contrast, the target diagram does not identify the N(4,4)N(4, 4) prediction as the closest to the origin, since the N(4,4)N(4, 4) prediction has larger RMSE and CRMSE with values of 2.815418 and 2.813784, respectively. This occurs because RMSE and CRMSE depend on the pointwise pairing of data but two independently generated samples have no natural one-to-one correspondence. Consequently, the target diagram does not identify that

ModelsnnSigned centered W2W_{2}BiasW2W_{2}
N(4,4)N(4, 4)1,000-0.1444830.0959070.173417
N(3,2.25)N(3, 2.25)1,000-0.537894-0.9053421.053078
N(5,6.25)N(5, 6.25)1,0000.4924751.2453251.339166
N(5,2.25)N(5, 2.25)1,000-0.5061161.1687651.273642
N(3,6.25)N(3, 6.25)1,0000.463521-0.9670321.072382
GAM(5,1)GAM(5, 1)1,0000.4689851.0733271.171315
W(6,4)W(6, 4)1,000-1.314335-0.2222361.332991
LogN(1.5,0.2025)LogN(1.5, 0.2025)1,0000.7885391.1251711.373974
BimodalBimodal1,000-0.7633420.5905950.965139

Table 1. Wasserstein Target Diagram Measures for Samples of Equal Size. The reference sample consists of n=1,000n = 1{,}000 observations from the normal distribution N(4,4)N(4, 4), with a sample mean 3.920872 and standard deviation 2.001575. The predicted sample generated from each distribution also contains 1,000 data points.

Figure 3. Target Diagram and Wasserstein Target Diagram for Samples of Equal Size. The reference consists of 1,000 observations from the normal distribution N(4,4)N(4,4), and other predicted samples have the same size. The N(4,4)N(4,4) predicted sample lies closest to the origin in the Wasserstein target diagram, but not in the target diagram. This shows the usefulness of Wasserstein target diagram for probability distribution comparison, even for samples from the same distribution.

the N(4,4)N(4,4) prediction has the highest similarity with the observed data, while the Wasserstein target diagram shows such identification. The remaining predictions in the Wasserstein target diagram exhibit larger distances to the origin due to differences in their biases, sizes, and shapes. For example, the Weibull prediction has a relatively small bias of -0.222236 but a large centered 2-Wasserstein distance of 1.314335, indicating that bias alone does not adequately characterize its distributional discrepancy with the observed data. These results illustrate the usefulness of the Wasserstein target diagram for comparing probability distributions, even when samples are independently generated from the same distribution.

Simulation with Unequal Sample Sizes: Distribution Comparison

To illustrate the application of Wasserstein target diagram to samples of unequal sizes, we generate a reference sample of size n=10,000n = 10{,}000 from a gamma distribution GAM(4,1)GAM(4,1) as the observed data. The reference distribution has theoretical mean 4 and standard deviation 2, while the generated sample has a mean 4.024726 and standard deviation 1.997160. We independently generate nine other samples from the following distributions as predicted data:

  • GAM(4,1)GAM(4,1): n=20,000n = 20{,}000, the same underlying gamma distribution as the observed sample.

  • GAM(4,0.8)GAM(4,0.8): n=15,000n = 15{,}000, smaller mean 3.2 and standard deviation 1.6.

  • GAM(4,1.2)GAM(4,1.2): n=25,000n = 25{,}000, larger mean 4.8 and standard deviation 2.4.

  • GAM(9,0.5)GAM(9,0.5): n=18,000n = 18{,}000, larger mean 4.5 but smaller standard deviation 1.5.

  • GAM(4,1)+1GAM(4,1) + 1: n=22,000n = 22{,}000, a gamma distribution shifted by 1, having larger mean 5 but the same standard deviation 2.

  • W(2,4.5)W(2,4.5): n=30,000n = 30{,}000, a Weibull distribution with shape parameter 2 and scale parameter 4.5, having a slightly smaller mean ≈3.988\approx3.988 but a slightly larger standard deviation ≈2.085\approx2.085.

  • LogN(1.285,0.2025)LogN(1.285,0.2025): n=25,000n = 25{,}000, a lognormal distribution with log-scale mean 1.285 and log-scale standard deviation 0.45, having mean ≈4\approx4 and smaller standard deviation ≈1.895\approx1.895.

  • N(4,4)N(4,4): n=35,000n = 35{,}000, a normal distribution with the same theoretical mean 4 and standard deviation 2 as the reference distribution.

  • BimodalBimodal: n=40,000n = 40{,}000, an equally weighted mixture 12N(2.5,0.25)+12N(5.5,0.25)\frac{1}{2}N(2.5,0.25) + \frac{1}{2}N(5.5,0.25), having the same mean 4 but smaller standard deviation 2.5≈1.581\sqrt{2.5} \approx1.581.

Table 2 summarizes the resulting measures, where the 2-Wasserstein distance is calculated by Equation (6), and Figure 4 presents the corresponding Wasserstein target diagram. Note that the prediction generated from GAM(4,1)GAM(4,1) is closest to the origin in the diagram, with a 2-Wasserstein distance of 0.058370, despite the sample size being doubled compared to the observation. This is because the predicted and observed data follow the same distribution. The GAM(4,0.8)\mathit{GAM}(4, 0.8) prediction appears in the lower-left region because of its smaller mean and standard deviation, whereas GAM(4,1.2)\mathit{GAM}(4, 1.2) appears in the upper-right region. The GAM(9,0.5)\mathit{GAM}(9, 0.5) prediction combines a larger mean with smaller variability and appears in the upper-left region. The shifted Gamma prediction has a bias of 0.997958 but a centered 2-Wasserstein distance of only 0.050082, indicating a discrepancy primarily associated with bias.

Prediction distributionSize nnSigned centered W2W_2BiasW2W_2
GAM(4,1)GAM(4,1)20,0000.043151-0.0393070.058370
GAM(4,0.8)GAM(4,0.8)15,000-0.400569-0.8214870.913945
GAM(4,1.2)GAM(4,1.2)25,0000.3645010.7408790.825689
GAM(9,0.5)GAM(9,0.5)18,000-0.5250690.4887460.717336
GAM(4,1)+1GAM(4,1) + 122,0000.0500820.9979580.999214
W(2,4.5)W(2,4.5)30,0000.185145-0.0442700.190364
LogN(1.285,0.2025)LogN(1.285,0.2025)25,000-0.222552-0.0243300.223878
N(4,4)N(4,4)35,000-0.457811-0.0299270.458788
Bimodal40,000-0.854919-0.0247870.855278

Table 2. Wasserstein Target Diagram Measures for Samples of Unequal Sizes. The reference sample consists of n=10,000n = 10{,}000 observations from the Gamma distribution GAM(4,1)GAM(4,1), with a sample mean 4.024726 and standard deviation 1.997160. Predicted sample sizes are listed in the second column.

Figure 4. Wasserstein Target Diagram for Samples of Different Sizes. The reference sample consists of n=10,000n = 10{,}000 observations from GAM(4,1)\mathit{GAM}(4, 1), with a mean 4.024726 and standard deviation 1.997160. The horizontal coordinate represents the signed centered 2-Wasserstein distance (signed centered W2W_2), the vertical coordinate represents the bias, and the radial distance represents the 2-Wasserstein distance (W2W_2). The GAM(4,1)\mathit{GAM}(4, 1) prediction with n=20,000n = 20{,}000 has the highest distribution similarity with the reference since it is closest to the origin.

The Weibull, lognormal, normal, and bimodal predictions have small absolute biases but different centered 2-Wasserstein distances. In particular, the normal distribution has the same theoretical mean and standard deviation as the reference, but its centered 2-Wasserstein distance is 0.457811, demonstrating its sensitivity to shape beyond the first two moments. Its negative horizontal coordinate reflects a standard deviation slightly below that of the reference sample. The bimodal distribution also has theoretical mean 4, but its centered 2-Wasserstein distance of 0.854919 reflects distribution differences in both size and shape.

Wind Speed Data: Comparison of Fitted Distributions

Accurate modeling of wind speed distributions is important for a variety of applications, including building design and wind energy assessment [10]. A key task is to compare which candidate probability distribution can accurately estimate the wind speed distribution, and the Wasserstein target diagram provides a direct assessment for such distribution comparison. In this section, we use processed wind speed data described in [10], which are hourly averaged wind speed measurements collected at a height of 80 meters (m) from the year 2010 to 2020 at the Flatirons M2 meteorological tower, located about 8 kilometers south of Boulder, Colorado, USA [7, 6]. The observed sample contains n=96,432n = 96{,}432 measurements, with a mean 4.638 m/s and a standard deviation of 3.474 m/s.

We compare ten fitted distributions using the maximum likelihood estimates reported in Table 4 of [10], including beta-generalized Lindley (BGL), beta-exponential (BE), beta-Weibull (BW), beta-Lindley (BL), generalized Lindley (GL), Lindley (L), lognormal (LogN), gamma (GAM), Weibull (W), and generalized extreme value (GEV) distributions. Their notation and probability density functions appear in Table 3 of the same reference [10]. To find the 2-Wasserstein distance between each fitted distribution and the empirical distribution of wind speed, we first generate a sample of the same size n=96,432n = 96{,}432 from each fitted distribution and then use Equation (7) to calculate the corresponding 2-Wasserstein distance.

ModelSigned centered W2W_2BiasW2W_2
BGL0.102954-0.0062190.103142
BE-0.216582-0.0009990.216585
BW-0.219444-0.0025620.219459
BL-0.266436-0.0020810.266444
GL-0.610161-0.0232480.610604
L0.446706-0.0039180.446723
LogN0.7525230.0807400.756842
GAM-0.5183270.0033460.518338
W-0.6236160.0306350.624368
GEV1.0862300.0408831.086999

Table 3. Wasserstein Target Diagram Measures for Distribution Fitting of Wind Speed. The observed wind speed sample has a mean of 4.638 and a standard deviation of 3.474. The fitted parameters of candidate distributions are the maximum likelihood estimates in Table 4 of [10].

Table 3 summarizes the signed centered 2-Wasserstein distance, bias, and 2-Wasserstein distance for each probability distribution, and Figure 5 presents the corresponding Wasserstein target diagram. In the diagram, all the models are close to the horizontal axis, which means that the fitted distributions generally have small biases. Therefore, absolute bias alone provides limited information to evaluate the candidate distributions. For example, BE has a smaller absolute bias than BGL (0.000999 versus 0.006219), but BGL has a smaller centered 2-Wasserstein distance (0.102954 versus 0.216582) and 2-Wasserstein distance (0.103142 versus 0.216585) than BE. Furthermore, BGL is closest to the origin in the Wasserstein target diagram, with the smallest 2-Wasserstein distance (0.103142) among the fitted distributions, followed by BE and BW, with BW having a distance of 0.219459. GEV and LogN have the largest 2-Wasserstein distances (1.086999 and 0.756842). This means that BGL has the best performance in the wind speed distribution fitting among all candidate distributions, which also aligns with the results in [10].

Figure 5. Wasserstein Target Diagram for Distribution Fitting of Wind Speed. The reference wind speed sample has mean 4.638 and standard deviation 3.474. The horizontal coordinate represents the signed centered 2-Wasserstein distance (signed centered W2W_2), the vertical coordinate represents the bias, and the radial distance represents the 2-Wasserstein distance (W2W_2). Ten distributions are compared and BGL has the best performance, since it is closest to the origin and has the smallest 2-Wasserstein distance.

Discussion and Conclusion

In this paper, we lift the target diagram to the probability distribution level and propose the Wasserstein target diagram to compare the probability distributions of different models. The original target diagram evaluates errors between paired data of equal sample sizes, and shows the spatial or temporal matching between predicted and observed data. The Wasserstein target diagram instead provides a visual tool to compare probability distributions, and extends the applicability of target diagrams to settings involving unpaired samples, samples of different sizes, and probability distribution fitting to only observed data. Therefore, the choice between the target diagram and Wasserstein target diagram depends on the evaluation objective. Furthermore, Theorem 3.1 shows that the target diagram and the Wasserstein target diagram coincide if and only if the given data are comonotone, which is often seen for cumulative climate variables indexed by time.

The two simulation studies in § 4 illustrate the Wasserstein target diagram for samples with equal and unequal sizes. The equal-size simulation shows that the Wasserstein target diagram can capture distributional similarities between independent samples drawn from the same distribution, which have a small Wasserstein distance but a large RMSE. The unequal-size simulation demonstrates the probability comparisons of Wasserstein target diagram without a one-to-one correspondence between data. The real data application on wind speed further illustrates how the diagram distinguishes fitted distributions with small biases.

Note that the Wasserstein target diagram uses the bias, the sign of size (variability), and the centered 2-Wasserstein distance from the decomposition of 2-Wasserstein distance in Equation (9). This means that the magnitude of the size and the shape of the distributions are hidden in the Wasserstein target diagram. A natural modification is to include distribution size or shape information in the Wasserstein target diagram via using different sizes or face colors of the markers.

Data Availability

The code and data used in this paper can be found at the following link https://github.com/tiantiy/Wasserstein-Target-Diagram.

References

Department of Mathematics and Statistical Science, University of Idaho, Moscow, ID, USA, 83844 Email address: [email protected]

Department of Mathematics and Statistical Science, University of Idaho, Moscow, ID, USA, 83844 Email address: [email protected]

References

  1. [1]Dongwei Chen, Emily J King, and Hungjui Yu. Taylor diagram and Wasserstein distance for model evaluation. arXiv preprint arXiv:2609.17950, 2026.arxiv.org/abs/2609.17950
  2. [2]Alessio Figalli and Federico Glaudo. An Invitation to Optimal Transport, Wasserstein Distances, and Gradient Flows. Number 22 in EMS Textbooks in Mathematics. EMS Press, Berlin, Germany, 2021.DOI
  3. [3]Antonio Irpino and Elvira Romano. Optimal histogram representation of large data sets: Fisher vs piecewise linear approximations. Revue des nouvelles technologies de l’information, 1:99–110, 2007.
  4. [4]Antonio Irpino and Rosanna Verde. Basic statistics for distributional symbolic variables: a new metric-based approach. Advances in Data Analysis and Classification, 9(2):143–175, 2015.DOI
  5. [5]Antonio Irpino and Rosanna Verde. Linear regression for numeric symbolic variables: a least squares approach based on Wasserstein Distance. Advances in Data Analysis and Classification, 9(1):81–106, 2015.DOI
  6. [6]David Jager and Afshin Andreas. M2 meteorology tower dataset. https://midcdmz.nlr.gov/apps/sitehome.pl?site=NWTC. [Online; accessed 15-November-2026].
  7. [7]David Jager and Afshin Andreas. NREL National Wind Technology Center (NWTC): M2 Tower; Boulder, Colorado (Data). Technical report, National Renewable Energy Lab (NREL), Golden, CO, United States, 1996.DOI
  8. [8]Jason K Jolliff, John C Kindle, Igor Shulman, Bradley Penta, Marjorie AM Friedrichs, Robert Helber, and Robert A Arnone. Summary diagrams for coupled hydrodynamic-ecosystem model skill assessment. Journal of Marine Systems, 76(1-2):64–82, 2009.DOI
  9. [9]Karl E Taylor. Summarizing multiple aspects of model performance in a single diagram. Journal of geophysical research: atmospheres, 106(D7):7183–7192, 2001.DOI
  10. [10]Tiantian Yang and Dongwei Chen. Beta-generalized Lindley distribution: A novel probability model for wind speed. Renewable Energy, page 124216, 2025.arxiv.org/abs/2503.09912

Paper details

vlnc:W_9uyga4ge
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…
Contents