Introduction
The notion of entropy was first introduced by Clausius [4 ] as a measure of disorder in thermodynamics. This concept was later extended to information theory by Shannon [15 ] to measure the average uncertainty in a random variable. For a continuous non-negative random variable X X X with probability density function f X ( x ) f_X(x) f X ( x ) , the differential entropy is defined as,
H ( X ) = − ∫ 0 ∞ f ( x ) log f ( x ) d x = − E X [ log f ( X ) ] . (1) H(X)=-\int_{0}^{\infty} f(x)\log f(x)\,dx=-E_X[\log f(X)]. \tag*{(1)} H ( X ) = − ∫ 0 ∞ f ( x ) log f ( x ) d x = − E X [ log f ( X )] . (1)
Among the several generalizations of entropy measures, Rényi entropy introduced by Rényi [14 ] is one of the most important measures. For a continuous random variable X X X with probability density function f X ( x ) f_X(x) f X ( x ) the Rényi entropy of order α \alpha α is given by,
H α ( X ) = 1 1 − α log ( ∫ 0 ∞ f ( x ) α d x ) = 1 1 − α log ( E X [ f ( X ) α − 1 ] ) , α > 0 , α ≠ 1. (2) H_{\alpha}(X)=\frac{1}{1-\alpha}\log\left(\int_{0}^{\infty}f(x)^{\alpha}\,dx\right)=\frac{1}{1-\alpha}\log\left(E_X\left[f(X)^{\alpha-1}\right]\right),\quad\alpha>0,\ \alpha\ne1. \tag*{(2)} H α ( X ) = 1 − α 1 log ( ∫ 0 ∞ f ( x ) α d x ) = 1 − α 1 log ( E X [ f ( X ) α − 1 ] ) , α > 0 , α = 1. (2)
Tsallis [16 ] introduced another generalization called Tsallis entropy of order α \alpha α . Tsallis entropy is widely used for analyzing non extensive data. For a continuous non-negative random variable X X X with probability density function f X ( x ) f_X(x) f X ( x ) , the Tsallis entropy is defined as follows,
T α ( X ) = 1 α − 1 [ 1 − ∫ 0 ∞ f ( x ) α d x ] = 1 α − 1 [ 1 − E X ( f ( X ) α − 1 ) ] , α > 0 , α ≠ 1. (3) T_{\alpha}(X)=\frac{1}{\alpha-1}\left[1-\int_{0}^{\infty}f(x)^{\alpha}\,dx\right]=\frac{1}{\alpha-1}\left[1-E_X\left(f(X)^{\alpha-1}\right)\right],\quad\alpha>0,\ \alpha\ne1. \tag*{(3)} T α ( X ) = α − 1 1 [ 1 − ∫ 0 ∞ f ( x ) α d x ] = α − 1 1 [ 1 − E X ( f ( X ) α − 1 ) ] , α > 0 , α = 1. (3)
As α → 1 \alpha\to1 α → 1 , both equation (1.2) and (1.3) converge to equation (1.1).
Apart from entropy measures Kullback and Leibler [11 ] introduced the Kullback–Leibler (KL) divergence, provides a useful measure for comparing two probability distributions F X ( ⋅ ) F_X(\cdot) F X ( ⋅ ) and G Y ( ⋅ ) G_Y(\cdot) G Y ( ⋅ ) , and is denoted by d K L ( F X , G Y ) d_{KL}(F_X,G_Y) d K L ( F X , G Y ) . It is a measure of the inefficiency of assuming that the distribution is g Y ( x ) g_Y(x) g Y ( x ) when the true distribution is f X ( x ) f_X(x) f X ( x ) . Let X X X and Y Y Y be two continuous non-negative random variables with probability density functions f X ( x ) f_X(x) f X ( x ) and g Y ( x ) g_Y(x) g Y ( x ) , respectively. Then the Kullback–Leibler (KL) divergence is defined as,
d K L ( X , Y ) = ∫ 0 ∞ f ( x ) log f ( x ) g ( x ) d x = E X [ log ( f ( X ) g ( X ) ) ] . (4) d_{KL}(X,Y)=\int_{0}^{\infty} f(x)\log\frac{f(x)}{g(x)}\,dx=E_X\left[\log\left(\frac{f(X)}{g(X)}\right)\right]. \tag*{(4)} d K L ( X , Y ) = ∫ 0 ∞ f ( x ) log g ( x ) f ( x ) d x = E X [ log ( g ( X ) f ( X ) ) ] . (4)
Ebrahimi et al. [8 ] developed the bivariate case of Kullback–Leibler divergence and studied its properties. Let X = ( X 1 , X 2 ) \mathbf{X}=(X_1,X_2) X = ( X 1 , X 2 ) and Y = ( Y 1 , Y 2 ) \mathbf{Y}=(Y_1,Y_2) Y = ( Y 1 , Y 2 ) be two non-negative bivariate random vectors with absolutely continuous joint pdfs f X ( x 1 , x 2 ) f_X(x_1,x_2) f X ( x 1 , x 2 ) and g Y ( x 1 , x 2 ) g_Y(x_1,x_2) g Y ( x 1 , x 2 ) and the corresponding cdfs F X ( x 1 , x 2 ) F_X(x_1,x_2) F X ( x 1 , x 2 ) and G Y ( x 1 , x 2 ) G_Y(x_1,x_2) G Y ( x 1 , x 2 ) respectively. Then the bivariate KL divergence is defined as,
d K L ( F X , G Y ) = ∫ 0 ∞ ∫ 0 ∞ f X ( x 1 , x 2 ) log ( f X ( x 1 , x 2 ) g Y ( x 1 , x 2 ) ) d x 1 d x 2 = E X [ log ( f X ( X 1 , X 2 ) g Y ( X 1 , X 2 ) ) ] . (5) d_{KL}(F_X,G_Y)=\int_{0}^{\infty}\int_{0}^{\infty}f_X(x_1,x_2)\log\left(\frac{f_X(x_1,x_2)}{g_Y(x_1,x_2)}\right)\,dx_1\,dx_2
=\mathit{E}_X\left[\log\left(\frac{f_X(X_1,X_2)}{g_Y(X_1,X_2)}\right)\right]. \tag*{(5)} d K L ( F X , G Y ) = ∫ 0 ∞ ∫ 0 ∞ f X ( x 1 , x 2 ) log ( g Y ( x 1 , x 2 ) f X ( x 1 , x 2 ) ) d x 1 d x 2 = E X [ log ( g Y ( X 1 , X 2 ) f X ( X 1 , X 2 ) ) ] . (5)
For more information one can refer Contreras-Reyes and Arellano-Valle [5 ] , Zhang et al. [18 ] , Rafflesia Chackochan et al. [13 ] . Several generalizations of the KL divergence exist in the literature. Among them, the Rényi and Tsallis divergence measures of order α \alpha α are the most studied. The Rényi divergence of order α \alpha α between two continuous bivariate distributions F X F_X F X and G Y G_Y G Y is defined as
d R α ( F X , G Y ) = 1 α − 1 [ log ∫ 0 ∞ ∫ 0 ∞ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 ] = 1 α − 1 [ log E X ( f X ( X 1 , X 2 ) g Y ( X 1 , X 2 ) ) α − 1 ] , 0 < α ≠ 1 < ∞ . (6) d_R^\alpha(F_X,G_Y)=\frac{1}{\alpha-1}\left[\log\int_{0}^{\infty}\int_{0}^{\infty}f_X^\alpha(x_1,x_2)g_Y^{1-\alpha}(x_1,x_2)\,dx_1\,dx_2\right]
=\frac{1}{\alpha-1}\left[\log E_X\left(\frac{f_X(X_1,X_2)}{g_Y(X_1,X_2)}\right)^{\alpha-1}\right],\qquad0<\alpha\ne1<\infty. \tag*{(6)} d R α ( F X , G Y ) = α − 1 1 [ log ∫ 0 ∞ ∫ 0 ∞ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 ] = α − 1 1 [ log E X ( g Y ( X 1 , X 2 ) f X ( X 1 , X 2 ) ) α − 1 ] , 0 < α = 1 < ∞. (6)
Similarly, the Tsallis divergence of order α \alpha α is defined as
d T α ( F X , G Y ) = 1 α − 1 [ ∫ 0 ∞ ∫ 0 ∞ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 − 1 ] = 1 α − 1 [ E X ( f X ( X 1 , X 2 ) g Y ( X 1 , X 2 ) ) α − 1 − 1 ] , 0 < α ≠ 1 < ∞ . (7) d_T^\alpha(F_X,G_Y)=\frac{1}{\alpha-1}\left[\int_{0}^{\infty}\int_{0}^{\infty}f_X^\alpha(x_1,x_2)g_Y^{1-\alpha}(x_1,x_2)\,dx_1\,dx_2-1\right]
=\frac{1}{\alpha-1}\left[E_X\left(\frac{f_X(X_1,X_2)}{g_Y(X_1,X_2)}\right)^{\alpha-1}-1\right],\qquad0<\alpha\ne1<\infty. \tag*{(7)} d T α ( F X , G Y ) = α − 1 1 [ ∫ 0 ∞ ∫ 0 ∞ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 − 1 ] = α − 1 1 [ E X ( g Y ( X 1 , X 2 ) f X ( X 1 , X 2 ) ) α − 1 − 1 ] , 0 < α = 1 < ∞. (7)
It can be shown that, as α → 1 \alpha\to1 α → 1 , both the Rényi and Tsallis divergence measures converge to the Kullback–Leibler divergence given in (5) . Furthermore, when α = 0 \alpha=0 α = 0 , both divergence measures reduce to zero, that is, d R α ( F X , G Y ) = d T α ( F X , G Y ) = 0 d_R^\alpha(F_X,G_Y)=d_T^\alpha(F_X,G_Y)=0 d R α ( F X , G Y ) = d T α ( F X , G Y ) = 0 . By combining the concepts of Rényi entropy and Tsallis entropy, Wondie and Kumar [17 ] introduced a new measure called the Rényi–Tsallis entropy of order α \alpha α . This generalized entropy includes the Shannon entropy as a special case when the parameter α \alpha α approaches 1. Inspired by this, Khammar and Noughabi [10 ] introduced the differential Rényi–Tsallis divergence. This new measure combines ideas from Rényi and Tsallis divergences. Let X X X and Y Y Y be two continuous random variables with distribution functions F X ( x ) F_X(x) F X ( x ) and G Y ( x ) G_Y(x) G Y ( x ) , and corresponding probability density functions f X ( x ) f_X(x) f X ( x ) and g Y ( x ) g_Y(x) g Y ( x ) , respectively. The Rényi–Tsallis divergence between F X F_X F X and G Y G_Y G Y is defined as,
d R − T α ( F X , G Y ) = { 1 α − α − 1 [ log ∫ 0 ∞ f X α ( x ) g Y 1 − α ( x ) d x + ∫ 0 ∞ f X α ( x ) g Y 1 − α ( x ) d x − 1 ] , 0 < α ≠ 1 < ∞ , ∫ 0 ∞ f X ( x ) log ( f X ( x ) g Y ( x ) ) d x , α = 1. (8) d_{R-T}^\alpha(F_X,G_Y)=
\begin{cases}
\frac{1}{\alpha-\alpha^{-1}}\left[\log\int_{0}^{\infty}f_X^\alpha(x)g_Y^{1-\alpha}(x)\,dx+\int_{0}^{\infty}f_X^\alpha(x)g_Y^{1-\alpha}(x)\,dx-1\right], & 0<\alpha\ne1<\infty,\\
\int_{0}^{\infty}f_X(x)\log\left(\frac{f_X(x)}{g_Y(x)}\right)\,dx, & \alpha=1.
\end{cases} \tag*{(8)} d R − T α ( F X , G Y ) = { α − α − 1 1 [ log ∫ 0 ∞ f X α ( x ) g Y 1 − α ( x ) d x + ∫ 0 ∞ f X α ( x ) g Y 1 − α ( x ) d x − 1 ] , ∫ 0 ∞ f X ( x ) log ( g Y ( x ) f X ( x ) ) d x , 0 < α = 1 < ∞ , α = 1. (8)
d R − T α ( F X , G Y ) = { 1 α − α − 1 [ log [ E X ( g Y ( X ) f X ( X ) ) 1 − α ] + E X [ ( g Y ( X ) f X ( X ) ) 1 − α − 1 ] ] , 0 < α ≠ 1 < ∞ , E X [ log ( f X ( X ) g Y ( X ) ) ] , α = 1. d_{R-T}^{\alpha}(F_X,G_Y)=
\begin{cases}
\frac{1}{\alpha-\alpha^{-1}}\left[\log\left[E_X\left(\frac{g_Y(X)}{f_X(X)}\right)^{1-\alpha}\right]+E_X\left[\left(\frac{g_Y(X)}{f_X(X)}\right)^{1-\alpha}-1\right]\right], & 0<\alpha\ne1<\infty,\\
E_X\left[\log\left(\frac{f_X(X)}{g_Y(X)}\right)\right], & \alpha=1.
\end{cases} d R − T α ( F X , G Y ) = ⎩ ⎨ ⎧ α − α − 1 1 [ log [ E X ( f X ( X ) g Y ( X ) ) 1 − α ] + E X [ ( f X ( X ) g Y ( X ) ) 1 − α − 1 ] ] , E X [ log ( g Y ( X ) f X ( X ) ) ] , 0 < α = 1 < ∞ , α = 1.
Its major properties were studied and also applied to exponential family distributions and order statistics.
Although Rényi and Tsallis divergences have been extensively studied in multivariate settings, the differential Rényi–Tsallis divergence has received comparatively limited attention beyond the univariate framework. In particular, its bivariate properties, behavior for bivariate order statistics, and nonparametric estimation has not been developed in a unified framework. This motivates the present study.
Rest of this article is organized as follows. In Section 2 we introduce a bivariate differential Rényi–Tsallis divergence and discuss its fundamental properties, including non-negativity, pseudo-additivity under independence, and invariance under monotone and nonsingular linear transformations. We also obtain some bounds for the bivariate Rényi–Tsallis divergence measure. In Section 3 we develop Rényi–Tsallis divergence for bivariate order statistics. In Section 4, we study the non parametric estimation of the proposed measure and apply it to a real life dataset. The major conclusions are discussed in Section 5.
Bivariate Rényi–Tsallis Divergence Measure
Let X = ( X 1 , X 2 ) \mathbf{X}=(X_1,X_2) X = ( X 1 , X 2 ) and Y = ( Y 1 , Y 2 ) \mathbf{Y}=(Y_1,Y_2) Y = ( Y 1 , Y 2 ) be two absolutely continuous non-negative random vectors having cumulative distribution functions F X ( x 1 , x 2 ) F_X(x_1,x_2) F X ( x 1 , x 2 ) and G Y ( x 1 , x 2 ) G_Y(x_1,x_2) G Y ( x 1 , x 2 ) and joint probability density functions f X ( x 1 , x 2 ) f_X(x_1,x_2) f X ( x 1 , x 2 ) and g Y ( x 1 , x 2 ) g_Y(x_1,x_2) g Y ( x 1 , x 2 ) , respectively. For an order parameter α > 0 \alpha>0 α > 0 , α ≠ 1 \alpha\ne1 α = 1 , define
I α = ∫ 0 ∞ ∫ 0 ∞ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 . I_{\alpha}=\int_{0}^{\infty}\int_{0}^{\infty}f_X^{\alpha}(x_1,x_2)g_Y^{1-\alpha}(x_1,x_2)\,\mathrm{d}x_1\,\mathrm{d}x_2. I α = ∫ 0 ∞ ∫ 0 ∞ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 .
Whenever the integral is well-defined, the bivariate differential Rényi–Tsallis divergence of order α \alpha α is given by,
d R − T α ( F X , G Y ) = 1 α − α − 1 [ log I α + I α − 1 ] , α > 0 , α ≠ 1 , (9) d_{R-T}^{\alpha}(F_X,G_Y)=\frac{1}{\alpha-\alpha^{-1}}\left[\log I_{\alpha}+I_{\alpha}-1\right],\quad\alpha>0,\ \alpha\ne1, \tag*{(9)} d R − T α ( F X , G Y ) = α − α − 1 1 [ log I α + I α − 1 ] , α > 0 , α = 1 , (9)
At α = 1 \alpha=1 α = 1 , the divergence is defined as the Kullback–Leibler divergence:
d R − T α ( F X , G Y ) = ∫ 0 ∞ ∫ 0 ∞ f X ( x 1 , x 2 ) log ( f X ( x 1 , x 2 ) g Y ( x 1 , x 2 ) ) d x 1 d x 2 , α = 1. (10) d_{R-T}^{\alpha}(F_X,G_Y)=\int_{0}^{\infty}\int_{0}^{\infty}f_X(x_1,x_2)\log\left(\frac{f_X(x_1,x_2)}{g_Y(x_1,x_2)}\right)\,\mathrm{d}x_1\,\mathrm{d}x_2,\quad\alpha=1. \tag*{(10)} d R − T α ( F X , G Y ) = ∫ 0 ∞ ∫ 0 ∞ f X ( x 1 , x 2 ) log ( g Y ( x 1 , x 2 ) f X ( x 1 , x 2 ) ) d x 1 d x 2 , α = 1. (10)
provided the suitable support and integrability conditions are satisfied. The same divergence can be written in expectation form.
d R − T α ( F X , G Y ) = { 1 α − α − 1 [ log E X ( g Y ( X 1 , X 2 ) f X ( X 1 , X 2 ) ) 1 − α + E X ( g Y ( X 1 , X 2 ) f X ( X 1 , X 2 ) ) 1 − α − 1 ] , 0 < α ≠ 1 < ∞ , E X [ log ( f X ( X 1 , X 2 ) g Y ( X 1 , X 2 ) ) ] , α = 1. (11) d_{R-T}^{\alpha}(F_X,G_Y)=
\begin{cases}
\frac{1}{\alpha-\alpha^{-1}}\left[\log E_X\left(\frac{g_Y(X_1,X_2)}{f_X(X_1,X_2)}\right)^{1-\alpha}+E_X\left(\frac{g_Y(X_1,X_2)}{f_X(X_1,X_2)}\right)^{1-\alpha}-1\right], & 0<\alpha\ne1<\infty,\\
E_X\left[\log\left(\frac{f_X(X_1,X_2)}{g_Y(X_1,X_2)}\right)\right], & \alpha=1.
\end{cases} \tag*{(11)} d R − T α ( F X , G Y ) = ⎩ ⎨ ⎧ α − α − 1 1 [ log E X ( f X ( X 1 , X 2 ) g Y ( X 1 , X 2 ) ) 1 − α + E X ( f X ( X 1 , X 2 ) g Y ( X 1 , X 2 ) ) 1 − α − 1 ] , E X [ log ( g Y ( X 1 , X 2 ) f X ( X 1 , X 2 ) ) ] , 0 < α = 1 < ∞ , α = 1. (11)
where E X E_X E X denotes the expectation with respect to f X f_X f X . The proposed formulation is a mixture of the logarithmic and linear components of the power-integral functional. This establishes its connection between the Rényi and Tsallis divergences. Using (6) , (7) , and (9) , the differential Rényi–Tsallis divergence can be represented as
d R − T α ( F X , G Y ) = α α + 1 [ d R α ( F X , G Y ) + d T α ( F X , G Y ) ] . (12) d^\alpha_{R-T}(F_X,G_Y)=\frac{\alpha}{\alpha+1}\left[d^\alpha_R(F_X,G_Y)+d^\alpha_T(F_X,G_Y)\right]. \tag*{(12)} d R − T α ( F X , G Y ) = α + 1 α [ d R α ( F X , G Y ) + d T α ( F X , G Y ) ] . (12)
As α → 1 \alpha\to1 α → 1 , equation (12) reduces to the measure given in (5) . Moreover, (12) indicates that the behavior of the divergence measure d R − T α ( F X , G Y ) d^\alpha_{R-T}(F_X,G_Y) d R − T α ( F X , G Y ) is similar to that of d R α ( F X , G Y ) d^\alpha_R(F_X,G_Y) d R α ( F X , G Y ) and d T α ( F X , G Y ) d^\alpha_T(F_X,G_Y) d T α ( F X , G Y ) with respect to changes in α \alpha α .
Remark 2.1. Let I α ( F X , G Y ) = ∫ 0 ∞ ∫ 0 ∞ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 I_\alpha(F_X,G_Y)=\int_0^\infty\int_0^\infty f_X^\alpha(x_1,x_2)g_Y^{1-\alpha}(x_1,x_2)\,dx_1\,dx_2 I α ( F X , G Y ) = ∫ 0 ∞ ∫ 0 ∞ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 . Now by interchanging F X F_X F X and G Y G_Y G Y we get I α ( G Y , F X ) = ∫ 0 ∞ ∫ 0 ∞ g Y α ( x 1 , x 2 ) f X 1 − α ( x 1 , x 2 ) d x 1 d x 2 I_\alpha(G_Y,F_X)=\int_0^\infty\int_0^\infty g_Y^\alpha(x_1,x_2)f_X^{1-\alpha}(x_1,x_2)\,dx_1\,dx_2 I α ( G Y , F X ) = ∫ 0 ∞ ∫ 0 ∞ g Y α ( x 1 , x 2 ) f X 1 − α ( x 1 , x 2 ) d x 1 d x 2 . In general these two integrals are not equal. Hence,
d R − T α ( F X , G Y ) ≠ d R − T α ( G Y , F X ) . d^\alpha_{R-T}(F_X,G_Y)\ne d^\alpha_{R-T}(G_Y,F_X). d R − T α ( F X , G Y ) = d R − T α ( G Y , F X ) .
That is, the bivariate Rényi–Tsallis divergence is generally asymmetric. When α = 1 2 \alpha=\frac{1}{2} α = 2 1 , we have
I 1 2 ( F X , G Y ) = ∫ 0 ∞ ∫ 0 ∞ f X 1 2 ( x 1 , x 2 ) g Y 1 2 ( x 1 , x 2 ) d x 1 d x 2 = I 1 2 ( G Y , F X ) . I_{\frac{1}{2}}(F_X,G_Y)=\int_0^\infty\int_0^\infty f_X^{\frac{1}{2}}(x_1,x_2)g_Y^{\frac{1}{2}}(x_1,x_2)\,dx_1\,dx_2=I_{\frac{1}{2}}(G_Y,F_X). I 2 1 ( F X , G Y ) = ∫ 0 ∞ ∫ 0 ∞ f X 2 1 ( x 1 , x 2 ) g Y 2 1 ( x 1 , x 2 ) d x 1 d x 2 = I 2 1 ( G Y , F X ) .
Hence, the bivariate Rényi–Tsallis divergence becomes symmetric at α = 1 2 \alpha=\frac{1}{2} α = 2 1 since the underlying integral reduces to the Bhattacharyya coefficient, which is invariant under the interchange of the two distributions.
Marginal–conditional representation
Let X = ( X 1 , X 2 ) X=(X_1,X_2) X = ( X 1 , X 2 ) and Y = ( Y 1 , Y 2 ) Y=(Y_1,Y_2) Y = ( Y 1 , Y 2 ) be absolutely continuous non-negative bivariate random vectors with joint pdfs f X ( x 1 , x 2 ) = f X 1 ( x 1 ) f X 2 ∣ X 1 ( x 2 ∣ x 1 ) f_X(x_1,x_2)=f_{X_1}(x_1)f_{X_2\mid X_1}(x_2\mid x_1) f X ( x 1 , x 2 ) = f X 1 ( x 1 ) f X 2 ∣ X 1 ( x 2 ∣ x 1 ) and g Y ( x 1 , x 2 ) = g Y 1 ( x 1 ) g Y 2 ∣ Y 1 ( x 2 ∣ x 1 ) g_Y(x_1,x_2)=g_{Y_1}(x_1)g_{Y_2\mid Y_1}(x_2\mid x_1) g Y ( x 1 , x 2 ) = g Y 1 ( x 1 ) g Y 2 ∣ Y 1 ( x 2 ∣ x 1 ) . Using these representations, the integral in the definition of the bivariate Rényi–Tsallis divergence can be written as
I α ( F X , G Y ) = ∫ 0 ∞ ∫ 0 ∞ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 = ∫ 0 ∞ f X 1 α ( x 1 ) g Y 1 1 − α ( x 1 ) I α ( x 1 ) d x 1 . (13) \begin{aligned}
I_\alpha(F_X,G_Y)&=\int_0^\infty\int_0^\infty f_X^\alpha(x_1,x_2)g_Y^{1-\alpha}(x_1,x_2)\,dx_1\,dx_2\\
&=\int_0^\infty f_{X_1}^\alpha(x_1)g_{Y_1}^{1-\alpha}(x_1)I_\alpha(x_1)\,dx_1. \tag*{(13)}
\end{aligned} I α ( F X , G Y ) = ∫ 0 ∞ ∫ 0 ∞ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 = ∫ 0 ∞ f X 1 α ( x 1 ) g Y 1 1 − α ( x 1 ) I α ( x 1 ) d x 1 . (13)
where
I α ( x 1 ) = ∫ 0 ∞ f X 2 ∣ X 1 α ( x 2 ∣ x 1 ) g Y 2 ∣ Y 1 1 − α ( x 2 ∣ x 1 ) d x 2 . I_\alpha(x_1)=\int_0^\infty f_{X_2\mid X_1}^\alpha(x_2\mid x_1)g_{Y_2\mid Y_1}^{1-\alpha}(x_2\mid x_1)\,dx_2. I α ( x 1 ) = ∫ 0 ∞ f X 2 ∣ X 1 α ( x 2 ∣ x 1 ) g Y 2 ∣ Y 1 1 − α ( x 2 ∣ x 1 ) d x 2 .
Then, the bivariate Rényi–Tsallis divergence can be expressed as
d R − T α ( F X , G Y ) = 1 α − α − 1 [ log I α ( F X , G Y ) + I α ( F X , G Y ) − 1 ] . (14) d^\alpha_{R-T}(F_X,G_Y)=\frac{1}{\alpha-\alpha^{-1}}\left[\log I_\alpha(F_X,G_Y)+I_\alpha(F_X,G_Y)-1\right]. \tag*{(14)} d R − T α ( F X , G Y ) = α − α − 1 1 [ log I α ( F X , G Y ) + I α ( F X , G Y ) − 1 ] . (14)
where I α ( F X , G Y ) I_\alpha(F_X,G_Y) I α ( F X , G Y ) is given by (13) . This representation separates the contribution of the marginal densities of X 1 X_1 X 1 and Y 1 Y_1 Y 1 from the integral containing the conditional densities of X 2 X_2 X 2 given X 1 X_1 X 1 and Y 2 Y_2 Y 2 given Y 1 Y_1 Y 1 . It thus provides a convenient way to study how the bivariate divergence depends on marginal and conditional distributional information. However, it does not imply an additive decomposition of the divergence into marginal and conditional parts, in general.
Example 2.2. Let X = ( X 1 , X 2 ) X=(X_1,X_2) X = ( X 1 , X 2 ) and Y = ( Y 1 , Y 2 ) Y=(Y_1,Y_2) Y = ( Y 1 , Y 2 ) follow bivariate Pareto Type–II distributions with probability density functions f X ( x 1 , x 2 ) = θ 1 ( θ 1 + 1 ) ( 1 + x 1 + x 2 ) − ( θ 1 + 2 ) f_X(x_1,x_2)=\theta_1(\theta_1+1)(1+x_1+x_2)^{-(\theta_1+2)} f X ( x 1 , x 2 ) = θ 1 ( θ 1 + 1 ) ( 1 + x 1 + x 2 ) − ( θ 1 + 2 ) , and g Y ( x 1 , x 2 ) = g_Y(x_1,x_2)= g Y ( x 1 , x 2 ) = θ 2 ( θ 2 + 1 ) ( 1 + x 1 + x 2 ) − ( θ 2 + 2 ) \theta_2(\theta_2+1)(1+x_1+x_2)^{-(\theta_2+2)} θ 2 ( θ 2 + 1 ) ( 1 + x 1 + x 2 ) − ( θ 2 + 2 ) , where x 1 , x 2 > 0 x_1,x_2>0 x 1 , x 2 > 0 , θ 1 > 0 \theta_1>0 θ 1 > 0 , and θ 2 > 0 \theta_2>0 θ 2 > 0 . The Rényi–Tsallis divergence measure between F X F_X F X and G Y G_Y G Y is given by
d R − T α ( F X , G Y ) = { 1 α − α − 1 [ log I α + ( I α − 1 ) ] , α > 0 , α ≠ 1 , ( θ 2 − θ 1 ) ( 1 + 2 θ 1 ) θ 1 ( 1 + θ 1 ) + log ( θ 1 ( 1 + θ 1 ) θ 2 ( 1 + θ 2 ) ) , α = 1. (15) d^\alpha_{R-T}(F_X,G_Y)=
\begin{cases}
\frac{1}{\alpha-\alpha^{-1}}\left[\log I_\alpha+(I_\alpha-1)\right], & \alpha>0,\ \alpha\ne1,\\[6pt]
\frac{(\theta_2-\theta_1)(1+2\theta_1)}{\theta_1(1+\theta_1)}+\log\left(\frac{\theta_1(1+\theta_1)}{\theta_2(1+\theta_2)}\right), & \alpha=1.
\end{cases} \tag*{(15)} d R − T α ( F X , G Y ) = ⎩ ⎨ ⎧ α − α − 1 1 [ log I α + ( I α − 1 ) ] , θ 1 ( 1 + θ 1 ) ( θ 2 − θ 1 ) ( 1 + 2 θ 1 ) + log ( θ 2 ( 1 + θ 2 ) θ 1 ( 1 + θ 1 ) ) , α > 0 , α = 1 , α = 1. (15)
where
I α = [ θ 1 ( θ 1 + 1 ) ] α [ θ 2 ( θ 2 + 1 ) ] 1 − α ( α θ 1 + ( 1 − α ) θ 2 + 1 ) ( α θ 1 + ( 1 − α ) θ 2 ) . I_\alpha=\frac{[\theta_1(\theta_1+1)]^\alpha[\theta_2(\theta_2+1)]^{1-\alpha}}{(\alpha\theta_1+(1-\alpha)\theta_2+1)(\alpha\theta_1+(1-\alpha)\theta_2)}. I α = ( α θ 1 + ( 1 − α ) θ 2 + 1 ) ( α θ 1 + ( 1 − α ) θ 2 ) [ θ 1 ( θ 1 + 1 ) ] α [ θ 2 ( θ 2 + 1 ) ] 1 − α .
For α = 0.5 \alpha=0.5 α = 0.5 , the values of the bivariate Rényi–Tsallis divergence for different values of θ 1 \theta_1 θ 1 and θ 2 \theta_2 θ 2 are presented in Table 1 .
Table 1. Bivariate Rényi–Tsallis divergence values for different values of θ 1 \theta_1 θ 1 and θ 2 \theta_2 θ 2
Remark 2.3. Let X = ( X 1 , X 2 ) X=(X_1,X_2) X = ( X 1 , X 2 ) , Y = ( Y 1 , Y 2 ) Y=(Y_1,Y_2) Y = ( Y 1 , Y 2 ) , and Z = ( Z 1 , Z 2 ) Z=(Z_1,Z_2) Z = ( Z 1 , Z 2 ) be Pareto Type–II random vectors with parameters θ 1 = 1 \theta_1=1 θ 1 = 1 , θ 2 = 5 \theta_2=5 θ 2 = 5 , θ 3 = 10 \theta_3=10 θ 3 = 10 respectively. To verify the triangular inequality property of the Rényi–Tsallis divergence, consider
d R − T 1 2 ( X , Y ) = 0.528 , d R − T 1 2 ( Y , Z ) = 0.135 , d R − T 1 2 ( X , Z ) = 0.976. d^{\frac{1}{2}}_{R-T}(X,Y)=0.528,\quad d^{\frac{1}{2}}_{R-T}(Y,Z)=0.135,\quad d^{\frac{1}{2}}_{R-T}(X,Z)=0.976. d R − T 2 1 ( X , Y ) = 0.528 , d R − T 2 1 ( Y , Z ) = 0.135 , d R − T 2 1 ( X , Z ) = 0.976.
d R − T 1 2 ( X , Y ) + d R − T 1 2 ( Y , Z ) = 0.528158 + 0.135348 = 0.663506. d^{\frac{1}{2}}_{R-T}(X,Y)+d^{\frac{1}{2}}_{R-T}(Y,Z)=0.528158+0.135348=0.663506. d R − T 2 1 ( X , Y ) + d R − T 2 1 ( Y , Z ) = 0.528158 + 0.135348 = 0.663506.
d R − T 1 2 ( X , Z ) = 0.976563 > 0.663506 = d R − T 1 2 ( X , Y ) + d R − T 1 2 ( Y , Z ) , d^{\frac{1}{2}}_{R-T}(X,Z)=0.976563>0.663506=d^{\frac{1}{2}}_{R-T}(X,Y)+d^{\frac{1}{2}}_{R-T}(Y,Z), d R − T 2 1 ( X , Z ) = 0.976563 > 0.663506 = d R − T 2 1 ( X , Y ) + d R − T 2 1 ( Y , Z ) ,
d R − T α ( X , Z ) ≰ d R − T α ( X , Y ) + d R − T α ( Y , Z ) . d^\alpha_{R-T}(X,Z)\not\leq d^\alpha_{R-T}(X,Y)+d^\alpha_{R-T}(Y,Z). d R − T α ( X , Z ) ≤ d R − T α ( X , Y ) + d R − T α ( Y , Z ) .
Hence, the triangular inequality is not satisfied.
Remark 2.4. As discussed earlier, in general the Rényi–Tsallis divergence is not symmetric. However, for α = 1 2 \alpha=\frac{1}{2} α = 2 1 , the divergence becomes symmetric. This property is clearly visible in Table 1 , in which the divergence value for each pair ( θ 1 , θ 2 ) (\theta_1,\theta_2) ( θ 1 , θ 2 ) is equal to that of ( θ 2 , θ 1 ) (\theta_2,\theta_1) ( θ 2 , θ 1 ) . For example,
d R − T 1 2 ( 1 , 4 ) = d R − T 1 2 ( 4 , 1 ) = 0.401 d^{\frac{1}{2}}_{R-T}(1,4)=d^{\frac{1}{2}}_{R-T}(4,1)=0.401 d R − T 2 1 ( 1 , 4 ) = d R − T 2 1 ( 4 , 1 ) = 0.401
and
d R − T 1 2 ( 2 , 7 ) = d R − T 1 2 ( 7 , 2 ) = 0.373. d^{\frac{1}{2}}_{R-T}(2,7)=d^{\frac{1}{2}}_{R-T}(7,2)=0.373. d R − T 2 1 ( 2 , 7 ) = d R − T 2 1 ( 7 , 2 ) = 0.373.
Thus, the symmetry in Table 1 is due to the choice α = 1 2 \alpha=\frac{1}{2} α = 2 1 , but the Rényi–Tsallis divergence is not symmetric in general. So, from remark 1 and 2 it is not a metric. The following theorem proves that the bivariate Rényi–Tsallis divergence is non-negative.
Non–negativity
Theorem 2.5. Let X = ( X 1 , X 2 ) \mathbf{X}=(X_{1},X_{2}) X = ( X 1 , X 2 ) and Y = ( Y 1 , Y 2 ) \mathbf{Y}=(Y_{1},Y_{2}) Y = ( Y 1 , Y 2 ) be absolutely continuous bivariate random vectors with cdfs F X ( x 1 , x 2 ) F_{X}(x_{1},x_{2}) F X ( x 1 , x 2 ) and G Y ( x 1 , x 2 ) G_{Y}(x_{1},x_{2}) G Y ( x 1 , x 2 ) , and corresponding joint pdfs f X ( x 1 , x 2 ) f_{X}(x_{1},x_{2}) f X ( x 1 , x 2 ) and g Y ( x 1 , x 2 ) g_{Y}(x_{1},x_{2}) g Y ( x 1 , x 2 ) , respectively. Then d R − T α ( F X , G Y ) ≥ 0 d^{\alpha}_{R-T}(F_{X},G_{Y})\geq0 d R − T α ( F X , G Y ) ≥ 0 , for 0 < α ≠ 1 < ∞ 0<\alpha\neq1<\infty 0 < α = 1 < ∞ .
Proof. Let
I α ( F X , G Y ) = ∫ 0 ∞ ∫ 0 ∞ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 . I_{\alpha}(F_{X},G_{Y})=\int_{0}^{\infty}\int_{0}^{\infty}f_{X}^{\alpha}(x_{1},x_{2})g_{Y}^{1-\alpha}(x_{1},x_{2})\,dx_{1}\,dx_{2}. I α ( F X , G Y ) = ∫ 0 ∞ ∫ 0 ∞ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 .
The Rényi–Tsallis divergence is defined as
d R − T α ( F X , G Y ) = 1 α − α − 1 [ log I α + I α − 1 ] . d^{\alpha}_{R-T}(F_{X},G_{Y})=\frac{1}{\alpha-\alpha^{-1}}\left[\log I_{\alpha}+I_{\alpha}-1\right]. d R − T α ( F X , G Y ) = α − α − 1 1 [ log I α + I α − 1 ] .
Since f X ( x 1 , x 2 ) f_{X}(x_{1},x_{2}) f X ( x 1 , x 2 ) is a probability density, we can write I α ( F X , G Y ) = ∫ 0 ∞ ∫ 0 ∞ f X ( x 1 , x 2 ) [ g Y ( x 1 , x 2 ) f X ( x 1 , x 2 ) ] 1 − α d x 1 d x 2 I_{\alpha}(F_{X},G_{Y})=\int_{0}^{\infty}\int_{0}^{\infty}f_{X}(x_{1},x_{2})\left[\frac{g_{Y}(x_{1},x_{2})}{f_{X}(x_{1},x_{2})}\right]^{1-\alpha}\,dx_{1}\,dx_{2} I α ( F X , G Y ) = ∫ 0 ∞ ∫ 0 ∞ f X ( x 1 , x 2 ) [ f X ( x 1 , x 2 ) g Y ( x 1 , x 2 ) ] 1 − α d x 1 d x 2 .
Define Z = g Y ( X 1 , X 2 ) f X ( X 1 , X 2 ) Z=\frac{g_{Y}(X_{1},X_{2})}{f_{X}(X_{1},X_{2})} Z = f X ( X 1 , X 2 ) g Y ( X 1 , X 2 ) . Then I α ( F X , G Y ) = E X [ Z 1 − α ] I_{\alpha}(F_{X},G_{Y})=E_{X}\left[Z^{1-\alpha}\right] I α ( F X , G Y ) = E X [ Z 1 − α ] . Moreover,
E X ( Z ) = ∫ 0 ∞ ∫ 0 ∞ f X ( x 1 , x 2 ) g Y ( x 1 , x 2 ) f X ( x 1 , x 2 ) d x 1 d x 2 = ∫ 0 ∞ ∫ 0 ∞ g Y ( x 1 , x 2 ) d x 1 d x 2 = 1. \begin{aligned}
E_{X}(Z)&=\int_{0}^{\infty}\int_{0}^{\infty}f_{X}(x_{1},x_{2})\frac{g_{Y}(x_{1},x_{2})}{f_{X}(x_{1},x_{2})}\,dx_{1}\,dx_{2}\\
&=\int_{0}^{\infty}\int_{0}^{\infty}g_{Y}(x_{1},x_{2})\,dx_{1}\,dx_{2}=1.
\end{aligned} E X ( Z ) = ∫ 0 ∞ ∫ 0 ∞ f X ( x 1 , x 2 ) f X ( x 1 , x 2 ) g Y ( x 1 , x 2 ) d x 1 d x 2 = ∫ 0 ∞ ∫ 0 ∞ g Y ( x 1 , x 2 ) d x 1 d x 2 = 1.
We consider two cases.
Case 1: 0 < α < 1 0<\alpha<1 0 < α < 1 . In this case, 0 < 1 − α < 1 0<1-\alpha<1 0 < 1 − α < 1 . Hence, the function ϕ ( z ) = z 1 − α \phi(z)=z^{1-\alpha} ϕ ( z ) = z 1 − α , z > 0 z>0 z > 0 , is concave. By Jensen’s inequality for multivariate case Perlman [12 ] ,
I α ( F X , G Y ) = E X [ ϕ ( Z ) ] ≤ ϕ ( E X [ Z ] ) . I_{\alpha}(F_{X},G_{Y})=E_{X}[\phi(Z)]\leq\phi(E_{X}[Z]). I α ( F X , G Y ) = E X [ ϕ ( Z )] ≤ ϕ ( E X [ Z ]) .
Since E X ( Z ) = 1 E_{X}(Z)=1 E X ( Z ) = 1 , we obtain I α ( F X , G Y ) ≤ ϕ ( 1 ) = 1 I_{\alpha}(F_{X},G_{Y})\leq\phi(1)=1 I α ( F X , G Y ) ≤ ϕ ( 1 ) = 1 . Therefore, log I α ( F X , G Y ) ≤ 0 \log I_{\alpha}(F_{X},G_{Y})\leq0 log I α ( F X , G Y ) ≤ 0 and I α ( F X , G Y ) − 1 ≤ 0 I_{\alpha}(F_{X},G_{Y})-1\leq0 I α ( F X , G Y ) − 1 ≤ 0 . Thus, log I α ( F X , G Y ) + I α − 1 ≤ 0 \log I_{\alpha}(F_{X},G_{Y})+I_{\alpha}-1\leq0 log I α ( F X , G Y ) + I α − 1 ≤ 0 . For 0 < α < 1 0<\alpha<1 0 < α < 1 , α − α − 1 < 0 \alpha-\alpha^{-1}<0 α − α − 1 < 0 , and hence 1 α − α − 1 < 0 \frac{1}{\alpha-\alpha^{-1}}<0 α − α − 1 1 < 0 . Therefore,
d R − T α ( F X , G Y ) = 1 α − α − 1 [ log I α + I α − 1 ] ≥ 0. d^{\alpha}_{R-T}(F_{X},G_{Y})=\frac{1}{\alpha-\alpha^{-1}}\left[\log I_{\alpha}+I_{\alpha}-1\right]\geq0. d R − T α ( F X , G Y ) = α − α − 1 1 [ log I α + I α − 1 ] ≥ 0.
Case 2: α > 1 \alpha>1 α > 1 . In this case, 1 − α < 0 1-\alpha<0 1 − α < 0 . Thus, ϕ ( z ) = z 1 − α \phi(z)=z^{1-\alpha} ϕ ( z ) = z 1 − α is convex on ( 0 , ∞ ) (0,\infty) ( 0 , ∞ ) . By Jensen’s inequality for multivariate case,
I α ( F X , G Y ) = E X [ ϕ ( Z ) ] ≥ ϕ ( E X [ Z ] ) . I_{\alpha}(F_{X},G_{Y})=E_{X}[\phi(Z)]\geq\phi(E_{X}[Z]). I α ( F X , G Y ) = E X [ ϕ ( Z )] ≥ ϕ ( E X [ Z ]) .
Since E X ( Z ) = 1 E_{X}(Z)=1 E X ( Z ) = 1 , I α ( F X , G Y ) ≥ ϕ ( 1 ) = 1 I_{\alpha}(F_{X},G_{Y})\geq\phi(1)=1 I α ( F X , G Y ) ≥ ϕ ( 1 ) = 1 . Consequently, log I α ( F X , G Y ) ≥ 0 \log I_{\alpha}(F_{X},G_{Y})\geq0 log I α ( F X , G Y ) ≥ 0 and I α ( F X , G Y ) − 1 ≥ 0 I_{\alpha}(F_{X},G_{Y})-1\geq0 I α ( F X , G Y ) − 1 ≥ 0 . Hence, log I α ( F X , G Y ) + I α ( F X , G Y ) − 1 ≥ 0 \log I_{\alpha}(F_{X},G_{Y})+I_{\alpha}(F_{X},G_{Y})-1\geq0 log I α ( F X , G Y ) + I α ( F X , G Y ) − 1 ≥ 0 . Also, for α > 1 \alpha>1 α > 1 , α − α − 1 > 0 \alpha-\alpha^{-1}>0 α − α − 1 > 0 , so that 1 α − α − 1 > 0 \frac{1}{\alpha-\alpha^{-1}}>0 α − α − 1 1 > 0 . Therefore,
d R − T α ( F X , G Y ) ≥ 0. d^{\alpha}_{R-T}(F_{X},G_{Y})\geq0. d R − T α ( F X , G Y ) ≥ 0.
The equality condition holds when F X = G Y F_{X}=G_{Y} F X = G Y , that is, when the two bivariate random vectors have the same distribution function almost everywhere. This completes the proof.
The behavior of these divergence measures as α \alpha α varies is given in the following lemma.
Lemma 2.6. The divergence measures d R α ( F X , G Y ) d^{\alpha}_{R}(F_{X},G_{Y}) d R α ( F X , G Y ) , d T α ( F X , G Y ) d^{\alpha}_{T}(F_{X},G_{Y}) d T α ( F X , G Y ) , and d R − T α ( F X , G Y ) d^{\alpha}_{R-T}(F_{X},G_{Y}) d R − T α ( F X , G Y ) exhibit the following behavior based on α \alpha α ,
If α = 0 \alpha=0 α = 0 , then d R α ( F X , G Y ) = 0 d^{\alpha}_{R}(F_{X},G_{Y})=0 d R α ( F X , G Y ) = 0 , d T α ( F X , G Y ) = 0 d^{\alpha}_{T}(F_{X},G_{Y})=0 d T α ( F X , G Y ) = 0 , and d R − T α ( F X , G Y ) = 0 d^{\alpha}_{R-T}(F_{X},G_{Y})=0 d R − T α ( F X , G Y ) = 0 .
If 0 < α < 1 0<\alpha<1 0 < α < 1 , then d R α ( F X , G Y ) ≥ 0 d^{\alpha}_{R}(F_{X},G_{Y})\geq0 d R α ( F X , G Y ) ≥ 0 , d T α ( F X , G Y ) ≥ 0 d^{\alpha}_{T}(F_{X},G_{Y})\geq0 d T α ( F X , G Y ) ≥ 0 , and d R − T α ( F X , G Y ) ≥ 0 d^{\alpha}_{R-T}(F_{X},G_{Y})\geq0 d R − T α ( F X , G Y ) ≥ 0 .
If α = 1 \alpha=1 α = 1 , then d R α ( F X , G Y ) = d T α ( F X , G Y ) = d R − T α ( F X , G Y ) = K L ( F X , G Y ) ≥ 0 d^{\alpha}_{R}(F_{X},G_{Y})=d^{\alpha}_{T}(F_{X},G_{Y})=d^{\alpha}_{R-T}(F_{X},G_{Y})=\mathrm{KL}(F_{X},G_{Y})\geq0 d R α ( F X , G Y ) = d T α ( F X , G Y ) = d R − T α ( F X , G Y ) = KL ( F X , G Y ) ≥ 0 .
If α > 1 \alpha>1 α > 1 , then d R α ( F X , G Y ) ≥ 0 d^{\alpha}_{R}(F_{X},G_{Y})\geq0 d R α ( F X , G Y ) ≥ 0 , d T α ( F X , G Y ) ≥ 0 d^{\alpha}_{T}(F_{X},G_{Y})\geq0 d T α ( F X , G Y ) ≥ 0 , and d R − T α ( F X , G Y ) ≥ 0 d^{\alpha}_{R-T}(F_{X},G_{Y})\geq0 d R − T α ( F X , G Y ) ≥ 0 .
Pseudo-additivity
Theorem 2.7. Let X = ( X 1 , X 2 ) \mathbf{X}=(X_{1},X_{2}) X = ( X 1 , X 2 ) and Y = ( Y 1 , Y 2 ) \mathbf{Y}=(Y_{1},Y_{2}) Y = ( Y 1 , Y 2 ) be absolutely continuous non-negative bivariate random vectors with joint pdfs f X ( x 1 , x 2 ) f_{\mathbf{X}}(x_{1},x_{2}) f X ( x 1 , x 2 ) and g Y ( x 1 , x 2 ) g_{\mathbf{Y}}(x_{1},x_{2}) g Y ( x 1 , x 2 ) , respectively. Assume that the components are independent under both distributions, that is, f X ( x 1 , x 2 ) = f X 1 ( x 1 ) f X 2 ( x 2 ) f_{\mathbf{X}}(x_{1},x_{2})=f_{X_{1}}(x_{1})f_{X_{2}}(x_{2}) f X ( x 1 , x 2 ) = f X 1 ( x 1 ) f X 2 ( x 2 ) and g Y ( x 1 , x 2 ) = g Y 1 ( x 1 ) g Y 2 ( x 2 ) g_{\mathbf{Y}}(x_{1},x_{2})=g_{Y_{1}}(x_{1})g_{Y_{2}}(x_{2}) g Y ( x 1 , x 2 ) = g Y 1 ( x 1 ) g Y 2 ( x 2 ) . For α > 0 \alpha>0 α > 0 , α ≠ 1 \alpha\ne1 α = 1 , let I α , i = ∫ 0 ∞ f X i α ( x i ) g Y i 1 − α ( x i ) d x i I_{\alpha,i}=\int_{0}^{\infty}f_{X_{i}}^{\alpha}(x_{i})g_{Y_{i}}^{1-\alpha}(x_{i})\,dx_{i} I α , i = ∫ 0 ∞ f X i α ( x i ) g Y i 1 − α ( x i ) d x i , i = 1 , 2 i=1,2 i = 1 , 2 . Then the Rényi–Tsallis divergence satisfies the pseudo-additivity relation
d R − T α ( F X , G Y ) = d R − T α ( F X 1 , G Y 1 ) + d R − T α ( F X 2 , G Y 2 ) + ( I α , 1 − 1 ) ( I α , 2 − 1 ) α − α − 1 . d_{R-T}^{\alpha}(F_{\mathbf{X}},G_{\mathbf{Y}})=d_{R-T}^{\alpha}(F_{X_{1}},G_{Y_{1}})+d_{R-T}^{\alpha}(F_{X_{2}},G_{Y_{2}})+\frac{(I_{\alpha,1}-1)(I_{\alpha,2}-1)}{\alpha-\alpha^{-1}}. d R − T α ( F X , G Y ) = d R − T α ( F X 1 , G Y 1 ) + d R − T α ( F X 2 , G Y 2 ) + α − α − 1 ( I α , 1 − 1 ) ( I α , 2 − 1 ) . Proof. Consider I α ( F X , G Y ) = ∫ 0 ∞ ∫ 0 ∞ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 I_{\alpha}(F_{\mathbf{X}},G_{\mathbf{Y}})=\int_{0}^{\infty}\int_{0}^{\infty}f_{\mathbf{X}}^{\alpha}(x_{1},x_{2})g_{\mathbf{Y}}^{1-\alpha}(x_{1},x_{2})\,dx_{1}\,dx_{2} I α ( F X , G Y ) = ∫ 0 ∞ ∫ 0 ∞ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 . As the components are independent under both the distributions,
I α ( F X , G Y ) = ∫ 0 ∞ ∫ 0 ∞ f X 1 α ( x 1 ) f X 2 α ( x 2 ) g Y 1 1 − α ( x 1 ) g Y 2 1 − α ( x 2 ) d x 1 d x 2 = [ ∫ 0 ∞ f X 1 α ( x 1 ) g Y 1 1 − α ( x 1 ) d x 1 ] [ ∫ 0 ∞ f X 2 α ( x 2 ) g Y 2 1 − α ( x 2 ) d x 2 ] . \begin{aligned}
I_{\alpha}(F_{\mathbf{X}},G_{\mathbf{Y}})&=\int_{0}^{\infty}\int_{0}^{\infty}f_{X_{1}}^{\alpha}(x_{1})f_{X_{2}}^{\alpha}(x_{2})g_{Y_{1}}^{1-\alpha}(x_{1})g_{Y_{2}}^{1-\alpha}(x_{2})\,dx_{1}\,dx_{2}\\
&=\left[\int_{0}^{\infty}f_{X_{1}}^{\alpha}(x_{1})g_{Y_{1}}^{1-\alpha}(x_{1})\,dx_{1}\right]\left[\int_{0}^{\infty}f_{X_{2}}^{\alpha}(x_{2})g_{Y_{2}}^{1-\alpha}(x_{2})\,dx_{2}\right].
\end{aligned} I α ( F X , G Y ) = ∫ 0 ∞ ∫ 0 ∞ f X 1 α ( x 1 ) f X 2 α ( x 2 ) g Y 1 1 − α ( x 1 ) g Y 2 1 − α ( x 2 ) d x 1 d x 2 = [ ∫ 0 ∞ f X 1 α ( x 1 ) g Y 1 1 − α ( x 1 ) d x 1 ] [ ∫ 0 ∞ f X 2 α ( x 2 ) g Y 2 1 − α ( x 2 ) d x 2 ] . Thus, I α ( F X , G Y ) = I α , 1 I α , 2 I_{\alpha}(F_{\mathbf{X}},G_{\mathbf{Y}})=I_{\alpha,1}I_{\alpha,2} I α ( F X , G Y ) = I α , 1 I α , 2 . By the definition of the Rényi–Tsallis divergence,
d R − T α ( F X , G Y ) = 1 α − α − 1 [ log I α ( F X , G Y ) + I α ( F X , G Y ) − 1 ] . d_{R-T}^{\alpha}(F_{\mathbf{X}},G_{\mathbf{Y}})=\frac{1}{\alpha-\alpha^{-1}}\left[\log I_{\alpha}(F_{\mathbf{X}},G_{\mathbf{Y}})+I_{\alpha}(F_{\mathbf{X}},G_{\mathbf{Y}})-1\right]. d R − T α ( F X , G Y ) = α − α − 1 1 [ log I α ( F X , G Y ) + I α ( F X , G Y ) − 1 ] . Substituting I α ( F X , G Y ) = I α , 1 I α , 2 I_{\alpha}(F_{\mathbf{X}},G_{\mathbf{Y}})=I_{\alpha,1}I_{\alpha,2} I α ( F X , G Y ) = I α , 1 I α , 2 gives
d R − T α ( F X , G Y ) = 1 α − α − 1 [ log ( I α , 1 I α , 2 ) + I α , 1 I α , 2 − 1 ] . d_{R-T}^{\alpha}(F_{\mathbf{X}},G_{\mathbf{Y}})=\frac{1}{\alpha-\alpha^{-1}}\left[\log(I_{\alpha,1}I_{\alpha,2})+I_{\alpha,1}I_{\alpha,2}-1\right]. d R − T α ( F X , G Y ) = α − α − 1 1 [ log ( I α , 1 I α , 2 ) + I α , 1 I α , 2 − 1 ] . Using log ( I α , 1 I α , 2 ) = log I α , 1 + log I α , 2 \log(I_{\alpha,1}I_{\alpha,2})=\log I_{\alpha,1}+\log I_{\alpha,2} log ( I α , 1 I α , 2 ) = log I α , 1 + log I α , 2 and I α , 1 I α , 2 − 1 = ( I α , 1 − 1 ) + ( I α , 2 − 1 ) + ( I α , 1 − 1 ) ( I α , 2 − 1 ) I_{\alpha,1}I_{\alpha,2}-1=(I_{\alpha,1}-1)+(I_{\alpha,2}-1)+(I_{\alpha,1}-1)(I_{\alpha,2}-1) I α , 1 I α , 2 − 1 = ( I α , 1 − 1 ) + ( I α , 2 − 1 ) + ( I α , 1 − 1 ) ( I α , 2 − 1 ) , we obtain
d R − T α ( F X , G Y ) = 1 α − α − 1 [ log I α , 1 + I α , 1 − 1 ] + 1 α − α − 1 [ log I α , 2 + I α , 2 − 1 ] + ( I α , 1 − 1 ) ( I α , 2 − 1 ) α − α − 1 . \begin{aligned}
d_{R-T}^{\alpha}(F_{\mathbf{X}},G_{\mathbf{Y}})&=\frac{1}{\alpha-\alpha^{-1}}\left[\log I_{\alpha,1}+I_{\alpha,1}-1\right]+\frac{1}{\alpha-\alpha^{-1}}\left[\log I_{\alpha,2}+I_{\alpha,2}-1\right]\\
&\quad+\frac{(I_{\alpha,1}-1)(I_{\alpha,2}-1)}{\alpha-\alpha^{-1}}.
\end{aligned} d R − T α ( F X , G Y ) = α − α − 1 1 [ log I α , 1 + I α , 1 − 1 ] + α − α − 1 1 [ log I α , 2 + I α , 2 − 1 ] + α − α − 1 ( I α , 1 − 1 ) ( I α , 2 − 1 ) . Hence,
d R − T α ( F X , G Y ) = d R − T α ( F X 1 , G Y 1 ) + d R − T α ( F X 2 , G Y 2 ) + ( I α , 1 − 1 ) ( I α , 2 − 1 ) α − α − 1 . d_{R-T}^{\alpha}(F_{\mathbf{X}},G_{\mathbf{Y}})=d_{R-T}^{\alpha}(F_{X_{1}},G_{Y_{1}})+d_{R-T}^{\alpha}(F_{X_{2}},G_{Y_{2}})+\frac{(I_{\alpha,1}-1)(I_{\alpha,2}-1)}{\alpha-\alpha^{-1}}. d R − T α ( F X , G Y ) = d R − T α ( F X 1 , G Y 1 ) + d R − T α ( F X 2 , G Y 2 ) + α − α − 1 ( I α , 1 − 1 ) ( I α , 2 − 1 ) . Therefore, the Rényi–Tsallis divergence satisfies a pseudo-additivity property under independence.
Example 2.8. Let X = ( X 1 , X 2 ) \mathbf{X}=(X_{1},X_{2}) X = ( X 1 , X 2 ) and Y = ( Y 1 , Y 2 ) \mathbf{Y}=(Y_{1},Y_{2}) Y = ( Y 1 , Y 2 ) follow independent bivariate exponential distributions with parameters ( λ 1 , λ 2 ) (\lambda_{1},\lambda_{2}) ( λ 1 , λ 2 ) and ( μ 1 , μ 2 ) (\mu_{1},\mu_{2}) ( μ 1 , μ 2 ) , respectively. The joint density functions are f X ( x 1 , x 2 ) = λ 1 λ 2 exp ( − λ 1 x 1 − λ 2 x 2 ) f_{\mathbf{X}}(x_{1},x_{2})=\lambda_{1}\lambda_{2}\exp(-\lambda_{1}x_{1}-\lambda_{2}x_{2}) f X ( x 1 , x 2 ) = λ 1 λ 2 exp ( − λ 1 x 1 − λ 2 x 2 ) , g Y ( x 1 , x 2 ) = μ 1 μ 2 exp ( − μ 1 x 1 − μ 2 x 2 ) g_{\mathbf{Y}}(x_{1},x_{2})=\mu_{1}\mu_{2}\exp(-\mu_{1}x_{1}-\mu_{2}x_{2}) g Y ( x 1 , x 2 ) = μ 1 μ 2 exp ( − μ 1 x 1 − μ 2 x 2 ) , for x 1 , x 2 > 0 x_{1},x_{2}>0 x 1 , x 2 > 0 . Then,
I α ( F X , G Y ) = ∫ 0 ∞ ∫ 0 ∞ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 = ∏ i = 1 2 λ i α μ i 1 − α α λ i + ( 1 − α ) μ i . I_{\alpha}(F_{\mathbf{X}},G_{\mathbf{Y}})=\int_{0}^{\infty}\int_{0}^{\infty}f_{\mathbf{X}}^{\alpha}(x_{1},x_{2})g_{\mathbf{Y}}^{1-\alpha}(x_{1},x_{2})\,dx_{1}\,dx_{2}=\prod_{i=1}^{2}\frac{\lambda_{i}^{\alpha}\mu_{i}^{1-\alpha}}{\alpha\lambda_{i}+(1-\alpha)\mu_{i}}. I α ( F X , G Y ) = ∫ 0 ∞ ∫ 0 ∞ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 = i = 1 ∏ 2 α λ i + ( 1 − α ) μ i λ i α μ i 1 − α . Substituting into (2.1), we obtain
d R − T α ( F X , G Y ) = 1 α − α − 1 { log ∏ i = 1 2 ( λ i α μ i 1 − α α λ i + ( 1 − α ) μ i ) + ∏ i = 1 2 λ i α μ i 1 − α α λ i + ( 1 − α ) μ i − 1 } . (16) d_{R-T}^{\alpha}(F_{\mathbf{X}},G_{\mathbf{Y}})=\frac{1}{\alpha-\alpha^{-1}}\left\{\log\prod_{i=1}^{2}\left(\frac{\lambda_{i}^{\alpha}\mu_{i}^{1-\alpha}}{\alpha\lambda_{i}+(1-\alpha)\mu_{i}}\right)+\prod_{i=1}^{2}\frac{\lambda_{i}^{\alpha}\mu_{i}^{1-\alpha}}{\alpha\lambda_{i}+(1-\alpha)\mu_{i}}-1\right\}. \tag*{(16)} d R − T α ( F X , G Y ) = α − α − 1 1 { log i = 1 ∏ 2 ( α λ i + ( 1 − α ) μ i λ i α μ i 1 − α ) + i = 1 ∏ 2 α λ i + ( 1 − α ) μ i λ i α μ i 1 − α − 1 } . (16)
Further, the marginal divergences are given by
d R − T α ( F X 1 , G Y 1 ) = 1 α − α − 1 [ log ( λ 1 α μ 1 1 − α α λ 1 + ( 1 − α ) μ 1 ) + λ 1 α μ 1 1 − α α λ 1 + ( 1 − α ) μ 1 − 1 ] . d^\alpha_{R-T}(F_{X_1},G_{Y_1})=\frac{1}{\alpha-\alpha^{-1}}\left[\log\left(\frac{\lambda_1^\alpha\mu_1^{1-\alpha}}{\alpha\lambda_1+(1-\alpha)\mu_1}\right)+\frac{\lambda_1^\alpha\mu_1^{1-\alpha}}{\alpha\lambda_1+(1-\alpha)\mu_1}-1\right]. d R − T α ( F X 1 , G Y 1 ) = α − α − 1 1 [ log ( α λ 1 + ( 1 − α ) μ 1 λ 1 α μ 1 1 − α ) + α λ 1 + ( 1 − α ) μ 1 λ 1 α μ 1 1 − α − 1 ] .
d R − T α ( F X 2 , G Y 2 ) = 1 α − α − 1 [ log ( λ 2 α μ 2 1 − α α λ 2 + ( 1 − α ) μ 2 ) + λ 2 α μ 2 1 − α α λ 2 + ( 1 − α ) μ 2 − 1 ] . d^\alpha_{R-T}(F_{X_2},G_{Y_2})=\frac{1}{\alpha-\alpha^{-1}}\left[\log\left(\frac{\lambda_2^\alpha\mu_2^{1-\alpha}}{\alpha\lambda_2+(1-\alpha)\mu_2}\right)+\frac{\lambda_2^\alpha\mu_2^{1-\alpha}}{\alpha\lambda_2+(1-\alpha)\mu_2}-1\right]. d R − T α ( F X 2 , G Y 2 ) = α − α − 1 1 [ log ( α λ 2 + ( 1 − α ) μ 2 λ 2 α μ 2 1 − α ) + α λ 2 + ( 1 − α ) μ 2 λ 2 α μ 2 1 − α − 1 ] .
Hence,
d R − T α ( F X , G Y ) − [ d R − T α ( F X 1 , G Y 1 ) + d R − T α ( F X 2 , G Y 2 ) ] = 1 α − α − 1 [ ∏ i = 1 2 λ i α μ i 1 − α α λ i + ( 1 − α ) μ i − λ 1 α μ 1 1 − α α λ 1 + ( 1 − α ) μ 1 − λ 2 α μ 2 1 − α α λ 2 + ( 1 − α ) μ 2 + 1 ] . (17) \begin{aligned}
d^\alpha_{R-T}(F_X,G_Y)-\left[d^\alpha_{R-T}(F_{X_1},G_{Y_1})+d^\alpha_{R-T}(F_{X_2},G_{Y_2})\right] \\
&=\frac{1}{\alpha-\alpha^{-1}}\left[\prod_{i=1}^{2}\frac{\lambda_i^\alpha\mu_i^{1-\alpha}}{\alpha\lambda_i+(1-\alpha)\mu_i}-\frac{\lambda_1^\alpha\mu_1^{1-\alpha}}{\alpha\lambda_1+(1-\alpha)\mu_1}-\frac{\lambda_2^\alpha\mu_2^{1-\alpha}}{\alpha\lambda_2+(1-\alpha)\mu_2}+1\right]. \tag*{(17)}
\end{aligned} d R − T α ( F X , G Y ) − [ d R − T α ( F X 1 , G Y 1 ) + d R − T α ( F X 2 , G Y 2 ) ] = α − α − 1 1 [ i = 1 ∏ 2 α λ i + ( 1 − α ) μ i λ i α μ i 1 − α − α λ 1 + ( 1 − α ) μ 1 λ 1 α μ 1 1 − α − α λ 2 + ( 1 − α ) μ 2 λ 2 α μ 2 1 − α + 1 ] . (17)
The expression (17) verifies the pseudo-additive structure of the Rényi–Tsallis divergence for the independent bivariate exponential distribution.
Corollary 2.9. Under the assumptions of the pseudo-additivity theorem, suppose that the component integrals I α , 1 I_{\alpha,1} I α , 1 and I α , 2 I_{\alpha,2} I α , 2 are finite and well-defined. Then
{ d R − T α ( F X , G Y ) ≤ d R − T α ( F X 1 , G Y 1 ) + d R − T α ( F X 2 , G Y 2 ) , 0 < α < 1 , d R − T α ( F X , G Y ) ≥ d R − T α ( F X 1 , G Y 1 ) + d R − T α ( F X 2 , G Y 2 ) , α > 1. \begin{cases}
d^\alpha_{R-T}(F_X,G_Y)\le d^\alpha_{R-T}(F_{X_1},G_{Y_1})+d^\alpha_{R-T}(F_{X_2},G_{Y_2}), & 0<\alpha<1,\\
d^\alpha_{R-T}(F_X,G_Y)\ge d^\alpha_{R-T}(F_{X_1},G_{Y_1})+d^\alpha_{R-T}(F_{X_2},G_{Y_2}), & \alpha>1.
\end{cases} { d R − T α ( F X , G Y ) ≤ d R − T α ( F X 1 , G Y 1 ) + d R − T α ( F X 2 , G Y 2 ) , d R − T α ( F X , G Y ) ≥ d R − T α ( F X 1 , G Y 1 ) + d R − T α ( F X 2 , G Y 2 ) , 0 < α < 1 , α > 1.
Proof. By the pseudo-additivity theorem,
d R − T α ( F X , G Y ) = d R − T α ( F X 1 , G Y 1 ) + d R − T α ( F X 2 , G Y 2 ) + C α , (18) d^\alpha_{R-T}(F_X,G_Y)=d^\alpha_{R-T}(F_{X_1},G_{Y_1})+d^\alpha_{R-T}(F_{X_2},G_{Y_2})+C_\alpha, \tag*{(18)} d R − T α ( F X , G Y ) = d R − T α ( F X 1 , G Y 1 ) + d R − T α ( F X 2 , G Y 2 ) + C α , (18)
where C α = ( I α , 1 − 1 ) ( I α , 2 − 1 ) α − α − 1 C_\alpha=\frac{(I_{\alpha,1}-1)(I_{\alpha,2}-1)}{\alpha-\alpha^{-1}} C α = α − α − 1 ( I α , 1 − 1 ) ( I α , 2 − 1 ) . For 0 < α < 1 0<\alpha<1 0 < α < 1 , the inequalities I α , i ≤ 1 I_{\alpha,i}\le1 I α , i ≤ 1 , i = 1 , 2 i=1,2 i = 1 , 2 , imply ( I α , 1 − 1 ) ( I α , 2 − 1 ) ≥ 0 (I_{\alpha,1}-1)(I_{\alpha,2}-1)\ge0 ( I α , 1 − 1 ) ( I α , 2 − 1 ) ≥ 0 . Since α − α − 1 < 0 \alpha-\alpha^{-1}<0 α − α − 1 < 0 , it follows that C α ≤ 0 C_\alpha\le0 C α ≤ 0 . For α > 1 \alpha>1 α > 1 , we have I α , i ≥ 1 I_{\alpha,i}\ge1 I α , i ≥ 1 , so ( I α , 1 − 1 ) ( I α , 2 − 1 ) ≥ 0 (I_{\alpha,1}-1)(I_{\alpha,2}-1)\ge0 ( I α , 1 − 1 ) ( I α , 2 − 1 ) ≥ 0 . Since α − α − 1 > 0 \alpha-\alpha^{-1}>0 α − α − 1 > 0 , we obtain C α ≥ 0 C_\alpha\ge0 C α ≥ 0 . Substituting into (18) gives the result.
Theorem 2.10. Let X = ( X 1 , X 2 ) X=(X_1,X_2) X = ( X 1 , X 2 ) and Y = ( Y 1 , Y 2 ) Y=(Y_1,Y_2) Y = ( Y 1 , Y 2 ) be two bivariate random vectors admitting absolutely continuous distribution functions with joint pdfs f X ( x 1 , x 2 ) f_X(x_1,x_2) f X ( x 1 , x 2 ) and g Y ( x 1 , x 2 ) g_Y(x_1,x_2) g Y ( x 1 , x 2 ) , respectively. Consider S = ( S 1 , S 2 ) S=(S_1,S_2) S = ( S 1 , S 2 ) and T = ( T 1 , T 2 ) T=(T_1,T_2) T = ( T 1 , T 2 ) such that S i = ϕ i ( X i ) S_i=\phi_i(X_i) S i = ϕ i ( X i ) and T i = ϕ i ( Y i ) T_i=\phi_i(Y_i) T i = ϕ i ( Y i ) for i = 1 , 2 i=1,2 i = 1 , 2 , where ϕ i ( ⋅ ) \phi_i(\cdot) ϕ i ( ⋅ ) are monotone and differentiable. Then
d R − T α ( F S ∗ , G T ∗ ) = d R − T α ( F X , G Y ) . d^\alpha_{R-T}(F_S^*,G_T^*)=d^\alpha_{R-T}(F_X,G_Y). d R − T α ( F S ∗ , G T ∗ ) = d R − T α ( F X , G Y ) .
Proof.
d R − T α ( F X , G Y ) = 1 α − α − 1 [ log ∬ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 + ∬ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 − 1 ] . d^\alpha_{R-T}(F_X,G_Y)=\frac{1}{\alpha-\alpha^{-1}}\left[\log\iint f_X^\alpha(x_1,x_2)g_Y^{1-\alpha}(x_1,x_2)\,dx_1\,dx_2+\iint f_X^\alpha(x_1,x_2)g_Y^{1-\alpha}(x_1,x_2)\,dx_1\,dx_2-1\right]. d R − T α ( F X , G Y ) = α − α − 1 1 [ log ∬ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 + ∬ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 − 1 ] .
Let f S ∗ ( s 1 , s 2 ) f_S^*(s_1,s_2) f S ∗ ( s 1 , s 2 ) and g T ∗ ( s 1 , s 2 ) g_T^*(s_1,s_2) g T ∗ ( s 1 , s 2 ) denote the joint pdfs of S S S and T T T , respectively. Then f S ∗ ( s 1 , s 2 ) = f X ( x 1 , x 2 ) ∣ J ∣ − 1 f_S^*(s_1,s_2)=f_X(x_1,x_2)|J|^{-1} f S ∗ ( s 1 , s 2 ) = f X ( x 1 , x 2 ) ∣ J ∣ − 1 , g T ∗ ( s 1 , s 2 ) = g Y ( x 1 , x 2 ) ∣ J ∣ − 1 g_T^*(s_1,s_2)=g_Y(x_1,x_2)|J|^{-1} g T ∗ ( s 1 , s 2 ) = g Y ( x 1 , x 2 ) ∣ J ∣ − 1 , where J J J is the Jacobian of the transformation given by J = ∣ ∂ ( s 1 , s 2 ) ∂ ( x 1 , x 2 ) ∣ = ∣ ∂ s 1 ∂ x 1 ∂ s 1 ∂ x 2 ∂ s 2 ∂ x 1 ∂ s 2 ∂ x 2 ∣ J=\left|\frac{\partial(s_1,s_2)}{\partial(x_1,x_2)}\right|=\left|\begin{matrix}\frac{\partial s_1}{\partial x_1}&\frac{\partial s_1}{\partial x_2}\\[2pt]\frac{\partial s_2}{\partial x_1}&\frac{\partial s_2}{\partial x_2}\end{matrix}\right| J = ∂ ( x 1 , x 2 ) ∂ ( s 1 , s 2 ) = ∂ x 1 ∂ s 1 ∂ x 1 ∂ s 2 ∂ x 2 ∂ s 1 ∂ x 2 ∂ s 2 . Also d s 1 d s 2 = ∣ J ∣ d x 1 d x 2 \mathrm{d}s_1\mathrm{d}s_2=\lvert J\rvert\mathrm{d}x_1\mathrm{d}x_2 d s 1 d s 2 = ∣ J ∣ d x 1 d x 2
d R − T α ( F S ∗ , G T ∗ ) = 1 α − α − 1 [ log ∬ f S ∗ α ( s 1 , s 2 ) g T ∗ 1 − α ( s 1 , s 2 ) d s 1 d s 2 + ∬ f S ∗ α ( s 1 , s 2 ) g T ∗ 1 − α ( s 1 , s 2 ) d s 1 d s 2 − 1 ] = 1 α − α − 1 [ log ∬ [ f X ( x 1 , x 2 ) ∣ J ∣ − 1 ] α [ g Y ( x 1 , x 2 ) ∣ J ∣ − 1 ] 1 − α d s 1 d s 2 + ∬ [ f X ( x 1 , x 2 ) ∣ J ∣ − 1 ] α [ g Y ( x 1 , x 2 ) ∣ J ∣ − 1 ] 1 − α d s 1 d s 2 − 1 ] = 1 α − α − 1 [ log ∬ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) ∣ J ∣ − 1 d s 1 d s 2 + ∬ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) ∣ J ∣ − 1 d s 1 d s 2 − 1 ] = 1 α − α − 1 [ log ∬ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 + ∬ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 − 1 ] = d R − T α ( F X , G Y ) . \begin{aligned}
d_{R-T}^{\alpha}(F_S^*,G_T^*)={}&\frac{1}{\alpha-\alpha^{-1}}\left[\log\iint f_S^{*\alpha}(s_1,s_2)g_T^{*1-\alpha}(s_1,s_2)\,\mathrm{d}s_1\,\mathrm{d}s_2\right.\\
&\left.+\iint f_S^{*\alpha}(s_1,s_2)g_T^{*1-\alpha}(s_1,s_2)\,\mathrm{d}s_1\,\mathrm{d}s_2-1\right]\\
={}&\frac{1}{\alpha-\alpha^{-1}}\left[\log\iint[f_X(x_1,x_2)\lvert J\rvert^{-1}]^\alpha[g_Y(x_1,x_2)\lvert J\rvert^{-1}]^{1-\alpha}\,\mathrm{d}s_1\,\mathrm{d}s_2\right.\\
&\left.+\iint[f_X(x_1,x_2)\lvert J\rvert^{-1}]^\alpha[g_Y(x_1,x_2)\lvert J\rvert^{-1}]^{1-\alpha}\,\mathrm{d}s_1\,\mathrm{d}s_2-1\right]\\
={}&\frac{1}{\alpha-\alpha^{-1}}\left[\log\iint f_X^\alpha(x_1,x_2)g_Y^{1-\alpha}(x_1,x_2)\lvert J\rvert^{-1}\,\mathrm{d}s_1\,\mathrm{d}s_2\right.\\
&\left.+\iint f_X^\alpha(x_1,x_2)g_Y^{1-\alpha}(x_1,x_2)\lvert J\rvert^{-1}\,\mathrm{d}s_1\,\mathrm{d}s_2-1\right]\\
={}&\frac{1}{\alpha-\alpha^{-1}}\left[\log\iint f_X^\alpha(x_1,x_2)g_Y^{1-\alpha}(x_1,x_2)\,\mathrm{d}x_1\,\mathrm{d}x_2\right.\\
&\left.+\iint f_X^\alpha(x_1,x_2)g_Y^{1-\alpha}(x_1,x_2)\,\mathrm{d}x_1\,\mathrm{d}x_2-1\right]=d_{R-T}^{\alpha}(F_X,G_Y).
\end{aligned} d R − T α ( F S ∗ , G T ∗ ) = = = = α − α − 1 1 [ log ∬ f S ∗ α ( s 1 , s 2 ) g T ∗ 1 − α ( s 1 , s 2 ) d s 1 d s 2 + ∬ f S ∗ α ( s 1 , s 2 ) g T ∗ 1 − α ( s 1 , s 2 ) d s 1 d s 2 − 1 ] α − α − 1 1 [ log ∬ [ f X ( x 1 , x 2 ) ∣ J ∣ − 1 ] α [ g Y ( x 1 , x 2 ) ∣ J ∣ − 1 ] 1 − α d s 1 d s 2 + ∬ [ f X ( x 1 , x 2 ) ∣ J ∣ − 1 ] α [ g Y ( x 1 , x 2 ) ∣ J ∣ − 1 ] 1 − α d s 1 d s 2 − 1 ] α − α − 1 1 [ log ∬ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) ∣ J ∣ − 1 d s 1 d s 2 + ∬ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) ∣ J ∣ − 1 d s 1 d s 2 − 1 ] α − α − 1 1 [ log ∬ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 + ∬ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 − 1 ] = d R − T α ( F X , G Y ) .
Corollary 2.11. Let X = ( X 1 , X 2 ) X=(X_1,X_2) X = ( X 1 , X 2 ) and Y = ( Y 1 , Y 2 ) Y=(Y_1,Y_2) Y = ( Y 1 , Y 2 ) be absolutely continuous, bivariate random vectors with joint pdfs f X ( x 1 , x 2 ) f_X(x_1,x_2) f X ( x 1 , x 2 ) and g Y ( x 1 , x 2 ) g_Y(x_1,x_2) g Y ( x 1 , x 2 ) , respectively. Consider S = ( S 1 , S 2 ) S=(S_1,S_2) S = ( S 1 , S 2 ) and T = ( T 1 , T 2 ) T=(T_1,T_2) T = ( T 1 , T 2 ) such that S ′ = A X ′ + b S'=AX'+b S ′ = A X ′ + b , T ′ = A Y ′ + b T'=AY'+b T ′ = A Y ′ + b , where A A A is a non-singular 2 × 2 2\times2 2 × 2 matrix and b b b is a 2 × 1 2\times1 2 × 1 constant vector. Then
d R − T α ( F S ∗ , G T ∗ ) = d R − T α ( F X , G Y ) . d_{R-T}^{\alpha}(F_S^*,G_T^*)=d_{R-T}^{\alpha}(F_X,G_Y). d R − T α ( F S ∗ , G T ∗ ) = d R − T α ( F X , G Y ) . That is the bivariate Rényi–Tsallis divergence is invariant under linear transformation.
Proof. This result follows directly from the Theorem 2.7.
Relationship with some bivariate divergence measures
Let X = ( X 1 , X 2 ) X=(X_1,X_2) X = ( X 1 , X 2 ) and Y = ( Y 1 , Y 2 ) Y=(Y_1,Y_2) Y = ( Y 1 , Y 2 ) be two non-negative absolutely continuous bivariate random vectors with cdfs F X ( x 1 , x 2 ) F_X(x_1,x_2) F X ( x 1 , x 2 ) and G Y ( y 1 , y 2 ) G_Y(y_1,y_2) G Y ( y 1 , y 2 ) and density functions f X ( x 1 , x 2 ) f_X(x_1,x_2) f X ( x 1 , x 2 ) and g Y ( x 1 , x 2 ) g_Y(x_1,x_2) g Y ( x 1 , x 2 ) , respectively. Then,
d c α ( F X , G Y ) = 1 α ( 1 − α ) [ 1 − ∬ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 ] d_c^\alpha(F_X,G_Y)=\frac{1}{\alpha(1-\alpha)}\left[1-\iint f_X^\alpha(x_1,x_2)g_Y^{1-\alpha}(x_1,x_2)\,\mathrm{d}x_1\,\mathrm{d}x_2\right] d c α ( F X , G Y ) = α ( 1 − α ) 1 [ 1 − ∬ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 ]
is known as the bivariate α \alpha α -divergence measure introduced by Cichocki et al. [3 ] . From this, we obtain
∬ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 = 1 − α ( 1 − α ) d c α ( F X , G Y ) . \iint f_X^\alpha(x_1,x_2)g_Y^{1-\alpha}(x_1,x_2)\,\mathrm{d}x_1\,\mathrm{d}x_2=1-\alpha(1-\alpha)d_c^\alpha(F_X,G_Y). ∬ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 = 1 − α ( 1 − α ) d c α ( F X , G Y ) .
Substituting this into the equation for d R − T α ( F X , G Y ) d^\alpha_{R-T}(F_X,G_Y) d R − T α ( F X , G Y ) , we get
d R − T α ( F X , G Y ) = 1 α − α − 1 { log [ 1 − α ( 1 − α ) d R − T α ( F X , G Y ) ] − α ( 1 − α ) d R − T α ( F X , G Y ) } . d^\alpha_{R-T}(F_X,G_Y)=\frac{1}{\alpha-\alpha^{-1}}\left\{\log\left[1-\alpha(1-\alpha)d^\alpha_{R-T}(F_X,G_Y)\right]-\alpha(1-\alpha)d^\alpha_{R-T}(F_X,G_Y)\right\}. d R − T α ( F X , G Y ) = α − α − 1 1 { log [ 1 − α ( 1 − α ) d R − T α ( F X , G Y ) ] − α ( 1 − α ) d R − T α ( F X , G Y ) } .
d R − T 1 / 2 ( F X , G Y ) = − 2 3 { log ∬ f X 1 / 2 ( x 1 , x 2 ) g Y 1 / 2 ( x 1 , x 2 ) d x 1 d x 2 + ∬ f X 1 / 2 ( x 1 , x 2 ) g Y 1 / 2 ( x 1 , x 2 ) d x 1 d x 2 − 1 } = − 2 3 { log ∬ f X ( x 1 , x 2 ) g Y ( x 1 , x 2 ) d x 1 d x 2 + ∬ f X ( x 1 , x 2 ) g Y ( x 1 , x 2 ) d x 1 d x 2 − 1 } . (19) \begin{aligned}
d^{1/2}_{R-T}(F_X,G_Y)
&=-\frac{2}{3}\left\{\log\iint f_X^{1/2}(x_1,x_2)g_Y^{1/2}(x_1,x_2)\,dx_1\,dx_2\right.\\
&\quad\left.+\iint f_X^{1/2}(x_1,x_2)g_Y^{1/2}(x_1,x_2)\,dx_1\,dx_2-1\right\}\\
&=-\frac{2}{3}\left\{\log\iint\sqrt{f_X(x_1,x_2)}\sqrt{g_Y(x_1,x_2)}\,dx_1\,dx_2\right.\\
&\quad\left.+\iint\sqrt{f_X(x_1,x_2)}\sqrt{g_Y(x_1,x_2)}\,dx_1\,dx_2-1\right\}. \tag*{(19)}
\end{aligned} d R − T 1/2 ( F X , G Y ) = − 3 2 { log ∬ f X 1/2 ( x 1 , x 2 ) g Y 1/2 ( x 1 , x 2 ) d x 1 d x 2 + ∬ f X 1/2 ( x 1 , x 2 ) g Y 1/2 ( x 1 , x 2 ) d x 1 d x 2 − 1 } = − 3 2 { log ∬ f X ( x 1 , x 2 ) g Y ( x 1 , x 2 ) d x 1 d x 2 + ∬ f X ( x 1 , x 2 ) g Y ( x 1 , x 2 ) d x 1 d x 2 − 1 } . (19)
The Bhattacharya divergence measure proposed by Bhattacharyya (1943) is given by
B ( F X , G Y ) = ∬ f X ( x 1 , x 2 ) g Y ( x 1 , x 2 ) d x 1 d x 2 . B(F_X,G_Y)=\iint\sqrt{f_X(x_1,x_2)}\sqrt{g_Y(x_1,x_2)}\,dx_1\,dx_2. B ( F X , G Y ) = ∬ f X ( x 1 , x 2 ) g Y ( x 1 , x 2 ) d x 1 d x 2 .
Therefore, equation (2.10) can be written as
d R − T 1 / 2 ( F X , G Y ) = − 2 3 [ log B ( F X , G Y ) + B ( F X , G Y ) − 1 ] . (20) d^{1/2}_{R-T}(F_X,G_Y)=-\frac{2}{3}\left[\log B(F_X,G_Y)+B(F_X,G_Y)-1\right]. \tag*{(20)} d R − T 1/2 ( F X , G Y ) = − 3 2 [ log B ( F X , G Y ) + B ( F X , G Y ) − 1 ] . (20)
Hellinger [9 ] developed the divergence measure known as Hellinger divergence measure, given by
H 2 ( F X , G Y ) = 1 2 ∬ ( f X ( x 1 , x 2 ) − g Y ( x 1 , x 2 ) ) 2 d x 1 d x 2 = 1 2 ∬ ( f X ( x 1 , x 2 ) − 2 f X ( x 1 , x 2 ) g Y ( x 1 , x 2 ) + g Y ( x 1 , x 2 ) ) d x 1 d x 2 = 1 2 [ 1 − 2 ∬ f X ( x 1 , x 2 ) g Y ( x 1 , x 2 ) d x 1 d x 2 + 1 ] = 1 − ∬ f X ( x 1 , x 2 ) g Y ( x 1 , x 2 ) d x 1 d x 2 . \begin{aligned}
H^2(F_X,G_Y)
&=\frac{1}{2}\iint\left(\sqrt{f_X(x_1,x_2)}-\sqrt{g_Y(x_1,x_2)}\right)^2\,dx_1\,dx_2\\
&=\frac{1}{2}\iint\left(f_X(x_1,x_2)-2\sqrt{f_X(x_1,x_2)}\sqrt{g_Y(x_1,x_2)}+g_Y(x_1,x_2)\right)\,dx_1\,dx_2\\
&=\frac{1}{2}\left[1-2\iint\sqrt{f_X(x_1,x_2)}\sqrt{g_Y(x_1,x_2)}\,dx_1\,dx_2+1\right]\\
&=1-\iint\sqrt{f_X(x_1,x_2)}\sqrt{g_Y(x_1,x_2)}\,dx_1\,dx_2.
\end{aligned} H 2 ( F X , G Y ) = 2 1 ∬ ( f X ( x 1 , x 2 ) − g Y ( x 1 , x 2 ) ) 2 d x 1 d x 2 = 2 1 ∬ ( f X ( x 1 , x 2 ) − 2 f X ( x 1 , x 2 ) g Y ( x 1 , x 2 ) + g Y ( x 1 , x 2 ) ) d x 1 d x 2 = 2 1 [ 1 − 2 ∬ f X ( x 1 , x 2 ) g Y ( x 1 , x 2 ) d x 1 d x 2 + 1 ] = 1 − ∬ f X ( x 1 , x 2 ) g Y ( x 1 , x 2 ) d x 1 d x 2 .
Therefore, B ( F X , G Y ) = 1 − H 2 ( F X , G Y ) B(F_X,G_Y)=1-H^2(F_X,G_Y) B ( F X , G Y ) = 1 − H 2 ( F X , G Y ) . Hence, equation (2.11) becomes
d R − T 1 / 2 ( F X , G Y ) = − 2 3 [ log ( 1 − H 2 ( F X , G Y ) ) − H 2 ( F X , G Y ) ] . d^{1/2}_{R-T}(F_X,G_Y)=-\frac{2}{3}\left[\log\left(1-H^2(F_X,G_Y)\right)-H^2(F_X,G_Y)\right]. d R − T 1/2 ( F X , G Y ) = − 3 2 [ log ( 1 − H 2 ( F X , G Y ) ) − H 2 ( F X , G Y ) ] .
Moreover, d R − T 1 / 2 ( F X , G Y ) = − 2 3 [ − d B ( F X , G Y ) + exp ( − d B ( F X , G Y ) ) − 1 ] d^{1/2}_{R-T}(F_X,G_Y)=-\frac{2}{3}\left[-d_B(F_X,G_Y)+\exp\left(-d_B(F_X,G_Y)\right)-1\right] d R − T 1/2 ( F X , G Y ) = − 3 2 [ − d B ( F X , G Y ) + exp ( − d B ( F X , G Y ) ) − 1 ] , where d B ( F X , G Y ) = − log B ( F X , G Y ) d_B(F_X,G_Y)=-\log B(F_X,G_Y) d B ( F X , G Y ) = − log B ( F X , G Y ) is known as the Bhattacharya distance.
In the next example, the bivariate differential Rényi–Tsallis divergence measure of independent bivariate exponential distribution is compared with some other divergence measures.
Example 2.12. Let X = ( X 1 , X 2 ) X=(X_1,X_2) X = ( X 1 , X 2 ) follow the independent bivariate exponential distribution with density f ( x 1 , x 2 ) = λ 1 λ 2 exp ( − λ 1 x 1 − λ 2 x 2 ) f(x_1,x_2)=\lambda_1\lambda_2\exp(-\lambda_1x_1-\lambda_2x_2) f ( x 1 , x 2 ) = λ 1 λ 2 exp ( − λ 1 x 1 − λ 2 x 2 ) , x 1 , x 2 ≥ 0 x_1,x_2\geq0 x 1 , x 2 ≥ 0 , λ 1 , λ 2 > 0 \lambda_1,\lambda_2>0 λ 1 , λ 2 > 0 . Similarly, let Y = ( Y 1 , Y 2 ) Y=(Y_1,Y_2) Y = ( Y 1 , Y 2 ) follow the independent bivariate exponential distribution with density g ( y 1 , y 2 ) = μ 1 μ 2 exp ( − μ 1 y 1 − μ 2 y 2 ) g(y_1,y_2)=\mu_1\mu_2\exp(-\mu_1y_1-\mu_2y_2) g ( y 1 , y 2 ) = μ 1 μ 2 exp ( − μ 1 y 1 − μ 2 y 2 ) , y 1 , y 2 ≥ 0 y_1,y_2\geq0 y 1 , y 2 ≥ 0 , μ 1 , μ 2 > 0 \mu_1,\mu_2>0 μ 1 , μ 2 > 0 . We obtain the Rényi–Tsallis divergence measure as,
d R − T α ( F X , G Y ) = { 1 α − α − 1 [ log { ∏ i = 1 2 ( λ i α μ i 1 − α α λ i + ( 1 − α ) μ i ) } + ∏ i = 1 2 ( λ i α μ i 1 − α α λ i + ( 1 − α ) μ i ) − 1 ] , α > 0 , α ≠ 1 , ∑ i = 1 2 [ log ( λ i μ i ) + μ i − λ i λ i ] , α = 1. (21) d_{R-T}^{\alpha}(F_X,G_Y)=
\begin{cases}
\frac{1}{\alpha-\alpha^{-1}}\left[\log\left\{\prod_{i=1}^{2}\left(\frac{\lambda_i^{\alpha}\mu_i^{1-\alpha}}{\alpha\lambda_i+(1-\alpha)\mu_i}\right)\right\}+\prod_{i=1}^{2}\left(\frac{\lambda_i^{\alpha}\mu_i^{1-\alpha}}{\alpha\lambda_i+(1-\alpha)\mu_i}\right)-1\right], & \alpha>0,\alpha\ne1,\\
\sum_{i=1}^{2}\left[\log\left(\frac{\lambda_i}{\mu_i}\right)+\frac{\mu_i-\lambda_i}{\lambda_i}\right], & \alpha=1.
\end{cases} \tag*{(21)} d R − T α ( F X , G Y ) = ⎩ ⎨ ⎧ α − α − 1 1 [ log { ∏ i = 1 2 ( α λ i + ( 1 − α ) μ i λ i α μ i 1 − α ) } + ∏ i = 1 2 ( α λ i + ( 1 − α ) μ i λ i α μ i 1 − α ) − 1 ] , ∑ i = 1 2 [ log ( μ i λ i ) + λ i μ i − λ i ] , α > 0 , α = 1 , α = 1. (21)
Figure 1. Plots of Rényi, Tsallis, and Rényi–Tsallis divergence measures for independent bivariate exponential distributions with α = 0.5 \alpha=0.5 α = 0.5 , λ 1 = 1 \lambda_1=1 λ 1 = 1 , λ 2 = 2 \lambda_2=2 λ 2 = 2 , μ 1 ∈ ( 0 , 3 ] \mu_1\in(0,3] μ 1 ∈ ( 0 , 3 ] , and μ 2 ∈ [ 2 , 5 ] \mu_2\in[2,5] μ 2 ∈ [ 2 , 5 ] .
Figure 1 shows the Rényi, Tsallis and Rényi–Tsallis divergence measures for independent bivariate exponential distribution with α = 0.5 \alpha=0.5 α = 0.5 , λ 1 = 1 \lambda_1=1 λ 1 = 1 , λ 2 = 2 \lambda_2=2 λ 2 = 2 , μ 1 ∈ [ 0 , 3 ] \mu_1\in[0,3] μ 1 ∈ [ 0 , 3 ] , and μ 2 ∈ [ 2 , 5 ] \mu_2\in[2,5] μ 2 ∈ [ 2 , 5 ] . All three measures increase with increasing μ 2 \mu_2 μ 2 and decrease with increasing μ 1 \mu_1 μ 1 . They take their largest values for small μ 1 \mu_1 μ 1 and large μ 2 \mu_2 μ 2 . They all exhibit the same overall monotonic trend; however, the Rényi divergence is relatively more sensitive to changes in the parameters, the Tsallis divergence changes more smoothly, and the proposed Rényi–Tsallis divergence maintains the common sensitivity pattern while providing a balanced measure of the distributional difference. And Table 2 gives the different divergence measures of independent bivariate exponential distribution. Following proposition gives some useful bound for the proposed divergence measure.
Table 2. Some divergence measures for independent bivariate exponential distributions
Example 2.13. Sensitivity of divergence measures under dependent bivariate models.
The above example shows the behaviour of the Rényi, Tsallis, and Rényi–Tsallis divergence measures when the parameters of independent bivariate exponential distributions vary. In this example, we study their sensitivity to changes in dependence under two different bivariate models: the Marshall–Olkin model and a Gumbel-type bivariate exponential model.
As a common reference distribution, consider the independent bivariate exponential distribution with unit marginal rates, whose density is q ( x 1 , x 2 ) = e − x 1 − x 2 q(x_1,x_2)=e^{-x_1-x_2} q ( x 1 , x 2 ) = e − x 1 − x 2 , x 1 , x 2 ≥ 0 x_1,x_2\ge0 x 1 , x 2 ≥ 0 . For a dependent model with density p p p , we evaluate the Rényi, Tsallis, and Rényi–Tsallis divergence measures using equations (6) , (7) and (9) .
Marshall–Olkin bivariate exponential model
Let ( X 1 , X 2 ) (X_1,X_2) ( X 1 , X 2 ) follow a Marshall–Olkin bivariate exponential distribution with survival function
F ‾ c ( x 1 , x 2 ) = exp [ − ( 1 − c ) x 1 − ( 1 − c ) x 2 − c max ( x 1 , x 2 ) ] , \overline{F}_c(x_1,x_2)=\exp\left[-(1-c)x_1-(1-c)x_2-c\max(x_1,x_2)\right], F c ( x 1 , x 2 ) = exp [ − ( 1 − c ) x 1 − ( 1 − c ) x 2 − c max ( x 1 , x 2 ) ] ,
where x 1 , x 2 ≥ 0 x_1,x_2\ge0 x 1 , x 2 ≥ 0 and 0 ≤ c < 1 0\le c<1 0 ≤ c < 1 . The marginal distributions are standard exponential for every value of c c c . The parameter c c c controls the common-shock component, and c = 0 c=0 c = 0 corresponds to independence. The absolutely continuous part of the joint distribution has density
f c ( x 1 , x 2 ) = { ( 1 − c ) e − ( 1 − c ) x 1 − x 2 , x 1 < x 2 , ( 1 − c ) e − x 1 − ( 1 − c ) x 2 , x 2 < x 1 . f_c(x_1,x_2)=
\begin{cases}
(1-c)e^{-(1-c)x_1-x_2}, & x_1<x_2,\\
(1-c)e^{-x_1-(1-c)x_2}, & x_2<x_1.
\end{cases} f c ( x 1 , x 2 ) = { ( 1 − c ) e − ( 1 − c ) x 1 − x 2 , ( 1 − c ) e − x 1 − ( 1 − c ) x 2 , x 1 < x 2 , x 2 < x 1 .
For c > 0 c>0 c > 0 , the distribution also has a singular component on the diagonal x 1 = x 2 x_1=x_2 x 1 = x 2 , with density c e − ( 2 − c ) t ce^{-(2-c)t} c e − ( 2 − c ) t with respect to the diagonal parameter t ≥ 0 t\ge0 t ≥ 0 . Its total probability mass is c / ( 2 − c ) c/(2-c) c / ( 2 − c ) .
Using a common dominating measure that accounts for both the continuous and singular components, the power integral relative to the independent reference distribution is
I α M O ( c ) = 2 ( 1 − c ) α 2 − α c , 0 < α < 1. I_{\alpha}^{MO}(c)=\frac{2(1-c)^{\alpha}}{2-\alpha c}, \qquad0<\alpha<1. I α M O ( c ) = 2 − α c 2 ( 1 − c ) α , 0 < α < 1.
Consequently, the three divergence measures can be obtained by substituting I α M O ( c ) I_{\alpha}^{MO}(c) I α M O ( c ) into their respective definitions. The diagonal component contributes no term to this power integral because the reference distribution assigns zero probability to the diagonal.
The Kendall rank correlation coefficient for this model is
τ M O ( c ) = c 2 − c . \tau_{MO}(c)=\frac{c}{2-c}. τ M O ( c ) = 2 − c c .
Writing ρ = ∣ τ ∣ \rho=|\tau| ρ = ∣ τ ∣ , the common-shock parameter can therefore be expressed as
c = 2 ρ 1 + ρ . c=\frac{2\rho}{1+\rho}. c = 1 + ρ 2 ρ .
This parameterization allows the sensitivity of the divergence measures to be evaluated with respect to the magnitude of Kendall’s tau.
Gumbel-type bivariate exponential model
For comparison, consider a Gumbel-type bivariate exponential model with survival function
G ‾ θ ( x 1 , x 2 ) = exp ( − x 1 − x 2 − θ x 1 x 2 ) , 0 ≤ θ ≤ 1. \overline{G}_{\theta}(x_1,x_2)=\exp(-x_1-x_2-\theta x_1x_2), \qquad0\le\theta\le1. G θ ( x 1 , x 2 ) = exp ( − x 1 − x 2 − θ x 1 x 2 ) , 0 ≤ θ ≤ 1.
Both marginal distributions are standard exponential, and θ = 0 \theta= 0 θ = 0 corresponds to independence. Its joint density is
g θ ( x 1 , x 2 ) = e − x 1 − x 2 − θ x 1 x 2 [ ( 1 + θ x 1 ) ( 1 + θ x 2 ) − θ ] . g_\theta(x_1,x_2) = e^{-x_1-x_2-\theta x_1x_2} \left[(1+\theta x_1)(1+\theta x_2)-\theta\right]. g θ ( x 1 , x 2 ) = e − x 1 − x 2 − θ x 1 x 2 [ ( 1 + θ x 1 ) ( 1 + θ x 2 ) − θ ] .
The power integral relative to the same independent reference distribution is
I α G ( θ ) = ∫ 0 ∞ ∫ 0 ∞ e − ( x 1 + x 2 ) − α θ x 1 x 2 × [ ( 1 + θ x 1 ) ( 1 + θ x 2 ) − θ ] α d x 2 d x 1 . I_\alpha^G(\theta) = \int_0^\infty\int_0^\infty e^{-(x_1+x_2)-\alpha\theta x_1x_2} \times\left[(1+\theta x_1)(1+\theta x_2)-\theta\right]^\alpha\, dx_2 \, dx_1. I α G ( θ ) = ∫ 0 ∞ ∫ 0 ∞ e − ( x 1 + x 2 ) − α θ x 1 x 2 × [ ( 1 + θ x 1 ) ( 1 + θ x 2 ) − θ ] α d x 2 d x 1 .
Unlike the corresponding integral for the Marshall–Olkin model, this expression does not simplify to the same closed form and is evaluated numerically using Gauss–Legendre quadrature. The Kendall rank correlation coefficient is evaluated numerically from
τ G ( θ ) = 4 ∫ 0 ∞ ∫ 0 ∞ G θ ( x 1 , x 2 ) g θ ( x 1 , x 2 ) d x 2 d x 1 − 1 , \tau_G(\theta) = 4 \int_0^\infty\int_0^\infty G_\theta(x_1,x_2)g_\theta(x_1,x_2) \, dx_2 \, dx_1 - 1, τ G ( θ ) = 4 ∫ 0 ∞ ∫ 0 ∞ G θ ( x 1 , x 2 ) g θ ( x 1 , x 2 ) d x 2 d x 1 − 1 ,
where
G θ ( x 1 , x 2 ) = 1 − e − x 1 − e − x 2 + e − x 1 − x 2 − θ x 1 x 2 G_\theta(x_1,x_2) = 1-e^{-x_1}-e^{-x_2}+e^{-x_1-x_2-\theta x_1x_2} G θ ( x 1 , x 2 ) = 1 − e − x 1 − e − x 2 + e − x 1 − x 2 − θ x 1 x 2
is the joint distribution function. In the parameter range considered, this coefficient is non-positive. We therefore compare the two models using the magnitude ρ = ∣ τ ∣ \rho= |\tau| ρ = ∣ τ ∣ , matching the value of ρ \rho ρ in the Marshall–Olkin model to the corresponding value in the Gumbel-type model.
Sensitivity with respect to dependence
Let D α D^\alpha D α denote any one of the three divergence measures. For each model, define its absolute sensitivity with respect to the magnitude of Kendall’s tau by
S D α ( ρ ) = ∣ d D α d ρ ∣ . S_D^\alpha(\rho) = \left|\frac{dD^\alpha}{d\rho}\right|. S D α ( ρ ) = d ρ d D α .
For the Marshall–Olkin model, the chain rule gives
S D , M O α ( ρ ) = ∣ ∂ D M O α ∂ c ∣ 2 ( 1 + ρ ) 2 . S_{D,MO}^\alpha(\rho) = \left|\frac{\partial D_{MO}^\alpha}{\partial c}\right|\frac{2}{(1+\rho)^2}. S D , M O α ( ρ ) = ∂ c ∂ D M O α ( 1 + ρ ) 2 2 .
For the Gumbel-type model, the corresponding sensitivity is
S D , G α ( ρ ) = ∣ d D G α / d θ d ∣ τ G ( θ ) ∣ / d θ ∣ θ = θ ( ρ ) , S_{D,G}^\alpha(\rho) = \left|\frac{dD_G^\alpha/d\theta}{d|\tau_G(\theta)|/d\theta}\right|_{\theta=\theta(\rho)}, S D , G α ( ρ ) = d ∣ τ G ( θ ) ∣/ d θ d D G α / d θ θ = θ ( ρ ) ,
where θ ( ρ ) \theta(\rho) θ ( ρ ) is obtained by numerically solving
∣ τ G ( θ ) ∣ = ρ . |\tau_G(\theta)| = \rho. ∣ τ G ( θ ) ∣ = ρ .
We evaluate these sensitivities for α = 0.25 , 0.50 , 0.75 , \alpha= 0.25, 0.50, 0.75, α = 0.25 , 0.50 , 0.75 , and 0.90 0.90 0.90 over the common range of dependence magnitudes attainable by both models. This comparison places the results on a common dependence scale instead of comparing derivatives with respect to the model’s differently scaled parameters.
Table 3 shows that the mean absolute sensitivities of all three divergence measures are higher under the Marshall–Olkin model than under the Gumbel-type model for the different values of α \alpha α . The difference increases with α \alpha α ; at α = 0.90 \alpha= 0.90 α = 0.90 , the ratios are 25.63, 21.01, and 23.33 for the Rényi, Tsallis, and Rényi–Tsallis divergences, respectively. However, the ordering of relative sensitivity is different for different models and orders, showing that sensitivity depends on both the divergence measure and the dependence structure. Marshall–Olkin model has a singular diagonal component whereas the Gumbel-type model is absolutely continuous. These differences should not be attributed only to the divergence measures. Numerical accuracy was tested through quadrature convergence, verification of the Rényi–Tsallis identity, and matching of the dependence magnitudes using ∣ τ ∣ |\tau| ∣ τ ∣ .
Table 3. Mean absolute sensitivity of the divergence measures over the matched dependence grid.
Figure 2. Absolute sensitivity of the Rényi, Tsallis, and Rényi–Tsallis divergence measures with respect to the magnitude of Kendall’s tau under the Marshall–Olkin bivariate exponential model, for α = 0.25 , 0.50 , 0.75 , \alpha= 0.25, 0.50, 0.75, α = 0.25 , 0.50 , 0.75 , and 0.90 0.90 0.90 .
Figure 3. Absolute sensitivity of the Rényi, Tsallis, and Rényi–Tsallis divergence measures with respect to the magnitude of Kendall’s tau under the Gumbel-type bivariate exponential model, for α = 0.25 , 0.50 , 0.75 , \alpha= 0.25, 0.50, 0.75, α = 0.25 , 0.50 , 0.75 , and 0.90 0.90 0.90 .
Figure 4. Comparison of the absolute sensitivity of the Rényi–Tsallis divergence under the Marshall–Olkin and Gumbel-type bivariate exponential models at matched values of the magnitude of Kendall’s tau.
Proposition 2.14. Let X = ( x 1 , x 2 ) X=(x_1,x_2) X = ( x 1 , x 2 ) and Y = ( y 1 , y 2 ) Y=(y_1,y_2) Y = ( y 1 , y 2 ) be two non-negative absolutely continuous random vectors with density functions f X ( x 1 , x 2 ) f_X(x_1,x_2) f X ( x 1 , x 2 ) and g Y ( x 1 , x 2 ) g_Y(x_1,x_2) g Y ( x 1 , x 2 ) respectively. Since for all f X ( x 1 , x 2 ) , g Y ( x 1 , x 2 ) ≥ 0 f_X(x_1,x_2),g_Y(x_1,x_2)\ge0 f X ( x 1 , x 2 ) , g Y ( x 1 , x 2 ) ≥ 0 , we have
log ( ∬ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 ) ≤ ( ∬ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 ) − 1 , α > 0. \log\left(\iint f_X^\alpha(x_1,x_2)g_Y^{1-\alpha}(x_1,x_2)\,\mathrm{d}x_1\,\mathrm{d}x_2\right)\le\left(\iint f_X^\alpha(x_1,x_2)g_Y^{1-\alpha}(x_1,x_2)\,\mathrm{d}x_1\,\mathrm{d}x_2\right)-1,\quad\alpha>0. log ( ∬ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 ) ≤ ( ∬ f X α ( x 1 , x 2 ) g Y 1 − α ( x 1 , x 2 ) d x 1 d x 2 ) − 1 , α > 0. Also, d R − T α ( F X , G Y ) = α α + 1 ( d R α + d T α ) d_{R-T}^{\alpha}(F_X,G_Y)=\frac{\alpha}{\alpha+1}(d_R^\alpha+d_T^\alpha) d R − T α ( F X , G Y ) = α + 1 α ( d R α + d T α ) . If 0 < α < 1 0<\alpha<1 0 < α < 1 , then d R α ≥ d T α d_R^\alpha\ge d_T^\alpha d R α ≥ d T α . Therefore, 2 d T α ≤ d R α + d T α ≤ 2 d R α 2d_T^\alpha\le d_R^\alpha+d_T^\alpha\le2d_R^\alpha 2 d T α ≤ d R α + d T α ≤ 2 d R α . Hence,
2 α α + 1 d T α ≤ d R − T α ≤ 2 α α + 1 d R α . \frac{2\alpha}{\alpha+1}d_T^\alpha\le d_{R-T}^\alpha\le\frac{2\alpha}{\alpha+1}d_R^\alpha. α + 1 2 α d T α ≤ d R − T α ≤ α + 1 2 α d R α . If α > 1 \alpha>1 α > 1 , then d R α ≤ d T α d_R^\alpha\le d_T^\alpha d R α ≤ d T α . Therefore,
2 α α + 1 d R α ≤ d R − T α ≤ 2 α α + 1 d T α . \frac{2\alpha}{\alpha+1}d_R^\alpha\le d_{R-T}^\alpha\le\frac{2\alpha}{\alpha+1}d_T^\alpha. α + 1 2 α d R α ≤ d R − T α ≤ α + 1 2 α d T α . In addition, Dragomir and Gluscevic [6 ] obtained the bound
d K L ( F X , G Y ) ≤ log ( d χ 2 ( F X , G Y ) + 1 ) d_{KL}(F_X,G_Y)\le\log\left(d_{\chi^2}(F_X,G_Y)+1\right) d K L ( F X , G Y ) ≤ log ( d χ 2 ( F X , G Y ) + 1 ) where d χ 2 ( F X , G Y ) d_{\chi^2}(F_X,G_Y) d χ 2 ( F X , G Y ) is known as the χ 2 \chi^2 χ 2 -distance (Ben-Tal et al. [1 ] ). Hence,
{ 2 α α + 1 [ d T α ( F X , G Y ) ] ≤ d R − T α ( F X , G Y ) ≤ 2 α α + 1 [ d R α ( F X , G Y ) ] , 0 < α < 1 , d R − T α ( F X , G Y ) ≤ log [ d χ 2 ( F X , G Y ) + 1 ] , α = 1 , 2 α α + 1 [ d R α ( F X , G Y ) ] ≤ d R − T α ( F X , G Y ) ≤ 2 α α + 1 [ d T α ( F X , G Y ) ] , α > 1. (22) \begin{cases}
\frac{2\alpha}{\alpha+1}\left[d_T^\alpha(F_X,G_Y)\right]\le d_{R-T}^\alpha(F_X,G_Y)\le\frac{2\alpha}{\alpha+1}\left[d_R^\alpha(F_X,G_Y)\right], & 0<\alpha<1,\\
d_{R-T}^\alpha(F_X,G_Y)\le\log\left[d_{\chi^2}(F_X,G_Y)+1\right], & \alpha=1,\\
\frac{2\alpha}{\alpha+1}\left[d_R^\alpha(F_X,G_Y)\right]\le d_{R-T}^\alpha(F_X,G_Y)\le\frac{2\alpha}{\alpha+1}\left[d_T^\alpha(F_X,G_Y)\right], & \alpha>1. \tag*{(22)}
\end{cases} ⎩ ⎨ ⎧ α + 1 2 α [ d T α ( F X , G Y ) ] ≤ d R − T α ( F X , G Y ) ≤ α + 1 2 α [ d R α ( F X , G Y ) ] , d R − T α ( F X , G Y ) ≤ log [ d χ 2 ( F X , G Y ) + 1 ] , α + 1 2 α [ d R α ( F X , G Y ) ] ≤ d R − T α ( F X , G Y ) ≤ α + 1 2 α [ d T α ( F X , G Y ) ] , 0 < α < 1 , α = 1 , α > 1. (22) Equation (22) shows that the bivariate Rényi–Tsallis divergence lies between scaled versions of bivariate Tsallis divergence and bivariate Rényi divergence. The scale factor is 2 α α + 1 \frac{2\alpha}{\alpha+1} α + 1 2 α which lies between 0 and 2 for all α > 0 \alpha>0 α > 0 . This bound shows how the interval containing the proposed divergence measure varies with the order parameter α \alpha α . Also for α = 1 \alpha=1 α = 1 , 2 α α + 1 \frac{2\alpha}{\alpha+1} α + 1 2 α becomes 1 and all these three divergences will be same and equal to KL divergence.
Rényi-Tsallis Divergence for Bivariate Order Statistics {#sec:3} Let X 1 , X 2 , … , X n X_1,X_2,\ldots,X_n X 1 , X 2 , … , X n and Y 1 , Y 2 , … , Y n Y_1,Y_2,\ldots,Y_n Y 1 , Y 2 , … , Y n be two independent random samples of size n n n from absolutely continuous distributions with cdfs F X F_X F X and G Y G_Y G Y , and corresponding pdfs f X f_X f X and g Y g_Y g Y ,
respectively. Let X ( i ) , X ( j ) X_{(i)}, X_{(j)} X ( i ) , X ( j ) and Y ( i ) , Y ( j ) Y_{(i)}, Y_{(j)} Y ( i ) , Y ( j ) denote the i t h i^{th} i t h and j t h j^{th} j t h order statistics (i < j i<j i < j ) from the samples associated with F X F_X F X and G Y G_Y G Y , respectively.
For 0 < α < ∞ 0<\alpha<\infty 0 < α < ∞ , α ≠ 1 \alpha\ne1 α = 1 , the Rényi–Tsallis divergence between the bivariate order statistic distributions F i j F_{ij} F ij and G i j G_{ij} G ij is defined as
d R − T α ( F X , G Y ) = 1 α − α − 1 [ log ( ∫ 0 ∞ ∫ x i ∞ f i j α ( x i , x j ) g i j 1 − α ( x i , x j ) d x j d x i ) + ∫ 0 ∞ ∫ x i ∞ f i j α ( x i , x j ) g i j 1 − α ( x i , x j ) d x j d x i − 1 ] . (23) d_{R-T}^{\alpha}(F_X,G_Y)=\frac{1}{\alpha-\alpha^{-1}}\left[\log\left(\int_0^\infty\int_{x_i}^\infty f_{ij}^{\alpha}(x_i,x_j)g_{ij}^{1-\alpha}(x_i,x_j)\,dx_j\,dx_i\right)+\int_0^\infty\int_{x_i}^\infty f_{ij}^{\alpha}(x_i,x_j)g_{ij}^{1-\alpha}(x_i,x_j)\,dx_j\,dx_i-1\right]. \tag*{(23)} d R − T α ( F X , G Y ) = α − α − 1 1 [ log ( ∫ 0 ∞ ∫ x i ∞ f ij α ( x i , x j ) g ij 1 − α ( x i , x j ) d x j d x i ) + ∫ 0 ∞ ∫ x i ∞ f ij α ( x i , x j ) g ij 1 − α ( x i , x j ) d x j d x i − 1 ] . (23)
The joint pdf of the pair ( X ( i ) , X ( j ) ) (X_{(i)},X_{(j)}) ( X ( i ) , X ( j ) ) is given by
f i j ( x i , x j ) = n ! ( i − 1 ) ! ( j − i − 1 ) ! ( n − j ) ! [ F X ( x i ) ] i − 1 [ F X ( x j ) − F X ( x i ) ] j − i − 1 [ 1 − F X ( x j ) ] n − j f X ( x i ) f X ( x j ) , x i < x j . f_{ij}(x_i,x_j)=\frac{n!}{(i-1)!(j-i-1)!(n-j)!}[F_X(x_i)]^{i-1}[F_X(x_j)-F_X(x_i)]^{j-i-1}[1-F_X(x_j)]^{n-j}f_X(x_i)f_X(x_j),\qquad x_i<x_j. f ij ( x i , x j ) = ( i − 1 )! ( j − i − 1 )! ( n − j )! n ! [ F X ( x i ) ] i − 1 [ F X ( x j ) − F X ( x i ) ] j − i − 1 [ 1 − F X ( x j ) ] n − j f X ( x i ) f X ( x j ) , x i < x j .
Similarly, the joint pdf of the pair ( Y ( i ) , Y ( j ) ) (Y_{(i)},Y_{(j)}) ( Y ( i ) , Y ( j ) ) is
g i j ( y i , y j ) = n ! ( i − 1 ) ! ( j − i − 1 ) ! ( n − j ) ! [ G Y ( y i ) ] i − 1 [ G Y ( y j ) − G Y ( y i ) ] j − i − 1 [ 1 − G Y ( y j ) ] n − j g Y ( y i ) g Y ( y j ) , y i < y j . g_{ij}(y_i,y_j)=\frac{n!}{(i-1)!(j-i-1)!(n-j)!}[G_Y(y_i)]^{i-1}[G_Y(y_j)-G_Y(y_i)]^{j-i-1}[1-G_Y(y_j)]^{n-j}g_Y(y_i)g_Y(y_j),\qquad y_i<y_j. g ij ( y i , y j ) = ( i − 1 )! ( j − i − 1 )! ( n − j )! n ! [ G Y ( y i ) ] i − 1 [ G Y ( y j ) − G Y ( y i ) ] j − i − 1 [ 1 − G Y ( y j ) ] n − j g Y ( y i ) g Y ( y j ) , y i < y j .
The differential Rényi–Tsallis divergence between the bivariate order statistics of two samples admits a joint representation in terms of the Rényi divergence and the Tsallis divergence based on their joint probability density functions. That is,
d R − T α ( F X , G Y ) = α α + 1 [ d R α ( F X , G Y ) + d T α ( F X , G Y ) ] , (24) d_{R-T}^{\alpha}(F_X,G_Y)=\frac{\alpha}{\alpha+1}\left[d_R^{\alpha}(F_X,G_Y)+d_T^{\alpha}(F_X,G_Y)\right], \tag*{(24)} d R − T α ( F X , G Y ) = α + 1 α [ d R α ( F X , G Y ) + d T α ( F X , G Y ) ] , (24)
where d R α ( F X , G Y ) d_R^{\alpha}(F_X,G_Y) d R α ( F X , G Y ) and d T α ( F X , G Y ) d_T^{\alpha}(F_X,G_Y) d T α ( F X , G Y ) denote the Rényi divergence and Tsallis divergence, respectively, between the joint densities of the bivariate order statistics given by,
d R α ( F X , G Y ) = 1 α − 1 log [ ∫ 0 ∞ ∫ x i ∞ f i j α ( x i , x j ) g i j 1 − α ( x i , x j ) d x j d x i ] , d_R^{\alpha}(F_X,G_Y)=\frac{1}{\alpha-1}\log\left[\int_0^\infty\int_{x_i}^\infty f_{ij}^{\alpha}(x_i,x_j)g_{ij}^{1-\alpha}(x_i,x_j)\,dx_j\,dx_i\right], d R α ( F X , G Y ) = α − 1 1 log [ ∫ 0 ∞ ∫ x i ∞ f ij α ( x i , x j ) g ij 1 − α ( x i , x j ) d x j d x i ] ,
d T α ( F X , G Y ) = 1 α − 1 [ ∫ 0 ∞ ∫ x i ∞ f i j α ( x i , x j ) g i j 1 − α ( x i , x j ) d x j d x i − 1 ] . d_T^{\alpha}(F_X,G_Y)=\frac{1}{\alpha-1}\left[\int_0^\infty\int_{x_i}^\infty f_{ij}^{\alpha}(x_i,x_j)g_{ij}^{1-\alpha}(x_i,x_j)\,dx_j\,dx_i-1\right]. d T α ( F X , G Y ) = α − 1 1 [ ∫ 0 ∞ ∫ x i ∞ f ij α ( x i , x j ) g ij 1 − α ( x i , x j ) d x j d x i − 1 ] .
Rényi–Tsallis Divergence for Bivariate Extreme Order Statistics
Let X 1 , X 2 , … , X n X_1,X_2,\ldots,X_n X 1 , X 2 , … , X n be a random sample from a continuous distribution with cumulative distribution function F X ( x ) F_X(x) F X ( x ) and probability density function f X ( x ) f_X(x) f X ( x ) . Similarly, let Y 1 , Y 2 , … , Y n Y_1,Y_2,\ldots,Y_n Y 1 , Y 2 , … , Y n be a random sample from another continuous distribution with cumulative distribution function G Y ( x ) G_Y(x) G Y ( x ) and probability density function g Y ( x ) g_Y(x) g Y ( x ) .
Let X 1 : n X_{1:n} X 1 : n and X n : n X_{n:n} X n : n denote the minimum and maximum order statistics, respectively. Then, for x 1 < x n x_1<x_n x 1 < x n , the joint probability density function of the bivariate extreme order statistic vector ( X 1 : n , X n : n ) (X_{1:n},X_{n:n}) ( X 1 : n , X n : n ) is given by
f 1 , n ( x 1 , x n ) = n ( n − 1 ) [ F X ( x n ) − F X ( x 1 ) ] n − 2 f X ( x 1 ) f X ( x n ) . f_{1,n}(x_1,x_n)=n(n-1)[F_X(x_n)-F_X(x_1)]^{n-2}f_X(x_1)f_X(x_n). f 1 , n ( x 1 , x n ) = n ( n − 1 ) [ F X ( x n ) − F X ( x 1 ) ] n − 2 f X ( x 1 ) f X ( x n ) .
Similarly,
g 1 , n ( x 1 , x n ) = n ( n − 1 ) [ G Y ( x n ) − G Y ( x 1 ) ] n − 2 g Y ( x 1 ) g Y ( x n ) . g_{1,n}(x_1,x_n)=n(n-1)[G_Y(x_n)-G_Y(x_1)]^{n-2}g_Y(x_1)g_Y(x_n). g 1 , n ( x 1 , x n ) = n ( n − 1 ) [ G Y ( x n ) − G Y ( x 1 ) ] n − 2 g Y ( x 1 ) g Y ( x n ) .
Then, the Rényi–Tsallis divergence of order α > 0 \alpha> 0 α > 0 , α ≠ 1 \alpha\ne1 α = 1 , between the bivariate extreme order statistic distributions corresponding to F X F_X F X and G Y G_Y G Y is defined as
d R − T α ( F X , G Y ) = 1 α − α − 1 [ log I α + I α − 1 ] , (25) d_{R-T}^{\alpha}(F_X,G_Y)=\frac{1}{\alpha-\alpha^{-1}}\left[\log I_\alpha+I_\alpha-1\right], \tag*{(25)} d R − T α ( F X , G Y ) = α − α − 1 1 [ log I α + I α − 1 ] , (25)
where
I α = ∫ − ∞ ∞ ∫ x 1 ∞ f 1 , n α ( x 1 , x n ) g 1 , n 1 − α ( x 1 , x n ) d x n d x 1 = n ( n − 1 ) ∫ − ∞ ∞ ∫ x 1 ∞ [ F X ( x n ) − F X ( x 1 ) ] α ( n − 2 ) [ G Y ( x n ) − G Y ( x 1 ) ] ( 1 − α ) ( n − 2 ) × f X α ( x 1 ) g Y 1 − α ( x 1 ) f X α ( x n ) g Y 1 − α ( x n ) d x n d x 1 . (26) \begin{aligned}
I_\alpha&=\int_{-\infty}^{\infty}\int_{x_1}^{\infty}f_{1,n}^{\alpha}(x_1,x_n)g_{1,n}^{1-\alpha}(x_1,x_n)\,\mathrm{d}x_n\,\mathrm{d}x_1\\
&=n(n-1)\int_{-\infty}^{\infty}\int_{x_1}^{\infty}\left[F_X(x_n)-F_X(x_1)\right]^{\alpha(n-2)}\left[G_Y(x_n)-G_Y(x_1)\right]^{(1-\alpha)(n-2)}\\
&\qquad{}\times f_X^\alpha(x_1)g_Y^{1-\alpha}(x_1)f_X^\alpha(x_n)g_Y^{1-\alpha}(x_n)\,\mathrm{d}x_n\,\mathrm{d}x_1. \tag*{(26)}
\end{aligned} I α = ∫ − ∞ ∞ ∫ x 1 ∞ f 1 , n α ( x 1 , x n ) g 1 , n 1 − α ( x 1 , x n ) d x n d x 1 = n ( n − 1 ) ∫ − ∞ ∞ ∫ x 1 ∞ [ F X ( x n ) − F X ( x 1 ) ] α ( n − 2 ) [ G Y ( x n ) − G Y ( x 1 ) ] ( 1 − α ) ( n − 2 ) × f X α ( x 1 ) g Y 1 − α ( x 1 ) f X α ( x n ) g Y 1 − α ( x n ) d x n d x 1 . (26)
Example 3.1. : Exponential Distribution Let X 1 , … , X n ∼ i . i . d . Exp ( λ 1 ) X_1,\ldots,X_n \mathrel{\overset{\mathrm{i.i.d.}}{\sim}} \operatorname{Exp}(\lambda_1) X 1 , … , X n ∼ i.i.d. Exp ( λ 1 ) , Y 1 , … , Y n ∼ i . i . d . Exp ( λ 2 ) Y_1,\ldots,Y_n \mathrel{\overset{\mathrm{i.i.d.}}{\sim}} \operatorname{Exp}(\lambda_2) Y 1 , … , Y n ∼ i.i.d. Exp ( λ 2 ) , where λ 1 > 0 \lambda_1>0 λ 1 > 0 and λ 2 > 0 \lambda_2>0 λ 2 > 0 . The corresponding density and distribution functions are
f ( x ) = λ 1 e − λ 1 x , F X ( x ) = 1 − e − λ 1 x , x > 0 , f(x)=\lambda_1e^{-\lambda_1x},\qquad F_X(x)=1-e^{-\lambda_1x},\qquad x>0, f ( x ) = λ 1 e − λ 1 x , F X ( x ) = 1 − e − λ 1 x , x > 0 , and
g ( x ) = λ 2 e − λ 2 x , G Y ( x ) = 1 − e − λ 2 x , x > 0. g(x)=\lambda_2e^{-\lambda_2x},\qquad G_Y(x)=1-e^{-\lambda_2x},\qquad x>0. g ( x ) = λ 2 e − λ 2 x , G Y ( x ) = 1 − e − λ 2 x , x > 0. For the bivariate extreme order statistics ( X 1 : n , X n : n ) (X_{1:n},X_{n:n}) ( X 1 : n , X n : n ) and ( Y 1 : n , Y n : n ) (Y_{1:n},Y_{n:n}) ( Y 1 : n , Y n : n ) , the joint densities are
f 1 , n ( x 1 , x n ) = n ( n − 1 ) ( e − λ 1 x 1 − e − λ 1 x n ) n − 2 λ 1 2 e − λ 1 ( x 1 + x n ) , f_{1,n}(x_1,x_n)=n(n-1)\left(e^{-\lambda_1x_1}-e^{-\lambda_1x_n}\right)^{n-2}\lambda_1^2e^{-\lambda_1(x_1+x_n)}, f 1 , n ( x 1 , x n ) = n ( n − 1 ) ( e − λ 1 x 1 − e − λ 1 x n ) n − 2 λ 1 2 e − λ 1 ( x 1 + x n ) , and
g 1 , n ( x 1 , x n ) = n ( n − 1 ) ( e − λ 2 x 1 − e − λ 2 x n ) n − 2 λ 2 2 e − λ 2 ( x 1 + x n ) . g_{1,n}(x_1,x_n)=n(n-1)\left(e^{-\lambda_2x_1}-e^{-\lambda_2x_n}\right)^{n-2}\lambda_2^2e^{-\lambda_2(x_1+x_n)}. g 1 , n ( x 1 , x n ) = n ( n − 1 ) ( e − λ 2 x 1 − e − λ 2 x n ) n − 2 λ 2 2 e − λ 2 ( x 1 + x n ) . Then the Rényi–Tsallis divergence of order α > 0 \alpha>0 α > 0 , α ≠ 1 \alpha\ne1 α = 1 between these two bivariate extreme order statistics is given by,
d R − T α ( F X , G Y ) = 1 α − α − 1 [ log ( I α ) + I α − 1 ] , (27) d_{R-T}^{\alpha}(F_X,G_Y)=\frac{1}{\alpha-\alpha^{-1}}\left[\log(I_\alpha)+I_\alpha-1\right], \tag*{(27)} d R − T α ( F X , G Y ) = α − α − 1 1 [ log ( I α ) + I α − 1 ] , (27) where
I α = ∫ 0 ∞ ∫ x 1 ∞ f 1 , n α ( x 1 , x n ) g 1 , n 1 − α ( x 1 , x n ) d x n d x 1 . (28) I_\alpha=\int_0^\infty\int_{x_1}^{\infty}f_{1,n}^{\alpha}(x_1,x_n)g_{1,n}^{1-\alpha}(x_1,x_n)\,\mathrm{d}x_n\,\mathrm{d}x_1. \tag*{(28)} I α = ∫ 0 ∞ ∫ x 1 ∞ f 1 , n α ( x 1 , x n ) g 1 , n 1 − α ( x 1 , x n ) d x n d x 1 . (28) Substituting the corresponding joint densities, we obtain
I α = n ( n − 1 ) λ 1 2 α λ 2 2 ( 1 − α ) ∫ 0 ∞ ∫ x 1 ∞ ( e − λ 1 x 1 − e − λ 1 x n ) α ( n − 2 ) × ( e − λ 2 x 1 − e − λ 2 x n ) ( 1 − α ) ( n − 2 ) e − [ α λ 1 + ( 1 − α ) λ 2 ] ( x 1 + x n ) d x n d x 1 . (29) \begin{aligned}
I_\alpha={}&n(n-1)\lambda_1^{2\alpha}\lambda_2^{2(1-\alpha)}\int_0^\infty\int_{x_1}^{\infty}\left(e^{-\lambda_1x_1}-e^{-\lambda_1x_n}\right)^{\alpha(n-2)}\\
&\times\left(e^{-\lambda_2x_1}-e^{-\lambda_2x_n}\right)^{(1-\alpha)(n-2)}e^{-[\alpha\lambda_1+(1-\alpha)\lambda_2](x_1+x_n)}\,\mathrm{d}x_n\,\mathrm{d}x_1. \tag*{(29)}
\end{aligned} I α = n ( n − 1 ) λ 1 2 α λ 2 2 ( 1 − α ) ∫ 0 ∞ ∫ x 1 ∞ ( e − λ 1 x 1 − e − λ 1 x n ) α ( n − 2 ) × ( e − λ 2 x 1 − e − λ 2 x n ) ( 1 − α ) ( n − 2 ) e − [ α λ 1 + ( 1 − α ) λ 2 ] ( x 1 + x n ) d x n d x 1 . (29)
Example 3.2. Weibull Distribution
Let X 1 , … , X n ∼ i . i . d . Weibull ( λ 1 , β 1 ) X_1,\ldots,X_n \mathrel{\overset{\mathrm{i.i.d.}}{\sim}} \operatorname{Weibull}(\lambda_1,\beta_1) X 1 , … , X n ∼ i.i.d. Weibull ( λ 1 , β 1 ) , Y 1 , … , Y n ∼ i . i . d . Weibull ( λ 2 , β 2 ) Y_1,\ldots,Y_n \mathrel{\overset{\mathrm{i.i.d.}}{\sim}} \operatorname{Weibull}(\lambda_2,\beta_2) Y 1 , … , Y n ∼ i.i.d. Weibull ( λ 2 , β 2 ) , where λ 1 , λ 2 , β 1 , β 2 > 0 \lambda_1,\lambda_2,\beta_1,\beta_2>0 λ 1 , λ 2 , β 1 , β 2 > 0 . The corresponding density and distribution functions are
f ( x ) = β 1 λ 1 β 1 x β 1 − 1 e − ( λ 1 x ) β 1 , F X ( x ) = 1 − e − ( λ 1 x ) β 1 , x > 0 , f(x)=\beta_1\lambda_1^{\beta_1}x^{\beta_1-1}e^{-(\lambda_1x)^{\beta_1}},\qquad F_X(x)=1-e^{-(\lambda_1x)^{\beta_1}},\qquad x>0, f ( x ) = β 1 λ 1 β 1 x β 1 − 1 e − ( λ 1 x ) β 1 , F X ( x ) = 1 − e − ( λ 1 x ) β 1 , x > 0 , and
g ( x ) = β 2 λ 2 β 2 x β 2 − 1 e − ( λ 2 x ) β 2 , G Y ( x ) = 1 − e − ( λ 2 x ) β 2 , x > 0. g(x)=\beta_2\lambda_2^{\beta_2}x^{\beta_2-1}e^{-(\lambda_2x)^{\beta_2}},\qquad G_Y(x)=1-e^{-(\lambda_2x)^{\beta_2}},\qquad x>0. g ( x ) = β 2 λ 2 β 2 x β 2 − 1 e − ( λ 2 x ) β 2 , G Y ( x ) = 1 − e − ( λ 2 x ) β 2 , x > 0.
For the bivariate extreme order statistics ( X 1 : n , X n : n ) (X_{1:n}, X_{n:n}) ( X 1 : n , X n : n ) and ( Y 1 : n , Y n : n ) (Y_{1:n}, Y_{n:n}) ( Y 1 : n , Y n : n ) , the joint densities are
f 1 , n ( x 1 , x n ) = n ( n − 1 ) ( e − ( λ 1 x 1 ) β 1 − e − ( λ 1 x n ) β 1 ) n − 2 × β 1 2 λ 1 2 β 1 x 1 β 1 − 1 x n β 1 − 1 e − ( λ 1 x 1 ) β 1 − ( λ 1 x n ) β 1 , f_{1,n}(x_1,x_n)=n(n-1)\left(e^{-(\lambda_1x_1)^{\beta_1}}-e^{-(\lambda_1x_n)^{\beta_1}}\right)^{n-2}\times\beta_1^2\lambda_1^{2\beta_1}x_1^{\beta_1-1}x_n^{\beta_1-1}e^{-(\lambda_1x_1)^{\beta_1}-(\lambda_1x_n)^{\beta_1}}, f 1 , n ( x 1 , x n ) = n ( n − 1 ) ( e − ( λ 1 x 1 ) β 1 − e − ( λ 1 x n ) β 1 ) n − 2 × β 1 2 λ 1 2 β 1 x 1 β 1 − 1 x n β 1 − 1 e − ( λ 1 x 1 ) β 1 − ( λ 1 x n ) β 1 ,
and
g 1 , n ( x 1 , x n ) = n ( n − 1 ) ( e − ( λ 2 x 1 ) β 2 − e − ( λ 2 x n ) β 2 ) n − 2 × β 2 2 λ 2 2 β 2 x 1 β 2 − 1 x n β 2 − 1 e − ( λ 2 x 1 ) β 2 − ( λ 2 x n ) β 2 . g_{1,n}(x_1,x_n)=n(n-1)\left(e^{-(\lambda_2x_1)^{\beta_2}}-e^{-(\lambda_2x_n)^{\beta_2}}\right)^{n-2}\times\beta_2^2\lambda_2^{2\beta_2}x_1^{\beta_2-1}x_n^{\beta_2-1}e^{-(\lambda_2x_1)^{\beta_2}-(\lambda_2x_n)^{\beta_2}}. g 1 , n ( x 1 , x n ) = n ( n − 1 ) ( e − ( λ 2 x 1 ) β 2 − e − ( λ 2 x n ) β 2 ) n − 2 × β 2 2 λ 2 2 β 2 x 1 β 2 − 1 x n β 2 − 1 e − ( λ 2 x 1 ) β 2 − ( λ 2 x n ) β 2 .
Then the Rényi–Tsallis divergence of order α > 0 \alpha>0 α > 0 , α ≠ 1 \alpha\ne1 α = 1 between these two bivariate extreme order statistics is given by
d R − T α ( F X , G Y ) = 1 α − α − 1 [ log I α + I α − 1 ] , (30) d_{R-T}^{\alpha}(F_X,G_Y)=\frac{1}{\alpha-\alpha^{-1}}\left[\log I_\alpha+I_\alpha-1\right], \tag*{(30)} d R − T α ( F X , G Y ) = α − α − 1 1 [ log I α + I α − 1 ] , (30)
where
I α = ∫ 0 ∞ ∫ x 1 ∞ f 1 , n α ( x 1 , x n ) g 1 , n 1 − α ( x 1 , x n ) d x n d x 1 . (31) I_\alpha=\int_0^\infty\int_{x_1}^\infty f_{1,n}^{\alpha}(x_1,x_n)g_{1,n}^{1-\alpha}(x_1,x_n)\,dx_n\,dx_1. \tag*{(31)} I α = ∫ 0 ∞ ∫ x 1 ∞ f 1 , n α ( x 1 , x n ) g 1 , n 1 − α ( x 1 , x n ) d x n d x 1 . (31)
Substituting the corresponding joint densities, we obtain
I α = n ( n − 1 ) β 1 2 α β 2 2 ( 1 − α ) λ 1 2 α β 1 λ 2 2 ( 1 − α ) β 2 ∫ 0 ∞ ∫ x 1 ∞ ( e − ( λ 1 x 1 ) β 1 − e − ( λ 1 x n ) β 1 ) α ( n − 2 ) × ( e − ( λ 2 x 1 ) β 2 − e − ( λ 2 x n ) β 2 ) ( 1 − α ) ( n − 2 ) ( x 1 x n ) α ( β 1 − 1 ) + ( 1 − α ) ( β 2 − 1 ) × e − α [ ( λ 1 x 1 ) β 1 + ( λ 1 x n ) β 1 ] e − ( 1 − α ) [ ( λ 2 x 1 ) β 2 + ( λ 2 x n ) β 2 ] d x n d x 1 . (32) \begin{aligned}
I_\alpha={}&n(n-1)\beta_1^{2\alpha}\beta_2^{2(1-\alpha)}\lambda_1^{2\alpha\beta_1}\lambda_2^{2(1-\alpha)\beta_2}\int_0^\infty\int_{x_1}^\infty\left(e^{-(\lambda_1x_1)^{\beta_1}}-e^{-(\lambda_1x_n)^{\beta_1}}\right)^{\alpha(n-2)}\\
&\times\left(e^{-(\lambda_2x_1)^{\beta_2}}-e^{-(\lambda_2x_n)^{\beta_2}}\right)^{(1-\alpha)(n-2)}(x_1x_n)^{\alpha(\beta_1-1)+(1-\alpha)(\beta_2-1)}\\
&\times e^{-\alpha[(\lambda_1x_1)^{\beta_1}+(\lambda_1x_n)^{\beta_1}]}e^{-(1-\alpha)[(\lambda_2x_1)^{\beta_2}+(\lambda_2x_n)^{\beta_2}]}\,dx_n\,dx_1. \tag*{(32)}
\end{aligned} I α = n ( n − 1 ) β 1 2 α β 2 2 ( 1 − α ) λ 1 2 α β 1 λ 2 2 ( 1 − α ) β 2 ∫ 0 ∞ ∫ x 1 ∞ ( e − ( λ 1 x 1 ) β 1 − e − ( λ 1 x n ) β 1 ) α ( n − 2 ) × ( e − ( λ 2 x 1 ) β 2 − e − ( λ 2 x n ) β 2 ) ( 1 − α ) ( n − 2 ) ( x 1 x n ) α ( β 1 − 1 ) + ( 1 − α ) ( β 2 − 1 ) × e − α [( λ 1 x 1 ) β 1 + ( λ 1 x n ) β 1 ] e − ( 1 − α ) [( λ 2 x 1 ) β 2 + ( λ 2 x n ) β 2 ] d x n d x 1 . (32)
Example 3.3. Pareto Type-I Distribution
Let X 1 , … , X n ∼ i . i . d . Pareto ( λ 1 , β 1 ) X_1,\ldots,X_n\mathrel{\mathop{\sim}^{\mathrm{i.i.d.}}}\operatorname{Pareto}(\lambda_1,\beta_1) X 1 , … , X n ∼ i.i.d. Pareto ( λ 1 , β 1 ) , Y 1 , … , Y n ∼ i . i . d . Pareto ( λ 2 , β 2 ) Y_1,\ldots,Y_n\mathrel{\mathop{\sim}^{\mathrm{i.i.d.}}}\operatorname{Pareto}(\lambda_2,\beta_2) Y 1 , … , Y n ∼ i.i.d. Pareto ( λ 2 , β 2 ) , where λ 1 , λ 2 , β 1 , β 2 > 0 \lambda_1,\lambda_2,\beta_1,\beta_2>0 λ 1 , λ 2 , β 1 , β 2 > 0 . The corresponding density and distribution functions are
f ( x ) = β 1 λ 1 β 1 x − ( β 1 + 1 ) , F X ( x ) = 1 − ( λ 1 x ) β 1 , x ≥ λ 1 , f(x)=\beta_1\lambda_1^{\beta_1}x^{-(\beta_1+1)},\qquad F_X(x)=1-\left(\frac{\lambda_1}{x}\right)^{\beta_1},\qquad x\ge\lambda_1, f ( x ) = β 1 λ 1 β 1 x − ( β 1 + 1 ) , F X ( x ) = 1 − ( x λ 1 ) β 1 , x ≥ λ 1 ,
and
g ( x ) = β 2 λ 2 β 2 x − ( β 2 + 1 ) , G Y ( x ) = 1 − ( λ 2 x ) β 2 , x ≥ λ 2 . g(x)=\beta_2\lambda_2^{\beta_2}x^{-(\beta_2+1)},\qquad G_Y(x)=1-\left(\frac{\lambda_2}{x}\right)^{\beta_2},\qquad x\ge\lambda_2. g ( x ) = β 2 λ 2 β 2 x − ( β 2 + 1 ) , G Y ( x ) = 1 − ( x λ 2 ) β 2 , x ≥ λ 2 .
For the bivariate extreme order statistics ( X 1 : n , X n : n ) (X_{1:n}, X_{n:n}) ( X 1 : n , X n : n ) and ( Y 1 : n , Y n : n ) (Y_{1:n}, Y_{n:n}) ( Y 1 : n , Y n : n ) , the joint densities are
f 1 , n ( x 1 , x n ) = n ( n − 1 ) [ ( λ 1 x 1 ) β 1 − ( λ 1 x n ) β 1 ] n − 2 β 1 2 λ 1 2 β 1 ( x 1 x n ) − ( β 1 + 1 ) , f_{1,n}(x_1,x_n)=n(n-1)\left[\left(\frac{\lambda_1}{x_1}\right)^{\beta_1}-\left(\frac{\lambda_1}{x_n}\right)^{\beta_1}\right]^{n-2}\beta_1^2\lambda_1^{2\beta_1}(x_1x_n)^{-(\beta_1+1)}, f 1 , n ( x 1 , x n ) = n ( n − 1 ) [ ( x 1 λ 1 ) β 1 − ( x n λ 1 ) β 1 ] n − 2 β 1 2 λ 1 2 β 1 ( x 1 x n ) − ( β 1 + 1 ) ,
and
g 1 , n ( x 1 , x n ) = n ( n − 1 ) [ ( λ 2 x 1 ) β 2 − ( λ 2 x n ) β 2 ] n − 2 β 2 2 λ 2 2 β 2 ( x 1 x n ) − ( β 2 + 1 ) . g_{1,n}(x_1,x_n)=n(n-1)\left[\left(\frac{\lambda_2}{x_1}\right)^{\beta_2}-\left(\frac{\lambda_2}{x_n}\right)^{\beta_2}\right]^{n-2}\beta_2^2\lambda_2^{2\beta_2}(x_1x_n)^{-(\beta_2+1)}. g 1 , n ( x 1 , x n ) = n ( n − 1 ) [ ( x 1 λ 2 ) β 2 − ( x n λ 2 ) β 2 ] n − 2 β 2 2 λ 2 2 β 2 ( x 1 x n ) − ( β 2 + 1 ) .
Then the Rényi–Tsallis divergence of order α > 0 \alpha>0 α > 0 , α ≠ 1 \alpha\ne1 α = 1 between these two bivariate extreme order statistics is given by
d R − T α ( F X , G Y ) = 1 α − α − 1 [ log I α + I α − 1 ] , (33) d_{R-T}^{\alpha}(F_X,G_Y)=\frac{1}{\alpha-\alpha^{-1}}\left[\log I_\alpha+I_\alpha-1\right], \tag*{(33)} d R − T α ( F X , G Y ) = α − α − 1 1 [ log I α + I α − 1 ] , (33)
where
I α = ∫ λ 1 ∞ ∫ x 1 ∞ f 1 , n α ( x 1 , x n ) g 1 , n 1 − α ( x 1 , x n ) d x n d x 1 . (34) I_\alpha=\int_{\lambda_1}^\infty\int_{x_1}^\infty f_{1,n}^{\alpha}(x_1,x_n)g_{1,n}^{1-\alpha}(x_1,x_n)\,dx_n\,dx_1. \tag*{(34)} I α = ∫ λ 1 ∞ ∫ x 1 ∞ f 1 , n α ( x 1 , x n ) g 1 , n 1 − α ( x 1 , x n ) d x n d x 1 . (34)
with λ = max ( λ 1 , λ 2 ) \lambda= \max(\lambda_1,\lambda_2) λ = max ( λ 1 , λ 2 ) . Substituting the corresponding joint densities, we obtain
I α = n ( n − 1 ) β 1 2 α β 2 2 ( 1 − α ) λ 1 2 α λ 2 2 ( 1 − α ) ∫ λ ∞ ∫ x 1 ∞ [ ( λ 1 x 1 ) β 1 − ( λ 1 x n ) β 1 ] α ( n − 2 ) [ ( λ 2 x 1 ) β 2 − ( λ 2 x n ) β 2 ] ( 1 − α ) ( n − 2 ) ( x 1 x n ) − [ α ( β 1 + 1 ) + ( 1 − α ) ( β 2 + 1 ) ] d x n d x 1 . (35) I_\alpha= n(n-1)\beta_1^{2\alpha}\beta_2^{2(1-\alpha)}\lambda_1^{2\alpha}\lambda_2^{2(1-\alpha)}\int_{\lambda}^{\infty}\int_{x_1}^{\infty}\left[\left(\frac{\lambda_1}{x_1}\right)^{\beta_1}-\left(\frac{\lambda_1}{x_n}\right)^{\beta_1}\right]^{\alpha(n-2)}
\left[\left(\frac{\lambda_2}{x_1}\right)^{\beta_2}-\left(\frac{\lambda_2}{x_n}\right)^{\beta_2}\right]^{(1-\alpha)(n-2)}
(x_1x_n)^{-[\alpha(\beta_1+1)+(1-\alpha)(\beta_2+1)]}\,dx_n\,dx_1. \tag*{(35)} I α = n ( n − 1 ) β 1 2 α β 2 2 ( 1 − α ) λ 1 2 α λ 2 2 ( 1 − α ) ∫ λ ∞ ∫ x 1 ∞ [ ( x 1 λ 1 ) β 1 − ( x n λ 1 ) β 1 ] α ( n − 2 ) [ ( x 1 λ 2 ) β 2 − ( x n λ 2 ) β 2 ] ( 1 − α ) ( n − 2 ) ( x 1 x n ) − [ α ( β 1 + 1 ) + ( 1 − α ) ( β 2 + 1 )] d x n d x 1 . (35)
Example 3.4. Rayleigh Distribution
Let X 1 , … , X n ∼ i . i . d . Rayleigh ( β 1 ) X_1,\ldots,X_n \overset{\mathrm{i.i.d.}}{\sim} \operatorname{Rayleigh}(\beta_1) X 1 , … , X n ∼ i.i.d. Rayleigh ( β 1 ) , Y 1 , … , Y n ∼ i . i . d . Rayleigh ( β 2 ) Y_1,\ldots,Y_n \overset{\mathrm{i.i.d.}}{\sim} \operatorname{Rayleigh}(\beta_2) Y 1 , … , Y n ∼ i.i.d. Rayleigh ( β 2 ) , where β 1 , β 2 > 0 \beta_1,\beta_2>0 β 1 , β 2 > 0 . The corresponding density and distribution functions are
f ( x 1 ) = x 1 β 1 2 exp ( − x 1 2 2 β 1 2 ) , F X ( x 1 ) = 1 − exp ( − x 1 2 2 β 1 2 ) , x 1 > 0 , f(x_1)=\frac{x_1}{\beta_1^2}\exp\left(-\frac{x_1^2}{2\beta_1^2}\right),\qquad F_X(x_1)=1-\exp\left(-\frac{x_1^2}{2\beta_1^2}\right),\qquad x_1>0, f ( x 1 ) = β 1 2 x 1 exp ( − 2 β 1 2 x 1 2 ) , F X ( x 1 ) = 1 − exp ( − 2 β 1 2 x 1 2 ) , x 1 > 0 ,
and
g ( x 1 ) = x 1 β 2 2 exp ( − x 1 2 2 β 2 2 ) , G Y ( x 1 ) = 1 − exp ( − x 1 2 2 β 2 2 ) , x 1 > 0. g(x_1)=\frac{x_1}{\beta_2^2}\exp\left(-\frac{x_1^2}{2\beta_2^2}\right),\qquad G_Y(x_1)=1-\exp\left(-\frac{x_1^2}{2\beta_2^2}\right),\qquad x_1>0. g ( x 1 ) = β 2 2 x 1 exp ( − 2 β 2 2 x 1 2 ) , G Y ( x 1 ) = 1 − exp ( − 2 β 2 2 x 1 2 ) , x 1 > 0.
For the bivariate extreme order statistics ( X 1 : n , X n : n ) (X_{1:n},X_{n:n}) ( X 1 : n , X n : n ) and ( Y 1 : n , Y n : n ) (Y_{1:n},Y_{n:n}) ( Y 1 : n , Y n : n ) , the joint densities are
f 1 , n ( x 1 , x n ) = n ( n − 1 ) [ e − x 1 2 / ( 2 β 1 2 ) − e − x n 2 / ( 2 β 1 2 ) ] n − 2 x 1 x n β 1 4 exp ( − x 1 2 + x n 2 2 β 1 2 ) , 0 < x 1 < x n < ∞ , f_{1,n}(x_1,x_n)=n(n-1)\left[e^{-x_1^2/(2\beta_1^2)}-e^{-x_n^2/(2\beta_1^2)}\right]^{n-2}\frac{x_1x_n}{\beta_1^4}\exp\left(-\frac{x_1^2+x_n^2}{2\beta_1^2}\right),\qquad0<x_1<x_n<\infty, f 1 , n ( x 1 , x n ) = n ( n − 1 ) [ e − x 1 2 / ( 2 β 1 2 ) − e − x n 2 / ( 2 β 1 2 ) ] n − 2 β 1 4 x 1 x n exp ( − 2 β 1 2 x 1 2 + x n 2 ) , 0 < x 1 < x n < ∞ ,
and
g 1 , n ( x 1 , x n ) = n ( n − 1 ) [ e − x 1 2 / ( 2 β 2 2 ) − e − x n 2 / ( 2 β 2 2 ) ] n − 2 x 1 x n β 2 4 exp ( − x 1 2 + x n 2 2 β 2 2 ) , 0 < x 1 < x n < ∞ . g_{1,n}(x_1,x_n)=n(n-1)\left[e^{-x_1^2/(2\beta_2^2)}-e^{-x_n^2/(2\beta_2^2)}\right]^{n-2}\frac{x_1x_n}{\beta_2^4}\exp\left(-\frac{x_1^2+x_n^2}{2\beta_2^2}\right),\qquad0<x_1<x_n<\infty. g 1 , n ( x 1 , x n ) = n ( n − 1 ) [ e − x 1 2 / ( 2 β 2 2 ) − e − x n 2 / ( 2 β 2 2 ) ] n − 2 β 2 4 x 1 x n exp ( − 2 β 2 2 x 1 2 + x n 2 ) , 0 < x 1 < x n < ∞.
Then the Rényi–Tsallis divergence of order α > 0 \alpha>0 α > 0 , α ≠ 1 \alpha\ne1 α = 1 between these two bivariate extreme order statistics is given by
d R − T α ( F X , G Y ) = 1 α − α − 1 [ log I α + I α − 1 ] , (36) d^\alpha_{R-T}(F_X,G_Y)=\frac{1}{\alpha-\alpha^{-1}}\left[\log I_\alpha+I_\alpha-1\right], \tag*{(36)} d R − T α ( F X , G Y ) = α − α − 1 1 [ log I α + I α − 1 ] , (36)
where
I α = ∫ 0 ∞ ∫ x 1 ∞ f 1 , n α ( x 1 , x n ) g 1 , n 1 − α ( x 1 , x n ) d x n d x 1 . (37) I_\alpha=\int_0^\infty\int_{x_1}^\infty f_{1,n}^{\alpha}(x_1,x_n)g_{1,n}^{1-\alpha}(x_1,x_n)\,dx_n\,dx_1. \tag*{(37)} I α = ∫ 0 ∞ ∫ x 1 ∞ f 1 , n α ( x 1 , x n ) g 1 , n 1 − α ( x 1 , x n ) d x n d x 1 . (37)
Substituting the corresponding joint densities, we obtain
I α = n ( n − 1 ) β 1 − 4 α β 2 − 4 ( 1 − α ) ∫ 0 ∞ ∫ x 1 ∞ x 1 x n [ e − x 1 2 / ( 2 β 1 2 ) − e − x n 2 / ( 2 β 1 2 ) ] α ( n − 2 ) × [ e − x 1 2 / ( 2 β 2 2 ) − e − x n 2 / ( 2 β 2 2 ) ] ( 1 − α ) ( n − 2 ) exp [ − α x 1 2 + x n 2 2 β 1 2 ] × exp [ − ( 1 − α ) x 1 2 + x n 2 2 β 2 2 ] d x n d x 1 . (38) \begin{aligned}
I_\alpha={}&n(n-1)\beta_1^{-4\alpha}\beta_2^{-4(1-\alpha)}\int_0^\infty\int_{x_1}^\infty x_1x_n\left[e^{-x_1^2/(2\beta_1^2)}-e^{-x_n^2/(2\beta_1^2)}\right]^{\alpha(n-2)}\\
&\times\left[e^{-x_1^2/(2\beta_2^2)}-e^{-x_n^2/(2\beta_2^2)}\right]^{(1-\alpha)(n-2)}
\exp\left[-\alpha\frac{x_1^2+x_n^2}{2\beta_1^2}\right]\\
&\times\exp\left[-(1-\alpha)\frac{x_1^2+x_n^2}{2\beta_2^2}\right]\,dx_n\,dx_1. \tag*{(38)}
\end{aligned} I α = n ( n − 1 ) β 1 − 4 α β 2 − 4 ( 1 − α ) ∫ 0 ∞ ∫ x 1 ∞ x 1 x n [ e − x 1 2 / ( 2 β 1 2 ) − e − x n 2 / ( 2 β 1 2 ) ] α ( n − 2 ) × [ e − x 1 2 / ( 2 β 2 2 ) − e − x n 2 / ( 2 β 2 2 ) ] ( 1 − α ) ( n − 2 ) exp [ − α 2 β 1 2 x 1 2 + x n 2 ] × exp [ − ( 1 − α ) 2 β 2 2 x 1 2 + x n 2 ] d x n d x 1 . (38)
The above examples have no closed form expression and are evaluated numerically. The following figure illustrates the Rényi–Tsallis divergence for exponential, Weibull, Pareto, and Rayleigh extreme order statistic distributions. Figure 5 shows that the Rényi–Tsallis divergence increases monotonically with α \alpha α for the extreme order statistics of all four distributions. This indicates that the measure becomes more sensitive to differences between the distributions at larger values of α \alpha α . The increase is most pronounced for the Pareto distribution, particularly for α > 0.7 \alpha>0.7 α > 0.7 , followed by the Weibull distribution. The Rayleigh distribution shows a moderate and steady increase, whereas the exponential distribution exhibits the lowest and smoothest growth.
Figure 5. Rényi–Tsallis divergence for exponential, Weibull, Pareto, and Rayleigh extreme order statistic distributions
Nonparametric Estimation of Bivariate Rényi–Tsallis Divergence
In this section, nonparametric estimators of the proposed bivariate Rényi–Tsallis divergence measure is developed. Let ( X 1 , X 2 , … , X n 1 ) (X_1, X_2, \ldots, X_{n_1}) ( X 1 , X 2 , … , X n 1 ) and ( Y 1 , Y 2 , … , Y n 2 ) (Y_1, Y_2, \ldots, Y_{n_2}) ( Y 1 , Y 2 , … , Y n 2 ) be two independent random samples drawn from two populations with unknown bivariate density functions f 1 ( x ) f_1(\mathbf{x}) f 1 ( x ) and f 2 ( x ) f_2(\mathbf{x}) f 2 ( x ) , respectively, where x = ( x 1 , x 2 ) T \mathbf{x} = (x_1,x_2)^{T} x = ( x 1 , x 2 ) T . Here, X i = ( X i 1 , X i 2 ) T \mathbf{X}_i = (X_{i1}, X_{i2})^{T} X i = ( X i 1 , X i 2 ) T , i = 1 , 2 , … , n 1 i = 1,2,\ldots,n_1 i = 1 , 2 , … , n 1 , and Y j = ( Y j 1 , Y j 2 ) T \mathbf{Y}_j = (Y_{j1}, Y_{j2})^{T} Y j = ( Y j 1 , Y j 2 ) T , j = 1 , 2 , … , n 2 j = 1,2,\ldots,n_2 j = 1 , 2 , … , n 2 .
Since the underlying distributions are unknown, the density functions are estimated nonparametrically using the multivariate kernel density estimator proposed by Duong [7 ] . The kernel density estimators of f 1 ( x ) f_1(\mathbf{x}) f 1 ( x ) and f 2 ( x ) f_2(\mathbf{x}) f 2 ( x ) are given by
f ^ 1 ( x ; H 1 ) = 1 n 1 ∑ i = 1 n 1 K H 1 ( x − X i ) , f ^ 2 ( x ; H 2 ) = 1 n 2 ∑ j = 1 n 2 K H 2 ( x − Y j ) . (39) \widehat{f}_1(\mathbf{x};H_1) = \frac{1}{n_1}\sum_{i=1}^{n_1} K_{H_1}(\mathbf{x}-\mathbf{X}_i), \qquad
\widehat{f}_2(\mathbf{x};H_2) = \frac{1}{n_2}\sum_{j=1}^{n_2} K_{H_2}(\mathbf{x}-\mathbf{Y}_j). \tag*{(39)} f 1 ( x ; H 1 ) = n 1 1 i = 1 ∑ n 1 K H 1 ( x − X i ) , f 2 ( x ; H 2 ) = n 2 1 j = 1 ∑ n 2 K H 2 ( x − Y j ) . (39)
where H 1 H_1 H 1 and H 2 H_2 H 2 are symmetric positive definite bandwidth matrices and
K H ( x ) = ∣ H ∣ − 1 / 2 K ( H − 1 / 2 x ) . K_H(\mathbf{x}) = |H|^{-1/2}K\left(H^{-1/2}\mathbf{x}\right). K H ( x ) = ∣ H ∣ − 1/2 K ( H − 1/2 x ) .
Throughout this work, the Gaussian kernel
K ( x ) = 1 ( 2 π ) d / 2 exp ( − 1 2 x T x ) K(\mathbf{x}) = \frac{1}{(2\pi)^{d/2}}\exp\left(-\frac{1}{2}\mathbf{x}^{T}\mathbf{x}\right) K ( x ) = ( 2 π ) d /2 1 exp ( − 2 1 x T x )
is employed, where d = 2 d = 2 d = 2 denotes the dimension of the random vector. The Gaussian kernel is widely used because it provides smooth density estimates and possesses desirable theoretical properties. The bandwidth matrices H 1 H_1 H 1 and H 2 H_2 H 2 are selected using the plug-in bandwidth selection method of Duong [7 ] , implemented through the Hpi() function in the ks package in R.
Replacing the unknown densities by their kernel estimates yields the following estimators of the proposed divergence measures. The estimator of the proposed bivariate Rényi–Tsallis divergence is
D ^ R T , α = 1 α − α − 1 [ log ( ∫ R 2 f ^ 1 ( x ) α f ^ 2 ( x ) 1 − α d x ) + ( ∫ R 2 f ^ 1 ( x ) α f ^ 2 ( x ) 1 − α d x − 1 ) ] . (40) \widehat{D}_{RT,\alpha} = \frac{1}{\alpha-\alpha^{-1}}\left[\log\left(\int_{\mathbb{R}^{2}} \widehat{f}_{1}(\mathbf{x})^{\alpha}\widehat{f}_{2}(\mathbf{x})^{1-\alpha}\,\mathrm{d}\mathbf{x}\right)+\left(\int_{\mathbb{R}^{2}} \widehat{f}_{1}(\mathbf{x})^{\alpha}\widehat{f}_{2}(\mathbf{x})^{1-\alpha}\,\mathrm{d}\mathbf{x}-1\right)\right]. \tag*{(40)} D R T , α = α − α − 1 1 [ log ( ∫ R 2 f 1 ( x ) α f 2 ( x ) 1 − α d x ) + ( ∫ R 2 f 1 ( x ) α f 2 ( x ) 1 − α d x − 1 ) ] . (40)
where α > 0 \alpha> 0 α > 0 , α ≠ 1 \alpha\ne1 α = 1 . Since these integrals generally do not admit closed-form solutions, they are evaluated numerically over a common two-dimensional grid after normalizing the estimated kernel densities so that they integrate to one.
Application to ACTG315 Longitudinal HIV Data
The practical applicability of the Rényi–Tsallis divergence measures was demonstrated through the analysis of the ACTG315 longitudinal HIV dataset. The dataset consists of repeated measurements of HIV-1 RNA viral load (RNA) and CD4 lymphocyte counts (CD4) for patients infected with HIV who were receiving antiretroviral therapy. These biomarkers are commonly used to monitor disease progression and treatment response.
The day of measurement was used to classify the observations into three clinical stages: Control, Treatment, and Follow-up. Since multiple observations were available for each patient within a given stage, patient-level averages of RNA and CD4 measurements were calculated. For each stage, the joint distribution of RNA viral load and CD4 lymphocyte count was estimated using bivariate Gaussian kernel density estimation with plug-in bandwidth selection.
Pairwise Rényi, Tsallis, and Rényi-Tsallis divergence measures were calculated between the Control–Treatment, Control–Follow-up, and Treatment–Follow-up comparisons, using the estimated densities for α = 0.4 \alpha= 0.4 α = 0.4 . The corresponding values are presented in Table 4 , Table 5 , and Table 6 . Results showed clear distributional differences between the three clinical stages. The biggest divergence values were observed between the Control and Follow-up groups. The Treatment and Follow-up groups had relatively smaller divergence values, indicating more similarity in their joint distributions of biomarkers. Also, out of the three divergence measures Rényi–Tsallis divergence gives the smallest values for the pairwise comparison.
Table 4. Rényi divergence values between the Control, Treatment, and Follow-up stages
Table 5. Tsallis divergence values between the Control, Treatment, and Follow-up stages
Table 6. Rényi–Tsallis divergence values between the Control, Treatment, and Follow-up stages
To assess uncertainty of the estimated divergences, patient-level nonparametric bootstrap (1000 replications) was performed. Kernel densities and divergence measures were recomputed for each bootstrap sample and bootstrap bias, variance, standard error, mean squared error (MSE) and 95% bootstrap percentile confidence intervals were obtained.
Table 7 presents the bootstrap comparison of the estimators of Rényi, Tsallis and Rényi-Tsallis divergence for bias, variance, standard error, mean squared error (MSE) and confidence interval. For both control versus treatment and treatment versus follow-up Rényi-Tsallis divergence has the lowest variance, standard error and MSE. So for most of the pairwise clinical stage comparison we can say that the Rényi-Tsallis divergence estimator showed lower variability and MSE, suggesting greater numerical stability.
Conclusion
In this article, we have proposed bivariate Rényi-Tsallis divergence, a natural extension for the univariate Rényi-Tsallis divergence which combines features of Rényi and Tsallis measures in a single framework. Several properties of bivariate Rényi-Tsallis divergence including non-negativity, pseudo additivity, behavior under linear and monotone transformations are studied. A nonparametric kernel based estimator was also developed in cases where the underlying distributions are unknown. The methodology has been successfully applied to ACTG315 longitudinal HIV data set. Bootstrap analysis has showed that the proposed estimator is able to provide accurate and stable divergence estimates. The present work can be extended to divergence measures based on bivariate distribution functions, bivariate survival functions and bivariate cumulative residual distributions. In addition, the proposed Rényi–Tsallis divergence may be generalized to the multivariate setting for comparing higher dimensional random vectors.
Table 7. Bootstrap bias, variance, standard error, MSE, and 95% percentile confidence intervals of the divergence estimators for pairwise clinical-stage comparisons
Declarations
Conflict of interest On behalf of all authors, the corresponding author states that there is no conflict of interest.
Aharon Ben-Tal, Adi Ben-Israel, and Marc Teboulle. Certainty equivalents and information measures: Duality and extremal principles. Journal of Mathematical Analysis and Applications , 157(1):211–236, 1991. doi: 10.1016/0022-247X(91)90140-N.
Anil Bhattacharyya. On a measure of divergence between two statistical populations defined