WorksMathematics
The Csisz{á}r order for ranking the importance of input random variables
Abstract
Csisz{á}r indices , defined as the -divergence between the joint distribution of a general random couple and the product of its marginal distributions, are widely used to quantify dependence between the input variable and the output variable . We show that the ranking of two input variables and in terms of their importance w.r.t.\ the output variable, given by the comparison of the Csisz{á}r indices and depends on the choice of . We introduce the Csisz{á}r order of input variables as the uniform ordering of Csisz{á}r indices over all proper convex functions . We give a sufficient condition to prove Csiszár order of two input multivariate random variables. We fully characterize it in two cases: when is a Gaussian vector (in the one-dimensional case this amount to compare the square of their correlation); and when , and are Bernoulli random variables. Finally, motivated by sensitivity analysis, we show that, when is a three-dimensional Gaussian random vector, thresholding , , and either preserves the Csisz{á}r order or makes the input variables incomparable; in particular thresholding never reverses the ranking.