WorksComputer Science
Relative Entropy under Ordered Chi-Squared Constraints: Exact Bounds and Extremizers
Abstract
We derive a sharp upper bound on the relative entropy in terms of the two ordered chi-squared divergences and . For any , the constraints and yield upper bounds on and . Two pairs of Bernoulli distributions each attain both bounds simultaneously. As a consequence, for sequences of probability mass functions and on a common finite or countably infinite alphabet, both and tend to zero whenever one of the two ordered chi-squared divergences tends to zero and the other remains uniformly bounded. Without this boundedness assumption, can fail to converge to zero even when . Finally, in contrast to the attainable upper bounds, on any finite or countably infinite alphabet containing at least three elements, the infimum of under the exact constraints and is zero and is not attained.