betaFeedback

WorksComputer Science

Relative Entropy under Ordered Chi-Squared Constraints: Exact Bounds and Extremizers

Abstract

We derive a sharp upper bound on the relative entropy D(P∥Q)D(P\|Q) in terms of the two ordered chi-squared divergences χ2(P∥Q)χ^2(P\|Q) and χ2(Q∥P)χ^2(Q\|P). For any A,B>0A,B>0, the constraints χ2(P∥Q)≤Aχ^2(P\|Q)\leq A and χ2(Q∥P)≤Bχ^2(Q\|P)\leq B yield upper bounds on D(P∥Q)D(P\|Q) and D(Q∥P)D(Q\|P). Two pairs of Bernoulli distributions each attain both bounds simultaneously. As a consequence, for sequences of probability mass functions {Pn}\{P_n\} and {Qn}\{Q_n\} on a common finite or countably infinite alphabet, both D(Pn∥Qn)D(P_n\|Q_n) and D(Qn∥Pn)D(Q_n\|P_n) tend to zero whenever one of the two ordered chi-squared divergences tends to zero and the other remains uniformly bounded. Without this boundedness assumption, D(Pn∥Qn)D(P_n\|Q_n) can fail to converge to zero even when χ2(Qn∥Pn)→0χ^2(Q_n\|P_n)\to 0. Finally, in contrast to the attainable upper bounds, on any finite or countably infinite alphabet containing at least three elements, the infimum of D(P∥Q)D(P\|Q) under the exact constraints χ2(P∥Q)=Aχ^2(P\|Q)=A and χ2(Q∥P)=Bχ^2(Q\|P)=B is zero and is not attained.

Paper details

vlnc:W_kysfxn3f
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
arXiv nonexclusive license
  • Computer Science

ChatGPT 6.1

View statement
Looking for related papers…