Skip to main content
SpringerLink
Log in

Mutual information, variation, and Fano’s inequality

  • Information Theory
  • Published:
Problems of Information Transmission Aims and scope Submit manuscript

Abstract

Some upper and lower bounds are obtained for the maximum of the absolute value of the difference between the mutual information |I(X; Y) − I(X′; Y′)| of two pairs of discrete random variables (X, Y) and (X′, Y′) via the variational distance between the probability distributions of these pairs. In particular, the upper bound obtained here substantially generalizes and improves the upper bound of [1]. In some special cases, our upper and lower bounds coincide or are rather close. It is also proved that the lower bound is asymptotically tight in the case where the variational distance between (X, Y) and (X′ Y′) tends to zero.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Zhang, Z., Estimating Mutual Information via Kolmogorov Distance, IEEE Trans. Inform. Theory, 2007, vol. 53, no. 9, pp. 3280–3282.

    Article  MathSciNet  Google Scholar 

  2. Csiszár, I. and Körner, J., Information Theory: Coding Theorems for Discrete Memoryless Systems, New York: Academic; Budapest: Akad. Kiadó, 1981. Translated under the title Teoriya informatsii: teoremy kodirovaniya dlya diskretnykh sistem bez pamyati, Moscow: Mir, 1985.

    MATH  Google Scholar 

  3. Pinsker, M.S., On Estimation of Information via Variation, Probl. Peredachi Inf., 2005, vol. 41, no. 2, pp. 3–8 [Probl. Inf. Trans. (Engl. Transl.), 2005, vol. 41, no. 2, pp. 71–75].

    MathSciNet  Google Scholar 

  4. Prelov, V.V., On Inequalities between Mutual Information and Variation, Probl. Peredachi Inf., 2007, vol. 43, no. 1, pp. 15–27 [Probl. Inf. Trans. (Engl. Transl.), 2007, vol. 43, no. 1, pp. 12–23].

    MathSciNet  Google Scholar 

  5. Fedotov, A.A., Harremöes, P., and Topsøe, F., Refinements of Pinsker’s Inequality, IEEE Trans. Inform. Theory, 2003, vol. 49, no. 6, pp. 1491–1498.

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Kharkevich Institute for Information Transmission Problems, RAS, Moscow, Russia

    V. V. Prelov

  2. Katholieke Universiteit Leuven, Leuven, Belgium

    E. C. van der Meulen

Authors
  1. V. V. Prelov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. E. C. van der Meulen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to V. V. Prelov.

Additional information

Original Russian Text © Prelov, van der Meulen, 2008, published in Problemy Peredachi Informatsii, 2008, Vol. 44, No. 3, pp. 19–32.

Supported in part by the Russian Foundation for Basic Research, project no. 06-01-00226, and the Research Fund K. U. Leuven, project GOA/98/06.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Prelov, V.V., van der Meulen, E.C. Mutual information, variation, and Fano’s inequality. Probl Inf Transm 44, 185–197 (2008). https://doi.org/10.1134/S0032946008030022

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S0032946008030022

Keywords

Search

Navigation