Skip to main content

Advertisement

Springer Nature Link
Log in

Formalization of Shannon’s Theorems

  • Published:
Journal of Automated Reasoning Aims and scope Submit manuscript

Abstract

The most fundamental results of information theory are Shannon’s theorems. These theorems express the bounds for (1) reliable data compression and (2) data transmission over a noisy channel. Their proofs are non-trivial but are rarely detailed, even in the introductory literature. This lack of formal foundations is all the more unfortunate that crucial results in computer security rely solely on information theory: this is the so-called “unconditional security”. In this article, we report on the formalization of a library for information theory in the SSReflect extension of the Coq proof-assistant. In particular, we produce the first formal proofs of the source coding theorem, that introduces the entropy as the bound for lossless compression, and of the channel coding theorem, that introduces the capacity as the bound for reliable communication over a noisy channel.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Affeldt, R., Hagiwara, M.: Formalization of Shannon’s Theorems in SSReflect-Coq. In: Proceedings of the 3rd International Conference on Interactive Theorem Proving (ITP 2012), Princeton, NJ, USA, August 13–15, 2012. Lecture Notes in Computer Science, vol. 7406, pp. 233–249. Springer, Heidelberg (2012)

    Google Scholar 

  2. Affeldt, R., Hagiwara, M., Sénizergues, J.: Formalization of Shannon’s Theorems in SSReflect-Coq. Coq documentation and scripts available at http://staff.aist.go.jp/reynald.affeldt/shannon

  3. Audebaud, P., Paulin-Mohring, C.: Proofs of randomized algorithms in Coq. Sci. Comput. Prog. 74(8), 568–589 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  4. Barthe, G., Crespo, J.M., Grégoire, B., Kunz, C., Zanella Béguelin, S.: Computer-Aided Cryptographic Proofs. In: Proceedings of the 3rd International Conference on Interactive Theorem Proving (ITP 2012), Princeton, NJ, USA, August 2012. Lecture Notes in Computer Science, vol. 7406, pp. 11–27. Springer, Heidelberg (2012)

    Google Scholar 

  5. Bertot, Y., Gonthier, G., Ould Biha, S., Pasca, I.: Canonical Big Operators. In: Proceedings of the 21st International Conference on Theorem Proving in Higher Order Logics (TPHOLs 2008), Montreal, Canada, August 18–21, 2008. Lecture Notes in Computer Science, vol. 5170, pp. 86–101. Springer, Heidelberg (2008)

    Google Scholar 

  6. Coble, A.R.: Anonymity, Information, and Machine-Assisted Proof. PhD Thesis, King’s College. University of Cambridge, UK (2010)

    Google Scholar 

  7. The Coq Development Team. Reference Manual. Version 8.4. Available at http://coq.inria.fr. INRIA (2004–2012)

  8. Cover, T.M., Thomas, J.A.: Elements of Information Theory, 2nd edn. Wiley-Interscience (2006)

  9. Csiszár, I.: The method of types. IEEE Trans. Inform. Theory 44(6), 2505–2523 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  10. Csiszár, I., Körner, J.: Information Theory—Coding Theorems for Discrete Memoryless Systems, 2nd edn. Cambridge University Press (2011)

  11. Gonthier, G., Mahboubi, A., Tassi, E.: A Small Scale Reflection Extension for the Coq system. Version 10. Technical Report RR-6455, INRIA (2011)

  12. Hagiwara, M.: Coding Theory: Mathematics for Digital Communication. In Japanese. http://www.nippyo.co.jp/book/5977.html. Nippon Hyoron Sha (2012)

  13. Hasan, O., Tahar, S.: Verification of expectation using theorem proving to verify expectation and variance for discrete random variables. J. Autom. Reason. 41, 295–323 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  14. Hölzl, J., Heller, A.: Three Chapters of Measure Theory in Isabelle/HOL. In: Proceedings of the 2nd International Conference on Interactive Theorem Proving (ITP 2011), Berg en Dal, The Netherlands, August 22–25, 2011. Lecture Notes in Computer Science, vol. 6898, pp. 135–151. Springer, Heidelberg (2011)

    Google Scholar 

  15. Hurd, J.: Formal Verification of Probabilistic Algorithms. PhD Thesis, Trinity College, University of Cambridge, UK (2001)

    Google Scholar 

  16. Mhamdi, T., Hasan, O., Tahar, S.: On the Formalization of the Lebesgue Integration Theory in HOL. In: Proceedings of the 1st International Conference on Interactive Theorem Proving (ITP 2010), Edinburgh, UK, July 11–14, 2010. Lecture Notes in Computer Science, vol. 6172, pp. 387–402. Springer, Heidelberg (2010)

    Google Scholar 

  17. Mhamdi, T., Hasan, O., Tahar, S.: Formalization of Entropy Measures in HOL. In: Proceedings of the 2nd International Conference on Interactive Theorem Proving (ITP 2011), Berg en Dal, The Netherlands, August 22–25, 2011. Lecture Notes in Computer Science, vol. 6898, pp. 233–248. Springer, Heidelberg (2011)

    Google Scholar 

  18. Khudanpur, S.: Information Theoretic Methods in Statistics. Lecture Notes. Available at http://old-site.clsp.jhu.edu/sanjeev/520.674/notes/GISAlgorithm.pdf (1999). Accessed 02 May 2013

  19. Shannon, C.E.: A Mathematical theory of communication. Bell Syst. Tech. J. 27, 379–423 and 623–656 (1948)

    Article  MATH  MathSciNet  Google Scholar 

  20. Shannon, C.E.: Communication theory of secrecy systems. Bell Sys. Tech. J. 28, 656–715 (1949)

    Article  MATH  MathSciNet  Google Scholar 

  21. Uyematsu, T.: Modern Shannon Theory, Information Theory with Types. In Japanese. Baifukan (1998)

  22. Verdú, S.: Fifty years of Shannon theory. IEEE Trans. Inform. Theory 44(6), 2057–2078 (1998)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Research Institute for Secure Systems, National Institute of Advanced Industrial Scienceand Technology, Tsukuba, Ibaraki, Japan

    Reynald Affeldt

  2. Department of Mathematics and Informatics, Faculty of Science, Chiba University, Chiba, Japan

    Manabu Hagiwara

  3. École Normale Supérieure de Cachan, Cachan, France

    Jonas Sénizergues

Authors
  1. Reynald Affeldt
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Manabu Hagiwara
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Jonas Sénizergues
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Reynald Affeldt.

Additional information

This article is a revised and extended version of a conference paper [1].

This work was essentially carried out when the second and third author were affiliated with Research Institute for Secure Systems, National Institute of Advanced Industrial Science and Technology, Japan.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Affeldt, R., Hagiwara, M. & Sénizergues, J. Formalization of Shannon’s Theorems. J Autom Reasoning 53, 63–103 (2014). https://doi.org/10.1007/s10817-013-9298-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10817-013-9298-1

Keywords