Abstract
The most fundamental results of information theory are Shannon’s theorems. These theorems express the bounds for (1) reliable data compression and (2) data transmission over a noisy channel. Their proofs are non-trivial but are rarely detailed, even in the introductory literature. This lack of formal foundations is all the more unfortunate that crucial results in computer security rely solely on information theory: this is the so-called “unconditional security”. In this article, we report on the formalization of a library for information theory in the SSReflect extension of the Coq proof-assistant. In particular, we produce the first formal proofs of the source coding theorem, that introduces the entropy as the bound for lossless compression, and of the channel coding theorem, that introduces the capacity as the bound for reliable communication over a noisy channel.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Affeldt, R., Hagiwara, M.: Formalization of Shannon’s Theorems in SSReflect-Coq. In: Proceedings of the 3rd International Conference on Interactive Theorem Proving (ITP 2012), Princeton, NJ, USA, August 13–15, 2012. Lecture Notes in Computer Science, vol. 7406, pp. 233–249. Springer, Heidelberg (2012)
Affeldt, R., Hagiwara, M., Sénizergues, J.: Formalization of Shannon’s Theorems in SSReflect-Coq. Coq documentation and scripts available at http://staff.aist.go.jp/reynald.affeldt/shannon
Audebaud, P., Paulin-Mohring, C.: Proofs of randomized algorithms in Coq. Sci. Comput. Prog. 74(8), 568–589 (2009)
Barthe, G., Crespo, J.M., Grégoire, B., Kunz, C., Zanella Béguelin, S.: Computer-Aided Cryptographic Proofs. In: Proceedings of the 3rd International Conference on Interactive Theorem Proving (ITP 2012), Princeton, NJ, USA, August 2012. Lecture Notes in Computer Science, vol. 7406, pp. 11–27. Springer, Heidelberg (2012)
Bertot, Y., Gonthier, G., Ould Biha, S., Pasca, I.: Canonical Big Operators. In: Proceedings of the 21st International Conference on Theorem Proving in Higher Order Logics (TPHOLs 2008), Montreal, Canada, August 18–21, 2008. Lecture Notes in Computer Science, vol. 5170, pp. 86–101. Springer, Heidelberg (2008)
Coble, A.R.: Anonymity, Information, and Machine-Assisted Proof. PhD Thesis, King’s College. University of Cambridge, UK (2010)
The Coq Development Team. Reference Manual. Version 8.4. Available at http://coq.inria.fr. INRIA (2004–2012)
Cover, T.M., Thomas, J.A.: Elements of Information Theory, 2nd edn. Wiley-Interscience (2006)
Csiszár, I.: The method of types. IEEE Trans. Inform. Theory 44(6), 2505–2523 (1998)
Csiszár, I., Körner, J.: Information Theory—Coding Theorems for Discrete Memoryless Systems, 2nd edn. Cambridge University Press (2011)
Gonthier, G., Mahboubi, A., Tassi, E.: A Small Scale Reflection Extension for the Coq system. Version 10. Technical Report RR-6455, INRIA (2011)
Hagiwara, M.: Coding Theory: Mathematics for Digital Communication. In Japanese. http://www.nippyo.co.jp/book/5977.html. Nippon Hyoron Sha (2012)
Hasan, O., Tahar, S.: Verification of expectation using theorem proving to verify expectation and variance for discrete random variables. J. Autom. Reason. 41, 295–323 (2008)
Hölzl, J., Heller, A.: Three Chapters of Measure Theory in Isabelle/HOL. In: Proceedings of the 2nd International Conference on Interactive Theorem Proving (ITP 2011), Berg en Dal, The Netherlands, August 22–25, 2011. Lecture Notes in Computer Science, vol. 6898, pp. 135–151. Springer, Heidelberg (2011)
Hurd, J.: Formal Verification of Probabilistic Algorithms. PhD Thesis, Trinity College, University of Cambridge, UK (2001)
Mhamdi, T., Hasan, O., Tahar, S.: On the Formalization of the Lebesgue Integration Theory in HOL. In: Proceedings of the 1st International Conference on Interactive Theorem Proving (ITP 2010), Edinburgh, UK, July 11–14, 2010. Lecture Notes in Computer Science, vol. 6172, pp. 387–402. Springer, Heidelberg (2010)
Mhamdi, T., Hasan, O., Tahar, S.: Formalization of Entropy Measures in HOL. In: Proceedings of the 2nd International Conference on Interactive Theorem Proving (ITP 2011), Berg en Dal, The Netherlands, August 22–25, 2011. Lecture Notes in Computer Science, vol. 6898, pp. 233–248. Springer, Heidelberg (2011)
Khudanpur, S.: Information Theoretic Methods in Statistics. Lecture Notes. Available at http://old-site.clsp.jhu.edu/sanjeev/520.674/notes/GISAlgorithm.pdf (1999). Accessed 02 May 2013
Shannon, C.E.: A Mathematical theory of communication. Bell Syst. Tech. J. 27, 379–423 and 623–656 (1948)
Shannon, C.E.: Communication theory of secrecy systems. Bell Sys. Tech. J. 28, 656–715 (1949)
Uyematsu, T.: Modern Shannon Theory, Information Theory with Types. In Japanese. Baifukan (1998)
Verdú, S.: Fifty years of Shannon theory. IEEE Trans. Inform. Theory 44(6), 2057–2078 (1998)
Author information
Authors and Affiliations
Corresponding author
Additional information
This article is a revised and extended version of a conference paper [1].
This work was essentially carried out when the second and third author were affiliated with Research Institute for Secure Systems, National Institute of Advanced Industrial Science and Technology, Japan.
Rights and permissions
About this article
Cite this article
Affeldt, R., Hagiwara, M. & Sénizergues, J. Formalization of Shannon’s Theorems. J Autom Reasoning 53, 63–103 (2014). https://doi.org/10.1007/s10817-013-9298-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10817-013-9298-1