Abstract
The connection between text compression and the measure of entropy of a source seems to be well known but poorly documented. We try to partially remedy this situation by showing that the topological entropy is a lower bound for the compression ratio of any compressor. We show that for factorial sources the 1978 version of the Ziv-Lempel compression algorithm achieves this lower bound.
This work was done with partial support from FAPESP and from BID/USP (project 30.01).
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
R. Adler, A. Konheim, and M. McAndrew, Topological entropy. Trans. Amer. Math. Soc, 114:309–319, 1965.
T. C. Bell, J. G. Cleary, and I. H. Witten. Text Compression. Prentice Hall, Englewood Cliffs, N.J., 1990.
N. Chomsky and G. A. Miller. Finite state languages. Information and Control 1:91–112, 1958.
A. de Luca. On the entropy of a formal language. In H. Brakhage, editor, Automata Theory and Formal Languages, pages 103–109, Springer-Verlag, Berlin, 1975. Lecture Notes in Computer Science, 33.
A. V. Goldberg and M. Sipser. Compression and ranking. SIAM J. Comput., 20:524–536, 1991.
G. Hansel. Estimation of the entropy by the Lempel-Ziv method. In M. Gross and D. Perrin, editors, Electronic Dictionaries and Automata in Computational Linguistics, pages 51–65, Springer-Verlag, Berlin, 1989. Lecture Notes in Computer Science, 377.
A. I. Khinchin. Mathematical Foundations of Information Theory. Dover, New York, 1957.
D. E. Knuth. The Art of Computer Programming, Vol. 1, Fundamental Algorithms. Addison-Wesley Pu. Co., Reading, MA, 1968.
W. Kuich. On the entropy of context-free languages. Information and Control, 16:173–200, 1970.
M. Lothaire. Combinatorics on Words. Volume 17 of Encyclopedia of Mathematics and its Applications, Addison-Wesley Pu. Co., Reading, MA, 1983.
N. F. G. Martin and J. W. England. Mathematical Theory of Entropy. Volume 12 of Encyclopedia of Mathematics and its Applications, Addison-Wesley Pu. Co., Reading, MA, 1981.
S. E. Newhouse. Entropy in smooth dynamical systems. 1990. To appear in Proceedings of the 1990 World Congress of Mathematicians.
D. Ornstein and B. Weiss. Entropy and data compression schemes. 1990. manuscript.
D. S. Ornstein and P. C. Shields. Universal almost sure data compression. The Annals of Probability, 18:441–452, 1990.
A. Rényi. A Diary on Information Theory. John Wiley & Sons, New York, NY, 1984.
C. E. Shannon. A mathematical theory of communication. Bell System Technical J., 27:398–403, 1948.
J. A. Storer. Data Compression—Methods and Theory. Computer Science Press, Rockville,MD, 1988.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Rights and permissions
Copyright information
© 1992 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hansel, G., Perrin, D., Simon, I. (1992). Compression and entropy. In: Finkel, A., Jantzen, M. (eds) STACS 92. STACS 1992. Lecture Notes in Computer Science, vol 577. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-55210-3_209
Download citation
DOI: https://doi.org/10.1007/3-540-55210-3_209
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-55210-9
Online ISBN: 978-3-540-46775-5
eBook Packages: Springer Book Archive