Abstract
Most pattern recognition problems are solved by highly task specific algorithms. However, all recognition and classification architectures are related in at least one aspect: They rely on compressed representations of the input. It is therefore an interesting question how much compression itself contributes to the pattern recognition process. The question has been answered by Benedetto et al. (2002) for the domain of text, where a common compression program (gzip) is capable of language recognition and authorship attribution. The underlying principle is estimating the mutual information from the obtained compression factor. While this principle appears to be well-suited for strings of symbols, it was to date believed to be not applicable to continuous valued real world sensory data. But here we show that compression achieves astonishingly high recognition rates even for complex tasks like visual object recognition, texture classification, and image retrieval. Though, naturally, specialized recognition algorithms still outperform compressors, our results are remarkable, since none of the applied compression programs (gzip, bzip2) was ever designed to solve this type of tasks. Compression is the only known method that solves such a wide variety of tasks without any modification, data preprocessing, feature extraction, even without parametrization. We conclude that compression can be seen as the “core” of a yet to develop theory of unified pattern recognition.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Benedetto, D., Caglioti, E., Loreto, V.: Language Trees and Zipping. Phys. Rev. Lett. 88(4) (2002)
Benedetto, D., Caglioti, E., Loreto, V.: Zipping out relevant information. Computing in Science and Engineering 5, 80–85 (2003)
Cho, A.: Reading the Bits of Shakespeare. ScienceNOW (January 24, 2002)
Ball, P.: Algorithm makes tongue tree. Nature Science Update (2002)
Khmelev, D.V., Teahan, W.J.: Comment on Language Trees and Zipping. Physical Review Letters 90(8), 89803–1 (2003)
Lempel, A., Ziv, J.: A Universal Algorithm for Sequential Data Compression. IEEE Trans. Inf. Th. 23(3), 337–343 (1977)
Burrows, M., Wheeler, D.J.: A Block-sorting Lossless Data Compression Algorithm. Research Report 124, Digital Systems Research Center (1994)
Hirschberg, D.S., Lelewer, D.A.: Efficient Decoding of Prefix Codes. Communications of the ACM 33(4), 449–459 (1990)
Sinkkonen, J., Kaski, S.: Clustering Based on Conditional Distributions in an Auxiliary Space. Neural Computation 14(1), 217–239 (2002)
Hulle, M.M.V.: Joint Entropy Maximization in Kernel-Based Topographic Maps. Neural Computation 14(8), 1887–1906 (2002)
Imaoka, H., Okajima, K.: An Algorithm for the Detection of Faces on the Basis of Gabor Features and Information Maximization. Neural Computation 16(6), 1163–1191 (2004)
Erdogmus, D., Hild, K.E., Rao, Y.N., Príncipe, J.C.: Minimax Mutual Information Approach for Independent Component Analysis. Neural Computation 16(6), 1235–1252 (2004)
Wyner, A.D.: 1994 Shannon Lecture. Typical Sequences and All That: Entropy, Pattern Matching, and Data Compression, AT & T Bell Laboratories, Murray Hill, New Jersey, USA (1994)
Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, New York (1991)
Rissanen, J.: Modeling by Shortest Data Description. Automatica 14, 465–471 (1978)
Vitanyi, P.M.B., Li, M.: Ideal MDL and its Relation to Bayesianism. In: Proc. ISIS: Information, Statistics and Induction in Science, pp. 282–291. World Scientific, Singapore (1996)
Leclerc, Y.G.: Constructing simple stable descriptions for image partitioning. Int’l J. of Computer Vision 3, 73–102 (1989)
Keeler, A.: Minimal length encoding of planar subdivision topologies with application to image segmentation. In: AAAI 1990 Spring Symposium of the Theory and Application of Minimal Length Encoding (1990)
Kanungo, T., Dom, B., Niblack, W., Steele, D.: A fast algorithm for MDL-based multi-band image segmentation. In: Proc. Conf. Computer Vision and Pattern Recognition CVPR (1994)
Nene, S.A., Nayar, S.K., Murase, H.: Columbia Object Image Library: COIL-100. Technical Report CUCS-006-96, Dept. Computer Science, Columbia Univ. (1996)
Picard, R., Graczyk, C., Mann, S., Wachman, J., Picard, L., Campbell, L.: Vision Texture Database (VisTex). Copyright 1995 by the Massachusetts Institute of Technology (1995)
Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-Based Image Retrieval at the End of the Early Years. IEEE Trans. on Pattern Analysis and Machine Intelligence 22(12), 1349–1380 (2000)
Corel: Corel GALLERYTM Magic 65000, Corel Corp., 1600 Carling Ave., Ottawa, Ontario, Canada K1Z 8R7 (1997)
Tarr, M.J., Bülthoff, H.H.: Image-Based Object Recognition in Man, Monkey and Machine. Cognition 67, 1–20 (1998)
Murase, H., Nayar, S.K.: Visual Learning and Recognition of 3-D Objects from Appearance. Int’l J. of Computer Vision 14, 5–24 (1995)
Paulus, D., Ahrlichs, U., Heigl, B., Denzler, J., Hornegger, J., Zobel, M., Niemann, H.: Active Knowledge-Based Scene Analysis. Videre 1(4) (2000)
Rui, Y., Huang, T.S., Chang, S.F.: Image Retrieval: Current Techniques, Promising Directions and Open Issues. J. of Visual Communications and Image Representation 10, 1–23 (1999)
Laaksonen, J.T., Koskela, J.M., Laakso, S.P., Oja, E.: PicSOM – Content-Based Image Retrieval with Self-Organizing Maps. Pattern Recognition Letters 21(13-14), 1199–1207 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Heidemann, G., Ritter, H. (2009). Data Compression - A Generic Principle of Pattern Recognition?. In: Ranchordas, A., Araújo, H.J., Pereira, J.M., Braz, J. (eds) Computer Vision and Computer Graphics. Theory and Applications. VISIGRAPP 2008. Communications in Computer and Information Science, vol 24. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-10226-4_16
Download citation
DOI: https://doi.org/10.1007/978-3-642-10226-4_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-10225-7
Online ISBN: 978-3-642-10226-4
eBook Packages: Computer ScienceComputer Science (R0)