Abstract
Information theoretic measures provide quantitative entropic divergences between two probability distributions or data sets. In this paper, we analyze the theoretical properties of the Jensen-Rényi divergence which is defined between any arbitrary number of probability distributions. Using the theory of majorization, we derive its maximum value, and also some performance upper bounds in terms of the Bayes risk and the asymptotic error of the nearest neighbor classifier. To gain further insight into the robustness and the application of the Jensen-Rényi divergence measure in imaging, we provide substantial numerical experiments to show the power of this entopic measure in image registration and segmentation.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Ali, S., Silvey, S.: A general class of coefficients of divergence of one distribution from another. J. Roy. Soc. 28, 131–142 (1966)
Kullback, S., Liebler, R.: On information and sufficiency. Ann. Math. Statist. 22, 79–86 (1951)
Stoica, R., Zerubia, J.: Image retrieval and indexing: A hierarchical approach in computing the distance between textured images. In: IEEE Int. Conf. on Image Processing, Chicago (1998)
Hero, A.O., Ma, B., Michel, O., Gorman, J.: Applications of entropic spanning graphs. IEEE Signal Processing Magazine 19(5), 85–95 (2002)
Rényi, A.: On Measures of Entropy and Information. Selected Papers of Alfréd Rényi 2, 525–580 (1961)
Lin, J.: Divergence Measures Based on the Shannon Entropy. IEEE Trans. Information Theory 37(1), 145–151 (1991)
Gomez, J.F., Martinez, J., Robles, A.M., Roman, R.: An analysis of edge detection by using the Jensen-Shannon divergence. Journal of Mathematical Imaging and Vision 13, 35–56 (2000)
Roman, R., Bernaola, P., Oliver, J.L.: Sequence compositional complexity of DNA through an entropic segmentation method. Physical Review Letters 80(6), 1344–1347 (1998)
Viola, P., Wells, W.M.: A lignment by maximization of mutual information. International Journal of Computer Vision 24(2), 154–173 (1997)
Maes, F., Collignon, A., Vandermeulen, D., Marchal, G., Suetens, P.: Multimodality image registration by maximization of mutual information. IEEE Trans. on Medical Imaging 16(2), 187–198 (1997)
He, Y., Ben Hamza, A., Krim, H., Chen, V.C.: An information theoretic measure for ISAR imagery focusing. In: Proc. SPIE, San Diego, vol. 4116 (2000)
He, Y., Ben Hamza, A., Krim, H.: A generalized divergence measure for robust image registration. IEEE Trans. Signal Processing 51(5) (2003)
Marshall, A.W., Olkin, I.: Inequalities: Theory of Majorization and Its Applications. Academic Press, London (1979)
Devroye, L., Gyorfi, L., Lugosi, G.: A probabilistic theory of pattern recognition, New York. Springer, New York (1996)
Figueiredo, M.A., Jain, A.K.: Unsupervised learning of finite mixture models. IEEE Trans. on pattern analysis and machine intelligence 24(3), 381–396 (2002)
Paragios, N., Deriche, R.: Geo desic active contours and level sets for the detection and tracking of moving objects. IEEE Trans. on pattern analysis and machine intelligence 22(3), 266–280 (2000)
Hellman, M., Raviv, J.: Probability of error, equivocation, and the Chernoff bound. IEEE Trans. Information Theory 16, 368–372 (1970)
Cover, T.M., Hart, P.E.: Nearest neighbor pattern classification. IEEE Trans. Inform. Theory 13, 21–27 (1967)
Katuri, R., Jain, R.C.: Computer Vision: Principles. IEEE Computer Society Press, Los Alamitos (1991)
Jensen, J.R.: Introductory digital image processing: a remote sensing perspective, 2nd edn. Prentice Hall, Upper Saddle River (1996)
Van den Elsen, P.A., Pol, E.J.D., Viergever, M.A.: Medical image matching-a review with classification. IEEE Engineering in Medicine and Biology Magazine 12(1), 26–39 (1993)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hamza, A.B., Krim, H. (2003). Image Registration and Segmentation by Maximizing the Jensen-Rényi Divergence. In: Rangarajan, A., Figueiredo, M., Zerubia, J. (eds) Energy Minimization Methods in Computer Vision and Pattern Recognition. EMMCVPR 2003. Lecture Notes in Computer Science, vol 2683. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-45063-4_10
Download citation
DOI: https://doi.org/10.1007/978-3-540-45063-4_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40498-9
Online ISBN: 978-3-540-45063-4
eBook Packages: Springer Book Archive