Abstract
We address the problem of entropy estimation for high-dimensional finite-accuracy data. Our main application is evaluating high-order mutual information image similarity criteria for multimodal image registration. The basis of our method is an estimator based on k-th nearest neighbor (NN) distances, modified so that only distances greater than some constant R are evaluated. This modification requires a correction which is found numerically in a preprocessing step using quadratic programming. We compare experimentally our new method with k-NN and histogram estimators on synthetic data as well as for evaluation of mutual information for image similarity.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Beirlant, J., Dudewicz, E.J.L.G., van der Meulen, E.C.: Nonparametric entropy estimation: an overview. International J. Math. Stat. Sci. (6), 17–39 (1997)
Harner, E.J., Singh, H., Li, S., Tan, J.: Computational challenges in computing nearest neighbor estimates of entropy for large molecules. Computing Science and Statistics 35 (2003)
Victor, J.D.: Binless strategies for estimation of information from neural data. Physical Review E, 66(5), 051903(15) (2002)
Viola, P., Wells III, W.M.: Alignment by maximization of mutual information. International Journal of Computer Vision (2), 137–154 (1997)
Pluim, J.P.W., Maintz, J.B.A., Viergever, M.A.: Mutual-information-based registration of medical images: A survey. IEEE Transactions on Medical Imaging 22(8), 986–1004 (2003)
Maes, F., Collignon, A., Vandermeulen, D., Marchal, G., Suetens, P.: Multimodality image registration by maximization of mutual information. IEEE Transactions on Medical Imaging 16(2), 187–198 (1997)
Pluim, J.P.W., Maintz, J.B.A., Viergever, M.A.: Image registration by maximization of combined mutual information and gradient information. IEEE Transactions Med. Imag. 19(8) (August 2000)
Rueckert, D., Clarkson, M.J., Hill, D.L.G., Hawkes, D.J.: Non-rigid registration using higher-order mutual information. In: Proceedings of SPIE Medical Imaging,: Image Processing 2000, pp. 438–447 (2000)
Russakoff, D.B., Tomasi, C., Rohlfing, T., Maurer, Jr,C.R.: Image similarity using mutual information of regions. In: Pajdla, T., Matas, J. (eds.) ECCV 2004. LNCS, vol. 3023, pp. 596–607. Springer, Heidelberg (2004)
Sabuncu, M.R., Ramadge, P.J.: Spatial information in entropy-based image registration. In: Gee, J.C., Maintz, J.B.A., Vannier, M.W. (eds.) WBIR 2003. LNCS, vol. 2717, pp. 132–141. Springer, Heidelberg (2003)
Darbellay, G.A., Vajda, I.: Estimation of the information by an adaptive partitioning of the observation space. IEEE Transactions on Information Theory 45(4), 1315–1321 (1999)
Miller, E.G.: A new class of entropy estimators for multi-dimensional densities. In: Proceedings of ICASSP2003 (2003)
Scott, D.W. (ed.): Multivariate Density Estimation : Theory, Practice, and Visualization. In: Wiley Series in Probability and Mathematical Statistics, John Wiley & Sons, New York (1992)
Kozachenko, L.F., Leonenko, N.N.: On statistical estimation of entropy of random vector. Probl. Inf. Trans. (in Russian) 23(9) (1987)
Kraskov, A., Stögbauer, H., Grassberger, P.: Estimating mutual information. Physical Review E, (69 066138) (2004)
Singh, H., Misra, N., Hnizdo, V., Fedorowicz, A., Demchuk, E.: Nearest neighbor estimates of entropy. American journal of mathematical and management sciences 23(3–4), 301–321 (2003)
Kybic, J.: Incremental updating of nearest neighbor-based high-dimensional entropy estimation. In: Duhamel, P., Vandendorpe, L. eds.: ICASSP2006, Toulouse, France, IEEE, III–804 DVD proceedings (2006)
Callahan, P.B., Kosaraju, S.R.: A decomposition of multi-dimensional point-sets with applications to k-nearest-neighbors and n-body potential fields. In: Proceedings 24th Annual AMC Symposium on the Theory of Computing 1992. pp. 546–556 (1992)
Smid, M.: Closest-point problems in computational geometry. In: Sack, J.-R.(ed.) Handbook on Computational Geometry, North Holland, Amsterdam (To appear 1997)
Beis, J.S., Lowe, D.G.: Shape indexing using approximate nearest-neighbour search in high-dimensional spaces. In: Proceedings of Conference on Computer Vision and Pattern Recognition, June 1997, pp. 1000–1006 (1997)
Preparata, F.P., Shamos, M.I.: Computational geometry: An introduction. In: Texts and Monographs in Computer Science, Springer, Heidelberg (1985)
Sedgewick, R.: Algorithms. Addison-Wesley, Reading (1989)
Hero, A.O., Ma, B., Michel, O., Gorman, J.: Applications of entropic spanning graphs. IEEE Signal Proc. Magazine 19(5), 85–95 (2002)
Neemuchwala, H., Hero, A., Carson, P.: Image matching using alpha-entropy measures and entropic graphs. Signal Process. 85(2), 277–296 (2005)
Šára, R.: A modification of Kozachenko-Leonenko entropy estimator for quantized data. (Unpublished notes 2006)
Neumaier, A.: MINQ — general definite and bound constrained indefinite quadratic programming (1998), http://www.mat.univie.ac.at/neum/software/minq/
Johnson, K.A., Becker, J.A.: (The whole brain atlas) http://www.med.harvard.edu/AANLIB/
García-Arteaga, J.D., Kybic, J., Li, W.: Elastic image registration for movement compensation in digital colposcopy. In Jan, J., Kozumplík, J., Provazník, I., (eds.): Buiosignal: Analysis of Biomedical Signals and Images, Brno, Czech Republic June 2006, Eurasip, pp. 236–238 Vutium Press (2006)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Kybic, J. (2007). High-Dimensional Entropy Estimation for Finite Accuracy Data: R-NN Entropy Estimator. In: Karssemeijer, N., Lelieveldt, B. (eds) Information Processing in Medical Imaging. IPMI 2007. Lecture Notes in Computer Science, vol 4584. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73273-0_47
Download citation
DOI: https://doi.org/10.1007/978-3-540-73273-0_47
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-73272-3
Online ISBN: 978-3-540-73273-0
eBook Packages: Computer ScienceComputer Science (R0)