Abstract
Gaussian mixtures (i.e. linear combinations of multivariate Gaussian probability densities) appear in numerous applications due to their universal ability to approximate multimodal probability distributions. Finding the modes (maxima) of a Gaussian mixture is a fundamental problem arising in many practical applications such as machine learning and digital image processing. In this paper, we propose a computationally efficient method for finding a significant mode of the Gaussian mixture. Such a mode represents an area of large probability, and it often coincides with the global mode of the mixture. The proposed method uses a Gaussian convolution in order to remove undesired local maxima of the Gaussian mixture and preserve its underlying structure. The transformation between the maximizers of the smoothed Gaussian mixture and the original one is formulated as a differential equation. A robust trust region method for tracing the solution curve of this differential equation is described. Our formulation also allows mixtures with negative weights or even negative values, which occur in some applications related to machine learning or quantum mechanics. The applicability of the method to mode-finding of Gaussian kernel density estimates obtained from experimental data is illustrated. Finally, some numerical results are given to demonstrate the ability of the method to find significant modes of Gaussian mixtures and kernel density estimates.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abramowitz M., Stegun I.: Handbook of Mathematical Functions. Dover, New York (1964)
Allgover E.L., Georg K.: Numerical Continuation Methods: An Introduction. Springer, Berlin (1990)
Bishop C.M., Svensén M., Williams C.K.I.: GTM: the generative topographic mapping. Neural Comput. 10(1), 215–234 (1998)
Carreira-Perpiñán, M.A.: Reconstruction of sequential data with probabilistic models and continuity constraints. In: Solla, S.A., Leen, T.K., Müller, K.R. (eds.) Advances in Neural Information Processing Systems, vol. 12, pp. 414–420 (1999)
Carreira-Perpiñán M.A.: Mode-finding for mixtures of Gaussian distributions. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1318–1323 (2000)
Carreira-Perpiñán, M.A., Renals, S.: A latent variable modelling approach to the acoustic-to-articulatory mapping problem. In: Ohala, J.J., Hasegawa, Y., Ohala, M., Granville, D., Bailey, A.C. (eds.) 14th International Congress of Phonetic Sciences (ICPhS’99), pp. 2013–2016 (1999)
Chaudhuri P., Marron J.S.: Sizer for exploration of structures in curves. J. Am. Stat. Assoc. 94(447), 807–823 (1999)
Cheng Y.: Mean shift, mode seeking, and clustering. Pattern Anal. Mach. Intell. 17(8), 790–799 (1995)
Duong T., Hazelton M.L.: Cross-validation bandwidth matrices for multivariate kernel density estimation. Scand. J. Stat. 32(3), 485–506 (2005)
Enzweiler, M., Gavrila, D.M.: Integrated pedestrian classification and orientation estimation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 982–989 (2010)
Fan J., Yao Q.: Nonlinear Time Series: Nonparametric and Parametric Methods. Springer, Berlin (2005)
Hall P., Sheather S.J., Jones M.C., Marron J.S.: On optimal data-based bandwidth selection in kernel density estimation. Biometrika 78(2), 263–269 (1991)
Han B., Davis L.: On-line density-based appearance modeling for object tracking. Tenth IEEE Int. Conf. Comput. Vis. 2, 1492–1499 (2005)
Hartle J.B.: Quantum mechanics with extended probabilities. Phys. Rev. A 78(1), 1–13 (2008)
Horn R.A., Johnson C.R.: Matrix Analysis. Cambridge University Press, Cambridge (1985)
Härdle W., Müller M., Sperlich S., Wervatz A.: Nonparametric and Semiparametric Models. Springer Series in Statistics. Springer, Berlin (2004)
Jain A.K., Duin R.P.W., Mao J.: Statistical pattern recognition: a review. IEEE Trans. Pattern Anal. Mach. Intell. 22(1), 4–37 (2000)
Jones D.R., Schonlau M., Welch W.J.: Efficient global optimization of expensive black-box functions. J. Glob. Optim. 13(4), 455–492 (1998)
Jones M.C., Marron J.S., Sheather S.J.: A brief survey of bandwidth selection for density estimation. J. Am. Stat. Assoc. 91(433), 401–407 (1996)
Koch W.: On “negative” information in tracking and sensor data fusion: discussion of selected examples. Seventh Int. Conf. Inf. Fusion 2, 1–20 (2004)
Marron J.S., Wand M.P.: Exact mean integrated squared error. Ann. Stat. 20(2), 712–736 (1992)
Moré J.J., Wu Z.: Global continuation for distance geometry problems. SIAM J. Optim. 7(3), 814–836 (1997)
Mückenheim W., Ludwig G., Dewdney C., Holland P.R., Kyprianidis A., Vigier J.P., Petroni N.C., Bartlett M.S., Jaynes E.T.: A review of extended probabilities. Phys. Rep. 133(6), 337–401 (1986)
Nocedal J., Wright S.J.: Numerical Optimization, 2nd edn. Springer, New York (2006)
Paris, S., Durand, F.: A topological approach to hierarchical segmentation using mean shift. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)
Scott D.W.: Multivatiate Density Estimation: Theory Practice and Visualization. Wiley, New York (1992)
Sheather S.J., Jones M.C.: A reliable data-based bandwidth selection method for kernel density estimation. J. R. Stat. Soc. Ser. B (Methodological) 53(3), 683–690 (1991)
Shen C., Brooks M.J., van den Hengel A.: Fast global kernel density mode seeking: applications to localization and tracking. IEEE Trans. Image Process. 16(5), 1457–1469 (2007)
Silverman B.W.: Density Estimation for Statistics and Data Analysis. Chapman and Hall, London (1986)
Steihaug T.: The conjugate gradient method and trust regions in large scale optimization. SIAM J. Numer. Anal. 20(3), 626–637 (1983)
Stewart G.W.: Matrix Algorithms, vol. II: Eigensystems. SIAM, Philadelphia (2001)
Titterington D.M., Smith A.F.M., Makov U.E.: Statistical Analysis of Finite Mixture Distributions. Wiley, Chichester (1985)
Wand M.P., Jones M.C.: Kernel Smoothing. Chapman and Hall, London (1995)
Wu Z.: The effective energy transformation scheme as a special continuation approach to global optimization with application to molecular conformation. SIAM J. Optim. 6(3), 748–768 (1996)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Pulkkinen, S., Mäkelä, M.M. & Karmitsa, N. A continuation approach to mode-finding of multivariate Gaussian mixtures and kernel density estimates. J Glob Optim 56, 459–487 (2013). https://doi.org/10.1007/s10898-011-9833-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10898-011-9833-8