Abstract
Density-based nonparametric clustering techniques, such as the mean shift algorithm, are well known for their flexibility and effectiveness in real-world vision-based problems. The underlying kernel density estimation process can be very expensive on large datasets. In this paper, the divide-and-conquer method is proposed to reduce these computational requirements. The dataset is first partitioned into a number of small, compact clusters. Components of the kernel estimator in each local cluster are then fit to a single, representative density function. The key novelty presented here is the efficient derivation of the representative density function using concepts from function approximation, such that the expensive kernel density estimator can be easily summarized by a highly compact model with very few basis functions. The proposed method has a time complexity that is only linear in the sample size and data dimensionality. Moreover, the bandwidth of the resultant density model is adaptive to local data distribution. Experiments on color image filtering/segmentation show that, the proposed method is dramatically faster than both the standard mean shift and fast mean shift implementations based on kd-trees while producing competitive image segmentation results.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Barbay, J., Golynski, A., Munro, J.I., Rao, S.S., 2007. Adaptive searching in succinctly encoded binary relations and tree-structured documents. Theor. Comput. Sci., 387(3):284–297. [doi:10.1016/j.tcs.2007.07.015]
Bouezmarni, T., Rombouts, J.V.K., 2010. Nonparametric density estimation for multivariate bounded data. J. Statist. Plan. Infer., 140(1):139–152. [doi:10.1016/j.jspi. 2009.07.013]
Chang, D.X., Zhang, X.D., Zheng, C.W., 2009. A genetic algorithm with gene rearrangement for k-means clustering. Pattern Recogn., 42(7):1210–1222. [doi:10. 1016/j.patcog.2008.11.006]
Chang, H., Yeung, D.Y., 2008. Robust path-based spectral clustering. Pattern Recogn., 41(1):191–203. [doi:10.1016/ j.patcog.2007.04.010]
Comaniciu, D., Meer, P., 2002. Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell., 24(5):603–619. [doi:10.1109/34.1000236]
de Berg, M., van Kreveld, M., Overmars, M., Cheong, O., 2008. Computational Geometry: Algorithms and Applications. Springer-Verlag, Berlin, Germany, p.105–120.
Fashing, M., Tomasi, C., 2005. Mean shift is a bound optimization. IEEE Trans. Pattern Anal. Mach. Intell., 27(3):471–474. [doi:10.1109/TPAMI.2005.59]
Georgescu, B., Shimshoni, I., Meer, P., 2003. Mean Shift Based Clustering in High Dimensions: a Texture Classification Example. Proc. 9th IEEE Int. Conf. on Computer Vision, p.456–463. [doi:10.1109/ICCV.2003. 1238382]
Han, B., Comaniciu, D., Zhu, Y., Davis, L., 2004. Incremental Density Approximation and Kernel-Based Bayesian Filtering for Object Tracking. Proc. IEEE Computer Society Conf. on Computer Version and Pattern Recognition, p.638–644. [doi:10.1109/CVPR.2004.1315092]
Mokkadem, A., Pelletier, M., Slaoui, Y., 2009. The stochastic approximation method for the estimation of a multivariate probability density. J. Statist. Plan. Infer., 139(7):2459–2478. [doi:10.1016/j.jspi.2008.11.012]
Ozertem, U., Erdogmus, D., Jenssen, R., 2008. Mean shift spectral clustering. Pattern Recogn., 41(6):1924–1938. [doi:10.1016/j.patcog.2007.09.009]
Parzen, E., 1962. On estimation of a probability density function and mode. Ann. Math. Statist., 33(3):1065–1076. [doi:10.1214/aoms/1177704472]
Rao, S., de Martins Martins, A., Principe, J.C., 2009. Mean shift: an information theoretic perspective. Pattern Recogn. Lett., 30(3):222–230. [doi:10.1016/j.patrec.2008. 09.011]
Ren, W., Singh, S., Singh, M., Zhu, Y.S., 2009. State of the art on spatio-temporal information based video retrieval. Pattern Recogn., 42(2):267–282. [doi:10.1016/j.patcog. 2008.08.033]
Ruslan, S., Sam, R., 2003. Adaptive Overrelaxed Bound Optimization Methods. Proc. 20th Int. Conf. on Machine Learning, p.664–671.
Shen, C., Brooks, M., 2005. Adaptive Over-Relaxed Mean Shift. Proc. 8th Int. Symp. on Signal Processing and Its Applications, p.575–578. [doi:10.1109/ISSPA.2005.1581 003]
Shen, C., Brooks, M., van den Hengel, A., 2007. Fast global kernel density mode seeking: application to localisation and tracking. IEEE Trans. Image Process., 16(5):1457–1469. [doi:10.1109/TIP.2007.894233]
Wang, X.H., Liu, J.L., 2009. Tracking multiple people under occlusion and across cameras using probabilistic models. J. Zhejing Univ.-Sci. A, 10(7):985–996. [doi:10.1631/jzus. A0820474]
Xu, G., Xu, J.H., 2010. Efficient approximation algorithms for clustering point-sets. Comput. Geom., 43(1):59–66. [doi:10.1016/j.comgeo.2007.12.002]
Yu, S.Y., Wang, F.L., Xue, Y.F., Yang, J., 2009. Bayesian moving object detection in dynamic scenes using an adaptive foreground model. J. Zhejing Univ.-Sci. A, 10(12):1750–1758. [doi:10.1631/jzus.A0820743]
Zhang, J., Zhang, K., Xu, X., Tse, C.K., Small, M., 2009. Seeding the kernels in graphs: towards multi-resolution community analysis. New J. Phys., 11(11):113003. [doi:10.1088/1367-2630/11/11/113003]
Zhang, K., Kwok, J.T., 2007. Simplifying Mixture Models Through Function Approximation. Advances in Neural Information Processing Systems 19. MIT Press, Cambridge, MA, p.1577–1584.
Zhang, K., Tang, M., Kwok, J.T., 2005. Applying Neighborhood Consistency for Fast Clustering and Kernel Density Estimation. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, 2:1001–1007. [doi:10.1109/CVPR.2005.73]
Zivkovic, Z., Cemgil, A.T., Krose, B., 2009. Approximate Bayesian methods for kernel-based object tracking. Comput. Vis. Image Understand., 113(6):743–749. [doi:10.1016/j.cviu.2008.12.008]
Author information
Authors and Affiliations
Corresponding author
Additional information
Project (No. 9140C1204060809) supported by the National Key Laboratory Foundation of China
Rights and permissions
About this article
Cite this article
Jin, Yx., Zhang, K., Kwok, J.T. et al. Fast and accurate kernel density approximation using a divide-and-conquer approach. J. Zhejiang Univ. - Sci. C 11, 677–689 (2010). https://doi.org/10.1631/jzus.C0910668
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1631/jzus.C0910668