Skip to main content
SpringerLink
Log in

Nonlinear Mean Shift over Riemannian Manifolds

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

The original mean shift algorithm is widely applied for nonparametric clustering in vector spaces. In this paper we generalize it to data points lying on Riemannian manifolds. This allows us to extend mean shift based clustering and filtering techniques to a large class of frequently occurring non-vector spaces in vision. We present an exact algorithm and prove its convergence properties as opposed to previous work which approximates the mean shift vector. The computational details of our algorithm are presented for frequently occurring classes of manifolds such as matrix Lie groups, Grassmann manifolds, essential matrices and symmetric positive definite matrices. Applications of the mean shift over these manifolds are shown.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Absil, P.-A., Mahony, R., & Sepulchre, R. (2003). Riemannian geometry of Grassmann manifolds with a view on algorithmic computation. Acta Applicandae Mathematicae, 80(2), 199–220.

    Article  MathSciNet  Google Scholar 

  • Arsigny, V., Fillard, P., Pennec, X., & Ayache, N. (2006). Geometric means in a novel vector space structure on symmetric positive-definite matrices. SIAM Journal on Matrix Analysis and Applications, 29(1), 328–347.

    Article  MathSciNet  Google Scholar 

  • Basser, P. J., Mattiello, J., & LeBihan, D. (1994). MR diffusion tensor spectroscopy and imaging. Biophysical Journal, 66, 259–267.

    Article  Google Scholar 

  • Begelfor, E., & Werman, M. (2006). Affine invariance revisited. In Proc. IEEE conf. on computer vision and pattern recognition, New York, NY (Vol. II, pp. 2087–2094).

  • Birchfield, S., & Rangarajan, S. (2005). Spatiograms vs. histograms for region-based tracking. In Proc. IEEE conf. on computer vision and pattern recognition, San Diego, CA, June 2005 (Vol. II, pp. 1158–1163).

  • Carreira-Perpinan, M. A. (2007). Gaussian mean-shift is an EM algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(5), 767–776.

    Article  Google Scholar 

  • Chen, H., & Meer, P. (2005). Robust fusion of uncertain information. IEEE Transactions on Systems, Man, Cybernetics Part B, 35, 578–586.

    Article  Google Scholar 

  • Cheng, Y. (1995). Mean shift, mode seeking, and clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17, 790–799.

    Article  Google Scholar 

  • Chikuse, Y. (2003). Statistics on special manifolds. Berlin: Springer.

    MATH  Google Scholar 

  • Christoudias, C. M., Georgescu, B., & Meer, P. (2002). Synergism in low level vision. In Proc. 16th intl. conf. on pattern recognition, Quebec, Canada (Vol. IV, pp. 150–155).

  • Collins, R. (2003). Mean shift blob tracking through scale space. In Proc. IEEE conf. on computer vision and pattern recognition, Madison, WI (Vol. II, pp. 234–240).

  • Comaniciu, D. (2003). Variable bandwidth density-based fusion. In Proc. IEEE conf. on computer vision and pattern recognition, Madison, WI, June 2003 (Vol. 1, pp. 59–66).

  • Comaniciu, D., & Meer, P. (2002). Mean shift: A robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24, 603–619.

    Article  Google Scholar 

  • Comaniciu, D., Ramesh, V., & Meer, P. (2003). Kernel-based object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25, 564–577.

    Article  Google Scholar 

  • Davis, B., Fletcher, P. T., Bullitt, E., & Joshi, S. (2007). Population shape regression from random design data. In Proc. 11th intl. conf. on computer vision, Rio de Janeiro, Brazil, Oct. 2007.

  • Edelman, A., Arias, T. A., & Smith, S. T. (1998). The geometry of algorithms with orthogonality constraints. SIAM Journal on Matrix Analysis and Applications, 20(2), 303–353.

    Article  MATH  MathSciNet  Google Scholar 

  • Elgammal, A., Duraiswami, R., & Davis, L. S. (2003). Efficient kernel density estimation using the efficient kernel density estimation using the color modeling and tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(11), 1499–1504.

    Article  Google Scholar 

  • Fashing, M., & Tomasi, C. (2005). Mean shift is a bound optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(3), 471–474.

    Article  Google Scholar 

  • Ferreira, R., & Xavier, J. (2006). Hessian of the Riemannian squared-distance function on connected locally symmetric spaces with applications. In Controlo 2006, 7th Portuguese conference on automatic control.

  • Fletcher, P., Joshi, S., Lu, C., & Pizer, S. (2004). Principal geodesic analysis for the study of nonlinear statistics of shape. IEEE Transactions on Medical Imaging, 23(8), 995–1005.

    Article  Google Scholar 

  • Fletcher, P. T., Lu, C., & Joshi, S. (2003). Statistics of shape via principal geodesic analysis on Lie groups. In Proc. IEEE conf. on computer vision and pattern recognition, Madison, WI (pp. 95–101).

  • Forstner, W., & Moonen, B. (1999). A metric for covariance matrices (Technical report). Dept. of Geodesy and Geoinformatices, Stuttgart University.

  • Fukunaga, K., & Hostetler, L. D. (1975). The estimation of the gradient of a density function, with applications in pattern recognition. IEEE Transactions on Information Theory, 21, 32–40.

    Article  MATH  MathSciNet  Google Scholar 

  • Geyer, C., Bajcsy, R., & Sastry, S. (2004). Euclid meets Fourier: Applying harmonic analysis to essential matrix estimation in omnidirectional cameras. In Proc. of workshop on omnidirectional vision, camera networks and non-classical cameras.

  • Golub, G. H., & Van Loan, C. F. (1989). Matrix computations (2nd ed.). Baltimore: John Hopkins Press.

    MATH  Google Scholar 

  • Govindu, V. M. (2004). Lie-algebraic averaging for globally consistent motion estimation. In Proc. IEEE conf. on computer vision and pattern recognition, Washington, DC (Vol. I, pp. 684–691).

  • Hager, G., Dewan, M., & Stewart, C. (2004). Multiple kernel tracking with SSD. In Proc. IEEE conf. on computer vision and pattern recognition, Washington, DC (Vol. I, pp. 790–797).

  • Haralick, R., Lee, C., Ottenberg, K., & Nolle, M. (1991). Analysis and solutions of the three point perspective pose estimation problem. In Proc. IEEE conf. on computer vision and pattern recognition, Maui, HA (pp. 592–598).

  • Hartley, R. I., & Zisserman, A. (2000). Multiple view geometry in computer vision. Cambridge: Cambridge University Press.

    MATH  Google Scholar 

  • Helm, P. A., Winslow, R. L., & McVeigh, E. (2006). Center for cardiovascular bioinformatics and modeling (Technical report). Johns Hopkins University.

  • Helmke, U., Huper, K., Lee, P., & Moore, J. (2007). Essential matrix estimation using Gauss-Newton iterations on a manifold. International Journal of Computer Vision, 74(2), 117–136.

    Article  Google Scholar 

  • Kanatani, K. (1990). Group theoretical methods in image understanding. Berlin: Springer.

    MATH  Google Scholar 

  • Karcher, H. (1977). Riemannian center of mass and mollifier smoothing. Communications on Pure and Applied Mathematics, 30(5), 509–541.

    Article  MATH  MathSciNet  Google Scholar 

  • Lenglet, C., Deriche, R., & Faugeras, O. (2004). Inferring white matter geometry from diffusion tensor MRI: Application to connectivity mapping. In Proc. European conf. on computer vision, Prague, Czech Republic (Vol. IV, pp. 127–140).

  • Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2), 91–110.

    Article  Google Scholar 

  • Ma, Y., Kosecka, J., & Sastry, S. (2001). Optimization criteria and geometric algorithms for motion and structure estimation. International Journal of Computer Vision, 44(3), 219–249.

    Article  MATH  Google Scholar 

  • Maybank, S. J. (1992). Theory of reconstruction from image motion. Berlin: Springer.

    Google Scholar 

  • Mordohai, P., & Medioni, G. (2007). Tensor voting: a perceptual organization approach to computer vision and machine learning. Morgan and Claypool Publishers.

  • Nister, D. (2005). Preemptive RANSAC for live structure from motion. Machine Vision and Applications, 16(5), 321–329.

    Article  Google Scholar 

  • O’Neill, B. (1983). Semi-Riemannian manifolds:with applications to relativity. San Diego: Academic Press.

    MATH  Google Scholar 

  • Pelletier, B. (2005). Kernel density estimation on Riemannian manifolds. Statistics and Probability Letters, 73(3), 297–304.

    Article  MATH  MathSciNet  Google Scholar 

  • Pennec, X., & Ayache, N. (1998). Uniform distribution, distance and expectation problems for geometric feature processing. Journal of Mathematical Imaging and Vision, 9(1), 49–67.

    Article  MATH  MathSciNet  Google Scholar 

  • Pennec, X., Fillard, P., & Ayache, N. (2006). A Riemannian framework for tensor computing. International Journal of Computer Vision, 66(1), 41–66.

    Article  MathSciNet  Google Scholar 

  • Rossmann, W. (2003). Lie groups: an introduction through linear groups. London: Oxford University Press.

    Google Scholar 

  • Singh, M., Arora, H., & Ahuja, N. (2004). A robust probabilistic estimation framework for parametric image models. In Proc. European conf. on computer vision, Prague, Czech Republic, May 2004 (Vol. I, pp. 508–522).

  • Smith, S. T. (1994). Optimization techniques on Riemannian manifolds. Fields Institute Communications, 3, 113–146.

    Google Scholar 

  • Soatto, S., Perona, P., Frezza, R., & Picci, G. (1994). Motion estimation via dynamic vision. In Proc. European conf. on computer vision, Stockholm, Sweden (Vol. II, pp. 61–72).

  • Srivastava, A., Jermyn, I., & Joshi, S. (2007). Riemannian analysis of probability density functions with applications in vision. In Proc. IEEE conf. on computer vision and pattern recognition, Minneapolis, MN.

  • Srivastava, A., Joshi, S., Mio, W., & Liu, X. (2005). Statistical shape analysis: Clustering, learning and testing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(4), 590–602.

    Article  Google Scholar 

  • Subbarao, R., Genc, Y., & Meer, P. (2007). Nonlinear mean shift for robust pose estimation. In 8th IEEE workshop on applications of computer vision, Austin, TX, February 2007.

  • Subbarao, R., & Meer, P. (2006). Nonlinear mean shift for clustering over analytic manifolds. In Proc. IEEE conf. on computer vision and pattern recognition, New York, NY (Vol. I, pp. 1168–1175).

  • Subbarao, R., & Meer, P. (2007). Discontinuity preserving filtering over analytic manifolds. In Proc. IEEE conf. on computer vision and pattern recognition, Minneapolis, MN.

  • Sugaya, Y., & Kanatani, K. (2004). Geometric structure of degeneracy for multi-body motion segmentation. In LNCS : Vol. 3247. The 2nd workshop on statistical methods in video processing (SMVP 2004) (pp. 13–25). Berlin: Springer.

    Google Scholar 

  • Tschumperle, D., & Deriche, R. (2003). DT-MRI images: Estimation, regularization and application. In Computer aided systems theory—EUROCAST 2003 (pp. 531–541).

  • Tschumperle, D., & Deriche, R. (2005). Vector-valued image regularization with PDEs: A common framework for different applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(4), 506–517.

    Article  Google Scholar 

  • Tuzel, O., Subbarao, R., & Meer, P. (2005). Simultaneous multiple 3D motion estimation via mode finding on Lie groups. In Proc. 10th intl. conf. on computer vision, Beijing, China (Vol. 1, pp. 18–25).

  • Vemuri, B. C., Chen, Y., Rao, M., McGraw, T., Wang, Z., & Mareci, T. (2001). Fiber tract mapping from diffusion tensor MRI. In Proceedings of the IEEE workshop on variational and level set methods (pp. 81–88).

  • Wang, J., Thiesson, B., Xu, Y., & Cohen, M. (2004). Image and video segmentation by anisotropic kernel mean shift. In Proc. European conf. on computer vision, Prague, Czech Republic (Vol. II, pp. 238–249).

  • Yang, C., Duraiswami, R., DeMenthon, D., & Davis, L. (2003). Mean-shift analysis using quasi-Newton methods. In Intl. conf. on image processing (Vol. II, pp. 447–450).

  • Zhang, Z. (2000). A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11), 1330–1334.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Electrical and Computer Engineering Department, Rutgers University, 94 Brett Road, Piscataway, NJ, 08854-8058, USA

    Raghav Subbarao & Peter Meer

Authors
  1. Raghav Subbarao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peter Meer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Raghav Subbarao.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Subbarao, R., Meer, P. Nonlinear Mean Shift over Riemannian Manifolds. Int J Comput Vis 84, 1–20 (2009). https://doi.org/10.1007/s11263-008-0195-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-008-0195-8

Keywords

Search

Navigation