Skip to main content
SpringerLink
Log in

Multi-object segmentation by stereo mismatch

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

A new stereo mismatch based foreground object segmentation method is described. It efficiently locates objects over a wide range of depths against backgrounds of known 3D geometry, even in the presence of rapidly changing lighting and dynamic textures, such as projected video. Not relying on full stereo reconstruction, it is fast enough in software for some real-time applications, robust to camera quality, and requires little parameter tuning. Experimental results validate the approach, demonstrating its ability to simultaneously distinguish multiple objects in a complex scene, even when close together or partially occluded.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ayer, S., Sawhney, H.S.: Layered representation of motion video using robust maximum-likelihood estimation of mixture models and MDL encoding. In: Int’l Conf. on Computer Vision, pp. 777–784 (1995)

  2. Baker, S., Szeliski, R., Anandan, P.: A layered approach to stereo reconstruction. In: Computer Vision and Pattern Recognition, pp. 434–441 (1998)

  3. Birchfield, S., Tomasi, C.: Multiway cut for stereo and motion with slanted surfaces. In: Int’l Conf. on Computer Vision, pp. 489–495 (1999)

  4. Cucchiara, R., Grana, C., Piccardi, M., Prati, A.: Detecting moving objects, ghosts and shadows in video streams. IEEE Trans. Pattern Anal. Mach. Intell. 25(10), 1337–1342 (2003)

    Article  Google Scholar 

  5. Elder, J.H., Goldberg, R.M.: Ecological statistics of Gestalt laws for the perceptual organization of contours. J. Vis. 2, 324–353 (2002)

    Article  Google Scholar 

  6. Elder, J.H., Krupnik, A., Johnston, L.A.: Contour grouping with prior models. IEEE Trans. Pattern Anal. Mach. Intell. 25(25), 1–14 (2003)

    Google Scholar 

  7. Foley, J.D., van Dam, A., Feiner, S.K., Hughes, J.F.: Computer Graphics: Principles and Practice in C, 2 edn. Addison-Wesley (1997)

  8. Fusiello, A., Trucco, E., Verri, A.: A compact algorithm for rectification of stereo pairs. Mach. Vis. Appl. 12(1), 16–22 (2000)

    Article  Google Scholar 

  9. Gutchess, D., Trajkovic, M., Cohen-Solal, E., Lyons, D., Jain, A.: A background model initialization algorithm for video surveillance. In: Int’l Conf. on Computer Vision, pp. 733–740 (2001)

  10. Hsu, S., Anandan, P., Peleg, S.: Accurate computation of optical flow by using layered motion representations. In: Int’l Conf. on Pattern Recognition, pp. 743–746 (1994)

  11. Ivanov, Y., Bobick, A., Liu, J.: Fast lighting independent background subtraction. Intl. J. Comput. Vis. 37(2), 199–207 (2000)

    Article  MATH  Google Scholar 

  12. Jain, R., Wakimoto, K.: Multiple perspective interactive video. In: Int’l Conf Multimedia Computing and Systems, pp. 202–211 (1995)

  13. Jepson, A.D., Black, M.J.: Mixture models for optical flow computation. In: Computer Vision and Pattern Recognition, pp. 760–761 (1993)

  14. Kolmogorov, V.: Software. http://www.adastral.ucl.ac.uk/~vladkolm/software.html (2001-2003)

  15. Kolmogorov, V., Criminisi, A., Blake, A., Cross, G., Rother, C.: Bi-layer segmentation of binocular stereo video. In: Computer Vision and Pattern Recognition, pp. 407–414 (2005)

  16. Kolmogorov, V., Zabih, R.: Multi-camera scene reconstruction via graph cuts. In: European Conf. on Computer Vision, pp. 82–96 (2002)

  17. Lin, M.H., Tomasi, C.: Surfaces with occlusions from layered stereo. IEEE Trans. Pattern Anal. Mach. Intell. 26(8), 1073–1078 (2004)

    Article  Google Scholar 

  18. Narayanan, P.J., Rander, P.W., Kanade, T.: Constructing virtual worlds using dense stereo. In: Int’l Conf. on Computer Vision, pp. 3–10 (1998)

  19. Ohta, Y., Kanade, T.: Stereo by intra- and inter-scanline search using dynamic programming. IEEE Trans. Pattern Anal. Mach. Intell. 7(2), 139–154 (1985)

    Article  Google Scholar 

  20. Oliver, N.M., Rosario, B., Pentland, A.P.: A Bayesian computer vision system for modeling human interactions. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 831–843 (2000)

    Article  Google Scholar 

  21. Rittscher, J., Kato, J., Joga, S., Blake, A.: A probabilistic background model for tracking. In: European Conf. on Computer Vision, pp. 336–350 (2000)

  22. Roy, S., Cox, I.J.: A maximum-flow formulation of the N-camera stereo correspondence problem. In: Int’l Conf. on Computer Vision, pp. 492–499 (1998)

  23. Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Intl. J. Comput. Vis. 47(1–3), 7–42 (2002)

    Article  MATH  Google Scholar 

  24. Stauffer, C., Grimson, W.: Adaptive background mixture models for real-time tracking. In: Computer Vision and Pattern Recognition, pp. 246–252 (1999)

  25. Sun, J., Zheng, N.N., Shum, H.Y.: Stereo matching using belief propagation. IEEE Trans. Pattern Anal. Mach. Intell. 25(7), 787–800 (2003)

    Article  Google Scholar 

  26. Sun, W., Cooperstock, J.R.: An empirical evaluation of factors influencing camera calibration accuracy using three publicly available techniques. Mach. Vis. Appl. 17(1), 51–67 (2006)

    Article  Google Scholar 

  27. Torr, P.H., Szeliski, R., Anandan, P.: An integrated Bayesian approach to layer extraction from image sequences. IEEE Trans. Pattern Anal. Mach. Intell. 23(3), 297–303 (2001)

    Article  Google Scholar 

  28. Toyama, K., Krumm, J., Brumitt, B., Meyers, B.: Wallflower: principles and practice of background maintenance. In: Intl Conf. on Computer Vision, pp. 255–261 (1999)

  29. Wang, J.Y., Adelson, E.H.: Layered representation for motion analysis. In: Computer Vision and Pattern Recognition, pp. 361–366 (1993)

  30. Weiss, Y., Adelson, E.H.: A unified mixture framework for motion segmentation: incorporating spatial coherence and estimating the number of models. In: Computer Vision and Pattern Recognition, pp. 321–326 (1996)

  31. Wren, C.R., Azarbayejani, A.J., Darrell, T.J., Pentland, A.P.: Pfinder: real-time tracking of the human body. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 780–785 (1997)

    Article  Google Scholar 

  32. Zhang, L., Curless, B., Seitz, S.M.: Rapid shape acquisition using color structured light and multi-pass dynamic programming. In: Intl Symposium on 3D Data Processing Visualization and Transmission, pp. 24–36 (2002)

Download references

Author information

Authors and Affiliations

  1. Intel Corporation, 2200 Mission College Blvd., Santa Clara, CA, 95054, USA

    Wei Sun

  2. Quantum Corporation, 1650 Technology Drive, Suite 800, San Jose, CA, 95110, USA

    Stephen P. Spackman

Authors
  1. Wei Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stephen P. Spackman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei Sun.

Additional information

This research was undertaken at Centre for Intelligent Machines, McGill University, Montreal, QC, Canada.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sun, W., Spackman, S.P. Multi-object segmentation by stereo mismatch. Machine Vision and Applications 20, 339–352 (2009). https://doi.org/10.1007/s00138-008-0127-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-008-0127-1

Keywords

Search

Navigation