Abstract
A new stereo mismatch based foreground object segmentation method is described. It efficiently locates objects over a wide range of depths against backgrounds of known 3D geometry, even in the presence of rapidly changing lighting and dynamic textures, such as projected video. Not relying on full stereo reconstruction, it is fast enough in software for some real-time applications, robust to camera quality, and requires little parameter tuning. Experimental results validate the approach, demonstrating its ability to simultaneously distinguish multiple objects in a complex scene, even when close together or partially occluded.
Similar content being viewed by others
References
Ayer, S., Sawhney, H.S.: Layered representation of motion video using robust maximum-likelihood estimation of mixture models and MDL encoding. In: Int’l Conf. on Computer Vision, pp. 777–784 (1995)
Baker, S., Szeliski, R., Anandan, P.: A layered approach to stereo reconstruction. In: Computer Vision and Pattern Recognition, pp. 434–441 (1998)
Birchfield, S., Tomasi, C.: Multiway cut for stereo and motion with slanted surfaces. In: Int’l Conf. on Computer Vision, pp. 489–495 (1999)
Cucchiara, R., Grana, C., Piccardi, M., Prati, A.: Detecting moving objects, ghosts and shadows in video streams. IEEE Trans. Pattern Anal. Mach. Intell. 25(10), 1337–1342 (2003)
Elder, J.H., Goldberg, R.M.: Ecological statistics of Gestalt laws for the perceptual organization of contours. J. Vis. 2, 324–353 (2002)
Elder, J.H., Krupnik, A., Johnston, L.A.: Contour grouping with prior models. IEEE Trans. Pattern Anal. Mach. Intell. 25(25), 1–14 (2003)
Foley, J.D., van Dam, A., Feiner, S.K., Hughes, J.F.: Computer Graphics: Principles and Practice in C, 2 edn. Addison-Wesley (1997)
Fusiello, A., Trucco, E., Verri, A.: A compact algorithm for rectification of stereo pairs. Mach. Vis. Appl. 12(1), 16–22 (2000)
Gutchess, D., Trajkovic, M., Cohen-Solal, E., Lyons, D., Jain, A.: A background model initialization algorithm for video surveillance. In: Int’l Conf. on Computer Vision, pp. 733–740 (2001)
Hsu, S., Anandan, P., Peleg, S.: Accurate computation of optical flow by using layered motion representations. In: Int’l Conf. on Pattern Recognition, pp. 743–746 (1994)
Ivanov, Y., Bobick, A., Liu, J.: Fast lighting independent background subtraction. Intl. J. Comput. Vis. 37(2), 199–207 (2000)
Jain, R., Wakimoto, K.: Multiple perspective interactive video. In: Int’l Conf Multimedia Computing and Systems, pp. 202–211 (1995)
Jepson, A.D., Black, M.J.: Mixture models for optical flow computation. In: Computer Vision and Pattern Recognition, pp. 760–761 (1993)
Kolmogorov, V.: Software. http://www.adastral.ucl.ac.uk/~vladkolm/software.html (2001-2003)
Kolmogorov, V., Criminisi, A., Blake, A., Cross, G., Rother, C.: Bi-layer segmentation of binocular stereo video. In: Computer Vision and Pattern Recognition, pp. 407–414 (2005)
Kolmogorov, V., Zabih, R.: Multi-camera scene reconstruction via graph cuts. In: European Conf. on Computer Vision, pp. 82–96 (2002)
Lin, M.H., Tomasi, C.: Surfaces with occlusions from layered stereo. IEEE Trans. Pattern Anal. Mach. Intell. 26(8), 1073–1078 (2004)
Narayanan, P.J., Rander, P.W., Kanade, T.: Constructing virtual worlds using dense stereo. In: Int’l Conf. on Computer Vision, pp. 3–10 (1998)
Ohta, Y., Kanade, T.: Stereo by intra- and inter-scanline search using dynamic programming. IEEE Trans. Pattern Anal. Mach. Intell. 7(2), 139–154 (1985)
Oliver, N.M., Rosario, B., Pentland, A.P.: A Bayesian computer vision system for modeling human interactions. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 831–843 (2000)
Rittscher, J., Kato, J., Joga, S., Blake, A.: A probabilistic background model for tracking. In: European Conf. on Computer Vision, pp. 336–350 (2000)
Roy, S., Cox, I.J.: A maximum-flow formulation of the N-camera stereo correspondence problem. In: Int’l Conf. on Computer Vision, pp. 492–499 (1998)
Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Intl. J. Comput. Vis. 47(1–3), 7–42 (2002)
Stauffer, C., Grimson, W.: Adaptive background mixture models for real-time tracking. In: Computer Vision and Pattern Recognition, pp. 246–252 (1999)
Sun, J., Zheng, N.N., Shum, H.Y.: Stereo matching using belief propagation. IEEE Trans. Pattern Anal. Mach. Intell. 25(7), 787–800 (2003)
Sun, W., Cooperstock, J.R.: An empirical evaluation of factors influencing camera calibration accuracy using three publicly available techniques. Mach. Vis. Appl. 17(1), 51–67 (2006)
Torr, P.H., Szeliski, R., Anandan, P.: An integrated Bayesian approach to layer extraction from image sequences. IEEE Trans. Pattern Anal. Mach. Intell. 23(3), 297–303 (2001)
Toyama, K., Krumm, J., Brumitt, B., Meyers, B.: Wallflower: principles and practice of background maintenance. In: Intl Conf. on Computer Vision, pp. 255–261 (1999)
Wang, J.Y., Adelson, E.H.: Layered representation for motion analysis. In: Computer Vision and Pattern Recognition, pp. 361–366 (1993)
Weiss, Y., Adelson, E.H.: A unified mixture framework for motion segmentation: incorporating spatial coherence and estimating the number of models. In: Computer Vision and Pattern Recognition, pp. 321–326 (1996)
Wren, C.R., Azarbayejani, A.J., Darrell, T.J., Pentland, A.P.: Pfinder: real-time tracking of the human body. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 780–785 (1997)
Zhang, L., Curless, B., Seitz, S.M.: Rapid shape acquisition using color structured light and multi-pass dynamic programming. In: Intl Symposium on 3D Data Processing Visualization and Transmission, pp. 24–36 (2002)
Author information
Authors and Affiliations
Corresponding author
Additional information
This research was undertaken at Centre for Intelligent Machines, McGill University, Montreal, QC, Canada.
Rights and permissions
About this article
Cite this article
Sun, W., Spackman, S.P. Multi-object segmentation by stereo mismatch. Machine Vision and Applications 20, 339–352 (2009). https://doi.org/10.1007/s00138-008-0127-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00138-008-0127-1