Skip to main content
SpringerLink
Log in

Who Blocks Who: Simultaneous Segmentation of Occluded Objects

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

In this paper, we present a simultaneous segmentation algorithm for multiple highly-occluded objects, which combines high-level knowledge and low-level information in a unified framework. The high-level knowledge provides sophisticated shape priors with the consideration of blocking relationship between nearby objects. Different from conventional layered model which attempts to solve the full ordering problem, we decompose the problem into a series of pairwise ones and this makes our algorithm scalable to a large number of objects. Objects are segmented in pixel level with higher-order soft constraints from superpixels, by a dual-level conditional random field. The model is optimized alternately by object layout and pixel-wise segmentation. We evaluate our system on different objects, i.e., clothing and pedestrian, and show impressive segmentation results and significant improvement over state-of-the-art segmentation algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Comaniciu D, Meer P. Mean shift: A robust approach toward feature space analysis. IEEE Trans. Pattern Analysis and Machine Intelligence, 2002, 24(5): 603-619.

    Article  Google Scholar 

  2. Shi J, Malik J. Normalized cuts and image segmentation. IEEE Trans. Pattern Analysis and Machine Intelligence, 2000, 22(8): 888-905.

    Article  Google Scholar 

  3. Deng Y, Manjunath B. Unsupervised segmentation of color-texture regions in image and video. IEEE Trans. Pattern Analysis and Machine Intelligence, 2001, 23(8): 800-810.

    Article  Google Scholar 

  4. Rother C, Kolmogorov V, Blake A. “GrabCut”—Interactive foreground extraction using iterated graph cuts. ACM Trans. Graphics, 2004, 23(3): 309-314.

    Article  Google Scholar 

  5. Pham V Q, Takahashi K, Naemura T. Foreground-back-ground segmentation using iterated distribution matching. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2011, pp.2113-2120.

  6. Viola P, Jones M. Robust real-time face detection. Int. J. Computer Vision, 2004, 52(2): 137-154.

    Article  Google Scholar 

  7. Dalal N, Triggs B. Histogram of oriented gradients for human detection. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 27-July 2, 2004, Vol.1, pp.886-893.

  8. Felzenszwalb P F, Girshick R B, McAllester D, Ra-manan D. Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Analysis and Machine Intelligence, 2010, 32(9): 1627-1645.

    Article  Google Scholar 

  9. Andriluka M, Roth S, Schiele B. Pictorial structures revisited: People detection and articulated pose estimation. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2009, pp.1014-1021.

  10. Borenstein E, Ullman S. Combined top-down/bottom-up segmentation. IEEE Trans. Pattern Analysis and Machine Intelligence, 2008, 30(12): 2019-2125.

    Article  Google Scholar 

  11. Kumar M P, Torr P, Zisserman A. Objcut: Efficient segmentation using top-down and bottom-up cues. IEEE Trans. Pattern Analysis and Machine Intelligence, 2010, 32(3): 530-545.

    Article  Google Scholar 

  12. Levin A, Weiss Y. Learning to combine bottom-up and top-down segmentation. Int. J. Computer Vision, 2009, 81(1): 105-118.

    Article  Google Scholar 

  13. Zhu L, Chen Y, Yuille A. Learning a hierarchical deformable template for rapid deformable object parsing. IEEE Trans. Pattern Analysis and Machine Intelligence, 2010, 32(6): 1029-1043.

    Article  Google Scholar 

  14. Gallagher A, Chen T. Understanding groups of images of people. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2009.

  15. Andriluka M, Roth S, Schiele B, People-tracking-by-detection and people-detection-by-tracking. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2008.

  16. Yang Y, Hallman S, Ramanan D, Fowlkes C. Layered object detection for multi-class segmentation. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2010, pp.3113-3120.

  17. Leibe B, Schiele B. Interleaved object categorization and segmentation. In Proc. British Machine Vision Conference, September 2003.

  18. Winn J, Jojic N. Locus: Learning object classes with unsupervised segmentation. In Proc. the 10th Int. Conf. Computer Vision, October 2005, Vol.1, pp.756-763.

  19. Winn J, Shotton J. The layout consistent random field for recognizing and segmenting partially occluded objects. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2006, pp.37-44.

  20. Hoiem D, Rother C, Winn J. 3D layoutCRF for multi-view object class recognition and segmentation. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2007.

  21. Gould S, Rodgers J, Cohen D, Elidan G, Koller D. Multi-class segmentation with relative location prior. Int. J. Computer Vision, 2008, 80(3): 300-316.

    Article  Google Scholar 

  22. Wu B, Nevatia R. Detection and segmentation of multiple, partially occluded objects by grouping, merging, assigning part detection responses. Int. J. Computer Vision, 2009, 82(2): 185-204.

    Article  Google Scholar 

  23. Gao W, Ai H, Lao S. Adaptive contour features in oriented granualar space for human detection and segmentation. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2009, pp.1786-1793.

  24. Gallagher A C, Chen T. Clothing cosegmentation for recognizing people. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2008.

  25. Vu N, Manjunath B. Shape prior segmentation of multiple objects with graph cuts. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2008.

  26. Ladicky L, Sturgess P, Alahari K, Russell C, Torr P H. What, where and how many? Combining object detectors and CRFs. In Proc. the 11th European Conf. Computer Vision, September 2010, pp.424-437.

  27. Kohli P, Ladicky L, Torr P H. Robust higher order potentials for enforcing label consistency. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2008.

  28. Maire M, Yu S X, Perona P. Object detection and segmentation from joint embedding of parts and pixels. In Proc. Int. Conf. Computer Vision, November 2011, pp.2142-2149.

  29. Wang N, Ai H. Who blocks who: Simultaneous clothing segmentation for grouping images. In Proc. Int. Conf. Computer Vision, November 2011, pp.1535-1542.

  30. Lafferty J, McCallum A, Pereira F. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proc. the 18th Int. Conf. Machine Learning, June 28-July 1, 2001, pp.282-289.

  31. Wang N, Ai H, Lao S. A compositional exemplar-based model for hair segmentation. In Proc. the 10th Asian Conf. Computer Vision, November 2010, Vol.3, pp.171-184.

  32. Ladicky L, Russell C, Kohli P, Torr P H. Associative hierarchical CRFs for object class image segmentation. In Proc. Int. Conf. Computer Vision, September 27-October 4, 2009, pp.739-746.

  33. Boykov Y, Jolly M. Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images. In Proc. the 8th Int. Conf. Computer Vision, July 2001, Vol.1, pp.105-112.

  34. Geurts P, Ernst D, Wehenkel L. Extremely randomized trees. Machine Intelligence Learning, 2006, 63(1): 3-42.

    Article  MATH  Google Scholar 

  35. Murphy K P, Weiss Y, Jordan M I. Loopy belief propagation for approximate inference: An empirical study. In Proc. the 15th Conf. Uncertainty in Artificial Intelligence, July 30-August 1, 1999, pp.467-475.

  36. Breiman L. Random forests. Machine Learning, 2001, 45(1): 5-32.

    Article  MATH  Google Scholar 

  37. Koller D, Friedman N. Probabilistic Graphical Models: Principles and Techniques (1st edition). MIT Press, 2009.

  38. Kolmogorov V, Zabih R. What energy functions can be minimized via graph cuts? IEEE Trans. Pattern Analysis and Machine Intelligence, 2004, 26(2): 147-159.

    Article  Google Scholar 

  39. Boykov Y, Kolmogorov V. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Analysis and Machine Intelligence, 2004, 26(9): 1124-1137.

    Article  Google Scholar 

  40. Bradski G. The opencv library. In Dr. Dobb’s Journal of Software Tools, 2000, http://www.drdobbs.com/open-source/the-opencv-library/184404319, July 2013.

  41. Huang C, Ai H, Li Y, Lao S. High performance rotation invariant multiview face detection. IEEE Trans. Pattern Analysis and Machine Intelligence, 2007, 29(4): 671-686.

    Article  Google Scholar 

  42. Eichner M, Ferrari V. We are family: Joint pose estimation of multiple persons. In Proc. the 11th European Conf. Computer Vision, September 2010, Part 1, pp.228-242.

  43. Duan G, Ai H, Lao S. A structural filter approach to human detection. In Proc. European Conf. Computer Vision, September 2010, Part 6, pp.238-251.

  44. Xing J, Ai H, Lao S. Multi-object tracking through occlusions by local tracklets filtering and global tracklets association with detection responses. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2009, pp.1200-1207.

  45. Bibby C, Reid I. Robust real-time visual tracking using pixel-wise posteriors. In Proc. European Conf. Computer Vision, October 2008, pp.831-844.

  46. Horber E, Rematas K, Leibe B. Level-set person segmentation and tracking with multi-region appearance models and top-down shape information. In Proc. Int. Conf. Computer Vision, November 2011, pp.1871-1878.

  47. Brox T, Bourdev L, Maji S, Malik J. Object segmentation by alignment of poselet activations to image contours. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2011, pp.2225-2232.

  48. Ramanan D. Using segmentation to verify hypotheses. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2006.

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Technology, Tsinghua University, Beijing, 100084, China

    Nan Wang & Hai-Zhou Ai

  2. Multimedia Interaction and Understanding Lab, HP Labs, Palo Alto, CA, 94304-1126, U.S.A.

    Feng Tang

Authors
  1. Nan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hai-Zhou Ai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Feng Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nan Wang.

Additional information

This work is supported in part by the National Natural Science Foundation of China under Grant No.61075026 and the National Basic Research 973 Program of China under Grant No.2011CB302203.

Electronic supplementary material

Below is the link to the electronic supplementary material.

ESM 1

(DOCX 16 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wang, N., Ai, HZ. & Tang, F. Who Blocks Who: Simultaneous Segmentation of Occluded Objects. J. Comput. Sci. Technol. 28, 890–906 (2013). https://doi.org/10.1007/s11390-013-1385-6

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-013-1385-6

Keywords

Search

Navigation