Skip to main content
SpringerLink
Log in

Shape-Based Object Localization for Descriptive Classification

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Discriminative tasks, including object categorization and detection, are central components of high-level computer vision. However, sometimes we are interested in a finer-grained characterization of the object’s properties, such as its pose or articulation. In this paper we develop a probabilistic method (LOOPS) that can learn a shape and appearance model for a particular object class, and be used to consistently localize constituent elements (landmarks) of the object’s outline in test images. This localization effectively projects the test image into an alternative representational space that makes it particularly easy to perform various descriptive tasks. We apply our method to a range of object classes in cluttered images and demonstrate its effectiveness in localizing objects and performing descriptive classification, descriptive ranking, and descriptive clustering.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., & Davis, J. (2005). Scape: shape completion and animation of people. In SIGGRAPH ’05: ACM SIGGRAPH 2005 papers (pp. 408–416). New York: ACM. doi:http://doi.acm.org/10.1145/1186822.1073207.

    Chapter  Google Scholar 

  • Basri, R., Costa, L., Geiger, D., & Jacobs, D. (1998). Determining the similarity of deformable shapes. Vision Research, 38, 2365–2385.

    Article  Google Scholar 

  • Belongie, S., Malik, J., & Puzicha, J. (2000) Shape context: A new descriptor for shape matching and object recognition. In Neural Information Processing Systems (pp. 831–837).

  • Berg, A., Berg, T., & Malik, J. (2005). Shape matching and object recognition using low distortion correspondence. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR).

  • Borenstein, E., Sharon, E., & Ullman, S. (2004). Combining top-down and bottom-up segmentation. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (p. 46). Los Alamitos: IEEE Computer Society. ISBN 0-7695-2158-4.

    Google Scholar 

  • Borgefors, G. (1988). Hierarchical chamfer matching: A parametric edge matching algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence, 10(6), 849–865. ISSN 0162-8828. doi:10.1109/34.9107.

    Article  Google Scholar 

  • Boyd, S., & Vandenberghe, L. (2004). Convex optimization. Cambridge: Cambridge University Press.

    MATH  Google Scholar 

  • Caselles, V., Kimmel, R., & Sapiro, G. (1995). Geodesic active contours. In International conference on computer vision (pp. 694–699).

  • Cootes, T. F., Taylor, C. J., Cooper, D. H., & Graham, J. (1995). Active shape models: their training and application. Computer Vision and Image Understanding, 61(1), 38–59. ISSN 1077-3142. doi:10.1006/cviu.1995.1004.

    Article  Google Scholar 

  • Cootes, T. F., Edwards, G. J., & Taylor, C. J. (1998). Active appearance models. In European conference on computer vision (vol. 2, pp. 484–498).

  • Cover, T. M., & Thomas, J. A. (1991). Elements of information theory. New York: Wiley.

    Book  MATH  Google Scholar 

  • Crandall, D. J., & Huttenlocher, D. P. (2006). Weakly supervised learning of part-based spatial models for visual object recognition. In A. Leonardis, H. Bischof, & A. Pinz (Eds.), Lecture notes in computer science : Vol. 3951. European conference on computer vision (Vol. 1, pp. 16–29). Berlin: Springer.

    Google Scholar 

  • Crandall, D., Felzenszwalb, P., & Huttenlocher, D. (2005). Spatial priors for part-based recognition using statistical models. In Proceedings of the 2005 IEEE Computer Society conference on computer vision and pattern recognition (CVPR’05) (vol. 1).

  • Cremers, D., Tischhäuser, F., Weickert, J., & Schnörr, C. (2002). Diffusion snakes: Introducing statistical shape knowledge into the Mumford-Shah functional. International Journal of Computer Vision, 50(3), 295–313. ISSN 0920-5691. doi:10.1023/A:1020826424915.

    Article  MATH  Google Scholar 

  • Dryden, I., & Mardia, K. (1998). Statistical shape analysis. New York: Wiley.

    MATH  Google Scholar 

  • Elidan, G., Heitz, G., & Koller, D. (2006a). Learning object shape: From cartoons to images. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR).

  • Elidan, G., McGraw, I., & Koller, D. (2006b). Residual belief propagation: Informed scheduling for asynchronous message passing. In Uncertainty in artificial intelligence.

  • Fei-Fei, L., Fergus, R., & Perona, P. (2004). Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR).

  • Felzenszwalb, P. F., & Huttenlocher, D. P. (2000). Efficient matching of pictorial structures. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (pp. 66–73).

  • Felzenszwalb, P. F., & Schwartz, J. D. (2007). Hierarchical matching of deformable shapes. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR).

  • Fergus, R., Perona, P., & Zisserman, A. (2003). Object class recognition by unsupervised scale-invariant learning. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (Vol. 2, pp. 264–271)

  • Fergus, R., Perona, P., & Zisserman, A. (2005). A sparse object category model for efficient learning and exhaustive recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, San Diego (Vol. 1, pp. 380–397).

  • Ferrari, V., Tuytelaars, T., & Van Gool, L. (2006). Object detection by contour segment networks. In European conference on computer vision (ECCV).

  • Ferrari, V., Jurie, F., & Schmid, C. (2007). Accurate object detection with deformable shape models learnt from images. In IEEE conference on computer vision and pattern recognition. IEEE, June 2007. New York: IEEE.

    Google Scholar 

  • Ferrari, V., Fevrier, L., Jurie, F., & Schmid, C. (2008). Groups of adjacent contour segments for object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(5), 36–51.

    Article  Google Scholar 

  • Fink, M., & Ullman, S. (2007). From aardvark to zorro: A benchmark for mammal image classification. International Journal of Computer Vision, 77, 143–156.

    Article  Google Scholar 

  • Grauman, K., & Darrell, T. (2005). Pyramid match kernels: Discriminative classification with sets of image features. In International conference on computer vision, October 2005.

  • Hill, A., & Taylor, C. (1996). A method of non-rigid correspondence for automatic landmark identification. In Proceedings of the British machine vision conference.

  • Hillel, A. B., Hertz, T., & Weinshall, D. (2005). Efficient learning of relational object class models. In International conference on computer vision (pp. 1762–1769), Washington, DC, USA. Los Alamitos: IEEE Computer Society. ISBN 0-7695-2334-X.

    Chapter  Google Scholar 

  • Kumar, M. P., Torr, P. H. S., & Zisserman, A. (2005). OBJ CUT. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR).

  • Leibe, B., Leonardis, A., & Schiele, B. (2004). Combined object categorization and segmentation with an implicit shape model. In ECCV’04 workshop on statistical learning in computer vision (pp. 17–32), Prague, Czech Republic, May 2004.

  • Leordeanu, M., Hebert, M., & Sukthankar, R. (2007). Beyond local appearance: Category recognition from pairwise interactions of simple features. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR).

  • Lowe, D. (2003). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 20, 91–110.

    Google Scholar 

  • Murphy, K. P., Torralba, A., Eaton, D., & Freeman, W. T. (2006). Object detection and localization using local and global features. In J. Ponce, M. Hebert, C. Schmid, & A. Zisserman (Eds.), Toward category-level object recognition. Cambridge: MIT Press.

    Google Scholar 

  • Opelt, A., Pinz, A., & Zisserman, A. (2006a). Fusing shape and appearance information for object category detection. In Proceedings of the British machine vision conference.

  • Opelt, A., Pinz, A., & Zisserman, A. (2006b). Incremental learning of object detectors using a visual shape alphabet. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR) (Vol. 1, pp. 3–10).

  • Pearl, J. (1988). Probabilistic reasoning in intelligent systems. San Mateo: Morgan Kaufmann.

    Google Scholar 

  • Prasad, M., & Fitzgibbon, A. (2006). Single view reconstruction of curved surfaces. In Proceedings of the 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR ’06), Washington, DC, USA (pp. 1345–1354). Los Alamitos: IEEE Computer Society. ISBN 0-7695-2597-0. doi:10.1109/CVPR.2006.281.

    Chapter  Google Scholar 

  • Schapire, R. E., & Singer, Y. (1999). Improved boosting using confidence-rated predictions. Machine Learning, 37(3), 297–336.

    Article  MATH  Google Scholar 

  • Sebastian, T. B., Klein, P. N., & Kimia, B. B. (2004). Recognition of shapes by editing their shock graphs. IEEE Transactions on Pattern Analysis Machine Intelligence, 26(5), 550–571. ISSN 0162-8828. doi:10.1109/TPAMI.2004.1273924.

    Article  Google Scholar 

  • Sethian, J. (1998). Level set methods and fast marching methods: evolving interfaces in computational geometry, fluid mechanics, computer vision, and materials science. Cambridge: Cambridge University Press.

    Google Scholar 

  • Shotton, J., Blake, A., & Cipolla, R. (2005). Contour-based learning for object detection. In International conference on computer vision.

  • Thayananthan, A., Stenger, B., Torr, P., & Cipolla, R. (2003). Shape context and chamfer matching in cluttered scenes. In IEEE Computer Society conference on computer vision and pattern recognition (CVPR).

  • Torralba, A., Murphy, K. P., & Freeman, W. T. (2005). Contextual models for object detection using boosted random fields. In L. K. Saul, Y. Weiss, & L. Bottou (Eds.), Advances in neural information processing systems (Vol. 17, pp. 1401–1408). Cambridge: MIT Press.

    Google Scholar 

  • Winn, J., & Shotton, J. (2006). The layout consistent random field for recognizing and segmenting partially occluded objects. In Proceedings of the 2006 IEEE Computer Society conference on computer vision and pattern recognition (CVPR ’06), Washington, DC, USA (pp. 37–44). Los Alamitos: IEEE Computer Society. ISBN 0-7695-2597-0.

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Stanford University, Stanford, CA, 94305, USA

    Geremy Heitz, Benjamin Packer & Daphne Koller

  2. Department of Statistics, Hebrew University of Jerusalem, Jerusalem, 91905, Israel

    Gal Elidan

Authors
  1. Geremy Heitz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gal Elidan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Benjamin Packer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daphne Koller
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Geremy Heitz.

Additional information

Authors G. H., G. E. and B. P. contributed equally to this manuscript.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Heitz, G., Elidan, G., Packer, B. et al. Shape-Based Object Localization for Descriptive Classification. Int J Comput Vis 84, 40–62 (2009). https://doi.org/10.1007/s11263-009-0228-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-009-0228-y

Keywords

Search

Navigation