Skip to main content
SpringerLink
Log in

Semi-supervised annotation of faces in image collection

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

The objective of this work is to correctly detect and recognize faces in an image collection using a database of known faces. This has applications in photo-tagging, video indexing, surveillance and recognition in wearable computers. We propose a two-stage approach for both detection and recognition tasks. In the first stage, we generate a seed set from the given image collection using off-the-shelf face detection and recognition algorithms. In the second stage, the obtained seed set is used to improve the performance of these algorithms by adapting them to the domain at hand. We propose an exemplar-based semi-supervised framework for improving the detections. For recognition of images, we use sparse representation classifier and generate seed images based on a confidence measure. The labels of the seed set are then propagated to other faces using label propagation framework by imposing appropriate constraints. Unlike traditional approaches, our approach exploits the similarities among the faces in collection to obtain improved performance. We conduct extensive experiments on two real-world photo-album and video collections. Our approach consistently provides an improvement of \({\sim } 4\)% for detection and \(5{-}9\)% for recognition on all these datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Belkin, M., Niyogi, P.: Laplacian Eigenmaps for dimensionality reduction and data representation. Neural Comput. 15, 1373–1396 (2003)

    Article  MATH  Google Scholar 

  2. Buml, M., Tapaswi, M., Stiefelhagen, R.: Semi-supervised learning with constraints for person identification in multimedia data. In: IEEE Conference on Computer Vision and Pattern Recognition (2013)

  3. De Marsico, M., Nappi, M., Riccio, D.: Face authentication with undercontrolled pose and illumination. Signal Image Video Process. 5, 401–413 (2011)

    Article  Google Scholar 

  4. Ding, C., Bao, T., Karmoshi, S., Zhu, M.: Single sample per person face recognition with KPCANet and a weighted voting scheme. Signal Image Video Process. (2017). doi:10.1007/s11760-017-1077-8

  5. Gallagher, A., Chen, T.: Clothing cosegmentation for recognizing people. In: IEEE Conference on Computer Vision and Pattern Recognition (2008)

  6. Guillaumin, M., Verbeek, J., Schmid, C.: Is that you? Metric learning approaches for face identification. In: International Conference on Computer Vision (2009)

  7. Jain, V., Learned-Miller, E.: Online domain adaptation of a pre-trained cascade of classifiers. In: IEEE Conference on Computer Vision and Pattern Recognition (2011)

  8. Kumar, N., Berg, A.C., Belhumeur, P.N., Nayar, S.K.: Attribute and simile classifiers for face verification. In: IEEE International Conference on Computer Vision (2009)

  9. Kumar, V., Namboodiri, A., Jawahar, C.: Face recognition in videos by label propagation. In: International Conference on Pattern Recognition (2014)

  10. Kumar, V., Namboodiri, A., Jawahar, C.: Visual phrases for exemplar face detection. In: IEEE International Conference on Computer Vision (2015)

  11. Li, H., Hua, G., Lin, Z., Brandt, J., Yang, J.: Probabilistic elastic part model for unsupervised face detector adaptation. In: IEEE International Conference on Computer Vision (2013)

  12. Li, H., Lin, Z., Shen, X., Brandt, J., Hua, G.: A convolutional neural network cascade for face detection. In: IEEE Conference on Computer Vision and Pattern Recognition (2015)

  13. Li, Y., Sun, B., Wu, T., Wang, Y.: Face detection with end-to-end integration of a CONvNet and a 3D model. In: European Conference on Computer Vision (2016)

  14. Mathias, M., Benenson, R., Pedersoli, M., Van Gool, L.: Face detection without bells and whistles. In: IEEE European Conference on Computer Vision (2014)

  15. Ortiz, E., Wright, A., Shah, M.: Face recognition in movie trailers via mean sequence sparse representation-based classification. In: IEEE Conference on Computer Vision and Pattern Recognition (2013)

  16. Ozerov, A., Vigouroux, J.R., Chevallier, L., Perez, P.: Clothing cosegmentation for recognizing people. In: International Conference on Image Processing (2013)

  17. Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. In: British Machine Vision Conference (2015)

  18. Roli, F., Marcialis, G.L.: Semi-supervised PCA-based face recognition using self training. In: Joint International Association Of Pattern Recognition Workshop (2006)

  19. Schroff, F., Kalenichenko, D., Philbin, J.: Facenet: A unified embedding for face recognition and clustering. In: IEEE Conference on Computer Vision and Pattern Recognition (2015)

  20. Sebe, N., Cohen, I., Huang, T.S., Gevers, T.: Semi-supervised face detection. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (2005)

  21. Shan, C.: Face recognition and retrieval in video. In: Video Search and Mining. Springer, pp 235–260. doi:10.1007/978-3-642-12900-1_9 (2010)

  22. Shen, X., Lin, Z., Brandt, J., Wu, Y.: Detecting and aligning faces by image retrieval. In: IEEE Conference on Computer Vision and Pattern Recognition (2013)

  23. Tapaswi, M., Bäuml, M., Stiefelhagen, R.: “Knock! Knock! Who is it?” Probabilistic person identification in TV-series. In: IEEE Conference on Computer Vision and Pattern Recognition (2012)

  24. Turk, M., Pentland, A.: Eigenfaces for recognition. J. Cogn. Neurosci. 3, 71–86 (1991)

    Article  Google Scholar 

  25. Viola, P.A., Jones, M.J.: Rapid object detection using a boosted cascade of simple features. In: IEEE Conference on Computer Vision and Pattern Recognition (2001)

  26. Wright, J., Yang, A.Y., Ganesh, A., Sastry, S.S., Ma, Y.: Robust face recognition via sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 31, 210–227 (2009)

    Article  Google Scholar 

  27. Yang, C., Ye, M., Tang, S., Xiang, T., Liu, Z.: Semi-supervised low-rank representation for image classification. Signal Image Video Process. 11, 73–80 (2017)

    Article  Google Scholar 

  28. Yu, W., Gan, L., Yang, S., Ding, Y., Jiang, P., Wang, J., Li, S.: An improved LBP algorithm for texture and face classification. Signal Image Video Process. 8, 155–161 (2014)

    Article  Google Scholar 

  29. Zhang, L., Yang, M., Feng, X.: Sparse representation or Collaborative representation: Which helps face recognition?. In: IEEE International Conference on Computer Vision (2011)

  30. Zhang, N., Paluri, M., Taigman, Y., Fergus, R., Bourdev, L.: Beyond frontal faces: improving person recognition using multiple cues. In: IEEE Conference on Computer Vision and Pattern Recognition (2015)

  31. Zhao, X., Evans, N.W., Dugelay, J.L.: Semi-supervised face recognition with LDA self-training. In: International Conference on Image Processing (2011)

  32. Zhu, X., Ghahramani, Z.: Learning from labeled and unlabeled data with label propagation. Technical Report, CMU (2002)

  33. Zhu, X., Ramanan, D.: Face detection, pose estimation, and landmark localization in the wild. In: IEEE Conference on Computer Vision and Pattern Recognition (2012)

Download references

Acknowledgements

This work is partly supported by the MCIT, New Delhi. Vijay Kumar is supported by TCS PhD fellowship 2012.

Author information

Authors and Affiliations

  1. International Institute of Information Technology, Hyderabad, India

    Vijay Kumar, Anoop Namboodiri & C. V. Jawahar

Authors
  1. Vijay Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anoop Namboodiri
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. C. V. Jawahar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vijay Kumar.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kumar, V., Namboodiri, A. & Jawahar, C.V. Semi-supervised annotation of faces in image collection. SIViP 12, 141–149 (2018). https://doi.org/10.1007/s11760-017-1140-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-017-1140-5

Keywords

Search

Navigation