Skip to main content
SpringerLink
Log in

Cross-modality based celebrity face naming for news image collections

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

For automatically mining the underlying relationships between different famous persons in daily news, for example, building a news person based network with the faces as icons to facilitate face-based person finding, we need a tool to automatically label faces in new images with their real names. This paper studies the problem of linking names with faces from large-scale news images with captions. In our previous work, we proposed a method called Person-based Subset Clustering which is mainly based on face clustering for all face images derived from the same name. The location where a name appears in a caption, as well as the visual structural information within a news image provided informative cues such as who are really in the associated image. By combining the domain knowledge from the captions and the corresponding image we propose a novel cross-modality approach to further improve the performance of linking names with faces. The experiments are performed on the data sets including approximately half a million news images from Yahoo! news, and the results show that the proposed method achieves significant improvement over the clustering-only methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Berg T, Berg A, Edwards J, Maire M, White R (2004) Names and faces in the news. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), page 848–854

  2. Berg A, Berg A, Edwards J, Forsyth D (2005) Who’s in the picture. Adv Neural Inf Process Syst 17:137–144

    Google Scholar 

  3. Cunningham H, Maynard D, Bontcheva K, Tablan V (2002) Gate: a framework and graphical development environment for robust nlp tools and applications. In: 40th Anniversary Meeting of the Association for Computational Linguistics ( ACL), page 168–175

  4. Frey BJ, Dueck D (2007) Clustering by passing messages between data points. Science 315:972–976

    Article  MATH  MathSciNet  Google Scholar 

  5. Guillaumin M, Mensink T, Verbeek JJ, Schmid C (2008) Automatic face naming with caption-based supervision. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), page 1–8

  6. Guillaumin M, Verbeek J, Schmid C (2010) Multiple instance metric learning from automatically labeled bags of faces. In: European Conference on Computer Vision (ECCV), page 634–647

  7. Le D, Satoh S (2008) Unsupervised face annotation by mining the web. In: International Conference on Data Mining (ICDM), page 383–392

  8. Mensink T, Verbeek J (2008) Improving people search using query expansions: how friends help to find people. In: European Conference on Computer Vision (ECCV), page 86–99

  9. Mensink T, Verbeek J (2008) Improving people search using query expansions. In: European Conference on Computer Vision (ECCV), page 86–99

  10. Ozkan D, Duygulu P (2010) Interesting faces: a graph-based approach for finding people in news. Pattern Recogn 43(5):1717–1735

    Article  Google Scholar 

  11. Peng Y, Ganesh A (2010) RASL: robust batch alignment of images by sparse and low-rank decomposition. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), page 763–770

  12. Pham PT, Moens MF, Tuytelaars T (2010) Cross-media alignment of names and faces. IEEE Trans Multimed 12:13–27

    Article  Google Scholar 

  13. Pham PT, Tuytelaars T, Moens M-F (2010) Naming persons in news video with label propagation. IEEE Multimedia 18:44–55

    Article  Google Scholar 

  14. Poppe R (2012) Facing scalability: naming faces in an online social network. Pattern Recogn 45:2335–2347

    Article  Google Scholar 

  15. Su X-P, Peng J-Y, Feng X-Y, Wu J, Fan J-P (2011) Linking names and faces by person-based subset clustering. In: Proceedings of the Third International Conference on Internet Multimedia Computing and Service (ACM ICIMCS), page 120–123

  16. Viola P, Jones M (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154

    Article  Google Scholar 

  17. Yang J, Chen M-Y, Hauptmann A (2004) Finding person x: correlating names with visual appearances. In: International Conference on Image and Video Retrieval, page 270–278

  18. Zhang B, Shan S, Gao W, Chen X (2005) Local Gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition. In: International Conference on Computer Vision (ICCV), page 786–791

Download references

Author information

Authors and Affiliations

  1. School of Electronics and Information, Northwestern Polytechnical University, Xi’an, China

    Xueping Su, Jinye Peng, Xiaoyi Feng, Jun Wu, Jianping Fan & Li Cui

Authors
  1. Xueping Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jinye Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaoyi Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jun Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jianping Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Li Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xueping Su.

Additional information

This work is supported by the doctorate foundation of Northwestern Polytechnical University under CX201114, Ministry of Education Fund for Doctoral Students Newcomer Awards of China, National Natural Science Foundation of China under Grant 61075014, 61272285, 61103062, The Research Fund for the Doctoral Program of Higher Education under Grant 20106102110028, 20116102110027, 20116102120031, 20126101110022, The Science and technology project of Shaanxi Province under Grant 2013K06-29, and NPU Basic Research Foundation under Grant JC201249.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Su, X., Peng, J., Feng, X. et al. Cross-modality based celebrity face naming for news image collections. Multimed Tools Appl 73, 1643–1661 (2014). https://doi.org/10.1007/s11042-013-1578-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-013-1578-6

Keywords

Search

Navigation