Skip to main content
SpringerLink
Log in

Automatic identification of focus personage in multi-lingual news images

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Annotations of character IDs in news images are critical as ground truth for news retrieval and recommendation system. Universality and accuracy optimization of deep neural network models constitutes the key technology to improve the precision and computing efficiency of automatic news character identification, which is attracting increased attention globally. This paper explores the optimized deep neural network model for automatic focus personage identification in multi-lingual news. First, the face model of the focus personage is trained by using the corresponding face images from German news as positive samples. Next, the scheme of Recurrent Convolutional Neural Network (RCNN) + Bi-directional Long-Short Term Memory (Bi-LSTM) + Conditional Random Field (CRF) is utilized to label the focus name, and the RCNN-RCNN encoder–decoder is applied to translate names of people into multiple languages. Third, face features are described by combining the advantages of Local Gabor Binary Pattern Histogram Sequence (LGBPHS) and RCNN, and iterative quantization (ITQ) is used to binarize codes. Finally, a name semantic network is built for different domains. Experiments are performed on a dataset which comprises approximately 100,000 news images. The experimental results demonstrate that the proposed method achieves a significant improvement over other algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Chen X, Zhou E, Mo Y, Liu J (2017) Delving deep into coarse-to-fine framework for facial land mark localization. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp 2088–2095

  2. Cheng J, Li D, Mirella L (2016) Long short term memory-networks for machine Reading. In: Proceedings of Conference on Empirical Methods in Natural Language Processing, pp. 551–561

  3. Cho K, Van Merriënboer B, Gulcehre C (2014) Learning phrase representations using RNN encoder–decoder for statistical machine translation. arxiv:1–15

  4. Fan D-P, Wang W, Cheng M-M (2019) Shifting more attention to video salient object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pp. 8554–8564

  5. Gong Y, Lazebnik S, Gordo A (2011) Iterative quantization: A procrustean approach to learning binary codes. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 1–15

  6. Guillaumin M, Verbeek J, Schmid C (2011) Multiple instance metric learning from automatically labeled bags of faces. European Conference on Computer Vision, In, pp 634–647

    Google Scholar 

  7. Huang Z, Xu W, Yu K (2015) Bidirectional LSTM-CRF models for sequence tagging. CoRR abs/1508.01991

  8. Le D, Satoh S (2012) Auto face re-ranking by mining the web and video archives. In: Proceeding of IEEE Computer Society Conference on Computer Vision and. Pattern Recogn:2965–2972

  9. Li L, Feng X, Boulkenafet Z, Xia Z, Li M (2017) An original face anti-spoofing approachusing partial convolutional neural network. In: Proceeding of International Conference on Image Processing Theory Tools and Applications, pp 1–6

  10. Liang M, Hu X (2015) Recurrent convolution neural network for object recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3367–3375

  11. Liu LDY (2018) Deep learning in natural language processing. Springer, Berlin

    Google Scholar 

  12. Liu L, Lao S, Fieguth P, Guo Y, Wang X, Pietikainen M (2016) Median robust extended local binary pattern for texture classification. IEEE Trans Image Process 25(3):1368–1381

    Article  MathSciNet  Google Scholar 

  13. Liu L, Fieguth P, Guo Y, Wang X, Pietikainen M (2017) Local binary features for texture classification: Taxonomy and experimental study. Pattern Recogn 62:135–160

    Article  Google Scholar 

  14. Lu J, Liong VE, Wang G (2017) Joint feature learning for face recognition. IEEE Trans Info Forensics Sec 10(7):1371–1383

    Article  Google Scholar 

  15. Luo G, Huang X, Lin CY, Nie Z (2015) Joint named entity recognition and disambiguation. In: Proceedings of Conference on Empirical Methods in Natural Language Processing, pp. 879–888

  16. Milborrow S, Nicolls F (2008) Locating facial features with an extended active shape model. In: European Conference on Computer Vision, pp. 504–513

  17. Ozkan D, Duygulu P (2010) Interesting faces: a graph-based approach for finding people in news. Pattern Recogn 43(5):1717–1735

    Article  Google Scholar 

  18. Satpathy A, Jiang X, Eng H (2014) Lbp based edge texture features for object recognition. IEEE Trans Image Process 23(5):1953–1964

    Article  MathSciNet  Google Scholar 

  19. Su X, Peng J, Feng X, Fan J (2014) Cross-modality based celebrity face naming for news image collections. Multimed Tools Appl 73(3):1643–1661

    Article  Google Scholar 

  20. Su X, Peng J, Feng X, Wu J (2016) Labeling faces with names based on the namesemantic network. Multimed Tools Appl 75(11):6445–6462

    Article  Google Scholar 

  21. Su X, Zhou H, Draghici VP, Rätsch M (2018) Face naming in news images via multiple instance learning and hybrid recurrent convolution neural network. J Electron Imag 27(3):033–036

    Google Scholar 

  22. Suand X, Zhou H (2017) Automatic focus personage identification in multi-lingual news image. In: Proceedings of International Conference on the Frontiers and Advances in DataScience, pp 64–69

  23. Sun Y, Wang X, Tang X (2014) Deep learning face representation by joint identification verification. Neural Information Processing Systems: 1988–1996

  24. Sun Y, Wang X, Tang X (2016) Sparsifying neural network connections for face recognition. In: IEEE Conference on Computer Vision and Pattern Recogn, pp 4856–4864

  25. Sundermeyer M, Ney H, Schlüter R (2015) From feed forward to recurrent LSTM neural networks for Language modeling. IEEE Trans Audio Speech Lang Process 23(3):517–529

    Article  Google Scholar 

  26. Tran K, Bisazza A, Monz C (2016) Recurrent memory networks for language modeling. The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp 321–331

  27. Wang W, Lu X, Shen J (2019) Zero-shot video object segmentation via attentive graph neural networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp 9236–9245

  28. Wohlhart P, Köstinger M, Roth PM, Bischof H (2011) Multiple instance boosting for face recognition in videos. In: International Conference on. Pattern Recogn:132–141

  29. Yang J, Yan R, Hauptmann AG (2005) Multiple instance learning for labeling faces in broadcasting news video. In: Association Computing Machinery International Conference on Multimedia, pp 31–40

  30. Zhang B, Shan S, Gao Wand Chen X (2005) Local Gabor binary pattern histogram sequence: a novel non-statistical model for face representation and recognition. In: International Conference on Computer Vision, pp 786–791

  31. Zhou J, Hong X, Su F, Zhao G (2016) Recurrent convolutional neural network regression for continuous pain intensity estimation in video. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition workshop on Context-Based Affect Recognition and Affective Face in-the-wild, pp 84–92

Download references

Funding

This work is supported by National Natural Science Foundation of China under Grant 61902301, Shaanxi natural science basic research project under Grant 2019JQ-255, the Scientific Research Program funded by Shaanxi Provincial Education Department, under Grant 19JK0364, and Graduate Scientific Innovation Fund for Xi’an Polytechnic University No.chx2020018.

Author information

Authors and Affiliations

  1. School of Electronics and Information, Xi’an Polytechnic University, Xi’an, China

    Xueping Su, Danyao Zhu & Jie Ren

  2. Interactive and Mobile Robotics and Artificial Intelligence, Department of Engineering, Reutlingen University, Reutlingen, Germany

    Matthias Rätsch

Authors
  1. Xueping Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Danyao Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jie Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Matthias Rätsch
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xueping Su.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Su, X., Zhu, D., Ren, J. et al. Automatic identification of focus personage in multi-lingual news images. Multimed Tools Appl 80, 11015–11030 (2021). https://doi.org/10.1007/s11042-020-10254-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-10254-4

Keywords

Search

Navigation