skip to main content
10.1145/3129416.3129451acmotherconferencesArticle/Chapter ViewAbstractPublication PageshtConference Proceedingsconference-collections
research-article

Deformable part models with CNN features for facial landmark detection under occlusion

Published: 26 September 2017 Publication History

Abstract

Detecting and localizing facial regions in images is a fundamental building block of many applications in the field of affective computing and human-computer interaction. This allows systems to do a variety of higher level analysis such as facial expression recognition. Facial expression recognition is based on the effective extraction of relevant facial features. Many techniques have been proposed to deal with the robust extraction of these features under a wide variety of poses and occlusion conditions. These techniques include Deformable Part Models (DPM's), and more recently deep Convolutional neural networks (CNN's). Recently, hybrid models based on DPMs and CNNs have been proposed considering the generalization properties of CNNs and DPMs. In this work we propose a combined system, using CNN's as features for a DPM with a focus on dealing with occlusion.
We also propose a method of face detection allowing occluded regions to be detected and explicitly ignored during the detection step. The resulting system is quite robust to a wide variety of occlusions achieving accuracies comparable to that of other state of the art systems.

References

[1]
Peter N Belhumeur, David W Jacobs, David J Kriegman, and Narendra Kumar. 2013. Localizing parts of faces using a consensus of exemplars. Pattern Analysis and Machine Intelligence, IEEE Transactions on 35, 12 (2013), 2930--2940.
[2]
X P Burgos-Artizzu, P Perona, and Dollár P. 2013. Robust face landmark estimation under occlusion. In Computer Vision (ICCV), 2013 IEEE International Conference on. IEEE, 1513--1520.
[3]
Navneet Dalal and Bill Triggs. 2005. Histograms of oriented gradients for human detection. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, Vol. 1. IEEE, 886--893.
[4]
Yuan Dong and Yue Wu. 2015. Adaptive cascade deep convolutional neural networks for face alignment. Computer standards & interfaces 42 (2015), 105--112.
[5]
Pedro F Felzenszwalb, Ross B Girshick, David McAllester, and Deva Ramanan. 2010. Object detection with discriminatively trained part-based models. IEEE transactions on pattern analysis and machine intelligence 32, 9 (2010), 1627--1645.
[6]
Kunihiko Fukushima. 1988. Neocognitron: A hierarchical neural network capable of visual pattern recognition. Neural networks 1, 2 (1988), 119--130.
[7]
Golnaz Ghiasi and Charless C. Fowlkes. 2014. Occlusion coherence: Localizing occluded faces with a hierarchical deformable part model. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[8]
Ross Girshick, Forrest Iandola, Trevor Darrell, and Jitendra Malik. 2015. Deformable part models are convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 437--446.
[9]
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press. http://www.deeplearningbook.org.
[10]
David H Hubel and Torsten N Wiesel. 1968. Receptive fields and functional architecture of monkey striate cortex. The Journal of physiology 195, 1 (1968), 215--243.
[11]
P Ivan. 2007. Active appearance models for face recognition. Vrije Universiteit Amsterdam, Amsterdam (2007).
[12]
Pengfei Ji, Yonghwa Kim, Yong Yang, and Yoo-Sung Kim. 2016. Face occlusion detection using skin color ratio and LBP features for intelligent video surveillance systems. In Computer Science and Information Systems (FedCSIS), 2016 Federated Conference on. IEEE, 253--259.
[13]
Vuong Le, Jonathan Brandt, Zhe Lin, Lubomir Bourdev, and Thomas Huang. 2012. Interactive facial feature localization. Computer Vision-ECCV 2012 (2012), 679--692.
[14]
Masakazu Matsugu, Katsuhiko Mori, Yusuke Mitari, and Yuji Kaneda. 2003. Subject independent facial expression recognition with robust face detection using a convolutional neural network. Neural Networks 16, 5 (2003), 555--559.
[15]
S Nass, J Steuer, and E R Tauber (Eds.). 1994. Computers are social actors. ACM.
[16]
Dinh-Luan Nguyen, Vinh-Tiep Nguyen, Minh-Triet Tran, and Atsuo Yoshitaka. 2015. Deep convolutional neural network in deformable part models for face detection. In Pacific-Rim Symposium on Image and Video Technology. Springer, 669--681.
[17]
Joseph Roth, Yiying Tong, and Xiaoming Liu. 2016. Adaptive 3D face reconstruction from unconstrained photo collections. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4197--4206.
[18]
Pierre-André Savalle, Stavros Tsogkas, George Papandreou, and Iasonas Kokkinos. 2014. Deformable part models with CNN features. In European Conference on Computer Vision, Parts and Attributes Workshop.
[19]
Danai Triantafyllidou, Paraskevi Nousi, and Anastasios Tefas. 2017. Fast deep convolutional face detection in the wild exploiting hard sample mining. Big Data Research (2017).
[20]
Alan L Yuille, David S Cohen, and Peter W Hallinan. 1989. Feature extraction from faces using deformable templates. In Computer Vision and Pattern Recognition, 1989. Proceedings CVPR'89., IEEE Computer Society Conference on. IEEE, 104--109.
[21]
Wei Zhang, Gregory Zelinsky, and Dimitris Samaras. 2007. Real-time accurate object detection using multiple resolutions. In Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on. IEEE, 1--8.

Cited By

View all
  • (2022)Cusp Pixel Labelling Model for Objects Outline Using R-CNNIEEE Access10.1109/ACCESS.2021.313989610(8883-8890)Online publication date: 2022
  • (2020)An improved object detection algorithm based on multi-scaled and deformable convolutional neural networksHuman-centric Computing and Information Sciences10.1186/s13673-020-00219-910:1Online publication date: 11-Apr-2020
  • (2020)Deep Partial Occlusion Facial Expression Recognition via Improved CNNAdvances in Visual Computing10.1007/978-3-030-64556-4_35(451-462)Online publication date: 7-Dec-2020

Index Terms

  1. Deformable part models with CNN features for facial landmark detection under occlusion

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    SAICSIT '17: Proceedings of the South African Institute of Computer Scientists and Information Technologists
    September 2017
    384 pages
    ISBN:9781450352505
    DOI:10.1145/3129416
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 26 September 2017

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. affective computing
    2. convolutional neural networks
    3. deformable part models
    4. facial feature extraction
    5. occlusion

    Qualifiers

    • Research-article

    Conference

    SAICSIT '17

    Acceptance Rates

    SAICSIT '17 Paper Acceptance Rate 39 of 108 submissions, 36%;
    Overall Acceptance Rate 187 of 439 submissions, 43%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)1
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 25 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2022)Cusp Pixel Labelling Model for Objects Outline Using R-CNNIEEE Access10.1109/ACCESS.2021.313989610(8883-8890)Online publication date: 2022
    • (2020)An improved object detection algorithm based on multi-scaled and deformable convolutional neural networksHuman-centric Computing and Information Sciences10.1186/s13673-020-00219-910:1Online publication date: 11-Apr-2020
    • (2020)Deep Partial Occlusion Facial Expression Recognition via Improved CNNAdvances in Visual Computing10.1007/978-3-030-64556-4_35(451-462)Online publication date: 7-Dec-2020

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media