research-article

Deformable part models with CNN features for facial landmark detection under occlusion

Authors:

Hima B. VadapalliAuthors Info & Claims

SAICSIT '17: Proceedings of the South African Institute of Computer Scientists and Information Technologists

Article No.: 5, Pages 1 - 9

https://doi.org/10.1145/3129416.3129451

Published: 26 September 2017 Publication History

Abstract

Detecting and localizing facial regions in images is a fundamental building block of many applications in the field of affective computing and human-computer interaction. This allows systems to do a variety of higher level analysis such as facial expression recognition. Facial expression recognition is based on the effective extraction of relevant facial features. Many techniques have been proposed to deal with the robust extraction of these features under a wide variety of poses and occlusion conditions. These techniques include Deformable Part Models (DPM's), and more recently deep Convolutional neural networks (CNN's). Recently, hybrid models based on DPMs and CNNs have been proposed considering the generalization properties of CNNs and DPMs. In this work we propose a combined system, using CNN's as features for a DPM with a focus on dealing with occlusion.

We also propose a method of face detection allowing occluded regions to be detected and explicitly ignored during the detection step. The resulting system is quite robust to a wide variety of occlusions achieving accuracies comparable to that of other state of the art systems.

References

[1]

Peter N Belhumeur, David W Jacobs, David J Kriegman, and Narendra Kumar. 2013. Localizing parts of faces using a consensus of exemplars. Pattern Analysis and Machine Intelligence, IEEE Transactions on 35, 12 (2013), 2930--2940.

Digital Library

[2]

X P Burgos-Artizzu, P Perona, and Dollár P. 2013. Robust face landmark estimation under occlusion. In Computer Vision (ICCV), 2013 IEEE International Conference on. IEEE, 1513--1520.

Digital Library

[3]

Navneet Dalal and Bill Triggs. 2005. Histograms of oriented gradients for human detection. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, Vol. 1. IEEE, 886--893.

Digital Library

[4]

Yuan Dong and Yue Wu. 2015. Adaptive cascade deep convolutional neural networks for face alignment. Computer standards & interfaces 42 (2015), 105--112.

Digital Library

[5]

Pedro F Felzenszwalb, Ross B Girshick, David McAllester, and Deva Ramanan. 2010. Object detection with discriminatively trained part-based models. IEEE transactions on pattern analysis and machine intelligence 32, 9 (2010), 1627--1645.

[6]

Kunihiko Fukushima. 1988. Neocognitron: A hierarchical neural network capable of visual pattern recognition. Neural networks 1, 2 (1988), 119--130.

[7]

Golnaz Ghiasi and Charless C. Fowlkes. 2014. Occlusion coherence: Localizing occluded faces with a hierarchical deformable part model. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

Digital Library

[8]

Ross Girshick, Forrest Iandola, Trevor Darrell, and Jitendra Malik. 2015. Deformable part models are convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 437--446.

[9]

Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press. http://www.deeplearningbook.org.

[10]

David H Hubel and Torsten N Wiesel. 1968. Receptive fields and functional architecture of monkey striate cortex. The Journal of physiology 195, 1 (1968), 215--243.

[11]

P Ivan. 2007. Active appearance models for face recognition. Vrije Universiteit Amsterdam, Amsterdam (2007).

[12]

Pengfei Ji, Yonghwa Kim, Yong Yang, and Yoo-Sung Kim. 2016. Face occlusion detection using skin color ratio and LBP features for intelligent video surveillance systems. In Computer Science and Information Systems (FedCSIS), 2016 Federated Conference on. IEEE, 253--259.

[13]

Vuong Le, Jonathan Brandt, Zhe Lin, Lubomir Bourdev, and Thomas Huang. 2012. Interactive facial feature localization. Computer Vision-ECCV 2012 (2012), 679--692.

Digital Library

[14]

Masakazu Matsugu, Katsuhiko Mori, Yusuke Mitari, and Yuji Kaneda. 2003. Subject independent facial expression recognition with robust face detection using a convolutional neural network. Neural Networks 16, 5 (2003), 555--559.

Digital Library

[15]

S Nass, J Steuer, and E R Tauber (Eds.). 1994. Computers are social actors. ACM.

[16]

Dinh-Luan Nguyen, Vinh-Tiep Nguyen, Minh-Triet Tran, and Atsuo Yoshitaka. 2015. Deep convolutional neural network in deformable part models for face detection. In Pacific-Rim Symposium on Image and Video Technology. Springer, 669--681.

[17]

Joseph Roth, Yiying Tong, and Xiaoming Liu. 2016. Adaptive 3D face reconstruction from unconstrained photo collections. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4197--4206.

[18]

Pierre-André Savalle, Stavros Tsogkas, George Papandreou, and Iasonas Kokkinos. 2014. Deformable part models with CNN features. In European Conference on Computer Vision, Parts and Attributes Workshop.

[19]

Danai Triantafyllidou, Paraskevi Nousi, and Anastasios Tefas. 2017. Fast deep convolutional face detection in the wild exploiting hard sample mining. Big Data Research (2017).

[20]

Alan L Yuille, David S Cohen, and Peter W Hallinan. 1989. Feature extraction from faces using deformable templates. In Computer Vision and Pattern Recognition, 1989. Proceedings CVPR'89., IEEE Computer Society Conference on. IEEE, 104--109.

[21]

Wei Zhang, Gregory Zelinsky, and Dimitris Samaras. 2007. Real-time accurate object detection using multiple resolutions. In Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on. IEEE, 1--8.

Cited By

Reddy SKandasamy G(2022)Cusp Pixel Labelling Model for Objects Outline Using R-CNNIEEE Access10.1109/ACCESS.2021.313989610(8883-8890)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2021.3139896
Cao DChen ZGao L(2020)An improved object detection algorithm based on multi-scaled and deformable convolutional neural networksHuman-centric Computing and Information Sciences10.1186/s13673-020-00219-910:1Online publication date: 11-Apr-2020
https://doi.org/10.1186/s13673-020-00219-9
Chen YLiu S(2020)Deep Partial Occlusion Facial Expression Recognition via Improved CNNAdvances in Visual Computing10.1007/978-3-030-64556-4_35(451-462)Online publication date: 7-Dec-2020
https://doi.org/10.1007/978-3-030-64556-4_35

Index Terms

Deformable part models with CNN features for facial landmark detection under occlusion
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Object detection

Recommendations

Facial Landmark Detection and Tracking for Facial Behavior Analysis
ICMR '16: Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval

The face is the most dominant and distinct communication tool of human beings. Automatic analysis of facial behavior allows machines to understand and interpret a human's states and needs for natural interactions. This research focuses on developing ...
A novel approach to classification of facial expressions from 3D-mesh datasets using modified PCA

We propose a novel approach to human facial expression recognition using only the shape information at a finite set of fiducial points, extracted from the 3D neutral and expressive faces. In the course of applying the technique to the facial database, ...
Facial expression recognition in still pictures and videos using active appearance models: a comparison approach
CompSysTech '07: Proceedings of the 2007 international conference on Computer systems and technologies

The paper highlights the performance of video sequence-oriented facial expression recognition using Active Appearance Model -- AAM, in a comparison with the analysis based on still pictures. The AAM is used to extract relevant information regarding the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

SAICSIT '17: Proceedings of the South African Institute of Computer Scientists and Information Technologists

September 2017

384 pages

ISBN:9781450352505

DOI:10.1145/3129416

Conference Chair:
Muthoni Masinde
Central University of Technology

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 September 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SAICSIT '17

SAICSIT '17: South African Institute of Computer Scientists and Information Technologists

September 26 - 28, 2017

Thaba 'Nchu, South Africa

Acceptance Rates

SAICSIT '17 Paper Acceptance Rate 39 of 108 submissions, 36%;

Overall Acceptance Rate 187 of 439 submissions, 43%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
121
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Reddy SKandasamy G(2022)Cusp Pixel Labelling Model for Objects Outline Using R-CNNIEEE Access10.1109/ACCESS.2021.313989610(8883-8890)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2021.3139896
Cao DChen ZGao L(2020)An improved object detection algorithm based on multi-scaled and deformable convolutional neural networksHuman-centric Computing and Information Sciences10.1186/s13673-020-00219-910:1Online publication date: 11-Apr-2020
https://doi.org/10.1186/s13673-020-00219-9
Chen YLiu S(2020)Deep Partial Occlusion Facial Expression Recognition via Improved CNNAdvances in Visual Computing10.1007/978-3-030-64556-4_35(451-462)Online publication date: 7-Dec-2020
https://doi.org/10.1007/978-3-030-64556-4_35

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten