Fast and robust occluded face detection in ATM surveillance
Introduction
As a kind of popular financial terminals, ATMs are widely distributed all over cities. When using the ATM, a customer is asked to insert a bank card and input password for personal identification. ATM crimes such as peeping passwords and beating machine maliciously [19] occur constantly. Usually, ATM criminals wear sunglasses, masks or helmets to cover their faces in order to avoid being identified by video surveillance systems. If a surveillance system can detect these abnormal activities automatically and alarm security staff promptly, it will greatly improve the financial security of banks and facilitate the detective work of the police. Up till now, there have been many developmental systems about ATM surveillance. A more comprehensive survey of existing works is presented in Section 2.
Among various existing approaches [1], [5], [11], [30], [37], [38], [39], [40], detecting users’ facial images are widely used in practice, where the acquired facial images are crucial for verifying occlusion. Some recent methods are proposed related to face recognition and classification [8], [9], [10], [26]. However, the performance of existing systems degrades significantly when dealing with faces with severe occlusions.
In ATM related crimes, in order to avoid their face picture being captured, suspects often occlude their eyes and mouths. This has become an obstacle for many existing approaches. Therefore, this paper targets the severe occlusion problem, and proposes a fast and robust face occlusion detection algorithm by innovatively making use of the Omega shape formed by the head and shoulder of the person. Note that, this work simply targets to justify whether the face in front of an ATM has been occluded or not. Whether a person is a criminal or not will need to be further investigated by security or police. Our contributions are mainly three folded:
1) First, we propose to make use of the Omega shape formed by the head and shoulder of the person for accurate localization of heads to tackle severe face occlusion. This overcomes the problems with most existing approaches that rely on the presence of facial organs, which are not available under severe occlusion. For this purpose, we construct a new energy function for elliptical head contour detection.
2) Secondly, we develop a fast and robust head tracking algorithm, which utilizes gradient and shape cues in a Bayesian framework. This not only avoids unnecessary computation and thus speeds up the system but also contributes to a high detection rate.
3) Lastly, to verify whether a detected face is occluded, we propose to combine a skin color classifier and a face template matching classifier using the AdaBoost algorithm. This helps to address the problems associated with skin color variation due to different races, sexes, and ages as well as uneven illumination.
The rest of this paper is organized as follows. Section 2 reviews related existing works. Section 3 describes our proposed energy-based head localization and elliptical head tracking algorithm. Section 4 describes our classifier based face occlusion verification method. In Section 5, experimental results and analysis are presented. Finally, conclusions are drawn in Section 6.
Section snippets
Related works
The work described in this paper targets the problem of face occlusion detection, which aims to determine whether a captured head image is with occlusion or not in ATM surveillance. This problem constitutes the steps of many computer vision and surveillance applications, such as face recognition, face retrieval, face tracking, etc. Verifying whether there is face occlusion from an image is generally done in two steps, i.e., head region detection which is to locate the head regions from images,
Proposed occluded face detection scheme
In this section, we first present our proposed solution for detecting faces with severe occlusions. We create a potential energy function and convert the problem of extracting head contour to an energy minimization problem.
Face occlusion verification
In order to determine whether a face is occluded or not, we build two weak classifiers, each considering one type of face features only, i.e., the skin color and face shape respectively. Then, the AdaBoost learning algorithm is employed to learn a strong classifier with a higher accuracy from the cascaded weak classifiers.
Pre-processing
Pan et al. [33] proposed a content similarity based fast reference frame selection algorithm for reducing the computational complexity of the multiple reference frames based interframe prediction. Based on the best motion vector selection correlation among the different size prediction modes, Pan et al. [34], [35] proposed a fast motion estimation (ME) method to reduce the encoding complexity. The proposed algorithm can yield a quite promising coding performance in terms of RD performance and
Conclusion
In this paper, we have constructed a simple but fast and robust face occlusion detection algorithm for ATM surveillance applications. We innovatively made use of the Omega shape formed by head and shoulder and proposed a novel potential energy function for head contour detection. We also developed a fast and robust head tracking algorithm, which utilizes the gradient and shape cues in a Bayesian framework. The newly developed head detection module has been demonstrated to be robust for its
Acknowledgments
This research was partly supported by National Natural Science Foundation, China (Nos. 61702226, 61672263, 21365008), the Natural Science Foundation of Jiangsu Province (Grants nos. BK20170200, BK20161135).
References (43)
- et al.
A skin tone detection algorithm for an adaptive approach to steganography
Signal. Process.
(2009) - et al.
Pose-invariant face recognition with homography-based normalization
Pattern Recog.
(2017) - et al.
Robust skin detection in real-world images
J. Vis. Commun. Image Represent.
(2015) - et al.
A survey of skin-color modeling and detection methods
Pattern Recog.
(2007) - et al.
Fast reference frame selection based on content similarity for low complexity hevc encoder
J. Vis. Commun. Image Represent
(2016) - et al.
Recognizability assessment of facial images for automated teller machine applications
Pattern Recognit.
(2012) - et al.
Recognizing occluded faces by exploiting psychophysically inspired similarity maps
Pattern Recognit. Lett.
(2013) - et al.
Tracking a detected face with dynamic programming
Image. Vis. Comput.
(2006) - et al.
Face recognition on partially occluded images using compressed sensing
Pattern. Recognit. Lett.
(2014) - et al.
Kurtosis: a critical review
Am. Stat.
(1988)
Elliptical head tracking using intensity gradients and color histograms
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
Facial fraud discrimination using detection and classification
Int. Conf. Adv. Vis. Comput.
Face recognition using support vector machine and multiscale directional image representation methods: a comparative study
Int. Joint. Conf. Neural Netw.
Finding happiest moments in a social context
ACCV
Robust face recognition via multimodal deep face representation
ACM Trans Intell Syst Technol
A comprehensive survey on pose-invariant face recognition
ACM Trans. Intell. Syst. Technol.
Image-based fraud detection in automatic teller machine
Int. J. Comput. Sci. Netw. Security
Progressive search space reduction for human pose estimation
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
Combining reconstructive and discriminative subspace methods for robust classification and regression by subsampling
IEEE Trans. Pattern. Anal. Mach. Intell.
On the removal of shadows from images
IEEE Trans. Pattern. Anal. Mach. Intell.
Real time finger tracking and contour detection for gesture recognition using opencv
Int. Conf. Ind. Instrument. Cont.
Cited by (45)
Face mask detection using deep convolutional neural network and multi-stage image processing
2023, Image and Vision ComputingAn investigational FW-MPM-LSTM approach for face recognition using defective data
2023, Image and Vision ComputingA comprehensive survey on deep facial expression recognition: challenges, applications, and future guidelines
2023, Alexandria Engineering JournalCollaborative algorithms that combine AI with IoT towards monitoring and control system
2021, Future Generation Computer SystemsCitation Excerpt :We conclude the paper and discuss the direction of our future work in Section 6. Designing an appropriate system detection framework for IoT is difficult because IoT is heterogeneous [2–5], concurrent by nature, and sensitive to timing. Some difficulties happen all the time due to the difficult fusion problem of hardware and software when IoT uses some pattern recognition algorithms for computation [6–9].
Self-learning soft computing algorithms for prediction machines of estimating crowd density
2021, Applied Soft ComputingCitation Excerpt :Each function module describes a state, implemented by one or more machine learning algorithms [69–77]. Mode changes characterize the relationship of each functional module of the system, and several approaches were explored to solve deadlines misses problems [78–80]. The proposed IoT-based prediction machine in this paper uses those function modes to keep the security, robustness and reliability of designed system; in addition, the proposed prediction machine switches to stop mode immediately to prevent damage to the system in the event of a failure or unexpected event.
Optimized multi-scale affine shape registration based on an unsupervised Bayesian classification
2024, Multimedia Tools and Applications