Abstract
Emotional body language constitutes an important channel of non-verbal information. Of this large set, hand-over-face gestures are treated as noise because they occlude facial expressions. In this paper, we propose an alternative facial processing framework where face occlusions instead of being removed, are detected, localized and eventually classified into communicative gestures. We present a video corpus of hand-over-face gestures and describe a multi-stage methodology for detecting and localizing these gestures. For pre-processing, we show that force fields form a better representation of images compared to edge detectors. For feature extraction, detection and localization, we show that Local Binary Patterns outperform Gabor filters in accuracy and speed. Our methodology yields an average detection rate of 97%, is robust to changes in facial expressions, hand shapes, and limited head motion, and preliminary testing with spontaneous videos suggests that it may generalize successfully to naturally evoked videos.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Buciu, I., Kotsia, I., Pitas, I.: Facial expression analysis under partial occlusion. Proc. of Acoustics, Speech, and Signal Processing 5, 453–456 (2005)
Comaniciu, D., Ramesh, V., Meer, P.: Real-time tracking of non-rigid objects using Mean Shift. In: Proc. of Computer Vision and Pattern Recognition, vol. 1, pp. 142–149 (2000)
de Gelder, B.: Towards the neurobiology of emotional body language. Nature Reviews Neuroscience 7, 242–249 (2006)
Ekman, P., Friesen, W.: The repertoire of nonverbal behavior. Categories, origins, usage, and coding. Mouton de Gruyter, Berlin (1969)
Fei, H., Reid, I.D.: Joint Bayes Filter: A Hybrid Tracker for Non-rigid Hand Motion Recognition. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3023, pp. 497–508. Springer, Heidelberg (2004)
Hurley, D., Nixon, M., Carter, J.: Forcefield energy functionals for image feature extraction. Image and Vision Computing 20, 311–317 (2002)
el Kaliouby, R., Robinson, P.: Real-Time Inference of Complex Mental States from Facial Expressions and Head Gestures. In: Real-time vision for human computer interaction, pp. 181–200. Springer, Heidelberg (2005)
Ma, W., Manjunath, B.: Texture features and learning similarity. In: Proc. of Computer Vision and Pattern Recognition, pp. 425–430 (1996)
Ojala, T., Pietikainen, M., Harwood, D.: Comparative study of texture measures with classification based on feature distributions. Pattern Recognition 29(1), 51–59 (1996)
Pease, A., Pease, B.: The definitive book of body language. Orion (2004)
Sherrah, J., Gong, S.: Resolving visual uncertainty and occlusion through probabilistic reasoning. In: Proc. of British Machine Vision Conference (BMVC), vol. 1, pp. 252–261 (2000)
Smith, P., da Vitoria Lobo, N., Shah, M.: Resolving hand over face occlusion. Image and Vision Computing 25(9), 1432–1448 (2007)
Stenger, B., Thayananthan, A., Torr, P.H.S., Cipolla, R.: Model-based hand tracking using a hierarchical bayesian filter. IEEE Trans. on Pattern Analysis and Machine Intelligence 28(9), 1372–1384 (2006)
Tong, Y., Liao, W., Ji, Q.: Facial action unit recognition by exploiting their dynamic and semantic relationships. IEEE Trans. on Pattern Analysis and Machine Intelligence 29(10), 1683–1699 (2007)
Triesch, J., von der Malsburg, C.: Classification of hand postures against complex backgrounds using elastic graph matching. Image and Vision Computing 20(13-14), 937–943 (2002)
Yilmaz, A., Javed, O., Shah, M.: Object tracking: A survey. ACM computing surveys 38(4), 1–45 (2006)
Zhou, H., Huang, T.S.: Tracking articulated hand motion with eigen dynamics analysis. In: Proc. of International Conference on Computer Vision, pp. 1102–1109 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Mahmoud, M., El-Kaliouby, R., Goneid, A. (2009). Towards Communicative Face Occlusions: Machine Detection of Hand-over-Face Gestures. In: Kamel, M., Campilho, A. (eds) Image Analysis and Recognition. ICIAR 2009. Lecture Notes in Computer Science, vol 5627. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02611-9_48
Download citation
DOI: https://doi.org/10.1007/978-3-642-02611-9_48
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02610-2
Online ISBN: 978-3-642-02611-9
eBook Packages: Computer ScienceComputer Science (R0)