Skip to main content
SpringerLink
Log in

A comparison of face detection methods using spontaneous videos

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Real time face detection techniques are needed in a wide range of fields. Therefore, developing a high, accurate and efficient near-real-time face detection method has become a major concern for both industrial and research communities. This paper introduces a critical comparison to a variety of face-detection methods, namely, (1) Haar-like cascade, (2) Linear Binary Pattern cascade (LBP), (3) Histogram of Oriented Gradients with Support Vector Machine (HOG) and (4) Convolutional Neural Network based algorithms (CNN) using video sequences rather than static images. Different experiments were conducted to evaluate the performance of these techniques on constraint and spontaneous video sequences from the database for Remote Collaborative and Affective Interactions (RECOLA) and the Database for Emotion Analysis using Physiological Signals (DEAP). The experimental results show that CNN based algorithm is more efficient compared to other approaches. It achieves an average detection rate of 99.99% for the DEAP database and 84.23% for the RECOLA database. However, it is the slowest when it comes to detecting faces with an average number of frames per second (FPS) of 2.12 and 2.58. Meanwhile LBP method is the fastest among the proposed methods with an average FPS of 25.58 and 33.79.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23

Similar content being viewed by others

References

  1. Acasandrei L, Barriga A (2014) Embedded Face Detection Application Based on Local Binary Patterns. In: 2014 IEEE Intl Conf on High Performance Computing and Communications, 2014 IEEE 6th Intl Symp on Cyberspace Safety and Security, 2014 IEEE 11th Intl Conf on Embedded Software and Syst (HPCC,CSS,ICESS), pp 641–644

    Chapter  Google Scholar 

  2. Alobaidi WH, Aziz IT, Jawad T, Flaih FMF, Azeez AT (2018) Face detection based on probability of amplitude distribution of local binary patterns algorithm. In: 2018 6th International Symposium on Digital Forensic and Security (ISDFS), pp 1–5

    Google Scholar 

  3. An S, Ma X, Song R, Li Y (2009) Face detection and recognition with SURF for human-robot interaction. In: 2009 IEEE International Conference on Automation and Logistics, pp 1946–1951

    Chapter  Google Scholar 

  4. Artan Y, Burry A, Kozitsky V, Paul P (2012) Efficient SMQT features for snow-based classification on face detection and character recognition tasks. In: 2012 Western New York Image Processing Workshop, pp 45–48

    Chapter  Google Scholar 

  5. Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720

    Article  Google Scholar 

  6. Bradski G, Kaehler A (2008) Learning OpenCV: Computer vision with the OpenCV library. O’Reilly Media, Inc

    Google Scholar 

  7. Byeon Y-H, Pan S-B, Moh S-M, Kwak K-C (2016) A surveillance system using CNN for face recognition with object, human and face detection. In: Information Science and Applications (ICISA) 2016, pp 975–984

    Chapter  Google Scholar 

  8. Chawla D, Trivedi MC (2018) Difference in lights and color background differentiates the color skin model in face detection for security surveillance. In: Networking Communication and Data Knowledge Engineering, pp 127–135

    Chapter  Google Scholar 

  9. Dagnes N, Marcolin F, Nonis F, Tornincasa S, Vezzetti E (2019) 3D geometry-based face recognition in presence of eye and mouth occlusions. Int J Interact Des Manuf 13(4):1617–1635

    Article  Google Scholar 

  10. Dalal N, Triggs B (2005) Histograms of Oriented Gradients for Human Detection. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), pp 886–893

    Google Scholar 

  11. Ding C, Tao D (2015) Robust face recognition via multimodal deep face representation. IEEE Transactions on Multimedia 17(11):2049–2058

    Article  Google Scholar 

  12. dlib C++ Library (n.d.) [Online]. Available: http://dlib.net/

  13. Du H, Wang Z, Wang D, Wang X (2018) Multi-Feature Fusion Method Applied in Texture Image Segmentation. In: 2018 14th International Conference on Computational Intelligence and Security (CIS), pp 135–139

    Chapter  Google Scholar 

  14. Duan S, Wang X, Wan W (2013) The LogitBoost Based on Joint Feature for Face Detection. In: 2013 Seventh International Conference on Image and Graphics, pp 483–488

    Chapter  Google Scholar 

  15. Erdem C, Ulukaya S, Karaali A, Erdem T (2011) Combining Haar Feature and skin color based classifiers for face detection. In: Presented at the ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, pp 1497–1500

    Google Scholar 

  16. Filali H, Riffi J, Mahraz AM, Tairi H (2018) Multiple face detection based on machine learning. In: 2018 International Conference on Intelligent Systems and Computer Vision (ISCV), pp 1–8

    Google Scholar 

  17. Gao W et al (2008) The CAS-PEAL large-scale Chinese face database and baseline evaluations. IEEE Trans Syst Man Cybern Syst Hum 38(1):149–161

    Article  Google Scholar 

  18. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich featurehierarchies for accurate object detection and semantic segmentation. In: Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, pp 580–587

    Chapter  Google Scholar 

  19. Guo-yun Z, Guo L-y, Wu J-h, Hong-min L, Guo G-q (2010) Face detection using Gabor wavelet and SVM. In: 2010 International Conference on Computer Application and System Modeling (ICCASM 2010), vol 2, pp V2–588–V2–590

    Google Scholar 

  20. Hadid A, Zhao G, Ahonen T, Pietikainen M (2008) Face analysis using Local binary Patterns. World Scientific Volume, pp 347–350

    Google Scholar 

  21. Hernández M, Cheplygina V, Plasencia-Calaña Y (2015) Dissimilarity Representations for Low-Resolution Face Recognition, pp 70–83

  22. Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507

    Article  MathSciNet  MATH  Google Scholar 

  23. Hu L, Li Z, Liu H (2015) Age Group Estimation on Single Face Image Using Blocking ULBP and SVM. In: Proceedings of the 2015 Chinese Intelligent Automation Conference, pp 431–438

    Chapter  Google Scholar 

  24. Huang GB, Ramesh M, Berg T, Learned-Miller E (2007) Labeled faces in the wild: a database for studying face recognition in unconstrained environments. University of Massachusetts, Amherst

    Google Scholar 

  25. Huang D, Chen C, Chen T, Wu J, Ko C (2018) Real-Time Face Detection Using a Moving Camera. In: 2018 32nd International Conference on Advanced Information Networking and Applications Workshops (WAINA), pp 609–614

    Chapter  Google Scholar 

  26. Jain V, Learned-Miller E (2010) Fddb: A benchmark for face detection in unconstrained settings. Technical Report UM-CS-2010–009, University of Massachusetts, Amherst

  27. Ji Z, Zhu H, Wang Q (2016) LFHOG: A discriminative descriptor for live face detection from light field image. In: 2016 IEEE International Conference on Image Processing (ICIP), pp 1474–1478

    Chapter  Google Scholar 

  28. Jin H, Liu Q, Lu H, Tong X (2004) Face Detection Using Improved LBP Under Bayesian Framework. In: Proceedings of the Third International Conference on Image and Graphics, Washington, DC, USA, pp 306–309

    Google Scholar 

  29. Jun Z, Jizhao H, Zhenglan T, Feng W (2017) Face detection based on LBP. In: 2017 13th IEEE International Conference on Electronic Measurement Instruments (ICEMI), pp 421–425

    Chapter  Google Scholar 

  30. Kikuchi T, Ozasa Y Watch, Listen Once, and Sync: Audio-Visual Synchronization With Multi-Modal Regression Cnn. In: 2018 IEEE international conference on acoustics, Speech and Signal Processing (ICASSP), vol 2018, pp 3036–3040

  31. Kim Y, Shahdoost H, Jadhav S, Gloster CS (2017) Improving the Accuracy of Arctan for Face Detection. In: 2017 IEEE 25th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM), p 202

    Chapter  Google Scholar 

  32. Koelstra S, Muhl C, Soleymani M, Lee JS, Yazdani A, Ebrahimi T, Pun T, Nijholt A, Patras I (2011) DEAP: A Database for Emotion Analysis Using Physiological Signals. IEEE Trans Affect Comput 3(1):18–31

    Article  Google Scholar 

  33. Köstinger M, Wohlhart P, Roth PM, Bischof H (2011) Annotatedfacial landmarks in the wild: A large-scale, realworld database for facial landmark localization. In: ComputerVision Workshops (ICCV Workshops), 2011 IEEE International Conference on, pp 2144–2151

    Chapter  Google Scholar 

  34. Lee S, Jang S, Kim J, Choi B (2016) A hardware architecture of face detection for human-robot interaction and its implementation. In: 2016 IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia), pp 1–2

    Google Scholar 

  35. Li J (2017) Parallel two-class 3D-CNN classifiers for video classification. In: 2017 International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS), pp 7–11

    Chapter  Google Scholar 

  36. S. Li, L. Zhu, Z. Zhang, and H. Zhang, “Learning to detect multi-view faces in real-time,” 2002, pp. 172–177.

  37. Li Q et al (2016) Multi-view face detector using a single cascade classifier. In: 2016 10th International Conference on Software, Knowledge, Information Management Applications (SKIMA), pp 464–468

    Google Scholar 

  38. Lienhart R, Maydt J (2002) An extended set of Haar-like features for rapid object detection. In: Proceedings. International Conference on Image Processing, vol 1, p I

    Chapter  Google Scholar 

  39. Liu C, Shum H-Y (2003) Kullback-Leibler boosting. In: 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 587–594

    Google Scholar 

  40. Loy CC, Zhang Z, Luo P, Tang X (2014) Facial landmark detection by deep multi-task learning. In: Proceedings of European Conference on Computer Vision (ECCV)

    Google Scholar 

  41. Marcolin F, Vezzetti E (2017) Novel descriptors for geometrical 3D face analysis. Multimed Tools Appl 76(12):13805–13834

    Article  Google Scholar 

  42. Martinez AM, Benavente R (1998) The AR face database. Tech. Rep. 24 CVC Technical Report

  43. McElroy D, Walsh C, Markwick AJ, Cordiner MA, Smith K, Millar TJ (2012) The UMIST database for astrochemistry 2012. Astron Astrophys 550:A36

    Article  Google Scholar 

  44. Mishra D, Chugh S (2013) Face detection using genetic based SIFT algorithm. In: 2013 International Conference on Green Computing, Communication and Conservation of Energy (ICGCE), pp 276–281

    Chapter  Google Scholar 

  45. Mita T, Kaneko T, Hori O (2005) Joint Haar-like features for face detection. In: Presented at the IEEE Int Conf Comp Vis, vol 2, pp 1619–1626

    Google Scholar 

  46. Mollahosseini A, Chan D, Mahoor MH (2016) Going deeper in facial expression recognition using deep neural networks. In: 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), pp 1–10

    Google Scholar 

  47. Munikrishna D. C, Raja K. B , Venugopal K. R (2018) Spatial Domain Face Recognition System Using Convolution of PDV and LBP. In: 2018 International conference on intelligent informatics and biomedical sciences (ICIIBMS), vol 3, pp 321–326

  48. Ojala T, Pietikäinen M, Mäenpää T (2002) Multiresolution gray-scale and rotation invariant texture classification with localbinary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987

    Article  MATH  Google Scholar 

  49. OpenCV library (n.d.) [Online]. Available: https://opencv.org

  50. Oualla M, Sadiq A (2016) Rotated asymetrique haar features for face detection. In: 2016 4th IEEE International Colloquium on Information Science and Technology (CiSt), pp 471–475

    Chapter  Google Scholar 

  51. Papageorgiou CP, Oren M, Poggio T (1998) A general framework for object detection. In: Proc. of ICCV, pp 555–562

    Google Scholar 

  52. Phillips P, Wechsler H, Huang J, Rauss P (1998) The FERET Database and Evaluation Procedure for Face recognition Algorithms. Image Vision Computing 5(16):295–306

    Article  Google Scholar 

  53. Pudil P, Novovičová J, Kittler J (Nov. 1994) Floating search methods in feature selection. Pattern Recogn Lett 15(11):1119–1125

    Article  Google Scholar 

  54. Qin H, Yan J, Li X, Hu X (2016) Joint training of cascaded cnn for face detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3456–3465

    Google Scholar 

  55. Rao GA, Syamala K, Kishore PV, Sastry ASCS (2018) Deep convolutional neural networks for sign language recognition. In: 2018 Conference on Signal Processing And Communication Engineering Systems (SPACES), pp 194–197

    Chapter  Google Scholar 

  56. Ringeval F, Sonderegger A, Sauer J, Lalanne D (2013) Introducing the RECOLA multimodal corpus of remote collaborative and affective interactions. In: 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), pp 1–8

    Google Scholar 

  57. Schapire RE, Singer Y (1999) Improved boosting algorithms using confidence-rated predictions. Mach Learn 37:297–336

    Article  MATH  Google Scholar 

  58. Sim T, Baker S, Bsat M (2002) The CMU Pose, Illumination, and Expression (PIE) database. In: Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition, pp 53–58

    Chapter  Google Scholar 

  59. Singh H, Agrawal D (2016) An analysis based on local binary pattern (LBP) and color moment (CM) for efficient image retrieval. In: 2016 International Conference on Emerging Technological Trends (ICETT), pp 1–6

    Google Scholar 

  60. Sun H, Xie Y, Sun B, Zhang H, Shang B, Fan G (2012) Fast Face Detection Based on Enhanced AdaBoost. 124:511–517

  61. Suse V, Ionescu D (2015) A real-time reconfigurable architecture for face detection. In: 2015 International Conference on ReConFigurable Computing and FPGAs (ReConFig), pp 1–6

    Google Scholar 

  62. Triantafyllidou D, Tefas A (2016) Face detection based on deep convolutional neural networks exploiting incremental facial part learning. In: 2016 23rd International Conference on Pattern Recognition (ICPR), pp 3560–3565

    Chapter  Google Scholar 

  63. Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 511–518

    Google Scholar 

  64. Viola P, Jones M (2002) Fast and Robust Classification using Asymmetric AdaBoost and a Detector Cascade. Proceedings of Advances in Neural Information Processing Systems 14

  65. Viola P, Jones M (2004) Robust real-time face detection. Int J Comput Vis 57:137–154

    Article  Google Scholar 

  66. Wang J, Wang B, Zheng Y, Liu W Research and Implementation on Face Detection Approach Based on Cascaded Convolutional Neural Networks. In: 2017 International conference on vision, Image and Signal Processing (ICVISP), vol 2017, pp 34–39

  67. Wu B, Ai H, Huang C, Lao S (2004) Fast rotation invariant multi-view face detection based on Real AdaBoost. In: Proc. of IEEE Conf. on Automatic Face and Gesture Recognition, pp 79–84

    Google Scholar 

  68. Xia Y, Zhang B, Coenen F (2015) Face occlusion detection based on multi-task convolution neural network. In: 2015 12th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), pp 375–379

    Google Scholar 

  69. Yanagisawa H, Yamashita T, Watanabe H (2018) A study on object detection method from manga images using CNN. In: 2018 International Workshop on Advanced Image Technology (IWAIT), pp 1–4

    Google Scholar 

  70. Yang HC, Xu AW (2015) Cascade face detection based on histograms of gradients and support vector machine. In: Parallel, grid, sCloud, and internet computing (3PGCIC), 2015 10th international conference on IEEE

    Google Scholar 

  71. Yann L, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE:2278–2324

  72. Ye X, Chen X, Chen H, Gu Y, Lv Q (2015) Deep learning network for face detection. In: 2015 IEEE 16th International Conference on Communication Technology (ICCT), pp 504–509

    Chapter  Google Scholar 

  73. Yi S, Yoon I, Oh C, Yi Y (2014) Real-time integrated face detection and recognition on embedded GPGPUs. In: 2014 IEEE 12th Symposium on Embedded Systems for Real-time Multimedia (ESTIMedia), pp 98–107

    Chapter  Google Scholar 

  74. Zhang C, Zhang Z (2010) A survey of recent advances in face detection. In: Tech. rep., Microsoft Research

    Google Scholar 

  75. Zhang C, Zhang Z (2014) Improving multiview face detection with multi-task deep convolutional neural networks. In: IEEE Winter Conference on Applications of Computer Vision, pp 1036–1041

    Chapter  Google Scholar 

  76. Zhou Y, Liu D, Huang T (2018) Survey of Face Detection on Low-Quality Images. In: 2018 13th IEEE International Conference on Automatic Face Gesture Recognition (FG 2018), pp 769–773

    Chapter  Google Scholar 

Download references

Acknowledgments

The authors of this study would like to show their sincere gratitude to the DEAP and RECOLA databases teams for offering these multimodal databases to evaluate this work: http://www.eecs.qmul.ac.uk/mmv/datasets/deap/index.html, https://diuf.unifr.ch/main/diva/recola/.

Author information

Authors and Affiliations

  1. Electrical Engineering Department, SITI Laboratory, National School of Engineers of Tunis, University of Tunis El Manar, BP. 37 Belvdre, 1002, Tunis, Tunisia

    Amal Adouani, Wiem Mimoun Ben Henia & Zied Lachiri

Authors
  1. Amal Adouani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wiem Mimoun Ben Henia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zied Lachiri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amal Adouani.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Adouani, A., Henia, W.M.B. & Lachiri, Z. A comparison of face detection methods using spontaneous videos. Multimed Tools Appl 81, 23163–23191 (2022). https://doi.org/10.1007/s11042-022-12781-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12781-8

Keywords

Search

Navigation