Skip to main content
SpringerLink
Log in

Statistical shape modelling for expression-invariant face analysis and recognition

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Paper introduces a 3-D shape representation scheme for automatic face analysis and identification, and demonstrates its invariance to facial expression. The core of this scheme lies on the combination of statistical shape modelling and non-rigid deformation matching. While the former matches 3-D faces with facial expression, the latter provides a low-dimensional feature vector that controls the deformation of model for matching the shape of new input, thereby enabling robust identification of 3-D faces. The proposed scheme is also able to handle the pose variation without large part of missing data. To assist the establishment of dense point correspondences, a modified free-form-deformation based on B-spline warping is applied with the help of extracted landmarks. The hybrid iterative closest point method is introduced for matching the models and new data. The feasibility and effectiveness of the proposed method was investigated using standard publicly available Gavab and BU-3DFE datasets, which contain faces with expression and pose changes. The performance of the system was compared with that of nine benchmark approaches. The experimental results demonstrate that the proposed scheme provides a competitive solution for face recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. Chellappa R, Sinha P, Phillips PJ (2010) Face recognition by computers and humans. Comput IEEE Comput Soc 43(2):46–55

    Article  Google Scholar 

  2. Chellppa R, Wilson C, Sirohey C (1995) Human and machine recognition of faces: a survey. Proc IEEE 83(5):705–740

    Article  Google Scholar 

  3. Jafri R, Arabnia HR (2009) A survery of face recognition techniques. J Inf Process Syst 53(2):41–66

    Article  Google Scholar 

  4. Kong SG, Heo J, Abidi BR, Paik J, Abidi M (2005) Recent advances in visual and infrared face recognition—a review. Comput Vis Image Underst 97:103–135

    Article  Google Scholar 

  5. Samal A, Iyengar P (1992) Automatic recognition and analysis of human faces and facial expressions: a survey. Pattern Recognit 25:65–77

    Article  Google Scholar 

  6. Lu Y, Zhou J, Yu S (2003) A survey of face detection. Extract Recognit Comput Inform Pattern Recognit 22:163–195

    MathSciNet  Google Scholar 

  7. Hadid A, Pietikainen M, Ahonen T (2004) A discriminative feature space for detecting and recognising faces. In: IEEE conference on computer vision and pattern recognition, pp 797–804

  8. Turk M, Pentland A (1991) Eigenfaces for recognition. J Cogn Neurosci 3(1):71–86

    Article  Google Scholar 

  9. Heseltine T, Pears N, Austin J, Chen Z (2003) Face recognition: a comparison of appearance-based approaches. Digital Image Comput Tech Appl 1:59–68

    Google Scholar 

  10. Delac K, Grgic M, Liatsis P (2005) Appearance-based statistical methods for face recognition. In: 47th international symposium ELMAR-2005 focused on multimedia systems and applications, pp 151–158

  11. Hesher C, Srivastava A, Erlebacher G (2003) A novel technique for face recognition using range imaging. In: International symposium on signal processing and its applications, pp 201–204

  12. Belhumeur P, Hespanha J, Kriegma D (1997) Eigenfaces vs. fisheraces: recognition using class specific projection. IEEE Trans Pattern Anal Mach Intell 19(7):328–340

    Article  Google Scholar 

  13. Ahonen T, Hadid A, Pietikainen M (2004) Face recognition with local binary patterns. In: European conference on computer vision, pp 469–481

  14. Bronstein A, Bronstein M, Kimmel R (2005) Three-dimensional face recognition. Int J Comput Vis 64(1):5–30

    Article  Google Scholar 

  15. Mpiperis I, Malassiotis S, Strintzis M (2007) 3-D face recognition with the geodesic polar representation. IEEE Trans Inform Forensic Secur 2(3):537–547

    Article  Google Scholar 

  16. Li X, Jia T, Zhang H (2009) Expression-insensitive 3D face recognition using sparse representation. In: Conference on computer vision and pattern recognition, pp 2575–2582

  17. Huang D, Ardabilian M, Wang Y, Chen L (2012) 3-D face recognition using elbp-based facial description and local feature hybrid matching. IEEE Trans Inf Forensic Secur 7(5):1551–1565

    Article  Google Scholar 

  18. Mohammadzade H, Hatzinakos D (2013) Iterative closest normal point for 3D face recognition. IEEE Trans Pattern Anal Mach Intell 35(2):381–397

    Article  Google Scholar 

  19. Shi J, Samal A, Marx D (2006) Effective are landmarks and their geometry for face recognition. Comput Vis Image Underst 102(2006):117–133

    Article  Google Scholar 

  20. Gupta S, Aggatwal JK, Markey MK, Bovik AC (2007) 3D face recognition founded on the structural diversity of human faces. In: IEEE international conference on computer vision and pattern recognition

  21. Zhang L, Razdan A, Farin G, Femiani J, Bae M, Lockwood C (2006) 3D face authentication and recognition based on bilateral symmetry analysis. Vis Comput Comput Vis Image Underst 22(1)

  22. Nagamine T, Uemura T, Masuda I (1992) 3-D facial image analysis for human identification. In: International conference on computer vision and pattern recognition

  23. Samir C, Srivastava A, Acheroy M (2006) Three-dimensional face recognition using shapes of facial curves. IEEE Trans Pattern Anal Mach Intell 28(11):1858–1863

    Article  Google Scholar 

  24. Drira H, Amor BB, Mohamed D, Srivastava A (2010) Pose and expression-invariant 3-D face recognition using elastic radial curves. In: British machine vision conference

  25. Queirolo CC, Silva L, Bellon ORP, Segundo MP (2010) 3D face recognition using simulated annealing and the surface interpenetration measure. IEEE Trans Pattern Anal Mach Intell 32(2):206–219

    Article  Google Scholar 

  26. Moreno AB, Sanchez A, Velez JF, Diaz FJ (2003) Face recognition using 3D surface-extracted descriptors. In: Conference on Irish machine vision and image processing

  27. Hajati F, Raie AA, Goa Y (2012) 2.5D face recognition using patch geodesic moments. Pattern Recognit 45(2012):969–982

    Article  MATH  Google Scholar 

  28. Gordan GG (1992) Face recognition based on depth and curvature features. In: International conference on computer vision and pattern recognition, pp 808–810

  29. Tanaka HT, Ikeda M, Chiaki H (1998) Curvature-based face surface recognition using spherical correlation—principal directions for curved object recognition. In: International conference on automatic face and gesture recognition, pp 372–377

  30. Wang Y, Liu J, Tang X (2010) Robust 3D face recognition by local shape difference boosting. IEEE Trans Pattern Anal Mach Intell 32(10):1858–1870

    Article  Google Scholar 

  31. Srivastava A, Liu X, Hesher C (2006) Face recognition using optimal linear components of face images. J Image Vis Comput 24(3):291–299

    Article  Google Scholar 

  32. Lu X, Jain AK (2008) Deformation modelling for robust 3D face matching. In: International conference on computer vision and pattern recognition

  33. Haar FB, Veltkamp RC (2010) Expression modeling for expression-invariant face recognition. J Comput Graph 34(2010):231–241

    Article  Google Scholar 

  34. Besl P, McKay N (1997) A method for registration of 3-D shapes. IEEE Trans Pattern Anal Mach Intell 14(2):239–256

    Article  Google Scholar 

  35. Zhang Z (1994) Iterative point matching for registration of free-form curve and surfaces. Int J Comput Vis 13(1):119–152

    Article  Google Scholar 

  36. He X, Yan S, Hu Y, Niyogi P, Zhang HJ (2005) Face recognition using Laplacianfaces. IEEE Trans Pattern Anal Mach Intell 27(3):328–340

    Article  Google Scholar 

  37. Belkin M, Niyogi P (2001) Laplacian eigenmaps and spectral techniques for embedding and clustering. In: Conference on advances in neural information processing system

  38. Jahne HB, HauBecker F (2000) Computer vision and application. Academic Press, London

  39. Papiez BW, Matuszewski BJ, Shark L-K, Quan W (2012) Facial expression recognition using LOG-euclidean statistical shape models. In: International conference on pattern recognition applications and methods, ICPRAM

  40. Papiez BW, Matuszewski BJ, Shark L-K, Quan W (2012) Facial expression recognition using diffeomorphic image registration framework. Math Methodol Pattern Recognit Mach Learn 30:179–194

    Article  MATH  Google Scholar 

  41. Rohr K, Stiehl HS, Sprengel R, Buzug TM, Weese J, Kuhn MH (2001) Landmark-based elastic registration using approximating thin-plate splines. IEEE Trans Med Imaging 20(6):526–534

    Article  Google Scholar 

  42. Joshi A (2009) Optimization of landmark selection for cortical surface registration. In: International conferences on computer vision and pattern recognition, pp 669–706

  43. Lu X, Jain AK, Colbry D (2006) Matching 2.5D face scans to 3D models. IEEE Trans Pattern Anal Mach Intell 28(1):31–43

    Article  Google Scholar 

  44. Quan W, Matuszewski BJ, Shark L-K (2012) Facial asymmetry analysis based on 3-D dynamic scans. In: IEEE international conference on system, man and cybernetics, pp 2676–2681

  45. Kimmel R, Sethian JA (1998) Computing geodesic on manifolds. US Natl Acad Sci 95:8431–8435

    Article  MathSciNet  MATH  Google Scholar 

  46. Bronstein MM, Bronstein AM, Kimmel R, Yavneh I (2006) Multigrid multidimensional scaling. Numer Linear Algebra Appl 13:149–171

    Article  MathSciNet  MATH  Google Scholar 

  47. Pighin F, Heaker J, Lischinski D, Szeliski R, Salesin DH (1998) Synthesizing realistic facial expression from photographs. In: ACM SIGGRAPH, pp 75–84

  48. Noh J, Neumann U (2001) Expression cloning. In: ACM SIGGRAPH, pp 277–288

  49. Quan W, Matuszewski BJ, Shark L-K, Ait-Boudaoud D (2009) Facial expression biometrics using statistical shape models. EURASIP J Adv Signal Process 1–17

  50. Ruechert D, Sonoda L, Hayes C, Hill D, Leach M, Hawkes D (1999) Nonrigid registration using free-form deformation: application to breast MR images. IEEE Trans Med Imaging 18(8):712–721

    Article  Google Scholar 

  51. Sandbach G, Zafeiriou S, Pantic M, Rueckert D (2011) A dynamic approach to the recognition of 3-D facial expressions and their temporal models. In: IEEE conference on automatic face and gesture recognition, pp 406–413

  52. Sandbach G, Zafeiriou S, Pantic M, Rueckert D (2012) Recognition of 3-D facial expression. Image Vis Comput 30(10):762–773

    Article  Google Scholar 

  53. Bookstein FL (1989) Principal warps: thin-plate splines and the decomposition of deformation. IEEE Trans Pattern Anal Mach Intell 11(6):567–585

    Article  MATH  Google Scholar 

  54. Lee S, Wolberd G, Shin S (1997) Scattered data interpolation with multilevel B-splines. IEEE Trans Vis Comput Graph 3(3):228–244

    Article  Google Scholar 

  55. Dhillon IS, Guan Y, Kulis B (2007) Weighted graph cuts without eigenvectors: a multilevel approach. IEEE Trans Pattern Anal Mach Intell 29(11):1–14

    Article  Google Scholar 

  56. Qiu D, May S, Nuchter A (2009) GPU-accelerated nearest neighbor search for 3D registration. In: Proceedings of the 7th international conference on computer vision systems: computer vision systems, pp 194–203

  57. Yin L, Wei X, Sun Y, Wang J, Rosato M (2006) A 3D facial expression database for facial behaviour research. In: 7th international conference on automatic face and gesture recognition, pp 211–216

  58. Moreno AB, Sanchez A (2004) GavabDB: a 3D face database. In: COST workshop on biometrics on internet: fundamentals, advances and applications, pp 77–82

  59. Moghaddam B, Pentland A (1997) Probabilistic visual learning for object representation. IEEE Trans Pattern Anal Mach Intell 19(6):696–710

    Article  Google Scholar 

  60. Phillips PJ (1998) Support vector machines applied to face recognition. Adv Neural Inf Process Syst 11:803–809

    Google Scholar 

  61. Rizvi SA, Philips PJ, Moon H (1998) The FERET verification testing protocol for face recognition algorithms. In: IEEE international conference on automatic face and gesture recognition

  62. Moreno AB, Sanchez A, Velez J, Diaz F (2005) Face recognition using 3D local geometrical features: PCA vs SVM. In: International symposium on image and signal processing and analysis

  63. Mahoor MH, Abdel-Mottaleb M (2009) Face recognition based on 3D ridge images obtained from range data. Pattern Recognit 42(3):445–451

    Article  MATH  Google Scholar 

  64. Berrett S, Bimbo A, Pala P (2006) 3D face recognition by modeling the arrangement of concave and convex regions. In: Adaptive multimedia retrieval, pp 108–118

Download references

Acknowledgments

The authors would like to acknowledge Dr. Lijun Yin from Binghamton University (USA) for making available to the BU-3DFE database.

Author information

Authors and Affiliations

  1. Robotics and Computer Vision Research Laboratory, ADSIP Research Centre, School of Computing, Engineering and Physical Sciences, University of Central Lancashire, CM124, C&T Building, Preston, PR1 2HE, Lancashire, UK

    Wei Quan

  2. Robotics and Computer Vision Research Laboratory, ADSIP Research Centre, School of Computing, Engineering and Physical Sciences, University of Central Lancashire, CM149, C&T Building, Preston, PR1 2HE, Lancashire, UK

    Bogdan J. Matuszewski

  3. Robotics and Computer Vision Research Laboratory, ADSIP Research Centre, School of Computing, Engineering and Physical Sciences, University of Central Lancashire, CM147, C&T Building, Preston, PR1 2HE, Lancashire, UK

    Lik-Kwan Shark

Authors
  1. Wei Quan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bogdan J. Matuszewski
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lik-Kwan Shark
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei Quan.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Quan, W., Matuszewski, B.J. & Shark, LK. Statistical shape modelling for expression-invariant face analysis and recognition. Pattern Anal Applic 19, 765–781 (2016). https://doi.org/10.1007/s10044-014-0439-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-014-0439-x

Keywords

Search

Navigation