Skip to main content
SpringerLink
Log in

A deformation model to reduce the effect of expressions in 3D face recognition

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In 3D face recognition, most work utilizes the rigid parts of face surfaces for matching to exclude the distortion caused by expressions. However, across a broad range of expressions, the rigid parts may not always be uniform and cover large parts of faces. On the other hand, the non-rigid regions of face surfaces also contain useful information for recognition. In this paper, we include the non-rigid regions besides the rigid parts for 3D face recognition. A deformation model is proposed to deform the non-rigid regions to the shapes that are more similar between intra-personal samples but less similar between inter-personal samples. Together with the rigid regions, the deformed parts make samples more discriminable so that the effect of expressions is reduced. The first part of our model uses the target gradient fields from enrolled samples to depress the distortion of the non-rigid regions. The gradient field works in the differential domain. According to the Poisson equation, a smooth deformed shape can be computed by a linear system. The second part of the model is the definition of a surface property that determines the deformation ability of different face regions. Unlike the target gradient fields that improve the similarity of intra-personal samples, the original topology and surface property can keep inter-personal samples sufficiently dissimilar. Our deformation model can be used to improve existing 3D face recognition methods. Experiments are carried out on FRGC and BU-3DFE databases. There are about 8–10% improvements obtained after applying this deformation model to the baseline ICP method. Compared with other deformation models, the experimental results show that our model has advantages on both recognition performance and computational efficiency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Medioni, G., Waupotitsch, R.: Face modeling and recognition in 3-D. In: Proceedings of the IEEE International Workshop on Analysis and Modeling of Faces and Gestures, pp. 232–233 (2003)

    Google Scholar 

  2. Zhao, W., Chellappa, R., Phillips, P.J., Rosenfeld, A.: Face recognition: a literature survey. ACM Comput. Surv. 35(4), 399–458 (2003)

    Article  Google Scholar 

  3. Besl, P.J., McKay, N.D.: A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992)

    Article  Google Scholar 

  4. Scheenstra, A., Ruifrok, A., Veltkamp, R.C.: A survey of 3D face recognition methods. In: Proceedings of the International Conference on Audio- and Video-Based Biometric Person Authentication, pp. 891–899 (2005)

    Chapter  Google Scholar 

  5. Chang, K., Bowyer, K.W., Flynn, P.: Effects on facial expression in 3D face recognition. In: Proceedings of the SPIE Conference on Biometric Technology for Human Identification, vol. 5779, pp. 132–143 (2005)

    Google Scholar 

  6. Chang, K., Bowyer, K.W., Flynn, P.: Multiple nose region matching for 3D face recognition under varying facial expression. IEEE Trans. Pattern Anal. Mach. Intell. 28(10), 1695–1700 (2006)

    Article  Google Scholar 

  7. Bowyer, K.W., Chang, K., Flynn, P.: A survey of approaches and challenges in 3D and multi-modal 2D + 3D face recognition. Comput. Vis. Image Underst. 101(1), 1–15 (2006)

    Article  Google Scholar 

  8. Faltemier, T., Bowyer, K.W., Flynn, P.: Using multi-instance enrollment to improve performance 3D face recognition. Comput. Vis. Image Underst. 112(2), 114–125 (2008)

    Article  Google Scholar 

  9. Nagamine, T., Uemura, T., Masuda, I.: 3D facial image analysis for human identification. In: Proceedings of the International Conference on Pattern Recognition, pp. 324–327 (1992)

    Google Scholar 

  10. Samir, C., Srivastava, A., Daoudi, M.: Three-dimensional face recognition using shapes of facial curves. IEEE Trans. Pattern Anal. Mach. Intell. 28(11), 1858–1863 (2006)

    Article  Google Scholar 

  11. Moreno, A.B., Sánchez, A., Vélez, J.F., Díaz, F.J.: Face recognition using 3d surface-extracted descriptors. In: Proceedings of the Irish Machine Vision and Image Processing Conference (2003)

    Google Scholar 

  12. Gupta, S., Aggatwal, J.K., Markey, M.K., Bovik, A.C.: 3D face recognition founded on the structural diversity of human faces. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2007)

    Google Scholar 

  13. Wang, Y., Pan, G., Wu, Z., Wang, Y.: Exploring facial expression effects in 3D face recognition using partial ICP. Proc. Asian Conf. Comput. Vis. 3851, 581–590 (2006)

    Google Scholar 

  14. Lu, X., Jain, A.K., Colbry, D.: Matching 2.5D face scans to 3D models. IEEE Trans. Pattern Anal. Mach. Intell. 28(1), 31–43 (2006)

    Article  Google Scholar 

  15. Gordon, G.G.: Face recognition based on depth and curvature features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 808–810 (1992)

    Chapter  Google Scholar 

  16. Tanaka, H.T., Ikeda, M., Chiaki, H.: Curvature-based face surface recognition using spherical correlation—principal directions for curved object recognition. In: Proceedings of the International Conference on Automatic Face and Gesture Recognition, pp. 372–377 (1998)

    Chapter  Google Scholar 

  17. Achermann, B., Bunke, H.: Classifying range images of human faces with Hausdorff distance. In: Proceedings of the International Conference on Pattern Recognition, pp. 809–813 (2000)

    Google Scholar 

  18. Achermann, B., Jiang, X., Bunke, H.: Face recognition using range images. In: Proceedings of the International Conference on Virtual Systems and MultiMedia, pp. 129–136 (1997)

    Google Scholar 

  19. Hesher, C., Srivastava, A., Erlebacher, G.: A novel technique for face recognition using range imaging. In: Proceedings of the International Symposium on Signal Processing and Its Applications, pp. 201–204 (2003)

    Chapter  Google Scholar 

  20. Lee, Y., Shim, J.: Curvature-based human face recognition using depth-weighted Hausdorff distance. In: Proceedings of the International Conference on Image Processing, pp. 1429–1432 (2004)

    Google Scholar 

  21. Russ, T.D., Koch, M.W., Little, C.Q.: A 2D range Hausdorff approach for 3D face recognition. In: Proceedings of the IEEE Workshop on Face Recognition Grand Challenge Experiments (2005)

    Google Scholar 

  22. Lu, X., Jain, A.K.: Deformation modeling for robust 3D race matching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1377–1383 (2006)

    Google Scholar 

  23. Lu, X., Jain, A.K.: Deformation modeling for robust 3D face matching. IEEE Trans. Pattern Anal. Mach. Intell. 30(8), 1346–1357 (2008)

    Article  Google Scholar 

  24. Xu, C., Tan, T., Li, S., Wang, Y., Zhong, C.: Learning effective intrinsic features to boost 3D-based face recognition. In: Proceedings of the European Conference on Computer Vision, pp. 416–427 (2006)

    Google Scholar 

  25. Chua, C., Han, F., Ho, Y.: 3D human face recognition using point signature. In: Proceedings of the International Conference on Automatic Face and Gesture Recognition, pp. 233–238 (2000)

    Google Scholar 

  26. Maurer, T., Guigonis, D., Maslov, I., Pesenti, B., Tsaregorodtsev, A., West, D., Medioni, G.: Performance of geometrix ActiveIDTM 3D face recognition engine on the FRGC data. In: IEEE Workshop on Face Recognition Grand Challenge Experiments (2005)

    Google Scholar 

  27. Pan, G., Wu, Y., Wu, Z., Liu, W.: 3D face recognition by profile and surface matching. In: Proceedings of the IEEE International Joint Conference on Neural Network, pp. 2168–2174 (2003)

    Google Scholar 

  28. Phillips, P.J., Grother, P., Michaels, R.J., Blackburn, D.M., Tabassi, E., Bone, J.: FRVT 2002: overview and summary. Available at www.frvt.org

  29. Phillips, P.J., Scruggs, W.T., O’Toole, A.J., Flynn, P., Bowyer, K.W., Schott, C.L., Sharpe, M.: FRVT 2006 and ICE 2006 large-scale results. Available at www.frvt.org

  30. Bronstein, A.M., Bronstein, M.M., Kimmel, R.: Expression-invariant 3D face recognition. In: Proceedings of the International Conference on Audio- and Video-Based Biometric Person Authentication, pp. 62–70 (2003)

    Chapter  Google Scholar 

  31. Bronstein, A.M., Bronstein, M.M., Kimmel, R.: Three-dimensional face recognition. Int. J. Comput. Vis. 64(1), 5–30 (2005)

    Article  Google Scholar 

  32. Phillips, P.J., Flynn, P., Scruggs, T., Bowyer, K.W., Chang, J., Hoffman, K., Marques, J., Min, J., Worek, W.: Overview of the face recognition grand challenge. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 947–954 (2005)

    Google Scholar 

  33. Yin, L., Wei, X., Sun, Y., Wang, J., Rosato, M.: A 3D facial expression database for facial behavior research. In: Proceedings of the International Conference on Automatic Face and Gesture Recognition, pp. 211–216 (2006)

    Google Scholar 

  34. Wang, Y., Pan, G., Wu, Z.: 3D face recognition in the presence of expression: a guidance-based constraint deformation approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2007)

    Google Scholar 

  35. Kakadiaris, I.A., Passalis, G., Toderici, G., Murtuza, N., Lu, Y., Karampatziakis, N., Theoharis, T.: Three-dimensional face recognition in the presence of facial expressions: an annotated deformable model approach. IEEE Trans. Pattern Anal. Mach. Intell. 29(4), 640–649 (2007)

    Article  Google Scholar 

  36. Mian, A.S., Bennamoun, M., Owens, R.: An efficient multimodal 2D-3D hybrid approach to automatic face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 29(11), 1927–1943 (2007)

    Article  Google Scholar 

  37. Mian, A.S., Bennamoun, M., Owens, R.: Keypoint detection and local feature matching for textured 3D face recognition. Int. J. Comput. Vis. 79(1), 1–12 (2008)

    Article  Google Scholar 

  38. Faltemier, T., Bowyer, K.W., Flynn, P.: A region ensemble for 3d face recognition. IEEE Trans. Inf. Forensics Secur. 3(1), 62–73 (2008)

    Article  Google Scholar 

  39. Wang, Y., Tang, X., Liu, J., Pan, G., Xiao, R.: 3D face recognition by local shape difference boosting. In: Proceedings of the European Conference on Computer Vision, pp. 603–616 (2008)

    Google Scholar 

  40. Meyer, M., Desbrun, M., Schoroder, P., Barr, A.: Discrete differential-geometry operators for triangulated 2-manifolds. In: Proceedings of VisMath (2002)

    Google Scholar 

  41. Tong, Y., Lombeyda, S., Hirani, A.N., Desbrun, M.: Discrete multiscale vector field decomposition. ACM Trans. Graph. 22(3), 445–452 (2003)

    Article  Google Scholar 

  42. Polthier, K., Preuss, E.: Variational approach to vector field decomposition. In: Proceedings of the Eurographics Workshop on Scientific Visualization (2000)

    Google Scholar 

  43. Polthier, K., Preuss, E.: Identifying vector field singularities using a discrete Hodge decomposition. In: Visualization and Mathematics (2002)

    Google Scholar 

  44. Yu, Y., Zhou, K., Xu, D., Shi, X., Bao, H., Guo, B., Shum, H.-Y.: Mesh editing with Poisson-based gradient field manipulation. ACM Trans. Graph. 23(3), 644–651 (2004)

    Article  Google Scholar 

  45. Desbrun, M., Meyer, M., Alliez, P.: Intrinsic parameterizations of surface meshes. In: Proceedings of the Eurographics (2002)

    Google Scholar 

  46. Bronstein, A.M., Bronstein, M.M., Kimmel, R.: Generalized multidimensional scaling: a framework for isometry-invariant partial surface matching. Proc. Natl. Acad. Sci. 103(5), 1168–1172 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  47. Bronstein, A.M., Bronstein, M.M., Kimmel, R.: Robust expression-invariant face recognition from partially missing data. In: Proceedings of the European Conference on Computer Vision, pp. 396–408 (2006)

    Google Scholar 

  48. Boehnen, C., Peters, T., Flynn, P.: 3D signatures for fast 3D face recognition. In: Proceedings of the IAPR/IEEE International Conference on Biometrics, pp. 12–21 (2009)

    Google Scholar 

  49. Queirolo, C.C., Silva, L., Bellon, O.R.P., Segundo, M.P.: 3D face recognition using simulated annealing and the surface interpenetration measure. IEEE Trans. Pattern Anal. Mach. Intell. 32(2), 206–219 (2010)

    Article  Google Scholar 

  50. Abraham, R., Marsden, J., Ratiu, T.: Manifolds, Tensor Analysis, and Applications. Applied Mathematical Sciences, vol. 75. Springer, Berlin (1988)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information Engineering, The Chinese University of Hong Kong, Hong Kong, China

    Yueming Wang & Jianzhuang Liu

  2. Qiushi Academy for Advanced Studies, Zhejiang University, Hangzhou, China

    Yueming Wang

  3. College of Computer Science, Zhejiang University, Hangzhou, China

    Gang Pan

Authors
  1. Yueming Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gang Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianzhuang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gang Pan.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wang, Y., Pan, G. & Liu, J. A deformation model to reduce the effect of expressions in 3D face recognition. Vis Comput 27, 333–345 (2011). https://doi.org/10.1007/s00371-010-0530-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-010-0530-2

Keywords

Search

Navigation