Skip to main content

Advertisement

SpringerLink
Log in

A multimodal approach for 3D face modeling and recognition using 3D deformable facial mask

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

We present a multimodal approach for face modeling and recognition. The algorithm uses three cameras to capture stereo images, two frontal and one profile, of the face. 2D facial features are extracted from one of the frontal images and a dense disparity map is computed from the two frontal images. Using the extracted 2D features and their corresponding disparities, we compute their 3D coordinates. We next align a low resolution 3D mesh model to the 3D features, re-project its vertices onto the frontal 2D image and adjust its profile silhouette vertices using the profile view image. We increase the resolution of the resulting 2D model at its center region to obtain a facial mask model covering distinctive features of the face. The 2D coordinates of the vertices, along with their disparities, result in a deformed 3D mask model specific to a given subject’s face. Our method integrates information from the extracted facial features from the 2D image modality with information from the 3D modality obtained from the stereo images. Application of the models in 3D face recognition, for 112 subjects, validates the algorithm with a 95% identification rate and 92% verification rate at 0.1% false acceptance rate.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ansari, A., Abdel-Mottaleb, M.: Automatic facial feature extraction and 3D face modeling using two orthogonal views with application to 3D face recognition. Pattern Recognit. 38(12), 2549–2563 (2005)

    Article  Google Scholar 

  2. Ansari, A., Abdel-Mottaleb, M., Mahoor, M.: Disparity-based 3D face modeling using 3D deformable facial mask for 3D face recognition. In: IEEE International Conference on Multimedia and Expo, Toronto, Ontario, Canada, July 2006

  3. Blanz, V., Vetter, T.: A morphable model for the syntheses of 3D faces. In: Proc. of int. Conference on Computer Graphics, pp. 187–194 (1999)

  4. Bowyer, K., Chang, K., Flynn, P.: A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition. Comput. Vis. Image Underst. 101, 1–15 (2006)

    Article  Google Scholar 

  5. Chang, K., Bowyer, K., Flynn, P.: Face recognition using 2D and 3D facial data. In: Multimodal User Authentication Workshop, pp. 25–32 (2003)

  6. Chang, K., Bowyer, K., Flynn, P.: An evaluation of multimodal 2D+3D face biometrics. IEEE Trans. Pattern Anal. Mach. Intell. 27(4), 619–624 (2005)

    Article  Google Scholar 

  7. Chen, Q., Medioni, G.: A volumetric stereo matching method: application to image-based modeling. In: Proceedings of IEEE CVPR (1999)

  8. Chen, M., Hauptmann, A.: Towards robust face recognition from multiple views. In: International Conference on Multimedia and Expo, Taipei, Taiwan, 27–30 June 2004

  9. Chen, X., Faltemier, T., Flynnn, P., Bowyer, K.: Human face modeling and recognition through multi-view high resolution stereopsis. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshop, June 2006

  10. Chowdhury, A., Chellappa, R.: Face reconstruction from monocular video using uncertainty analysis and a generic model. Comput. Vis. Image Underst. 91, 188–213 (2003)

    Article  Google Scholar 

  11. Chua, C., Han, F., Ho, Y.: 3D human face recognition using point signature. In: Proceedings of AFGR (2000)

  12. Fisher, R.: The use of multiple measurements in taxonomic problems. Ann Eugen. 7(part II), 179–188 (1936)

    Google Scholar 

  13. Fua, P.: Using model-driven bundle-adjustment to model heads from raw video sequences. In: International Conference on Computer Vision, Corfu, Greece, pp. 46–53 (1999)

  14. Godil, A., Ressler, S., Grother, P.: Face recognition using 3D facial shape and color map information comparison and combination. In: Biometric Technology for Human Identification, SPIE, vol. 5404, pp. 351–361 (2005)

  15. Gordon, G.: Face recognition based on depth maps and surface curvature. In: Proceedings of SPIE: Geometric Methods in Computer Vision, vol. 1570, pp. 234–247 (1990)

  16. Gordon, G.: Face recognition from frontal and profile views. In: Proceedings of IWAFGR, Zurich, pp. 47–52 (1995)

  17. Guenter, B., Grimm, C., Malvar, H., Wood, D.: Making faces. In: Proceedings of SIGGRAPH (1998)

  18. Harris, C.G., Stephens, M.J.: A combined corner and edge detector. In: Proceedings of AVC Conference, Manchester (1988)

  19. Hehser, C., Srivastava, A., Erlebacher, G.: A novel technique for face recognition using range images. In: Seventh International Symposium on Signal Processing and its Application (2003)

  20. Hsu, R.L., Jain, A.K.: Face modeling for recognition. In: Proceedings of the International Conference on Image Processing, Greece, October 2001

  21. Hsu, R., Abdel-Mottaleb, M., Jain, A.: Face detection in color images. IEEE Trans. PAMI 24(5), 696–706 (2002)

    Google Scholar 

  22. Hurley, J., Cattell, R.: The procrustes program: producing direct rotation to test a hypothesized factor structure. Behav. Sci. 7, 258–262 (1962)

    Article  Google Scholar 

  23. Husken, M., Brauckmann, M., Gehlen, S., von der Malsburg, C.: Strategies and benefits of fusion of 2D and 3D face recognition. In: IEEE Workshop on Face Recognition Grand Challenge Experiments, June 2005

  24. Lanitis, A., Taylor, C.J., Cootes, T.F.: Automatic interpretation and coding of face images using flexible models. IEEE Trans. PAMI 19(7), 743–756 (1997)

    Google Scholar 

  25. Lee, Y., Terzopuolos, D., Waters, K.: Realistic modeling for facial animation. In: Proc. of SIGGRAPH, Los Angeles, pp. 55–61 (1995)

  26. Lee, W., Thalmann, N.: Fast head modeling for animation. Image Vis. Comput. 18(4), 355–364 (2000)

    Article  Google Scholar 

  27. Lee, Y., Park, K., Shim, J., Yi, T.: 3D face recognition using statistical multiple features for the local depth information. In: Proceedings of Vision Interface (2003)

  28. Lengagne, R., Fua, P., Monga, O.: 3D face modeling from stereo and differential constraints. In: Proceedings of ICAFGR, pp. 148–153 (1998)

  29. Lu, X., Jain, A.: Integrating range and texture information for 3D face recognition. In: 7th IEEE Workshop on Applications of Computer Vision (WACV ’05), pp. 155–163 (2005)

  30. Lu, X., Colbry, D., Jain, A.: Three-dimensional model based face recognition. In: Proceedings of Patter Recognition, UK (2004)

  31. Lu, X., Hsu, R., Jain, A., Kamgar-Parsi, B., Kamgar-Parsi, B.: Face recognition with 3D model-based synthesis. In: Proceedings of International Conference on Biometric Authentication, vol. 3072 of Lecture Notes in Computer Science, Hong Kong, pp. 139–146 (2004)

  32. Liu, Z.: A fully automatic system to model faces from a single image. Microsoft research technical report, TR-2003–55 (2003)

  33. Mahoor, M., Abdel-Mottaleb, M., Ansari, A.: Improved active shape model for facial feature extraction in color images. J. Multimed. 1(4), 21–28 (2006)

    Google Scholar 

  34. Mallick, S., Trivedi, M.: Parametric face modeling and affect synthesis. In: Proceedings of IEEE on ICME, Baltimore, Maryland, July 2003

  35. Maurer, T., Guigonis, D., Maslov, I., Pesenti, B., Tsaregorodtsev, A., West, D., Medioni, G.: Performance of geometrix activeID∧TM 3D face recognition engine on the FRGC data. In: IEEE Workshop on Face Recognition Grand Challenge Experiments, June 2005

  36. Medioni, G., Waupotitsch, R.: Face recognition and modeling in 3D. In: IEEE International Workshop on Analysis and Modeling of Faces and Gestures, pp. 232–233 (2003)

  37. Moreno, A., Sanchez, A., Velez, J., Diaz, F.: Face recognition using 3D surface-extracted descriptors. In: Irish Machine Vision and Image Processing Conference (2003)

  38. Papatheodorou, T., Reuckert, D.: Evaluation of automatic 4D face recognition using surface and texture registration. In: Sixth International Conference on Automated Face and Gesture Recognition, pp. 321–326 (2004)

  39. Reinders, M., Beek, P., Sankur, B., Lubbe, J.: Facial feature localization and adaptation of a generic face model for model based coding. SP Image Commun 7(1), 57–74 (1995)

    Article  Google Scholar 

  40. Salma, C.: Manual of Photometry, 4th ed. American Society of Photometry, Virginia (1980)

    Google Scholar 

  41. Sampson, P.D.: Fitting conic sections to very scattered data: an iterative refinement of the bookstein algorithm. In: CVGIP (1982)

  42. Shan, Y., Liu, Z., Zhang, Z.: Model-based bundle adjustment with application to face modeling. ICCV 2001 2, 644–651 (2001)

    Google Scholar 

  43. Tsalakanidou, F., Tzocaras, D., Strintzis, M.: Use of depth and colour eigenfaces for face recognition. Pattern Recognit. Lett. 24, 1427–1435 (2003)

    Article  MATH  Google Scholar 

  44. Tsalakanidou, F., Malassiotis, S., Strintzis, M.: Integration of 2D and 3D images for enhanced face authentication. In: Sixth International Conference on Automated Face and Gesture Recognition, pp. 266–271 (2004)

  45. Turk, M., Pentland, A.: Eigenfaces for recognition. J. Cogn. Neurosci. 3(1), 71–86 (1991)

    Article  Google Scholar 

  46. Yan, J., Zhang, H.: Synthesized virtual view-based Eigenspace for face recognition. In: Fifth IEEE Workshop on Application of Computer Vision (2000)

  47. Yin, L., Yourst, M.: 3D face recognition based on high-resolution 3D face modeling from frontal and profile view. In: Proceedings of ACM SIGMM workshop on biometrics methods and applications, Berkley, California (2003)

  48. Zhaho, W., Chellappa, R., Phillips, P., Rosenfeld, A.: Face recognition: a literature survey. ACM Computing Survey, pp. 399–458 (2003)

  49. Zhang, Z.: Flexible camera calibration by viewing a plane from unknown orientations. In: Proc. of ICCV, Greece, pp. 666–673 (1999)

  50. Zhang, C., Cohen, F.: 3-D face structure extraction and recognition from images using 3-D morphing and distance mapping. IEEE Trans. Image Process. 11(11), 1249–1259 (2002)

    Article  MathSciNet  Google Scholar 

  51. Zhang, H., Negahdaripour, S.: Fast and robust progressive stereo reconstruction via symmetry guided fusion. In: Proceedings of Oceans ’05 Europe, Brest, France, June 2005

  52. Zhang, Z., Jacobs, C., Cohen, M.: Rapid modeling of animated faces from video. Technical Report, Microsoft Research (1999)

  53. Zhang, Y., Sim, T., Tan, C.: From range data to animated anatomy-based faces: a model adaptation method. In: 3dim, Fifth International Conference on 3-D Digital Imaging and Modeling, pp. 343–350 (2005)

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, University of Miami, 1251 Memorial Drive, Coral Gables, FL, 33124, USA

    A-Nasser Ansari, Mohamed Abdel-Mottaleb & Mohammad H. Mahoor

Authors
  1. A-Nasser Ansari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohamed Abdel-Mottaleb
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammad H. Mahoor
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohamed Abdel-Mottaleb.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ansari, AN., Abdel-Mottaleb, M. & Mahoor, M.H. A multimodal approach for 3D face modeling and recognition using 3D deformable facial mask. Machine Vision and Applications 20, 189–203 (2009). https://doi.org/10.1007/s00138-008-0123-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-008-0123-5

Keywords

Search

Navigation