Skip to main content
SpringerLink
Log in

Nearest neighbor weighted average customization for modeling faces

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

In this paper, we present an anatomically accurate generic wireframe face model and an efficient customization method for modeling human faces. We use a single 2D image for customization of the generic model. We employ perspective projection to estimate 3D coordinates of the 2D facial landmarks in the image. The non-landmark vertices of the 3D model are shifted using the translations of k nearest landmark vertices, inversely weighted by the square of their distances. We demonstrate on Photoface and Bosphorus 3D face data sets that the proposed method achieves substantially low relative error values with modest time complexity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Parke, F.I.: “A parametric model for human faces”. Ph.D. Dissertation, The University of Utah, AAI7508697 (1974)

  2. Vizireanu, N., Udrea, R.: Visual-oriented morphological foreground content grayscale frames interpolation method. J. Electron. Imaging 18(2), 020502–020502-3 (2009)

    Article  Google Scholar 

  3. Zhang, Z., et al.: Robust and rapid generation of animated faces from video images: A model-based modeling approach. Int. J. Comput. Vision 58(2), 93–119 (2004)

    Article  Google Scholar 

  4. Blanz, V., Vetter, T.: “A morphable model for the synthesis of 3D faces”. in Proc of the SIGGRAPH’99, pp. 187–194 (1999)

  5. Lee, W.S., Magnenat, N.T.: Fast head modeling for animation. Image Vis. Comput. J. 4, 355–364 (2000)

    Article  Google Scholar 

  6. Marschner, S.R., et al.: “Modeling and rendering for realistic facial animation”. In: Proceedings of the 11th Eurographics Workshop on Rendering (2000)

  7. Blanz, V., et al.: “Face identification across different poses and illuminations with a 3D morphable model”. In: Proceedings of the 5th International Conference on Automatic Face and Gesture Recognition (FG), pp. 202–207 (2002)

  8. Weyrauch, B., et al.: “Component-based face recognition with 3D morphable models”. In: Computer Vision and Pattern Recognition Workshop CVPRW ’04, pp. 85 June (2004)

  9. Okada, E.: Three-dimensional facial simulations and measurements: Changes of facial contour and units associated with facial expression. J.Craniofac. Surg. 12, 167–174 (2001)

    Article  Google Scholar 

  10. Koch, R.M., et al.: “Simulating facial surgery using finite element models”. In: Proceedings of the SIGGRAPH 96 (1996)

  11. Park, U., Jain, A.K.: “3D model-based face recognition in video”. In: Proceedings of the 2nd International Conference on Biometrics (2007)

  12. Duhn, S.V., et al.: “Three-view surveillance video based face modeling for recognition”. In: Proceedings of the Biometrics Symposium (2007)

  13. Shan, Y., et al.: “Model-based bundle adjustment with application to face modeling”. In: Proceedings of the 8th International Conference on Computer Vision (ICCV01) 2, 624–651 (2001)

  14. Fua, P.: Regularized bundle-adjustment to model heads from image sequences without calibration data. Int. J. Comput. Vis. 2, 153–171 (2000)

    Article  Google Scholar 

  15. DeCarlo, D., et al.: “An anthropometric face model using variational techniques”. In: Proceedings of the SIGGGRAPH’98 pp. 67–74 (1998)

  16. Beumier, C., Acheroy, M.: Face verication from 3D and grey level cues. Patt. Recogn. Lett. 22, 1321–1329 (2001)

    Article  MATH  Google Scholar 

  17. Savran, A., et al.: “Bosphorus database for 3D face analysis”. In: The First COST 2101 Workshop on Biometrics and Identity Management (BIOID 2008) (2008)

  18. Yin, L., et al.: “A 3D facial expression database for facial behavior research”. In Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition (FGR06), pp. 211–216 (2006)

  19. Yin, L., et al.: “A high-resolution 3D dynamic facial expression database”. In: Proceedings of the 8th International Conference on Automatic Face and Gesture Recognition (FGR08) (2008)

  20. Stan, Z.L., et al.: “The HFB face database for heterogeneous face biometrics research”. In: 6th IEEE Workshop on Object Tracking and Classification Beyond and in the Visible Spectrum (OTCBVS, in conjunction with CVPR 2009). Miami, June (2009)

  21. Gao, W., et al.: The CAS-PEAL large-scale Chinese face database and baseline evaluations. IEEE Trans. Syst. Man Cybern. (Part A) 38(1), 149–161 (2008)

    Article  Google Scholar 

  22. Phillips, P.J., et al.: Overview of the face recognition grand challenge. Comput. Vis. Patt. Recognit. 1, 947–954 (2005)

    Google Scholar 

  23. Moreno, A.B., Sanchez, A.: “GavabDB: A 3D face database”. In: Proceedings 2nd COST Workshop on Biometrics on the Internet: Fundamentals, Advances and Applications pp. 77–82 (2004)

  24. Troje, N., Blthoff, H.H.: Face recognition under varying poses: The role of texture and shape. Vis. Res. 36, 1761–1771 (1996)

    Article  Google Scholar 

  25. Faltemier, T.C., et al.: “Using a multi-instance enrollment representation to improve 3D face recognition” in Biometrics: Theory, Applications, and Systems. pp. 1–6 (2007)

  26. Zafeiriou, S., et al.: “The photoface database”. In: IEEE Computer Vision and Pattern Recognition Workshops. (IEEE Workshop on Biometrics 2011) June 20–25, Colorado Springs, Co., USA (2011)

  27. Gupta, S., et al.: Texas 3D face recognition database. IEEE Southwest Symposium on Image Analysis and, Interpretation, pp. 97–100 (2010)

  28. Messer, K., et al.: “XM2VTSDB: The extended M2VTS database”. In: Second International Conference on Audio and Video-based Biometric Person Authentication March (1999)

  29. Heseltine, T., et al.: Three-dimensional face recognition using combinations of surface feature map subspace components. Image Vis. Comput. 26, 382–396 (2008)

    Article  Google Scholar 

  30. Cristinacce, D., Cootes, T.: Facial feature detection using adaboost with shape constraints. Proceedings of British Machine Vision Conference 1, 231–240 (2003)

    Google Scholar 

  31. Cristinacce, D., Cootes, T.: Feature detection and tracking with constrained local models. Proceedings of British Machine Vision Conference , Norwich, 3, 929–938 (2006)

  32. Salah, A., et al.: Robust facial landmarking for registration. Ann.Telecommun. 62, 1608–1633 (2007)

    Google Scholar 

  33. Asteriadis, S., et al.: Facial feature detection using distance vector fields. Patt. Recognit. 42(7), 1388–1398 (2009)

    Article  MATH  Google Scholar 

  34. Kozakaya, T., et al.: Facial feature localization using weighted vector concentration approach. Image Vis. Comput. 28(5), 772–780 (2010)

    Article  Google Scholar 

  35. McKenna, S.J., et al.: “Tracking facial feature points with Gabor wavelets and shape models”. In: Proceedings of International Conference on Audio- and Video-based Biometric Person Authentication pp. 35–42 (1997)

  36. Rogers, M., Graham, J.: Robust active shape model search. In: Proceedings of ECCV 4, 517–530 (2002)

  37. Celiktutan, O., et al.: “Multi-attribute robust facial feature localization, Automatic Face and Gesture Recognition”. In: FG 08. 8th IEEE International Conference, pp. 1–6 (2008)

  38. Valenti, R., et al.: “Facial expression recognition: A fully integrated approach”. In: Workshop on Visual and Multimedia Digital Libraries (2007)

  39. Valstar, M.F., Pantic, M.: “Combined support vector machines and hidden Markov models for modeling facial action temporal dynamics”. In: Proceedings of the 2007 IEEE international conference on Human-computer interaction (HCI’07) 118–127 (2007)

  40. Longuet-Higgins, H.: “A computer algorithm for reconstructing a scene from two projections”. Nature, pp. 133–135 (1981)

  41. D’Apuzzo, N.: “Motion capture from multi image video sequences”. in Proceedings of the XVIIIth Congress of the International Society of Biomechanics (2001)

  42. D’Apuzzo, N., et al.: Markerless full body shape and motion capture from video sequences. Int. Arch. Photogramm. Rem. Sens. 31, 256–261 (2002)

    Google Scholar 

  43. Lee, Y., et al.: “Realistic modeling for facial animation”. In Proceedings of the 22nd annual conference on Computer graphics and interactive techniques (SIGGRAPH ’95) ACM, New York, pp. 55–62 (1995)

  44. Proesmans, M., Gool, L.V.: “Reading between the linesa method for extracting dynamic 3D with texture”. In: Proceedings of the ACM symposium on Virtual reality software and technology (VRST ’97) ACM, New York, 95–102 (1997)

  45. Horn, K.P., Brooks, M.J.: Shape from shading. MIT Press, Cambridge (1989)

  46. Adm, M.B., Said, A.M.: “3D human face reconstruction from single image using interactive shape from shading”. In: National Postgraduate Conference, pp. 1–7 (2011)

  47. Kemelmacher-Shlizerman, I., Basri, R.: 3D face reconstruction from a single image using a single reference face shape. IEEE Trans. Pattern Anal. Mach. Intell. 33(2), 394–405 (2011)

    Article  Google Scholar 

  48. Kittler, J., et al.: “3D assisted face recognition: A survey of 3D imaging, modelling and recognition approaches”. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition pp. 114 (2005)

  49. Torresani, L., Hertzmann, A.: “Nonrigid structure-from motion: Estimating shape and motion with hierarchical priors”. In: IEEE Transactions on Pattern Analysis and Machine Intelligence 30(5) (2008)

  50. Hartleyand, R.I., Zisserrnan, A.: Multiple view geometry. Cambridge University Press, Cambridge (2004)

    Book  Google Scholar 

  51. Ullman, S.: “The interpretation of structure from motion”. In: Proceedings of the Royal Society of London, pp. 405–426 (1979)

  52. Krinidis, S., Pitas, I.: “Facial expression synthesis through facial expressions statistical analysis”. European Signal Processing Conference (EU-SIPCO06) Florence (2006)

  53. Essa, I.A.: Coding, analysis, interpretation, and recognition of facial expressions. IEEE Trans. Patt. Anal. Mach. Intell. 19, 757–763 (1998)

    Article  Google Scholar 

  54. Moghaddam, B., Pentland, A.: “Face recognition using view-based and modular eigenspaces”. Automatic Systems for the Identification of Humans 2277 SPIE (1994)

  55. Cootes, T.F., et al.: Active shape models—their training and application. Comput. Vis. Image Underst. 61(1), 38–59 (1995)

    Article  Google Scholar 

  56. Cootes, T.F., et al.: Active appearance models. Proc. Euro. Conf. Comput. Vis. 2, 484–498 (1998)

    Google Scholar 

  57. Ahlberg, J.: An active model for facial feature tracking. EURASIP J. Appl. Sign. Process. 2001, 566–571 (2001)

    Google Scholar 

  58. Dornaika, F., Ahlberg, J.: Fitting 3D face models for tracking and active appearance model training. Image Vis. Comput. 24, 1010–1024 (2006)

    Article  Google Scholar 

  59. Dornaika, F., Ahlberg, J.: “Fast and reliable active appearance model search for 3D face tracking”. In: International Conference on Model-based Imaging, Rendering, Image Analysis and Graphical Special Effects (MIRAGE) pp. 113–122, Paris, March (2003)

  60. Schonemann, P.H.: A generalized solution of the orthogonal Procrustes problem. Psychometrika 31, 1–10 (1966)

    Article  MathSciNet  Google Scholar 

  61. Besl, P.J., McKay, H.D.: “A method for registration of 3-D shapes”. In: IEEE Transactions on Pattern Analysis and Machine Intelligence (1992)

  62. Gower, J.: Generalized procrustes analysis. Psychometrika 40, 1–10 (1966)

    Google Scholar 

  63. Gower, J., Dijksterhuis, G.B.: Procrustes problems. Oxford University Press, Oxford (2004)

  64. Chen, Y., Medioni, G.: “Object modeling by registration of multiple range images”. In: Proceedings of the 1992 IEEE International Conference on Robotics and Automation 2724–2729 (1992)

  65. Du, S.Y., et al.: “An extension of the ICP algorithm considering scale factor”. In: Proceedings 14th IEEE International Conference on Image Processing (ICIP) 193–196 (2007)

  66. Rydfalk, M.: “Candide, a parameterized face”. Technical Report Report No. LiTH-ISY-I-866, Department of Electrical Engineering, Linkping University, Sweden (1987)

  67. Ahlberg, J.: “CANDIDE-3–an updated parameterized face”. Report No. LiTH-ISY-R-2326, Department of Electrical Engineering, Linkping University, Sweden (2001)

  68. Breton, G., et al.: “FaceEngine a 3d facial animation engine for real time applications”. In: Proceedings of 6th International Conference on 3D Web Technology, pp.15–22 (2001)

  69. Eskil, T.: “HIgh polygon GEneric face Model”. Internet: http://pi.isikun.edu.tr, 21 Dec (2012)

  70. Luximon, Y., et al.: The 3D Chinese head and face modeling. Comput. Aid. Des. 44(1), 40–47 (2012)

    Article  Google Scholar 

  71. Tsai, R.Y.: A versatile camera calibration technique for high accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE J. Robot. Autom. 4, 323–344 (1987)

    Article  Google Scholar 

Download references

Acknowledgments

This research is part of project “Expression Recognition based on Facial Anatomy”, grant number 109E061, supported by The Support Programme for Scientific and Technological Research Projects (1001) of The Scientific and Technological Research Council of Turkey (TÜBİTAK).

Author information

Authors and Affiliations

  1. Pattern Recognition and Machine Intelligence Laboratory, IŞIK University, Şile, Istanbul, Turkey

    H. Pasindu Abeysundera & Kristin S. Benli

  2. Computer Science and Engineering Department, IŞIK University, Şile, Istanbul, Turkey

    M. Taner Eskil

Authors
  1. H. Pasindu Abeysundera
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kristin S. Benli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Taner Eskil
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Taner Eskil.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Abeysundera, H.P., Benli, K.S. & Eskil, M.T. Nearest neighbor weighted average customization for modeling faces. Machine Vision and Applications 24, 1525–1537 (2013). https://doi.org/10.1007/s00138-013-0489-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-013-0489-x

Keywords

Search

Navigation