Skip to main content
SpringerLink
Log in

3D face and motion estimation from sparse points using adaptive bracketed minimization

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper presents a novel method for estimating camera motion and reconstructing human face from a video sequence. The coarse-to-fine method is applied via combining the concepts of Powell’s minimization with gradient descent. Sparse points defining the human face in every frame are tracked using the active appearance model. The case of occluded points, even for self-occlusion, does not pose a problem in the proposed method. Robustness in the presence of noise and 3D accuracy using this method is also demonstrated. Examples of face reconstruction using other methods including trifocal tensor, Powell’s minimization, and gradient descent are also compared to the proposed method. Experiments on both synthetic and real faces are presented and analyzed. Also, different camera movement paths are illustrated. All real-world experiments used an off-the-shelf digital camera carried by a human walking without using any dolly to demonstrate the robustness and practicality of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Avidan S, Shashua A (1998) Novel view synthesis by cascading trilinear tensors. IEEE Trans Vis Comput Graph 4(4):293–306, Oct-Dec 1998

    Article  Google Scholar 

  2. Bozek J, Grgic M, Delac K (2010) Comparative analysis of interpolation methods for bilateral asymmetry. IEEE Int Symp ELMAR, pp. 1–7, 15–17 Sep 2010.

  3. Brent RP (2002) Algorithms for minimization without derivatives, 1st ed. Dover, 2002.

  4. Chen OT-C (2000) Motion estimation using a one-dimensional gradient descent search. IEEE Trans Circuits Syst Video Technol 10(4):608–616, Jun 2000

    Article  Google Scholar 

  5. Chouvatut V, Madarasmi S, Tuceryan M (2009) Face reconstruction and camera pose using multi-dimensional descent. Proc Int Conf Comput Electr Syst Sci Eng (CESSE), pp. 730–735, 25–27 Dec 2009.

  6. Chouvatut V, Madarasmi S, Tuceryan M (2010) 3D reconstruction and camera pose from video sequence using multi-dimensional descent. 4th Int Conf Inf Syst Tech Manag (ICISTM), pp. 282–292, 10–12 Mar 2010.

  7. Edwards GJ, Taylor CJ, Cootes TF (1998) Interpreting face images using active appearance models. 3rd IEEE Int Conf Autom Face Gesture Recognit, pp. 300–305, 14–16 Apr 1998.

  8. Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun Assoc Comput Mach (ACM) 24(6):381–395, Jun 1981

    MathSciNet  Google Scholar 

  9. Galpin F, Morin L (2002) Sliding adjustment for 3D video representation. EURASIP J Appl Signal Process 2002(10):1088–1101, 2002

    Article  MATH  Google Scholar 

  10. Hartley RI (1994) Projective reconstruction and invariants from multiple images. IEEE Trans Pattern Anal Mach Intell 16(10):1036–1041, Oct 1994

    Article  Google Scholar 

  11. Hartley R, Zisserman A (2006) Multiple view geometry in computer vision, 2nd ed. Cambridge, 2006.

  12. Karayiannis NB (2000) Reformulated radial basis neural networks trained by gradient descent. IEEE Trans Neural Netw 10(3):657–671, May 2000

    Article  Google Scholar 

  13. Kato H, Billinghurst M (1999) Marker tracking and HMD calibration for a video-based augmented reality conferencing system. Proc 2nd IEEE and ACM Int Workshop Augment Real, pp. 85–94, Oct 1999.

  14. Li J, Chellappa R (2005) A factorization method for structure from planar motion. IEEE Workshop Motion Video Comput (WACV/MOTIONS) 2:154–159, Jan 2005

    Google Scholar 

  15. Okuma T, Sakaue K, Takemura H, Yokoya N (2000) Real-time camera parameter estimation from images for a mixed reality system. IEEE Proc 15th Int Conf Pattern Recognit 4:482–486, 3–7 Sep 2000

    Google Scholar 

  16. Park SW, Heo J, Savvides M (2008) 3D face reconstruction from a single 2D face image. IEEE Comput Soc Conf Comput Vis Pattern Recognit Workshops (CVPR), pp. 1–8, 23–28 Jun 2008.

  17. Po LM, Ng KH, Cheung KW, Wong KM, Uddin Y, Ting CW (2009) Novel directional gradient descent searches for fast block motion estimation. IEEE Trans Circuits Syst Video Technol 19(8):1189–1195, Aug 2009

    Article  Google Scholar 

  18. Press WH, Teukolsky SA, Vetterling WT, Flannery BP (2007) Numerical recipes – the art of scientific computing, 3rd ed. Cambridge, 2007.

  19. Repko J, Pollefeys M (2005) 3D models from extended uncalibrated video sequences: addressing key-frame selection and projective drift. Int Conf 3-D Digit Imaging Model, 2005.

  20. Smolic A (2002) Robust generation of 360-degree panoramic views from consumer video sequences. 4th EURASIP-IEEE Reg 8 Int Symp Video/Image Process Multimed Commun (VIPromCom), pp. 431–435, 16–19 Jun 2002.

  21. Tsai RY (1987) A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV camera and lenses. IEEE J Robot Autom RA-3(4):323–344, Aug 1987

    Article  Google Scholar 

  22. Xu X, Dony RD (2004) Differential evolution with powell’s direction set method in medical image registration. IEEE Int Symp Biomed Imaging: Nano to Micro 1:732–735, 15–18 Apr 2004

    Google Scholar 

  23. Zheng Y, Wang Z (2008) Robust depth estimation for efficient 3D face reconstruction. 15th IEEE Int Conf Image Process, pp. 1516–1519, 12–15 Oct 2008.

  24. Zheng Y, Chang J, Zheng Z, Wang Z (2007) 3D face reconstruction from stereo: a model based approach. IEEE Int Conf Image Process (ICIP) 3:III-65–III-68, 16 Sep 2007 – 19 Oct 2007

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Engineering Department, King Mongkut’s University of Technology Thonburi, Bangkok, Thailand

    Varin Chouvatut & Suthep Madarasmi

  2. Computer and Information Science Department, Indiana University-Purdue University Indianapolis, Indianapolis, IN, 46202-5132, USA

    Mihran Tuceryan

Authors
  1. Varin Chouvatut
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Suthep Madarasmi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mihran Tuceryan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Varin Chouvatut.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chouvatut, V., Madarasmi, S. & Tuceryan, M. 3D face and motion estimation from sparse points using adaptive bracketed minimization. Multimed Tools Appl 63, 569–589 (2013). https://doi.org/10.1007/s11042-011-0925-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-011-0925-8

Keywords

Search

Navigation