Claudia Arellano
Rozenn Dahyot
ABSTRACT
We propose to fit automatically a 3D morphable face model to a point cloud captured with a RGB-D sensor. Both data sets, the shape model and the target point cloud are modelled as two probability density functions (pdfs). Rigid registration (rotation and translation) and reconstruction on the model is performed by minimising the Euclidean distance between these two pdfs augmented with a multivariate Gaussian prior. Our resulting process is robust and it does not require point to point correspondence. Experimental results on synthetic and real data illustrates the performance of this novel approach.
- B. Amberg, A. Blake, A. Fitzgibbon, S. Romdhani, and T. Vetter. Reconstructing high quality face-surfaces using model based stereo. IEEE International Conference on Computer Vision, ICCV, pages 1--8, 2007.Google Scholar
Cross Ref
- B. Amberg, S. Romdhani, and T. Vetter. Optimal step nonrigid icp algorithms for surface registration. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR '07), pages 1--8, 2007.Google ScholarCross Ref
- C. Arellano and R. Dahyot. Mean shift algorithm for robust rigid registration between gaussian mixture models. In 20th European Signal Processing Conference (Eusipco), pages 1154--1158, Bucharest, Romania, August 2012.Google Scholar
- C. Arellano and R. Dahyot. Shape model fitting using non-isotropic gmm. In 23rd IET Irish Signals and Systems Conference, 2012.Google ScholarCross Ref
- C. Arellano and R. Dahyot. Shape model fitting without point correspondence. In 20th European Signal Processing Conference (Eusipco), pages 934--938, Bucharest, Romania, August 2012.Google Scholar
- S. Baker and I. Matthews. Equivalence and efficiency of image alignment algorithms. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR, pages 1090--1097, 2001.Google ScholarCross Ref
- C. Basso, T. Vetter, and V. Blanz. Regularized 3d morphable models. In IEEE International Workshop on Higher-Level Knowledge in 3D Modeling and Motion Analysis, pages 3--12, 2003. Google ScholarDigital Library
- A. Basu, I. Harris, N. Hjort, and M. Jones. Robust and efficient estimation by minimising a density power divergence. Biometrika, 85(3): 549--559, 1998.Google ScholarCross Ref
- P. Besl and N. McKay. A method for registration of 3-d shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14: 239--256, 1992. Google ScholarDigital Library
- V. Blanz and T. Vetter. A morphable model for the synthesis of 3d faces. Proceedings of the annual conference on Computer Graphics and Interactive Techniques, SIGGRAPH, pages 187--194, 1999. Google ScholarDigital Library
- D. Bradley, W. Heidrich, T. Popa, and A. Sheffer. High resolution passive facial performance capture. ACM Transactions Graphics, 29(4): 41:1--41:10, July 2010. Google ScholarDigital Library
- H. Bulthoff. Shape from X: psychophysics and computation. In: Landy M, Movshon A, editors. Computational models of visual processing. Cambridge: MIT Press, 1991.Google Scholar
- S.-H. Cha. Comprehensive survey on distance/similarity measures between probability density functions. International Journal of Mathematical Models and Methods in Applied Sciences, 1: 300--307, 2007.Google Scholar
- H. Chui and A. Rangarajan. A feature registration framework using mixture models. In IEEE Workshop on Mathematical Methods in Biomedical Image Analysis, pages 190--197, 2000. Google ScholarDigital Library
- Y. Cui, S. Schuon, C. Derek, S. Thrun, and C. Theobalt. 3d shape scanning with a time-of-flight camera. In IEEE conference on Computer Vision and Patern Recognition, CVPR, 2010.Google ScholarCross Ref
- B. Freedman, A. Shpunt, M. Machline, and Y. Arieli. Depth mapping using projected patterns. Patent Publication number: US 2010/0118123 A1, 2008.Google Scholar
- P. Fua. Regularized bundle-adjustment to model heads from image sequences without calibration data. International Journal of Computer Vision, volume 38: 153--171, 2000. Google ScholarDigital Library
- S. Granger and X. Pennec. Multi-scale em-icp: A fast and robust approach for surface registration. European Conference on Computer Vision ECCV, pages 418--432, 2002. Google ScholarDigital Library
- E. Hasanbelliu, L. S. Giraldo, and J. Principe. A robust point matching algorithm for non-rigid registration using the cauchy-schwarz divergence. IEEE International Workshop on Machine Learning for Signal Processing, pages 1--6, 2011.Google ScholarCross Ref
- M. Helén and T. Virtanen. Audio query by example using similarity measures between probability density functions of features. EURASIP Journal on Audio, Speech, and Music Processing, 2010. Google ScholarDigital Library
- P. Henry, M. Krainin, E. Herbst, X. Ren, and D. Fox. Rgbd mapping: Using depth cameras for dense 3d modeling of indoor environments. In In RGB-D: Advanced Reasoning with Depth Cameras Workshop in conjunction with RSS, 2010.Google Scholar
- B. Huhle, T. Schairer, P. Jenke, and W. Straßer. Fusion of range and color images for denoising and resolution enhancement with a non-local filter. Computer Vision and Image Understanding, 114(12): 1336--1345, 2010. Special issue on Time-of-Flight Camera Based Computer Vision. Google ScholarDigital Library
- T. Jebara, R. Kondor, and A. Howard. Probability product kernels. Journal of Machine Learning Research, volume 5: 819--844, 2004. Google ScholarDigital Library
- B. Jian and B. Vemuri. Robust point set registration using gaussian mixture models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011. Google ScholarDigital Library
- J. Lee, B. Moghaddam, H. Pfister, and R. Machiraju. Silhouette-based 3d face shape recovery. In Graphics Interface, pages 21--30, 2003.Google Scholar
- R. Lengagne, P. Fua, and O. Monga. 3d stereo reconstruction of human faces driven by differential constraints. Image and Vision Computing, volume 18(4): 337--343, 2000.Google ScholarCross Ref
- M. D. Levine and Y. (Chris) Yu. State-of-the-art of 3d facial reconstruction methods for face recognition based on a single 2d training image per person. Pattern Recogn. Lett., volume 30(10): 908--913, 2009. Google ScholarDigital Library
- R. Lienhart and J. Maydt. An extended set of haar-like features for rapid object detection. In International Conference on Image Processing (ICIP), volume 1, pages I-900--I-903, 2002.Google Scholar
- H. M, J. Choi, and G. Medioni. Laser scan quality 3-d face modelling using a low cost depth map. In 20th European Signal Processing Conference (Eusipco), pages 1995--1999, Bucharest, Romania, August 2012.Google Scholar
- P. Merrell, A. Akbarzadeh, L. Wang, J. michael Frahm, and R. Y. D. NistÃl'r. Real-time visibility-based fusion of depth maps. In International Conference on Computer Vision and Pattern Recognition (CVPR), 2007.Google ScholarCross Ref
- A. Myronenko and X. S. Point set registration: Coherent point drift. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32: 2262--2275, 2010. Google ScholarDigital Library
- R. Newcombe and A. Davison. Live dense reconstruction with a single moving camera. In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, pages 1498--1505, june 2010.Google ScholarCross Ref
- R. A. Newcombe, S. Izadi, O. Hilliges, D. Molyneaux, D. Kim, A. J. Davison, P. Kohli, J. Shotton, S. Hodges, and A. Fitzgibbon. Kinectfusion: Real-time dense surface mapping and tracking. In Proceedings of the 2011 10th IEEE International Symposium on Mixed and Augmented Reality, ISMAR '11, pages 127--136, Washington, DC, USA, 2011. IEEE Computer Society. Google ScholarDigital Library
- K. Ouji, M. Ardabilian, L. Chen, and F. Ghorbel. A space-time depth super-resolution scheme for 3d face scanning. In Proceedings of the 13th international conference on Advanced concepts for intelligent vision systems, ACIVS'11, pages 658--668, Berlin, Heidelberg, 2011. Springer-Verlag. Google ScholarDigital Library
- A. Patel and W. A. Smith. 3d morphable face models revisited. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pages 1327--1334, 2009.Google ScholarCross Ref
- P. Paysan, R. Knothe, B. Amberg, S. Romdhani, and T. Vetter. A 3d face model for pose and illumination invariant face recognition. IEEE Conference on Advanced Video and Signal Based Surveillance, pages 296--301, 2009. Google ScholarDigital Library
- C. P. Robert and G. Casella. Monte Carlo Statistical Methods. Springer Verlag, 1999. Google ScholarDigital Library
- S. Romdhani, V. Blanz, and T. Vetter. Face identification by fitting a 3d morphable model using linear shape and texture error functions. European Conference on Computer Vision, ECCV, pages 3--19, 2002. Google ScholarDigital Library
- S. Romdhani and T. Vetter. Efficient, robust and accurate fitting of a 3d morphable model. In Proceedings of the IEEE International Conference on Computer Vision, ICCV, pages 59--66, 2003. Google ScholarDigital Library
- S. Romdhani and T. Vetter. Estimating 3d shape and texture using pixel intensity, edges, specular highlights, texture constraints and a prior. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pages 986--993, 2005. Google ScholarDigital Library
- F. Romeiro and T. Zickler. Model-based stereo with occlusions. In Proceedings of the 3rd international conference on Analysis and modeling of faces and gestures, AMFG, pages 31--45, 2007. Google ScholarDigital Library
- S. Rusinkiewicz and M. Levoy. Efficient variants of the icp algorithm. International Conference on 3-D Digital Imaging and Modeling, 2001.Google ScholarCross Ref
- J. Ruttle. Statistical Framework for Multi-Sensor Fusion and 3D Reconstruction. PhD thesis, School of Computer Science and Statistics, Trinity College Dublin, Dublin, Ireland, 2012.Google Scholar
- J. Ruttle, C. Arellano, and R. Dahyot. Extrinsic camera parameters estimation for shape-from-depths. In 20th European Signal Processing Conference (Eusipco), pages 1985--1989, Bucharest, Romania, August, 27--31 2012.Google Scholar
- D. Schneider and P. Eisert. Algorithms for automatic and robust registration of 3d head scans. Journal of Virtual Reality and Broadcasting, 7, 2010.Google Scholar
- D. Scott. Parametric statistical modeling by minimum integrated square error. Technometrics, 43: 274--285, 2001.Google ScholarCross Ref
- C. Shen, M. Brooks, and A. van den Hengel. Fast global kernel density mode seeking: Applications to localization and tracking. IEEE Transactions on Image Processing, 16, 2007. Google ScholarDigital Library
- W. A. Smith and E. R. Hancock. Recovering face shape and reflectance properties from single images. In IEEE International Conference on Automatic Face Gesture Recognition, pages 1--8, 2008.Google ScholarCross Ref
- Y. Tsin and T. Kanade. A correlation-based approach to robust point set registration. In European Conference in Computer Vision ECCV, pages 558--569, 2004.Google ScholarCross Ref
- J. J. Verbeek, N. Vlassis, and B. J. A. Kröse. Self-organizing mixture models. Neurocomput., 63: 99--123, Jan. 2005. Google ScholarDigital Library
- P. Viola and M. J. Jones. Rapid object detection using a boosted cascade of simple features. In International Conference on Computer Vision and Pattern Recognition (CVPR), 2001.Google ScholarCross Ref
- X. Wang, W. Liang, and L. Zhang. Morphable face reconstruction with multiple views. International Conference on Intelligent Human-Machine Systems and Cybernetics, IHMSC, pages 250--253, 2010. Google ScholarDigital Library
- T. Weise, S. Bouaziz, H. Li, and M. Pauly. Realtime performance-based facial animation. ACM Transactions Graphics, 30: 1--10, 2011. Google ScholarDigital Library
- Z. Zhang. Iterative point matching for registration of free-form curves and surfaces. Int. Journal Comput. Vision, 13: 119--152, 1994. Google ScholarDigital Library
- M. Zhao, T.-S. Chua, and T. Sim. Morphable face reconstruction with multiple images. In IEEE International Conference on Automatic Face and Gesture Recognition, FGR, pages 597--602, 2006. Google ScholarDigital Library
- M. Zollhöfer, M. Martinek, G. Greiner, M. Stamminger, and J. Süßmuth. Automatic reconstruction of personalized avatars from 3d face scans. In Proceedings of Computer Animation and Virtual Worlds (CASA), volume 22, pages 195--202, 2011. Google ScholarDigital Library
Index Terms
- Robust Bayesian fitting of 3D morphable model
Recommendations
Gaussian mixture 3D morphable face model
A Gaussian Mixture 3DMM (GM-3DMM) which models the global population as a mixture of Gaussian subpopulations.A ESO-based model selection strategy for GM-3DMM fitting.A GM-3DMM-based face recognition framework by fusing multiple experts, which has ...
2D face fitting-assisted 3D face reconstruction for pose-robust face recognition
Special issue on Digital Information ForensicsRecent face recognition algorithm can achieve high accuracy when the tested face samples are frontal. However, when the face pose changes largely, the performance of existing methods drop drastically. Efforts on pose-robust face recognition are highly ...
Efficient 3D reconstruction for face recognition
Face recognition with variant pose, illumination and expression (PIE) is a challenging problem. In this paper, we propose an analysis-by-synthesis framework for face recognition with variant PIE. First, an efficient two-dimensional (2D)-to-three-...
Comments