Skip to main content
SpringerLink
Log in
Book cover

Handbook of Face Recognition pp 323–351Cite as

Face Tracking and Recognition in Video

  • Chapter

Abstract

In this chapter, we describe the utility of videos in enhancing performance of image-based recognition tasks. We discuss a joint tracking-recognition framework that allows for using the motion information in a video to better localize and identify the person in the video using still galleries. We discuss how to jointly capture facial appearance and dynamics to obtain a parametric representation for video-to-video recognition. We discuss recognition in multi-camera networks where the probe and gallery both consist of multi-camera videos. Concluding remarks and directions for future research are provided.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Aggarwal, G., Roy-Chowdhury, A., Chellappa, R.: A system identification approach for video-based face recognition. In: International Conference on Pattern Recognition, Cambridge, UK, August 2004

    Google Scholar 

  2. Anderson, B., Moore, J.: Optimal Filtering. Prentice Hall, Englewood Cliffs (1979)

    MATH  Google Scholar 

  3. Arandjelovic, O., Cipolla, R.: Face recognition from video using the generic shape-illumination manifold. In: European Conference on Computer Vision, pp. 27–40 (2006)

    Google Scholar 

  4. Arandjelovic, O., Shakhnarovich, G., Fisher, J., Cipolla, R., Darrell, T.: Face recognition with image sets using manifold density divergence. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 581–588, San Diego, USA, June 2005

    Google Scholar 

  5. Basri, R., Jacobs, D.: Lambertian reflectance and linear subspaces. In: Proceedings of IEEE International Conference on Computer Vision, vol. 2, pp. 383–390 (2001)

    Google Scholar 

  6. Bissacco, A., Chiuso, A., Ma, Y., Soatto, S.: Recognition of human gaits. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 52–57, Hawaii, USA, December 2001

    Google Scholar 

  7. Brocker, T., Dieck, T.: Representations of Compact Lie Groups. Springer, Berlin (2003)

    Google Scholar 

  8. Chikuse, Y.: Statistics on Special Manifolds. Lecture Notes in Statistics. Springer, New York (2003)

    MATH  Google Scholar 

  9. Choudhury, T., Clarkson, B., Jebara, T., Pentland, A.: Multimodal person recognition using unconstrained audio and video. In: Proc. of Intl. Conf. on Audio- and Video-Based Person Authentication, pp. 176–181 (1999)

    Google Scholar 

  10. Cock, K.D., Moor, B.D.: Subspace angles between ARMA models. Syst. Control Lett. 46, 265–270 (2002)

    Article  MATH  Google Scholar 

  11. Doretto, G., Chiuso, A., Wu, Y.N., Soatto, S.: Dynamic textures. Int. J. Comput. Vis. 51(2), 91–109 (2003)

    Article  MATH  Google Scholar 

  12. Doucet, A., Godsill, S., Andrieu, C.: On sequential Monte Carlo sampling methods for Bayesian filtering. Stat. Comput. 10(3), 197–209 (2000)

    Article  Google Scholar 

  13. Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20(2), 303–353 (1999)

    Article  MathSciNet  Google Scholar 

  14. Fan, W., Yeung, D.-Y.: Locally linear models on face appearance manifolds with application to dual-subspace based classification. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1384–1390, New York, NY, USA, June 2006

    Google Scholar 

  15. Hamm, J., Lee, D.D.: Grassmann discriminant analysis: a unifying view on subspace-based learning. In: International Conference on Machine Learning, pp. 376–383, Helsinki, Finland, June 2008

    Google Scholar 

  16. Kazhdan, M., Funkhouser, T., Rusinkiewicz, S.: Rotation invariant spherical harmonic representation of 3d shape descriptors. In: Proceedings of Eurographics/ACM SIGGRAPH Symposium on Geometry Processing, pp. 156–164 (2003)

    Google Scholar 

  17. Kim, T.K., Kittler, J., Cipolla, R.: Discriminative learning and recognition of image set classes using canonical correlations. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 1005–1018 (2007)

    Article  Google Scholar 

  18. Kitagawa, G.: Monte Carlo filter and smoother for non-Gaussian nonlinear state space models. J. Comput. Graph. Stat. 5, 1–25 (1996)

    Article  MathSciNet  Google Scholar 

  19. Lee, K.-C., Ho, J., Kriegman, D.J.: Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans. Pattern Anal. Mach. Intell. 27(5), 684–698 (2005)

    Article  Google Scholar 

  20. Lee, K.-C., Ho, J., Yang, M.-H., Kriegman, D.J.: Visual tracking and recognition using probabilistic appearance manifolds. Comput. Vis. Image Underst. 99(3), 303–331 (2005)

    Article  Google Scholar 

  21. Li, Y., Gong, S., Liddell, H.: Video-based online face recognition using identity surfaces. In: Proceedings of the IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, pp. 40–46 (2001)

    Chapter  Google Scholar 

  22. Liu, J., Chen, R.: Sequential Monte Carlo for dynamic systems. J. Am. Stat. Assoc. 93, 1031–1041 (1998)

    Google Scholar 

  23. Liu, X., Chen, T.: Video-based face recognition using adaptive hidden Markov model. In: Proceedings of Computer Vision and Pattern Recognition, vol. 1, pp. 340–345 (2003)

    Google Scholar 

  24. Liu, X., Chen, T., Thornton, S.M.: Eigenspace updating for non-stationary process and its application to face recognition. Pattern Recognit. 36, 1945–1959 (2003)

    Article  MATH  Google Scholar 

  25. Moghaddam, B., Pentland, A.: Probabilistic visual learning for object representation. IEEE Trans. Pattern Anal. Mach. Intell. 19, 696–710 (1997)

    Article  Google Scholar 

  26. O’Toole, A.J., Roark, D., Abdi, H.: Recognizing moving faces: a psychological and neural synthesis. Trends Cogn. Sci. 6, 261–266 (2002)

    Article  Google Scholar 

  27. Overschee, P.V., Moor, B.D.: Subspace algorithms for the stochastic identification problem. Automatica 29(3), 649–660 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  28. Park, U., Jain, A.K., Ross, A.: Face recognition in video: adaptive fusion of multiple matchers. In: Proceedings of IEEE Computer Society Workshop on Biometrics (In Conjunction with CVPR), pp. 1–8 (2007)

    Google Scholar 

  29. Philipps, P., Moon, H., Rivzi, S., Ross, P.: The Feret evaluation methodology for face-recognition algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 22, 1090–1104 (2000)

    Article  Google Scholar 

  30. Ramamoorthi, R.: Analytic pca construction for theoretical analysis of lighting variability in images of a Lambertian object. IEEE Trans. Pattern Anal. Mach. Intell. 24(10), 1322–1333 (2002)

    Article  Google Scholar 

  31. Roark, D.A., Barrett, S.E., O’Toole, A.J., Abdi, H.: Learning the moves: The effect of familiarity and facial motion on person recognition across large changes in viewing format. Perception 761–773 (2006)

    Google Scholar 

  32. Ross, A.A., Nandakumar, K., Jain, A.K.: Handbook of Multibiometrics. International Series on Biometrics. Springer, New York (2006)

    Google Scholar 

  33. Saisan, P., Doretto, G., Wu, Y.N., Soatto, S.: Dynamic texture recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 58–63, Hawaii, USA, December 2001

    Google Scholar 

  34. Shakhnarovich, G., Fisher, J.W., Darrell, T.: Face recognition from long-term observations. In: Proceedings of the European Conference on Computer Vision, vol. 3, pp. 851–865, May 2002

    Google Scholar 

  35. Turaga, P., Veeraraghavan, A., Chellappa, R.: Statistical analysis on Stiefel and Grassmann manifolds with applications in computer vision. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8, Alaska, USA, June 2008

    Chapter  Google Scholar 

  36. Turaga, P., Veeraraghavan, A., Srivastava, A., Chellappa, R.: Statistical analysis on manifolds and its applications to video analysis. In: Schonfeld, D., Shan, C., Tao, D., Wang, L. (eds.) Video Search and Mining. Studies in Computational Intelligence, Chap. 5. Springer, Berlin (2010)

    Google Scholar 

  37. Veeraraghavan, A., Roy-Chowdhury, A., Chellappa, R.: Matching shape sequences in video with an application to human movement analysis. IEEE Trans. Pattern Anal. Mach. Intell. 27(12), 1896–1909 (2005)

    Article  Google Scholar 

  38. Xu, Y., Roy-Chowdhury, A., Patel, K.: Pose and illumination invariant face recognition in video. In: Proceedings of IEEE Computer Society Workshop on Biometrics (In Conjunction with CVPR), pp. 1–7 (2007)

    Google Scholar 

  39. Zhang, L., Samaras, D.: Face recognition from a single training image under arbitrary unknown lighting using spherical harmonics. IEEE Trans. Pattern Anal. Mach. Intell. 28(3), 351–363 (2006)

    Article  Google Scholar 

  40. Zhao, W.Y., Chellappa, R., Rosenfeld, A., Phillips, P.: Face recognition: a literature survey. ACM Comput. Surv. 35 (2003)

    Google Scholar 

  41. Zhou, S.K., Chellappa, R.: From sample similarity to ensemble similarity: probabilistic distance measures in reproducing kernel Hilbert space. IEEE Trans. Pattern Anal. Mach. Intell. 28(6), 917–929 (2006)

    Article  Google Scholar 

  42. Zhou, S., Krueger, V., Chellappa, R.: Probabilistic recognition of human faces from video. Comput. Vis. Image Underst. 91, 214–245 (2003)

    Article  Google Scholar 

  43. Zhou, S., Chellappa, R., Moghaddam, B.: Visual tracking and recognition using appearance-adaptive models in particle filters. IEEE Trans. Image Process. (2004)

    Google Scholar 

Download references

Acknowledgements

Supported by a MURI Grant N00014-08-1-0638 from the Office of Naval Research. The authors would like to thank Dr. Aswin Sankaranarayanan for helpful discussions related to Sect. 13.5.

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, Center for Automation Research, University of Maryland, College Park, MD, 20742, USA

    Rama Chellappa, Ming Du & Pavan Turaga

  2. Siemens Corporate Research, 755 College Road East, Princeton, NJ, 08540, USA

    Shaohua Kevin Zhou

Authors
  1. Rama Chellappa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ming Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pavan Turaga
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shaohua Kevin Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rama Chellappa .

Editor information

Editors and Affiliations

  1. Institute of Automation, Center Biometrics Research & Security, Chinese Academy of Science, Room 1227, Zhongguancun 95, Beijing Donglu, 100080, China, People's Republic

    Stan Z. Li

  2. Dept. Computer Science & Engineering, Michigan State University, East Lansing, 48824-1226, Michigan, USA

    Anil K. Jain

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag London Limited

About this chapter

Cite this chapter

Chellappa, R., Du, M., Turaga, P., Zhou, S.K. (2011). Face Tracking and Recognition in Video. In: Li, S., Jain, A. (eds) Handbook of Face Recognition. Springer, London. https://doi.org/10.1007/978-0-85729-932-1_13

Download citation

  • DOI: https://doi.org/10.1007/978-0-85729-932-1_13

  • Publisher Name: Springer, London

  • Print ISBN: 978-0-85729-931-4

  • Online ISBN: 978-0-85729-932-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation