Skip to main content
SpringerLink
Log in
Book cover

Asian Conference on Computer Vision

ACCV 2016: Computer Vision – ACCV 2016 pp 344–360Cite as

Prototype Discriminative Learning for Face Image Set Classification

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 10113))

Abstract

This paper presents a novel Prototype Discriminative Learning (PDL) method to solve the problem of face image set classification. We aim to simultaneously learn a set of prototypes for each image set and a linear discriminative transformation to make projections on the target subspace satisfy that each image set can be optimally classified to the same class with its nearest neighbor prototype. For an image set, its prototypes are actually “virtual” as they do not certainly appear in the set but are only assumed to belong to the corresponding affine hull, i.e., affine combinations of samples in the set. Thus, the proposed method not only inherits the merit of classical affine hull in revealing unseen appearance variations implicitly in an image set, but more importantly overcomes its flaw caused by too loose affine approximation via efficiently shrinking each affine hull with a set of discriminative prototypes. The proposed method is evaluated by face identification and verification tasks on three challenging and large-scale databases, YouTube Celebrities, COX and Point-and-Shoot Challenge, to demonstrate its superiority over the state-of-the-art.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    The source code of PDL is available at http://vipl.ict.ac.cn/resources/codes.

References

  1. Cevikalp, H., Triggs, B.: Face recognition based on image sets. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2010)

    Google Scholar 

  2. Hu, Y., Mian, A.S., Owens, R.: Sparse approximated nearest points for image set classification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2011)

    Google Scholar 

  3. Yang, M., Zhu, P., Gool, L.V., Zhang, L.: Face recognition based on regularized nearest points between image sets. In: IEEE Conference on Automatic Face and Gesture Recognition (FG) (2013)

    Google Scholar 

  4. Wang, W., Wang, R., Shan, S., Chen, X.: Probabilistic nearest neighbor search for robust classification of face image sets. In: IEEE Conference on Automatic Face and Gesture Recognition (FG) (2015)

    Google Scholar 

  5. Zhu, P., Zhang, L., Zuo, W., Zhang, D.: From point to set: extend the learning of distance metrics. In: IEEE International Conference on Computer Vision (ICCV) (2013)

    Google Scholar 

  6. Leng, M., Moutafis, P., Kakadiaris, I.A.: Joint prototype and metric learning for set-to-set matching: application to biometrics. In: IEEE Conference on Biometrics Theory, Applications and Systems (BTAS) (2015)

    Google Scholar 

  7. Yamaguchi, O., Fukui, K., Maeda, K.: Face recognition using temporal image sequence. In: IEEE Conference on Automatic Face and Gesture Recognition (FG) (1998)

    Google Scholar 

  8. Kim, T.K., Kittler, J., Cipolla, R.: Discriminative learning and recognition of image set classes using canonical correlations. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 29, 1005–1018 (2007)

    Article  Google Scholar 

  9. Hamm, J., Lee, D.D.: Grassmann discriminant analysis: a unifying view on subspace-based learning. In: International Conference on Machine Learning (ICML) (2008)

    Google Scholar 

  10. Harandi, M.T., Sanderson, C., Shirazi, S., Lovell, B.C.: Graph embedding discriminant analysis on grassmannian manifolds for improved image set matching. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2011)

    Google Scholar 

  11. Wang, R., Shan, S., Chen, X., Gao, W.: Manifold-manifold distance with application to face recognition based on image set. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2008)

    Google Scholar 

  12. Wang, R., Chen, X.: Manifold discriminant analysis. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2009)

    Google Scholar 

  13. Cui, Z., Shan, S., Zhang, H., Lao, S., Chen, X.: Image sets alignment for video-based face recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012)

    Google Scholar 

  14. Chen, S., Sanderson, C., Harandi, M.T., Lovell, B.C.: Improved image set classification via joint sparse approximated nearest subspaces. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2013)

    Google Scholar 

  15. Li, Z., Liu, J., Tang, J., Lu, H.: Robust structured subspace learning for data representation. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 37, 2085–2098 (2015)

    Article  Google Scholar 

  16. Shakhnarovich, G., Fisher, J.W., Darrell, T.: Face recognition from long-term observations. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2352, pp. 851–865. Springer, Heidelberg (2002). doi:10.1007/3-540-47977-5_56

    Chapter  Google Scholar 

  17. Arandjelović, O., Shakhnarovich, G., Fisher, J., Cipolla, R., Darrell, T.: Face recognition with image sets using manifold density divergence. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2005)

    Google Scholar 

  18. Wang, R., Guo, H., Davis, L.S., Dai, Q.: Covariance discriminative learning: a natural and efficient approach to image set classification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012)

    Google Scholar 

  19. Harandi, M.T., Salzmann, M., Hartley, R.: From manifold to manifold: geometry-aware dimensionality reduction for SPD matrices. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8690, pp. 17–32. Springer, Heidelberg (2014). doi:10.1007/978-3-319-10605-2_2

    Google Scholar 

  20. Huang, Z., Wang, R., Shan, S., Li, X., Chen, X.: Log-Euclidean metric learning on symmetric positive definite manifold with application to image set classification. In: International Conference on Machine Learning (ICML) (2015)

    Google Scholar 

  21. Wang, W., Wang, R., Huang, Z., Shan, S., Chen, X.: Discriminant analysis on Riemannian manifold of Gaussian distributions for face recognition with image sets. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)

    Google Scholar 

  22. Chen, Y.-C., Patel, V.M., Phillips, P.J., Chellappa, R.: Dictionary-based face recognition from video. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 766–779. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33783-3_55

    Chapter  Google Scholar 

  23. Chen, Y.C., Patel, V.M., Shekhar, S., Chellappa, R., Phillips, P.J.: Video-based face recognition via joint sparse representation. In: IEEE Conference on Automatic Face and Gesture Recognition (FG) (2013)

    Google Scholar 

  24. Cui, Z., Chang, H., Shan, S., Ma, B., Chen, X.: Joint sparse representation for video-based face recognition. Neurocomputing 135, 306–312 (2014)

    Article  Google Scholar 

  25. Lu, J., Wang, G., Deng, W., Moulin, P.: Simultaneous feature and dictionary learning for image set based face recognition. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 265–280. Springer, Cham (2014). doi:10.1007/978-3-319-10590-1_18

    Google Scholar 

  26. Hayat, M., Bennamoun, M., An, S.: Learning non-linear reconstruction models for image set classification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014)

    Google Scholar 

  27. Chen, S., Wiliem, A., Sanderson, C., Lovell, B.C.: Matching image sets via adaptive multi convex hull. arXiv preprint arXiv:1403.0320 (2014)

  28. Chen, L.: Dual linear regression based classification for face cluster recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014)

    Google Scholar 

  29. Paredes, R., Vidal, E.: Learning prototypes and distances: a prototype reduction technique based on nearest neighbor error minimization. Pattern Recogn. 39, 180–188 (2006)

    Article  MATH  Google Scholar 

  30. Paredes, R., Vidal, E.: Learning weighted metrics to minimize nearest-neighbor classification error. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 28, 1100–1110 (2006)

    Article  Google Scholar 

  31. Villegas, M., Paredes, R.: Simultaneous learning of a discriminative projection and prototypes for nearest-neighbor classification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2008)

    Google Scholar 

  32. Le, Q.V., Ngiam, J., Coates, A., Lahiri, A., Prochnow, B., Ng, A.Y.: On optimization methods for deep learning. In: International Conference on Machine Learning (ICML) (2011)

    Google Scholar 

  33. Garcia, S., Derrac, J., Cano, J.R., Herrera, F.: Prototype selection for nearest neighbor classification: taxonomy and empirical study. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 34, 417–435 (2012)

    Article  Google Scholar 

  34. Ma, M., Shao, M., Zhao, X., Fu, Y.: Prototype based feature learning for face image set classification. In: IEEE Conference on Automatic Face and Gesture Recognition (FG) (2013)

    Google Scholar 

  35. Kim, M., Kumar, S., Pavlovic, V., Rowley, H.: Face tracking and recognition with visual constraints in real-world videos. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2008)

    Google Scholar 

  36. Huang, Z., Wang, R., Shan, S., Chen, X.: Learning euclidean-to-riemannian metric for point-to-set classification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014)

    Google Scholar 

  37. Ross, B., Phillips, J., Bolme, D., Draper, B., Givens, G., Lui, Y.M., Teli, M.N., Zhang, H., Scruggs, W.T., Bowyer, K., Flynn, P., Cheng, S.: The challenge of face recognition from digital point-and-shoot cameras. In: IEEE Conference on Biometrics Theory, Applications and Systems (BTAS) (2013)

    Google Scholar 

  38. Beveridge, J.R., Zhang, H., Draper, B.A., Flynn, P.J., Feng, Z., Huber, P., Kittler, J., Huang, Z., Li, S., Li, Y., Kan, M., Wang, R., Shan, S., Chen, X.: Report on the FG 2015 video person recognition evaluation. In: IEEE Conference and Workshops on Automatic Face and Gesture Recognition (FG) (2015)

    Google Scholar 

  39. Zhang, X., Zhang, L., Wang, X.J., Shum, H.Y.: Finding celebrities in billions of web images. IEEE Trans. Multimedia 14, 995–1007 (2012)

    Article  Google Scholar 

  40. Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. In: ACM International Conference on Multimedia (2014)

    Google Scholar 

Download references

Acknowledgement

This work is partially supported by 973 Program under contract No. 2015CB351802, Natural Science Foundation of China under contracts Nos. 61390511, 61379083, 61272321, and Youth Innovation Promotion Association CAS No. 2015085.

Author information

Authors and Affiliations

  1. Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CAS, Beijing, 100190, China

    Wen Wang, Ruiping Wang, Shiguang Shan & Xilin Chen

  2. University of Chinese Academy of Sciences, Beijing, 100049, China

    Wen Wang, Ruiping Wang, Shiguang Shan & Xilin Chen

  3. Cooperative Medianet Innovation Center, Beijing, China

    Ruiping Wang, Shiguang Shan & Xilin Chen

Authors
  1. Wen Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ruiping Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shiguang Shan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xilin Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ruiping Wang .

Editor information

Editors and Affiliations

  1. National Tsing Hua University, Hsinchu, Taiwan

    Shang-Hong Lai

  2. Graz University of Technology, Graz, Austria

    Vincent Lepetit

  3. Drexel University, Philadelphia, Pennsylvania, USA

    Ko Nishino

  4. The University of Tokyo , Tokyo, Japan

    Yoichi Sato

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 148 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Wang, W., Wang, R., Shan, S., Chen, X. (2017). Prototype Discriminative Learning for Face Image Set Classification. In: Lai, SH., Lepetit, V., Nishino, K., Sato, Y. (eds) Computer Vision – ACCV 2016. ACCV 2016. Lecture Notes in Computer Science(), vol 10113. Springer, Cham. https://doi.org/10.1007/978-3-319-54187-7_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-54187-7_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-54186-0

  • Online ISBN: 978-3-319-54187-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation