Skip to main content
SpringerLink
Log in
Book cover

Asian Conference on Computer Vision

ACCV 2018: Computer Vision – ACCV 2018 pp 142–158Cite as

Nonlinear Subspace Feature Enhancement for Image Set Classification

  • Conference paper
  • First Online:

  • 1770 Accesses

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11364))

Abstract

While several methods have been proposed for modeling and recognizing image sets, the success of these methods relies heavily on how well the image data follows the assumptions of the underlying models. Among the models that have been utilized by many image set classification methods, the physically inspired subspace model assumes that the images of an object lie on a union of low-dimensional subspaces. Despite their successful performance in controlled environments, the performance of such subspace-based classifiers suffers in practical unconstrained settings, where the data may not strictly follow the assumptions necessary for the subspace model to hold. In this paper, we propose Nonlinear Subspace Feature Enhancement (NSFE), an approach for nonlinearly embedding image sets into a space where they adhere to a more discriminative subspace structure. In turn, this improves the performance of subspace-based classifiers such as sparse representation-based classification. We describe how the structured loss function of NSFE can be optimized in a batch-by-batch fashion by a two-step alternating algorithm. The algorithm makes very few assumptions about the form of the embedding to be learned and is compatible with stochastic gradient descent and back-propagation. This makes NSFE usable with deep, feed-forward embeddings and trainable in an end-to-end fashion. We experiment with two different types of features and nonlinear embeddings over three image set datasets and we show that our method compares favorably to state-of-the-art image set classification methods.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Wang, R., Shan, S., Chen, X., Gao, W.: Manifold-manifold distance with application to face recognition based on image set. In: CVPR, pp. 1–8 (2008)

    Google Scholar 

  2. Wang, R., Chen, X.: Manifold discriminant analysis. In: CVPR, pp. 429–436 (2009)

    Google Scholar 

  3. Cevikalp, H., Triggs, B.: Face recognition based on image sets. In: CVPR, pp. 2567–2573 (2010)

    Google Scholar 

  4. Hu, Y., Mian, A.S., Owens, R.: Sparse approximated nearest points for image set classification. In: CVPR, pp. 121–128 (2011)

    Google Scholar 

  5. Harandi, M.T., Sanderson, C., Shirazi, S., Lovell, B.C.: Graph embedding discriminant analysis on Grassmannian manifolds for improved image set matching. In: CVPR, pp. 2705–2712 (2011)

    Google Scholar 

  6. Mahmood, A., Mian, A.: Hierarchical sparse spectral clustering for image set classification. In: BMVC, pp. 1–11 (2012)

    Google Scholar 

  7. Chen, Y.-C., Patel, V.M., Phillips, P.J., Chellappa, R.: Dictionary-based face recognition from video. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 766–779. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33783-3_55

    Chapter  Google Scholar 

  8. Wang, R., Guo, H., Davis, L.S., Dai, Q.: Covariance discriminative learning: a natural and efficient approach to image set classification. In: CVPR, pp. 2496–2503 (2012)

    Google Scholar 

  9. Harandi, M., Sanderson, C., Shen, C., Lovell, B.C.: Dictionary learning and sparse coding on Grassmann manifolds: an extrinsic solution. In: ICCV, pp. 3120–3127 (2013)

    Google Scholar 

  10. Ortiz, E.G., Wright, A., Shah, M.: Face recognition in movie trailers via mean sequence sparse representation-based classification. In: CVPR, pp. 3531–3538 (2013)

    Google Scholar 

  11. Chen, S., Sanderson, C., Harandi, M.T., Lovell, B.C.: Improved image set classification via joint sparse approximated nearest subspaces. In: CVPR, pp. 452–459 (2013)

    Google Scholar 

  12. Chen, L.: Dual linear regression based classification for face cluster recognition. In: CVPR, pp. 2673–2680 (2014)

    Google Scholar 

  13. Mahmood, A., Mian, A., Owens, R.: Semi-supervised spectral clustering for image set classification. In: CVPR, pp. 121–128 (2014)

    Google Scholar 

  14. Hayat, M., Bennamoun, M., An, S.: Learning non-linear reconstruction models for image set classification. In: CVPR, pp. 1915–1922 (2014)

    Google Scholar 

  15. Hayat, M., Bennamoun, M., An, S.: Reverse training: an efficient approach for image set classification. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 784–799. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10599-4_50

    Chapter  Google Scholar 

  16. Lu, J., Wang, G., Deng, W., Moulin, P.: Simultaneous feature and dictionary learning for image set based face recognition. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 265–280. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_18

    Chapter  Google Scholar 

  17. Lu, J., Wang, G., Deng, W., Moulin, P., Zhou, J.: Multi-manifold deep metric learning for image set classification. In: CVPR (2015) 1137–1145

    Google Scholar 

  18. Huang, Z., Wang, R., Shan, S., Li, X., Chen, X.: Log-Euclidean metric learning on symmetric positive definite manifold with application to image set classification. In: ICML, pp. 720–729 (2015)

    Google Scholar 

  19. Wang, W., Wang, R., Huang, Z., Shan, S., Chen, X.: Discriminant analysis on Riemannian manifold of Gaussian distributions for face recognition with image sets. In: CVPR, pp. 2048–2057 (2015)

    Google Scholar 

  20. Basri, R., Jacobs, D.W.: Lambertian reflectance and linear subspaces. PAMI 25, 218–233 (2003)

    Article  Google Scholar 

  21. Wright, J., Yang, A.Y., Ganesh, A., Sastry, S.S., Ma, Y.: Robust face recognition via sparse representation. PAMI 31, 210–227 (2009)

    Article  Google Scholar 

  22. Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. JMLR 12, 2121–2159 (2011)

    MathSciNet  MATH  Google Scholar 

  23. Sutskever, I., Martens, J., Dahl, G.E., Hinton, G.E.: On the importance of initialization and momentum in deep learning. In: ICML, pp. 1139–1147 (2013)

    Google Scholar 

  24. Hamm, J., Lee, D.D.: Grassmann discriminant analysis: a unifying view on subspace-based learning. In: ICML, pp. 376–383 (2008)

    Google Scholar 

  25. Huang, Z., Wang, R., Shan, S., Chen, X.: Projection metric learning on Grassmann manifold with application to video based face recognition. In: CVPR, pp. 140–149 (2015)

    Google Scholar 

  26. Zhu, P., Zhang, L., Zuo, W., Zhang, D.: From point to set: extend the learning of distance metrics. In: ICCV, pp. 2664–2671 (2013)

    Google Scholar 

  27. Harandi, M., Salzmann, M., Baktashmotlagh, M.: Beyond Gauss: image-set matching on the Riemannian manifold of PDFs. In: ICCV, pp. 4112–4120 (2015)

    Google Scholar 

  28. Hayat, M., Bennamoun, M., An, S.: Deep reconstruction models for image set classification. PAMI 37, 713–727 (2015)

    Article  Google Scholar 

  29. Zhang, L., Yang, M., Feng, X.: Sparse representation or collaborative representation: which helps face recognition? In: ICCV, pp. 471–478 (2011)

    Google Scholar 

  30. Zhu, P., Zuo, W., Zhang, L., Shiu, S.C.K., Zhang, D.: Image set-based collaborative representation for face recognition. IEEE Trans. Inf. Forens. Secur. 9, 1120–1132 (2014)

    Article  Google Scholar 

  31. Zhang, H., Zhang, Y., Huang, T.S.: Simultaneous discriminative projection and dictionary learning for sparse representation based classification. Pattern Recogn. 46, 346–354 (2013)

    Article  Google Scholar 

  32. Qiu, Q., Sapiro, G.: Learning transformations for clustering and classification. JMLR 16, 187–225 (2015)

    MathSciNet  MATH  Google Scholar 

  33. Fathy, M.E., Alavi, A., Chellappa, R.: Discriminative Log-Euclidean feature learning for sparse representation-based recognition of faces from videos, pp. 3359–3367 (2016)

    Google Scholar 

  34. Weinberger, K.Q., Blitzer, J., Saul, L.K.: Distance metric learning for large margin nearest neighbor classification. In: NIPS, pp. 1473–1480 (2005)

    Google Scholar 

  35. Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: CVPR, pp. 815–823 (2015)

    Google Scholar 

  36. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)

    Google Scholar 

  37. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: ICML, pp. 448–456 (2015)

    Google Scholar 

  38. Mairal, J., Bach, F., Ponce, J., Sapiro, G.: Online dictionary learning for sparse coding. In: ICML, pp. 689–696 (2009)

    Google Scholar 

  39. Kim, M., Kumar, S., Pavlovic, V., Rowley, H.: Face tracking and recognition with visual constraints in real-world videos. In: CVPR, pp. 1–8 (2008)

    Google Scholar 

  40. Viola, P., Jones, M.J.: Robust real-time face detection. IJCV 57, 137–154 (2004)

    Article  Google Scholar 

  41. Asthana, A., Zafeiriou, S., Cheng, S., Pantic, M.: Robust discriminative response map fitting with constrained local models. In: CVPR, pp. 3444–3451 (2013)

    Google Scholar 

  42. Wolf, L., Hassner, T., Maoz, I.: Face recognition in unconstrained videos with matched background similarity. In: CVPR, pp. 529–534 (2011)

    Google Scholar 

  43. Fathy, M.E., Patel, V.M., Chellappa, R.: Face-based active authentication on mobile devices. In: ICASSP, pp. 1687–1691 (2015)

    Google Scholar 

Download references

Acknowledgment

This research is based upon work supported by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), via IARPA R&D Contract No. 2014-14071600012. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the ODNI, IARPA, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation thereon.

Author information

Authors and Affiliations

  1. Center for Automation Research, University of Maryland, College Park, MD, 20742, USA

    Mohammed E. Fathy, Azadeh Alavi & Rama Chellappa

Authors
  1. Mohammed E. Fathy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Azadeh Alavi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rama Chellappa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammed E. Fathy .

Editor information

Editors and Affiliations

  1. IIIT Hyderabad, Hyderabad, India

    C.V. Jawahar

  2. ANU, Canberra, ACT, Australia

    Hongdong Li

  3. Simon Fraser University, Burnaby, BC, Canada

    Greg Mori

  4. ETH Zurich, Zurich, Zürich, Switzerland

    Konrad Schindler

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Fathy, M.E., Alavi, A., Chellappa, R. (2019). Nonlinear Subspace Feature Enhancement for Image Set Classification. In: Jawahar, C., Li, H., Mori, G., Schindler, K. (eds) Computer Vision – ACCV 2018. ACCV 2018. Lecture Notes in Computer Science(), vol 11364. Springer, Cham. https://doi.org/10.1007/978-3-030-20870-7_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-20870-7_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-20869-1

  • Online ISBN: 978-3-030-20870-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation