Skip to main content
SpringerLink
Log in

Query set centered sparse projection learning for set based image classification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Set based image classification technology has been developed successfully in recent decades. Previous approaches dispose set based image classification by employing all the gallery sets to learn metrics or construct the model using a typical number of parameters. However, they are based on the assumption that the global structure is consistent with the local structure, which is rigid in real applications. Additionally, the participation of all gallery sets increases the influence of outliers. This paper conducts this task via sparse projection learning by employing ℓ2,1 norm from the perspective of the query set. Instead of involving all the image sets, this work devotes to searching for a local region, which is centered with a query set and constructed by the candidates selected from different classes in the gallery sets. By maximizing the inter-class while minimizing the intra-class of the candidates from the gallery sets from the query set, this work can learn a discriminate and sparse projection for image set feature extraction. In order to learn the projection, an alternative updating algorithm to solve the optimization problem is proposed and the convergence and complexity are analyzed. Finally, the distance is measured in the discriminate low-dimensional space using Euclidean distance between the central data point of the query set and the central one of images from the same class. The proposed approach learns the projection in the local set centered with the query set with ℓ2,1 norm, which contributes to more discriminative feature. Compared with the existing algorithms, the experiments on the challenging databases demonstrate that the proposed simple yet effective approach obtains the best classification accuracy with comparable time cost.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. In this paper, factitious set are the set constructed by devices and only indicates the image individuals are multi-view appearances of same subject. Conversely, inherent set can indicate the label information. If two sets are belonging to the same class, they are in the same inherent set.

  2. http://www.ri.cmu.edu/publicationview.html?pubid=3904

References

  1. Arandjelovic O, Shakhnarovich G, Fisher J, Cipolla R (2005) Face recognition with image sets using manifold density divergence. In: CVPR, pp 581–588

  2. Cevikalp H, Triggs B (2010) Face recognition based on image sets. In: CVPR, pp 2567–2573

  3. Chen L (2014) Dual linear regression based classification for face cluster recognition. In: CVPR, pp 2673–2680

  4. Cheng G, Zhou PC, Han JW (2017) Duplex metric learning for image set classification. IEEE Trans Image Process 27(1):281–292

    Article  MathSciNet  Google Scholar 

  5. Crisp DJ, Burges CJC (1999) A geometric interpretation of v-svm classifiers. In: NIPS, pp 244–250

  6. Du X, Wang JY (2015) Support image set machine: jointly learning representation and classifier for image set classification. Knowl-Based Syst 78:51–58

    Article  Google Scholar 

  7. Hayat M, Bennamoun M, An SJ (2015) Deep learning non-linear reconstruction models for image set classification. IEEE Trans Pattern Anal Machine Intell 37(4):713–727

    Article  Google Scholar 

  8. Hu YQ, Mian AS, Owens R (2011) Sparse approximated nearest points for image set classification. In: CVPR, pp 121–128

  9. Huang ZW, Wang RP, Shan SG, Chen XL (2015) Projection metric learning on grassmann manifold with application to video based face recognition. In: CVPR, pp 140–149

  10. Kim M (2018) Dynamic sparse coding for sparse time-series modeling via first-order smooth optimization. Appl Intell 48(11):3889–3901

    Article  Google Scholar 

  11. Kim TK, Kittler J, Cipolla R (2007) Discriminative learning and recognition of image set classes using canonical correlations. IEEE Trans Pattern Anal Mach Intell 29(6):1005–1018

    Article  Google Scholar 

  12. Lee KC, Ho J, Yang MH, Kriegman D (2003) Video-based face recognition using probabilistic appearance manifolds. In: CVPR, pp I–I

  13. Liu B, Jing LP, Li J, Yu J, Gittens A, Mahoney MW (2019) Group collaborative representation for image set classification. Int J Comput Vis 127(2):181–206

    Article  Google Scholar 

  14. Lu JW, Wang G, Moulin P (2013) Image set classification using holistic multiple order statistics features and localized multi-kernel metric learning. In: ICCV, pp 329–336

  15. Mensink T, Verbeek J, Perronnin F, Csurka G (2013) Distance-based image classification: generalizing to new classes at near-zero cost. IEEE Trans Pattern Anal Machine Intell 35(11):2624–2637

    Article  Google Scholar 

  16. Muja M, Lowe DG (2014) Scalable nearest neighbor algorithms for high dimensional data. IEEE Trans Pattern Anal Mach Intell 36(11):2227–2240

    Article  Google Scholar 

  17. Nie FP, Huang H, Xiao C, Ding CHQ (2010) Efficient and robust feature selection via joint l21-norms minimization. In: NIPS, pp 1813–1821

  18. Ren ZW, Sun QS, Yang C (2019) Nonnegative discriminative encoded nearest points for image set classification. Neural Computing and Applications. https://doi.org/10.1007/s00521-019-04419-y

  19. Ross DA, Lim J, Lin RS, Yang MH (2008) Incremental learning for robust visual tracking. Int J Comput Vis 77:125–141

    Article  Google Scholar 

  20. Shah SAA, Nadeem U, Bennamoun M, Sohel F, Togneri R (2017) Efficient image set classification using linear regression based image reconstruction. In: CVPRW, pp 601–610

  21. Stallkamp J, Ekenel HK, Stiefelhagen R (2007) Video-based face recognition on real-world data. In: ICCV, pp 1–8

  22. Tan HL, Gao Y, Ma ZM (2018) Regularized constraint subspace based method for image set classification. Pattern Recogn 76:434–448

    Article  Google Scholar 

  23. Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154

    Article  Google Scholar 

  24. Wagner A, Wright J, Ganesh A, Zhou Z, Mobahi H, Ma Y (2012) Toward a practical face recognition system: robust alignment and illumination by sparse representation. IEEE Trans Pattern Anal Mach Intell 34(2):372–386

    Article  Google Scholar 

  25. Wang RP, Chen XL (2009) Manifold discriminant analysis. In: CVPR, pp 429–436

  26. Wang RP, Shan SG, Chen XL, Gao W (2008) Manifold-manifold distance with application to face recognition based on image set. In: CVPR, pp 1–8

  27. Wang RP, Guo HM, DL S, D QH (2012) Covariance discriminative learning: a natural and efficient approach to image set classification. In: CVPR, pp 2496–2503

  28. Wang W, Wang RP, Shan SG, Chen XL (2016) Prototype discriminative learning for face image set classification. In: ACCV, pp 344–360

  29. Wang Y, Wang A, Ai Q, Sun HJ (2019) Ensemble based fuzzy weighted extreme learning machine for gene expression classification. Appl Intell 49(3):1161–1171

    Article  Google Scholar 

  30. Wolf L, Hassner T, Maoz I (2011) Face recognition in unconstrained videos with matched background similarity. In: CVPR, pp 529–534

  31. Yamaguchi O, Fukui K, Maeda K (1998) Face recognition using temporal image sequence. In: IEEE international conference on automatic face and gesture recognition, pp 318–323

  32. Yan WZ, Sun HJ, Sun QS, Zheng ZC, Gao XZ, Zhang Q, Ren ZW (2019) Multiple kernel dimensionality reduction based on collaborative representation for set oriented image classification. Expert Systems with Applications 137:380– 391

    Article  Google Scholar 

  33. Yan WZ, Sun QS, Sun HJ, Li YM, Ren ZW (2019) Multiple kernel dimensionality reduction based on linear regression virtual reconstruction for image set classification. Neurocomputing 361:256–269

    Article  Google Scholar 

  34. Yang M, Zhu PF, Gool LV, Zhang L (2013) Face recognition based on regularized nearest points between image sets. In: IEEE international conference and workshops on automatic face and gesture recognition, pp 1–7

  35. Zheng P, Zhao ZQ, Gao J, Wu XD (2018) A set-level joint sparse representation for image set classification. Inf Sci 448: 75–90

    Article  MathSciNet  Google Scholar 

  36. Zhu PF, Zhang L, Zuo WM, Zhang D (2013) From point to set: extend the learning of distance metrics. In: ICCV, pp 2664– 2671

  37. Zhu PF, Zuo WM, Zhang L, Shiu CK (2014) Image set-based collaborative representation for face recognition. IEEE Trans Inform Forensics Secur 9(7):1120–1132

    Article  Google Scholar 

  38. Zhu WJ, Yan YH (2018) Joint linear regression and nonnegative matrix factorization based on self-organized graph for image clustering and classification. IEEE Access 6:38820–38834

    Article  Google Scholar 

  39. Zhu WJ, Yan YH, Peng YS (2016) Dictionary learning based on discriminative energy contribution for image classification. Knowl-Based Syst 113:116–124

    Article  Google Scholar 

  40. Zhu WJ, Yan YH, Peng YS (2017) Pair of projections based on sparse consistence with applications to efficient face recognition. Signal Processing:Image Communication 55C:32–40

    Google Scholar 

Download references

Acknowledgments

This work is in part by Natural Science Foundation of Zhejiang Province (NO. LQ20F030015).

Author information

Authors and Affiliations

  1. Key Laboratory of Electromagnetic Wave Information Technology and Metrology of Zhejiang Province, College of Information Engineering, China Jiliang University, Hangzhou, 310018, China

    Wenjie Zhu, Bo Peng, Han Wu & Binhao Wang

Authors
  1. Wenjie Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bo Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Han Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Binhao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenjie Zhu.

Ethics declarations

Conflict of interests

The authors declare that they have no conflicts of interest to this work.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, W., Peng, B., Wu, H. et al. Query set centered sparse projection learning for set based image classification. Appl Intell 50, 3400–3411 (2020). https://doi.org/10.1007/s10489-020-01730-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-01730-3

Keywords

Search

Navigation