Skip to main content
Springer Nature Link
Log in

Collaborative coding and dictionary learning for nearest subspace classification

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Sparse representation (SR) has been widely studied and successfully applied to many areas of computer science in recent years. However, whether sparsity is essential to improve the classification performance is still an open question. Some studies reveal that it is the collaborative representation (CR) rather than SR that truly improves the classification performance. In this paper, the advantage of CR is further investigated and exploited, and a CR-based coding method is proposed. This method improves the classification performance by applying CR to the traditional nearest subspace (NS) method. Compared to the other NS method which codes the test sample on each class separately, the proposed method employs all samples to code the test sample collaboratively and preserves the subspace structure at the same time. The test sample is then classified to the class with the smallest representation error. Besides, a corresponding dictionary learning algorithm is also proposed so that the coding can be conducted on a dictionary learned from the training dataset. Since analytical solutions for coding and dictionary learning have been derived, our algorithm can be implemented efficiently. Experiments are conducted on seven face databases and the USPS handwritten digit database, and the results show that the proposed algorithm outperforms many state-of-the-art coding methods and dictionary learning methods, which demonstrates the power brought by CR.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Aharon M, Elad M, Bruckstein A (2006) K-svd: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans Signal Process 54:4311–4322

    Article  Google Scholar 

  • Bao C, Ji H, Quan Y, Shen Z (2014) \(l_0\) norm based dictionary learning by proximal methods with global convergence. In: IEEE conference on computer vision and pattern recognition, pp 3858–3865

  • Cai S, Zuo W, Zhang L, Feng X, Wang P (2014) Support vector guided dictionary learning. In: ECCV, 2014, pp 624–639. http://www4.comp.polyu.edu.hk/cslzhang/papers.htm

  • Chao Y-W, Yeh Y-R, Chen Y-W, Lee Y-J, Wang Y-C (2011) Locality-constrained group sparse representation for robust face recognition. In: IEEE international conference on image processing, pp 761–764

  • Cheng B, Yang J, Yan S, Fu Y, Huang TS (2010) Learning with \(l_1\) graph for image analysis. IEEE Trans Image Process 19:858–866

    Article  MathSciNet  Google Scholar 

  • Chi Y-T, Ali M, Rajwade A, Ho J (2013) Block and group regularized sparse modeling for dictionary learning. In: IEEE conference on computer vision and pattern recognition, pp 377–382

  • Chi Y, Porikli F (2012) Connecting the dots in multi-class classification: from nearest subspace to collaborative representation. In: IEEE conference on computer vision and pattern recognition. IEEE, pp 3602–3609

  • Dong LX, Zhang D, Shi G (2011) Sparsity-based image denoising via dictionary learning and structural clustering. In: IEEE conference on computer vision and pattern recognition, pp 457–464

  • Dong W, Zhang D, Shi G, Centralized sparse representation for image restoration. In: IEEE international conference on computer vision, pp 1259–1266

  • Donoho DL (2006) For most large underdetermined systems of linear equations the minimal l1-norm solution is also the sparsest solution. Commun Pure Appl Math 59:797–829

    Article  Google Scholar 

  • Foroughi H, Ray N, Zhang H (2015) Robust people counting using sparse representation and random projection. Pattern Recogn 48(10):3038–3052

    Article  Google Scholar 

  • Georghiades AS, Belhumeur PN, Kriegman D (2001) From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Mach Intell 23:643–660

    Article  Google Scholar 

  • Huang GB, Mattar M, Berg T, Learned-Miller E et al (2008) Labeled faces in the wild: a database forstudying face recognition in unconstrained environments. In: Workshop on faces in’real-life’images: detection, alignment, and recognition

  • Huang K, Aviyente S (2006) Sparse representation for signal classification. In: Advances in neural information processing systems, pp 609–616

  • Jiang Z, Lin Z, Davis LS (2013) Label consistent k-svd: learning a discriminative dictionary for recognition. IEEE Trans Pattern Anal Mach Intell 35:2651–2664

    Article  Google Scholar 

  • Jiang Z, Lin Z, Davis LS (2011) Learning a discriminative dictionary for sparse coding via label consistent k-svd. In: IEEE conference on computer vision and pattern recognition, pp. 1697–1704. http://www.umiacs.umd.edu/zhuolin/projectlcksvd.html

  • Kong S, Wang D (2012) A dictionary learning approach for classification: separating the particularity and the commonality. In: ECCV 2012, pp 186–199. https://github.com/tiepvupsu/DICTOL

  • Lee K-C, Ho J, Kriegman D (2005) Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans Pattern Anal Mach Intell 27:684–698

    Article  Google Scholar 

  • Lee H, Battle A, Raina R, Ng AY (2006)Efficient sparse coding algorithms. In: Advances in neural information processing systems, pp 801–808

  • Liu J, Ji S, Ye J (2009) SLEP: sparse learning with efficient projections. Arizona State University

  • Li K, Yang J, Jiang J (2015) Nonrigid structure from motion via sparse representation. In: IEEE international conference on multimedia and expo, pp 1–6

  • Lu C-Y, Min H, Gui J, Zhu L, Lei Y-K (2013) Face recognition via weighted sparse representation. J Vis Commun Image Represent 24:111–116

    Article  Google Scholar 

  • Mairal J, Bach F, Ponce J, Sapiro G (2009b) Online dictionary learning for sparse coding. In: International conference on machine learning, pp 689–696

  • Mairal J, Bach F, Ponce J, Sapiro G, Zisserman A (2009) Non-local sparse models for image restoration. In: IEEE international conference on computer vision, pp 2272–2279

  • Mairal J, Ponce J, Sapiro G, Zisserman A, Bach FR (2009a) Supervised dictionary learning. In: Advances in neural information processing systems, pp 1033–1040

  • Majumdar A, Ward RK (2009) Classification via group sparsity promoting regularization. In: IEEE international conference on acoustics, speech and signal processing, pp 861–864

  • Martinez AM (1998) The ar face database. CVC Technical Report 24

  • Naseem I, Togneri R, Bennamoun M (2010) Linear regression for face recognition. IEEE Trans Pattern Anal Mach Intell 32:2106–2112

    Article  Google Scholar 

  • Ou W, You X, Tao D, Zhang P, Tang Y, Zhu Z (2014) Robust face recognition via occlusion dictionary learning. Pattern Recogn 47:1559–1572

    Article  Google Scholar 

  • Peng X, Zhang L, Yi Z, Tan KK (2014) Learning locality-constrained collaborative representation for robust face recognition. Pattern Recogn 47:2794–2806

    Article  Google Scholar 

  • Phillips PJ, Flynn PJ, Scruggs T, Bowyer KW, Chang J, Hoffman K, Marques J, Min J, Worek W (2005) Overview of the face recognition grand challenge. In: IEEE conference on computer vision and pattern recognition, vol 1, pp 947–954

  • Phillips PJ, Wechsler H, Huang J, Rauss PJ (1998) The feret database and evaluation procedure for face-recognition algorithms. Image Vis Comput 16:295–306

    Article  Google Scholar 

  • Ramirez I, Sprechmann P, Sapiro G (2010) Classification and clustering via dictionary learning with structured incoherence and shared features. In: IEEE conference on computer vision and pattern recognition, pp 3501–3508

  • Rigamonti R, Brown MA, Lepetit V (2011) Are sparse representations really relevant for image classification? In: IEEE conference on computer vision and pattern recognition, pp 1545–1552

  • Samaria FS, Harter AC (1994) Parameterisation of a stochastic model for human face identification. In: IEEE workshop on applications of computer vision, pp 138–142

  • Shi Q, Eriksson A, Van Den Hengel A, Shen C (2011) Is face recognition really a compressive sensing problem? In: IEEE conference on computer vision and pattern recognition, pp 553–560

  • Shrivastava A, Pillai JK, Patel VM (2015) Multiple kernel-based dictionary learning for weakly supervised classification. Pattern Recogn 48:2667–2675

    Article  Google Scholar 

  • Sim T, Baker S, Bsat M (2002) The cmu pose, illumination, and expression (pie) database. In: IEEE international conference on automatic face and gesture recognition, pp 46–51

  • Sun Y, Liu Q, Tang J, Tao D (2014) Learning discriminative dictionary for group sparse representation. IEEE Trans Image Process 23:3816–3828

    Article  MathSciNet  Google Scholar 

  • USPS Handwritten Digit Database (1990)

  • Wang D, Kong S (2014) A classification-oriented dictionary learning model: Explicitly learning the particularity and commonality across categories. Pattern Recogn 47:885–898

    Article  Google Scholar 

  • Wang J, Yang J, Yu K, Lv F, Huang T, Gong Y (2010) Locality-constrained linear coding for image classification. In: IEEE conference on computer vision and pattern recognition, pp 3360–3367

  • Wei CP, Chao YW, Yeh YR, Wang YCF (2013) Locality-sensitive dictionary learning for sparse representation based classification. Pattern Recogn 46:1277–1287

    Article  Google Scholar 

  • Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31:210–227

    Article  Google Scholar 

  • Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19:2861–2873

    Article  MathSciNet  Google Scholar 

  • Yang M, Dai D, Shen L, Gool LV (2014) Latent dictionary learning for sparse representation based classification. In: IEEE conference on computer vision and pattern recognition, pp 4138–4145

  • Yang J, Yu K, Gong Y, Huang T (2009) Linear spatial pyramid matching using sparse coding for image classification. In: IEEE conference on computer vision and pattern recognition, pp 1794–1801

  • Yang J, Yu K, Huang T (2010) Supervised translation-invariant sparse coding. In: IEEE conference on computer vision and pattern recognition, pp 3517–3524

  • Yang M, Zhang D, Feng X (2011) Fisher discrimination dictionary learning for sparse representation. In: IEEE international conference on computer vision, pp 543–550. http://www4.comp.polyu.edu.hk/~cslzhang/papers.htm

  • Yang M, Zhang L, Feng X, Zhang D (2014) Sparse representation based fisher discrimination dictionary learning for image classification. Int J Comput Vis 1–24

  • Yang M, Zhang D, Wang S (2012) Relaxed collaborative representation for pattern classification. In: IEEE conference on computer vision and pattern recognition. IEEE, pp 2224–2231

  • Yang M, Zhang D, Yang J (2011) Robust sparse coding for face recognition. In: IEEE conference on computer vision and pattern recognition, pp 625–632

  • Yang M, Zhang L, Yang J, Zhang D (2010) Metaface learning for sparse representation based face recognition. In: IEEE international conference on image processing, pp 1601–1604

  • Yuan M, Lin Y (2006) Model selection and estimation in regression with grouped variables. J R Stat Soc Ser B (Stat Methodol) 68:49–67

    Article  MathSciNet  Google Scholar 

  • Yuan X, Yan S (2010) Visual classification with multi-task joint sparse representation. In: IEEE conference on computer vision and pattern recognition, pp 3493–3500

  • Zhang X, Pham D-S, Venkatesh S, Liu W, Phung D (2015) Mixed-norm sparse representation for multi view face recognition. Pattern Recogn 48:2935–2946

    Article  Google Scholar 

  • Zhang Q, Li B (2010) Discriminative k-svd for dictionary learning in face recognition. In: IEEE conference on computer vision and pattern recognition, pp 2691–2698

  • Zhang L, Yang M, Feng X (2011) Sparse representation or collaborative representation: which helps face recognition? In: IEEE international conference on computer vision, pp 471–478. http://www4.comp.polyu.edu.hk/cslzhang/papers.htm

  • Zheng M, Bu J, Chen C, Wang C, Zhang L, Qiu G, Cai D (2011) Graph regularized sparse coding for image representation. IEEE Trans Image Process 20:1327–1336

    Article  MathSciNet  Google Scholar 

  • Zhou N, Shen Y, Peng J, Fan J (2012) Learning inter-related visual dictionary for object recognition. In: IEEE conference on computer vision and pattern recognition. IEEE, pp 3490–3497

Download references

Funding

This work is partially supported by the NSFC under Grant Nos. 61272338 and 61673018.

Author information

Authors and Affiliations

  1. School of Mathematics, Sun Yat-sen University, Guangzhou, 510275, Guangdong, China

    Zhong Zhao, Qi Shen & Guocan Feng

  2. Guangdong Province Key Laboratory of Computational Science, Guangzhou, 510275, Guangdong, China

    Zhong Zhao, Qi Shen & Guocan Feng

  3. Department of Mathematical Sciences, Georgia Southern University, Statesboro, GA, 30460, USA

    Jiehua Zhu

Authors
  1. Zhong Zhao
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Qi Shen
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Guocan Feng
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Jiehua Zhu
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

ZZ and QS designed the model and the computational framework. ZZ carried out the implementation and performed the calculations. ZZ and QS wrote the original draft with input from all authors. QS, GCF and JHZ contributed to reviewing and editing.

Corresponding author

Correspondence to Qi Shen.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Qi Shen: This work is partially supported by the NSFC under Grant Nos. 61272338 and 61673018.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, Z., Shen, Q., Feng, G. et al. Collaborative coding and dictionary learning for nearest subspace classification. Soft Comput 25, 7627–7643 (2021). https://doi.org/10.1007/s00500-021-05723-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-021-05723-3

Keywords