Skip to main content
SpringerLink
Log in

Sparse coding and dictionary learning with class-specific group sparsity

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In recent years, sparse coding via dictionary learning has been widely used in many applications for exploiting sparsity patterns of data. For classification, useful sparsity patterns should have discrimination, which cannot be well achieved by standard sparse coding techniques. In this paper, we investigate structured sparse coding for obtaining discriminative class-specific group sparsity patterns in the context of classification. A structured dictionary learning approach for sparse coding is proposed by considering the \(\ell _{2,0}\) norm on each class of data. An efficient numerical algorithm with global convergence is developed for solving the related challenging \(\ell _{2,0}\) minimization problem. The learned dictionary is decomposed into class-specific dictionaries for the classification that is done according to the minimum reconstruction error among all the classes. For evaluation, the proposed method was applied to classifying both the synthetic data and real-world data. The experiments show the competitive performance of the proposed method in comparison with several existing discriminative sparse coding methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. For fair comparison, the dictionary size of SRC is set the same as our method in all the experiments.

  2. In practice, we found that the parameter \(\alpha\) can be simply set to a sufficiently small positive constant.

References

  1. Aharon M, Elad M, Bruckstein A (2006) K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans Sig Process 54(11):4311–4322

    Article  MATH  Google Scholar 

  2. Bach F, Jenatton R, Mairal J, Obozinski G et al (2012) Structured sparsity through convex optimization. Stat Sci 27(4):450–468

    Article  MathSciNet  MATH  Google Scholar 

  3. Bao C, Ji H, Quan Y, Shen Z (2014) \(\ell _0\) Norm-based dictionary learning by proximal methods with global convergence. In: CVPR, pp 3858–3865

  4. Bao C, Ji H, Quan Y, Shen Z (2016) Dictionary learning for sparse coding: algorithms and convergence analysis. IEEE Trans Pattern Anal 38(7):1356–1369

    Article  Google Scholar 

  5. Boureau YL, Bach F, LeCun Y, Ponce J (2010) Learning mid-level features for recognition. In: CVPR. IEEE, pp 2559–2566

  6. Cai S, Zuo W, Zhang L, Feng X, Wang P (2014) Support vector guided dictionary learning. In: ECCV. Springer, pp 624–639

  7. Cai X, Nie F, Cai W, Huang H (2013) New graph structured sparsity model for multi-label image annotations. In: ICCV. IEEE, pp 801–808

  8. Cai X, Nie F, Huang H (2013) Exact top-\(k\) feature selection via \(\ell _{2,0}\)-norm constraint. In: IJCAI. AAAI Press, pp 1240–1246

  9. Candes EJ, Romberg JK, Tao T (2006) Stable signal recovery from incomplete and inaccurate measurements. Commun Pure Appl Math 59(8):1207–1223

    Article  MathSciNet  MATH  Google Scholar 

  10. Castrodad A, Sapiro G (2012) Sparse modeling of human actions from motion imagery. Int J Comput Vis 100(1):1–15

    Article  Google Scholar 

  11. Chen YC, Patel VM, Shekhar S, Chellappa R, Phillips PJ (2013) Video-based face recognition via joint sparse representation. In: ICAFGR. IEEE, pp 1–8

  12. Chi YT, Ali M, Rajwade A, Ho J (2013) Block and group regularized sparse modeling for dictionary learning. In: CVPR. IEEE, pp 377–382

  13. Donoho DL (2006) Compressed sensing. IEEE Trans Inf Theory 52(4):1289–1306

    Article  MathSciNet  MATH  Google Scholar 

  14. Elad M, Aharon M (2006) Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans Image Process 15(12):3736–3745

    Article  MathSciNet  Google Scholar 

  15. Elhamifar E, Vidal R (2012) Block-sparse recovery via convex optimization. IEEE Trans Sig Process 60(8):4094–4107

    Article  MathSciNet  Google Scholar 

  16. Gao S, Tsang WH, Ma Y (2013) Learning category-specific dictionary and shared dictionary for fine-grained image categorization. IEEE Trans Image Process 23(2):623–634

    MathSciNet  MATH  Google Scholar 

  17. Georghiades AS, Belhumeur PN, Kriegman D (2001) From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal 23(6):643–660

    Article  Google Scholar 

  18. Huang J, Zhang T, Metaxas D (2011) Learning with structured sparsity. J Mach Learn Res 12:3371–3412

    MathSciNet  MATH  Google Scholar 

  19. Jacob L, Obozinski G, Vert JP (2009) Group lasso with overlap and graph lasso. In: ICML. ACM, pp 433–440

  20. Jenatton R, Mairal J, Obozinski G, Bach F (2011) Proximal methods for hierarchical sparse coding. J Mach Learn Res 12(7):2297–2334

    MathSciNet  MATH  Google Scholar 

  21. Jenatton R, Gramfort A, Thirion B et al (2011) Multiscale mining of fmri data with hierarchical structured sparsity. SIAM J Imaging Sci 5(3):835–856

    Article  MathSciNet  MATH  Google Scholar 

  22. Jerome B, Shoham S, Teboulle M (2014) Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math Program 146(1–2):459–494

    MathSciNet  MATH  Google Scholar 

  23. Jiang Z, Lin Z, Davis L (2013) Label consistent K-SVD: learning a discriminative dictionary for recognition. IEEE Trans Pattern Anal 35(11):2651–2664

    Article  Google Scholar 

  24. Kavukcuoglu K, Ranzato M, Fergus R, LeCun Y (2009) Learning invariant features through topographic filter maps. In: CVPR. IEEE, pp 1605–1612

  25. Kim S, Xing EP (2010) Tree-guided group lasso for multi-task regression with structured sparsity. In: ICML, pp 543–550

  26. Kong S, Wang D (2012) A dictionary learning approach for classification: separating the particularity and the commonality. In: ECCV. Springer, pp 186–199

  27. Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: CVPR, vol 2. IEEE, pp 2169–2178

  28. Lee H, Ekanadham C, Ng AY (2008) Sparse deep belief net model for visual area v2. In: NIPS, pp 873–880

  29. Lian XC, Li Z, Lu BL, Zhang L (2010) Max-margin dictionary learning for multiclass image categorization. In: ECCV. Springer, pp 157–170

  30. Mairal J, Bach F, Ponce J (2012) Task-driven dictionary learning. IEEE Trans Pattern Anal 34(4):791–804

    Article  Google Scholar 

  31. Majumdar A, Ward RK (2009) Classification via group sparsity promoting regularization. In: ICASSP. IEEE, pp 861–864

  32. Martinez AM (1998) The AR face database. CVC Technical Report 24

  33. Nie F, Huang H, Cai X, Ding CH (2010) Efficient and robust feature selection via joint l2, 1-norms minimization. In: NIPS, pp 1813–1821

  34. Pham DS, Venkatesh S (2008) Joint learning and dictionary construction for pattern recognition. In: CVPR. IEEE, pp 1–8

  35. Quan Y, Xu Y, Sun Y, Huang Y (2016) Supervised dictionary learning with multiple classifier integration. Pattern Recognit 55:247–260

    Article  Google Scholar 

  36. Quan Y, Xu Y, Sun Y, Huang Y, Ji H (2016) Sparse coding for classification via discrimination ensemble. In: CVPR. IEEE, pp 5839–5847

  37. Ramirez I, Sprechmann P, Sapiro G (2010) Classification and clustering via dictionary learning with structured incoherence and shared features. In: CVPR. IEEE, pp 3501–3508

  38. Rao N, Nowak R, Cox C, Rogers T (2016) Classification with the sparse group lasso. IEEE Trans Sig Process 64(2):448–463

    Article  MathSciNet  Google Scholar 

  39. Rodriguez JAM, Shah M (2008) Action mach: a spatio-temporal maximum average correlation height filter for action recognition. In: CVPR. IEEE, pp 1–8

  40. Zelnik-Manor L, Rosenblum K, Eldar Y (2012) Dictionary optimization for block-sparse representations. IEEE Trans Sig Process 60(5):2386–2395

    Article  MathSciNet  Google Scholar 

  41. Sadanand S, Corso JJ (2012) Action bank: a high-level representation of activity in video. In: CVPR. IEEE, pp 1234–1241

  42. Shen Z, Toh KC, Yun S (2011) An accelerated proximal gradient algorithm for frame-based image restoration via the balanced approach. SIAM J Imaging Sci 4(2):573–596

    Article  MathSciNet  MATH  Google Scholar 

  43. Sprechmann P, Ramirez I, Sapiro G, Eldar YC (2011) C-Hilasso: a collaborative hierarchical sparse modeling framework. IEEE Trans Sig Process 59(9):4183–4198

    Article  MathSciNet  Google Scholar 

  44. Sun F, Xu M, Hu X, Jiang X (2015) Graph regularized and sparse nonnegative matrix factorization with hard constraints for data representation. Neurocomputing 173:233–244

    Article  Google Scholar 

  45. Szlam A, Gregor K, LeCun Y (2012) Fast approximations to structured sparse coding and applications to object classification. In: ECCV. Springer, pp 200–213

  46. Wang J, Yang J, Yu K, Lv F, Huang T, Gong Y (2010) Locality-constrained linear coding for image classification. In: CVPR. IEEE, pp 3360–3367

  47. Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal 31(2):210–227

    Article  Google Scholar 

  48. Xu Y, Sun Y, Quan Y, Luo Y (2015) Structured sparse coding for classification via reweighted\(\ell _ {2, 1}\) minimization. In: CCCV. Springer, pp 189–199

  49. Xu Y, Sun Y, Quan Y, Zheng B (2015) Discriminative structured dictionary learning with hierarchical group sparsity. Comput Vis Image Underst 136:59–68

    Article  Google Scholar 

  50. Yang J, Yu K, Gong Y, Huang T (2009) Linear spatial pyramid matching using sparse coding for image classification. In: CVPR. IEEE, pp 1794–1801

  51. Yang M, Zhang D, Feng X (2011) Fisher discrimination dictionary learning for sparse representation. In: ICCV. IEEE, pp 543–550

  52. Yang M, Zhang D, Yang J (2011) Robust sparse coding for face recognition. In: CVPR. IEEE, pp 625–632

  53. Yang M, Dai D, Shen L, Gool LV (2014) Latent dictionary learning for sparse representation-based classification. In: CVPR. IEEE, pp 4138–4145

  54. Deng W, Yin W, Zhang Y (2013) Group sparse optimization by alternating direction method. Proc. SPIE, Wavelets and Sparsity XV, 88580R

  55. Yuan M, Lin Y (2006) Model selection and estimation in regression with grouped variables. J R Stat Soc B 68(1):49–67

    Article  MathSciNet  MATH  Google Scholar 

  56. Zhang D, Liu P, Zhang K, Zhang H, Wang Q, Jing X (2015) Class relatedness oriented-discriminative dictionary learning for multiclass image classification. Pattern Recognit 59:168–175

    Article  Google Scholar 

  57. Zhang Q, Li B (2010) Discriminative K-SVD for dictionary learning in face recognition. In: CVPR. IEEE, pp 2691–2698

  58. Zhang Y, Jiang Z, Davis LS (2013) Learning structured low-rank representations for image classification. In: CVPR. IEEE, pp 676–683

  59. Zhou N, Shen Y, Peng J, Fan J (2012) Learning inter-related visual dictionary for object recognition. In: CVPR. IEEE, pp 3490–3497

Download references

Acknowledgements

Yuping Sun would like to thank the support by Natural Science Foundation of Guangdong Province (Grand No. 2016A030313516). Yuhui Quan would like to thank the support by National Nature Science Foundation of China (Grand No. 61602184). Jia Fu would like to thank the support by National Social Science Foundation of China (Grand No. 16BXW020), Fundamental Research Funds for the Central Universities (Grand No. 2014XM520), and Philosophy and Social Science Research of Guangdong Province (Grand No. GD14XXW03).

Author information

Authors and Affiliations

  1. School of Automation Science and Engineering, South China University of Technology, Guangzhou, 510641, China

    Yuping Sun

  2. School of Computer Science and Engineering, South China University of Technology, Guangzhou, 510006, China

    Yuhui Quan

  3. School of Journalism and Communication, South China University of Technology, Guangzhou, 510006, China

    Jia Fu

Authors
  1. Yuping Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuhui Quan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jia Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jia Fu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, Y., Quan, Y. & Fu, J. Sparse coding and dictionary learning with class-specific group sparsity. Neural Comput & Applic 30, 1265–1275 (2018). https://doi.org/10.1007/s00521-016-2764-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-016-2764-z

Keywords

Search

Navigation