Skip to main content
SpringerLink
Log in

MLK-SVD, the new approach in deep dictionary learning

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

The aim of this study is to improve the classification efficiency of advanced methods using a multilayered dictionary learning framework. This paper presents the new idea of “multilayered K-singular value decomposition (MLK-SVD)” dictionary learning as a multilayer method of classification. This method starts by building a sparse representation at the patch level and relies on a hierarchy of learned dictionaries to output a global sparse representation for the whole image. In this research using class labels of training data, the label information is associated with each dictionary item (columns of the dictionary matrix) to enforce discrimination in sparse codes during the dictionary learning process. Also, this algorithm instead of learning one shallow dictionary learned multiple levels of dictionaries. The proposed formulation of deep dictionary learning provides the basis to develop more efficient dictionary learning algorithms. It relies on a succession of sparse coding and pooling steps in order to find an efficient representation of the data for classification. The performance of the proposed method is evaluated on MNIST and CIFAR-10 datasets, and results show that this method can help in advancing the state of the art.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Olshausen, B., Field, D.J.: Sparse coding with an overcomplete basis set: a strategy employed by V1. Vis. Res. 37(23), 33113325 (1997)

    Article  Google Scholar 

  2. Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401, 788791 (1999)

    Article  Google Scholar 

  3. Liu, Y., Yu, D., Chen, X., Li, Z., Fan, J.: TOP-SIFT: the selected SIFT descriptor based on dictionary learning. The Visual Computer 35(5), 667–677 (2019)

    Article  Google Scholar 

  4. Zheng, H., Zhu, J., Yang, Z., Jin, Z.: Effective micro-expression recognition using relaxed K-SVD algorithm. Int. J. Mach. Learn. 8(6), 2043–2049 (2017)

    Article  Google Scholar 

  5. Schnass, K.: Convergence radius and sample complexity of ITKM algorithms for dictionary learning. Appl. Comput. Harmonic Anal. 45(1), 22–58 (2018)

    Article  MathSciNet  Google Scholar 

  6. Mitro, J., Bridge, D., Prestwich, S.: Denoising Dictionary Learning against Adversarial Perturbations. arXiv:1801.02257 (2018)

  7. Ataee, Z., Mohseni, H.: Structured dictionary learning using mixed-norms and group-sparsity constraint. Visual Comput. (2019)

  8. Wen, B., Ravishankar, S., Bresler, Y.: High-dimensional sparsifying transform learning for online video denoising. IEEE Trans. Image Process (2019)

  9. Naderahmadian, Y., Beheshti, S.: Generalized adaptive weighted recursive least squares dictionary learning for retinal vessel inpainting. IEEE Statistical Signal Processing Workshop (SSP) (2018)

  10. Son, C.H., Choo, H.: Local learned dictionaries optimized to edge orientation for inverse halftoning. IEEE Trans. Image Process. 23(6), 25422556 (2014)

    MathSciNet  MATH  Google Scholar 

  11. Caballero, J., Price, A.N., Rueckert, D., Hajnal, J.V.: Dictionary learning and time sparsity for dynamic mr data reconstruction. IEEE Trans. Med. Imag. 33(4), 979994 (2014)

    Article  Google Scholar 

  12. Zayyani, H., Korki, M., Marvasti, F.: Dictionary learning for blind one bit compressed sensing. IEEE Signal Process. Lett. (2016)

  13. Pan, M., Guan, F., Hu, H., Yang,Y: Compressed sensing based on dictionary learning for reconstructing blade tip timing signals, Prognostics and System Health Management (2017)

  14. Huang, Y., De Bortoli, V., Zhou, F., Gilles, J.: Review of wavelet-based unsupervised texture segmentation, advantage of adaptive wavelets. IET Image Process. 12(9), 1626–1638 (2018)

    Article  Google Scholar 

  15. Saha, M., Naskar, M.K., Chatterji, B.N.: Wavelet and curvelet transforms for biomedical image processing. Biomedical Signal Processing (2018)

  16. Hagargi, P.A., Shubhangi, D.: Brain tumor MR image fusion using most dominant features extraction from wavelet and curvelet transforms - Brain (2018)

  17. Zhang, Z., Jiang, W., Qin, J., Zhang, L., Li, F.: Jointly learning structured analysis discriminative dictionary and analysis multiclass classifier. IEEE Trans. Neural Netw. Learn. Syst. 29(8), 3798–3814 (2018)

    Article  MathSciNet  Google Scholar 

  18. Wang, H., Wang, L.: Cross-agent action recognition. IEEE Trans. Circuits Syst. 28(10), 2908–2919 (2018)

    Google Scholar 

  19. Jin, W., Wang, L., Zeng, X., Liu, Z., Fu, R.: Classication of clouds in satellite imagery using over-complete dictionary via sparse representation. Pattern Recognit. Lett. 49(1), 193200 (2014)

    Google Scholar 

  20. Li, Z.Q., Sun, J., Wu, X.J., Yin, H.F.: Learning efficient structured dictionary for image classification. arXiv:2002.03271 (2020)

  21. Han, Z., Liu, Z., Vong, C.M., Liu, Y.S., Bu, S.: Deep spatiality: unsupervised learning of spatially-enhanced global and local 3D features by deep neural network with coupled softmax. Image Process. 27(6), 3049–3063 (2018)

    Article  MathSciNet  Google Scholar 

  22. Khan, N., Tappen, M.F.: Stable discriminative dictionary learning via discriminative deviation. In: Proc. Int. Conf. Pattern Recognit (2012)

  23. Wang, R., Shen, M., Li, Y., Gomes, S.: Multi-task joint sparse representation classification based on fisher discrimination dictionary learning. Comput. Mater. Contin 57(1), 25–48 (2018)

    Article  Google Scholar 

  24. Zhang, Z., Zhang, L., Tan, Y., Zhang, L.: Joint discriminative dictionary and classifier learning for ALS point cloud classification. IEEE Trans. Geosci. Remote Sens. 56(1), 524–538 (2018)

    Article  Google Scholar 

  25. Quan, Y., Xu, Y., Sun, Y., Huang, Y., Ji, H.: Sparse coding for classification via discrimination ensemble. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

  26. Gu, B., Quan, X., Gu, Y., Sheng, V.S., Zheng, G.: Chunk incremental learning for cost-sensitive hinge loss support vector machine. Pattern Recognit. 83, 196–208 (2018)

    Article  Google Scholar 

  27. Zhou, P., Fang, C., Lin, Z.: https://scholar.google.com/citations?user=NeCCx-kAAAAJ&hl=en&oi=sra, Chang. EY, Dictionary learning with structured noise, Neurocomputing (2018)

  28. Zhang, Q., Li, B.: Discriminative K-SVD for dictionary learning in face recognition. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit (2010)

  29. Jiang, Z., Lin, Z., Davis, L.S.: Label consistent K-SVD: learning a discriminative dictionary for recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(11), 26512664 (2013)

    Google Scholar 

  30. Pacheco, A.G.C., Krohling, R.A., Silva, C.A.S.: Restricted Boltzmann machine to determine the input weights for extreme learning machines. Expert Syst. Appl. 96, 77–85 (2018)

    Article  Google Scholar 

  31. Bourlard, H., Kamp, Y.: Auto-association by multilayer perceptrons and singular value decomposition. Biol. Cybern. 59(45), 291294 (1998)

    MathSciNet  MATH  Google Scholar 

  32. Tang, X., Dai, Y., Liu, Q., Dang, X., Xu, J.: Application of bidirectional recurrent neural network combined with deep belief network in short-term load forecasting, IEEE Access. ieeexplore.ieee.org (2019)

  33. Li, H., Xu, Q., He, Y., Fan, X., Li, S.: Modeling and predicting reservoir landslide displacement with deep belief network and EWMA control charts: a case study in Three Gorges Reservoir, Landslides. Springer, Berlin (2020)

    Google Scholar 

  34. Zhu, X., Chen, Z.: Dual-modality spatiotemporal feature learning for spontaneous facial expression recognition in e-learning using hybrid deep neural network. Visual Comput. (2019)

  35. Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.A.: Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11(12), 33713408 (2010)

    MathSciNet  MATH  Google Scholar 

  36. Sulam, J., Papyan, V., Romano, Y., Elad, M.: Multi-layer convolutional sparse modeling: pursuit and dictionary learning. arXiv:1708.08705V2[cs.CV] (2018)

  37. Chan Wai Tim, S., Rombaut, M., Pellerin, D.: Multi-layer dictionary learning for image classification. In: International Conference on Advanced Concepts for Intelligent Vision Systems (ACIVS) (2016)

  38. Yankelevsky Y, et al.: Structure-aware classification using supervised dictionary learning. Computer Vision and Pattern Recognition (2016)

  39. Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images, Tech. rep, University of Toronto, (2009)

  40. Tariyal, S., Majumdar, A., Singh, R., Vatsa, M.: Deep dictionary learning. Digital Object Identifier (2016). https://doi.org/10.1109/ACCESS.2016.2611583. IEEE Access

    Article  Google Scholar 

  41. Mahdizadehaghdam, S., Panahi, A., et al.: Deep dictionary learning: a parametric network approach. IEEE Transactions on Image Processing. ieeexplore.ieee.org (2019)

Download references

Author information

Authors and Affiliations

  1. Department of Information Technology, Faculty of Computer, University of Qom, Qom, Iran

    Azadeh Montazeri & Rouhollah Dianat

  2. Department of Computer, Faculty of Computer and Electrical, Qom University of Technology, Qom, Iran

    Mahboubeh Shamsi

Authors
  1. Azadeh Montazeri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mahboubeh Shamsi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rouhollah Dianat
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Azadeh Montazeri.

Ethics declarations

Conflict of interest

Author Azadeh Montazeri declares that she has no conflict of interest. Author Mahboubeh Shamsi declares that she has no conflict of interest. Author Rouhollah Dianat declares that he has no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Montazeri, A., Shamsi, M. & Dianat, R. MLK-SVD, the new approach in deep dictionary learning. Vis Comput 37, 707–715 (2021). https://doi.org/10.1007/s00371-020-01970-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-020-01970-x

Keywords

Search

Navigation