Skip to main content
SpringerLink
Log in

Robust and fast low-rank deep convolutional feature recovery: toward information retention and accelerated convergence

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

Notwithstanding the great progress on deep convolutional neural networks (CNNs) has been made during last decade, the representation ability may still be restricted and it usually needs more epochs to converge in training, due to the information loss caused by the up-/down-sampling operations. In this paper, we propose a general deep feature recovery layer, termed Low-rank Deep Feature Recovery (LDFR), to enhance the representation of convolutional features by seamlessly integrating the low-rank recovery into CNNs, which can be easily extended to all CNNs-based models. To be specific, to recover the lost useful information, LDFR learns the low-rank projections to embed feature maps onto a low-rank subspace based on the selected informative convolutional feature maps. Such operation can ensure all the convolutional feature maps to be reconstructed easily to recover the underlying subspace, with more useful detailed information discovered, e.g., the strokes of characters or the texture information of clothes. To make the learnt low-rank subspaces more powerful for feature recovery, we design a fusion strategy to obtain a generalized subspace, which averages over all learnt subspaces in each LDFR layer, so that the convolutional features in test phase can be recovered effectively via low-rank embedding. We also present a fast version of LDFR, called FLDFR, to speedup the optimization of LDFR by flattening all feature maps of batch images to recover the lost information. Extensive simulations on several image datasets show that the existing CNN models equipped with our LDFR layers can obtain better performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22

Similar content being viewed by others

References

  1. Wright J, Yang AY, Ganesh A, Sastry S, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31(2):210–227

    Article  Google Scholar 

  2. Bengio Y, Courville A, Vincent P (2013) Representation learning: a review and new perspectives. IEEE Trans Pattern Anal Mach Intell 35(8):1798–1828

    Article  Google Scholar 

  3. Zhang Y, Zhang Z, Wang Y, Zhang Z, Zhang L, Yan S, Wang M (Nov2021) Dual-constrained deep semi-supervised coupled factorization network with enriched prior. Int J Computer Vision (IJCV) 129(12):3233–3254

    Article  Google Scholar 

  4. Zhang Z, Jiang W, Qin J, Zhang L, Li F, Zhang M, Yan S (2018) Jointly learning structured analysis discriminative dictionary and analysis multiclass classifier. IEEE Trans Neural Netw Learn Syst 29(8):3798–3814

    Article  MathSciNet  Google Scholar 

  5. Liu G, Lin Z, Yan SC (2013) Robust recovery of subspace structures by low-rank representation. IEEE Trans Pattern Anal Mach Intell 35(1):171–184

    Article  Google Scholar 

  6. Candès EJ, Li X, Ma Y (2011) Robust principal component analysis? J ACM 58(3):1–11

    Article  MathSciNet  MATH  Google Scholar 

  7. Zhang Z, Li F, Zhao M, Zhang L, Yan S (2016) Joint low-rank and sparse principal feature coding for enhanced robust representation and visual classification. IEEE Trans Image Process 25(6):2429–2443

    Article  MathSciNet  MATH  Google Scholar 

  8. Zhang Y, Jiang ZL, and Davis L (2013) Learning structured low-rank representations for image classification,” In Proceedings of the IEEE Conf. on Computer Vision and Pattern Recognition, Portland

  9. Hull J (1994) A database for handwritten text recognition research. IEEE Trans Pattern Anal Mach Intell 16(5):550–554

    Article  Google Scholar 

  10. Jiang S, Ding Z and Fu Y (2017) “Deep low-rank sparse collective factorization for cross-domain recommendation,” In Proceedings of ACM Conference on Multimedia, pp.163–171

  11. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  12. Krizhevsky A, Sutskever I, Hinton G (2012) imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst (NIPS) 25:1097–1105

    Google Scholar 

  13. Simonyan K, Zisserman A (2014) “Very Deep Convolutional Networks for Large-Scale Image Recognition,” arXiv preprint, arXiv: 1409. 1556

  14. He K, Zhang X, Ren S and Sun J (2016) “Deep residual learning for image recognition,” In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pp.770–778

  15. Gao R, Hou X, Qin J, Shen Y, Long Y, Liu L, Zhang Z, Shao L (2022) Visual-semantic aligned bidirectional network for zero-shot learning. IEEE Trans Multim 2:87

    Google Scholar 

  16. Lin Z, Chen M, Wu L,Ma Y (2009) “The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices,” Tech Tep UILU-ENG-09–2215

  17. Nie F, Huang H, Cai X, and Ding C (2010) “Efficient and robust feature selection via joint l21-norms minimization,” in: advances in neural information processing systems, Vancouver, British Columbia, Canada, pp.1813–1821

  18. Zhang Z, Yan SC, Zhao MB (2014) Similarity preserving low-rank representation for enhanced data representation and effective subspace learning. Neural Netw 53:81–94

    Article  MATH  Google Scholar 

  19. Bengio Y, Lamblin P, Popovici D, Larochelle H (2007) “Greedy layer- wise training of deep networks. Adv Neural Inf Process Syst 19:153

    Google Scholar 

  20. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D and Rabinovich A (2005) “Going deeper with convolutions,” In Proceedings of IEEE Conference on Computer Vision and pattern Recognition, pp.1–9

  21. Huang G,. Liu Z, Van Der Maaten L, and Weinberger K (2017) “Densely connected convolutional networks,” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.4700- 4708

  22. Jia Y, Zhang H, Zhang Z, Liu M (2021) CNN-based encoder-decoder networks for salient object detection: a comprehensive review and recent advances. Inf Sci 546:835–857

    Article  MathSciNet  Google Scholar 

  23. Xiao H, Rasul K, and Vollgraf R (2017) “Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,” arXiv:1708. 07747v2,

  24. Nene SA , Nayar SK and Murase H (1996) “Columbia Object Image Library (COIL-100)”, Technical Report CUCS-006–96

  25. Krawetz N (2017) Kind of Like That. http://www.hackerfactor.com/ blog/?/archives/529-Kind-of-Like-That.html. Accessed 12 Jan

  26. Wang Z, Bovik AC, Sheikh HR, Simoncelli P (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  27. Fan J, Ding L, Yang C, Udell M, Zhang Z (2022) "Euclidean-norm- induced schatten-p quasi-norm regularization for low-rank tensor completion and tensor robust principal component analysis. Trans Mach Learn Res 25:5

    Google Scholar 

  28. Fan J, Chow TWS (2020) Exactly robust kernel principal component analysis. IEEE Trans Neural Netw Learn Syst 31(3):749–761

    Article  MathSciNet  Google Scholar 

  29. Li Z, Liu J, Jiang Y, Tang J, and Lu H (2012)“Low rank metric learning for social image retrieval,” In: Proceedings of ACM International Conference on Multimedia, pp.853–856

  30. Ota K, Dao M, Mezaris V, Natale FGB (2017) Deep learning for mobile multimedia: a survey. ACM Trans Multim Comput Commun Appl 13(3):34–22

    Google Scholar 

  31. Lan X, Ye M, Shao R, Zhong B, Yuen PC, Zhou H (2019) Learning modality-consistency feature templates: A robust RGB-infrared tracking system. IEEE Trans Industrial Electron 66(12):9887–9897

    Article  Google Scholar 

  32. Baldwin RW, Almatrafi M, Asari VK, and Hirakawa K (2020) “Event probability mask (EPM) and event denoising convolutional neural network (EDnCNN) for neuromorphic cameras,” IEEE Conference on Computer Vision and Pattern Recognition, pp.1698–1707

  33. Mok T, Chung A (2020) “Fast symmetric diffeomorphic image registration with convolutional neural networks,” IEEE Conference on Computer Vision and Pattern Recognition, pp.4643–4652

  34. Bao BK, Liu GC, Xu C (2012) “Inductive robust principal component analysis. IEEE Trans Image Proc 21(8):3794–3800

    Article  MathSciNet  MATH  Google Scholar 

  35. Liu G, Zhang Z, Liu Q, Xiong H (2019) Robust subspace clustering with compressed data. IEEE Trans Image Process 28(10):5161–5170

    Article  MathSciNet  MATH  Google Scholar 

  36. Yu S, Wu Y (2018) Subspace clustering based on latent low rank representation with Frobenius norm minimization. Neurocomputing 275:2479–2489

    Article  Google Scholar 

  37. Ren J, Zhang Z, Fan J, Zhang H, Xu M and Wang M (2021)" Robust Low-rank Deep Feature Recovery in CNNs: Toward Low Information Loss and Fast Convergence," In: Proceedings of the 21th IEEE International Conference on Data Mining (ICDM), Auckland, New Zealand,

  38. Alex K (2009) “Learning multiple layers of features from tiny images

  39. Martinez A and Benavente R (1998) “The AR face database,” CVC Technical Report

  40. Werfel J, Xie X, and Sebastian Seung H (2003) “Learning curves for stochastic gradient descent in linear feedforward networks,” In: Proceedings of the Annual Conference on Neural Information Processing Systems, pp.1197–1204

  41. Zhu G, Zeng X, Jin X, Zhang J (2021) Metro passengers counting and density estimation via dilated-transposed fully convolutional neural network. Knowl Inf Syst 63(6):1557–1575

    Article  Google Scholar 

  42. Zhang S, Zhang W, Niu J (2019) Improving short-text representation in convolutional networks by dependency parsing. Knowl Inf Syst 61(1):463–484

    Article  Google Scholar 

  43. Du M, Liu N, Yang F, Hu X (2021) Learning credible DNNs via incorporating prior knowledge and model local explanation. Knowl Inf Syst 63(2):305–332

    Article  Google Scholar 

Download references

Acknowledgements

The work described in this paper is partially supported by National Natural Science Foundation of China (62072151), and Anhui Provincial Natural Science Fund for the Distinguished Young Scholars (2008085J30). Zhao Zhang is the corresponding author of this paper. Jicong Fan is the co-corresponding author of this paper.

Author information

Authors and Affiliations

  1. School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, China

    Jiahuan Ren, Zhao Zhang & Meng Wang

  2. Key Laboratory of Knowledge Engineering With Big Data (Ministry of Education) and Intelligent Interconnected Systems Laboratory of Anhui Province, Hefei University of Technology, Hefei, China

    Jiahuan Ren, Zhao Zhang & Meng Wang

  3. School of Data Science, The Chinese University of Hong Kong Shenzhen, Shenzhen, China

    Jicong Fan

  4. Department of Computer Science, Harbin Institute of Technology, Shenzhen, China

    Haijun Zhang

  5. School of Information Engineering, Zhengzhou University, Zhengzhou, China

    Mingliang Xu

Authors
  1. Jiahuan Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jicong Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Haijun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mingliang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Meng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Zhao Zhang or Jicong Fan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ren, J., Zhang, Z., Fan, J. et al. Robust and fast low-rank deep convolutional feature recovery: toward information retention and accelerated convergence. Knowl Inf Syst 65, 1287–1315 (2023). https://doi.org/10.1007/s10115-022-01795-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-022-01795-1

Keywords

Search

Navigation