Skip to main content

Incorporating Discrete Wavelet Transformation Decomposition Convolution into Deep Network to Achieve Light Training

  • Conference paper
  • First Online:
Artificial Neural Networks and Machine Learning – ICANN 2021 (ICANN 2021)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12892))

Included in the following conference series:

Abstract

The deep neural network achieves superior performance in various tasks. However, it is notoriously known that its training needs a considerable time cost to refine a large number of parameters. We proposed a deep wavelet network to tackle the issue. The proposed network is built by processing blocks with various decomposition levels which named Discrete Wavelet Transformation Decomposition Convolution (DWTDC). The DWTDC aims to fulfill the task of feature map discrete wavelet transformation decomposition and subbands differential fusion. We employ the DWTDC block to act as the convolution layer so that the parameters are estimated within the frequency domain space. Because of the merits of economic representation in the wavelet domain, the training parameters are greatly reduced, only requiring \(33\%\) of the parameters of the popular networks. Extensive experiments by comparing with benchmark models show that the proposed DWTDC dramatically reduced the number of parameters and achieved light training without sacrificing the classification performance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Candès, E.J., Recht, B.: Exact matrix completion via convex optimization. Found. Comput. Math. 9(6), 717 (2009)

    Article  MathSciNet  Google Scholar 

  2. Chen, W., Wilson, J., Tyree, S., Weinberger, K., Chen, Y.: Compressing neural networks with the hashing trick. In: International Conference on Machine Learning, pp. 2285–2294 (2015)

    Google Scholar 

  3. Chollet, F.: Xception: deep learning with depthwise separable convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1251–1258 (2017)

    Google Scholar 

  4. Cotter, F., Kingsbury, N.: Deep learning in the wavelet domain. arXiv preprint arXiv:1811.06115 (2018)

  5. Courbariaux, M., Hubara, I., Soudry, D., El-Yaniv, R., Bengio, Y.: Binarized neural networks: training deep neural networks with weights and activations constrained to +1 or \(-\)1. arXiv preprint arXiv:1602.02830 (2016)

  6. Daubechies, I.: The wavelet transform, time-frequency localization and signal analysis. IEEE Trans. Inform. Theory 36(5), 961–1005 (1990)

    Article  MathSciNet  Google Scholar 

  7. Denton, E.L., Zaremba, W., Bruna, J., LeCun, Y., Fergus, R.: Exploiting linear structure within convolutional networks for efficient evaluation. In: Advances in Neural Information Processing Systems, pp. 1269–1277 (2014)

    Google Scholar 

  8. Gong, Y., Liu, L., Yang, M., Bourdev, L.: Compressing deep convolutional networks using vector quantization. arXiv preprint arXiv:1412.6115 (2014)

  9. Kim, Y.D., Park, E., Yoo, S., Choi, T., Yang, L., Shin, D.: Compression of deep convolutional neural networks for fast and low power mobile applications. arXiv preprint arXiv:1511.06530 (2015)

  10. Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images. Tech. rep. Citeseer (2009)

    Google Scholar 

  11. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)

    Article  Google Scholar 

  12. Mahanta, D.D.D.L.B.: Childhood medulloblastoma microscopic images (IEEE Dataport (2020). https://dxdoi.org/1021227/w0m0-mw21. https://doi.org/10.21227/w0m0-mw21

  13. Mallat, S.: A Wavelet Tour of Signal Processing. Elsevier (1999)

    Google Scholar 

  14. Nene, S.A., Nayar, S.K., Murase, H.: Columbia object image library (coil-20. Tech. rep. (1996)

    Google Scholar 

  15. Nene, S.A., Nayar, S.K., Murase, H.: object image library (coil-100). Tech. rep. (1996)

    Google Scholar 

  16. Rastegari, M., Ordonez, V., Redmon, J., Farhadi, A.: XNOR-Net: ImageNet classification using binary convolutional neural networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 525–542. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_32

    Chapter  Google Scholar 

  17. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Mobilenetv 2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510–4520 (2018)

    Google Scholar 

  18. Shrestha, A., Mahmood, A.: Review of deep learning algorithms and architectures. IEEE Access 7, 53040–53065 (2019)

    Article  Google Scholar 

  19. Subhashini, A., Victor, S.: A new approach on denoising for 1D, 2D and 3D images based on discrete wavelet transformation and thresholding. Int. J. Adv. Res. Comp. Sci. 9(1), 708–710 (2018)

    Article  Google Scholar 

  20. Wen, W., Wu, C., Wang, Y., Chen, Y., Li, H.: Learning structured sparsity in deep neural networks. arXiv preprint arXiv:1608.03665 (2016)

  21. Yang, L., Yang, P., Ni, R., Zhao, Y.: Xception-based general forensic method on small-size images. In: Pan, J.-S., Li, J., Tsai, P.-W., Jain, L.C. (eds.) Advances in Intelligent Information Hiding and Multimedia Signal Processing. SIST, vol. 157, pp. 361–369. Springer, Singapore (2020). https://doi.org/10.1007/978-981-13-9710-3_38

    Chapter  Google Scholar 

  22. Zhang, Y.: DNN classifier of wide-angle retinal images in computer-aided screening for ROP (IEEE Dataport 2018). https://dxdoi.org/1021227/q5jw-t682. https://doi.org/10.21227/q5jw-t682

  23. Zhao, J., Zhang, Y., He, X., Xie, P.: COVID-CT-dataset: a CT scan dataset about COVID-19. arXiv preprint arXiv:2003.13865 (2020)

Download references

Acknowledgment

This work was partially supported by the National Natural Science Foundation of China (61771007), Key-Area Research and Development of Guangdong Province (2020B010166002, 2020B111119001), Science and Technology Planning Project of Guangdong Province (2017B020226004), and the Health & Medical Collaborative Innovation Project of Guangzhou City (202002020049).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Hongmin Cai .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Tao, G., Rong, W., Weng, W., Dan, T., Zhang, B., Cai, H. (2021). Incorporating Discrete Wavelet Transformation Decomposition Convolution into Deep Network to Achieve Light Training. In: Farkaš, I., Masulli, P., Otte, S., Wermter, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2021. ICANN 2021. Lecture Notes in Computer Science(), vol 12892. Springer, Cham. https://doi.org/10.1007/978-3-030-86340-1_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-86340-1_25

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-86339-5

  • Online ISBN: 978-3-030-86340-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics