Skip to main content
SpringerLink
Log in

Development of a novel activation function based on Chebyshev polynomials: an aid for classification and denoising of images

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

The main objective of this paper is to improve the efficiency and accuracy of convolutional neural network models for image classification and denoising tasks. The focus of the study is on enhancing the activation layer of these models, which is a critical component that determines the output of each neuron in the network. To achieve this goal, we propose a novel activation function based on Chebyshev polynomials, which is both data-driven and self-learnable. In addition to proposing the LIP model, the authors investigate its performance in approximating various nonlinearities and determine its Lipschitz bound. The study then evaluates the performance of the proposed activation function by conducting experiments on multiple datasets using different convolutional neural network models. The results show that the proposed activation function outperforms other activation layers and significantly enhances the accuracy of image classification and denoising tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

Datasets analyzed during this study are included in the manuscript references section. Requests for any material regarding this manuscript should be made to the corresponding author.

References

  1. Ranzato MA, Boureau Y-L, Cun YL (2007) Sparse feature learning for deep belief network. Advances in neural information processing systems, vol 20. Curran Associates Inc, New York, pp 1185–1192

    Google Scholar 

  2. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  3. Li X, Sun Y (2021) Application of RBF neural network optimal segmentation algorithm in credit rating. Neural Comput Appl 33:8227–8235

    Article  Google Scholar 

  4. Zong C, Wang H et al (2022) An improved 3D point cloud instance segmentation method for overhead catenary height detection. Comput Electr Eng 98:107685

    Article  Google Scholar 

  5. Agarap AF (2018) Deep learning using rectified linear units (relu). arXiv preprint arXiv:1803.08375

  6. Van Amersfoort J et al. (2020) Simple and scalable epistemic uncertainty estimation using a single deep deterministic neural network. arXiv preprint arXiv:2003.02037

  7. Huang CQ, Jiang F et al. (2022) Dual-graph attention convolution network for 3-D point cloud classification. In: IEEE Transactions on Neural Networks and Learning Systems, pp 1–13

  8. Bank D, Koenigstein N, Giryes R (2020) Autoencoders. arXiv preprint arXiv:2003.05991

  9. Vincent P et al (2010) Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J Mach Learn Res 11(12):3371–3408

    MathSciNet  MATH  Google Scholar 

  10. Wang Y, Yao H, Zhao S (2015) Auto-encoder based dimensionality reduction. Neurocomputing 184:232–242

    Article  Google Scholar 

  11. Dugas C, Bengio Y, Bélisle F, Nadeau C, Garcia R (2001) Incorporating second-order functional knowledge for better option pricing. Advances in neural information processing systems. MIT Press, Cambridge, pp 472–478

    MATH  Google Scholar 

  12. Clevert D-A, Unterthiner T, Hochreiter S (2015) Fast and accurate deep network learning by exponential linear units (elus). arXiv preprint arXiv:1511.07289

  13. Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural networks. In: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pp 315–323

  14. Maas AL, Hannun AY, Ng AY (2013) Rectifier nonlinearities improve neural network acoustic models. In Proc. ICML 30(1):3

    Google Scholar 

  15. Ramachandran P, Zoph B, Le QV (2017) Searching for activation functions. arXiv preprint arXiv:1710.05941

  16. Misra D (2019) A self regularized non-monotonic neural activation function. arXiv preprint arXiv:1908.08681, vol 4, no 2, pp 10–48550

  17. Agostinelli F, Hoffman M, Sadowski P, Baldi P (2014) Learning activation functions to improve deep neural networks. arXiv preprint arXiv:1412.6830

  18. Carini A, Sicuranza GL (2013) Even mirror Fourier nonlinear filters. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, pp 5608–5612

  19. Nocedal J, Wright S (2006) Numerical optimization. Springer Science and Business Media, Berlin

    MATH  Google Scholar 

  20. Goyal M, Goyal R, Reddy PV, Lall B (2020) Activation functions. Deep learning: algorithms and applications. Springer, New york, pp 1–30

    Google Scholar 

  21. Venkatappareddy P et al (2021) A legendre polynomial based activation function: an aid for modeling of max pooling. Digit Signal Process 115:103093

    Article  Google Scholar 

  22. Carini A, Sicuranza GL (2016) A study about Chebyshev nonlinear filters. Signal Process 122:24–32

    Article  Google Scholar 

  23. Rudin W et al (1964) Principles of mathematical analysis, vol 3. McGraw-hill, New York

    MATH  Google Scholar 

  24. Agarap AF (2018) Deep learning using rectified linear units (relu). arXiv preprint arXiv:1803.08375

  25. https://www.yann.lecun.com/exdb/mnist/

  26. https://www.kaggle.com/zalando-research/fashionmnist/./

  27. https://www.github.com/meghanabhange/FashionMNIST-3-Layer-CNN

  28. https://www.github.com/umbertogriffo/Fashion-mnist-cnn-keras

  29. Byerly A et al (2020) A branching and merging convolutional network with homogeneous filter capsules. arXiv preprint arXiv:2001.09136

  30. Tanveer MS et al (2020) Fine-tuning DARTS for image classification. arXiv preprint arXiv:2006.09042

  31. Zagoruyko S et al (2016) Wide residual networks. preprint, arXiv:1605.07146

  32. Sun Z, Li Y, Li Y, Jiang T, Sun W (2021) Steady-state mean-square error analysis for non-negative least lncosh algorithm. IEEE transactions on circuits and systems II: express briefs

  33. Goyal B, Agrawal S, Sohi BS (2018) Noise issues prevailing in various types of medical images. Biomed Pharmacol J 11(3):1227

    Article  Google Scholar 

  34. Sadri AR, Zekri M, Sadri S, Gheissari N (2012) Impulse noise cancellation of medical images using wavelet networks and median filters. J Med Signals Sensors 2(1):25

    Article  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

Not applicable.

Author information

Authors and Affiliations

  1. Department of Electronics and Communication Engineering, Vignan’s Foundation for Science Technology and Research, Vadlamudi, Guntur, Andhra Pradesh, India

    M. Deepthi & G. N. V. R. Vikram

  2. Image Processing Division, Azista Industries Private Limited, Hyderabad, Telangana, India

    P. Venkatappareddy

Authors
  1. M. Deepthi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. G. N. V. R. Vikram
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. P. Venkatappareddy
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

MD and GNVRV prepared manuscript and PV reviewed the manuscript.

Corresponding author

Correspondence to M. Deepthi.

Ethics declarations

Ethical approval

Not applicable.

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Deepthi, M., Vikram, G.N.V.R. & Venkatappareddy, P. Development of a novel activation function based on Chebyshev polynomials: an aid for classification and denoising of images. J Supercomput 79, 20515–20531 (2023). https://doi.org/10.1007/s11227-023-05466-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-023-05466-y

Keywords

Search

Navigation