Skip to main content
Springer Nature Link
Log in

LSTM-based adaptive whale optimization model for classification of fused multimodality medical image

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Multimodality medical image fusion is the important area in the medical imaging field which enhances the reliability of medical diagnosis. Medical image fusion as well as their classification is employed to achieve significant multimodality of medical image data. The single modality image does not provide the adequate information needed for an accurate diagnosis. An adaptive whale optimization algorithm (AWOA) with long short-term memory (LSTM) based efficient multimodal medical image fusion classification is proposed to enhance diagnostic accuracy. To obtain the fused images, discrete wavelet transform with an arithmetic optimization algorithm is used for the fusion process by taking the multimodal medical images. In this AWOA algorithm, the classification accuracy is enhanced, and also the weight of the LSTM is optimized. The three dataset images used in evaluating the experimental set with the representation of several diseases like mild Alzheimer’s encephalopathy, hypertensive encephalopathy and glioma to validate the proposed method are demonstrated. The classification accuracy obtained for each respective dataset is 98.25%, 98.54% and 98.75%. The proposed classifier has achieved better accuracy as compared to other classifiers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

Data sharing is not applicable to this article as no new data were created or analyzed in this study.

References

  1. Zhang, Y., Sidibé, D., Morel, O., Mériaudeau, F.: Deep multimodal fusion for semantic image segmentation: a survey. Image Vis. Comput. 104042, 1 (2020)

    Google Scholar 

  2. Kaur, M., Singh, D.: Multi-modality medical image fusion technique using multi-objective differential evolution based deep neural networks. J. Ambient Intell. Hum. Comput. 1, 1 (2021)

    Google Scholar 

  3. Velmurugan, S.P., Sivakumar, P., Rajasekaran, M.P.: Multimodality image fusion using center-based genetic algorithm and fuzzy logic. Int. J. Biomed. Eng. Technol. 28(4), 322–348 (2018)

    Article  Google Scholar 

  4. Rajalingam, B., Priya, R.: Multimodal medical image fusion using various hybrid fusion techniques for clinical treatment analysis. Smart Construct. Res. 2(2), 1–20 (2018)

    Google Scholar 

  5. Shahdoosti, H.R., Mehrabi, A.: Multimodal image fusion using sparse representation classification in tetrolet domain. Digital Signal Process. 79, 9–22 (2018)

    Article  MathSciNet  Google Scholar 

  6. Guo, Z., Li, X., Huang, H., Guo, N., Li, Q.: Deep learning-based image segmentation on multimodal medical imaging. IEEE Trans. Radiat. Plasma Med. Sci. 3(2), 162–169 (2019)

    Article  Google Scholar 

  7. Rajalingam, B., Priya, R.: Multimodal medical image fusion based on deep learning neural network for clinical treatment analysis”. Int. J. ChemTech. Res. 11(06), 160–176 (2018)

    Google Scholar 

  8. Yousif, A., Omar, Z.B., Sheikh, U.U.: A Survey on multi-scale medical images fusion techniques: brain diseases. J. Biomed. Eng. Med. Imaging 7(1), 18–38 (2020)

    Article  Google Scholar 

  9. Yadav, S.P., Yadav, S.: Image fusion using hybrid methods in multimodality medical images. Med. Biol. Eng. Comput. 58(4), 669–687 (2020)

    Article  Google Scholar 

  10. Ganasala, P., Kumar, V.: Feature-motivated simplified adaptive PCNN-based medical image fusion algorithm in NSST domain. J. Digit. Imag. 29(1), 73–85 (2016)

    Article  Google Scholar 

  11. Achanta, S.D.M., Karthikeyan, T., Vinothkanna, R.: A novel hidden Markov model-based adaptive dynamic time warping (HMDTW) gait analysis for identifying physically challenged persons. Soft. Comput. 23(18), 8359–8366 (2019)

    Article  Google Scholar 

  12. Achanta, S.D.M., Karthikeyan, T.: A wireless IOT system towards gait detection technique using FSR sensor and wearable IOT devices. Int. J. Intell. Unmanned Syst. (2019)

  13. Sampath Dakshina Murthy, A., Karthikeyan, T., Vinoth Kanna, R.: Gait-based person fall prediction using deep learning approach. Soft Comput. 1, 1–9 (2021)

    Google Scholar 

  14. Achanta, S.D.M., Karthikeyan, T., Kanna, R.V.: Wearable sensor based acoustic gait analysis using phase transition-based optimization algorithm on IoT. Int. J. Speech Technol. 1, 1–11 (2021)

    Google Scholar 

  15. Ganasala, P., Kumar, V., Prasad, A.D.: Performance evaluation of color models in the fusion of functional and anatomical images. J. Med. Syst. 40(5), 122 (2016)

    Article  Google Scholar 

  16. Ghimpet, G., Eanu, T., Batard, M., Bertalmío, et al.: A decomposition framework for image denoising algorithms. IEEE Trans. Image Process. 25(1), 388–399 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  17. Bhardwaj J, Nayak A, Gambhir D (2021) Multimodal medical image fusion based on discrete wavelet transform and genetic algorithm. In: International Conference on Innovative Computing and Communications, pp. 1047–1057. Springer

  18. Wang, K., Zheng, M., Wei, H., Qi, G., Li, Y.: Multi-modality medical image fusion using convolutional neural network and contrast pyramid. Sensors 20(8), 2169 (2020)

    Article  Google Scholar 

  19. Arif, M., Wang, G.: Fast curvelet transform through genetic algorithm for multimodal medical image fusion. Soft. Comput. 24(3), 1815–1836 (2020)

    Article  Google Scholar 

  20. Subbiah Parvathy, V., Pothiraj, S., Sampson, J.: A novel approach in multimodality medical image fusion using optimal shearlet and deep learning. Int. J. Imaging Syst. Technol. 30(4), 847–859 (2020)

    Article  Google Scholar 

  21. Maqsood, S., Javed, U.: Multi-modal medical image fusion based on two-scale image decomposition and sparse representation. Biomed. Signal Process. Control 57, 101810 (2020)

    Article  Google Scholar 

  22. Parvathy, V.S., Pothiraj, S.: Multi-modality medical image fusion using hybridization of binary crow search optimization. Health Care Manag. Sci. 1, 1–9 (2019)

    Google Scholar 

  23. Parvathy, V.S., Pothiraj, S., Sampson, J.: Optimal deep neural network model based multimodality fused medical image classification. Phys. Commun. 41, 101119 (2020)

    Article  Google Scholar 

  24. Abualigah, L., Diabat, A., Mirjalili, S., Abd Elaziz, M., Gandomi, A.H.: The arithmetic optimization algorithm. Comput. Methods Appl. Mech. Eng. 1(376), 113609 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  25. Habib, M.K., Cherri, A.K.: Parallel quaternary signed-digit arithmetic operations: addition, subtraction, multiplication and division. Opt. Laser Technol. 30(8), 515–525 (1998)

    Article  Google Scholar 

  26. Shahid, F., Zameer, A., Muneeb, M.: Predictions for COVID-19 with deep learning models of LSTM GRU and Bi-LSTM. Chaos Solitons Fract. 140, 110212 (2020)

    Article  MathSciNet  Google Scholar 

  27. Yu, R., Gao, J., Yu, M., Lu, W., Xu, T., Zhao, M., Zhang, Z.: LSTM-EFG for wind power forecasting based on sequential correlation features. Fut. Gen. Comput. Syst. 93, 33–42 (2019)

    Article  Google Scholar 

  28. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput 9, 1735–1780 (1997)

    Article  Google Scholar 

  29. Zaremba W, Sutskever I and Vinyals O(2014) Recurrent neural network regularization. arXiv preprint arXiv:1409.2329

  30. Pawar, M., Sale, D., Dypit, P.: MRI and CT image denoising using gaussian filter, wavelet transform and curvelet transform. Int. J. Eng. Sci. Comput. 7(5), 12013–6 (2017)

    Google Scholar 

  31. Malegori, C., Franzetti, L., Guidetti, R., Casiraghi, E., Rossi, R.: GLCM, an image analysis technique for early detection of biofilm. J. Food Eng. 185, 48–55 (2016)

    Article  Google Scholar 

  32. Mirjalili, S., Lewis, A.: The whale optimization algorithm. Adv. Eng. Softw. 95, 51–67 (2016)

    Article  Google Scholar 

  33. Mafarja, M.M., Mirjalili, S.: Hybrid whale optimization algorithm with simulated annealing for feature selection. Neurocomputing 260, 302–312 (2017)

    Article  Google Scholar 

  34. Tu, J., Chen, H., Liu, J., Heidari, A.A., Zhang, X., Wang, M., Ruby, R., Pham, Q.V.: Evolutionary biogeography-based whale optimization methods with communication structure: Towards measuring the balance. Knowl.-Based Syst. 212, 106642 (2021)

    Article  Google Scholar 

  35. Simon, D.: Biogeography-based optimization. IEEE Trans. Evol. Comput. 12(2008), 702–713 (2008)

    Article  Google Scholar 

  36. Johnson KA, Becker JA, The whole brain altas [Online]. http://www.med.harvard.edu/aanlib/.

  37. Tan, W., Tiwari, P., Pandey, H.M., Moreira, C., Jaiswal, A.K.: Multimodal medical image fusion algorithm in the era of big data. Neural Comput. Appl. 1, 1–21 (2020)

    Google Scholar 

  38. Subbiah Parvathy, V., Pothiraj, S., Sampson, J.: A novel approach in multimodality medical image fusion using optimal shearlet and deep learning. Int. J. Imag. Syst. Technol. 30(4), 847–859 (2020)

    Article  Google Scholar 

  39. Naveena, C., Rangappa, S., Chethan, H.K.: Texture features in palmprint recognition system. Int. J. Nat. Comput. Res. (IJNCR) 10(1), 41–57 (2021)

    Article  Google Scholar 

  40. Hinrichs, C., Singh, V., Mukherjee, L., Xu, G., Chung, M.K., Johnson, S.C., Initiative, A.D.N.: Spatially augmented LPboosting for AD classification with evaluations on the ADNI dataset. Neuroimage 48(1), 138–149 (2009)

    Article  Google Scholar 

  41. Ramya, V.J., Lakshmi, S.: Enhanced deep CNN based arithmetic optimization algorithm for acute myelogenous leukemia detection. Ann. Roman. Soc. Cell Biol. 1, 7333–7352 (2021)

    Google Scholar 

  42. Almasri, M.M., Alajlan, A.M.: Artificial intelligence-based multimodal medical image fusion using hybrid S2 optimal CNN. Electronics 11(14), 2124 (2022)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computing Science and Engineering, Galgotias University, Greater Noida, India

    Vipin Rai

  2. School of Engineering and Technology, Sharda University, Greater Noida, India

    Ganesh Gupta

  3. Department of Computer Science and Engineering, GL Bajaj Institute of Technology & Management, Greater Noda, Uttar Pradesh, India

    Shivani Joshi & Rajiv Kumar

  4. Department of Computer Science and Engineering, JIMS Engineering Management Technical Campus Guru Gobind Singh Indraprastha University, New Delhi, India

    Avinash Dwivedi

Authors
  1. Vipin Rai
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Ganesh Gupta
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Shivani Joshi
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Rajiv Kumar
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Avinash Dwivedi
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

VR agreed on the content of the study. VR, GG, SJ, RK and AD collected all the data for analysis. VR agreed on the methodology. VR, GG, SJ, RK and AD completed the analysis based on agreed steps. Results and conclusions are discussed and written together. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Shivani Joshi.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Human and animal rights

This article does not contain any studies with human or animal subjects performed by any of the authors.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rai, V., Gupta, G., Joshi, S. et al. LSTM-based adaptive whale optimization model for classification of fused multimodality medical image. SIViP 17, 2241–2250 (2023). https://doi.org/10.1007/s11760-022-02439-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-022-02439-1

Keywords