Skip to main content
SpringerLink
Log in

RESPNet: resource-efficient and structure-preserving network for deformable image registration

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Different deep learning-based architectures have been developed for medical image registration in the last few years. The architectures of these methods are complex and require a considerable amount of memory. The scalability of the architectures is limited due to their compatibility with low or moderate memory devices. The deformable image registration attained attention in the field of medical image registration. The uprising in deep learning has paved the path for more sophisticated solutions to the many medical imaging problems. We have proposed the resource-efficient and structure-preserving network (RESPNet) to solve medical image registration issues. RESPNet is convolutional neural networks (CNNs)-based architecture to predict the deformation vector field (DVF), which signifies the displacement of each pixel in all directions. We have developed the three CNNs modules (i.e., StraightCNN, UpCNN, and DownCNN) to reduce parameter size and preserve the structure of the images. In this paper, the architecture of RESPNet utilizes the structural properties of images for 2D registration of the retina images and 3D registration of brain magnetic resonance (MR) images. The proposed architecture requires less than 25% memory compared to the current state-of-the-art methods and can be trained in 6-8 hours on a 13 GB GPU to produce the results. The dice score of 2D and 3D images registration is 0.8784 and 0.7515, respectively. The suggested architecture reduces the memory requirements by more than 75% and achieves better performance in dice score, mutual information, and processing time. We have developed a memory-efficient deep learning architecture for medical image registration. This architecture can be employed to register 2D and 3D images efficiently.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data Availability

The datasets used in this work will be available from the corresponding author on reasonable request.

References

  1. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 25, pp. 1097–1105. Curran Associates, Inc.,

  2. Mamoshina P, Vieira A, Putin E, Zhavoronkov A (2016) Applications of deep learning in biomedicine. Mol Pharm 13(5):1445–1454

    Article  Google Scholar 

  3. Shanker R, Bhattacharya M (2022) Classification of brain mr images using modified version of simplified pulse-coupled neural network and linear programming twin support vector machines. J Supercomput 78:13831–13863

    Article  Google Scholar 

  4. Sotiras A, Davatzikos C, Paragios N (2013) Deformable medical image registration: A survey. IEEE Trans Med Imaging 32(7):1153

    Article  Google Scholar 

  5. Cortés X, Serratosa F (2015) An interactive method for the image alignment problem based on partially supervised correspondence. Expert Syst Appl 42(1):179–192

    Article  Google Scholar 

  6. Cao W (2021) Applying image registration algorithm combined with cnn model to video image stitching. J Supercomput 77(12):13879–13896

    Article  Google Scholar 

  7. Wyawahare MV, Patil PM, Abhyankar HK et al (2009) Image registration techniques: an overview. Int J Signal Process, Image Process Pattern Recognition 2(3):11–28

    Google Scholar 

  8. Crum WR, Hartkens T, Hill DLG (2004) Non-rigid image registration: theory and practice. Br J Radiol 77(suppl–2):S140–S153

    Article  Google Scholar 

  9. Seungjong O, Kim S (2017) Deformable image registration in radiation therapy. Radiat Oncol J 35(2):101

    Article  Google Scholar 

  10. Lundervold AS, Lundervold Arvid (2019) An overview of deep learning in medical imaging focusing on mri. Z Med Phys 29(2):102–127

    Article  Google Scholar 

  11. Zaharchuk G, Gong E, Wintermark M, Rubin D, Langlotz CP (2018) Deep learning in neuroradiology. Am J Neuroradiol 39(10):1776–1784

    Article  Google Scholar 

  12. Johnson HJ, Christensen Gary E (2002) Consistent landmark and intensity-based image registration. IEEE Trans Med Imaging 21(5):450–461

    Article  Google Scholar 

  13. Khader M, Hamza AB (2012) An information-theoretic method for multimodality medical image registration. Expert Syst Appl 39(5):5548–5556

    Article  Google Scholar 

  14. Hill DLG, Batchelor PG, Holden M, Hawkes David J (2001) Medical image registration. Phys Med Biol 46(3):R1

    Article  Google Scholar 

  15. Maes F, Collignon A, Vandermeulen D, Marchal G, Suetens P (1997) Multimodality image registration by maximization of mutual information. IEEE Trans Med Imaging 16(2):187–198

    Article  Google Scholar 

  16. Bhattacharya M, Das A (2007) Multi resolution medical image registration using maximization of mutual information & optimization by genetic algorithm. In: 2007 IEEE Nuclear Science Symposium Conference Record, vol 4, pp. 2961–2964. IEEE,

  17. Devarriya D, Gulati C, Mansharamani V, Sakalle A, Bhardwaj A (2019) Unbalanced breast cancer data classification using novel fitness functions in genetic programming. Expert Syst Appl, p. 112866

  18. Das A, Bhattacharya M (2011) Affine-based registration of ct and mr modality images of human brain using multiresolution approaches: comparative study on genetic algorithm and particle swarm optimization. Neural Comput Appl 20(2):223–237

    Article  Google Scholar 

  19. Zheng Z, Cao W, Ye D, Cao G, Lian D (2022) Multi-strategy mutual learning network for deformable medical image registration. Neurocomputing

  20. An Q, Chen S, Liang J, Snyder M, Di Y (2022) Evaluation of dir schemes on tumor/organ with progressive shrinkage: accuracy of tumor/organ internal tissue tracking during the radiation treatment. Radiotherapy and Oncol

  21. Yang T, Bai X, Cui X, Gong Y, Li L (2022) Transdir: Deformable imaging registration network based on transformer to improve the feature extraction ability. Med Phys 49(2):952–965

    Article  Google Scholar 

  22. Amor BB, Arguillère S, Shao L (2021) Resnet-lddmm: advancing the lddmm framework using deep residual networks. arXiv preprint arXiv:2102.07951

  23. Shanker R, Bhattacharya Mahua (2019) Brain tumor segmentation of normal and lesion tissues using hybrid clustering and hierarchical centroid shape descriptor. Comput Methods in Biomechan Biomed Eng: Imaging Vis 7(5–6):676–689

    Google Scholar 

  24. Shanker R, Bhattacharya M (2020) An automated computer-aided diagnosis system for classification of mr images using texture features and gbest-guided gravitational search algorithm. Biocybernet Biomed Eng 40(2):815–835

    Article  Google Scholar 

  25. Rawat W, Wang Z (2017) Deep convolutional neural networks for image classification: a comprehensive review. Neural Comput 29(9):2352–2449

    Article  MathSciNet  MATH  Google Scholar 

  26. Kearney V, Haaf S, Sudhyadhom A, Valdes G, Solberg TD (2018) An unsupervised convolutional neural network-based algorithm for deformable image registration. Phys Med Biol 63(18):185017

    Article  Google Scholar 

  27. Chen Z, Zekai X, Yi W, Yang X, Hou W, Ding M, Granichin O (2019) Real-time and multimodal brain slice-to-volume registration using cnn. Expert Syst Appl 133:86–96

    Article  Google Scholar 

  28. Yang X, Kwitt R, Styner M, Niethammer Marc (2017) Quicksilver: Fast predictive image registration-a deep learning approach. Neuroimage 158:378–396

    Article  Google Scholar 

  29. Cao Y, Miller M, Winslow RL, Younes L (2005) Large deformation diffeomorphic metric mapping of vector fields. IEEE Trans Med Imaging 24(9):1216–1230

    Article  Google Scholar 

  30. Balakrishnan G, Zhao A, Sabuncu MR, Guttag J, Dalca AV (2019) Voxelmorph: a learning framework for deformable medical image registration. IEEE Trans Med Imag

  31. Köpüklü O, Kose N, Gunduz A, Rigoll G (2019) Resource efficient 3d convolutional neural networks. arXiv preprint arXiv:1904.02422

  32. Speiser A, Yan J, Archer EW, Buesing L, Turaga SC, Macke JH (2017) Fast amortized inference of neural activity from calcium imaging data with variational autoencoders. In: Advances in Neural Information Processing Systems, pp. 4024–4034

  33. Hernandez-Matas C, Zabulis X, Triantafyllou A, Anyfanti P, Douma S, Argyros Antonis A (2017) Fire: fundus image registration dataset. J Model Ophthalmol 1(4):16–28

    Google Scholar 

  34. Gousias IS, Edwards AD, Rutherford MA, Counsell SJ, Hajnal JoV, Rueckert D, Hammers A (2012) Magnetic resonance imaging of the newborn brain: manual segmentation of labelled atlases in term-born and preterm infants. Neuroimage 62(3):1499–1509

    Article  Google Scholar 

  35. Jaderberg M, Simonyan K, Zisserman A, et al. (2015) Spatial transformer networks. In: Advances in neural information processing systems, pp. 2017–2025

  36. Hecht-Nielsen R (1992) Theory of the backpropagation neural network. In: Neural networks for perception, pp. 65–93. Elsevier

  37. Kingma Diederik P, Ba Jimmy (2014) Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

  38. Hua B-S, Tran M-K, Yeung S-K (2018) Pointwise convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 984–993

  39. Huang Gao, Liu S, Van der ML, Weinberger KQ (2018) Condensenet: An efficient densenet using learned group convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2752–2761

  40. Szegedy C, Ioffe S, Vanhoucke V, Alemi AA (2017) Inception-v4, inception-resnet and the impact of residual connections on learning. In: Thirty-First AAAI Conference on Artificial Intelligence

  41. Iandola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ, Keutzer K (2016) Squeezenet: Alexnet-level accuracy with 50x fewer parameters and \(< 0.5 mb\) model size. arXiv preprint arXiv:1602.07360,

  42. Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167

  43. Li Y, Yuan Y (2017) Convergence analysis of two-layer neural networks with relu activation. In: Advances in Neural Information Processing Systems, pp. 597–607

  44. Zhang X, Zhou X, Lin M, Sun J (2018) Shufflenet: An extremely efficient convolutional neural network for mobile devices. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6848–6856

  45. Boureau Y-L, Ponce J, LeCun Y (2010) A theoretical analysis of feature pooling in visual recognition. In: Proceedings of the 27th international conference on machine learning (ICML-10), pp. 111–118

  46. Sun B, Zhou S (2012) Study on the 3d interpolation models used in color conversion. Int J Eng Technol 4(1):57

    Article  MathSciNet  Google Scholar 

  47. Keane RD, Adrian RJ (1992) Theory of cross-correlation analysis of piv images. Appl Sci Res 49(3):191–215

    Article  Google Scholar 

  48. Zhang L, Zhang L, Mou X, Zhang D (2011) Fsim: A feature similarity index for image quality assessment. IEEE Trans Image Process 20(8):2378–2386

    Article  MathSciNet  MATH  Google Scholar 

  49. Lowekamp BC, Chen DT, Ibáñez L, Blezek D (2013) The design of simpleitk. Front Neuroinform 7:45

    Article  Google Scholar 

  50. Garyfallidis E, Brett M, Amirbekian B, Rokem A, Van Der Walt S, Descoteaux M, Nimmo-Smith I (2014) Dipy, a library for the analysis of diffusion mri data. Front Neuroinform 8:8

    Article  Google Scholar 

  51. Rigaud B, Simon A, Castelli J, Gobeli M, Ospina Arango J-D, Cazoulat G, Henry O, Haigron P, De Crevoisier R (2015) Evaluation of deformable image registration methods for dose monitoring in head and neck radiotherapy. BioMed Res Int

  52. Kirby N, Chuang C, Ueda U, Pouliot J (2013) The need for application-based adaptation of deformable image registration. Med Phys 40(1):011702

    Article  Google Scholar 

  53. Haussler D, Opper Manfred et al (1997) Mutual information, metric entropy and cumulative relative entropy risk. Ann Stat 25(6):2451–2492

    Article  MathSciNet  MATH  Google Scholar 

  54. Baez JC, Fritz T, Leinster T (2011) A characterization of entropy in terms of information loss. Entropy 13(11):1945–1957

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. ABV-Indian Institute of Information Technology and Management, Morena Link Road, Gwalior, 474015, India

    Ravi Shanker & Mahua Bhattacharya

  2. Department of Computer Science and Engineering, Indian Institute of Information Technology, Vadodara, Gandhinagar, 382028, India

    Heet Sankesara

  3. School of Computer Scinece Engineering and Technology, Bennett University, Greator Noida, 201310, India

    Surendra Nagar

Authors
  1. Ravi Shanker
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Heet Sankesara
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Surendra Nagar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mahua Bhattacharya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ravi Shanker.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shanker, R., Sankesara, H., Nagar, S. et al. RESPNet: resource-efficient and structure-preserving network for deformable image registration. J Supercomput 79, 4713–4736 (2023). https://doi.org/10.1007/s11227-022-04840-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04840-6

Keywords

Search

Navigation