Skip to main content

Advertisement

SpringerLink
Log in

Deformable registration of lung 3DCT images using an unsupervised heterogeneous multi-resolution neural network

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

Lung image registration is more challenging than other organs. This is because the breath of the human body causes large deformations in the lung parenchyma and small deformations in tissues such as the pulmonary vascular. Many studies have recently used multi-resolution networks to solve the lung registration problem. However, they use the same structure of registration modules on each level, which makes it difficult to handle complex and small deformations. We propose an unsupervised heterogeneous multi-resolution network (UHMR-Net) to overcome the above problem. The image detail registration module (IDRM) is designed on the highest resolution level. Within this module, the cascaded network is used on the same resolution image to continuously learn the “remaining” detail deformation fields. The shallow shrinkage loss (SS-Loss) is designed to supervise the cascaded network, thus further improving the ability of the network to handle small deformations. Moreover, with the lightweight feature local correlation layer we proposed, the image boundary registration module (IBRM), on multiple low-resolution levels, can better solve the large deformation registration problem. The target registration error on the public DIR-Lab 4DCT dataset was 1.56 ± 1.39 mm, which was significantly better than the classic conventional methods and advanced deep-based methods.

Graphical Abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Murphy K, Pluim JP, van Rikxoort EM, de Jong PA, de Hoop B et al (2012) Toward automatic regional analysis of pulmonary function using inspiration and expiration thoracic CT. Med Phys 39:1650–1662. https://doi.org/10.1118/1.3687891

    Article  PubMed  Google Scholar 

  2. Weiss E, Wijesooriya K, Dill SV, Keall PJ (2007) Tumor and normal tissue motion in the thorax during respiration: analysis of volumetric and positional variations using 4D CT. Int J Radiat Oncol Biol Phys 67:296–307. https://doi.org/10.1016/j.ijrobp.2006.09.009

    Article  PubMed  Google Scholar 

  3. De Vos BD, Berendsen FF, Viergever MA, Sokooti H, Staring M et al (2019) A deep learning framework for unsupervised affine and deformable image registration. Med Image Anal 52:128–143. https://doi.org/10.1016/j.media.2018.11.010

    Article  PubMed  Google Scholar 

  4. Zhao S, Dong Y, Chang EI and Xu Y (2019) Recursive cascaded networks for unsupervised medical image registration. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV) (10599–10609). https://doi.org/10.1109/ICCV.2019.01070

  5. Fang Q, Gu X, Yan J, Zhao J and Li Q (2019) A FCN-based unsupervised learning model for deformable chest CT image registration. In: IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC), p. 1–14

  6. Eppenhof KA, Pluim JP (2018) Pulmonary CT registration through supervised learning with convolutional neural networks. IEEE Trans Med Imaging 38:1097–1105. https://doi.org/10.1109/TMI.2018.2878316

    Article  PubMed  Google Scholar 

  7. Fu Y, Lei Y, Wang T, Higgins K, Bradley JD et al (2020) LungRegNet: an unsupervised deformable image registration method for 4D-CT lung. Med Phys 47:1763–1774. https://doi.org/10.1002/mp.14065

    Article  PubMed  Google Scholar 

  8. Hering A, Ginneken Bv and Heldmann S (2019) mlvirnet: multilevel variational image registration network. In International Conference on Medical Image Computing and Computer-Assisted Intervention (257–265). https://doi.org/10.1007/978-3-030-32226-7_29

  9. Mok TC and Chung A (2020) Large deformation diffeomorphic image registration with Laplacian pyramid networks. In International Conference on Medical Image Computing and Computer-Assisted Intervention (211–221). https://doi.org/10.1007/978-3-030-59716-0_21

  10. He X, Guo J, Zhang X, Bi H, Gerard S et al (2021) Recursive refinement network for deformable lung registration between exhale and inhale CT scans. arXiv preprint arXiv:210607608. https://doi.org/10.48550/arXiv.2106.07608

  11. Jiang Z, Yin F-F, Ge Y, Ren L (2020) A multi-scale framework with unsupervised joint training of convolutional neural networks for pulmonary deformable image registration. Phys Med Biol 65:015011. https://doi.org/10.1088/1361-6560/ab5da0

    Article  PubMed  PubMed Central  Google Scholar 

  12. Shu Y, Wang H, Xiao B, Bi X and Li W (2021) Medical image registration based on uncoupled learning and accumulative enhancement. In International Conference on Medical Image Computing and Computer-Assisted Intervention (3–13). https://doi.org/10.1007/978-3-030-87202-1_1

  13. Zhang L, Zhou L, Li R, Wang X, Han B et al (2021) Cascaded feature warping network for unsupervised medical image registration. In 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI) (913–916). https://doi.org/10.1109/ISBI48211.2021.9433880

  14. Jaderberg M, Simonyan K and Zisserman A (2015) Spatial transformer networks. In: Proceedings of the 28th International Conference on Neural Information Processing Systems 2:2017–2025

  15. Balakrishnan G, Zhao A, Sabuncu MR, Guttag J and Dalca AV (2018) An unsupervised learning model for deformable medical image registration. In Proceedings of the IEEE conference on computer vision and pattern recognition (9252–9260). https://doi.org/10.1109/CVPR.2018.00964

  16. Xu J, Ranftl R and Koltun V (2017) Accurate optical flow via direct cost volume processing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (1289–1297). https://doi.org/10.1109/CVPR.2017.615

  17. Dosovitskiy A, Fischer P, Ilg E, Hausser P, Hazirbas C et al (2015) FlowNet: learning optical flow with convolutional networks. Proceedings of the IEEE International Conference on Computer Vision (ICCV) (2758–2766) https://doi.org/10.1109/iccv.2015.316

  18. Hui T-W, Tang X and Loy CC (2018) Liteflownet: a lightweight convolutional neural network for optical flow estimation. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (8981–8989), https://doi.org/10.1109/CVPR.2018.00936

  19. Kang M, Hu X, Huang W, Scott MR, Reyes M (2022) Dual-stream pyramid registration network. Med Image Anal 78:102379. https://doi.org/10.1016/j.media.2022.102379

    Article  PubMed  Google Scholar 

  20. Sun D, Yang X, Liu M-Y and Kautz J (2018) Pwc-net: Cnns for optical flow using pyramid, warping, and cost volume. In Proceedings of the IEEE conference on computer vision and pattern recognition (8934–8943). https://doi.org/10.1109/CVPR.2018.00931

  21. Huang G, Liu Z, Van Der Maaten L and Weinberger KQ (2017) Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (4700–4708). https://doi.org/10.1109/CVPR.2017.243

  22. Rueckert D, Sonoda LI, Hayes C, Hill DLG, Leach MO et al (1999) Nonrigid registration using free-form deformations: application to breast MR images. IEEE Trans Med Imaging 18:712–721. https://doi.org/10.1109/42.796284

    Article  CAS  PubMed  Google Scholar 

  23. Hugo GD, Weiss E, Sleeman WC, Balik S, Keall PJ et al (2017) A longitudinal four-dimensional computed tomography and cone beam computed tomography dataset for image-guided radiation therapy research in lung cancer. Med Phys 44:762–771. https://doi.org/10.1002/mp.12059

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  24. Castillo E, Castillo R, Martinez J, Shenoy M, Guerrero T (2009) Four-dimensional deformable image registration using trajectory modeling. Phys Med Biol 55:305–327. https://doi.org/10.1088/0031-9155/55/1/018

    Article  Google Scholar 

  25. Castillo R, Castillo E, Guerra R, Johnson VE, McPhail T et al (2009) A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets. Phys Med Biol 54:1849–1870. https://doi.org/10.1088/0031-9155/54/7/001

    Article  PubMed  Google Scholar 

  26. Paszke A, Gross S, Massa F, Lerer A, Bradbury J et al (2019) Pytorch: an imperative style, high-performance deep learning library. Adv Neural Inf Process Syst 32:8026–8037

    Google Scholar 

  27. Avants BB, Tustison NJ, Song G, Cook PA, Klein A et al (2011) A reproducible evaluation of ANTs similarity metric performance in brain image registration. Neuroimage 54:2033–2044. https://doi.org/10.1016/j.neuroimage.2010.09.025

    Article  PubMed  Google Scholar 

  28. Marstal K, Berendsen F, Staring M and Klein S (2016) SimpleElastix: a user-friendly, multi-lingual library for medical image registration. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops ( 134–142). https://doi.org/10.1109/CVPRW.2016.78

  29. Delmon V, Rit S, Pinho R, Sarrut D (2013) Registration of sliding objects using direction dependent B-splines decomposition. Phys Med Biol 58:1303–1314. https://doi.org/10.1088/0031-9155/58/5/1303

    Article  CAS  PubMed  Google Scholar 

Download references

Funding

This work was supported by the National Natural Science Foundation of China under award number 61976091.

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, East China University of Science and Technology, Shanghai, China

    Qing Chang & Jieming Zhang

Authors
  1. Qing Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jieming Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qing Chang.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chang, Q., Zhang, J. Deformable registration of lung 3DCT images using an unsupervised heterogeneous multi-resolution neural network. Med Biol Eng Comput 61, 2353–2365 (2023). https://doi.org/10.1007/s11517-023-02834-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-023-02834-x

Keywords

Search

Navigation