skip to main content
10.1145/3651623.3651630acmotherconferencesArticle/Chapter ViewAbstractPublication PagesapitConference Proceedingsconference-collections
research-article

Multi-feature Combination Medical Image Registration with Keypoints Correction

Published: 07 May 2024 Publication History

Abstract

As a fundamental step in medical image analysis, medical image registration aims to align images from different sources or time, facilitating accurate diagnosis and treatment planning. In recent years, deep learning has revolutionized this field, offering novel solutions to the challenges posed by traditional registration methods. However, some existing methods still face some challenges in registering large deformation images. We propose a multi-feature combination medical image registration with keypoints correction method (MFCNet) that can effectively combine local and global features to solve the long-distance problem faced by large deformation images. The keypoint correction module can extract and optimize anatomical point information of images to assist in the training process. This makes the registered images have better anatomical rationality. Finally, we conducted comparative experiments on two public brain MR datasets (OASIS and LPBA40) with registration methods based on traditional and deep learning in recent years, and performed qualitative and quantitative analyses. The results show that the method we proposed achieve an excellent registration accuracy while ensuring real-time.

References

[1]
John Ashburner. 2007. A fast diffeomorphic image registration algorithm. Neuroimage 38, 1 (2007), 95–113.
[2]
Brian B Avants, Charles L Epstein, Murray Grossman, and James C Gee. 2008. Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Medical image analysis 12, 1 (2008), 26–41.
[3]
Ruzena Bajcsy and Stane Kovačič. 1989. Multiresolution elastic matching. Computer vision, graphics, and image processing 46, 1 (1989), 1–21.
[4]
Guha Balakrishnan, Amy Zhao, Mert R Sabuncu, John Guttag, and Adrian V Dalca. 2019. VoxelMorph: a learning framework for deformable medical image registration. IEEE transactions on medical imaging 38, 8 (2019), 1788–1800.
[5]
Junyu Chen, Eric C Frey, Yufan He, William P Segars, Ye Li, and Yong Du. 2022. Transmorph: Transformer for unsupervised medical image registration. Medical image analysis 82 (2022), 102615.
[6]
Junyu Chen, Yufan He, Eric C Frey, Ye Li, and Yong Du. 2021. Vit-v-net: Vision transformer for unsupervised volumetric medical image registration. arXiv preprint arXiv:2104.06468 (2021).
[7]
Lee R Dice. 1945. Measures of the amount of ecologic association between species. Ecology 26, 3 (1945), 297–302.
[8]
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020).
[9]
Alexey Dosovitskiy, Philipp Fischer, Eddy Ilg, Philip Hausser, Caner Hazirbas, Vladimir Golkov, Patrick Van Der Smagt, Daniel Cremers, and Thomas Brox. 2015. Flownet: Learning optical flow with convolutional networks. In Proceedings of the IEEE international conference on computer vision. 2758–2766.
[10]
Jianyuan Guo, Kai Han, Han Wu, Yehui Tang, Xinghao Chen, Yunhe Wang, and Chang Xu. 2022. Cmt: Convolutional neural networks meet vision transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12175–12185.
[11]
Grant Haskins, Uwe Kruger, and Pingkun Yan. 2020. Deep learning in medical image registration: a survey. Machine Vision and Applications 31, 1 (2020), 1–18.
[12]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Identity mappings in deep residual networks. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14. Springer, 630–645.
[13]
Max Jaderberg, Karen Simonyan, Andrew Zisserman, 2015. Spatial transformer networks. Advances in neural information processing systems 28 (2015).
[14]
Hongming Li and Yong Fan. 2017. Non-rigid image registration using fully convolutional networks with deep self-supervision. arXiv preprint arXiv:1709.00799 (2017).
[15]
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision. 10012–10022.
[16]
Tianyu Ma, Ajay Gupta, and Mert R Sabuncu. 2020. Volumetric landmark detection with a multi-scale shift equivariant neural network. In 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI). IEEE, 981–985.
[17]
Daniel S Marcus, Tracy H Wang, Jamie Parker, John G Csernansky, John C Morris, and Randy L Buckner. 2007. Open Access Series of Imaging Studies (OASIS): cross-sectional MRI data in young, middle aged, nondemented, and demented older adults. Journal of cognitive neuroscience 19, 9 (2007), 1498–1507.
[18]
Long Short-Term Memory. 2010. Long short-term memory. Neural computation 9, 8 (2010), 1735–1780.
[19]
Tony CW Mok and Albert CS Chung. 2020. Large deformation diffeomorphic image registration with laplacian pyramid networks. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part III 23. Springer, 211–221.
[20]
David W Shattuck, Mubeena Mirza, Vitria Adisetiyo, Cornelius Hojatkashani, Georges Salamon, Katherine L Narr, Russell A Poldrack, Robert M Bilder, and Arthur W Toga. 2008. Construction of a 3D probabilistic atlas of human cortical structures. Neuroimage 39, 3 (2008), 1064–1080.
[21]
Amitojdeep Singh, Sourya Sengupta, and Vasudevan Lakshminarayanan. 2020. Explainable deep learning models in medical image analysis. Journal of imaging 6, 6 (2020), 52.
[22]
Hessam Sokooti, Bob de Vos, Floris Berendsen, Boudewijn PF Lelieveldt, Ivana Išgum, and Marius Staring. 2017. Nonrigid image registration using multi-scale 3D convolutional neural networks. In International conference on medical image computing and computer-assisted intervention. Springer, 232–239.
[23]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).
[24]
Bob D de Vos, Floris F Berendsen, Max A Viergever, Marius Staring, and Ivana Išgum. 2017. End-to-end unsupervised deformable image registration with a convolutional neural network. In Deep learning in medical image analysis and multimodal learning for clinical decision support. Springer, 204–212.
[25]
Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao. 2021. Pyramid vision transformer: A versatile backbone for dense prediction without convolutions. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 568–578.
[26]
Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao. 2022. Pvt v2: Improved baselines with pyramid vision transformer. Computational Visual Media 8, 3 (2022), 415–424.
[27]
Haiping Wu, Bin Xiao, Noel Codella, Mengchen Liu, Xiyang Dai, Lu Yuan, and Lei Zhang. 2021. Cvt: Introducing convolutions to vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 22–31.
[28]
Xiao Yang, Roland Kwitt, Martin Styner, and Marc Niethammer. 2017. Quicksilver: Fast predictive image registration–a deep learning approach. NeuroImage 158 (2017), 378–396.
[29]
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision. 2223–2232.

Index Terms

  1. Multi-feature Combination Medical Image Registration with Keypoints Correction

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Other conferences
        APIT '24: Proceedings of the 2024 6th Asia Pacific Information Technology Conference
        January 2024
        105 pages
        ISBN:9798400716218
        DOI:10.1145/3651623
        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 07 May 2024

        Permissions

        Request permissions for this article.

        Check for updates

        Author Tags

        1. Deep learning
        2. Keypoints correction
        3. Medical image registration
        4. Multi-feature combination

        Qualifiers

        • Research-article
        • Research
        • Refereed limited

        Funding Sources

        • the Fundamental Research Foundation of Shenzhen under Grant

        Conference

        APIT 2024

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • 0
          Total Citations
        • 19
          Total Downloads
        • Downloads (Last 12 months)19
        • Downloads (Last 6 weeks)1
        Reflects downloads up to 13 Jan 2025

        Other Metrics

        Citations

        View Options

        Login options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format.

        HTML Format

        Media

        Figures

        Other

        Tables

        Share

        Share

        Share this Publication link

        Share on social media