Fuyang Xing
ABSTRACT
As a fundamental step in medical image analysis, medical image registration aims to align images from different sources or time, facilitating accurate diagnosis and treatment planning. In recent years, deep learning has revolutionized this field, offering novel solutions to the challenges posed by traditional registration methods. However, some existing methods still face some challenges in registering large deformation images. We propose a multi-feature combination medical image registration with keypoints correction method (MFCNet) that can effectively combine local and global features to solve the long-distance problem faced by large deformation images. The keypoint correction module can extract and optimize anatomical point information of images to assist in the training process. This makes the registered images have better anatomical rationality. Finally, we conducted comparative experiments on two public brain MR datasets (OASIS and LPBA40) with registration methods based on traditional and deep learning in recent years, and performed qualitative and quantitative analyses. The results show that the method we proposed achieve an excellent registration accuracy while ensuring real-time.
- John Ashburner. 2007. A fast diffeomorphic image registration algorithm. Neuroimage 38, 1 (2007), 95–113.Google Scholar
Cross Ref
- Brian B Avants, Charles L Epstein, Murray Grossman, and James C Gee. 2008. Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Medical image analysis 12, 1 (2008), 26–41.Google Scholar
- Ruzena Bajcsy and Stane Kovačič. 1989. Multiresolution elastic matching. Computer vision, graphics, and image processing 46, 1 (1989), 1–21.Google Scholar
- Guha Balakrishnan, Amy Zhao, Mert R Sabuncu, John Guttag, and Adrian V Dalca. 2019. VoxelMorph: a learning framework for deformable medical image registration. IEEE transactions on medical imaging 38, 8 (2019), 1788–1800.Google Scholar
- Junyu Chen, Eric C Frey, Yufan He, William P Segars, Ye Li, and Yong Du. 2022. Transmorph: Transformer for unsupervised medical image registration. Medical image analysis 82 (2022), 102615.Google Scholar
- Junyu Chen, Yufan He, Eric C Frey, Ye Li, and Yong Du. 2021. Vit-v-net: Vision transformer for unsupervised volumetric medical image registration. arXiv preprint arXiv:2104.06468 (2021).Google Scholar
- Lee R Dice. 1945. Measures of the amount of ecologic association between species. Ecology 26, 3 (1945), 297–302.Google ScholarCross Ref
- Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020).Google Scholar
- Alexey Dosovitskiy, Philipp Fischer, Eddy Ilg, Philip Hausser, Caner Hazirbas, Vladimir Golkov, Patrick Van Der Smagt, Daniel Cremers, and Thomas Brox. 2015. Flownet: Learning optical flow with convolutional networks. In Proceedings of the IEEE international conference on computer vision. 2758–2766.Google ScholarDigital Library
- Jianyuan Guo, Kai Han, Han Wu, Yehui Tang, Xinghao Chen, Yunhe Wang, and Chang Xu. 2022. Cmt: Convolutional neural networks meet vision transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12175–12185.Google ScholarCross Ref
- Grant Haskins, Uwe Kruger, and Pingkun Yan. 2020. Deep learning in medical image registration: a survey. Machine Vision and Applications 31, 1 (2020), 1–18.Google ScholarDigital Library
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Identity mappings in deep residual networks. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14. Springer, 630–645.Google Scholar
- Max Jaderberg, Karen Simonyan, Andrew Zisserman, 2015. Spatial transformer networks. Advances in neural information processing systems 28 (2015).Google Scholar
- Hongming Li and Yong Fan. 2017. Non-rigid image registration using fully convolutional networks with deep self-supervision. arXiv preprint arXiv:1709.00799 (2017).Google Scholar
- Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision. 10012–10022.Google ScholarCross Ref
- Tianyu Ma, Ajay Gupta, and Mert R Sabuncu. 2020. Volumetric landmark detection with a multi-scale shift equivariant neural network. In 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI). IEEE, 981–985.Google ScholarCross Ref
- Daniel S Marcus, Tracy H Wang, Jamie Parker, John G Csernansky, John C Morris, and Randy L Buckner. 2007. Open Access Series of Imaging Studies (OASIS): cross-sectional MRI data in young, middle aged, nondemented, and demented older adults. Journal of cognitive neuroscience 19, 9 (2007), 1498–1507.Google ScholarDigital Library
- Long Short-Term Memory. 2010. Long short-term memory. Neural computation 9, 8 (2010), 1735–1780.Google Scholar
- Tony CW Mok and Albert CS Chung. 2020. Large deformation diffeomorphic image registration with laplacian pyramid networks. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part III 23. Springer, 211–221.Google ScholarDigital Library
- David W Shattuck, Mubeena Mirza, Vitria Adisetiyo, Cornelius Hojatkashani, Georges Salamon, Katherine L Narr, Russell A Poldrack, Robert M Bilder, and Arthur W Toga. 2008. Construction of a 3D probabilistic atlas of human cortical structures. Neuroimage 39, 3 (2008), 1064–1080.Google ScholarCross Ref
- Amitojdeep Singh, Sourya Sengupta, and Vasudevan Lakshminarayanan. 2020. Explainable deep learning models in medical image analysis. Journal of imaging 6, 6 (2020), 52.Google ScholarCross Ref
- Hessam Sokooti, Bob de Vos, Floris Berendsen, Boudewijn PF Lelieveldt, Ivana Išgum, and Marius Staring. 2017. Nonrigid image registration using multi-scale 3D convolutional neural networks. In International conference on medical image computing and computer-assisted intervention. Springer, 232–239.Google ScholarDigital Library
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).Google Scholar
- Bob D de Vos, Floris F Berendsen, Max A Viergever, Marius Staring, and Ivana Išgum. 2017. End-to-end unsupervised deformable image registration with a convolutional neural network. In Deep learning in medical image analysis and multimodal learning for clinical decision support. Springer, 204–212.Google Scholar
- Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao. 2021. Pyramid vision transformer: A versatile backbone for dense prediction without convolutions. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 568–578.Google ScholarCross Ref
- Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao. 2022. Pvt v2: Improved baselines with pyramid vision transformer. Computational Visual Media 8, 3 (2022), 415–424.Google ScholarCross Ref
- Haiping Wu, Bin Xiao, Noel Codella, Mengchen Liu, Xiyang Dai, Lu Yuan, and Lei Zhang. 2021. Cvt: Introducing convolutions to vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 22–31.Google ScholarCross Ref
- Xiao Yang, Roland Kwitt, Martin Styner, and Marc Niethammer. 2017. Quicksilver: Fast predictive image registration–a deep learning approach. NeuroImage 158 (2017), 378–396.Google ScholarCross Ref
- Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision. 2223–2232.Google ScholarCross Ref
Index Terms
- Multi-feature Combination Medical Image Registration with Keypoints Correction
Recommendations
Cascade connection-based channel attention network for bidirectional medical image registration
AbstractMedical image registration is an essential task in researching and applying medical images. Doctors can observe and extract relevant pathological features to quickly analyze the disease by registered images to diagnose the infection. After more ...
Medical Image Registration Method Based on Simulated CT
Intelligent Computing MethodologiesAbstractIn recent years, artificial intelligence (AI) has been widely used in medicine, for example, tumor screening, qualitative diagnosis, radiotherapy organ delineation, curative effect evaluation and prognosis, etc. Medical image registration is a ...
Learning deformable registration of medical images with anatomical constraints
AbstractDeformable image registration is a fundamental problem in the field of medical image analysis. During the last years, we have witnessed the advent of deep learning-based image registration methods which achieve state-of-the-art ...
Comments