Crossmodal Matching Transformer based X-ray and CT image registration for TEVAR
Authors: 
Meng Li, Changyan Lin, Lixia Shu, Xin Pu, Yu Chen, Heng Wu, Jiasong Li, Hongshuai CaoAuthors Info & Claims
Meng Li, Changyan Lin, Lixia Shu, Xin Pu, Yu Chen, Heng Wu, Jiasong Li, Hongshuai CaoAuthors Info & ClaimsAbstract
Since the mapping relationship between definitized intra-interventional 2D X-ray and undefined pre-interventional 3D Computed Tomography(CT) is uncertain, auxiliary positioning devices or body markers, such as medical implants, are commonly used to determine this relationship. However, such approaches can not be widely used in clinical due to the complex realities. To determine the mapping relationship, and achieve a initializtion post estimation of human body without auxiliary equipment or markers, a cross-modal matching transformer network is proposed to matching 2D X-ray and 3D CT images directly. The proposed approach first learns skeletal features from 2D X-ray and 3D CT images. The features are then converted into 1D X-ray and CT representation vectors, which are combined using a transformer module. As a result, the well-trained network can directly predict the spatial correspondence between arbitrary 2D X-ray and 3D CT. The experimental results show that when combining our approach with the conventional approach, the achieved accuracy and speed can meet the basic clinical intervention needs, and it provides a new direction for intra-interventional registration.
References
[1]
Xi Cheng, Li Zhang, and Yefeng Zheng. 2018. Deep similarity learning for multimodal medical images. Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization 6, 3 (2018), 248–252.
[2]
Özgün Çiçek, Ahmed Abdulkadir, Soeren S Lienkamp, Thomas Brox, and Olaf Ronneberger. 2016. 3D U-Net: learning dense volumetric segmentation from sparse annotation. In International conference on medical image computing and computer-assisted intervention. Springer, 424–432.
[3]
Sayan Ghosal and Nilanjan Ray. 2017. Deep deformable registration: Enhancing accuracy by fully convolutional neural net. Pattern Recognition Letters 94 (2017), 81–86.
[4]
Guofa Li, Yifan Yang, Xingda Qu, Dongpu Cao, and Keqiang Li. 2020. A deep learning based image enhancement approach for autonomous driving at night. Knowledge-Based Systems(2020), 106617.
[5]
Haofu Liao, Wei-An Lin, Jiarui Zhang, Jingdan Zhang, Jiebo Luo, and S Kevin Zhou. 2019. Multiview 2D/3D rigid registration via a point-of-interest network for tracking and triangulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12638–12647.
[6]
Haofu Liao, Wei-An Lin, Jiarui Zhang, Jingdan Zhang, Jiebo Luo, and S Kevin Zhou. 2019. Multiview 2d/3d rigid registration via a point-of-interest network for tracking and triangulation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 12638–12647.
[7]
Stefan Matl, Richard Brosig, Maximilian Baust, Nassir Navab, and Stefanie Demirci. 2017. Vascular image registration techniques: A living review. Medical image analysis 35 (2017), 1–17.
[8]
Shun Miao, Sebastien Piat, Peter Fischer, Ahmet Tuysuzoglu, Philip Mewes, Tommaso Mansi, and Rui Liao. 2018. Dilated fcn for multi-agent 2d/3d medical image registration. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32.
[9]
Tulin Ozturk, Muhammed Talo, Eylul Azra Yildirim, Ulas Baran Baloglu, Ozal Yildirim, and U Rajendra Acharya. 2020. Automated detection of COVID-19 cases using deep neural networks with X-ray images. Computers in biology and medicine 121 (2020), 103792.
[10]
Tran Minh Quan, David GC Hildebrand, and Won-Ki Jeong. 2016. Fusionnet: A deep fully residual convolutional neural network for image segmentation in connectomics. arXiv preprint arXiv:1612.05360(2016).
[11]
Volker Rasche, B Schreiber, C Graeff, Thomas Istel, Hermann Schomberg, Michael Grass, Reiner Koppe, Erhard Klotz, and Georg Rose. 2003. Performance of image intensifier-equipped X-ray systems for three-dimensional imaging. In International Congress Series, Vol. 1256. Elsevier, 187–192.
[12]
Adnan Z Rizvi, M Hassan Murad, Ronald M Fairman, Patricia J Erwin, and Victor M Montori. 2009. The effect of left subclavian artery coverage on morbidity and mortality in patients undergoing endovascular thoracic aortic interventions: a systematic review and meta-analysis. Journal of vascular surgery 50, 5 (2009), 1159–1169.
[13]
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention. Springer, 234–241.
[14]
Siyang Song, Shashank Jaiswal, Linlin Shen, and Michel Valstar. 2020. Spectral representation of behaviour primitives for depression analysis. IEEE Transactions on Affective Computing(2020).
[15]
Siyang Song, Linlin Shen, and Michel Valstar. 2018. Human behaviour-based automatic depression analysis using hand-crafted statistics and deep learned spectral features. In 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018). IEEE, 158–165.
[16]
PJ Tang, HL Wang, and LX Zuo. 2016. Parallel cross deep convolution neural networks model. Journal of Image and Graphics 21, 3 (2016), 339–347.
[17]
Yao-Hung Hubert Tsai, Shaojie Bai, Paul Pu Liang, J Zico Kolter, Louis-Philippe Morency, and Ruslan Salakhutdinov. 2019. Multimodal transformer for unaligned multimodal language sequences. In Proceedings of the conference. Association for Computational Linguistics. Meeting, Vol. 2019. NIH Public Access, 6558.
[18]
Guorong Wu, Minjeong Kim, Qian Wang, Brent C Munsell, and Dinggang Shen. 2015. Scalable high-performance image registration framework by unsupervised deep feature representations learning. IEEE Transactions on Biomedical Engineering 63, 7 (2015), 1505–1516.
[19]
Kun Xu, Hang Su, Jun Zhu, Ji-Song Guan, and Bo Zhang. 2016. Neuron segmentation based on CNN with semi-supervised regularization. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 20–28.
[20]
Hengfen Yang, Xin Jin, and Dongming Zhou. 2015. Block medical image fusion based on adaptive PCNN. In 2015 6th IEEE International Conference on Software Engineering and Service Science (ICSESS). IEEE, 330–333.
[21]
Xin Yi, Ekta Walia, and Paul Babyn. 2019. Generative adversarial network in medical imaging: A review. Medical image analysis 58 (2019), 101552.
[22]
Liya Zhao and Kebin Jia. 2015. Deep adaptive log-demons: diffeomorphic image registration with very large deformations. Computational and Mathematical Methods in Medicine 2015 (2015).
Index Terms
- Crossmodal Matching Transformer based X-ray and CT image registration for TEVAR
Index terms have been assigned to the content through auto-classification.
Recommendations
X-Ray to CT Rigid Registration Using Scene Coordinate Regression
Medical Image Computing and Computer Assisted Intervention – MICCAI 2023AbstractIntraoperative fluoroscopy is a frequently used modality in minimally invasive orthopedic surgeries. Aligning the intraoperatively acquired X-ray image with the preoperatively acquired 3D model of a computed tomography (CT) scan reduces the mental ...
Joint segmentation and registration for 4D lung CT images based on Markov random field
ICIMCS '18: Proceedings of the 10th International Conference on Internet Multimedia Computing and ServiceThe study of segmentation and registration of lung volume in medical images has been an active area with the emergence and development of 4D CT (Computed Tomography) medical imaging techniques. Precise image segmentation and registration methods are ...
Registration of 3-D CT and 2-D Flat Images of Mouse via Affine Transformation
It is difficult to directly coregister the 3-D fluorescence molecular tomography (FMT) image of a small tumor in a mouse whose maximal diameter is only a few millimeters with a larger CT image of the entire animal that spans about 10 cm. This paper ...
Comments
Information & Contributors
Information
Published In
August 2021
91 pages
ISBN:9781450390507
DOI:10.1145/3484424
Copyright © 2021 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 08 November 2021
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICBIP '21
ICBIP '21: 2021 6th International Conference on Biomedical Signal and Image Processing
August 20 - 22, 2021
Suzhou, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 95Total Downloads
- Downloads (Last 12 months)8
- Downloads (Last 6 weeks)0
Reflects downloads up to 08 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format