Locally adaptive 2D–3D registration using vascular structure model for liver catheterization

doi:10.1016/j.compbiomed.2016.01.009

Computers in Biology and Medicine

Volume 70, 1 March 2016, Pages 119-130

https://doi.org/10.1016/j.compbiomed.2016.01.009 Get rights and content

Highlights

•
A registration algorithm between 2D DSA image and 3D CTA scans is proposed.
•
A vascular structure model is constructed using the augmented fast marching method.
•
The tree model is divided into subtrees based on the connectivity and the length.
•
Our method performs the registration focusing on the selected subtree.
•
Our method performs the registration very accurately.

Abstract

Two-dimensional–three-dimensional (2D–3D) registration between intra-operative 2D digital subtraction angiography (DSA) and pre-operative 3D computed tomography angiography (CTA) can be used for roadmapping purposes. However, through the projection of 3D vessels, incorrect intersections and overlaps between vessels are produced because of the complex vascular structure, which makes it difficult to obtain the correct solution of 2D–3D registration. To overcome these problems, we propose a registration method that selects a suitable part of a 3D vascular structure for a given DSA image and finds the optimized solution to the partial 3D structure. The proposed algorithm can reduce the registration errors because it restricts the range of the 3D vascular structure for the registration by using only the relevant 3D vessels with the given DSA. To search for the appropriate 3D partial structure, we first construct a tree model of the 3D vascular structure and divide it into several subtrees in accordance with the connectivity. Then, the best matched subtree with the given DSA image is selected using the results from the coarse registration between each subtree and the vessels in the DSA image. Finally, a fine registration is conducted to minimize the difference between the selected subtree and the vessels of the DSA image. In experimental results obtained using 10 clinical datasets, the average distance errors in the case of the proposed method were 2.34±1.94 mm. The proposed algorithm converges faster and produces more correct results than the conventional method in evaluations on patient datasets.

Introduction

Pre-operative three-dimensional (3D) computed tomography (CT) angiographic images can aid in planning intravascular interventions and guiding catheter-directed procedures. Since intra-operative two-dimensional (2D) angiography images scan the limited areas near the catheter, it is difficult to figure out the overall structure of the vasculature and the location of the catheter. Moreover, geometric information is lost through projection, which makes it difficult to recognize the connectivity between vessels. Pre-operative computed tomography angiography (CTA) data can compensate for these limitations of 2D angiography because they provide the overall 3D vascular structure. Hence, it can be used for planning the path for the catheter to be moved before the intervention, and for making a roadmap that shows the currently traversed vessels and the distal vessel tree.

To utilize the 3D CTA data effectively, an accurate registration between 2D digital subtraction angiography (DSA) and 3D CTA images is needed. Therefore, many approaches have been proposed for 2D–3D registration in abdominal [1], [2], [3], [4], cardiac [5], [6], [7], [8], [9], and neurological [10], [11], [12] interventions, and Markelj et al. [13] has provided a review of the existing 2D–3D registration methods.

Many algorithms follow the strategy of minimizing the difference between the vessels of a 2D image and the projection of 3D vessels. Groher et al. [1] optimizes the transformation parameters by using the difference between centerlines and the projection of 3D vessels. Further, they use topological information such as the degree of bifurcation points in order to enhance the accuracy of the registration algorithm. However, accurate 2D–3D branching point matching might be difficult when enhanced 2D vessel regions are considerably smaller than the whole 3D vessel regions. Further, this method detects the branching points of the thickest 2D and 3D vessels as root nodes and finds the x–y translation in the initialization step. The correspondence of the root nodes of the 2D and 3D vessels might be incorrectly generated and accordingly degrade the registration accuracy when the enhanced range of 2D vessels is different from that of 3D vessels. Jomier et al. [3] proposed the sum of the Gaussian-blurred intensity values in the 2D image at the projected model points as a registration metric. This method improves the registration accuracy by using bi-plane images. In clinical practice, bi-plane images are not acquired for all clinical applications. Rivest-Hénault et al. [6] proposed a non-rigid registration algorithm with a distance measure and regularization constraints that maintain the smoothness and a certain degree of rigidity of the vessels. Although these algorithms show a high accuracy and a robust convergence, they assume that the contrasted vessels in DSA have a similar range to that of the 3D vessels from CTA scans. However, in the liver catheterization, a contrast medium is injected, and only the limited areas near the catheter are displayed in the angiography images while the catheter is moved through the vascular structure. Ambrosini et al. [24] overcame this problem by selecting an appropriate leaf vessel centerline from the 3D vascular structure according to the shape similarity with the catheter centerline. This method selects the appropriate vessel centerline in most cases and shows a high accuracy in the registration results. However, it fails to find a correct solution when a small part of the catheter is visible on the image or the shape of the catheter does not have a predominant feature.

In this paper, we propose a method that finds a currently traversed area in a 3D vascular structure for a given DSA image and registers the 3D vessels to the 2D vessels on the DSA by using the subtree structure of the 3D vessels. We improve the accuracy of the registration by dividing a 3D vascular tree model into several subtrees and using only one of the subtrees in the registration process. The 3D vascular structure is very complex, and therefore, the projection of the entire structure causes a considerable number of overlapped vessels and incorrect intersection points. This leads to a false local minima in the registration process, particularly when a DSA image shows only a part of the vascular structure. We overcome these problems by dividing the 3D structure into several subtrees and using only the relevant part of the given DSA image. To search for the appropriate part of the DSA image, we first construct a tree model of the 3D vascular structure considering their connectivity and divide it into several subtrees. Subsequently, the best matched subtree for the given DSA image is selected according to the dissimilarity measure from the coarse registration between each subtree and the DSA image. Finally, fine registration is conducted for the selected subtree by using a distance metric. Then, by restricting the range for the registration process, we obtain better convergence and higher accuracy than the registration using all the 3D vessels.

The remainder of this paper is organized as follows: the next section describes a method of constructing the 3D vascular structure model. Section 3 explains the locally adaptive registration algorithm using the 3D model. Section 4 discusses the experimental results and is followed by Section 5 that presents the conclusion and future work.

Section snippets

Construction of 3D vascular structure model

The proposed method consists of four main steps, as shown in Fig. 1. First, we analyze the pre-operative 3D CTA and the intra-operative 2D DSA. From the 3D CTA, a 3D vascular structural model is constructed and divided into several subtrees. During intervention, a 2D vessel centerline and its distance transform are computed from the 2D DSA. Next, the best matched subtree for a 2D vessel centerline is selected by comparing the dissimilarity among the subtrees. Finally, fine registration is

Locally adaptive registration

To use 3D CTA data as a roadmap during an intervention procedure, an accurate registration between the CTA and the DSA images is necessary. During an intervention, a catheter moves through an artery and contrast medium is injected near the catheter. Then, the injected area is projected to produce new DSA images. Through this process, very limited areas of the vascular structure are displayed in the DSA. Therefore, it is difficult to match the DSA with the entire 3D vessel structure. In this

Experimental results

Ten breath-hold DSA datasets were tested to verify that the proposed algorithm found the appropriate subtree and computed the correct transformation parameters. Each DSA dataset had the corresponding CTA scans, and we used four CTA scans obtained from different patients (Table 1). All datasets were obtained using a C-arm CT scanner of Siemens. The number of slices per CTA scan ranged from 343 to 441, and each slice had a size of 512×512. The pixel spacing and the slice interval were all 0.4 mm.

Conclusion

In this paper, we presented a 2D–3D registration algorithm between pre-operative 3D CTA scans and intra-operative 2D DSA images for liver catheterization. The proposed approach improves the accuracy of the registration by detecting the areas of interest in the 3D vascular structure used for the registration process. The structure constructed from the 3D CTA scans is divided into several subtrees with respect to their connectivity. In the registration process, the dissimilarity of each subtree

Conflict of interest statement

None declared.

Acknowledgments

This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT and future Planning (No. 2014R1A2A2A03002574).

Jihye Kim is a PhD candidate in the School of Computer Science and Engineering from Seoul National University, Korea. She received the BS degree in the Computer Science from KAIST in 2005, and the MS degree in the School of Computer Science and Engineering from KAIST in 2007. Since 2007, she has worked as a software engineer in DMC R&D center, Samsung Electronics. Her research interests are image segmentation and registration.

References (24)

N. Baka et al.
Statistical coronary motion models for 2D+ t/3D registration of X-ray coronary angiography and CTA
Med. Image Anal.
(2013)
P. Markelj et al.
A review of 3D/2D registration methods for image-guided interventions
Med. Image Anal.
(2012)
A.W. Fitzgibbon
Robust registration of 2D and 3D point sets
Image Vis. Comput.
(2003)
M. Groher et al.
New CTA protocol and 2D–3D registration method for liver catheterization
M. Groher et al.
Deformable 2D–3D registration of vascular structures in a one view scenario
Med. Imaging IEEE Trans.
(2009)
J. Jomier et al.
3D/2D model-to-image registration applied to TIPS surgery
R. Liao et al.
An efficient graph-based deformable 2D/3D registration algorithm with applications for abdominal aortic aneurysm interventions
D. Ruijters et al.
Vesselness-based 2D–3D registration of the coronary arteries
Int. J. Comput.-Assist. Radiol. Surg.
(2009)
D. Rivest-Hénault et al.
Nonrigid 2D/3D registration of coronary artery models with live fluoroscopy for guidance of cardiac interventions
Med. Imaging IEEE Trans.
(2012)
G.A. Turgeon et al.
2D–3D registration of coronary angiograms for cardiac procedure planning and guidance
Med. Phys.
(2005)

E. Serradell, A. Romero, R. Leta, C. Gatta F, Moreno–Noguer simultaneous correspondence and non-rigid 3D reconstruction...

R. McLaughlin et al.

A comparison of a similarity-based and a feature-based 2-D–3-D registration method for neurointerventional use

Med. Imaging IEEE Trans.

(2005)

Cited by (13)

Topological recovery for non-rigid 2D/3D registration of coronary artery models
2021, Computer Methods and Programs in Biomedicine
Citation Excerpt :
Additionally, the strategies involved in the transformation method primarily affect the precision of registration. Although many 2D-3D registration methods have been reported for bones and liver arteries [12,13], these objects are relatively static compared to coronary arteries [14]; i.e., the beating of the heart and respiration induce the motion of coronary arteries during intra-operative imaging. A 2D-3D real-time image registration technique using deep learning has recently been proposed, but the method is limited to rigid-body registration and requires synthetic learning data [15].
Background and Objective: Intra-operative X-ray angiography, the current standard method for visualizing and diagnosing cardiovascular disease, is limited in its ability to provide essential 3D information. These limitations are disadvantages in treating patients. For example, it is a cause of lowering the success rate of interventional procedures. Here, we propose a novel 2D-3D non-rigid registration method to understand vascular geometry during percutaneous coronary intervention. Methods: The proposed method uses the local bijection pair distance as a cost function to minimize the effect of inconsistencies from center-line extraction. Moreover, novel cage-based 3D deformation and multi-threaded particle swarm optimization are utilized to implement real-time registration. We evaluated the proposed method for 154 examinations from 10 anonymous patients by coverage percentage, comparing the average distance of the 2D extracted center-line with that of the registered 3D center-line. Results: The proposed 2D-3D non-rigid registration method achieved an average distance of 1.98 mm with a 0.54 s computation time. Additionally, in aiming to reduce the uncertainty of XA images, we used the proposed method to retrospectively visualize the connections between 2D vascular segments and the distal part of occlusions. Conclusions: Ultimately, the proposed 2D/3D non-rigid registration method can successfully register the 3D center-line of coronary arteries with corresponding 2D XA images, and is computationally sufficient for online usage. Therefore, this method can improve the success rate of such procedures as a percutaneous coronary intervention and provide the information necessary to diagnose cardiovascular diseases better.
Deep multispectral image registration network
2021, Computerized Medical Imaging and Graphics
Multispectral imaging (MSI) of the ocular fundus provides a sequence of narrow-band images to show the different depths in the retina and choroid. One challenge in analyzing MSI images comes from the image-to-image spatial misalignment, which occurs because the acquisition time of eye MSI images is commonly longer than the natural time scale of the eye's saccadic movement. It is necessary to align images because ophthalmologists usually overlay two of the images to analyze specific features when analyzing MSI images. In this paper, we propose a weakly supervised MSI image registration network, called MSI-R-NET, for multispectral fundus image registration. Compared to other deep-learning-based registration methods, MSI-R-NET utilizes the blood vessel segmentation label to provide spatial correspondence. In addition, we employ a feature equilibrium module to connect the aggregating layers better, and propose a multiresolution auto-context structure to adapt the registration task. In the testing stage, given a new pair of MSI images, the trained model can predict the pixelwise spatial correspondence without labeled blood vessel information. The experimental results demonstrate that the proposed segmentation-driven registration method is highly accurate.
Image processing for precise three-dimensional registration and stitching of thick high-resolution laser-scanning microscopy image stacks
2018, Computers in Biology and Medicine
Citation Excerpt :
Such a perspective could be judged as relatively narrow in a general computer vision context. However, as recently illustrated in this journal [14–17], it is actually specifically meaningful in biomedical imaging where important communities of life scientists work on the same types of samples. In this study, we have developed an optimized automatic registration and stitching algorithm, 2D-SIFT-in-3D-Space, specifically adapted for thick high-resolution laser-scanning microscopy image stacks.
The possible depth of imaging of laser-scanning microscopy is limited not only by the working distances of objective lenses but also by image degradation caused by attenuation and diffraction of light passing through the specimen. To tackle this problem, one can either flip the sample to record images from both sides of the specimen or consecutively cut off shallow parts of the sample after taking serial images of certain thickness. Multiple image substacks acquired in these ways should be combined afterwards to generate a single stack. However, subtle movements of samples during image acquisition cause mismatch not only in the translation along x-, y-, and z-axes and rotation around z-axis but also tilting around x- and y-axes, making it difficult to register the substacks precisely. In this work, we developed a novel approach called 2D-SIFT-in-3D-Space using Scale Invariant Feature Transform (SIFT) to achieve robust three-dimensional matching of image substacks. Our method registers the substacks by separately fixing translation and rotation along x-, y-, and z-axes, through extraction and matching of stable features across two-dimensional sections of the 3D stacks. To validate the quality of registration, we developed a simulator of laser-scanning microscopy images to generate a virtual stack in which noise levels and rotation angles are controlled with known parameters. We illustrate quantitatively the performance of our approach by registering an entire brain of Drosophila melanogaster consisting of 800 sections. Our approach is also demonstrated to be extendable to other types of data that share large dimensions and need of fine registration of multiple image substacks. This method is implemented in Java and distributed as ImageJ/Fiji plugin. The source code is available via Github (http://www.creatis.insa-lyon.fr/site7/fr/MicroTools).
Technical and Clinical Progress on Robot-Assisted Endovascular Interventions: A Review
2023, Micromachines
A Survey on the Progress of Computer-Assisted Vascular Intervention
2022, Jisuanji Fuzhu Sheji Yu Tuxingxue Xuebao/Journal of Computer-Aided Design and Computer Graphics
Techniques and Algorithms for Hepatic Vessel Skeletonization in Medical Images: A Survey
2022, Entropy

View all citing articles on Scopus

Jeongjin Lee is an associate professor in the School of Computer Science and Engineering at Soongsil University, Korea. He received the B.S. degree in mechanical engineering, and the M.S. and Ph.D. degrees in computer science and engineering from Seoul National University, Korea in 2002 and 2008, respectively. He worked as a research professor in Department of Radiology, University of Ulsan, Korea (Seoul Asan Medical Center) from October 2007 to February 2009. He also worked as a C.T.O. in Clinical Imaging Solution from January 2008 to May 2010. He worked as an assistant professor in Department of Digital Media, The Catholic University of Korea from March 2009 to February 2013. His research interests include image registration, image segmentation, computer-aided diagnosis, computer-aided surgery, and virtual endoscopy.

Jin Wook Chung is the section chief of Interventional Radiology at Seoul National University Hospital. He graduated from Seoul National University College of Medicine in 1985, completed his residency training in Radiology at Seoul National University Hospital in 1989. His primary clinical and research interest is intraarterial management of hepatocellular carcinoma, especially chemoembolization. Until now, he performed more than 20,000 chemoembolization procedures during the past 20 years and published many scientific papers about the basic vascular anatomy, techniques and outcome of chemoembolization in international journals. He also conducted multiple preclinical studies to test and validate new concepts for intraarterial management of liver tumors. Currently, he is one of key speakers in international scientific meetings of interventional radiology.

Yeong-Gil Shin is a professor in the School of Computer Science and Engineering and the director of the Computer Graphics and Image Processing Laboratory at Seoul National University, Korea. He received the BS and MS degrees in computer science from Seoul National University, and the PhD degree in computer science from the University of Southern California, USA in 1990. His research interests include computer graphics, volume visualization, medical imaging, and image processing.

View full text

Locally adaptive 2D–3D registration using vascular structure model for liver catheterization

Highlights

Abstract

Introduction

Section snippets

Construction of 3D vascular structure model

Locally adaptive registration

Experimental results

Conclusion

Conflict of interest statement

Acknowledgments

Med. Image Anal.

Med. Image Anal.

Image Vis. Comput.

New CTA protocol and 2D–3D registration method for liver catheterization

Deformable 2D–3D registration of vascular structures in a one view scenario

Med. Imaging IEEE Trans.

3D/2D model-to-image registration applied to TIPS surgery

An efficient graph-based deformable 2D/3D registration algorithm with applications for abdominal aortic aneurysm interventions

Vesselness-based 2D–3D registration of the coronary arteries

Int. J. Comput.-Assist. Radiol. Surg.

Nonrigid 2D/3D registration of coronary artery models with live fluoroscopy for guidance of cardiac interventions

Med. Imaging IEEE Trans.

2D–3D registration of coronary angiograms for cardiac procedure planning and guidance

Med. Phys.

A comparison of a similarity-based and a feature-based 2-D–3-D registration method for neurointerventional use

Med. Imaging IEEE Trans.