Skip to main content
SpringerLink
Log in

A parallelized 4D reconstruction algorithm for vascular structures and motions based on energy optimization

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In this paper, we present a parallel 4D vessel reconstruction algorithm that simultaneously recovers 3D structure, shape, and motion based on multiple views of X-ray angiograms. The fundamental goal is to assist the analysis and diagnosis of interventional surgery in the most efficient way towards interactive and accurate performance. We start with a fully parallelized algorithm to extract vessels as well as their skeletons and topologies from dynamic image sequences. Then, instead of resorting to registration, we present an algorithm to formulate the reconstruction problem as an energy minimization problem with color, coherence, and topology constraints to reconstruct the 3D vessel initially, which is robust to combat noise and incomplete information in images. Next, we incorporate temporal information into our energy optimization framework to track and reconstruct 4D kinematics of the dynamic vessels, which is also capable of recovering previous incomplete and misleading shapes acquired from static images otherwise. We demonstrate our system in coronary arteries reconstruction and movement tracking for percutaneous coronary intervention surgery to help medical practitioners learn about the 3D shapes and their motions of the coronary arteries of specific patient. We envision that our system would be of high assistance for diagnosis and therapy to treat vessel-related diseases in a clinical setting in the near future.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Sarry, L., Boire, J.-Y.: Three-dimensional tracking of coronary arteries from biplane angiographic sequences using parametrically deformable models. IEEE Trans. Med. Imaging 20(12), 1341–1351 (2001)

    Article  Google Scholar 

  2. Cañero, C., Vilariño, F., Mauri, J., Radeva, P.: Predictive (un) distortion model and 3-d reconstruction by biplane snakes. IEEE Trans. Med. Imaging 21(9), 1188–1201 (2002)

    Article  Google Scholar 

  3. Dumay, A.C., Gerbrands, J.J., Reiber, J.H.: Automated extraction, labelling and analysis of the coronary vasculature from arteriograms. Int. J. Card. Imaging 10(3), 205–215 (1994)

    Article  Google Scholar 

  4. Aylward, S.R., Bullitt, E.: Initialization, noise, singularities, and scale in height ridge traversal for tubular object centerline extraction. IEEE Trans. Med. Imaging 21(2), 61–75 (2002)

    Article  Google Scholar 

  5. Sen, A., Lan, L., Doi, K., Hoffmann, K.R.: Quantitative evaluation of vessel tracking techniques on coronary angiograms. Med. Phys. 26(5), 698–706 (1999)

    Article  Google Scholar 

  6. Kirbas, C., Quek, F.: A review of vessel extraction techniques and algorithms. ACM Comput. Surv. (CSUR) 36(2), 81–121 (2004)

    Article  Google Scholar 

  7. Hoover, A., Kouznetsova, V., Goldbaum, M.: Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response. IEEE Trans. Med. Imaging 19(3), 203–210 (2000)

    Article  Google Scholar 

  8. Li, Q., You, J., Zhang, D.: Vessel segmentation and width estimation in retinal images using multiscale production of matched filter responses. Expert Syst. Appl. 39(9), 7600–7610 (2012)

    Article  Google Scholar 

  9. Frangi, A.F., Niessen, W.J., Vincken, K.L., Viergever, M.A.: Multiscale vessel enhancement filtering. In: MICCAI’98, Springer. pp. 130–137 (1998)

  10. Condurache, A.-P., Aach, T.: Vessel segmentation in angiograms using hysteresis thresholding. In: IAPR Conference on Machine Vision Applications, pp. 269–272 (2005)

  11. Zhang, B., Zhang, L., Zhang, L., Karray, F.: Retinal vessel extraction by matched filter with first-order derivative of gaussian. Comput. Biol. Med. 40(4), 438–445 (2010)

    Article  Google Scholar 

  12. Van Uitert, R., Bitter, I.: Subvoxel precise skeletons of volumetric data based on fast marching methods. Med. Phys. 34(2), 627–638 (2007)

    Article  Google Scholar 

  13. Hassouna, M., Farag, A.: Multistencils fast marching methods: a highly accurate solution to the eikonal equation on cartesian domains. IEEE Trans. Pattern Anal. Mach. Intell. 29(9), 1563–1574 (2007). doi:10.1109/TPAMI.2007.1154

    Article  Google Scholar 

  14. Zhang, T.Y., Suen, C.Y.: A fast parallel algorithm for thinning digital patterns. Commun. ACM 27(3), 236–239 (1984)

    Article  Google Scholar 

  15. Wellnhofer, E., Wahle, A., Mugaragu, I., Gross, J., Oswald, H., Fleck, E.: Validation of an accurate method for three-dimensional reconstruction and quantitative assessment of volumes, lengths and diameters of coronary vascular branches and segments from biplane angiographic projections. Int. J. Card. Imaging 15(5), 339–353 (1999)

    Article  Google Scholar 

  16. Messenger, J.C., Chen, S.J., Carroll, J.D., Burchenal, J., Kioussopoulos, K., Groves, B.M.: 3d coronary reconstruction from routine single-plane coronary angiograms: clinical validation and quantitative analysis of the right coronary artery in 100 patients. Int. J. Card. Imaging 16(6), 413–427 (2000)

    Article  Google Scholar 

  17. Gollapudi, R.R., Valencia, R., Lee, S.S., Wong, G.B., Teirstein, P.S., Price, M.J.: Utility of three-dimensional reconstruction of coronary angiography to guide percutaneous coronary intervention. Catheter. Cardiovasc. Interv. 69(4), 479–482 (2007)

    Article  Google Scholar 

  18. Movassaghi, B., Rasche, V., Grass, M., Viergever, M.A., Niessen, W.J.: A quantitative analysis of 3-d coronary modeling from two or more projection images. IEEE Trans. Med. Imaging 23(12), 1517–1531 (2004)

    Article  Google Scholar 

  19. Sprague, K., Drangova, M., Lehmann, G., Slomka, P., Levin, D., Chow, B., et al.: Coronary x-ray angiographic reconstruction and image orientation. Med. Phys. 33, 707–718 (2006)

    Article  Google Scholar 

  20. Hansis, E., Schäfer, D., Dössel, O., Grass, M.: Projection-based motion compensation for gated coronary artery reconstruction from rotational x-ray angiograms. Phys. Med. Biol. 53(14), 3807–3820 (2008)

    Article  Google Scholar 

  21. Nguyen, T.V., Sklansky, J.: Reconstructing the 3-d medial axes of coronary arteries in single-view cineangiograms. IEEE Trans. Med. Imaging 13(1), 61–73 (1994)

    Article  Google Scholar 

  22. Fessler, J.A., Macovski, A.: Object-based 3-d reconstruction of arterial trees from magnetic resonance angiograms. IEEE Trans. Med. Imaging 10, 25–39 (1991)

    Article  Google Scholar 

  23. Liu, I., Sun, Y.: Fully automated reconstruction of three-dimensional vascular tree structures from two orthogonal views using computational algorithms and productionrules. Opt. Eng. 31(10), 2197–2207 (1992)

    Article  Google Scholar 

  24. Weng, J., Ahuja, N., Huang, T.S.: Optimal motion and structure estimation. IEEE Trans. Pattern Anal. Mach. Intell. 15(9), 864–884 (1993)

    Article  Google Scholar 

  25. Chen, S.-Y.J., Hoffmann, K.R., Carroll, J.D.: Three-dimensional reconstruction of coronary arterial tree based on biplane angiograms. Proc. SPIE Med. Imag. Image Process. 2710, 103–114 (1996)

    Google Scholar 

  26. Chen, S.Y.J., Metz, C.E.: Improved determination of biplane imaging geometry from two projection images and its application to 3-d reconstruction of coronary arterial trees. Med. Phys. 24, 633–654 (1997)

    Article  Google Scholar 

  27. Ruan, S., Bruno, A., Coatrieux, J.-L.: Three-dimensional motion and reconstruction of coronary arteries from biplane cineangiography. Image Vis. Comput. 12(10), 683–689 (1994)

    Article  Google Scholar 

  28. Puentes, J., Roux, C., Garreau, M., Coatrieux, J.-L.: Dynamic feature extraction of coronary artery motion using dsa image sequences. IEEE Trans. Med. Imaging 17(6), 857–871 (1998)

    Article  Google Scholar 

  29. Ingrassia, C., Windyga, P., Shah, M.: Segmentation and tracking of coronary arteries. In: BMES/EMBS Conference, vol. 1, IEEE, pp. 203–203 (1999)

  30. Chen, S.-Y., Carroll, J.D.: Kinematic and deformation analysis of 4-d coronary arterial trees reconstructed from cine angiograms. IEEE Trans. Med. Imaging 22(6), 710–721 (2003)

    Article  Google Scholar 

  31. Naegel, B., Passat, N., Ronse, C.: Grey-level hit-or-miss transforms-Part I: unified theory. Pattern Recognit. 40(2), 635–647 (2007)

    Article  MATH  Google Scholar 

  32. Shechter, G., Devernay, F., Coste-Manière, E., Quyyumi, A., McVeigh, E.R.: Three-dimensional motion tracking of coronary arteries in biplane cineangiograms. IEEE Trans. Med. Imaging 22(4), 493–503 (2003)

    Article  Google Scholar 

  33. Shechter, G., Resar, J.R., McVeigh, E.R.: Displacement and velocity of the coronary arteries: cardiac and respiratory motion. IEEE Trans. Med. Imaging 25(3), 369–375 (2006)

    Article  Google Scholar 

  34. Blondel, C., Malandain, G., Vaillant, R., Ayache, N.: Reconstruction of coronary arteries from a single rotational x-ray projection sequence. IEEE Trans. Med. Imaging 25(5), 653–663 (2006)

    Article  Google Scholar 

  35. Bouattour, S., Arndt, R., Paulus, D.: 4D reconstruction of coronary arteries from monoplane angiograms. In: Computer Analysis of Images and Patterns, Springer. pp. 724–731 (2005)

  36. Schoonenberg, G., Florent, R., Lelong, P., Wink, O., Ruijters, D., Carroll, J., ter Haar, B.: Projection-based motion compensation and reconstruction of coronary segments and cardiac implantable devices using rotational x-ray angiography. Med. Image Anal. 13(5), 785–792 (2009)

    Article  Google Scholar 

  37. Geman, S., Geman, D.: Stochastic relaxation, gibbs distributions, and the bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6, 721–741 (1984)

    Article  MATH  Google Scholar 

  38. Lafferty, J.D., McCallum, A., Pereira, F.C.N.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of the Eighteenth International Conference on Machine Learning, ICML ’01, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, pp. 282–289 (2001)

  39. Pearl, J.: Reverend bayes on inference engines: a distributed hierarchical approach. AAAI. pp. 133–136 (1982)

  40. Meltzer, T., Yanover, C., Weiss, Y.: Globally optimal solutions for energy minimization in stereo vision using reweighted belief propagation. In: Proceedings of the Tenth IEEE International Conference on Computer Vision, ICCV ’05, IEEE Computer Society, pp. 428–435 (2005)

  41. Murphy, K.P., Weiss, Y., Jordan, M.I.: Loopy belief propagation for approximate inference: An empirical study. In: Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence, Morgan Kaufmann Publishers Inc. pp. 467–475 (1999)

  42. Szeliski, R., Zabih, R., Scharstein, D., Veksler, O., Kolmogorov, V., Agarwala, A., Tappen, M., Rother, C.: A comparative study of energy minimization methods for markov random fields with smoothness-based priors. IEEE Trans. Pattern Anal. Mach. Intell. 30(6), 1068–1080 (2008)

  43. Tappen, M.F., Freeman, W.T.: Comparison of graph cuts with belief propagation for stereo, using identical mrf parameters. In: Ninth IEEE International Conference on Computer Vision, IEEE, pp. 900–906 (2003)

  44. Potetz, B., Lee, T.S.: Efficient belief propagation for higher-order cliques using linear constraint nodes. Comput. Vis. Image Underst. 112(1), 39–54 (2008)

    Article  Google Scholar 

  45. Brunton, A., Shu, C., Roth, G.: Belief propagation on the gpu for stereo vision. In: The 3rd Canadian Conference on Computer and Robot Vision, pp. 76–76 (2006)

  46. Coughlan, J., Shen, H.: Dynamic quantization for belief propagation in sparse spaces. Comput. Vis. Image Underst. 106(1), 47–58 (2007)

    Article  Google Scholar 

  47. Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient belief propagation for early vision. Int. J. Comput. Vis. 70(1), 41–54 (2006)

    Article  Google Scholar 

  48. Boykov, Y.Y., Jolly, M.-P.: Interactive graph cuts for optimal boundary & region segmentation of objects in nd images. In: Eighth IEEE International Conference on Computer Vision, vol. 1, pp. 105–112 (2001)

  49. Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)

    Article  Google Scholar 

  50. Kolmogorov, V., Zabin, R.: What energy functions can be minimized via graph cuts? IEEE Trans. Pattern Anal. Mach. Intell. 26(2), 147–159 (2004)

    Article  Google Scholar 

  51. Vezhnevets, V., Konouchine, V.: Growcut: Interactive multi-label nd image segmentation by cellular automata. In: Proc. of Graphicon, pp. 150–156 (2005)

  52. Rother, C., Kolmogorov, V., Blake, A.: Grabcut: Interactive foreground extraction using iterated graph cuts. ACM Trans. Gr. (TOG) ACM. 23, 309–314 (2004)

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported in part by National Natural Science Foundation of China (Grant No. 61190120, 61190121, 61190125, 61300068, 61300067), National Science Foundation of USA (Grant No. IIS-0949467, IIS-1047715, and IIS-1049448), the National High Technology Research and Development Program (863 Program) of China (Grant No. 012AA011503), Postdoctoral Science Foundation of China (Grant No. 2013M530512).

Author information

Authors and Affiliations

  1. State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China

    Xinglong Liu, Fei Hou & Aimin Hao

  2. Department of Computer Science, Stony Brook University, Stony Brook, USA

    Hong Qin

Authors
  1. Xinglong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fei Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aimin Hao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hong Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fei Hou.

Electronic supplementary material

Below is the link to the electronic supplementary material.

ESM 1 (MP4 85,218 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, X., Hou, F., Hao, A. et al. A parallelized 4D reconstruction algorithm for vascular structures and motions based on energy optimization. Vis Comput 31, 1431–1446 (2015). https://doi.org/10.1007/s00371-014-1024-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-014-1024-4

Keywords

Search

Navigation