Skip to main content
SpringerLink
Log in

Augmented reality in neurovascular surgery: feasibility and first uses in the operating room

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

The aim of this report is to present a prototype augmented reality (AR) intra-operative brain imaging system. We present our experience of using this new neuronavigation system in neurovascular surgery and discuss the feasibility of this technology for aneurysms, arteriovenous malformations (AVMs), and arteriovenous fistulae (AVFs).

Methods

We developed an augmented reality system that uses an external camera to capture the live view of the patient on the operating room table and to merge this view with pre-operative volume-rendered vessels. We have extensively tested the system in the laboratory and have used the system in four surgical cases: one aneurysm, two AVMs and one AVF case.

Results

The developed AR neuronavigation system allows for precise patient-to-image registration and calibration of the camera, resulting in a well-aligned augmented reality view. Initial results suggest that augmented reality is useful for tailoring craniotomies, localizing vessels of interest, and planning resection corridors.

Conclusion

Augmented reality is a promising technology for neurovascular surgery. However, for more complex anomalies such as AVMs and AVFs, better visualization techniques that allow one to distinguish between arteries and veins and determine the absolute depth of a vessel of interest are needed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. https://www.opengl.org/documentation/glsl/.

References

  1. Edwards P, Hawkes D, Hill D, Jewell D, Spink R, Strong A, Gleeson M (1995) Augmentation of reality using an operating microscope for otolaryngology and neurosurgical guidance. J Image Guid Surg 1(3):172–178

    Article  CAS  PubMed  Google Scholar 

  2. Stieg P, Batjer H, Samson D (2007) Intracranial arteriovenous malformations. CRC Press, New York

    Google Scholar 

  3. Hope TA, Hope MD, Purcell DD, von Morze C, Vigneron DB, Alley MT, Dillon WP (2010) Evaluation of intracranial stenoses and aneurysms with accelerated 4D flow. Magn Reson Imaging 28:41–46

    Article  PubMed  Google Scholar 

  4. Spetzler RF, Kondziolka DS, Higashida RT, Kalani MYS (2014) Comprehensive management of arteriovenous malformations of the brain and spine. Cambridge University Press, Cambridge

    Book  Google Scholar 

  5. Kersten-Oertel M, Jannin P, Collins DL (2013) The state of the art of visualization in mixed reality image guided surgery. Comput Med Imaging Graph 37:98–112

    Article  PubMed  Google Scholar 

  6. Kawamata T, Iseki H, Shibasaki T, Hori T (2002) Endoscopic augmented reality navigation system for endonasal transsphenoidal surgery to treat pituitary tumors: technical note. Neurosurgery 50:1393–1397

    PubMed  Google Scholar 

  7. Cabrilo I, Bijlenga P, Schaller K (2014) Augmented reality in the surgery of cerebral aneurysms: a technical report. Neurosurgery 10(Suppl. 2):252–260; discussion 260–261

  8. Cabrilo I, Bijlenga P, Schaller K (2014) Augmented reality in the surgery of cerebral arteriovenous malformations: technique assessment and considerations. Acta Neurochir (Wien) 156:1769–1774

    Article  Google Scholar 

  9. Paul P, Fleig O, Jannin P (2005) Augmented virtuality based on stereoscopic reconstruction in multimodal image-guided neurosurgery: methods and performance evaluation. IEEE Trans Med Imaging 24:1500–1511

    Article  PubMed  Google Scholar 

  10. Rosahl SK, Shahidi R (2008) The virtual operating field—how image guidance can become integral to microneurosurgery. In: Ramina R, de Aguiar PHP, Tatagiba M (eds) Samii’s essentials in neurosurgery. Springer, Berlin, pp 11–20

  11. Shahidi R, Bax MR, Maurer CR Jr, Johnson JA, Wilkinson EP, Bai W, West JB, Citardi MJ, Manwaring KH, Khadem R (2002) Implementation, calibration and accuracy testing of an image-enhanced endoscopy system. Med Imaging IEEE Trans 21:1524–1535

    Article  Google Scholar 

  12. King AP, Edwards PJ, Maurer CR, de Cunha DA, Gaston RP, Clarkson M, Hill DLG, Hawkes DJ, Fenlon MR, Strong AJ, Cox TCS, Gleeson MJ (2000) Stereo augmented reality in the surgical microscope. Presence-Teleoperators Virtual Environ 9:360–368

    Article  Google Scholar 

  13. Gleason PL, Kikinis R, Altobelli D, Wells W, Alexander E, Black PM, Jolesz F (1994) Video registration virtual reality for nonlinkage stereotactic surgery. Stereotact Funct Neurosurg 63:139–143

  14. Edwards PJ, King AP, Hawkes DJ, Fleig O, Maurer CR Jr, Hill DL, Fenlon MR, de Cunha DA, Gaston RP, Chandra S, Mannss J, Strong AJ, Gleeson MJ, Cox TC (1999) Stereo augmented reality in the surgical microscope. Stud Health Technol Inform 62:102–108

  15. Birkfellner W, Figl M, Matula C, Hummel J, Hanel R, Imhof H, Wanschitz F, Wagner A, Watzinger F, Bergmann H (2003) Computer-enhanced stereoscopic vision in a head-mounted operating binocular. Phys Med Biol 48:N49–N57

    Article  PubMed  Google Scholar 

  16. Birkfellner W, Figl M, Huber K, Watzinger F, Wanschitz F, Hummel J, Hanel R, Greimel W, Homolka P, Ewers R, Bergmann H (2002) A head-mounted operating binocular for augmented reality visualization in medicine—design and initial evaluation. IEEE Trans Med Imaging 21:991–997

    Article  PubMed  Google Scholar 

  17. Mercier L, Del Maestro RF, Petrecca K, Kochanowska A, Drouin S, Yan CX, Janke AL, Chen SJ, Collins DL (2011) New prototype neuronavigation system based on preoperative imaging and intraoperative freehand ultrasound: system description and validation. Int J Comput Assist Radiol Surg 6:507–522

    Article  PubMed  Google Scholar 

  18. Gerard I, Collins DL (2015) An analysis of tracking error in image guided neurosurgery. IJCARS 9. doi:10.1007/s11548-014-1145-2

  19. Zhang Z (2004) Camera calibration with one-dimensional objects. IEEE Trans Pattern Anal Mach Intell 26:892–899

    Article  PubMed  Google Scholar 

  20. Moghari MH, Abolmaesumi P (2010) Understanding the effect of bias in fiducial localization error on point-based rigid-body registration. IEEE Trans Med Imaging 29:1730–1738

    Article  PubMed  Google Scholar 

  21. Labadie RF, Davis BM, Fitzpatrick JM (2005) Image-guided surgery: what is the accuracy? Curr Opin Otolaryngol Head Neck Surg 13:27–31

    Article  PubMed  Google Scholar 

  22. Liu W, Ding H, Han H, Xue Q, Sun Z, Wang G (2009) The study of fiducial localization error of image in point-based registration. Conf Proc IEEE Eng Med Biol Soc 2009:5088–5091

    PubMed  Google Scholar 

  23. Kersten-Oertel M, Jannin P, Collins DL (2012) DVV: a taxonomy for mixed reality visualization in image guided surgery. IEEE Trans Vis Comput Graph 18:332–352

  24. Drouin S, Kersten-Oertel M, Chen SJS, Collins DL (2012) A realistic test and development environment for mixed reality in neurosurgery. In: Linte CA, Moore JT, Chen ECS, Holmes DR III (eds) Augmented environments for computer-assisted interventions (AE-CAI). Lecture notes in computer science, vol 7264, pp 13–23

  25. Hansen C, Wieferich J, Ritter F, Rieder C, Peitgen HO (2010) Illustrative visualization of 3D planning models for augmented reality in liver surgery. Int J Comput Assist Radiol Surg 5:133–141

    Article  PubMed  Google Scholar 

  26. Johnson L, Edwards P, Hawkes D (2002) Surface transparency makes stereo overlays unpredictable: the implications for augmented reality. Stud Health Technol Inform 94:131–136

    Google Scholar 

  27. Lerotic M, Chung AJ, Mylonas G, Yang GZ (2007) Pq-space based non-photorealistic rendering for augmented reality. Med Image Comput Comput Assist Interv 10:102–109

    PubMed  Google Scholar 

  28. Kersten-Oertel M, Chen SJ, Collins DL (2014) An evaluation of depth enhancing perceptual cues for vascular volume visualization in neurosurgery. IEEE Trans Vis Comput Graph 20(3):391–403

    Article  PubMed  Google Scholar 

  29. Kersten-Oertel M, Chen SJ, Collins DL (2013) An evaluation of depth enhancing perceptual cues for vascular volume visualization in neurosurgery. IEEE Trans Vis Comput Graph 20:391–403

    Article  Google Scholar 

  30. Kersten-Oertel M, Gerard I, Mok K, Sirhan D, Sinclair D, Collins DL (2014) Augmented reality in neurovascular surgery: first experiences. In: Linte CA, Yaniv Z, Fallavollita P, Abolmaesumi P, Holmes DR III (eds) Augmented environments for computer-assisted interventions (AE-CAI). Lecture notes in computer science, vol 8678, pp 80–89

  31. Aschke M, Wirtz CR, Raczkowsky J, Worn H, Kunze S (2003) Augmented reality in operating microscopes for neurosurgical interventions. In: Neural engineering, 2003. Conference Proceedings. First international IEEE EMBS conference on, pp 652–655

Download references

Conflict of interest

Marta Kersten-Oertel, Ian Gerard, Simon Drouin, Kelvin Mok, Denis Sirhan, David Sinclair, and D. Louis Collins declare that they have no conflict of interest.

Author information

Authors and Affiliations

  1. McConnell Brain Imaging Centre, McGill University, Montreal, Canada

    Marta Kersten-Oertel, Ian Gerard, Simon Drouin & D. Louis Collins

  2. Montreal Neurological Institute and Hospital, Montreal, Canada

    Kelvin Mok, Denis Sirhan & David S. Sinclair

Authors
  1. Marta Kersten-Oertel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ian Gerard
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Simon Drouin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kelvin Mok
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Denis Sirhan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. David S. Sinclair
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. D. Louis Collins
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marta Kersten-Oertel.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kersten-Oertel, M., Gerard, I., Drouin, S. et al. Augmented reality in neurovascular surgery: feasibility and first uses in the operating room. Int J CARS 10, 1823–1836 (2015). https://doi.org/10.1007/s11548-015-1163-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-015-1163-8

Keywords

Search

Navigation