Skip to main content

Advertisement

SpringerLink
Log in

Illustrative visualization of 3D planning models for augmented reality in liver surgery

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Augmented reality (AR) obtains increasing acceptance in the operating room. However, a meaningful augmentation of the surgical view with a 3D visualization of planning data which allows reliable comparisons of distances and spatial relations is still an open request.

Methods

We introduce methods for intraoperative visualization of 3D planning models which extend illustrative rendering and AR techniques. We aim to reduce visual complexity of 3D planning models and accentuate spatial relations between relevant objects. The main contribution of our work is an advanced silhouette algorithm for 3D planning models (distance-encoding silhouettes) combined with procedural textures (distance-encoding surfaces). In addition, we present a method for illustrative visualization of resection surfaces.

Results

The developed algorithms have been embedded into a clinical prototype that has been evaluated in the operating room. To verify the expressiveness of our illustration methods, we performed a user study under controlled conditions. The study revealed a clear advantage in distance assessment with the proposed illustrative approach in comparison to classical rendering techniques.

Conclusion

The presented illustration methods are beneficial for distance assessment in surgical AR. To increase the safety of interventions with the proposed approach, the reduction of inaccuracies in tracking and registration is a subject of our current research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Schenk A, Zidowitz S, Bourquain H, Hindennach M, Hansen C, Hahn H, Peitgen HO (2008) Clinical relevance of model based computer-assisted diagnosis and therapy. Proceedings of SPIE Medical Imaging, 6915(1):691502_1-19. doi:10.1117/12.780270

  2. Oldhafer KJ, Stavrou GA, Prause G, Peitgen HO, Lueth TC, Weber S (2009) How to operate a liver tumor you cannot see. Langenbecks Arch Surg 394(3): 489–494. doi:10.1007/s00423-009-0469-9

    Article  PubMed  Google Scholar 

  3. Hildebrand P, Schlichting S, Martens V, Besiveric A, Roblick U, Roblick U, Mirow L, Buerk C, Schweikard A, Bruch H (2008) Prototype of an intraoperative navigation and documentation system for laparoscopic radiofrequency ablations: first experiences. Eur J Surg Oncol 34(4): 418–421. doi:10.1016/j.ejso.2007.04.017

    CAS  PubMed  Google Scholar 

  4. Beller S, Eulenstein S, Lange T, Hünerbein M, Schlag PM (2009) Upgrade of an optical navigation system with a permanent electromagnetic position control: a first step towards “navigated control” for liver surgery. J Hepatobiliary Pancreat Surg 16(2): 165–170. doi:10.1007/s00534-008-0040-z

    Article  PubMed  Google Scholar 

  5. Cash DM, Miga MI, Glasgow SC, Dawant BM, Clements LW, Cao Z, Galloway RL, Chapman WC (2007) Concepts and preliminary data toward the realization of image-guided liver surgery. J Gastrointest Surg 11(7): 844–859. doi:10.1007/s11605-007-0090-6

    Article  PubMed  Google Scholar 

  6. Ritter F, Hansen C, Dicken V, Konrad O, Preim B, Peitgen HO (2006) Real-time illustration of vascular structures. IEEE Trans Vis Comput Graph J 12(5): 877–884. doi:10.1109/TVCG.2006.172

    Article  Google Scholar 

  7. Sielhorst T, Feuerstein M, Navab N (2008) Advanced Medical Displays: A literature review of augmented reality. IEEE/OSA J Disp Technol; Special Issue on Medical Displays 4(4): 451–467. doi:10.1109/JDT.2008.2001575

    Google Scholar 

  8. Ayache N (2003) Epidaure: a research project in medical image analysis, simulation and robotics at INRIA. IEEE Trans Med Imaging 22(10): 1185–1201. doi:10.1109/TMI.2003.812863

    Article  PubMed  Google Scholar 

  9. Samset E, Schmalstieg D, Vander SJ, Freudenthal A, Declerck J, Casciaro S, Rideng Ø, Gersak B (2008) Augmented reality in surgical procedures. Proc SPIE Med Imaging 6806(1):68060K_1-12. doi:10.1117/12.784155

    Google Scholar 

  10. Nicolau SA, Pennec X, Soler L, Buy X, Gangi A, Ayache N, Marescaux J (2009) An augmented reality system for liver thermal ablation: design and evaluation on clinical cases. Med Image Anal 13(3): 494–506. doi:10.1016/j.media.2009.02.003

    Article  CAS  PubMed  Google Scholar 

  11. Feuerstein M, Mussack T, Heining SM, Navab N (2008) Intraoperative laparoscope augmentation for port placement and resection planning in minimally invasive liver resection. IEEE Trans Med Imaging 27(1): 355–369. doi:10.1109/TMI.2007.907327

    Article  PubMed  Google Scholar 

  12. Scheuering M, Schneider A, Schenk A, Preim B, Greiner G (2003) Intraoperative augmented reality for minimally invasive liver interventions. Proc SPIE Med Imaging 5029(1): 407–417. doi:10.1117/12.480212

    Google Scholar 

  13. Marescaux J, Rubino F, Arenas M, Mutter D, Soler L (2004) Augmented-reality-assisted laparoscopic adrenalectomy. J Am Med Assoc 292(18): 2214–2215. doi:10.1001/jama.292.18.2214-c

    Article  CAS  Google Scholar 

  14. Krempien R, Hoppe H, Kahrs L, Daeuber S, Schorr O, Eggers G, Bischof M, Munter MW, Debus J, Harms W (2008) Projector-based augmented reality for intuitive intraoperative guidance in image-guided 3D interstitial brachytherapy. Int J Radiat Oncol Biol Phys 70(3): 944–952. doi:10.1016/j.ijrobp.2007.10.048

    PubMed  Google Scholar 

  15. Riechmann M, Kahrs LA, Hoppe H, Ulmer C, Raczkowsky J, Lamade W, Wörn H (2006) Visualisierungskonzept für die projektorbasierte Erweiterte Realität in der Leberchirurgie. Proc BMT 209(1): 1–2

    Google Scholar 

  16. Glossop ND, Wang Z (2003) Laser projection augmented reality system for computer-assisted surgery. Int Congr Ser 1256(1): 65–71. doi:10.1016/S0531-5131(03)00515-6

    Article  Google Scholar 

  17. Navab N, Feuerstein M, Bichlmeier C (2007) Laparoscopic virtual mirror—new interaction paradigm for monitor based augmented reality. Virtual Reality Conference IEEE, pp 43–50. doi:10.1109/VR.2007.352462

  18. Lerotic M, Chung AJ, Mylonas GP, Yang GZ (2007) Pq-space based non-photorealistic rendering for augmented reality. Proc MICCAI 4792: 102–109. doi:10.1007/978-3-540-75759-7

    Google Scholar 

  19. Bichlmeier C, Wimmer F, Heining SM, Navab N (2007) Contextual anatomic mimesis: hybrid in situ visualization method for improving multi-sensory depth perception in medical augmented reality. In: ISMAR ‘07: Proceedings of the 2007 6th IEEE and ACM international symposium on mixed and augmented reality, pp 1–10. doi:10.1109/ISMAR.2007.4538837

  20. Strothotte T, Schlechtweg S (2002) Non-photorealistic computer graphics. Morgan Kaufmann, San Francisco

    Google Scholar 

  21. Bruckner S (2008) Interactive illustrative volume visualization. PhD thesis, University of Technology, Vienna, Austria

  22. Fischer J, Bartz D (2005) Stylized augmented reality for improved immersion. Proceedings of the IEEE Conference on Virtual Reality, pp 195–202. doi:10.1109/VR.2005.1492774

  23. Freudenberg B (2004) Real-time stroke-based halftoning. PhD thesis, Otto-von-Guericke University Magdeburg, Germany

  24. Isenberg T, Halper N, Strothotte T (2002) Stylizing silhouettes at interactive rates: from silhouette edges to silhouette strokes. Computer Graphics Forum. 21(3): 249–258. doi:10.1111/1467-8659.00584

    Article  Google Scholar 

  25. Ericsson K, Simon H (1993) Protocol Analysis: Verbal Reports as Data. MIT Press, Boston

    Google Scholar 

  26. Blum J, Padoy N, Feußner H, Navab N (2008) Workflow mining for visualization and analysis of surgeries. Int J Comput Assist Radiol Surg 3(5): 379–386. doi:10.1007/s11548-008-0239-0

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Fraunhofer MEVIS, Insitute for Medical Image Computing, Bremen, Germany

    Christian Hansen, Jan Wieferich, Felix Ritter, Christian Rieder & Heinz-Otto Peitgen

Authors
  1. Christian Hansen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jan Wieferich
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Felix Ritter
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Christian Rieder
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Heinz-Otto Peitgen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christian Hansen.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hansen, C., Wieferich, J., Ritter, F. et al. Illustrative visualization of 3D planning models for augmented reality in liver surgery. Int J CARS 5, 133–141 (2010). https://doi.org/10.1007/s11548-009-0365-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-009-0365-3

Keywords

Search

Navigation