Skip to main content
SpringerLink
Log in

Projective biomechanical depth matching for soft tissue registration in laparoscopic surgery

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

A key component of computer- assisted surgery systems is the accurate and robust registration of preoperative planning data with intraoperative sensor data. In laparoscopic surgery, this image-based registration remains challenging due to soft tissue deformations. This paper presents a novel approach for biomechanical soft tissue registration of preoperative CT data with stereo endoscopic image data.

Methods

The proposed method consists of two registrations steps. First, we use a 3D surface mosaic from partial surfaces reconstructed from stereo endoscopic images to initially align the biomechanical model with the intraoperative position and shape of the organ. After this initialization, the biomechanical model is projected onto newly captured surfaces, resulting in displacement boundary conditions, which in turn are used to update the biomechanical model.

Results

The method is evaluated in silico, using a human liver model, and in vivo, using porcine data. The quantitative in silico data shows a stable behaviour of the biomechanical model and root-mean-square deviation of volume vertices of under 3 mm with adjusted biomechanical parameters.

Conclusion

This work contributes a fully automatic featureless non-rigid registration approach. The results of the in silico and in vivo experiments suggest that our method is able to handle dynamic deformations during surgery. Additional experiments, especially regarding human tissue behaviour, are an important next step towards clinical applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. https://www.sofa-framework.org/.

  2. http://mitk.org.

  3. http://doc.cgal.org/latest/Manual/packages.html#PartMeshing.

References

  1. Nicolau S, Soler L, Mutter D, Marescaux J (2011) Augmented reality in laparoscopic surgical oncology. Surg Oncol 20(3):189–201

    Article  PubMed  Google Scholar 

  2. Maier-Hein L, Mountney P, Bartoli A, Elhawary H, Elson D, Groch A, Kolb A, Rodrigues M, Sorger J, Speidel S, Stoyanov D (2013) Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery. Med Image Anal 17(8):974–996

    Article  CAS  PubMed  Google Scholar 

  3. Maier-Hein L, Groch A, Bartoli A, Bodenstedt S, Boissonnat G, Chang PL, Clancy NT, Elson DS, Haase S, Heim E, Hornegger J (2014) Comparative validation of single-shot optical techniques for laparoscopic 3-D surface reconstruction. IEEE Trans Med Imaging 33(10):1913–1930

    Article  CAS  PubMed  Google Scholar 

  4. Allan M, Kapoor A, Mewes P, Mountney P (2015) Non rigid registration of 3D images to laparoscopic video for image guided surgery. International workshop on computer-assisted and robotic endoscopy. Springer International Publishing, Berlin

    Google Scholar 

  5. Stefansic JD, Herline AJ, Shyr Y, Chapman WC, Fitzpatrick JM, Dawant BM, Galloway RL (2002) Registration of physical space to laparoscopic image space for use in minimally invasive hepatic surgery. In: 5th IEEE EMBS international summer school on biomedical imaging. IEEE, p 12

  6. Su LM, Vagvolgyi BP, Agarwal R, Reiley CE, Taylor RH, Hager GD (2009) Augmented reality during robot-assisted laparoscopic partial nephrectomy: toward real-time 3D-CT to stereoscopic video registration. Urology 73(4):896–900

    Article  PubMed  Google Scholar 

  7. Herline AJ, Stefansic JD, Debelak JP, Hartmann SL, Pinson CW, Galloway RL, Chapman WC (1999) Image-guided surgery: preliminary feasibility studies of frameless stereotactic liver surgery. Arch Surg 134(6):644–650

    Article  CAS  PubMed  Google Scholar 

  8. Gupta T, Shin D, Sivagnanadasan N, Hoiem D (2016) 3DFS: deformable dense depth fusion and segmentation for object reconstruction from a handheld camera. arXiv:1606.05002

  9. Newcombe RA, Fox D, Seitz SM (2015) Dynamicfusion: reconstruction and tracking of non-rigid scenes in real-time. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 343–352

  10. Bregler C, Hertzmann A, Biermann H (2000) Recovering non-rigid 3D shape from image streams. In: Proceedings of IEEE conference on computer vision and pattern recognition, 2000, vol 2. IEEE, pp 690–696

  11. Rucker DC, Wu Y, Clements L, Ondrake J, Pheiffer T, Simpson A, Jarnagin W, Miga M (2014) A mechanics-based nonrigid registration method for liver surgery using sparse intraoperative data. IEEE Trans Med Imaging 33(1):147–158

    Article  PubMed  Google Scholar 

  12. Plantefve R, Peterlik I, Haouchine N, Cotin S (2015) Patient-specific biomechanical modeling for guidance during minimally-invasive hepatic surgery. Ann. Biomed. Eng. 44(1):139–153

    Article  Google Scholar 

  13. Suwelack S, Röhl S, Bodenstedt S, Reichard D, Dillmann R, Thiago S, Maier-Hein L, Wagner M, Wnscher J, Kenngott H, Müller BP, Speidel S (2014) Physics-based shape matching for intraoperative image guidance. Med Phys 41(11):111901

    Article  PubMed  Google Scholar 

  14. Pratt P, Stoyanov D, Visentini-Scarzanella M, Yang G (2010) Dynamic guidance for robotic surgery using image-constrained biomechanical models. In: Medical image computing and computer-assisted intervention-MICCAI, pp 77–85

  15. Oktay O, Zhang L, Mansi T, Mountney P, Mewes P, Nicolau S, Soler L, Chefd’hotel C (2013) Biomechanically driven registration of pre-to intra-operative 3D images for laparoscopic surgery. In: Medical image computing and computer-assisted intervention-MICCAI, pp 1–9

  16. Nosrati MS, Abugharbieh R, Peyrat JM, Abinahed J, Al-Alao O, Al-Ansari A, Hamarneh G (2016) Simultaneous multi-structure segmentation and 3D nonrigid pose estimation in image-guided robotic surgery. IEEE Trans Med Imaging 35(1):1–9

    Article  PubMed  Google Scholar 

  17. Collins T, Bartoli A, Bourdel N, Canis M (2016) Robust, real-time, dense and deformable 3D organ tracking in laparoscopic videos. In: International conference on medical image computing and computer-assisted intervention. Springer International Publishing (2016)

  18. dos Santos TR, Seitel A, Kilgus T, Suwelack S, Wekerle AL, Kenngott H, Speidel S, Schlemmer H, Meinzer H, Heimann T, Maier-Hein L (2014) Pose-independent surface matching for intra-operative soft-tissue marker-less registration. Med Image Anal 18(7):1101–1114

    Article  PubMed  Google Scholar 

  19. Rusu RB, Blodow N, Beetz M (2009) Fast point feature histograms (FPFH) for 3D registration. In: IEEE international conference on robotics and automation, 2009. ICRA’09. IEEE

  20. Bodenstedt S, Goertler J,Wagner M, Kenngott H, Mïller-Stich B, Dillmann R, Speidel S (2016) Superpixel-based structure classification for laparoscopic surgery. In: Webster RJ, Yaniv ZR (eds) Medical Imaging 2016: Image-Guided Procedures, Robotic Interventions, and Modeling, vol 9786. Bellingham. doi:10.1117/12.2216750

  21. Roehl S, Bodenstedt S, Suwelack S, Kenngott H, Müller-Stich BP, Dillmann R, Speidel S (2012) Dense GPU-enhanced surface reconstruction from stereo endoscopic images for intraoperative registration. Med Phys 39(3):1632–1645

    Article  Google Scholar 

  22. Nolden M, Zelzer S, Seitel A, Wald D, Mller M, Franz AM, Maleike D, Fangerau M, Baumhauer M, Maier-Hein L, Maier-Hein KH, Meinzer HP, Wolf I (2013) The medical imaging interaction toolkit: challenges and advances. Int J Comput Assist Radiol Surg 8:607–620

    Article  PubMed  Google Scholar 

  23. Yeh WC, Li PC, Jeng YM, Hsu HC, Kuo PL, Li ML, Lee PH (2002) Elastic modulus measurements of human liver and correlation with pathology. Ultrasound Med Biol 28(4):467–474

    Article  PubMed  Google Scholar 

  24. Fung YC (1981) Biomechanics: mechanical properties of living tissues. Springer, New York

    Book  Google Scholar 

  25. Reichard D, Bodenstedt S, Suwelack S, Mayer B, Preukschas A, Wagner M, Kenngott H, Müller-Stich B, Dillmann R, Speidel S (2015) Intraoperative on-the-fly organ-mosaicking for laparoscopic surgery. J Med Imaging 2(4):045001

    Article  Google Scholar 

  26. Shi H (2007) Finite element modeling of soft tissue deformation. ProQuest Dissertations Publishing, University of Louisville

Download references

Acknowledgements

The present research was supported by the Klaus Tschira Foundation.

Author information

Authors and Affiliations

  1. Karlsruhe Institute of Technology, Adenauerring 2, Bldg. 50.20, Karlsruhe, Germany

    Daniel Reichard, Dominik Häntsch, Sebastian Bodenstedt, Stefan Suwelack, Rüdiger Dillmann & Stefanie Speidel

  2. Department of General, Abdominal and Transplantation Surgery, University of Heidelberg, Heidelberg, Germany

    Martin Wagner, Hannes Kenngott & Beat Müller-Stich

  3. Junior Group Computer-Assisted Interventions, Division of Medical and Biological Informatics, German Cancer Research Center (DKFZ), Heidelberg, Germany

    Lena Maier-Hein

Authors
  1. Daniel Reichard
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dominik Häntsch
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sebastian Bodenstedt
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Stefan Suwelack
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Martin Wagner
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hannes Kenngott
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Beat Müller-Stich
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Lena Maier-Hein
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Rüdiger Dillmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Stefanie Speidel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniel Reichard.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All applicable international, national and/or institutional guidelines for the care and use of animals were followed. All procedures performed in studies involving animals were in accordance with the ethical standards of the institution or practice at which the studies were conducted.

Informed consent

This article does not contain patient data.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 29376 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Reichard, D., Häntsch, D., Bodenstedt, S. et al. Projective biomechanical depth matching for soft tissue registration in laparoscopic surgery. Int J CARS 12, 1101–1110 (2017). https://doi.org/10.1007/s11548-017-1613-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-017-1613-6

Keywords

Search

Navigation