Skip to main content
Springer Nature Link
Log in

Towards Multi-modal Self-supervised Video and Ultrasound Pose Estimation for Laparoscopic Liver Surgery

  • Conference paper
  • First Online:
Simplifying Medical Ultrasound (ASMUS 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13565))

Included in the following conference series:

Abstract

Estimating a registration between intra-operative data and a pre-operative scan is a key step to enable image guidance, and is particularly challenging in the laparoscopic approach due to the limited field of views of the data sources in these interventions. In this paper, we propose the first multi-modal, self-supervised registration paradigm to perform simultaneous laparoscopic ultrasound and video alignment to CT of the liver. Preliminary experiments performed on a single, patient-specific anatomical CT model suggest that registration of multiple features can facilitate the alignment of both data sources, and we show an example registration on an instance of real clinical data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    visiblepatient.com.

References

  1. Ciria, R., Cherqui, D., Geller, D.A., Briceno, J., Wakabayashi, G.: Comparative short-term benefits of laparoscopic liver resection: 9000 cases and climbing. Ann. Surg. 263(4), 761–777 (2016)

    Article  Google Scholar 

  2. Fusaglia, M., et al.: A clinically applicable laser-based image-guided system for laparoscopic liver procedures. Int. J. Comput. Assist. Radiol. Surg. 11(8), 1499–1513 (2016)

    Article  Google Scholar 

  3. Guo, X., Xu, S., Lin, X., Sun, Y., Ma, X.: 3d hand pose estimation from a single RGB image through semantic decomposition of VAE latent space. Pattern Anal. Appl. 25(1), 157–167 (2022)

    Article  Google Scholar 

  4. Kendall, A., Cipolla, R.: Geometric loss functions for camera pose regression with deep learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5974–5983 (2017)

    Google Scholar 

  5. Kendall, A., Grimes, M., Cipolla, R.: PoseNet: a convolutional network for real-time 6-DOF camera relocalization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2938–2946 (2015)

    Google Scholar 

  6. Koo, B., Özgür, E., Le Roy, B., Buc, E., Bartoli, A.: Deformable registration of a preoperative 3d liver volume to a laparoscopy image using contour and shading cues. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10433, pp. 326–334. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66182-7_38

    Chapter  Google Scholar 

  7. Koo, B., et al.: Automatic, global registration in laparoscopic liver surgery. Int. J. Comput. Assist. Radiol. Surg. 17(1), 167–176 (2022)

    Article  Google Scholar 

  8. Liu, S., Li, T., Chen, W., Li, H.: Soft rasterizer: a differentiable renderer for image-based 3d reasoning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7708–7717 (2019)

    Google Scholar 

  9. Plantefeve, R., Peterlik, I., Haouchine, N., Cotin, S.: Patient-specific biomechanical modeling for guidance during minimally-invasive hepatic surgery. Ann. Biomed. Eng. 44(1), 139–153 (2016)

    Article  Google Scholar 

  10. Prince, S.J.: Computer Vision: Models, Learning, and Inference. Cambridge University Press (2012)

    Google Scholar 

  11. Rabbani, N., et al.: A methodology and clinical dataset with ground-truth to evaluate registration accuracy quantitatively in computer-assisted Laparoscopic Liver Resection. Comput. Meth. Biomech. Biomed. Eng. Imaging Vis. 10(4), 441–450 (2022)

    Article  Google Scholar 

  12. Ramalhinho, J., Tregidgo, H.F., Gurusamy, K., Hawkes, D.J., Davidson, B., Clarkson, M.J.: Registration of untracked 2d laparoscopic ultrasound to CT images of the liver using multi-labelled content-based image retrieval. IEEE Trans. Med. Imaging 40(3), 1042–1054 (2020)

    Article  Google Scholar 

  13. Ravi, N., et al.: Accelerating 3d deep learning with PyTorch3D. arXiv preprint arXiv:2007.08501 (2020)

  14. Robu, M.R., et al.: Global rigid registration of CT to video in laparoscopic liver surgery. Int. J. Comput. Assist. Radiol. Surg. 13(6), 947–956 (2018)

    Article  Google Scholar 

  15. Schneider, C., Allam, M., Stoyanov, D., Hawkes, D., Gurusamy, K., Davidson, B.: Performance of image guided navigation in laparoscopic liver surgery-a systematic review. Surg. Oncol. 38, 101637 (2021)

    Article  Google Scholar 

  16. Song, Y., et al.: Locally rigid, vessel-based registration for laparoscopic liver surgery. Int. J. Comput. Assist. Radiol. Surg. 10(12), 1951–1961 (2015)

    Article  Google Scholar 

  17. Thompson, S., et al.: Accuracy validation of an image guided laparoscopy system for liver resection. In: Medical imaging 2015: Image-Guided Procedures, Robotic Interventions, and Modeling, vol. 9415, pp. 52–63. SPIE (2015)

    Google Scholar 

  18. Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)

    Article  Google Scholar 

Download references

Acknowledgement

This work is supported by the Wellcome/EPSRC Centre for Interventional and Surgical Sciences (203145Z/16/Z). For the purpose of Open Access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission. NMB is supported by the EPSRC-funded UCL Centre for Doctoral Training in Intelligent, Integrated Imaging in Healthcare (i4health) (EP/S021930/1). JR is supported by the EPSRC grant (EP/T029404/1).

Author information

Authors and Affiliations

  1. Wellcome/EPSRC Centre for Surgical and Interventional Sciences, University College London, London, UK

    Nina Montaña-Brown, João Ramalhinho, Bongjin Koo, Yipeng Hu & Matthew J. Clarkson

  2. Centre for Medical Image Computing, University College London, London, UK

    Nina Montaña-Brown, João Ramalhinho, Bongjin Koo, Yipeng Hu & Matthew J. Clarkson

  3. Division of Surgery and Interventional Science, University College London, London, UK

    Moustafa Allam, Brian Davidson & Kurinchi Gurusamy

Authors
  1. Nina Montaña-Brown
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. João Ramalhinho
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bongjin Koo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Moustafa Allam
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Brian Davidson
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kurinchi Gurusamy
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yipeng Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Matthew J. Clarkson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nina Montaña-Brown .

Editor information

Editors and Affiliations

  1. Kitware Inc., Carrboro, NC, USA

    Stephen Aylward

  2. University of Oxford, Oxford, UK

    J. Alison Noble

  3. University College London, London, UK

    Yipeng Hu

  4. University College London, London, UK

    Su-Lin Lee

  5. University College London, London, UK

    Zachary Baum

  6. University College London, London, UK

    Zhe Min

Ethics declarations

Ethical Approval

All data involving human participants was acquired in accordance with standards of the institution and/or national research committee (study: REC = 14/L0/1264, IRAS = 158321), and with the 1964 declaration of Helsinki and its later amendments.

Conflict of Interest

The authors declare that they have no conflict of interest.

Consent to Participate

Informed consent was obtained from all individual participants included in the study.

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Montaña-Brown, N. et al. (2022). Towards Multi-modal Self-supervised Video and Ultrasound Pose Estimation for Laparoscopic Liver Surgery. In: Aylward, S., Noble, J.A., Hu, Y., Lee, SL., Baum, Z., Min, Z. (eds) Simplifying Medical Ultrasound. ASMUS 2022. Lecture Notes in Computer Science, vol 13565. Springer, Cham. https://doi.org/10.1007/978-3-031-16902-1_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-16902-1_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-16901-4

  • Online ISBN: 978-3-031-16902-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships