Skip to main content
SpringerLink
Log in

Intraoperative laparoscopic liver surface registration with preoperative CT using mixing features and overlapping region masks

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Laparoscopic liver resection is a minimal invasive surgery. Augmented reality can map preoperative anatomy information extracted from computed tomography to the intraoperative liver surface reconstructed from stereo 3D laparoscopy. However, liver surface registration is particularly challenging as the intraoperative surface is only partially visible and suffers from large liver deformations due to pneumoperitoneum. This study proposes a deep learning-based robust point cloud registration network.

Methods

This study proposed a low overlap liver surface registration algorithm combining local mixed features and global features of point clouds. A learned overlap mask is used to filter the non-overlapping region of the point cloud, and a network is used to predict the overlapping region threshold to regulate the training process.

Results

We validated the algorithm on the DePoLL (the Deformable Porcine Laparoscopic Liver) dataset. Compared with the baseline method and other state-of-the-art registration methods, our method achieves minimum target registration error (TRE) of 19.9 ± 2.7 mm.

Conclusion

The proposed point cloud registration method uses the learned overlapping mask to filter the non-overlapping areas in the point cloud, then the extracted overlapping area point cloud is registered according to the mixed features and global features, and this method is robust and efficient in low-overlap liver surface registration.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Fuks D, Cauchy F, Ftériche S, Nomi T, Schwarz L, Dokmak S, Scatton O, Fusco G, Belghiti J, Gayet B, Soubrane O (2016) Laparoscopy decreases pulmonary complications in patients undergoing major liver resection. Ann Surg 263(2):353–361

    Article  PubMed  Google Scholar 

  2. Ciria R, Cherqui D, Geller D, Briceno J, Wakabayashi G (2016) Comparative Short-term benefits of laparoscopic liver resection. Ann Surg 263(4):761–777

    Article  PubMed  Google Scholar 

  3. Wakabayashi G, Cherqui D, Geller D, Han H, Kaneko H, Buell J (2014) Laparoscopic hepatectomy is theoretically better than open hepatectomy: preparing for the 2nd international consensus conference on laparoscopic liver resection. J Hepato-Biliary Pancreat Sci 21:723–731

    Article  Google Scholar 

  4. Dagher I, O’Rourke N, Geller D, Cherqui D, Belli G, Gamblin T, Lainas P, Laurent A, Nguyen K, Marvin M, Thomas M, Ravindra K, Fielding G, Franco D, Buell J (2009) Laparoscopic major hepatectomy: an evolution in standard of care. Ann Surg 250(5):856–860

    Article  PubMed  Google Scholar 

  5. El-Gendi A, El-Shafei M, ShawkyA E-G (2018) Laparoscopic versus open hepatic resection for solitary hepatocellular carcinoma less than 5 cm in cirrhotic patients: a randomized controlled study. J Laparoendosc Adv Surg Tech A 28(3):302–310

    Article  PubMed  Google Scholar 

  6. Zhang W, Yin D, Chen X, Zhang S, Meng F, Guo H, Liang S, Zhou S, Liu S, Sun L, Guo X, Luo H, He B, Xiao D, Cai W, Fang C, Liu L, Jia F (2021) Morphologic change of in vivo porcine liver under 13 mm Hg pneumoperitoneum pressure. Surg Laparosc Endosc Percutan Tech 31(6):679–684

    Article  PubMed  Google Scholar 

  7. Kirchberg J, Reißfelder C, Weitz J, Koch M (2013) Laparoscopic surgery of liver tumors. Langenbecks Arch Surg 398(7):931–938

    Article  PubMed  Google Scholar 

  8. Cauchy F, Fuks D, Nomi T, Schwarz L, Barbier L, Dokmak S, Scatton O, Belghiti J, Soubrane O, Gayet B (2015) Risk factors and consequences of conversion in laparoscopic major liver resection. Br J Surg 102(7):785–795

    Article  CAS  PubMed  Google Scholar 

  9. Thompson S, Schneider C, Bosi M, Gurusamy K, Ourselin S, Davidson B, Hawkes D, Clarkson MJ (2018) In vivo estimation of target registration errors during augmented reality laparoscopic surgery. Int J Comput Assist Radiol Surg 13(6):865–874

    Article  PubMed  PubMed Central  Google Scholar 

  10. Espinel Y, Calvet L, Botros K, Buc E, Tilmant C, Bartoli A (2022) Using multiple images and contours for deformable 3D–2D registration of a preoperative CT in laparoscopic liver surgery. Int J Comput Assist Radio Surg 17:2211–2219

    Article  Google Scholar 

  11. Thompson S, Totz J, Song Y, Johnsen S, Stoyanov D, Ourselin S, Clarkson MJ (2015) Accuracy validation of an image guided laparoscopy system for liver resection. In: Medical imaging 2015: image-guided procedures, robotic interventions, and modelling vol 9415, pp 52-63. SPIE

  12. Schneider C, Thompson S, Totz J, Song Y, Allam M, Sodergren MH, Desjardins AE, Barratt D, Ourselin S, Gurusamy K, Stoyanov D, Clarkson MJ, Hawkes DJ, Davidson BR (2020) Comparison of manual and semi-automatic registration in augmented reality image-guided liver surgery: a clinical feasibility study. Surg Endosc 34:4702–4711

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Adagolodjo Y, Trivisonne R, Haouchine N, Cotin S, Courtecuisse H (2017) Silhouette-based pose estimation for deformable organs application to surgical augmented reality. IROS. pp 539–544

  14. Modrzejewski R, Collins T, Seeliger B, Bartoli A, Hostettler A, Marescaux J (2019) An in vivo porcine dataset and evaluation methodology to measure soft-body laparoscopic liver registration accuracy with an extended algorithm that handles collisions. Int J Comput Assist Radio Surg 14(7):1237–1245

    Article  Google Scholar 

  15. Zhang X, Wang J, Wang T, Ji X, Shen Y, Sun Z, Zhang X (2019) A markerless automatic deformable registration framework for augmented reality navigation of laparoscopy partial nephrectomy. Int J Comput Assist Radiol Surg 14(8):1285–1294

    Article  PubMed  Google Scholar 

  16. Besl PJ, McKay ND (1992) A method for registration of 3-D shapes. IEEE Trans Pattern Anal Mach Intell 14(2):239–256

    Article  Google Scholar 

  17. Yang J, Li H, Jia Y (2013) Go-ICP: Solving 3D registration efficiently and globally optimally. CVPR. pp 1457–1464

  18. dos Santos T, Seitel A, Kilgus T, Suwelack S, Wekerle A, Kenngott H, Speidel S, Schlemmer H, Meinzer H, Heimann T, Maier-Hein L (2014) Pose-independent surface matching for intra-operative soft-tissue marker-less registration. Med Image Anal 18(7):1101–1114

    Article  PubMed  Google Scholar 

  19. Wang Y, Solomon JM (2019a) Deep closest point: learning representations for point cloud registration. ICCV. pp 3523–3532

  20. Yuan W, Eckart B, Kim K, Jampani V, Fox D, Kautz J (2020) DeepGMR: Learning latent gaussian mixture models for registration. ECCV. pp 733–750

  21. Wang Y, Solomon JM (2019b) PRNet: Self-supervised learning for partial-to-partial registration. NeurIPS. pp 8814–8826

  22. Gold S, Rangarajan A, Lu C, Pappu S, Mjolsness E (1998) New algorithms for 2D and 3D point matching. Pattern Recognit 31(8):1019–1031

    Article  Google Scholar 

  23. Yew ZJ, Lee GH (2020) RPM-Net: robust point matching using learned features. CVPR. pp 11824–11833

  24. Li J, Zhang C, Xu Z, Zhou H, Zhang C (2020) Iterative distance-aware similarity matrix convolution with mutual-supervised point elimination for efficient point cloud registration. ECCV. pp 378–394

  25. Jang E, Gu S, Poole B (2016) Categorical reparameterization with Gumbel-Softmax. arXiv preprint arXiv:1611.01144

  26. Sinkhorn R (1964) A relationship between arbitrary positive matrices and doubly stochastic matrices. Ann Math Stat 35(2):876–879

    Article  Google Scholar 

  27. Aoki Y, Goforth H, Srivatsan RA, Lucey S (2019) PointNetLK: Robust & efficient point cloud registration using PointNet. CVPR. pp 7163–7172

  28. Sarode V, Li X, Goforth H, Aoki Y, Srivatsan RA, Lucey S, Choset H (2019) PCRNet: Point cloud registration network using pointnet encoding. arXiv preprint arXiv:1908.07906

  29. Huang X, Mei G, Zhang J (2020) Feature-metric registration: A fast semi-supervised approach for robust point cloud registration without correspondences. CVPR. pp 11366–11374

  30. Zhu L, Liu D, Lin C, Yan R, Gómez-Fernández F, Yang N, Feng Z (2021) Point cloud registration using representative overlapping points. arXiv preprint arXiv: 2107.02583

  31. Xu H, Liu S, Wang G, Liu G, Zeng B (2021) OMNet: Learning overlapping mask for partial-to-partial point cloud registration. arXiv preprint arXiv: 2103.00937

  32. Wu Z, Song S, Khosla A, Yu F, Zhang L, Tang X, Xiao J (2015) 3D ShapeNets: a deep representation for volumetric shapes. CVPR. pp 1912–1920

  33. Qi CR, Su H, Mo K, and Guibas LJ (2017) PointNet: Deep learning on point sets for 3D classification and segmentation. CVPR. pp 77–85

  34. Rusu R, Blodow N, Beetz M (2009) Fast point feature histograms (FPFH) for 3D registration. IEEE ICRA. pp 3212–3217

  35. Deng H, Tolga Birdal T, Ilic S (2018) PPFNet: Global context aware local features for robust 3D point matching. CVPR. pp 195–205

  36. Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. ICLR

  37. Reichard D, Bodenstedt S, Suwelack S, Mayer B, Preukschas A, Wagner M, Kenngott H, Muller-Stich B, Dillmann R, Speidel S (2015) Intraoperative on-the-fly organ-mosaicking for laparoscopic surgery. J Med Imaging 2(4):045001

    Article  Google Scholar 

  38. Robu MR, Ramalhinho J, Thompson S, Gurusamy K, Davidson B, Hawkes D, Stoyanov D, Clarkson MJ (2018) Global rigid registration of CT to video in laparoscopic liver surgery. Int J Comput Assist Radio Surg 13:947–956

    Article  Google Scholar 

  39. Robu MR, Edwards P, Ramalhinho J, Thompson S, Davidson B, Hawkes D, Stoyanov D, Clarkson MJ (2017) Intelligent viewpoint selection for efficient CT to video registration in laparoscopic liver surgery. Int J Comput Assist Radio Surg 12:1079–1088

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by grants from the NSFC Grant Program (Nos. 62172401 and 12026602); the Key-Area Research and Development Program of Guangdong Province (No. 2020B010165004); the National Key R&D Program, China (No. 2019YFC0118100); the Natural Science Foundation of Guangdong Province (Nos. 2022A1515010439 and 2022A1515010176); the Shenzhen Key Basic Science Program (Nos. JCYJ20180507182437217 and JCYJ20220531100614032); and the Shenzhen Key Laboratory Program (No. ZDSYS201707271637577).

Author information

Author notes

  1. Peidong Guan and Huoling Luo have equally contributed this work.

Authors and Affiliations

  1. Research Center for Medical AI, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China

    Peidong Guan, Huoling Luo & Fucang Jia

  2. Shenzhen College of Advanced Technology, University of Chinese Academy and Sciences, Shenzhen, China

    Peidong Guan & Fucang Jia

  3. Department of Interventional Radiology, Shenzhen People’s Hospital, Shenzhen, China

    Jianxi Guo & Yanfang Zhang

  4. Pazhou Lab, Guangzhou, China

    Fucang Jia

Authors
  1. Peidong Guan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huoling Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianxi Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yanfang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Fucang Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yanfang Zhang or Fucang Jia.

Ethics declarations

Conflict of interest

None.

Human and animal rights

The work used the public dataset only.

Inform consent

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 372 kb)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guan, P., Luo, H., Guo, J. et al. Intraoperative laparoscopic liver surface registration with preoperative CT using mixing features and overlapping region masks. Int J CARS 18, 1521–1531 (2023). https://doi.org/10.1007/s11548-023-02846-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-023-02846-w

Keywords

Search

Navigation