Skip to main content

Advertisement

Springer Nature Link
Log in

3d freehand ultrasound reconstruction by reference-based point cloud registration

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

This study aims to address the challenging estimation of trajectories from freehand ultrasound examinations by means of registration of automatically generated surface points. Current approaches to inter-sweep point cloud registration can be improved by incorporating heatmap predictions, but practical challenges such as label-sparsity or only partially overlapping coverage of target structures arise when applying realistic examination conditions.

Methods

We propose a pipeline comprising three stages: (1) Utilizing a Free Point Transformer for coarse pre-registration, (2) Introducing HeatReg for further refinement using support point clouds, and (3) Employing instance optimization to enhance predicted displacements. Key techniques include expanding point sets with support points derived from prior knowledge and leverage of gradient keypoints. We evaluate our method on a large set of 42 forearm ultrasound sweeps with optical ground-truth tracking and investigate multiple ablations.

Results

The proposed pipeline effectively registers free-hand intra-patient ultrasound sweeps. Combining Free Point Transformer with support-point enhanced HeatReg outperforms the FPT baseline by a mean directed surface distance of 0.96 mm (40%). Subsequent refinement using Adam instance optimization and DiVRoC further improves registration accuracy and trajectory estimation.

Conclusion

The proposed techniques enable and improve the application of point cloud registration as a basis for freehand ultrasound reconstruction. Our results demonstrate significant theoretical and practical advantages of heatmap incorporation and multi-stage model predictions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Code availability

Visualization of results, implementation and trained models can be accessed at https://github.com/MDL-UzL/HeatReg.

Notes

  1. https://github.com/MDL-UzL/HeatReg.

References

  1. Che C, Mathai TS, Galeotti J (2017) Ultrasound registration: a review. Methods 115:128–143

    Article  CAS  PubMed  Google Scholar 

  2. El Hadramy S, Verde J, Beaudet K-P, Padoy N, Cotin S (2023) Trackerless volume reconstruction from intraoperative ultrasound images. In International conference on medical image computing and computer-assisted intervention, pp 303–312. Springer

  3. Prevost R, Salehi M, Jagoda S, Kumar N, Sprung J, Ladikos A, Bauer R, Zettinig O, Wein W (2018) 3d freehand ultrasound without external tracking using deep learning. Med Image Anal 48:187–202

    Article  PubMed  Google Scholar 

  4. Chen J-F, Fowlkes JB, Carson PL, Rubin JM (1997) Determination of scan-plane motion using speckle decorrelation: theoretical considerations and initial test. Int J Imag Syst Technol 8(1):38–44

    Article  Google Scholar 

  5. Luo M, Yang X, Yan Z, Li J, Zhang Y, Chen J, Hu X, Qian J, Cheng J, Ni D (2023) Multi-imu with online self-consistency for freehand 3d ultrasound reconstruction. In International conference on medical image computing and computer-assisted intervention, pp 342–351. Springer

  6. Hansen L, Heinrich MP (2021) Deep learning based geometric registration for medical images: how accurate can we get without visual features? In Information processing in medical imaging: 27th international conference, IPMI 2021, Virtual Event, June 28–June 30, 2021, Proceedings 27, pp 18–30. Springer

  7. Shen Z, Feydy J, Liu P, Curiale AH, San Jose Estepar R, San Jose Estepar R, Niethammer M (2021) Accurate point cloud registration with robust optimal transport. Adv Neural Inf Process Syst 34:5373–5389

    Google Scholar 

  8. Wu W, Wang ZY, Li Z, Liu W, Fuxin L (2020) Ppointpwc-net: cost volume on point clouds for (self-)supervised scene flow estimation. In Computer vision–ECCV 2020: 16th European conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part V 16, pp 88–10. Springer

  9. Bigalke A, Heinrich MP (2023) A denoised mean teacher for domain adaptive point cloud registration. In International conference on medical image computing and computer-assisted intervention, pp 666–676. Springer

  10. Baum ZM, Hu Y, Barratt DC (2021) Real-time multimodal image registration with partial intraoperative point-set data. Med Image Anal 74:102231

    Article  PubMed  PubMed Central  Google Scholar 

  11. Sun X, Xiao B, Wei F, Liang S, Wei Y (2018) Integral human pose regression. In Proceedings of the European conference on computer vision (ECCV), pp 529–545

  12. Heinrich MP, Hansen L (2022) Voxelmorph++ Going beyond the cranial vault with keypoint supervision and multi-channel instance optimisation. In International workshop on biomedical image registration, pp 85–95. Springer

  13. Hermes N, Hansen L, Bigalke A, Heinrich MP (2022) Support point sets for improving contactless interaction in geometric learning for hand pose estimation. In Bildverarbeitung Für die Medizin 2022: proceedings, German workshop on medical image computing, Heidelberg, June 26-28, 2022, pp 89–94 . Springer

  14. Heinrich MP, Bigalke A, Großbröhmer C, Hansen L (2023) Chasing clouds: differentiable volumetric rasterisation of point clouds as a highly efficient and accurate loss for large-scale deformable 3D registration. In Proceedings of the IEEE/CVF international conference on computer vision, pp 8026–8036

  15. Hansen L, Lichtenstein J, Heinrich MP (2024) displacement representation for conditional point cloud registration: heatreg applied to 2d/3d freehand ultrasound reconstruction. In BVM workshop, pp 39–45. Springer

  16. Haxthausen F, Ipsen S, Schwegmann H, Bruder R, Ernst F, Garcia-Vazquez V (2020) A 3D slicer module for calibration of spatially tracked 3D ultrasound probes. In CARS 2020-computer assisted radiology and surgery proceedings of the 34th international congress and exhibition, pp. 14–16

  17. Qi CR, Su H, Mo K, Guibas LJ (2017) Ppointnet: deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 652–660

  18. Isensee F, Jaeger PF, Kohl SA, Petersen J, Maier-Hein KH (2021) nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat Methods 18(2):203–211

    Article  CAS  PubMed  Google Scholar 

  19. Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In 3rd international conference on learning representations, ICLR 2015, San Diego, CA, USA, May 7–9, 2015, Conference track proceedings

  20. Girshick R (2015) Fast R-CNN. In Proceedings of the IEEE international conference on computer vision, pp 1440–1448

Download references

Funding

This work has been funded by the German Federal Ministry of Education and Research (BMBF, KI-gesteuerte Ultraschall-Bildgebung von Frakturen im Kindes- und Jugendalter, FKZ 13GW0578C) and by the Business Developement and Technology Transfer Corporation of Schleswig-Holstein (WTSH, Interaktiver Kl-gestützter Freihand 3D-Ultraschall in der Knochenchirurgie, Application Nr. 22023016).

Author information

Authors and Affiliations

  1. Institute of Medical Informatics, University of Lübeck, Lübeck, Germany

    Christoph Großbröhmer & Mattias P. Heinrich

  2. EchoScout GmbH, Lübeck, Germany

    Lasse Hansen

  3. Clinic of Oral and Maxillofacial Surgery, UKSH, Kiel, Germany

    Jürgen Lichtenstein

  4. Clinic of Pediatric Surgery, UKSH, Lübeck, Germany

    Ludger Tüshaus

Authors
  1. Christoph Großbröhmer
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Lasse Hansen
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Jürgen Lichtenstein
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Ludger Tüshaus
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Mattias P. Heinrich
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Lasse Hansen.

Ethics declarations

Ethics approval and consent to participate

All subjects have given informed consent to the recording, processing and publication of the data.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Großbröhmer, C., Hansen, L., Lichtenstein, J. et al. 3d freehand ultrasound reconstruction by reference-based point cloud registration. Int J CARS 20, 475–484 (2025). https://doi.org/10.1007/s11548-024-03280-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-024-03280-2

Keywords