Skip to main content

Self-supervised Probe Pose Regression via Optimized Ultrasound Representations for US-CT Fusion

  • Conference paper
  • First Online:
Proceedings of 2023 International Conference on Medical Imaging and Computer-Aided Diagnosis (MICAD 2023) (MICAD 2023)

Abstract

Aligning 2D ultrasound images with 3D CT scans of the liver holds significant clinical value in enhancing diagnostic precision, surgical planning, and treatment delivery. Conventional approaches primarily rely on optimization techniques, which often have a limited capture range and are susceptible to initialization errors. To address these limitations, we define the problem as “probe pose regression” and leverage deep learning for a more robust and efficient solution for liver US-CT registration without access to paired data. The proposed method is a three-part framework that combines ultrasound rendering, generative model and pose regression. In the first stage, we exploit a differentiable ultrasound rendering model designed to synthesize ultrasound images given segmentation labels. We let the downstream task optimize the rendering parameters, enhancing the performance of the overall method. In the second stage, a generative model bridges the gap between real and rendered ultrasound images, enabling application on real B-mode images. Finally, we use a patient-specific pose regression network, trained self-supervised with only synthetic images and their known poses. We use ultrasound, and CT scans from a dual-modality human abdomen phantom to validate the proposed method.

Our experimental results indicate that the proposed method can estimate probe poses within an acceptable error margin, which can later be fine-tuned using conventional methods. This capability confirms that the proposed framework can serve as a reliable initialization step for US-CT fusion and achieve fully automated US-CT fusion when coupled with conventional methods. The code and the dataset are available at https://github.com/mfazampour/SS_Probe_Pose_Regression.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://www.kyotokagaku.com/en/products_data/us-22/.

  2. 2.

    Siemens Healthineers, Erlangen, Germany.

  3. 3.

    KUKA GmbH, Augsburg, Germany.

References

  1. Roche, A., Pennec, X., Malandain, G., Ayache, N.: Rigid registration of 3-D ultrasound with MR images: a new approach combining intensity and gradient information. IEEE Trans. Med. Imaging 20(10), 1038 (2001)

    Article  Google Scholar 

  2. Wein, W., Ladikos, A., Fuerst, B., Shah, A., Sharma, K., Navab, N.: Global registration of ultrasound to MRI using the LC2 metric for enabling neurosurgical guidance. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) Medical Image Computing and Computer-Assisted Intervention - MICCAI 2013. Lecture Notes in Computer Science, vol. 8149, pp. 34–41. Springer, Berlin (2013). https://doi.org/10.1007/978-3-642-40811-3_5

    Chapter  Google Scholar 

  3. Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V.: VoxelMorph: a learning framework for deformable medical image registration. IEEE Trans. Med. Imaging 38(8), 1788 (2019)

    Article  Google Scholar 

  4. Cao, X., Yang, J., Wang, L., Xue, Z., Wang, Q., Shen, D.: Deep learning based inter-modality image registration supervised by intra-modality similarity. In: Shi, Y., Suk, H.I., Liu, M. (eds.) Machine Learning in Medical Imaging. Lecture Notes in Computer Science(), vol. 11046, pp. 55–63. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00919-9_7

    Chapter  Google Scholar 

  5. Markova, V., Ronchetti, M., Wein, W., Zettinig, O., Prevost, R.: Global Multi-modal 2D/3D Registration via Local Descriptors Learning. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds.) Medical Image Computing and Computer Assisted Intervention - MICCAI 2022. Lecture Notes in Computer Science, vol. 13436, pp. 269–279. Springer, Cham (2022)

    Chapter  Google Scholar 

  6. Jaganathan, S., Kukla, M., Wang, J., Shetty, K., Maier, A.: Self-supervised 2D/3D registration for X-Ray to CT image fusion. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 2788–2798 (2023)

    Google Scholar 

  7. Zhang, B., et al.: A patient-specific self-supervised model for automatic X-Ray/CT registration. In: Greenspan, H., et al. (eds.) Medical Image Computing and Computer Assisted Intervention - MICCAI 2023. Lecture Notes in Computer Science, vol. 14228, pp. 515–524. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-43996-4_49

    Chapter  Google Scholar 

  8. Prevost, R., et al.: 3D freehand ultrasound without external tracking using deep learning. Med. Image Anal. 48, 187 (2018)

    Article  Google Scholar 

  9. Miura, K., Ito, K., Aoki, T., Ohmiya, J., Kondo, S.: Localizing 2D ultrasound probe from ultrasound image sequences using deep learning for volume reconstruction. In: Hu, Y., et al. (eds.) Medical Ultrasound, and Preterm, Perinatal and Paediatric Image Analysis. Lecture Notes in Computer Science(), vol. 12437, pp. 96–105. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-60334-2_10

    Chapter  Google Scholar 

  10. Ning, G., Liang, H., Zhou, L., Zhang, X., Liao, H.: Spatial position estimation method for 3D ultrasound reconstruction based on hybrid transfomers. In: 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI), pp. 1–5. IEEE (2022)

    Google Scholar 

  11. Guo, H., et al.: IEEE Trans. Ultras., Ferroelectr., Freq. Control (2022)

    Google Scholar 

  12. Liu, J., et al.: Biomed. Signal Process. Control 86, 105150 (2023)

    Google Scholar 

  13. Alsinan, A.Z., Rule, C., Vives, M., Patel, V.M., Hacihaliloglu, I.: GAN-based realistic bone ultrasound image and label synthesis for improved segmentation. In: Martel, A.L., et al. (eds.) Medical Image Computing and Computer Assisted Intervention - MICCAI 2020. Lecture Notes in Computer Science(), vol. 12266, pp. 795–804. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59725-2_77

    Chapter  Google Scholar 

  14. Peng, B., Huang, X., Wang, S., Jiang, J.: 2019 IEEE International Conference on Image Processing (ICIP), pp. 4629–4633. IEEE (2019)

    Google Scholar 

  15. Velikova, Y., Azampour, M.F., Simson, W., Gonzalez Duque, V., Navab, N.: LOTUS: learning to optimize task-based US representations. In: Greenspan, H., et al. (eds.) Medical Image Computing and Computer Assisted Intervention - MICCAI 2023. Lecture Notes in Computer Science, vol. 14220, pp. 492–501. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-43907-0_42

    Chapter  Google Scholar 

  16. Velikova, Y., Azampour, M.F., Simson, W., Duque, V.G., Navab, N.: arXiv preprint: arXiv:2307.16021 (2023)

  17. Lee, H.Y., et al.: Drit++: diverse image-to-image translation via disentangled representations. Int. J. Comput. Vision 128, 2402 (2020)

    Article  Google Scholar 

  18. Huang, X., Liu, M.Y., Belongie, S., Kautz, J.: Proceedings of the European Conference on Computer Vision (ECCV), pp. 172–189 (2018)

    Google Scholar 

  19. Park, T., Efros, A.A., Zhang, R., Zhu, J.Y.: Contrastive learning for unpaired image-to-image translation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.M. (eds.) Computer Vision - ECCV 2020. Lecture Notes in Computer Science(), vol. 12354, pp. 319–345. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58545-7_19

    Chapter  Google Scholar 

  20. Tan, M., Le, Q.: EfficientNet: rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114. PMLR (2019)

    Google Scholar 

  21. Wasserthal, J., Meyer, M., Breit, H.C., Cyriac, J., Yang, S., Segeroth, M.: arXiv preprint: arXiv:2208.05868 (2022)

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Mohammad Farid Azampour .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Azampour, M.F., Velikova, Y., Fatemizadeh, E., Dakua, S.P., Navab, N. (2024). Self-supervised Probe Pose Regression via Optimized Ultrasound Representations for US-CT Fusion. In: Su, R., Zhang, YD., Frangi, A.F. (eds) Proceedings of 2023 International Conference on Medical Imaging and Computer-Aided Diagnosis (MICAD 2023). MICAD 2023. Lecture Notes in Electrical Engineering, vol 1166. Springer, Singapore. https://doi.org/10.1007/978-981-97-1335-6_11

Download citation

  • DOI: https://doi.org/10.1007/978-981-97-1335-6_11

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-97-1334-9

  • Online ISBN: 978-981-97-1335-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics