Abstract
Aligning 2D ultrasound images with 3D CT scans of the liver holds significant clinical value in enhancing diagnostic precision, surgical planning, and treatment delivery. Conventional approaches primarily rely on optimization techniques, which often have a limited capture range and are susceptible to initialization errors. To address these limitations, we define the problem as “probe pose regression” and leverage deep learning for a more robust and efficient solution for liver US-CT registration without access to paired data. The proposed method is a three-part framework that combines ultrasound rendering, generative model and pose regression. In the first stage, we exploit a differentiable ultrasound rendering model designed to synthesize ultrasound images given segmentation labels. We let the downstream task optimize the rendering parameters, enhancing the performance of the overall method. In the second stage, a generative model bridges the gap between real and rendered ultrasound images, enabling application on real B-mode images. Finally, we use a patient-specific pose regression network, trained self-supervised with only synthetic images and their known poses. We use ultrasound, and CT scans from a dual-modality human abdomen phantom to validate the proposed method.
Our experimental results indicate that the proposed method can estimate probe poses within an acceptable error margin, which can later be fine-tuned using conventional methods. This capability confirms that the proposed framework can serve as a reliable initialization step for US-CT fusion and achieve fully automated US-CT fusion when coupled with conventional methods. The code and the dataset are available at https://github.com/mfazampour/SS_Probe_Pose_Regression.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
Siemens Healthineers, Erlangen, Germany.
- 3.
KUKA GmbH, Augsburg, Germany.
References
Roche, A., Pennec, X., Malandain, G., Ayache, N.: Rigid registration of 3-D ultrasound with MR images: a new approach combining intensity and gradient information. IEEE Trans. Med. Imaging 20(10), 1038 (2001)
Wein, W., Ladikos, A., Fuerst, B., Shah, A., Sharma, K., Navab, N.: Global registration of ultrasound to MRI using the LC2 metric for enabling neurosurgical guidance. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) Medical Image Computing and Computer-Assisted Intervention - MICCAI 2013. Lecture Notes in Computer Science, vol. 8149, pp. 34–41. Springer, Berlin (2013). https://doi.org/10.1007/978-3-642-40811-3_5
Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V.: VoxelMorph: a learning framework for deformable medical image registration. IEEE Trans. Med. Imaging 38(8), 1788 (2019)
Cao, X., Yang, J., Wang, L., Xue, Z., Wang, Q., Shen, D.: Deep learning based inter-modality image registration supervised by intra-modality similarity. In: Shi, Y., Suk, H.I., Liu, M. (eds.) Machine Learning in Medical Imaging. Lecture Notes in Computer Science(), vol. 11046, pp. 55–63. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00919-9_7
Markova, V., Ronchetti, M., Wein, W., Zettinig, O., Prevost, R.: Global Multi-modal 2D/3D Registration via Local Descriptors Learning. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds.) Medical Image Computing and Computer Assisted Intervention - MICCAI 2022. Lecture Notes in Computer Science, vol. 13436, pp. 269–279. Springer, Cham (2022)
Jaganathan, S., Kukla, M., Wang, J., Shetty, K., Maier, A.: Self-supervised 2D/3D registration for X-Ray to CT image fusion. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 2788–2798 (2023)
Zhang, B., et al.: A patient-specific self-supervised model for automatic X-Ray/CT registration. In: Greenspan, H., et al. (eds.) Medical Image Computing and Computer Assisted Intervention - MICCAI 2023. Lecture Notes in Computer Science, vol. 14228, pp. 515–524. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-43996-4_49
Prevost, R., et al.: 3D freehand ultrasound without external tracking using deep learning. Med. Image Anal. 48, 187 (2018)
Miura, K., Ito, K., Aoki, T., Ohmiya, J., Kondo, S.: Localizing 2D ultrasound probe from ultrasound image sequences using deep learning for volume reconstruction. In: Hu, Y., et al. (eds.) Medical Ultrasound, and Preterm, Perinatal and Paediatric Image Analysis. Lecture Notes in Computer Science(), vol. 12437, pp. 96–105. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-60334-2_10
Ning, G., Liang, H., Zhou, L., Zhang, X., Liao, H.: Spatial position estimation method for 3D ultrasound reconstruction based on hybrid transfomers. In: 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI), pp. 1–5. IEEE (2022)
Guo, H., et al.: IEEE Trans. Ultras., Ferroelectr., Freq. Control (2022)
Liu, J., et al.: Biomed. Signal Process. Control 86, 105150 (2023)
Alsinan, A.Z., Rule, C., Vives, M., Patel, V.M., Hacihaliloglu, I.: GAN-based realistic bone ultrasound image and label synthesis for improved segmentation. In: Martel, A.L., et al. (eds.) Medical Image Computing and Computer Assisted Intervention - MICCAI 2020. Lecture Notes in Computer Science(), vol. 12266, pp. 795–804. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59725-2_77
Peng, B., Huang, X., Wang, S., Jiang, J.: 2019 IEEE International Conference on Image Processing (ICIP), pp. 4629–4633. IEEE (2019)
Velikova, Y., Azampour, M.F., Simson, W., Gonzalez Duque, V., Navab, N.: LOTUS: learning to optimize task-based US representations. In: Greenspan, H., et al. (eds.) Medical Image Computing and Computer Assisted Intervention - MICCAI 2023. Lecture Notes in Computer Science, vol. 14220, pp. 492–501. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-43907-0_42
Velikova, Y., Azampour, M.F., Simson, W., Duque, V.G., Navab, N.: arXiv preprint: arXiv:2307.16021 (2023)
Lee, H.Y., et al.: Drit++: diverse image-to-image translation via disentangled representations. Int. J. Comput. Vision 128, 2402 (2020)
Huang, X., Liu, M.Y., Belongie, S., Kautz, J.: Proceedings of the European Conference on Computer Vision (ECCV), pp. 172–189 (2018)
Park, T., Efros, A.A., Zhang, R., Zhu, J.Y.: Contrastive learning for unpaired image-to-image translation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.M. (eds.) Computer Vision - ECCV 2020. Lecture Notes in Computer Science(), vol. 12354, pp. 319–345. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58545-7_19
Tan, M., Le, Q.: EfficientNet: rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114. PMLR (2019)
Wasserthal, J., Meyer, M., Breit, H.C., Cyriac, J., Yang, S., Segeroth, M.: arXiv preprint: arXiv:2208.05868 (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Azampour, M.F., Velikova, Y., Fatemizadeh, E., Dakua, S.P., Navab, N. (2024). Self-supervised Probe Pose Regression via Optimized Ultrasound Representations for US-CT Fusion. In: Su, R., Zhang, YD., Frangi, A.F. (eds) Proceedings of 2023 International Conference on Medical Imaging and Computer-Aided Diagnosis (MICAD 2023). MICAD 2023. Lecture Notes in Electrical Engineering, vol 1166. Springer, Singapore. https://doi.org/10.1007/978-981-97-1335-6_11
Download citation
DOI: https://doi.org/10.1007/978-981-97-1335-6_11
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-1334-9
Online ISBN: 978-981-97-1335-6
eBook Packages: Computer ScienceComputer Science (R0)