Abstract
Segmentation models achieved expert-level performance in a large variety of medical applications. However, their robustness to rotations is rarely discussed and can be crucial for clinical use with the risk of discarding subtle but diagnostically relevant anatomical structures. In medical images, complex structures appear in a wide range of positions and rotations, requiring rotation robustness. In this work, we investigate the robustness to rotations of a standard 3D nnU-Net in the context of two segmentation tasks: the hippocampus in MRI and the pulmonary airway system in CT. In addition, we introduce a 3D Locally Rotation Invariant (LRI) operator based on the bispectrum to achieve high robustness to input rotations. It is compared to a standard nnU-Net, a nnU-Net with extended rotational data augmentation and XEdgeConv, a state-of-the-art approach for RI. While all models performed similarly in terms of Dice score for right-angle rotations, the Bispectral U-Net outperformed other designs in the context of finer and more realistic rotations. Furthermore, the Bispectral U-Net and the XEdgeConv were more stable w.r.t. input rotation, i.e. the predictions are significantly more consistent across input rotations. Important inconsistencies of the nnU-Net were observed for lung airway segmentation, suggesting potential risks of using the model in clinical routine.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
http://medicaldecathlon.com/, March 2024.
- 2.
https://atm22.grand-challenge.org/, March 2024.
- 3.
An epoch was computed in \(\approx \) 15 min using up to 27 Gb for a batch size of two.
- 4.
An epoch was computed in \(\approx \) 45 min using up to 70 Gb for a batch size of one.
- 5.
A spline interpolation of order three was used when executing those rotations.
References
Andrearczyk, V., Depeursinge, A.: Rotational 3D texture classification using group equivariant CNNs. arXiv preprint arXiv:1810.06889 (2018)
Andrearczyk, V., Fageot, J., Oreiller, V., Montet, X., Depeursinge, A.: Local rotation invariance in 3D CNNs. Med. Image Anal. 65, 101756 (2020)
Antonelli, M., et al.: The medical segmentation decathlon. Nat. Commun. 13(1), 4128 (2022)
Esteves, C., Allen-Blanchette, C., Makadia, A., Daniilidis, K.: Learning SO(3) equivariant representations with spherical CNNs. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11217, pp. 54–70. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01261-8_4
Fageot, J., Uhlmann, V., Püspöki, Z., Beck, B., Unser, M., Depeursinge, A.: Principled design and implementation of steerable detectors. IEEE Trans. Image Process. 30, 4465–4478 (2021)
Hadid, A.: The local binary pattern approach and its applications to face analysis. In: 2008 First Workshops on Image Processing Theory, Tools and Applications, pp. 1–9. IEEE (2008)
Isensee, F., Jaeger, P.F., Kohl, S.A., Petersen, J., Maier-Hein, K.H.: nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18(2), 203–211 (2021)
Jiang, M., Wu, Y., Zhao, T., Zhao, Z., Lu, C.: PointSIFT: a SIFT-like network module for 3D point cloud semantic segmentation. arXiv preprint arXiv:1807.00652 (2018)
Kakarala, R.: Completeness of bispectrum on compact groups. arXiv preprint arXiv:0902.01961 (2009)
Nguyen, T., Hua, B.-S., Le, N.: 3D-UCaps: 3D capsules Unet for volumetric image segmentation. In: de Bruijne, M., et al. (eds.) MICCAI 2021, Part I. LNCS, vol. 12901, pp. 548–558. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87193-2_52
Oreiller, V., Andrearczyk, V., Fageot, J., Prior, J.O., Depeursinge, A.: 3D solid spherical bispectrum CNNs for biomedical texture analysis. arXiv preprint arXiv:2004.13371 (2020)
Oreiller, V., Fageot, J., Andrearczyk, V., Prior, J.O., Depeursinge, A.: Robust multi-organ nucleus segmentation using a locally rotation invariant bispectral U-Net. In: International Conference on Medical Imaging with Deep Learning, pp. 929–943. PMLR (2022)
Qin, Y., et al.: AirwayNet: a voxel-connectivity aware approach for accurate airway segmentation using convolutional neural Networks. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 212–220. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32226-7_24
Rao, Y., Lu, J., Zhou, J.: Spherical fractal convolutional neural networks for point cloud recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 452–460 (2019)
Sun, X., Lian, Z., Xiao, J.: SRINet: learning strictly rotation-invariant representations for point cloud classification and segmentation. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 980–988 (2019)
Unser, M., Chenouard, N.: A unifying parametric framework for 2D steerable wavelet transforms. SIAM J. Imag. Sci. 6(1), 102–135 (2013)
Weihsbach, C., Hansen, L., Heinrich, M.: XEdgeConv: leveraging graph convolutions for efficient, permutation- and rotation-invariant dense 3D medical image segmentation. In: Geometric Deep Learning in Medical Image Analysis, pp. 61–71. PMLR (2022)
Weiler, M., Geiger, M., Welling, M., Boomsma, W., Cohen, T.S.: 3D Steerable CNNs: learning rotationally equivariant features in volumetric data. Adv. Neural Inf. Process. Syst. 31 (2018)
Weiler, M., Hamprecht, F.A., Storath, M.: Learning steerable filters for rotation equivariant CNNs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 849–858 (2018)
Winkels, M., Cohen, T.S.: Pulmonary nodule detection in CT scans with equivariant CNNs. Med. Image Anal. 55, 15–26 (2019)
Worrall, D., Brostow, G.: CubeNet: equivariance to 3D rotation and translation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11209, pp. 585–602. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01228-1_35
Worrall, D.E., Garbin, S.J., Turmukhambetov, D., Brostow, G.J.: Harmonic networks: deep translation and rotation equivariance. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5028–5037 (2017)
Yu, W., Zheng, H., Zhang, M., Zhang, H., Sun, J., Yang, J.: BREAK: bronchi reconstruction by geodesic transformation and skeleton embedding. In: 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI), pp. 1–5. IEEE (2022)
Zhang, M., et al.: Multi-site, multi-domain airway tree modeling. Med. Image Anal. 90, 102957 (2023)
Zhang, M., Zhang, H., Yang, G.Z., Gu, Y.: CFDA: collaborative feature disentanglement and augmentation for pulmonary airway tree modeling of COVID-19 CTs. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds.) MICCAI 2022. LNCS, vol. 13431, pp. 506–516. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16431-6_48
Zhang, Z., Hua, B.S., Rosen, D.W., Yeung, S.K.: Rotation invariant convolutions for 3D point clouds deep learning. In: 2019 International Conference on 3D Vision (3DV), pp. 204–213. IEEE (2019)
Zheng, H., et al.: Alleviating class-wise gradient imbalance for pulmonary airway segmentation. IEEE Trans. Med. Imaging 40(9), 2452–2462 (2021)
Acknowledgments
This work was partially funded by the Swiss National Science Foundation (SNSF) with the projects 205320_219430 and 205320_179069, the Swiss Cancer Research foundation with the project TARGET (KFS-5549-02-2022-R), and the Hasler Foundation with the project MSxplain number 21042.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Disclosure of Interests
The authors have no competing interests to declare that are relevant to the content of this article.
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Chevalley, A., Oreiller, V., Fageot, J., Prior, J.O., Andrearczyk, V., Depeursinge, A. (2025). A Bispectral 3D U-Net for Rotation Robustness in Medical Segmentation. In: Chen, C., Singh, Y., Hu, X. (eds) Topology- and Graph-Informed Imaging Informatics. TGI3 2024. Lecture Notes in Computer Science, vol 15239. Springer, Cham. https://doi.org/10.1007/978-3-031-73967-5_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-73967-5_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-73966-8
Online ISBN: 978-3-031-73967-5
eBook Packages: Computer ScienceComputer Science (R0)
