Abstract
Many brain tissue segmentation methods generally utilize one-level fusion to explore complementary discrepancies among different modalities. However, this one-level fusion manner cannot fully explore potential characteristics of multi-modality images. To this end, we propose a multi-level fusion segmentation transformer framework (dubbed SF-SeFormer) for brain tissue segmentation. Specifically, the proposed SF-SegFormer consists of three parts: Double Paired-modality Encoding (DPE) network, Cross Feature Decoding (CFD) network and Semantical Double Boundary Generation (SDBG) branch. Firstly, our DPE network is introduced to extract features from two pairs of dual-modality for the first-level fusion. Secondly, we design CFD network for the second-level and the third-level fusion by using cross-feature updating block and Cross Feature Fusion (CFF) block. Thirdly, we propose multi-stage channel aggregation-based multi-layer perceptron to enrich channel-aggregation diversity for efficient feature representation. Besides, semantical double boundaries can help to distinguish brain tissues, so we design SDBG branch to predict boundary of each target region, which can regularize multi-resolution CFF features. A large number of experiments have shown that proposed method outperforms many state-of-the-art segmentation methods, when evaluating on BrainWeb dataset.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Wen, Y., Xie, K., He, L.: Segmenting medical MRI via recurrent decoding cell, pp. 12452–12459. AAAI Press (2020)
Jégou, S., Drozdzal, M., Vázquez, D., Romero, A., Bengio, Y.: The one hundred layers tiramisu: fully convolutional DenseNets for semantic segmentation. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2017, Honolulu, HI, USA, 21–26 July 2017, pp. 1175–1183. IEEE Computer Society (2017)
Valanarasu, J.M.J., Sindagi, V.A., Hacihaliloglu, I., Patel, V.M.: KiU-Net: towards accurate segmentation of biomedical images using over-complete representations. In: Martel, A.L. (ed.) Medical Image Computing and Computer Assisted Intervention - MICCAI 2020 - 23rd International Conference, Lima, Peru, 4–8 October 2020, Proceedings, Part IV, vol. 12264 of Lecture Notes in Computer Science, pp. 363–373. Springer (2020). https://doi.org/10.1007/978-3-030-59719-1_36
Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017)
Dolz, J., Desrosiers, C., Ben Ayed, I.: IVD-Net: intervertebral disc localization and segmentation in MRI with a multi-modal UNet. In: Zheng, G., Belavy, D., Cai, Y., Li, S. (eds.) CSI 2018. LNCS, vol. 11397, pp. 130–143. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-13736-6_11
Liu, Y., Cheng, M.-M., Fan, D.-P., Zhang, L., Bian, J.-W., Tao, D.: Semantic edge detection with diverse deep supervision. Int. J. Comput. Vis. 130(1), 179–198 (2022)
Chen, J., et al.: TransUNet: transformers make strong encoders for medical image segmentation. CoRR, abs/2102.04306 (2021)
Liu, Z., et al.: Swin transformer: hierarchical vision transformer using shifted windows. CoRR, abs/2103.14030 (2021)
Lin, A., Chen, B., Xu, J., Zhang, Z., Lu, G.: DS-TransUNet: dual swin transformer U-Net for medical image segmentation. CoRR, abs/2106.06716 (2021)
Wang, Z., Cun, X., Bao, J., Liu, J.: Uformer: a general U-shaped transformer for image restoration. CoRR, abs/2106.03106 (2021)
Vaswani, A., et al.: Attention is all you need. Adv. Neural Inf. Process. Syst. 30 (2017)
Paszke, A., et al.: PyTorch: an imperative style, high-performance deep learning library. In: Wallach, H.M., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E.B., Garnett, R. (eds.) Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, 8–14 December 2019, Vancouver, BC, Canada, pp. 8024–8035 (2019)
Pereira, S., Pinto, A., Amorim, J., Ribeiro, A., Alves, V., Silva, C.A.: Adaptive feature recombination and recalibration for semantic segmentation with fully convolutional networks. IEEE Trans. Medical Imaging 38(12), 2914–2925 (2019)
Acknowledge
This work is supported by Fundamental Research Program of Shanxi Province (No. 202103021223284) and Taiyuan University of Science and Technology Scientific Research Initial Funding (No. 20192023 and No. 20192055). This study is also support by Scientific and Technological Innovation Programs of Higher Education Institutions in Shanxi, China (Grant No. 2019L0580).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, J., Zhao, L., Zeng, J., Qin, P. (2022). SF-SegFormer: Stepped-Fusion Segmentation Transformer for Brain Tissue Image via Inter-Group Correlation and Enhanced Multi-layer Perceptron. In: Yang, G., Aviles-Rivero, A., Roberts, M., Schönlieb, CB. (eds) Medical Image Understanding and Analysis. MIUA 2022. Lecture Notes in Computer Science, vol 13413. Springer, Cham. https://doi.org/10.1007/978-3-031-12053-4_38
Download citation
DOI: https://doi.org/10.1007/978-3-031-12053-4_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-12052-7
Online ISBN: 978-3-031-12053-4
eBook Packages: Computer ScienceComputer Science (R0)