skip to main content
10.1145/3613307.3613320acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicbipConference Proceedingsconference-collections
research-article

Tumor Segmentation in Weakly Paired Anatomical and Functional MRI images with Multimodal Information Fusion

Published: 28 September 2023 Publication History

Abstract

Multimodal magnetic resonance imaging (MRI) contains complementary information in anatomical and functional images that help the accurate diagnosis and treatment evaluation of lung cancers. Accurately segmenting tumor regions in each modality can help to obtain comprehensive and precise information. Existing multimodal segmentation methods are mostly used for images with strict registration, but it is difficult to achieve it for lung MRI images. It is challenging to obtain accurate tumor segmentation in lung anatomical and functional MRI images simultaneously. In this paper, we propose a method for lung tumor segmentation in weakly paired anatomical and functional MRI images. Firstly, we use domain adaptation to narrow the gap in features across different modalities, enabling the segmentation network to adapt to different modality data simultaneously. At the same time, we explore a two-stage multimodal co-attention mechanism to help the extraction and effective fusion of multi-modal information. We evaluate the proposed method on lung tumor segmentation with a clinical dataset of 90 chest MRI scans of non-small cell lung cancer (NSCLC). The results show that this method effectively improves the segmentation accuracy of each modality, the DSC of anatomical MRI is increased to 0.81±0.19, and the DSC of functional MRI is increased to 0.77±0.23, which is significantly improved compared with several multimodal tumor segmentation methods (p <0.05).

References

[1]
Lung cancer statistics. Available from: https://www.wcrf.org/cancer-trends/lung-cancer-statistics/.
[2]
Sim, A.J., A review of the role of MRI in diagnosis and treatment of early stage lung cancer. Clinical and Translational Radiation Oncology, 2020. 24: p. 16-22.
[3]
Brown, S.e.a., The evoling role of radiotherapy in non-small cell lung cancer. British Journal of Radiology, 2019. vol. 92,1104 (2019): 20190524.
[4]
Bainbridge, H., Magnetic resonance imaging in precision radiation therapy for lung cancer. Translational Lung Cancer Research, 2017. 6(6): p. 689-707.
[5]
Otazo, R., MRI-guided Radiation Therapy: An Emerging Paradigm in Adaptive Radiation Oncology. Radiology, 2021. 298(2): p. 248-260.
[6]
Shen, Y., Gao, M., Brain Tumor Segmentation on MRI with Missing Modalities. International Conference on Information Processing in Medical Imaging, 2019: p. 417–428.
[7]
Kumar, A., Co-Learning Feature Fusion Maps From PET-CT Images of Lung Cancer. Ieee Transactions on Medical Imaging, 2020. 39(1): p. 204-217.
[8]
Zhou, T., S. Ruan, and S. Canu, A review: Deep learning for medical image segmentation using multi-modality fusion. Array, 2019. 3-4.
[9]
Fu, X.H., Multimodal Spatial Attention Module for Targeting Multimodal PET-CT Lung Tumor Segmentation. Ieee Journal of Biomedical and Health Informatics, 2021. 25(9): p. 3507-3516.
[10]
Zhou, T.X., Latent Correlation Representation Learning for Brain Tumor Segmentation With Missing MRI Modalities. Ieee Transactions on Image Processing, 2021. 30: p. 4263-4274.
[11]
Zhou, P., Coco-attention for Tumor Segmentation in Weakly Paired Multimodal MRI Images. IEEE Journal of Biomedical and Health Informatics, 2023.
[12]
Tsai, Y.H., Domain Adaptation for Structured Output via Discriminative Patch Representations. 2019 Ieee/Cvf International Conference on Computer Vision (Iccv 2019), 2019: p. 1456-1465.
[13]
Woo, S.H., CBAM: Convolutional Block Attention Module. Computer Vision - Eccv 2018, Pt Vii, 2018. 11211: p. 3-19.

Index Terms

  1. Tumor Segmentation in Weakly Paired Anatomical and Functional MRI images with Multimodal Information Fusion

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ICBIP '23: Proceedings of the 2023 8th International Conference on Biomedical Signal and Image Processing
    July 2023
    140 pages
    ISBN:9798400707698
    DOI:10.1145/3613307
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 28 September 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. co-attention
    2. domain adaptation
    3. multimodal tumor segmentation

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Funding Sources

    • National Natural Science Foundation of China
    • National Natural Science Foundation of China

    Conference

    ICBIP 2023

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 33
      Total Downloads
    • Downloads (Last 12 months)15
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 13 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media