Unsupervised domain selective graph convolutional network for preoperative prediction of lymph node metastasis in gastric cancer

https://doi.org/10.1016/j.media.2022.102467Get rights and content

Highlights

  • The proposed MSDA framework can promote LN metastasis prediction in multi-center learning.

  • A novel 3D IFPN can effectively extract the domain invariant features of small targets (i.e., LNs).

  • A novel UDS-GCN is designed to achieves the imbalanced knowledge transfer and class-aware feature alignment across domains.

Abstract

Preoperative prediction of lymph node (LN) metastasis based on computed tomography (CT) scans is an important task in gastric cancer, but few machine learning-based techniques have been proposed. While multi-center datasets increase sample size and representation ability, they suffer from inter-center heterogeneity. To tackle the above issue, we propose a novel multi-source domain adaptation framework for this diagnosis task, which not only considers domain-invariant and domain-specific features, but also achieves the imbalanced knowledge transfer and class-aware feature alignment across domains. First, we develop a 3D improved feature pyramidal network (i.e., 3D IFPN) to extract common multi-level features from the high-resolution 3D CT images, where a feature dynamic transfer (FDT) module can promote the network's ability to recognize the small target (i.e., LN). Then, we design an unsupervised domain selective graph convolutional network (i.e., UDS-GCN), which mainly includes three types of components: domain-specific feature extractor, domain selector and class-aware GCN classifier. Specifically, multiple domain-specific feature extractors are employed for learning domain-specific features from the common multi-level features generated by the 3D IFPN. A domain selector via the optimal transport (OT) theory is designed for controlling the amount of knowledge transferred from source domains to the target domain. A class-aware GCN classifier is developed to explicitly enhance/weaken the intra-class/inter-class similarity of all sample pairs across domains. To optimize UDS-GCN, the domain selector and the class-aware GCN classifier provide reliable target pseudo-labels to each other in the iterative process by collaborative learning. The extensive experiments are conducted on an in-house CT image dataset collected from four medical centers to demonstrate the efficacy of our proposed method. Experimental results verify that the proposed method boosts LN metastasis diagnosis performance and outperforms state-of-the-art methods. Our code is publically available at https://github.com/infinite-tao/LN_MSDA.

Introduction

Gastric cancer is a major problem threatening human health, and it is still the third leading cause of cancer-related death worldwide (Bray et al., 2018). Lymph node (LN) metastasis is one of the main prognostic factors of gastric cancer, and it plays a vital role in selecting reasonable surgical approaches and prognosis plans. Due to the high imaging density resolution, convenient examination, fast speed and non-invasiveness, computed tomography (CT) scan has become a routine imaging modality for the preoperative assessment of LN status (Kwee and Kwee, 2007). In clinical practice, traditional methods for determining nodal status using imaging tools are mainly based on the size of the LNs. However, the accuracy of previous work reports is unsatisfactory (Hwang et al., 2010; Saito et al., 2015). Moreover, the outlining process used to be completed manually on multi-slice images, which is highly subjective, labor-costing and time-consuming. To meet the clinical needs, it is urgent to develop an effective computer-aided diagnosis (CAD) system to supplement the conventional imaging tools and relieve the burden on radiologists.

To date, there is only one category of CAD method proposed to improve the prediction of LN metastasis in gastric cancer (Feng et al., 2019; Wang et al., 2020b). This category of method is machine learning-based radiomics, which mainly includes two stages: extracting hand-crafted features and building machine learning-based classifiers. However, the feature engineering is cumbersome and usually requires domain knowledge and experience. Besides, machine learning-based radiomics is not fully automatic, which is not suitable for studying multi-center data due to complex data distribution.

Compared to traditional machine learning methods with hand-crafted features, one of the main advantages of deep learning models is that they can effectively extract high-level semantic information and rich detailed information. Deep learning is an emerging technique that employs convolutional neural networks to learn the internal laws and deep representations of medical imaging data. Hence, deep learning has been widely applied to facilitate the diagnosis and prediction of various diseases in different imaging tools (Avendi et al., 2016; Gu et al., 2019; Li et al., 2018).By constructing robust models with refined multi-scale features, deep learning has been demonstrated to be useful in various challenging clinical tasks, including diagnosis in patients with abdominal diseases such as liver, kidney, and pancreas cancers (Heller et al., 2021; Ma et al., 2018; Yasaka et al., 2018). However, this fully automated technique has not been used to predict LN metastasis in gastric cancer.

To further improve disease identification in CAD, collecting data from multiple medical centers has become a hot research topic recently (Huang et al., 2020; Meng et al., 2020). Technically, multi-center disease identification is usually formulated as a multi-source domain adaptation (MSDA) problem, by first training models based on multiple source data distributions (i.e., multi-source domain) and then testing the learned model on a similar target data distribution (i.e., target domain). To mitigate multiple domain-shifts in data distribution, MSDA has been widely used for disease analysis (Wang et al., 2020a; Wang et al., 2019; Zhang et al., 2020). For instance, to alleviate the inter-site heterogeneity of multi-site data, a method of learning a latent representation space via low-rank representation decomposition was proposed and used for autism spectrum disorder identification (Wang et al., 2019). A supervised domain adaptation method based on instance weighting was designed for Alzheimer's disease diagnosis (Wachinger and Reuter, 2016). An augmented multi-center graph convolutional network (AM-GCN) was developed and evaluated for COVID-19 diagnosis, which considered the disease status of training samples (Song et al., 2021). These studies confirm that MSDA has obvious advantages in multi-center image analysis. However, these class-agnostic or domain-insufficient methods either ignore the class information of samples drawn from the source and target domains or fail to consider the domain discrepancy between the source and target domains. Therefore, they are unable to distinguish whether samples from the source and target domains should be aligned according to the class information (as shown in Fig.1b).

To tackle the above issues, we propose a novel multi-source domain adaptation framework, which consists of an early-stage 3D improved feature pyramidal network (3D IFPN) to extract common multi-level features from the initial CT scans, followed by an unsupervised domain selective graph convolutional network (UDS-GCN) to alleviate the domain shift between multiple source and target domains. Specifically, our 3D IFPN first employs feature pyramid network (FPN) to extract high-level semantic information and rich detailed information. Then, a feature dynamic transfer (FDT) module is designed, which improves the network's ability to recognize LNs by performing region-based spatial-aware transfer (SAT) and global-based channel-aware transfer (CAT) between high-level semantic information and rich detailed information. Our UDS-GCN achieves three key aspects that can help to learn a robust classifier for the target domain: domain selection, graph clustering and collaborative learning. To this end, we propose a domain selector and a class-aware GCN classifier, which are optimized in iterations by a collaborative learning scheme. The domain selector is composed of optimal transport (OT) theory and multiple domain-specific classifiers, which learns an ensemble classifier that can control the amount of knowledge transferred from multiple source domains to the target domain. Therefore, the class-aware GCN classifier can leverage the manual labels from the source domain data and target pseudo-labels generated by the ensemble classifier to enhance the intra-class compactness and inter-class separability of samples in all domains (see Fig.1c). To prevent these classifiers from over-fitting on their own noisy pseudo-labels, we develop a collaborative learning strategy with the domain selector and the class-aware GCN classifier to obtain more reliable pseudo-labels. Experiments on the self-collected CT image datasets show that our method achieves the promising performance in the prediction of LN metastasis and is superior to other state-of-the-art methods. In summary, this work has two main contributions: (1) We develop a novel 3D IFPN for LN metastasis prediction by fully exploiting high-level semantic and rich detailed information from multi-level features. Specifically, we develop a FDT module, which not only can capture the variable distribution of spatial semantics between adjacent levels, but also learn a set of weights to express the importance of the features at different levels. (2) We design a general unsupervised MSDA framework for multi-center LN metastasis prediction in gastric cancer, named UDS-GCN, which not only can selectively use the knowledge of multiple source domains to improve the performance of the classifier on the target domain, but also can achieve class-aware feature alignment in the MSDA scenario.

Section snippets

Related works

Since MSDA is an extension of single-source domain adaptation, many MSDA methods are inspired by single-source domain adaptation. In this section, we first give a brief description about single-source domain adaptation. We then introduce the existing work in MSDA. After that, we introduce the OT theory and its application progress in different fields.

Overview

Fig. 2 shows our two-stage deep neural network, which includes two deep neural subnets (i.e., 3D IFPN and UDS-GCN). Limited by the large amount of computer memory consumed by the 3D task, we first develop 3D IFPN to extract common and discriminative multi-level features from the initial CT images. The proposed 3D IFPN takes an abdominal CT volume as input and starts with 3D FPN (Lin et al., 2017a) as backbone to acquire multi-level features at different resolutions, and it is designed as a

Datasets and data preprocessing

The datasets are collected from the four medical centers (Taiyuan People Hospital, China; Xian People Hospital, China; Department of Radiology, China-Japan Friendship Hospital, Beijing, China; Heping Hospital, Changzi Medical College, China) by three kinds of medical instruments (Toshiba 320-slice CT, SOMATOM 64-slice CT, Philips 128-slice CT and SOMATOM force dual source CT), with a largely varying in-plane resolution from 0.5 to 1.0 mm and slice spacing from 5.0 to 8.0 mm. Our dataset

Parameter analysis

In our 3D IFPN, there are two important parameters (i.e., γ and α) to be tuned. In validation strategy 1, we use the cross-entropy loss function to train 3D IFPN, and use the five-fold group cross-validation strategy. Since we ignore the problem of sample imbalance, Table 4 shows that the classification results have low specificity. In validation strategy 2, we consider using focal loss instead of cross-entropy loss to alleviate the problem of sample imbalance. To investigate the impact of the

Conclusions

In this paper, we propose to investigate MSDA for preoperative prediction of LN metastasis in gastric cancer. To tackle the limitations of 3D data training in multi-source domains, we first propose a multi-task 3D IFPN, which extracts common multi-level features for implementation of the MSDA scenario. The key idea of 3D IFPN is to select useful complementary information from multi-level features by region-based SAT and global-based CAT to improve its ability to recognize small targets. Then,

Authorship conformation form

Please check the following as appropriate:

All authors have participated in (a) conception and design, or analysis and interpretation of the data; (b) drafting the article or revising it critically for important intellectual content; and (c) approval of the final version.

CRediT authorship contribution statement

Yongtao Zhang: Conceptualization, Visualization, Formal analysis, Data curation, Writing – original draft, Writing – review & editing. Ning Yuan: Conceptualization, Formal analysis, Data curation, Writing – review & editing. Zhiguo Zhang: Conceptualization, Formal analysis, Writing – review & editing. Jie Du: Conceptualization, Formal analysis, Writing – review & editing. Tianfu Wang: Conceptualization, Formal analysis. Bing Liu: Visualization, Data curation. Aocai Yang: Visualization, Data

Declaration of Competing Interest

None.

Acknowledgments

This work was supported partly by National Natural Science Foundation of China (Nos. 62071309, 61871274, 81971585 and 62006160), National Natural Science Foundation of Guangdong Province (Nos. 2019B1515120029 and 2019A1515111205), Shenzhen Key Basic Research Project (Nos. SGDX20201103095802007, KCXFZ20201221173213036, JCYJ20180507184647636, JCYJ20190808155618806, GJHZ20190822095414576, and JCYJ20190808145011259), National Key Research and Development Program of China (Nos. 2020YFC2003903,

References (55)

  • N. Courty et al.

    Joint distribution optimal transportation for domain adaptation

    Adv. Neural Inf. Process. Syst.

    (2017)
  • N. Courty et al.

    Optimal transport for domain adaptation

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2016)
  • M. Cuturi

    Sinkhorn distances: Lightspeed computation of optimal transport

    Adv. Neural Inform. Process. Syst.

    (2013)
  • B.B. Damodaran et al.

    Deepjdot: Deep joint distribution optimal transport for unsupervised domain adaptation

  • Y. Ganin et al.

    Domain-adversarial training of neural networks

    J. Mach. Learn. Res.

    (2016)
  • Z. Gu et al.

    CE-Net: context encoder network for 2D medical image segmentation

    IEEE Trans. Med. Imaging

    (2019)
  • J. Hoffman et al.

    Cycada: cycle-consistent adversarial domain adaptation

    Proceedings of the International Conference on Machine Learning

    PMLR

    (2018)
  • J. Hu et al.

    Squeeze-and-excitation networks

  • S.W. Hwang et al.

    Preoperative staging of gastric cancer by endoscopic ultrasonography and multidetector-row computed tomography

    J. Gastroenterol. Hepatol.

    (2010)
  • G. Kang et al.

    Contrastive adaptation network for single-and multi-source domain adaptation

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2020)
  • G. Kang et al.

    Contrastive adaptation network for unsupervised domain adaptation

  • Kipf, T.N., Welling, M., 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint...
  • R.M. Kwee et al.

    Imaging in local staging of gastric cancer: a systematic review

    J. Clin. Oncol.

    (2007)
  • X. Li et al.

    H-DenseUNet: hybrid densely connected UNet for liver and tumor segmentation from CT volumes

    IEEE Trans. Med. Imaging

    (2018)
  • X. Li et al.

    Selective kernel networks

  • T.Y. Lin et al.

    Feature pyramid networks for object detection

  • T.Y. Lin et al.

    Focal loss for dense object detection

  • Cited by (5)

    View full text