Unsupervised domain selective graph convolutional network for preoperative prediction of lymph node metastasis in gastric cancer
Introduction
Gastric cancer is a major problem threatening human health, and it is still the third leading cause of cancer-related death worldwide (Bray et al., 2018). Lymph node (LN) metastasis is one of the main prognostic factors of gastric cancer, and it plays a vital role in selecting reasonable surgical approaches and prognosis plans. Due to the high imaging density resolution, convenient examination, fast speed and non-invasiveness, computed tomography (CT) scan has become a routine imaging modality for the preoperative assessment of LN status (Kwee and Kwee, 2007). In clinical practice, traditional methods for determining nodal status using imaging tools are mainly based on the size of the LNs. However, the accuracy of previous work reports is unsatisfactory (Hwang et al., 2010; Saito et al., 2015). Moreover, the outlining process used to be completed manually on multi-slice images, which is highly subjective, labor-costing and time-consuming. To meet the clinical needs, it is urgent to develop an effective computer-aided diagnosis (CAD) system to supplement the conventional imaging tools and relieve the burden on radiologists.
To date, there is only one category of CAD method proposed to improve the prediction of LN metastasis in gastric cancer (Feng et al., 2019; Wang et al., 2020b). This category of method is machine learning-based radiomics, which mainly includes two stages: extracting hand-crafted features and building machine learning-based classifiers. However, the feature engineering is cumbersome and usually requires domain knowledge and experience. Besides, machine learning-based radiomics is not fully automatic, which is not suitable for studying multi-center data due to complex data distribution.
Compared to traditional machine learning methods with hand-crafted features, one of the main advantages of deep learning models is that they can effectively extract high-level semantic information and rich detailed information. Deep learning is an emerging technique that employs convolutional neural networks to learn the internal laws and deep representations of medical imaging data. Hence, deep learning has been widely applied to facilitate the diagnosis and prediction of various diseases in different imaging tools (Avendi et al., 2016; Gu et al., 2019; Li et al., 2018).By constructing robust models with refined multi-scale features, deep learning has been demonstrated to be useful in various challenging clinical tasks, including diagnosis in patients with abdominal diseases such as liver, kidney, and pancreas cancers (Heller et al., 2021; Ma et al., 2018; Yasaka et al., 2018). However, this fully automated technique has not been used to predict LN metastasis in gastric cancer.
To further improve disease identification in CAD, collecting data from multiple medical centers has become a hot research topic recently (Huang et al., 2020; Meng et al., 2020). Technically, multi-center disease identification is usually formulated as a multi-source domain adaptation (MSDA) problem, by first training models based on multiple source data distributions (i.e., multi-source domain) and then testing the learned model on a similar target data distribution (i.e., target domain). To mitigate multiple domain-shifts in data distribution, MSDA has been widely used for disease analysis (Wang et al., 2020a; Wang et al., 2019; Zhang et al., 2020). For instance, to alleviate the inter-site heterogeneity of multi-site data, a method of learning a latent representation space via low-rank representation decomposition was proposed and used for autism spectrum disorder identification (Wang et al., 2019). A supervised domain adaptation method based on instance weighting was designed for Alzheimer's disease diagnosis (Wachinger and Reuter, 2016). An augmented multi-center graph convolutional network (AM-GCN) was developed and evaluated for COVID-19 diagnosis, which considered the disease status of training samples (Song et al., 2021). These studies confirm that MSDA has obvious advantages in multi-center image analysis. However, these class-agnostic or domain-insufficient methods either ignore the class information of samples drawn from the source and target domains or fail to consider the domain discrepancy between the source and target domains. Therefore, they are unable to distinguish whether samples from the source and target domains should be aligned according to the class information (as shown in Fig.1b).
To tackle the above issues, we propose a novel multi-source domain adaptation framework, which consists of an early-stage 3D improved feature pyramidal network (3D IFPN) to extract common multi-level features from the initial CT scans, followed by an unsupervised domain selective graph convolutional network (UDS-GCN) to alleviate the domain shift between multiple source and target domains. Specifically, our 3D IFPN first employs feature pyramid network (FPN) to extract high-level semantic information and rich detailed information. Then, a feature dynamic transfer (FDT) module is designed, which improves the network's ability to recognize LNs by performing region-based spatial-aware transfer (SAT) and global-based channel-aware transfer (CAT) between high-level semantic information and rich detailed information. Our UDS-GCN achieves three key aspects that can help to learn a robust classifier for the target domain: domain selection, graph clustering and collaborative learning. To this end, we propose a domain selector and a class-aware GCN classifier, which are optimized in iterations by a collaborative learning scheme. The domain selector is composed of optimal transport (OT) theory and multiple domain-specific classifiers, which learns an ensemble classifier that can control the amount of knowledge transferred from multiple source domains to the target domain. Therefore, the class-aware GCN classifier can leverage the manual labels from the source domain data and target pseudo-labels generated by the ensemble classifier to enhance the intra-class compactness and inter-class separability of samples in all domains (see Fig.1c). To prevent these classifiers from over-fitting on their own noisy pseudo-labels, we develop a collaborative learning strategy with the domain selector and the class-aware GCN classifier to obtain more reliable pseudo-labels. Experiments on the self-collected CT image datasets show that our method achieves the promising performance in the prediction of LN metastasis and is superior to other state-of-the-art methods. In summary, this work has two main contributions: (1) We develop a novel 3D IFPN for LN metastasis prediction by fully exploiting high-level semantic and rich detailed information from multi-level features. Specifically, we develop a FDT module, which not only can capture the variable distribution of spatial semantics between adjacent levels, but also learn a set of weights to express the importance of the features at different levels. (2) We design a general unsupervised MSDA framework for multi-center LN metastasis prediction in gastric cancer, named UDS-GCN, which not only can selectively use the knowledge of multiple source domains to improve the performance of the classifier on the target domain, but also can achieve class-aware feature alignment in the MSDA scenario.
Section snippets
Related works
Since MSDA is an extension of single-source domain adaptation, many MSDA methods are inspired by single-source domain adaptation. In this section, we first give a brief description about single-source domain adaptation. We then introduce the existing work in MSDA. After that, we introduce the OT theory and its application progress in different fields.
Overview
Fig. 2 shows our two-stage deep neural network, which includes two deep neural subnets (i.e., 3D IFPN and UDS-GCN). Limited by the large amount of computer memory consumed by the 3D task, we first develop 3D IFPN to extract common and discriminative multi-level features from the initial CT images. The proposed 3D IFPN takes an abdominal CT volume as input and starts with 3D FPN (Lin et al., 2017a) as backbone to acquire multi-level features at different resolutions, and it is designed as a
Datasets and data preprocessing
The datasets are collected from the four medical centers (Taiyuan People Hospital, China; Xian People Hospital, China; Department of Radiology, China-Japan Friendship Hospital, Beijing, China; Heping Hospital, Changzi Medical College, China) by three kinds of medical instruments (Toshiba 320-slice CT, SOMATOM 64-slice CT, Philips 128-slice CT and SOMATOM force dual source CT), with a largely varying in-plane resolution from 0.5 to 1.0 mm and slice spacing from 5.0 to 8.0 mm. Our dataset
Parameter analysis
In our 3D IFPN, there are two important parameters (i.e., γ and α) to be tuned. In validation strategy 1, we use the cross-entropy loss function to train 3D IFPN, and use the five-fold group cross-validation strategy. Since we ignore the problem of sample imbalance, Table 4 shows that the classification results have low specificity. In validation strategy 2, we consider using focal loss instead of cross-entropy loss to alleviate the problem of sample imbalance. To investigate the impact of the
Conclusions
In this paper, we propose to investigate MSDA for preoperative prediction of LN metastasis in gastric cancer. To tackle the limitations of 3D data training in multi-source domains, we first propose a multi-task 3D IFPN, which extracts common multi-level features for implementation of the MSDA scenario. The key idea of 3D IFPN is to select useful complementary information from multi-level features by region-based SAT and global-based CAT to improve its ability to recognize small targets. Then,
Authorship conformation form
Please check the following as appropriate:
All authors have participated in (a) conception and design, or analysis and interpretation of the data; (b) drafting the article or revising it critically for important intellectual content; and (c) approval of the final version.
CRediT authorship contribution statement
Yongtao Zhang: Conceptualization, Visualization, Formal analysis, Data curation, Writing – original draft, Writing – review & editing. Ning Yuan: Conceptualization, Formal analysis, Data curation, Writing – review & editing. Zhiguo Zhang: Conceptualization, Formal analysis, Writing – review & editing. Jie Du: Conceptualization, Formal analysis, Writing – review & editing. Tianfu Wang: Conceptualization, Formal analysis. Bing Liu: Visualization, Data curation. Aocai Yang: Visualization, Data
Declaration of Competing Interest
None.
Acknowledgments
This work was supported partly by National Natural Science Foundation of China (Nos. 62071309, 61871274, 81971585 and 62006160), National Natural Science Foundation of Guangdong Province (Nos. 2019B1515120029 and 2019A1515111205), Shenzhen Key Basic Research Project (Nos. SGDX20201103095802007, KCXFZ20201221173213036, JCYJ20180507184647636, JCYJ20190808155618806, GJHZ20190822095414576, and JCYJ20190808145011259), National Key Research and Development Program of China (Nos. 2020YFC2003903,
References (55)
- et al.
A combined deep-learning and deformable-model approach to fully automatic segmentation of the left ventricle in cardiac MRI
Med. Image Anal.
(2016) - et al.
An intelligent clinical decision support system for preoperative prediction of lymph node metastasis in gastric cancer
J. Am. Coll. Radiol.
(2019) - et al.
The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: results of the KiTS19 challenge
Med. Image Anal.
(2021) - et al.
Self-weighted adaptive structure learning for ASD diagnosis via multi-template multi-center representation
Med. Image Anal.
(2020) - et al.
A support vector machine-recursive feature elimination feature selection method based on artificial contrast variables and mutual information
J. Chromatogr. B
(2012) - et al.
Disease prediction using graph convolutional networks: application to autism spectrum disorder and Alzheimer's disease
Med. Image Anal.
(2018) - et al.
Domain adaptation for Alzheimer's disease diagnostics
Neuroimage
(2016) - et al.
Distribution-matching embedding for visual domain adaptation
J. Mach. Learn. Res.
(2016) - et al.
Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries
CA Cancer J. Clin.
(2018) - et al.
Dynamic region-aware convolution
Joint distribution optimal transportation for domain adaptation
Adv. Neural Inf. Process. Syst.
Optimal transport for domain adaptation
IEEE Trans. Pattern Anal. Mach. Intell.
Sinkhorn distances: Lightspeed computation of optimal transport
Adv. Neural Inform. Process. Syst.
Deepjdot: Deep joint distribution optimal transport for unsupervised domain adaptation
Domain-adversarial training of neural networks
J. Mach. Learn. Res.
CE-Net: context encoder network for 2D medical image segmentation
IEEE Trans. Med. Imaging
Cycada: cycle-consistent adversarial domain adaptation
Proceedings of the International Conference on Machine Learning
PMLR
Squeeze-and-excitation networks
Preoperative staging of gastric cancer by endoscopic ultrasonography and multidetector-row computed tomography
J. Gastroenterol. Hepatol.
Contrastive adaptation network for single-and multi-source domain adaptation
IEEE Trans. Pattern Anal. Mach. Intell.
Contrastive adaptation network for unsupervised domain adaptation
Imaging in local staging of gastric cancer: a systematic review
J. Clin. Oncol.
H-DenseUNet: hybrid densely connected UNet for liver and tumor segmentation from CT volumes
IEEE Trans. Med. Imaging
Selective kernel networks
Feature pyramid networks for object detection
Focal loss for dense object detection
Cited by (5)
Noninvasive Assessment of HER2 Expression Status in Gastric Cancer Using <sup>18</sup>F-FDG Positron Emission Tomography/Computed Tomography-Based Radiomics: A Pilot Study
2024, Cancer Biotherapy and RadiopharmaceuticalsNucleus Detection Based on Adversarial Domain Adaptation with Cross-Domain Consistency
2023, Proceedings - 2023 1st IEEE International Conference on Medical Artificial Intelligence, MedAI 2023Boundary Guided Semantic Learning for Real-Time COVID-19 Lung Infection Segmentation System
2022, IEEE Transactions on Consumer Electronics