Abstract
Cerebral infarct volume measured in follow-up non-contrast CT (NCCT) scans is an important radiologic outcome measure evaluating the effectiveness of endovascular therapy of acute ischemic stroke (AIS) patients. In this paper, a dense Multi-Path Contextual Generative Adversarial Network (MPC-GAN) is proposed to automatically segment ischemic infarct volume from NCCT images of AIS patients. The developed MPC-GAN approach makes use of a dense multi-path U-Net as generator regularized by a discriminator network. Both generator and discriminator take contextual information as inputs, such as bilateral intensity difference, infarct location probability, and distance to cerebrospinal fluid (CSF). We collected 100 NCCT images with manual segmentations. Of 100 patients, 60 patients were used to train the MPC-GAN, 10 patients were used to tune the parameters, and the remained 30 patients were used for validation. Quantitative results in comparison with manual segmentations show that the proposed MPC-GAN achieved a dice coefficient (DC) of 72.6%, outperforming some state-of-the-art segmentation methods, such as U-Net, U-Net based GAN, and random forest based segmentation method.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
Endovascular therapy (EVT) has become the standard of care in patients with anterior circulation ischemic stroke [1]. Cerebral infarct volume measured on follow-up non-contrast CT (NCCT) scans of acute ischemic stroke (AIS) patients is an important radiologic outcome evaluating the EVT effectiveness [2], in addition to clinical measures, such as Modified Rankin Scale (mRS) and the National Institute of Health Stroke Scale (NIHSS). Measurement of post-treatment infarct volume, however, relies on manual segmentation in practice, which is tedious, time-consuming, and observer variant. A fully automated infarct segmentation approach would be desired to objectively measure infarct volume from the follow-up NCCT scans of AIS patients.
Accurately segmenting ischemic infarct from NCCT images is challenging, due to the poor image quality of brain CT scans with low signal to noise ratio (SNR), anisotropy, interference from chronic infarcts, leukoaraiosis, and partial volume effect around the cerebrospinal fluid (CSF). There are no well established methods for automatically segmenting ischemic lesions from CT scans, in spite of some successful approaches for MR images [3]. Existing automatic infarct segmentation approaches in NCCT are mostly based on machine learning by utilizing information including intensity, image texture, hemisphere attenuation distributions [4]. Most of these automated methods, however, required extensive post-editing. In order to reduce the detected false positive lesion voxels, human knowledge was used to guide the segmentation, leading to some semi-automated approaches [5, 6]. Although these methods achieved favorable segmentation accuracy, observer variability was introduced.
In this paper, a deep learning technique, Multi-Path Contextual Generative Adversarial Network (MPC-GAN), is proposed to automatically segment ischemic infarct from follow up NCCT images of AIS patients. The developed MPC-GAN approach makes use of a dense multi-path U-Net [7] as generator regularized by a discriminator network. Both generator and discriminator take contextual information as inputs, such as bilateral intensity difference, infarct location probability, and distance to cerebrospinal fluid (CSF). In order to account for this class imbalanced segmentation problem, a boundary loss based on distance metric is additionally introduced to mitigate the difficulties of regional loss using cross entropy for the generator in the proposed MPC-GAN.
2 Automated Infarct Segmentation Approach Using Dense Multi-Path Contextual GAN (MPC-GAN)
2.1 MPC-GAN Architecture
Generative adversarial networks (GAN) typically consists of two networks: a generator (Fig. 1(a)-A) and a discriminator (Fig. 1(a)-B) [8]. The generator is used to generate the output that ‘fools’ discriminator to distinguish from gold standard, while the discriminator tries to differentiate the gold standard from the ’fake’ output of the generator. In this study, we design two GAN structures to tackle this segmentation problem, a single path GAN (sGAN) without connections (1) and (2) in Fig. 1(a) and a MPC-GAN in Fig. 1(b). In the sGAN setting, a U-Net structure is used for the generator [7] while pixel-wise authenticity is determined for the discriminator (Fig. 1(a)). The sGAN only takes NCCT intensity information as input. The proposed MPC-GAN employs a densely connected 4-path U-Net structure for the generator, and both inputs and outputs of each path are connected to the pixel-wise discriminator (Fig. 1(b)). In addition to NCCT intensity used in the first path, image contexts are used as inputs for the other three paths, such as bilateral intensity difference of brain, spatial probability of ischemic infarct occurrence, and distance between any voxel and cerebrospinal fluid (CSF) regions. The bilateral difference is calculated by subtracting the original brain from its mirrored brain. The mirrored brain image is generated by deformable registration of horizontally flipped brain to the original brain. The spatial probability of ischemic infarct occurrence is obtained by non-linearly registering a NCCT atlas with a lesion location probability map onto the subject image. The distance map is calculated by applying Euclidean distance transform on the binary mask of CSF. These extracted contextual information are also input as multi-channels into the sGAN, leading to another version mGAN, shown as sGAN with two additional connections (1) and (2) in Fig. 1(a)).
2.2 Loss Function
The generator \(\mathcal {G}\) is a mapping from the input \(x_g\) to the ground truth, the manually segmented infarct image y, i.e., \(\mathcal {G}: (x_g,\varTheta _\mathcal {G}) \rightarrow y\), where \(\varTheta _\mathcal {G}\) is the parameters of the generator. The discriminator \(\mathcal {D}\) maps a pair of \(\{x_d,y\}\) to pixel wise binary classification \(\{0,1\}^{H \times W}\) with the parameter set \(\varTheta _\mathcal {D}\), where \(x_d\) is the input of the discriminator, \(\{0,1\}\) denotes that a pixel in y is algorithm segmented infarct or expert segmented infarct, and H and W denote the height and width of the input image. In our GAN framework, \(\mathcal {D}\) is trained by maximizing the probability of assigning the correct label to both training examples and the samples from \(\mathcal {G}\), and \(\mathcal {G}\) is simultaneously trained by minimizing the difference between the algorithm generated infarct label and ground truth. The loss function for the discriminator is formulated as
where \(\mathbb {E}(.)\) denotes the empirical estimate of expected value of the probability. To train a discriminator \(\mathcal {D}\) that can make correct judgement, \(\mathcal {D}(x_d, y)\) should be maximized while \(\mathcal {D}(x_d, \mathcal {G}(x_g))\) should be minimized. The generator should prevent the discriminator from making correct judgement by generating outputs that are difficult to be distinguished from the real data, and match the ground truth as well as possible. Therefore, the loss function for the generator should ensure that the generator can make its output matching very well with the gold standard, and is defined as:
where \(\lambda \) and \(\alpha \) are used to balance the three loss terms. \(\mathrm CE\) is the cross entropy to minimize the distance between the gold standard and generator’s output. In addition to this, a boundary loss term is particularly introduced and combined into the segmentation loss function in this study in order to mitigate the difficulties of cross entropy loss in this segmentation task with highly unbalanced sample distribution. Inspired by the work in [9], the boundary loss \(\mathcal {L}_B\) is calculating the changes between two regions of ground truth y and \(\mathcal {G}(x_g)\) (Fig. 2). Specifically, \(\mathcal {L}_B\) linearly sums the regional softmax probability outputs of the generator in the symmetrical difference region between y and \(\mathcal {G}(x_g)\) while weighted by a distance function, and is defined in Eq. (3) in which \(y\varDelta \mathcal {G}(x_g)\) denotes symmetrical difference of two regions, and \(\phi (y)\) is a level set function pre-computed from the ground-truth region y, which is used to measure a distance between a point in \(y\varDelta \mathcal {G}(x_g)\) and the boundary of y. Note that it is not necessary to provide this level set function during inference as prediction maps of the generator are achieved by a forward pass through the network whose parameters are already trained to generate segmentations.
The objective function of the MPC-GAN sums up both the discriminator and generator loss as:
3 Experiments and Results
Image Acquisitions: We collected 100 follow up NCCT scans of AIS patients with manual segmentations. Of 100 patients, 60 patients were used to train the MPC-GAN, 10 patients were used to optimize the parameters used in the method, and the remained 30 patients were used for validating the proposed MPC-GAN model. All infarcts were manually contoured by a well-trained clinician, and then verified by another neuroradiologist with more than 5 years’ experience using the ITK-SNAP software.
Implementation Details: The NCCT images, difference maps, infarct location maps, and distance maps were normalized using the z-score method. Online data augmentation was utilized to increase the diversity of samples. In the generator, U-Net in each path had 7 layers with 64 input features and max pooling kernel size of 2. The discriminator had 6 layers. Except the output convolution layers, all other convolution layers utilized ReLU activation function and batch normalization. Following the output convolution layer of the generator, softmax method was used to obtain the infarct probability map, which was used to generate segmentation results and was also used as inputs for the discriminator. Adam optimizer with a fixed learning rate of 0.0002 and \(\beta _1\) of 0.4 was used to optimize the defined discriminator and generator loss functions. The \(\lambda \) and \(\alpha \) in Eq. 2 are fixed to 0.1 and 0.5 respectively. The maximum number of iterations was set to 50,000 with a batch size of 5. The number of epochs is 50. All the parameters were tuned using the 10 patients for evaluations by changing one specific parameter while keeping the other parameters unchanged. All the tuned parameters were kept constant during the validation stage. All experiments were performed on a Linux workstation with 2.8 GHz Xeon processors (Intel, Santa Clara, CA, USA) with 16 GB RAM and a NVIDIA Quadro M4000 GPU using Python with Tensorflow.
Evaluation Metrics: Dice coefficient (DC), mean absolute surface distance (MAD), and maximum absolute surface distance (MAXD) with 95% percentile [10, 11] were utilized to quantitatively evaluate the performance of the proposed algorithm compared to the ground truth of manual segmentations. Besides, volumetric analyses using Pearson correlation and Bland-Altman were also performed.
Comparison Benchmarks: Our proposed MPC-GAN method was compared with a few other methods: U-Net [7]; single path GAN (sGAN) taking the first path only with intensity information as input; single path GAN with multi-channel inputs (intensity, bilateral difference, distance, and lesion probability) (mGAN) shown in Fig. 1(a)); and random forest method (RF) based on intensity, bilateral difference, distance, and lesion probability features [12]; For fair comparisons, all CNN methods shared the same pre-processed training images and the same loss functions for both generators and discriminators. The Otsu method was used to generate binary segmentation results from the probability maps output by each machine learning algorithm. The final segmentation for each method was obtained by removing isolated islands using morphological operations.
3.1 Results
Two examples of infarct segmentation are shown in Fig. 3. Quantitative results using metrics of DC, MAD, and MAXD are shown in Table 1 and box-plot of DC is shown in Fig. 4. The proposed MPC-GAN method obtained the highest DC (\(72.6\pm 12.5\)%) while keeping the MAD (\(3.9\pm 1.9\) mm) and MAXD (\(10.7\pm 5.8\) mm) the lowest compared to other four methods. One-way ANOVA analysis followed by multiple comparisons using the Dunn-Sidak method shows that the proposed MPC-GAN method is significantly better than the other 4 methods when assessing DC, MAD and MAXD (all \(p<0.01\)). Volume analyses show that the MPC-GAN segmented infarct volume is strongly correlated to the manually segmented infarct volume (\(r=0.936\), 95% confidence interval (CI): 0.895-0.960, \(p<0.001\)), and Bland-Altman plot shows that the mean difference between manual and MPC-GAN segmented volume is 7.0 \(\mathrm{mL}\). The average computational time per patient was \(2.05 \pm 0.63~\mathrm{min}\) including contextual information calculations.
4 Discussion and Conclusion
A novel segmentation approach is proposed to automatically segment ischemic infarct from post-treatment NCCT scans of acute ischemic patients, which makes use of a dense multi-path contextual generative adversarial network (MPC-GAN). Besides intensity information, image context including bilateral intensity difference, infarct location probability, and distance to CSF, are used to train the proposed MPC-GAN. In addition, a boundary loss is introduced to optimize the generator’s loss function in order to mitigate the difficulties of cross entropy for imbalanced segmentation problem. Quantitative evaluations against manual segmentation on 30 AIS patient NCCT images show that the proposed MPC-GAN is accurate regarding the metrics of DC, MAD, MAXD, and infarct volume, outperforming the state-of-the-art segmentation methods, such as U-Net and single path GAN.
The proposed MPC-GAN achieved a DC of \(72.6\%\), an increase of \(\approx 9\) % when compared to mGAN that used single path GAN structure while taking contextual information as multi-channel inputs. Additionally, the proposed MPC-GAN obtained faster convergence for both generator and discriminator than mGAN in our experiments. The mGAN method increased DC by \({\approx }7\) % compared to single path sGAN taking the intensity information as inputs only, demonstrating the efficacy of introducing contextual information into the deep learning framework. In our experiments, the random forest based method surprisingly obtained better results than another deep learning technique of U-Net. There were only a few methods proposed for auto- or semi-automatic segmentations of ischemic infarcts in CT scans in the literature. The DC obtained by the proposed MPC-GAN is higher than the \(67\%\) reported by an automated segmentation method [4]. Although the DC in this work is lower than the values reported by some semi-automatic segmentation methods, such as \(74\%\) in [13] and \(85\%\) in [5], the advantage of the fully automatic segmentation over semi-automatic segmentations eliminates any subjective interference of observer variability. Moreover, the MPC-GAN segmented infarct volume is strongly correlated to the gold standard of manual segmented volume, suggesting its clinical relevance.
This study has limitations. First, the proposed MPC-GAN segmentation does not address the cases where ischemic infarct and hemorrhagic transformation co-exist after treatment, which accounts for \(\approx \)10% of NCCT scans. Second, extra computational time is required to extract the defined contextual features. Third, a limited number of images were used to validate the proposed method. Validation on a larger population is required.
To summarize, a novel segmentation approach using dense multi-path contextual generative adversarial network is proposed to segment ischemic infarct from post-treatment NCCT scans of AIS patients. Quantitative evaluations using DC, MAD, and MAXD show that the proposed approach achieves the state-of-the-art performance. It is anticipated that the proposed method will permit automatic delineation of ischemic infarct, therefore providing an objective and efficient tool to measure radiologic outcome in AIS patients for both research and clinical purpose.
References
Goyal, M., et al.: Randomized assessment of rapid endovascular treatment of ischemic stroke. N. Engl. J. Med. 372(11), 1019–1030 (2015)
Boers, A.M., et al.: Mediation of the relationship between endovascular therapy and functional outcome by follow-up infarct volume in patients with acute ischemic stroke. JAMA Neurol. 76, 194–202 (2019)
Winzeck, S., et al.: ISLES 2016 and 2017-benchmarking ischemic stroke lesion outcome prediction based on multispectral MRI. Front. Neurol. 9, 679 (2018)
Gillebert, C.R., Humphreys, G.W., Mantini, D.: Automated delineation of stroke lesions using brain CT images. NeuroImage Clin. 4, 540–548 (2014)
de Haan, B., Clas, P., Juenger, H., Wilke, M., Karnath, H.O.: Fast semi-automated lesion demarcation in stroke. NeuroImage Clin. 9, 69–74 (2015)
Kuang, H., Menon, B.K., Qiu, W.: Segmenting hemorrhagic and ischemic infarct simultaneously from follow-up non-contrast CT images in patients with acute ischemic stroke. IEEE Access 7, 39842–39851 (2019)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)
Kervadec, H., Bouchtiba, J., Desrosiers, C., Granger, É., Dolz, J., Ayed, I.B.: Boundary loss for highly unbalanced segmentation. arXiv preprint arXiv:1812.07032 (2018)
Qiu, W., Yuan, J., Ukwatta, E., Sun, Y., Rajchl, M., Fenster, A.: Prostate segmentation: an efficient convex optimization approach with axial symmetry using 3D TRUS and MR images. IEEE Trans. Med. Imaging 33(4), 1–14 (2014)
Qiu, W., et al.: Automatic segmentation approach to extracting neonatal cerebral ventricles from 3D ultrasound images. Med. Image Anal. 35, 181–191 (2017)
Kuang, H., Najm, M., Menon, B.K., Qiu, W.: Joint segmentation of intracerebral hemorrhage and infarct from non-contrast CT images of post-treatment acute ischemic stroke patients. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11072, pp. 681–688. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00931-1_78
Boers, A.M., et al.: Automated cerebral infarct volume measurement in follow-up noncontrast ct scans of patients with acute ischemic stroke. Am. J. Neuroradiol. 34(8), 1522–1527 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Kuang, H., Menon, B.K., Qiu, W. (2019). Automated Infarct Segmentation from Follow-up Non-Contrast CT Scans in Patients with Acute Ischemic Stroke Using Dense Multi-Path Contextual Generative Adversarial Network. In: Shen, D., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. MICCAI 2019. Lecture Notes in Computer Science(), vol 11766. Springer, Cham. https://doi.org/10.1007/978-3-030-32248-9_95
Download citation
DOI: https://doi.org/10.1007/978-3-030-32248-9_95
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32247-2
Online ISBN: 978-3-030-32248-9
eBook Packages: Computer ScienceComputer Science (R0)