Deep convolutional neural network for accurate segmentation and quantification of white matter hyperintensities
Introduction
White matter hyperintensity (WMH) and ischemic stroke appear as regions of abnormally signal intensity on magnetic resonance image (MRI) images. WMH is commonly observed in elderly people and ischemic stroke disease patients [1]. Studies have shown that the presence of WMH increases the risk of ischemic stroke and the probability of death [2], [3]. Accurate characterization of the location and shape of WMH is important for diagnosis and prognosis of ischemic stroke diseases. While WMH and ischemic stroke lesion have different pathologies, they not only coexist in the cortical or sub-cortical, but also have consistent signal intensity in the fluid-attenuated inversion recovery (FLAIR) and the longitudinal relaxation time (T1) magnetic resonance imaging (MRI) images [4]. Differentiation of these two types of lesions could be error-prone and time-consuming for clinicians.
Manual segmentation done by the for clinicians to quantify WMHs is usually performed on FLAIR images. However, WMH usually has small, irregular and scattered form making precise manual segmentation difficult. Moreover, the WMH and ischemic stroke lesion share similar characteristics, that is, both lesion tissues are hyperintense in the FLAIR MRI and in the T1 MRI. Fig. 1(a) and (b) shows example FLAIR and T1 MRI images, respectively. Fig. 1(c) shows annotated WMHs and ischemic stroke lesions where green areas stand for WMH and red areas for stroke lesions. Fig. 1(d) separates all WMHs. As shown in Fig. 1(a) and (b), the boundary that separates the WMHs and ischemic stroke lesions is fuzzy and difficult to determine precisely. However, clinicians still need to correctly annotate WMHs shown in Fig. 1(d). Thus, the development of accurate and automatic tools for the segmentation of the WMH from ischemic stroke lesions is important to support the clinicians, and in the long run to develop treatment strategies and track disease progression.
Section snippets
Related work
Accurate segmentation and quantification of WMH are applications within a broader field of the computational analysis of the MRI images of the brain. A number of studies in the area of WMH detection and segmentation were published in the past [5], [6], [7]. The research efforts in this area are increasing. We review related works that focus on the WMH segmentation.
Before the machine learning methods were introduced to the medical image segmentation, most of the WMH segmentation techniques were
Data
We use multi-modality MRIs as training and test cases. All cases come from two public benchmark challenges: WMH challenge1 and SISS challenge 20152. The two challenges concern the WMH and ischemic stroke segmentation tasks, respectively. Both challenges have cases with both WMH and ischemic stroke lesions. However, neither challenge aims to differentiate the two similar lesions.
As shown in Table 1, the WMH challenge provides the
Methods
Recent years have seen an increased interest in the application of the deep learning models, a subtype of the supervised models, for the image segmentation. The main reason for that is these models often provide an improved predictive performance for the medical image segmentation, when compared with the other types of supervised and unsupervised models. We propose a deep learning network (M2DCNN) to accurately segment the WMHs and distinguish them from the ischemic stroke lesions.
Experimental setup
We describe the data used in this study in Section 3.1. Briefly, we collected the training and test samples from the WMH 2017 and SISS 2015 challenge. The MRI images have two modalities: FLAIR and T1. The test images are divided into four types as shown in Table 2. We use the deep network parameters from the training epoch with the lowest loss value to implement the M2DCNN method when performing tests on the test cases. The training process stops when ten consecutive epochs have the same low
Ablation study
We perform several ablation experiments to validate the value provided by the innovative elements that we introduce in M2DCNN.
Discussion
The high quality segmentation of WMH would provide useful information for the clinicians. We design, implement and test a novel deep neural network M2DCNN for the accurate and automatic segmentation of the WMHs, which particularly focuses on the problem of differentiation of these lesions from the stroke lesions. Our extensive empirical tests suggest that M2DCNN outperforms other state-of-the-art methods.
The U-shaped network were previously shown to perform well in the image segmentation tasks
Conclusion
WMH and ischemic stroke lesions appear as similar signals in MRI images, making it difficult to accurately segment WMHs. Our objective is to introduce a new tool that can accurately segment WMHs and differentiate them from the ischemic stroke lesions. To this end we design, implement and test a deep neural network called M2DCNN. Our design extends the previously developed networks with several important innovations that include the use of dense and dilated blocks, multi-scale features. This
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
This work was supported by the National Natural Science Foundation of China under grants No. 61772557, No. 61772552; the 111 Project (No. B18059); and the Hunan Provincial Science and Technology Program (2018WK4001); Shanghai Municipal Science and Technology Major Project (2018SHZDZX01) and ZJLab.
Liangliang Liu received his M.S. degree from Henan University in 2014. He is currently a Ph.D. candidate in School of Computer Science and Engineering, Central South University, Changsha, Hunan, P.R. China. His research interests include machine learning, deep learning and medical image analysis.
References (55)
- et al.
Neuroimaging standards for research into small vessel disease and its contribution to ageing and neurodegeneration
Lancet Neurol.
(2013) - et al.
White matter lesion segmentation based on feature joint occurrence probability and random field theory from magnetic resonance (mr) images
Pattern Recognit. Lett.
(2010) - et al.
3D graph cut with new edge weights for cerebral white matter segmentation
Pattern Recognit. Lett.
(2011) - et al.
Fully convolutional network ensembles for white matter hyperintensities segmentation in mr images
Neuroimage
(2018) - et al.
Fully automatic segmentation of white matter hyperintensities in mr images of the elderly
Neuroimage
(2005) - et al.
Automatic white matter lesion segmentation using an adaptive outlier detection method
Magn. Reson. Imaging
(2012) - et al.
Development and validation of morphological segmentation of age-related cerebral white matter hyperintensities
Neuroimage
(2009) - et al.
Computer-aided evaluation method of white matter hyperintensities related to subcortical vascular dementia based on magnetic resonance imaging
Comput. Med. Imaging Graph.
(2010) - et al.
Accurate white matter lesion segmentation by k nearest neighbor classification with tissue type priors (knn-ttps)
Neuroimage Clin.
(2013) - et al.
Probabilistic segmentation of white matter lesions in mr imaging
Neuroimage
(2004)
Computer-assisted segmentation of white matter lesions in 3d mr images using support vector machine
Acad. Radiol.
Automatic segmentation of mr brain images with a convolutional neural network
IEEE Trans. Med. Imaging
Automated segmentation and quantification of white matter hyperintensities in acute ischemic stroke patients with cerebral infarction
Plos One
Deepedge: a multi-scale bifurcated deep network for top-down contour detection
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Deep convolutional neural networks for multi-modality isointense infant brain image segmentation
NeuroImage
A large margin algorithm for automated segmentation of white matter hyperintensity
Pattern Recognit.
Tractseg - fast and accurate white matter tract segmentation
NeuroImage
Isles 2015 - a public evaluation benchmark for ischemic stroke lesion segmentation from multispectral mri
Med. Image Anal.
Densely connected convolutional networks
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
White matter hyperintensity on cranial magnetic resonance imaging: a predictor of stroke
Stroke
Development of white matter hyperintensity is preceded by reduced cerebrovascular reactivity
Ann. Neurol.
Rationale, design and methodology of the image analysis protocol for studies of patients with cerebral small vessel disease and mild stroke
Brain Behav.
Application of variable threshold intensity to segmentation for white matter hyperintensities in fluid attenuated inversion recovery magnetic resonance images
Neuroradiology
Validation of a regression technique for segmentation of white matter hyperintensities in alzheimer’s disease
IEEE Trans. Med. Imaging
Validation of t1w-based segmentations of white matter hyperintensity volumes in large-scale datasets of aging
Hum. Brain Mapp.
Extracting and summarizing white matter hyperintensities using supervised segmentation methods in alzheimer’s disease risk and aging studies
Hum. Brain Mapp.
Cited by (24)
Deep learning-based covert brain infarct detection from multiple MRI sequences
2023, NeurocomputingA spatiotemporal correlation deep learning network for brain penumbra disease
2023, NeurocomputingCitation Excerpt :In addition, building an end-to-end model with conventional methods is usually not straightforward. CNNs have been rapidly adopted for medical imaging analysis tasks [16–19], and represent the main building blocks in CAD systems for brain image segmentation [20–23]. The encoder-decoder is a advanced architecture that is used for segmentation tasks [24–27].
An enhanced multi-modal brain graph network for classifying neuropsychiatric disorders
2022, Medical Image AnalysisBrain MR images segmentation using 3D CNN with features recalibration mechanism for segmented CT generation
2022, NeurocomputingCitation Excerpt :It has been proven that the MR image segmentation based method for PET quantification is a simple and robust approach which has been applied in commercial PET/MR scanners [13]. Deep convolutional neural network (CNN) has been applied successfully to perform segmentation tasks in different medical imaging applications [14–18]. Various fully convolutional segmentation network architectures have been proposed, yet few of them emphasis on the characterization of spatial-wise and channel-wise features and patterns.
Automatic detection of white matter hyperintensities via mask region-based convolutional neural networks using magnetic resonance images
2022, Deep Learning for Medical Applications with Unique Data
Liangliang Liu received his M.S. degree from Henan University in 2014. He is currently a Ph.D. candidate in School of Computer Science and Engineering, Central South University, Changsha, Hunan, P.R. China. His research interests include machine learning, deep learning and medical image analysis.
Shaowu Chen received his M.S degree from Guizhou Medical University in 2012. He is currently an attending physician in Department of Pathology, Pingdingshan First People’s Hospital, Pingdingshan, P.R. China. His research interests include medical image analysis and diagnosis.
Xiaofeng Zhu obtained his PhD in computer science from The University of Queensland, Australia, and is focusing on mining useful knowledge or information from big multimedia or medical imaging data.
Xing-Ming Zhao received his Ph.D. degree from the University of Science and Technology of China in 2005. Currently, he is a professor at the Institute of Science and Technology for Brain-Inspired Intelligence (ISTBI), Fudan University. His research focuses on data mining and bioinformatics. He has published more than 70 journal papers, and is a senior member of IEEE, co-chair of IEEE SMC Technical Committee on Systems Biology and ACM SIGBio China. He is also the lead guest editor and editor member of several journals, e.g. IEEE/ACM TCBB and Neurocomputing.
Fang-Xiang Wu received the B.Sc. degree and the M.Sc. degree in applied mathematics, both from Dalian University of Technology, Dalian, China, in 1990 and 1993, respectively, the first Ph.D. degree in control theory and its applications from Northwestern Polytechnical University, Xi’an, China, in 1998, and the second Ph.D. degree in biomedical engineering from University of Saskatchewan (U of S), Saskatoon, Canada, in 2004. During 2004–2005, he worked as a Postdoctoral Fellow in the Laval University Medical Research Center (CHUL), Quebec City, Canada. He is currently a Professor of the Division of Biomedical Engineering and the Department of Mechanical Engineering at the U of S. His current research interests include computational and systems biology, genomic and proteomic data analysis, biological system identification and parameter estimation, applications of control theory to biological systems. Dr. Wu is serving as the editorial board member of five international journals, the guest editor of several international journals, and as the program committee chair or member of several international conferences. He has also reviewed papers for many international journals.
Jianxin Wang received his B.S. and M.S. degree in Computer Science from Central South University of Technology, P. R. China, and his PhD degree in Computer Science from Central South University. Currently, he is the vice dean and a professor in School of Computer Science and Engineering, Central South University, Changsha, Hunan, P.R. China. He is currently serving as the executive editor of International Journal of Bioinformatics Research and Applications and serving in the editorial board of International Journal of Data Mining and Bioinformatics. He has also served as the program committee member for many international conferences. His current research interests include algorithm analysis and optimization, parameterized algorithm, bioinformatics and computer network. He has published more than 200 papers in various International journals and refereed conferences. He is a senior member of the IEEE.