Elsevier

Neurocomputing

Volume 384, 7 April 2020, Pages 231-242
Neurocomputing

Deep convolutional neural network for accurate segmentation and quantification of white matter hyperintensities

https://doi.org/10.1016/j.neucom.2019.12.050Get rights and content

Highlights

  • We propose a deep convolutional neural network for improving the quality of the segmentation WMHs.

  • This network consists of two subnets which uses several architectural innovations to provide very accurate segmentation.

  • We also propose a novel loss function based on lesion similarity and background similarity.

  • Our network outperforms current segmentation methods on two public segmentation challenges.

Abstract

White matter hyperintensities (WMHs) appear as regions of abnormally signal intensity on magnetic resonance imaging (MRI) images, that can be identified in MRI images of elderly people and ischemic stroke patients. However, manual segmentation and quantification of images with WMHs is laborious and time-consuming. Moreover, ischemic stroke lesion and WMHs appear as similar signals in MRI images, making it difficult to accurately segment the WMHs. Analysis of WMH-containing images is important for clinical diagnosis, and thus several segmentation methods have been proposed. However, these methods cannot accurately differentiate WMH and ischemic stroke lesions. We propose a deep convolutional neural network, M2DCNN, that can accurately segment WMHs and distinguish them from ischemic stroke lesions. M2DCNN consists of two subnets that rely on a set of novel multi-scale features and a novel architecture (inclusion of dense and dilated blocks). Our model is trained and evaluated on two public segmentation challenges with multi-modality MRI images. Empirical tests demonstrate that M2DCNN outperforms current segmentation methods. We empirically demonstrate that M2DCNN effectively separates WMHs from stroke lesions. Finally, ablation experiments reveal that both multi-scale features and architectural elements in our method contribute to the improved predictive performance.

Introduction

White matter hyperintensity (WMH) and ischemic stroke appear as regions of abnormally signal intensity on magnetic resonance image (MRI) images. WMH is commonly observed in elderly people and ischemic stroke disease patients [1]. Studies have shown that the presence of WMH increases the risk of ischemic stroke and the probability of death [2], [3]. Accurate characterization of the location and shape of WMH is important for diagnosis and prognosis of ischemic stroke diseases. While WMH and ischemic stroke lesion have different pathologies, they not only coexist in the cortical or sub-cortical, but also have consistent signal intensity in the fluid-attenuated inversion recovery (FLAIR) and the longitudinal relaxation time (T1) magnetic resonance imaging (MRI) images [4]. Differentiation of these two types of lesions could be error-prone and time-consuming for clinicians.

Manual segmentation done by the for clinicians to quantify WMHs is usually performed on FLAIR images. However, WMH usually has small, irregular and scattered form making precise manual segmentation difficult. Moreover, the WMH and ischemic stroke lesion share similar characteristics, that is, both lesion tissues are hyperintense in the FLAIR MRI and in the T1 MRI. Fig. 1(a) and (b) shows example FLAIR and T1 MRI images, respectively. Fig. 1(c) shows annotated WMHs and ischemic stroke lesions where green areas stand for WMH and red areas for stroke lesions. Fig. 1(d) separates all WMHs. As shown in Fig. 1(a) and (b), the boundary that separates the WMHs and ischemic stroke lesions is fuzzy and difficult to determine precisely. However, clinicians still need to correctly annotate WMHs shown in Fig. 1(d). Thus, the development of accurate and automatic tools for the segmentation of the WMH from ischemic stroke lesions is important to support the clinicians, and in the long run to develop treatment strategies and track disease progression.

Section snippets

Related work

Accurate segmentation and quantification of WMH are applications within a broader field of the computational analysis of the MRI images of the brain. A number of studies in the area of WMH detection and segmentation were published in the past [5], [6], [7]. The research efforts in this area are increasing. We review related works that focus on the WMH segmentation.

Before the machine learning methods were introduced to the medical image segmentation, most of the WMH segmentation techniques were

Data

We use multi-modality MRIs as training and test cases. All cases come from two public benchmark challenges: WMH challenge1 and SISS challenge 20152. The two challenges concern the WMH and ischemic stroke segmentation tasks, respectively. Both challenges have cases with both WMH and ischemic stroke lesions. However, neither challenge aims to differentiate the two similar lesions.

As shown in Table 1, the WMH challenge provides the

Methods

Recent years have seen an increased interest in the application of the deep learning models, a subtype of the supervised models, for the image segmentation. The main reason for that is these models often provide an improved predictive performance for the medical image segmentation, when compared with the other types of supervised and unsupervised models. We propose a deep learning network (M2DCNN) to accurately segment the WMHs and distinguish them from the ischemic stroke lesions.

Experimental setup

We describe the data used in this study in Section 3.1. Briefly, we collected the training and test samples from the WMH 2017 and SISS 2015 challenge. The MRI images have two modalities: FLAIR and T1. The test images are divided into four types as shown in Table 2. We use the deep network parameters from the training epoch with the lowest loss value to implement the M2DCNN method when performing tests on the test cases. The training process stops when ten consecutive epochs have the same low

Ablation study

We perform several ablation experiments to validate the value provided by the innovative elements that we introduce in M2DCNN.

Discussion

The high quality segmentation of WMH would provide useful information for the clinicians. We design, implement and test a novel deep neural network M2DCNN for the accurate and automatic segmentation of the WMHs, which particularly focuses on the problem of differentiation of these lesions from the stroke lesions. Our extensive empirical tests suggest that M2DCNN outperforms other state-of-the-art methods.

The U-shaped network were previously shown to perform well in the image segmentation tasks

Conclusion

WMH and ischemic stroke lesions appear as similar signals in MRI images, making it difficult to accurately segment WMHs. Our objective is to introduce a new tool that can accurately segment WMHs and differentiate them from the ischemic stroke lesions. To this end we design, implement and test a deep neural network called M2DCNN. Our design extends the previously developed networks with several important innovations that include the use of dense and dilated blocks, multi-scale features. This

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This work was supported by the National Natural Science Foundation of China under grants No. 61772557, No. 61772552; the 111 Project (No. B18059); and the Hunan Provincial Science and Technology Program (2018WK4001); Shanghai Municipal Science and Technology Major Project (2018SHZDZX01) and ZJLab.

Liangliang Liu received his M.S. degree from Henan University in 2014. He is currently a Ph.D. candidate in School of Computer Science and Engineering, Central South University, Changsha, Hunan, P.R. China. His research interests include machine learning, deep learning and medical image analysis.

References (55)

  • Z. Lao et al.

    Computer-assisted segmentation of white matter lesions in 3d mr images using support vector machine

    Acad. Radiol.

    (2008)
  • A. Khan, A. Sohail, U. Zahoora, A.S. Qureshi, A survey of the recent architectures of deep convolutional neural...
  • P. Moeskops et al.

    Automatic segmentation of mr brain images with a convolutional neural network

    IEEE Trans. Med. Imaging

    (2016)
  • J.Z. Tsai et al.

    Automated segmentation and quantification of white matter hyperintensities in acute ischemic stroke patients with cerebral infarction

    Plos One

    (2014)
  • G. Bertasius et al.

    Deepedge: a multi-scale bifurcated deep network for top-down contour detection

    IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    (2015)
  • W. Zhang et al.

    Deep convolutional neural networks for multi-modality isointense infant brain image segmentation

    NeuroImage

    (2015)
  • C. Qin et al.

    A large margin algorithm for automated segmentation of white matter hyperintensity

    Pattern Recognit.

    (2018)
  • J. Wasserthal et al.

    Tractseg - fast and accurate white matter tract segmentation

    NeuroImage

    (2018)
  • O. Maier et al.

    Isles 2015 - a public evaluation benchmark for ischemic stroke lesion segmentation from multispectral mri

    Med. Image Anal.

    (2017)
  • G. Huang et al.

    Densely connected convolutional networks

    IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    (2017)
  • K. Lewis H et al.

    White matter hyperintensity on cranial magnetic resonance imaging: a predictor of stroke

    Stroke

    (2004)
  • K. Sam et al.

    Development of white matter hyperintensity is preceded by reduced cerebrovascular reactivity

    Ann. Neurol.

    (2016)
  • C.H.M. Valdés et al.

    Rationale, design and methodology of the image analysis protocol for studies of patients with cerebral small vessel disease and mild stroke

    Brain Behav.

    (2016)
  • B.I. Yoo et al.

    Application of variable threshold intensity to segmentation for white matter hyperintensities in fluid attenuated inversion recovery magnetic resonance images

    Neuroradiology

    (2014)
  • M. Dadar et al.

    Validation of a regression technique for segmentation of white matter hyperintensities in alzheimer’s disease

    IEEE Trans. Med. Imaging

    (2017)
  • M. Dadar et al.

    Validation of t1w-based segmentations of white matter hyperintensity volumes in large-scale datasets of aging

    Hum. Brain Mapp.

    (2018)
  • V. Ithapu et al.

    Extracting and summarizing white matter hyperintensities using supervised segmentation methods in alzheimer’s disease risk and aging studies

    Hum. Brain Mapp.

    (2014)
  • Cited by (24)

    • A spatiotemporal correlation deep learning network for brain penumbra disease

      2023, Neurocomputing
      Citation Excerpt :

      In addition, building an end-to-end model with conventional methods is usually not straightforward. CNNs have been rapidly adopted for medical imaging analysis tasks [16–19], and represent the main building blocks in CAD systems for brain image segmentation [20–23]. The encoder-decoder is a advanced architecture that is used for segmentation tasks [24–27].

    • Brain MR images segmentation using 3D CNN with features recalibration mechanism for segmented CT generation

      2022, Neurocomputing
      Citation Excerpt :

      It has been proven that the MR image segmentation based method for PET quantification is a simple and robust approach which has been applied in commercial PET/MR scanners [13]. Deep convolutional neural network (CNN) has been applied successfully to perform segmentation tasks in different medical imaging applications [14–18]. Various fully convolutional segmentation network architectures have been proposed, yet few of them emphasis on the characterization of spatial-wise and channel-wise features and patterns.

    View all citing articles on Scopus

    Liangliang Liu received his M.S. degree from Henan University in 2014. He is currently a Ph.D. candidate in School of Computer Science and Engineering, Central South University, Changsha, Hunan, P.R. China. His research interests include machine learning, deep learning and medical image analysis.

    Shaowu Chen received his M.S degree from Guizhou Medical University in 2012. He is currently an attending physician in Department of Pathology, Pingdingshan First People’s Hospital, Pingdingshan, P.R. China. His research interests include medical image analysis and diagnosis.

    Xiaofeng Zhu obtained his PhD in computer science from The University of Queensland, Australia, and is focusing on mining useful knowledge or information from big multimedia or medical imaging data.

    Xing-Ming Zhao received his Ph.D. degree from the University of Science and Technology of China in 2005. Currently, he is a professor at the Institute of Science and Technology for Brain-Inspired Intelligence (ISTBI), Fudan University. His research focuses on data mining and bioinformatics. He has published more than 70 journal papers, and is a senior member of IEEE, co-chair of IEEE SMC Technical Committee on Systems Biology and ACM SIGBio China. He is also the lead guest editor and editor member of several journals, e.g. IEEE/ACM TCBB and Neurocomputing.

    Fang-Xiang Wu received the B.Sc. degree and the M.Sc. degree in applied mathematics, both from Dalian University of Technology, Dalian, China, in 1990 and 1993, respectively, the first Ph.D. degree in control theory and its applications from Northwestern Polytechnical University, Xi’an, China, in 1998, and the second Ph.D. degree in biomedical engineering from University of Saskatchewan (U of S), Saskatoon, Canada, in 2004. During 2004–2005, he worked as a Postdoctoral Fellow in the Laval University Medical Research Center (CHUL), Quebec City, Canada. He is currently a Professor of the Division of Biomedical Engineering and the Department of Mechanical Engineering at the U of S. His current research interests include computational and systems biology, genomic and proteomic data analysis, biological system identification and parameter estimation, applications of control theory to biological systems. Dr. Wu is serving as the editorial board member of five international journals, the guest editor of several international journals, and as the program committee chair or member of several international conferences. He has also reviewed papers for many international journals.

    Jianxin Wang received his B.S. and M.S. degree in Computer Science from Central South University of Technology, P. R. China, and his PhD degree in Computer Science from Central South University. Currently, he is the vice dean and a professor in School of Computer Science and Engineering, Central South University, Changsha, Hunan, P.R. China. He is currently serving as the executive editor of International Journal of Bioinformatics Research and Applications and serving in the editorial board of International Journal of Data Mining and Bioinformatics. He has also served as the program committee member for many international conferences. His current research interests include algorithm analysis and optimization, parameterized algorithm, bioinformatics and computer network. He has published more than 200 papers in various International journals and refereed conferences. He is a senior member of the IEEE.

    View full text