Attention convolutional neural network for accurate segmentation and quantification of lesions in ischemic stroke disease

Highlights

• Development of an advanced deep CNN architecture that aims to improve predictive performance and that allows for accurate and simultaneous prediction of both lesion types.

• We extend the popular U-net architecture by inclusion of novel modules, layers and the attention mechanism.

• Visualization of the output layers before and after attention module contacting path in proposed DRANet.

Abstract

Ischemic stroke lesion and white matter hyperintensity (WMH) lesion appear as regions of abnormally signal intensity on magnetic resonance image (MRI) sequences. Ischemic stroke is a frequent cause of death and disability, while WMH is a risk factor for stroke. Accurate segmentation and quantification of ischemic stroke and WMH lesions are important for diagnosis and prognosis. However, radiologists have a difficult time distinguishing these two types of similar lesions. A novel deep residual attention convolutional neural network (DRANet) is proposed to accurately and simultaneously segment and quantify ischemic stroke and WMH lesions in the MRI images. DRANet inherits the advantages of the U-net design and applies a novel attention module that extracts high-quality features from the input images. Moreover, the Dice loss function is used to train DRANet to address data imbalance in the training data set. DRANet is trained and evaluated on 742 2D MRI images which are produced from the sub-acute ischemic stroke lesion segmentation (SISS) challenge. Empirical tests demonstrate that DRANet outperforms several other state-of-the-art segmentation methods. It accurately segments and quantifies both ischemic stroke lesion and WMH. Ablation experiments reveal that attention modules improve the predictive performance of DRANet.

Introduction

Ischemic stroke is a major cerebrovasular disease that prevents blood from reaching brain regions by directly blocking blood vessels. White matter hyperintensity (WMH), which is also called the signal of the silent cerebral small vessel diseases (CSVDs), is an abnormality in the cerebral tissue Cloonan et al. (2015). WMH lesions are frequently observed in elderly individuals and ischemic stroke patients. The presence of WMH is a risk of future symptomatic ischemic stroke Lewis H et al. (2004); Cho et al. (2015). WMH is also used to monitor ischemic stroke lesion growth Hakan et al. (2008) and to predict post-stroke outcomes Arsava et al. (2011). Accurate identification and segmentation of these two types of lesions would improve the ability of physicians to correctly diagnose patients. However, ischemic stroke and WMH lesions may coexist in some areas of the brain. In addition, these two lesions have similar signals in the MRI images. For example, both lesions appear as hyperintensities in the T2 or fluid-attenuated inversion recovery (FLAIR) MRI, and as hypointensity in the longitudinal relaxation time (T1) MRI. We show two examples of ischemic stroke lesion and WMH on FLAIR MRIs in Fig. 1. Radiologists often struggle to distinguish these two types of lesions.

Radiologists usually delineate the lesions slice by slice in the FLAIR images. Presence of similar lesions and the ambiguity of their boundaries may impede correct segmentation of the ischemic stroke and WMH lesions. In addition, both lesions may have different sizes and shapes, which further complicates the segmentation process. Manual segmentation is a time-consuming work and the quality of manual segmentation depends on the level of experience of radiologists. Computer programs can learn from large collections of images to rival performance of experienced radiologists and do not tire when applied to process many images in a short amount of time. Thus, the development of accurate, concurrent and automatic segmentation for the two types of lesions is highly desirable.

Many methods have been developed for the brain image analysis tasks in the past few years Liu et al. (2017); Kong et al. (2019); Liu et al. (2020). We review methods that are used for the segmentation of the stroke lesions, WMH, and other CSVD lesions. These methods use medical images of the brain as input to segment the target with the help of several different types of algorithms that include k-nearest neighbor (KNN) Steenwijk et al. (2013); Boer et al. (2009), support vector machine (SVM) Lao et al. (2008); Vamsi et al. (2014), random forest (RF) Dadar et al. (2018); Mitra et al. (2014) and various designs of the convolutional neural network (CNN) methods Griffanti et al. (2016); Maier et al. (2017); Zhang et al. (2018); Li et al. (2018); Liu et al. (2019).

Several studies aim to perform the segmentation of the stroke or WMH lesions. These methods can be broadly divided into two main categories: unsupervised methods and supervised methods. The unsupervised methods do not require labeled images during the training process. For instance, Erin et al. (2010) proposed an automated WMH segmentation method based on fuzzy C-means (FCM) clustering and thresholding. Matus et al. (2010) developed an unsupervised method to assess the diagnosis and treatment decisions for the acute ischemic stroke. Nguyen et al. (2015) published a location-sensitive deep network to accurately locate stroke lesions. However, the accuracy of these unsupervised segmentation methods is relatively low. Recently, supervised methods, which require labeled data to train the predictive model, have gained popularity. Many classical supervised machine learning methods were used for the stroke lesion or WMH segmentation tasks. For instance, Mitraa et al. (2014) proposed a supervised RF model to segment ischemic stroke lesions. The authors first predicted the probable lesion volumes. Next, they used RF to extract lesion regions based on multi-modality MRIs followed by the thresholding that generates segmentation. Similarly, Dadar et al. (2018) developed a segmentation model based on the RF model. Lao et al. (2008) proposed the SVM model that was trained on expert-defined white matter lesions.

More recently, supervised CNN-based methods have become popular for the medical image segmentation Yu et al. (2019). These methods incorporate novel types of network modules, such as U-net Ronneberger et al. (2015), residual module Chen et al. (2018), dense module Zhang et al. (2018), dilated module Lessmann et al. (2018), and attention module Jin et al. (2018), to improve predictive performance when compared with classical CNNs and the other supervised machine learning algorithms. For instance, a deep uResNet was used to segment the WMH Guerrero et al. (2017). In uResNet, residual blocks were embedded into the U-net network to improve the speed of convergence and to solve the vanishing gradient problem in the back-propagation flow. Similarly, Chen et al. (2018) proposed a novel deep network which with 25 layers and voxel-wise residual module. This network integrates multi-modal and multi-level MRIs for brain segmentation. Zhang et al. (2018) proposed a deep CNN for the segmentation of the acute ischemic stroke lesion from DWI MRIs. They used weakly labeled subjects for training the predictive model, which limited the predictive performance of the model. Similarly, Li et al. (2018) proposed an ensemble of FCNs for the prediction of WMHs. The development of accurate advanced designs of CNNs for real-world clinical use, including the detection of small-volume lesions in MRI images, remains a challenging problem. Several studies changed the model structure to extract more abundant features. Lessmann et al. (2018) used two consecutive dilated convolutional neural networks to automatic detection of coronary artery, thoracic aorta and cardiac valve calcifications in low-dose chest CT. In addition, Vesal et al. (2018) have developed a deep CNN for the left atrial segmentation. Jin et al. (2018) published a novel design of CNN named RA-UNet to annotate liver and tumor in medical images. Moeskops et al. (2018) released a multi-scale convolutional neural network for brain tissue segmentation. Just this year, Nakarmi et al. (2020) proposed a novel multi-scale deep learning framework for medical image segmentation that combined the deep model and prior knowledge. While these models have achieved promising results for the lesion segmentation task, most of them focus on a single lesion type, and those that can segment multiple lesion types struggle to do it well.

We propose a state-of-the-art deep network DRANet to simultaneously segment the ischemic stroke and WMH lesions from MRIs. We introduce a novel attention module to improve the predictive quality of the proposed network. Our contributions include:

• Development of an advanced deep CNN architecture that aims to improve predictive performance and that allows for accurate and simultaneous prediction of both lesion types. We extend the popular U-net architecture by inclusion of novel modules, layers and the attention mechanism. Details are explained in Section 2.2.

• Inclusion of comprehensive empirical tests that compare our novel design with the current solutions using multi-modality MRIs on a modern benchmark data set.

• Analysis and visualization of the results produced by our network with the goal to explain the benefits of our novel design features.

The rest of this paper is organized as follows. Section 2 introduces the architecture of the proposed deep network and provides a quantitative evaluation of our segmentation network. Section 3 introduces the data used in our study. Section 4 conducts empirical comparative experiments and ablation experiments that investigate the contributions of individual components of our proposed network. Finally, we discuss and summarize this study in Sections 5 and 6, respectively.