Brain tumor segmentation from multimodal magnetic resonance images via sparse representation
Introduction
A brain tumor is an abnormal tissue proliferation that will cause an increase in intracranial pressure, resulting in central nervous system damage, hence endangering the lives of patients. The reliable detection and segmentation of brain tumors from magnetic resonance (MR) images can aid surgical planning and therapy assessments in medical diagnosis. Currently, most brain tumors are segmented manually by medical experts, although this method is time consuming. Moreover, the segmentation technology shows inter- and intra-rater variability, and the results are difficult to reproduce [1]. Therefore, computer-aided semi-automatic and automatic tumor segmentation is a promising method and plays an increasing role in modern medical analysis. Although a large improvement has been acquired in medical imaging technologies and a great deal of effort has been paid in tumor segmentation methods [2], developing an automatic, accurate and repeatable tumor segmentation algorithm remains a challenging task due to the unpredictable appearance, location, shape, and size of the tumors and overlap between the intensity range of healthy tissues and that of tumors. Moreover, because magnetic resonance imaging (MRI) is affected by noise, bias-field effect, partial volume effect, and tissular movement, MR images are usually blurred and non-uniform in intensity, which further increase the difficulty in tumor segmentation.
In the past few decades, various brain tumor segmentation algorithms have been presented that can be generally divided into two categories: semi-automatic and automatic. Semi-automatic segmentation methods mainly include active contour models [3], level-set models [4], [5], and the newest developed tumor-cut (TC) model [6], [7]. Compared with automatic segmentation approaches, semi-automatic segmentation approaches can achieve more accurate segmentation results, whereas this type of method needs manual initialization and user interaction. Automatic segmentation approaches mainly include brain atlases and registration based methods, probability based methods, and supervised classification based methods. These types of methods can be performed independently by computer without the participation of humans, but the segmentation result is unsatisfactory.
The brain atlas-guided segmentation methods [8], [9], [10], [11], [12] can provide important spatial information prior to using an aligned template to extract the spatial information of each type of healthy brain tissues and then segment the tumor and outlier by measuring the deviation from the normal brain. This type of method is actually a fusion approach based on registration, and its accuracy of segmentation relies heavily on the accuracy of registration. Due to the intensity variations around the tumor and deformation of healthy tissue morphology caused by edema and infiltration, it is still difficult to achieve accurate segmentation results.
The Bayesian approach [13] uses multivariate normal distribution to estimate the mean and covariance from a training data set. The class labels are obtained by maximum a posteriori (MAP) probability. In [14], Corso et al. incorporated the Bayesian model into a graph-based affinity calculation to segment glioblastoma multiforme (GBM) brain tumors and edema into multichannel MR volumes. In [15], the expectation maximization (EM) algorithm was used to perform a Gaussian mixture model (GMM) to distribute every pixel/voxel of T1-weighted (T1) MR images to the tissue class with the highest probability. Due to using only T1-weighted MR images, the segmentation accuracy is not satisfactory. Moreover, when only the single T1-weighted modality is used, it is intractable to separate the active tumor from the edema.
The supervised classification approach based on machine learning is an often-used brain tumor segmentation method. The technology is used to assign each pixel/voxel to a certain category according to the corresponding label statistics. The support vector machine (SVM) is the most popular supervised classification approach that maps the trained samples vector into a high-dimensional feature space and seeks an optimal separating hyperplane to classify the lesion tissues and healthy tissues [16], [17], [18]. For a binary classification task of small samples, the SVM classifier performs very well with a fast running speed and high classification accuracy. However, for the multi-classification problem, SVM is not an effective approach. Especially for high dimensional data, SVM is intractable. The random forest (RF) model is a multi-classification learning algorithm [19], [20], [21], [22] that is an ensemble of many binary decision trees. This method generates many classifiers and aggregates their results to obtain better decisions. However, for the classification problem, feature representation, in most cases, is more important than the choice of the classifier.
For MR brain tumor segmentation, the commonly used features are intensity-based features that show inter- and intra-rater variability [1] and are unstable; thus, this method is often sensitive to noise.
Sparse representation is a recently developed machine learning method. This approach can approximate images as a sparse linear combination of base atoms from a designed dictionary by adapting the dictionary to a set of training samples. Compared with traditional subspace learning models, sparse representation is more robust for data containing many outliers and sparse noise. Currently, sparse representation has been successfully applied in various vision tasks such as MR image reconstruction [23], face recognition [24], [25], image denoising [26], image super-resolution [27], and image classification [28].
The sparse representation-based classification (SRC) algorithm [24] is one of the most popular classification methods. However, this method was originally designed for face recognition purposes and did not consider the spatial constraint information. For the segmentation task, it is easy to lead to fragmentary segmentation results and decrease the segmentation accuracy because the SRC-based models did not consider the spatial dependencies between neighboring voxels. Currently, its various improved versions have been presented and have been successfully applied in medical image classification and segmentation. In [29], the extended SRC is used to segment MR brain images. In [30], Wang et al. proposed a patch-based sparse representation to segment brain MR images, adding spatial regularization via level sets into the model. In [31], the spatial constraints are generated by introducing Markov random field (MRF) regularization to an SRC model to segment MR brain tumor. In this SRC-based model, however, Zhan et al. directly used the original training samples as the base atoms of the dictionary. To make the dictionary discriminative, it is necessary to provide sufficient training samples from each class to reconstruct the dictionary, leading to an increased dictionary size and decreased speed. For big sample training data, it is intractable.
Moreover, for [30], speed becomes a large problem in the application of the semi-automatic model because real-time conditions are required for an interactive segmentation scheme. In [32], Li et al. not only considered the regularization constraints but also learned the adaptive dictionaries to segment MR brain tumors. Accordingly, these learned dictionaries gained the discriminative capability for classification, and the size of the dictionary also shrank simultaneously. However, this method segments MR brain tumors on 2D slices, the learned dictionaries will miss much of the contextual information, and there will be no smoothness across slices; thus, the dictionaries become non-rotation-invariant.
Inspired by the SRC model [24] and MRF with the graph cuts model [33], we present a sparse representation-based automatic brain tumor segmentation algorithm. Fig. 1 presents an overview of the proposed framework. This method formulates tumor segmentation as a MAP problem that casts the sparse reconstruction residual as a likelihood and incorporates the prior spatial smoothness into MRF and then solves the optimization problem of minimum energy using the graph cuts algorithm [34].
In contrast to the current SRC model [24] and its improved version [30], [31], we not only learn the adaptive dictionaries but also consider the spatial information between adjacent voxels and add a regularization smoothness term into the model; thus, the segmentation accuracy can be increased. Different from [31], we use the online dictionary learning (OLD) algorithm [35] to train our dictionary that is suitable for the big training sample data. In [31], the sparse coefficients were obtained by minimizing a non-negative Elastic-Net problem. Especially, in the work of Zhan et al., the authors only segmented the multimodal brain tumor into the edema and the tumor and the model was evaluated based on the single Jaccard score. In this paper, however, sparse coefficients were obtained by solving a ℓ1 minimization problem using the LARS-Lasso algorithm [36]. We segment the tumor into four different tumor classes: necrosis, edema, non-enhancing tumor and enhancing tumor. Moreover, we discuss in detail for different parameters and train the model with cross-validation. The segmentation results are evaluated by several metrics – the Dice, positive predictive value, sensitivity and Cohen's kappa. Compared with [30], our model is a fully automatic segmentation algorithm that does not require real-time performance, so speed is no longer a key problem for the automatic brain tumor segmentation model. Particularly, the sparse representation calculated from multiple modalities should be more powerful than previous representations from a single modality in capturing the complex structure of the brain tumor because the T1 modality edema and non enhancing tumor areas have a very close range of intensity values to gray matter, whereas the corresponding Flair modality shows the edema well by a hyper-intense signal around the tumor [8], not only making the learned dictionaries contain richer features, but also further enhancing the discriminant of the learned dictionary between the edema and active core tumor. This property was not involved in [30].
Because little valid prior knowledge can be used in brain tumor segmentation, we proposed a data-driven patch-based segmentation scheme, which learns a feature base space via dictionary learning for training samples and then projects the intensity information of the test images to dictionary space via a sparse coding strategy and acquires the sparse vectors.
This paper is an extended version of our work in the conference paper [32]. Compared to [32], we extend our model from 2D to 3D by extracting the cubic patch samples and extend the pixel-wise labeling problem of three classes to a voxel-wise labeling problem with five classes: necrosis, edema, non-enhancing tumor, enhancing tumor, and health brain tissues. Furthermore, we discuss the parameters within the model in detail and extend our experiment validation not only on real high-grade data, but also on real low-grade data. Although we did not do any post-processing, an obvious improvement on the segmentation results has been obtained.
The remainder of this paper is organized as follows. In Section 2, a probability model is formulated and the detailed algorithm implementation is described. Section 3 gives the experimental results and analysis. Section 4 discusses the parameters of the proposed model and Section 5 concludes this paper.
Section snippets
Method
Given a set of registered multimodal MR brain volumes of the same patient comprising four modalities with the same size – Flair, T1, contrast enhanced T1-weighted (T1C), and T2-weighted (T2) – we can denote the features of each voxel as a vector , where , and are the intensity vectors of size cubic patches that voxel locates in the center of which in the corresponding modality, where n is the number of
Experimental results
In this section, we first introduce the setting of experiments, and then train and validate the proposed model on the training data sets. Finally, we evaluate our method and compare the experimental results with those of other algorithms on the testing data sets. The algorithm is run by Matlab R2013a on a workstation with an Intel i7 CPU (1.8G) and 64G RAM with Ubuntu 12.04.
Discussion
In this section, we will discuss and analyze the influences of the parameters (such as λ, c, K and patch size ) on the proposed model.
Conclusion
This paper formulates multimodal MR brain tumor segmentation as a voxel-wise labeling problem by estimating a probability maximization model. This probability model uses a sparse representation-based framework to obtain the likelihood estimation. The MAP is then estimated using MRF. Considering MAP as an NP-hard problem, the model is converted into a minimum energy optimization problem, and the graph cuts algorithm is employed to obtain an optimal partition.
Although brain tumor segmentation is
Conflicts of interest
None.
Acknowledgments
This work is supported by Key Joint Program of National Natural Science Foundation and Guangdong Province of China U1201257 and U1401254, a grant from the Shenzhen-Hong Kong Innovation Circle Funding Program (Nos. SGLH20131010151755080 and GHP/002/13SZ) and a grant from the Guangdong Natural Science Foudation (Nos. 2016A030313047 and 2016A030313180).
References (47)
- et al.
A novel content-based active model for brain tumor segmentation
Magn Reson Imaging
(2012) - et al.
Semi-automated brain tumor and edema segmentation using MRI
Eur J Radiol
(2005) - et al.
A Bayesian model of shape and appearance for subcortical brain segmentation
NeuroImaging
(2011) - et al.
Kernel feature selection to fuse multi-spectral MRI images for brain tumor segmentation
Comput Vis Image Underst
(2011) - et al.
Computer-assisted segmentation of white matter lesions in 3D MR images, using Support Vector Machine
Acad Radiol
(2008) - et al.
Pose-robust face recognition via sparse representation
Pattern Recognit
(2013) - et al.
Segmentation of MR images via discriminative dictionary learning and sparse coding: application to hippocampus labeling
NeuroImaging
(2013) - et al.
Brain tumor segmentation from multispectral MRIs using sparse representation classification and Markov Random Field regularization
IEEE Trans Med Imaging
(2004) - et al.
A survey of MRI based medical image analysis for brain tumor studies
Phys Med Biol
(2013) - et al.
Fluid vector flow and application in brain tumor segmentation
IEEE Trans Biomed Eng
(2009)
Tumor-cut: segmentation of brain tumors on contrast enhanced MR images for radiosurgery applications
IEEE Trans Med Imaging
Multimodal brain tumor segmentation using the “tumor-cut” method on the BraTS dataset
Deformable registration of glioma images using EM algorithm and diffusion reaction modeling
IEEE Trans Med Imaging
A generative model for image segmentation based on label fusion
IEEE Trans Med Imaging
A generative model for image segmentation based in multimodal images
IEEE Trans Med Imaging
Segmentation brain tumor using alignment-based features
3D variational brain tumor segmentation using a high dimensional feature set
Efficient multilevel brain tumor segmentation with integrated Bayesian model classification
IEEE Trans Med Imaging
Adaptive segmentation of MRI data
IEEE Trans Med Imaging
Fully automatic segmentation of brain tumor images using support vector machine classification in combination with hierarchical conditional random field regularization
Ants and Árboles
A hybrid model for multimodal brain tumor segmentation
Multi-class abnormal brain tissue segmentation using texture features
Cited by (105)
A comprehensive survey of sparse regularization: Fundamental, state-of-the-art methodologies and applications on fault diagnosis
2023, Expert Systems with ApplicationsGradient-assisted deep model for brain tumor segmentation by multi-modality MRI volumes
2023, Biomedical Signal Processing and ControlSystematic study and design of multimodal MRI image augmentation for brain tumor detection with loss aware exchange and residual networks
2024, International Journal of Imaging Systems and TechnologyDesign and assessment of improved Convolutional Neural Network based brain tumor segmentation and classification system
2024, Journal of Integrated Science and TechnologyAutomated Multimodal Brain Tumor Segmentation and Localization in MRI Images Using Hybrid Res2-UNeXt
2024, Journal of Electrical Engineering and Technology