Brain tumor segmentation from multimodal magnetic resonance images via sparse representation

https://doi.org/10.1016/j.artmed.2016.08.004Get rights and content

Highlights

  • An automated brain tumor segmentation model based on maximum a posteriori probabilistic (MAP) estimation is presented.

  • The likelihood probability of the model is estimated by sparse coding and dictionary learning.

  • The Markov random field (MRF) is introduced into the prior probability.

  • The MAP is converted into a minimum energy optimization problem and graph cuts is used to find its solution.

Abstract

Objective

Accurately segmenting and quantifying brain gliomas from magnetic resonance (MR) images remains a challenging task because of the large spatial and structural variability among brain tumors. To develop a fully automatic and accurate brain tumor segmentation algorithm, we present a probabilistic model of multimodal MR brain tumor segmentation. This model combines sparse representation and the Markov random field (MRF) to solve the spatial and structural variability problem.

Methods

We formulate the tumor segmentation problem as a multi-classification task by labeling each voxel as the maximum posterior probability. We estimate the maximum a posteriori (MAP) probability by introducing the sparse representation into a likelihood probability and a MRF into the prior probability. Considering the MAP as an NP-hard problem, we convert the maximum posterior probability estimation into a minimum energy optimization problem and employ graph cuts to find the solution to the MAP estimation.

Results

Our method is evaluated using the Brain Tumor Segmentation Challenge 2013 database (BRATS 2013) and obtained Dice coefficient metric values of 0.85, 0.75, and 0.69 on the high-grade Challenge data set, 0.73, 0.56, and 0.54 on the high-grade Challenge LeaderBoard data set, and 0.84, 0.54, and 0.57 on the low-grade Challenge data set for the complete, core, and enhancing regions.

Conclusions

The experimental results show that the proposed algorithm is valid and ranks 2nd compared with the state-of-the-art tumor segmentation algorithms in the MICCAI BRATS 2013 challenge.

Introduction

A brain tumor is an abnormal tissue proliferation that will cause an increase in intracranial pressure, resulting in central nervous system damage, hence endangering the lives of patients. The reliable detection and segmentation of brain tumors from magnetic resonance (MR) images can aid surgical planning and therapy assessments in medical diagnosis. Currently, most brain tumors are segmented manually by medical experts, although this method is time consuming. Moreover, the segmentation technology shows inter- and intra-rater variability, and the results are difficult to reproduce [1]. Therefore, computer-aided semi-automatic and automatic tumor segmentation is a promising method and plays an increasing role in modern medical analysis. Although a large improvement has been acquired in medical imaging technologies and a great deal of effort has been paid in tumor segmentation methods [2], developing an automatic, accurate and repeatable tumor segmentation algorithm remains a challenging task due to the unpredictable appearance, location, shape, and size of the tumors and overlap between the intensity range of healthy tissues and that of tumors. Moreover, because magnetic resonance imaging (MRI) is affected by noise, bias-field effect, partial volume effect, and tissular movement, MR images are usually blurred and non-uniform in intensity, which further increase the difficulty in tumor segmentation.

In the past few decades, various brain tumor segmentation algorithms have been presented that can be generally divided into two categories: semi-automatic and automatic. Semi-automatic segmentation methods mainly include active contour models [3], level-set models [4], [5], and the newest developed tumor-cut (TC) model [6], [7]. Compared with automatic segmentation approaches, semi-automatic segmentation approaches can achieve more accurate segmentation results, whereas this type of method needs manual initialization and user interaction. Automatic segmentation approaches mainly include brain atlases and registration based methods, probability based methods, and supervised classification based methods. These types of methods can be performed independently by computer without the participation of humans, but the segmentation result is unsatisfactory.

The brain atlas-guided segmentation methods [8], [9], [10], [11], [12] can provide important spatial information prior to using an aligned template to extract the spatial information of each type of healthy brain tissues and then segment the tumor and outlier by measuring the deviation from the normal brain. This type of method is actually a fusion approach based on registration, and its accuracy of segmentation relies heavily on the accuracy of registration. Due to the intensity variations around the tumor and deformation of healthy tissue morphology caused by edema and infiltration, it is still difficult to achieve accurate segmentation results.

The Bayesian approach [13] uses multivariate normal distribution to estimate the mean and covariance from a training data set. The class labels are obtained by maximum a posteriori (MAP) probability. In [14], Corso et al. incorporated the Bayesian model into a graph-based affinity calculation to segment glioblastoma multiforme (GBM) brain tumors and edema into multichannel MR volumes. In [15], the expectation maximization (EM) algorithm was used to perform a Gaussian mixture model (GMM) to distribute every pixel/voxel of T1-weighted (T1) MR images to the tissue class with the highest probability. Due to using only T1-weighted MR images, the segmentation accuracy is not satisfactory. Moreover, when only the single T1-weighted modality is used, it is intractable to separate the active tumor from the edema.

The supervised classification approach based on machine learning is an often-used brain tumor segmentation method. The technology is used to assign each pixel/voxel to a certain category according to the corresponding label statistics. The support vector machine (SVM) is the most popular supervised classification approach that maps the trained samples vector into a high-dimensional feature space and seeks an optimal separating hyperplane to classify the lesion tissues and healthy tissues [16], [17], [18]. For a binary classification task of small samples, the SVM classifier performs very well with a fast running speed and high classification accuracy. However, for the multi-classification problem, SVM is not an effective approach. Especially for high dimensional data, SVM is intractable. The random forest (RF) model is a multi-classification learning algorithm [19], [20], [21], [22] that is an ensemble of many binary decision trees. This method generates many classifiers and aggregates their results to obtain better decisions. However, for the classification problem, feature representation, in most cases, is more important than the choice of the classifier.

For MR brain tumor segmentation, the commonly used features are intensity-based features that show inter- and intra-rater variability [1] and are unstable; thus, this method is often sensitive to noise.

Sparse representation is a recently developed machine learning method. This approach can approximate images as a sparse linear combination of base atoms from a designed dictionary by adapting the dictionary to a set of training samples. Compared with traditional subspace learning models, sparse representation is more robust for data containing many outliers and sparse noise. Currently, sparse representation has been successfully applied in various vision tasks such as MR image reconstruction [23], face recognition [24], [25], image denoising [26], image super-resolution [27], and image classification [28].

The sparse representation-based classification (SRC) algorithm [24] is one of the most popular classification methods. However, this method was originally designed for face recognition purposes and did not consider the spatial constraint information. For the segmentation task, it is easy to lead to fragmentary segmentation results and decrease the segmentation accuracy because the SRC-based models did not consider the spatial dependencies between neighboring voxels. Currently, its various improved versions have been presented and have been successfully applied in medical image classification and segmentation. In [29], the extended SRC is used to segment MR brain images. In [30], Wang et al. proposed a patch-based sparse representation to segment brain MR images, adding spatial regularization via level sets into the model. In [31], the spatial constraints are generated by introducing Markov random field (MRF) regularization to an SRC model to segment MR brain tumor. In this SRC-based model, however, Zhan et al. directly used the original training samples as the base atoms of the dictionary. To make the dictionary discriminative, it is necessary to provide sufficient training samples from each class to reconstruct the dictionary, leading to an increased dictionary size and decreased speed. For big sample training data, it is intractable.

Moreover, for [30], speed becomes a large problem in the application of the semi-automatic model because real-time conditions are required for an interactive segmentation scheme. In [32], Li et al. not only considered the regularization constraints but also learned the adaptive dictionaries to segment MR brain tumors. Accordingly, these learned dictionaries gained the discriminative capability for classification, and the size of the dictionary also shrank simultaneously. However, this method segments MR brain tumors on 2D slices, the learned dictionaries will miss much of the contextual information, and there will be no smoothness across slices; thus, the dictionaries become non-rotation-invariant.

Inspired by the SRC model [24] and MRF with the graph cuts model [33], we present a sparse representation-based automatic brain tumor segmentation algorithm. Fig. 1 presents an overview of the proposed framework. This method formulates tumor segmentation as a MAP problem that casts the sparse reconstruction residual as a likelihood and incorporates the prior spatial smoothness into MRF and then solves the optimization problem of minimum energy using the graph cuts algorithm [34].

In contrast to the current SRC model [24] and its improved version [30], [31], we not only learn the adaptive dictionaries but also consider the spatial information between adjacent voxels and add a regularization smoothness term into the model; thus, the segmentation accuracy can be increased. Different from [31], we use the online dictionary learning (OLD) algorithm [35] to train our dictionary that is suitable for the big training sample data. In [31], the sparse coefficients were obtained by minimizing a non-negative Elastic-Net problem. Especially, in the work of Zhan et al., the authors only segmented the multimodal brain tumor into the edema and the tumor and the model was evaluated based on the single Jaccard score. In this paper, however, sparse coefficients were obtained by solving a ℓ1 minimization problem using the LARS-Lasso algorithm [36]. We segment the tumor into four different tumor classes: necrosis, edema, non-enhancing tumor and enhancing tumor. Moreover, we discuss in detail for different parameters and train the model with cross-validation. The segmentation results are evaluated by several metrics – the Dice, positive predictive value, sensitivity and Cohen's kappa. Compared with [30], our model is a fully automatic segmentation algorithm that does not require real-time performance, so speed is no longer a key problem for the automatic brain tumor segmentation model. Particularly, the sparse representation calculated from multiple modalities should be more powerful than previous representations from a single modality in capturing the complex structure of the brain tumor because the T1 modality edema and non enhancing tumor areas have a very close range of intensity values to gray matter, whereas the corresponding Flair modality shows the edema well by a hyper-intense signal around the tumor [8], not only making the learned dictionaries contain richer features, but also further enhancing the discriminant of the learned dictionary between the edema and active core tumor. This property was not involved in [30].

Because little valid prior knowledge can be used in brain tumor segmentation, we proposed a data-driven patch-based segmentation scheme, which learns a feature base space via dictionary learning for training samples and then projects the intensity information of the test images to dictionary space via a sparse coding strategy and acquires the sparse vectors.

This paper is an extended version of our work in the conference paper [32]. Compared to [32], we extend our model from 2D to 3D by extracting the cubic patch samples and extend the pixel-wise labeling problem of three classes to a voxel-wise labeling problem with five classes: necrosis, edema, non-enhancing tumor, enhancing tumor, and health brain tissues. Furthermore, we discuss the parameters within the model in detail and extend our experiment validation not only on real high-grade data, but also on real low-grade data. Although we did not do any post-processing, an obvious improvement on the segmentation results has been obtained.

The remainder of this paper is organized as follows. In Section 2, a probability model is formulated and the detailed algorithm implementation is described. Section 3 gives the experimental results and analysis. Section 4 discusses the parameters of the proposed model and Section 5 concludes this paper.

Section snippets

Method

Given a set of registered multimodal MR brain volumes of the same patient comprising four modalities with the same size – Flair, T1, contrast enhanced T1-weighted (T1C), and T2-weighted (T2) – we can denote the features of each voxel v as a vector x(v)=(xFlairT(v),xT1T(v),xT1CT(v),xT2T(v))T4n, where xFlair(v),xT1(v),xT1C(v), and xT2(v) are the intensity vectors of size n3×n3×n3 cubic patches that voxel v locates in the center of which in the corresponding modality, where n is the number of

Experimental results

In this section, we first introduce the setting of experiments, and then train and validate the proposed model on the training data sets. Finally, we evaluate our method and compare the experimental results with those of other algorithms on the testing data sets. The algorithm is run by Matlab R2013a on a workstation with an Intel i7 CPU (1.8G) and 64G RAM with Ubuntu 12.04.

Discussion

In this section, we will discuss and analyze the influences of the parameters (such as λ, c, K and patch size n3×n3×n3) on the proposed model.

Conclusion

This paper formulates multimodal MR brain tumor segmentation as a voxel-wise labeling problem by estimating a probability maximization model. This probability model uses a sparse representation-based framework to obtain the likelihood estimation. The MAP is then estimated using MRF. Considering MAP as an NP-hard problem, the model is converted into a minimum energy optimization problem, and the graph cuts algorithm is employed to obtain an optimal partition.

Although brain tumor segmentation is

Conflicts of interest

None.

Acknowledgments

This work is supported by Key Joint Program of National Natural Science Foundation and Guangdong Province of China U1201257 and U1401254, a grant from the Shenzhen-Hong Kong Innovation Circle Funding Program (Nos. SGLH20131010151755080 and GHP/002/13SZ) and a grant from the Guangdong Natural Science Foudation (Nos. 2016A030313047 and 2016A030313180).

References (47)

  • A. Hamamci et al.

    Tumor-cut: segmentation of brain tumors on contrast enhanced MR images for radiosurgery applications

    IEEE Trans Med Imaging

    (2012)
  • A. Hamamci et al.

    Multimodal brain tumor segmentation using the “tumor-cut” method on the BraTS dataset

  • A. Gooya et al.

    Deformable registration of glioma images using EM algorithm and diffusion reaction modeling

    IEEE Trans Med Imaging

    (2011)
  • M.R. Sabuncu et al.

    A generative model for image segmentation based on label fusion

    IEEE Trans Med Imaging

    (2010)
  • B. Menze et al.

    A generative model for image segmentation based in multimodal images

    IEEE Trans Med Imaging

    (2010)
  • M. Schmidt et al.

    Segmentation brain tumor using alignment-based features

  • D. Cobzas et al.

    3D variational brain tumor segmentation using a high dimensional feature set

  • J.J. Corso et al.

    Efficient multilevel brain tumor segmentation with integrated Bayesian model classification

    IEEE Trans Med Imaging

    (2008)
  • W.M. Wells et al.

    Adaptive segmentation of MRI data

    IEEE Trans Med Imaging

    (1996)
  • S. Bauer et al.

    Fully automatic segmentation of brain tumor images using support vector machine classification in combination with hierarchical conditional random field regularization

  • N. Tustison et al.

    Ants and Árboles

  • R. Meier et al.

    A hybrid model for multimodal brain tumor segmentation

  • S. Reza et al.

    Multi-class abnormal brain tissue segmentation using texture features

  • Cited by (105)

    View all citing articles on Scopus
    View full text