1 Introduction

High-grade glioblastoma (HGG) or glioblastoma represents tumors arising from the gluey or supportive tissue of the brain. HGG is considered the most aggressive type of brain tumor. According to the American Brain Tumor Association (ABTA) [1] HGGs represent 74.6% of all malignant tumors and 24.7% of all primary brain tumors. World Health Organization (WHO) categorize HGGs as stage IV brain cancer [2]. Typically, the survival duration of patients with HGG tumor is less than two years [3, 4]. Therefore, accurate and timely detection of HGG tumor is essential for devising an appropriate treatment plan that may improve patient survival duration.

Recent works [5,6,7,8,9,10] have focused on developing automated survival prediction techniques for patients with HGG tumor. Different studies analyze tumor heterogeneity using different types of imaging [11] such as Magnetic Resonance Imaging (MRI) [12, 13]. This suggests MR as a potential non-invasive imaging biomarker for Glioblastoma diagnostic, prognostic and survival prediction. Jain et al. [14] extract morphological imaging features represented by Visually Accessible Rembrandt Images (VASRAI) [15] from the non-enhancing region of GBM and then correlate these features to the relative cerebral blood volume of a non-enhancing region and non-enhancing region crossing the midline. Gutman et al. [16] utilize four VASARI imaging features that describe the size of the contrast-enhanced, necrosis, non-enhance, edema, and the size of the whole tumor. Then these imaging features are associated with genetic mutation, survival prediction and Verhaak subtypes [17]. Nicolasjilwan et al. [18] combine clinical factors, VASARI imaging features, and genomics in a stepwise multivariate Cox model to predict overall survival time. Prasanna et al. [19] extract radiomic texture features that characterize three tumor regions; enhancing tumor, peritumoral brain zone, and necrosis from MR images. These features are assessed to overall survival prediction. Itakura et al. [20] utilize quantitative MRI features that describe tumor histogram statistics, texture, edge sharpness, compactness, and roughness. The authors cluster the features into three MRI phenotypic imaging subtypes; pre-multifocal, spherical, and rim-enhancing tumor. The three distinct subtypes are then correlated with overall survival and associated with molecular pathways.

Our proposed deep learning and machine learning based survival prediction technique have shown the best performance in the BraTS 2017 survival prediction challenge on the validation and test dataset, respectively [9]. This work proposes a sophisticated computational modeling-based survival prediction method. Specifically, we approach the survival prediction task as a two-step process, brain tumor segmentation followed by survival prediction, considering the segmentation output as an input to the second step. The brain tumor segmentation task is performed by utilizing two state-of-the-art convolutional neural networks (CNN) models, U-Net and fully convolutional neural network (FCN). The outcome of these two models is fused together to achieve the final segmentation output. This segmentation output along with the original MRI volumes are considered as input to the survival prediction step. Several radiomics features such as texture, topological, histogram etc. are extracted from the raw MRI sequences and the segmented tumor volume. Furthermore, a state-of-the-art 3D CNN architecture is utilized to extract additional features useful for the survival prediction task. These features are then processed using a gradient boosting-based regression technique known as extended gradient boosting (XGBoost) to obtain the survival estimation of patients.

2 Dataset

In this study, we use MR images of 163 high-grade GBM patients from BtaTS18 training dataset [21,22,23,24,25]. The dataset provides the ground truth segmentation of tumor tissues which comprises of enhancing tumor (ET), edema (ED), and the necrosis and non-enhancing tumor (NCR/NET). The training dataset provides age and overall survival (in days) data. The available scans of the MRI are native (T1), post-contrast T1-weighted (T1Gd), T2-weighted (T2), and T2 Fluid Attenuated Inversion Recovery (FLAIR) volumes. The dataset is co-registered, re-sampled to 1 mm3 and skull-stripped. In addition, for overall survival validation proposes we use the 28 cases of BraTS18 validation dataset.

3 Methodology

3.1 Brain Tumor Segmentation

This work utilizes two state-of-the-art CNN architectures, U-Net and fully convolutional neural network (FCN) to perform the brain tumor segmentation task. The following two sub-sections provide a very brief outline of these two models.

Brain Tumor Segmentation Using Deep CNNs.

This work utilizes a CNN based U-Net model [26, 27] to perform the brain tumor segmentation task. Unlike patch based CNN segmentation pipeline where the model only sees a localized region of the brain, the U-Net based segmentation model captures the global information from a different region of the brain tissues which is essential to achieve robust segmentation performance. Moreover, U-Net based model allows achieving an end-to-end segmentation framework rather than a pixel-wise classification technique. The U-Net architecture utilized in this work is implemented following [26]. Rather than using regular cross-entropy based loss function, we utilize a soft dice similarity coefficient (DSC) metric based loss function to train the U-Net model [28]. The U-Net model is trained using mini-batch gradient descent (GD) technique which minimizes the soft dice cost function. This work also uses a fully convolutional network (FCNs) [29, 30] for tumor segmentation. We adapt VGG-11 [31] as a pre-trained model. The overall FCN architecture contains an encode and a decode stage. The encoding stage contains convolution and max-pooling steps whereas the decoding stage contains a deconvolution step to obtain the same output size as the input. The final segmentation output is achieved by fusing the outcomes obtained from the above mentioned CNN based tumor segmentation pipelines as shown in Fig. 1.

Fig. 1.
figure 1

Brain tumor segmentation pipeline using U-Net and FCN

Full size image

3.2 Survival Prediction

The proposed survival prediction pipeline essentially involves three stages, (1) feature extraction stage, (2) feature selection stage, and (3) feature classification/regression stage.

Feature Extraction.

Overall, approximately 31 thousand features representing texture, volume, area, and Euler characteristics are extracted from the tumor and the sub-regions (edema, enhancing tumor, and tumor core). Each feature type is selected to emphasize different characteristics of the tumor that may be relevant for survival prediction. For instance, texture features define the heterogeneity of the different tumor tissues, and Volumetric and Euler characteristic features define tumor shape.

We extract forty-one representative features [32] from three raw MRI sequences, and from eight texture representations of the tumor volume that includes several Texton filters [33], and fractal characterizations using algorithms such as PTPSA [34], mBm [35], and Holder exponent [36]. The features obtained from these representations include histograms, co-occurrence matrix, grey-tone difference matrix, and several other statistical measures. Furthermore, histogram-based features are extracted from the different modalities of different histogram graphs of the tumor tissue regions (edema, enhancing tumor, and necrosis). We also extract many representative volumetric features from the different tumor tissues with respect to the brain and whole tumor regions. In addition, we compute the Euler characteristics of the whole tumor, edema, enhancing and necrosis, for each slice as feature vectors. Euler characteristic [37] identifies the shape of a tumor by computing tumor vertices, edges, and faces. Combining feature vectors from all the above-mentioned methods constitute a total 31,000 feature points for the survival prediction task.

Feature Selection.

We perform recursive feature selection on the Euler features alone, another recursive feature selection on the other features (texture, volumetric, histogram-graph based).

Feature Classification and Regression.

Extreme Gradient Boosting (XGBoost) [38] is a tree boosting supervised machine learning technique that is highly effective. XGBoost is widely used in classification and regression tasks. In our study, XGBoost is utilized for classification and regression overall survival prediction on the selected features. The trained models are tuned to their optimized hyper-parameters when a tuned grid (search grid) is created by the different combination of the hyper-parameters. The complete pipeline for classification and regression overall survival is illustrated in Fig. 2.

Fig. 2.
figure 2

Survival prediction pipeline

Full size image

4 Experimental Results

Following the proposed pipeline in Fig. 1, we first perform the FCN and U-Net fused tumor segmentation task. Figure 3 shows an example from the BraTS 2018 training dataset of the fused segmentation outcome of FCN and U-Net. We perform leave-one-out cross-validation analysis on the BraTS 2018 training dataset using the proposed survival prediction pipeline. We evaluate the performance of our proposed method using the root mean square error (RMSE), and the classification accuracy for a three-class setting defined as follows: (1) long – more than 15 months, (2) Medium – between 10 to 15 months, (3) short – less than 10 months. The best model is picked from the analysis of training data to be used to process the validation dataset of the BraTS 2018 competition. The ground truth is considered for the segmented tumor in the survival analysis of the training dataset, while the segmented tumor obtained in stage one of the proposed pipeline is used as input in the evaluation of the validation dataset. The leave-one-out cross-validation RMSE of the training dataset is 391.25, and the three-class leave-one-out cross-validated survival classification accuracy is 0.73. Table 1 shows the online evaluation results we achieve with the BraTS18 validation dataset and the test dataset.

Fig. 3.
figure 3

Example input from training dataset and segmentation outcomes: (a) T1Gd sequence, (b) FLAIR sequence, (c) the segmentation outcome of FCN, (d) the segmentation outcome of U-Net, (e) ground truth (f) fused segmentation.

Full size image
Table 1. BraTS18 validation dataset and test dataset evaluation
Full size table

5 Conclusions

This work proposes a robust automated glioblastoma survival prediction using state-of-the-art computational modeling techniques. The survival prediction task is performed in two steps: tumor segmentation and survival prediction. A combination of hand-crafted and learned features are used in a regression technique to obtain the final survival prediction output. The performance of the proposed pipeline is evaluated using BraTS 2018 challenge training and validation datasets. Our results show a leave-one-out cross-validated classification accuracy of 0.679 for the validation dataset and that of 0.519 for the test dataset.