Multi-level features combined end-to-end learning for automated pathological grading of breast cancer on digital mammograms

https://doi.org/10.1016/j.compmedimag.2018.10.008Get rights and content

Highlights

  • Noninvasive and convenient mammograms are used for discriminating the pathological grades instead of pathological images.

  • Hand-crafted features focusing on low-level details are extracted and selected by supervised LASSO logistic regression.

  • Convolutional Neural Network (CNN) is designed to focus on extracting high-level semantic features.

  • The new feature-combined network makes different parts of the network learn to focus on different levels of features.

  • Results demonstrate that our proposed algorithm obtains a comparable performance compared with pathological images.

Abstract

We propose to discriminate the pathological grades directly on digital mammograms instead of pathological images. An end-to-end learning algorithm based on the combined multi-level features is proposed. Low-level features are extracted and selected by supervised LASSO logistic regression. Convolutional Neural Network (CNN) is designed to extract high-level semantic features. These extracted multi-level features are combined to optimize the new CNN end to end to make different parts of the network learn to pay attention to different level of features. Results demonstrate that our proposed algorithm is superior to other CNN models and obtain comparable performance compared with pathological images.

Introduction

Breast cancer is the most common cancer in women, and it has gradually become the second leading cause of death among women (McGuire et al., 2015). The number of breast cancer patients shows a continuous increase since 1970s, and more and more women die because of breast cancer. Pathological grading of breast cancer is an important tumor-related prognostic factor, that both recurrence-free interval and overall survival are worse in patients with poorly differentiated tumors than that of well-differentiated tumors. Pathological grades have an important influence in breast cancer diagnosis and prognosis analysis in clinical practice. And breast cancer grading system can provide momentous clinical information about the cancer, and discriminate breast cancer patient subpopulations with different recurrent risks.

Elston and Ellis modified Scarff-Bloom-Richardson (SBR) grading system (Elston and Ellis, 1991) is the most widely used and well-established criterion for pathological diagnosis based on pathological sections (Ruibal et al., 2001). The computation of SBR grade score is based on three morphologic features including degree of tubule formation, mitotic activity, and nuclear pleomorphism (Bansal et al., 2012), each of which is scored on a scale of 1–3 to produce a combined score ranged between 3 and 9. The final SBR score is divided into three grades (I-III): grade I, score is 3–5; grade II, score is 6–7; and grade III, score is 8-9. The higher the score, the higher the pathological grade, and the greater the risk of recurrence.

During histological analysis, pathologists perform grading by manually examining breast cancer tissue specimen under a microscope, which is a tedious and subjective process. Breast cancer grading identified by pathological images is considered as a standard clinical practice for the diagnosis and prognosis of breast cancer. Although many automatic breast cancer grading methods based on digitized pathological images had been presented to improve the diagnostic efficiency and confidence of pathologists (Veta et al., 2014; Petushi et al., 2006; Wan et al., 2014; Basavanhally et al., 2013), they were all dependent on images obtained by observing histological sections under the microscope. This ultimately needs a biopsy of the cancer tissue. Extracting cancerous tissue and staining with H&E involve a series of processing procedures and biopsy is invasive. Digital mammography screening is the most convenient, reliable and noninvasive examination, which benefits the early detection and diagnosis of breast cancer. Considering that different breast cancer grades might have different presentations on mammograms, we aim to find the mapping relationship between mammograms and breast cancer grades, and this can also help decrease tedious work in examining pathological sections.

Recently, many studies have been presented to predict the cancer stage. Mu et al. (2015) studied forty-two patients suffering from cervical cancer to identify the stage of cervical cancer. They computed three standardized uptake values (SUVmax, SUVmean, SUVpeak), metabolic tumor volume (MTV), and fifty-four 3D texture features based on PET images. The most discriminative features were selected through correlation analysis. Using the Support Vector Machine (SVM) classifier to classify the combined features, they get a higher accuracy and larger AUC (the area under the ROC curves). Liang et al. (2016) studied 494 consecutive patients with stage I-IV colorectal cancer (CRC) to establish a Radiomics signature. They extracted 150 texture features from a single 2-dimensional CT slice, including features from the gray-level co-occurrence matrix (GLCM) and histogram. Then the least absolute shrinkage and selection operator method (LASSO) logistic regression model was used to select the most valuable prognostic features, and Mann-Whitney U test was used to estimate the relationship between the Radiomics signature and CRC stages (I-II and III-IV). Genomic and Radiomic imaging profiles of invasive breast carcinomas were also integrated to predict the clinical phenotypes (Guo et al., 2015). A total of 38 Radiomic features and 144 genomic features were extracted from a collection of 91 cancer cases. All the features were standardized, and LASSO regression was also used to model the relationship between clinical outcomes and radiogenomic features.

As mentioned above, most recently proposed pathological grading methods are based on Radiomics. Radiomics (Lambin et al., 2012; Kumar et al., 2012) is a newly developing technology that can extract and analyze large amounts of advanced quantitative imaging features with high throughput from radiographic images. It captures intratumoral heterogeneity in a noninvasive way. Special and quantitative image features are computed and used to predict the structure of cancer. The most effective features are selected to guide final diagnosis and predict prognosis. On the other hand, pathological grade is very difficult to be recognized from the mammograms. Hand-crafted features are mainly bottom image details not containing mid-level and high-level semantic features and are insufficient. Hand-crafted features are limited by experience, and might omit some important and beneficial features for breast cancer grading. Hence, many useful information in the mammograms might be ignored. Deep learning especially Convolutional Neural Network (CNN), as an additional tool, has the ability of directly obtaining object category and training in an end to end manner though learning from the raw data. It could extract abstracted high-level semantic features from the network input. CNN has achieved remarkable performance in natural image classification (Simonyan and Zisserman, 2014; Szegedy et al., 2015) and other tasks, such as image segmentation (Wang et al., 2017). Many works have also been done in computer aided detection of mammographic lesions (Samala et al., 2016; Kooi et al., 2017) and breast mass classification (Dhungel et al., 2016; Jiao et al., 2016). Thus the combination of CNN extracted features and manually designed features could obtain a good performance in image classification. This can ultimately improve the grading accuracy.

In this paper, we presented an automatic method for breast cancer grading based on digital mammograms rather than common pathological images, which is gold standard for breast cancer grading. Representations of grade II and grade III in digital mammograms are quite similar. Thus, discrimination between grade II and grade III in mammograms is very difficult even if done by experienced professional radiologists. Therefore, we focus on the discrimination of grade II and grade III breast cancer. The proposed method combines multi-level selected features extracted from mammograms, which consist of bottom details based on Radiomics and high-level information based on CNN. We first extract the cancerous region according to the tumor contour delineated by professional radiologists. Then, a series of low-level features, such as gray-level, texture and wavelet features, are computed based on the segmented breast tumor region. LASSO logistic regression (Tibshirani, 1996) is applied to the extracted features to select the most relevant features for breast cancer grading. CNN is also designed to discriminate breast cancer pathological grades, and we combine selected hand-crafted features with pre-trained CNN extracted features in an end-to-end training algorithm to re-optimize the overall network. This method integrates multi-level image features, including the bottom visual features and high-level semantic features. We evaluate this method in collected clinical cases, and obtain comparable accuracy with pathological images (Wan et al., 2017).

The remainder of this paper is organized as follows. The second part contains details of our proposed algorithm. Then the results and discussion are presented to prove the performance of our proposed method. Finally, we create the conclusion based on the contents of the preceding sections.

Section snippets

Material and methods

Current breast cancer pathological grading mainly depends on pathological images, and it is hard for professional radiologists to discriminate the grades only through mammograms. Breast cancer pathological grading based on digital mammograms is a very difficult task, but exploiting the prediction based on potential characteristics in the mammograms can benefit diagnosis. Features extracted based on Radiomics (we named Radiomics features) is low-level features, which points to image bottom

Data

The mammograms data used in our research are digital molybdenum target mammograms collected from Department of Radiology in Henan Provincial People’s Hospital. All cases in our collected dataset have the final diagnosis from the Pathology Department of this hospital. The pathological grades are extracted from the pathological diagnosis. All tumor contours in the mammograms are labeled by more than two experienced and specialized radiologists. We collected 204 patient cases, and every case

Conclusions

In this paper, we proposed to grade breast cancer based on digital mammograms rather than pathological images, which is noninvasive and convenient. A feature-combined algorithm that integrates Radiomics and deep CNN is used. Multi-level features extracted by Radiomics and CNN could capture more potential information in the mammograms and improve the classification accuracy. Radiomics features are selected by supervised LASSO to find the most effective features. Our proposed algorithm do not

References (39)

  • J. Deng et al.

    Imagenet: a large-scale hierarchical image database

    IEEE Conference on Computer Vision and Pattern Recognition

    (2009)
  • N. Dhungel et al.

    The automated learning of deep features for breast mass classification from mammograms

    International Conference on Medical Image Computing and Computer-Assisted Intervention

    (2016)
  • C.W. Elston et al.

    Pathological prognostic factors in breast cancer. I. The value of histological grade in breast cancer: experience from a large study with long-term follow-up

    Histopathology

    (1991)
  • R.A. Fisher

    The use of multiple measurements in taxonomic problems

    Ann. Hum. Genet.

    (1936)
  • W. Guo et al.

    Prediction of clinical phenotypes in invasive breast carcinomas from the integration of radiomics and genomics data

    J. Med. Imaging

    (2015)
  • R.M. Haralick et al.

    Textural features for image classification

    IEEE Trans. Syst. Man Cybern.

    (1973)
  • K. He et al.

    Deep residual learning for image recognition.

    Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

    (2016)
  • G. Huang et al.

    Densely connected convolutional networks

    arXiv preprint arXiv:1608.06993

    (2016)
  • F.N. Iandola et al.

    SqueezeNet: AlexNet-level Accuracy With 50x Fewer Parameters and< 0.5 MB Model Size

    (2016)
  • Cited by (24)

    • A hybrid end-to-end learning approach for breast cancer diagnosis: convolutional recurrent network

      2023, Computers and Electrical Engineering
      Citation Excerpt :

      Recently, researchers have proposed several new methods for breast cancer detection [15]. Hai, Tan [9] performed cancer detection on breast mammography images using an end-to-end learning algorithm. That study, in contrast to ours, extracted textural features and used mammographic images to classify pathological grades.

    • Deep and machine learning techniques for medical imaging-based breast cancer: A comprehensive review

      2021, Expert Systems with Applications
      Citation Excerpt :

      Incoming mammographic diagnostic images of breast cancer may be classified into malignant, benign, and stable patients by CNNI-BCC. In Hai et al. (2019), the authors proposed differentiating the pathological degrees in digital mammograms explicitly. It proposed an end-to-end learning algorithm based on multilevel functions combined.

    • Special issue on machine learning in medical imaging

      2019, Computerized Medical Imaging and Graphics
    • Genetic algorithm for feature selection in mammograms for breast masses classification

      2024, Computer Methods in Biomechanics and Biomedical Engineering: Imaging and Visualization
    View all citing articles on Scopus
    View full text