Integrating segmentation information into CNN for breast cancer diagnosis of mammographic masses

https://doi.org/10.1016/j.cmpb.2020.105913Get rights and content

Highlights

  • Modification of every convolutional layer in a CNN is addressed which enables the network to not only encounter the original input image but also the corresponding mass segmentation map.

  • A new loss function is introduced which adds an extra spatially-aware loss function term to the standard cross-entropy, aiming to steer the attention of the network to the mass region during training.

  • The proposed method is evaluated using both ground-truth and automatically produced segmentation maps.

Abstract

Background and ObjectivesSegmentation of mammographic lesions has been proven to be a valuable source of information, as it can assist in both extracting shape-related features and providing accurate localization of the lesion. In this work, a methodology is proposed for integrating mammographic mass segmentation information into a convolutional neural network (CNN), aiming to improve the diagnosis of breast cancer in mammograms. MethodsThe proposed methodology involves modification of each convolutional layer of a CNN, so that information of not only the input image but also the corresponding segmentation map is considered. Furthermore, a new loss function is introduced, which adds an extra term to the standard cross-entropy, aiming to steer the attention of the network to the mass region, penalizing strong feature activations based on their location. The segmentation maps are acquired either from the provided ground-truth or from an automatic segmentation stage. ResultsPerformance evaluation in diagnosis is conducted on two mammographic mass datasets, namely DDSM-400 and CBIS-DDSM, with differences in quality of the corresponding ground-truth segmentation maps. The proposed method achieves diagnosis performance of 0.898 and 0.862 in terms AUC when using ground-truth segmentation maps and a maximum of 0.880 and 0.860 when a U-Net-based automatic segmentation stage is employed, for DDSM-400 and CBIS-DDSM, respectively. ConclusionsThe experimental results demonstrate that integrating segmentation information into a CNN leads to improved performance in breast cancer diagnosis of mammographic masses.

Introduction

According to breast cancer statistics released from American Cancer Society, in 2019, around 268,600 new breast cancer cases is projected to occur in the United States resulting in 41,760 deaths [1]. These facts make breast cancer the most frequently diagnosed and the third most lethal type of cancer among women [2]. However, death rates have been decreasing since 1989, which is attributed to advancements in treatment and earlier detection through screening. Mammography is still the best choice for general population screening. Assessment of mammographic findings is highly dependent on the expertise of the radiologist, leading to a high number of false readings and additional interventions [3].

The role of computer-aided diagnosis (CAD) systems is already established in providing assistance in the decision-making process of radiologists. There are multiple interconnected stages in the pipeline of a typical CAD system, usually starting from a detection stage (CADe), aiming at the localization of suspicious lesions in a mammogram. Subsequently, the pipeline may include a segmentation stage, where the specific boundaries of a lesion are outlined, to differentiate it from background tissue. Finally, distinguishing malignant and benign lesion status, i.e. diagnosis, is implemented by a stage (CADx) based on classifier training to calculate either a probability or a binary decision regarding the pathology of an unknown lesion.

Prior to the emergence of deep learning, the pathology classification stage was based on hand-crafted features. Researchers designed features to capture both textural and morphological characteristics and relied on accurate segmentation [4], [5], implemented as either an automatic or a manual stage. This is attributed to the fact that the likelihood of malignancy has been found to be highly correlated to the morphology of a lesion [6]. Thus, the incorporation of segmentation can significantly facilitate the extraction of representative feature towards this end. In our previous work [7], a diagnosis pipeline was proposed, utilizing a set of texture and shape features of mammographic masses, computed from segmentation maps, acquired by expert radiologists using a semi-automatic segmentation method [8]. During experimentation, it was shown that shape and texture features had a complementary role.

Deep learning has been increasingly applied in mammographic imaging [9], with convolutional neural networks (CNN) being the most popular type of architecture. Typically, in mammographic mass diagnosis, CNNs process directly the Region Of Interest (ROI), performing consecutive convolution and pooling operations and ultimately calculate a probability for malignancy. Segmentation of mammographic masses may have an assisting role in a CNN pipeline for automatic diagnosis, as it can be a source of shape-related characteristics and a means to provide accurate localization.

The contribution of this paper is a novel methodology for integrating segmentation information into a CNN, aiming to improve breast cancer diagnosis. Unlike previous approaches, that use the segmentation maps as input in a network, in the proposed approach, the segmentation maps are made available to the convolutional layers of the CNN, that are modified, resulting in a new architecture, that improves performance in diagnosis. Furthermore, during the training phase, for each layer a new spatially-aware loss term is introduced, aiming to penalize activations based on the mass location.

The remainder of the paper is organized as follows: Section 2 presents an analysis of the current state-of-the art. Section 3 presents a detailed description of the proposed methodology. The experimental results and discussion are presented in Sections 4 and 5, respectively and conclusions are drawn in Section 6.

Section snippets

Related work

Carneiro et al. [10] used a pretrained CNN, after fine-tuning it on segmented lesions. For multiple views, they extracted CNN features both from the segmentation map and the ROI, fusing the two paths in a subsequent stage, to produce a diagnosis for the case. Arevalo et al. [11] used a CNN for extracting features from mammographic masses and a linear SVM for classification. They compared the performance achieved with hand-crafted texture and shape descriptors with the corresponding performance

Proposed methodology

The proposed scheme is based on a typical classification pipeline, where a CNN classifier directly processes the ROI and produces a diagnosis. In addition, the network is modified in a particular way, to adopt the segmentation maps as an additional input. The employed network, as well as the modifications that have been applied, are described in Sections 3.2 and 3.3, respectively. During the training phase, a spatially-aware loss term is introduced (Section 3.3.3) that, given the segmentation

Datasets

For the performance evaluation of the proposed methodology, two datasets have been used: DDSM-400 This dataset contains 400 mammograms (196 benign and 204 malignant), with masses selected from the Digital Database for Screening Mammography (DDSM) [28] and was developed in our previous work [7]. The dataset was further processed by expert radiologists, to acquire accurate contour delineations, utilizing a semi-automatic segmentation method, described in [8]. The evaluation using this dataset is

Discussion

A preliminary step of our experimental work aims to assess the effect on diagnosis performance of different ROI selection strategies, as presented in Section 4.2.1, since they affect the amount of contextual information included in the ROI. Contextual information refers to data outside the specific region of the object that is being classified. In the case of mammograms, it refers to the adjacent normal tissue that does not belong to the lesion. It can be argued that there is a trade-off in

Conclusions

In this paper, a novel method for the integration of segmentation information into a CNN is proposed. Improved diagnosis performance of a CNN classifier was achieved, applying the method using both ground-truth and automatically produced segmentation maps. This demonstrates that the incorporation of a segmentation stage into a deep learning pipeline can lead to increased diagnosis performance, compared to the typical approach of a standalone CNN classifier.

As segmentation quality is shown to

Declaration of Competing Interest

The authors declare no conflict of interest.

Acknowledgements

We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan Xp GPU used for this research.

References (35)

  • T. Fawcett

    An introduction to ROC analysis

    Pattern Recognit Lett

    (2006)
  • R.L. Siegel et al.

    Cancer statistics, 2019

    CA Cancer J Clin

    (2019)
  • A.C. Society

    Breast cancer facts & figures

    (2015)
  • R.A. Hubbard et al.

    Cumulative probability of false-positive recall or biopsy recommendation after 10 years of screening mammography: a cohort study

    Ann. Intern. Med.

    (2011)
  • M.L. Giger et al.

    Breast image analysis for risk assessment, detection, diagnosis, and treatment of cancer

    Annu Rev Biomed Eng

    (2013)
  • W.L. Bi et al.

    Artificial intelligence in cancer imaging: clinical challenges and applications

    CA Cancer J Clin

    (2019)
  • N. Arikidis et al.

    Shortest paths of mass contour estimates in mammography

    Proceedings of the International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), Workshop on Breast Image Analysis (BIA)

    (2015)
  • Cited by (63)

    • Breast Cancer Identification using Hybrid Algorithm

      2024, International Journal of Intelligent Systems and Applications in Engineering
    View all citing articles on Scopus
    View full text