Integrating segmentation information into CNN for breast cancer diagnosis of mammographic masses
Introduction
According to breast cancer statistics released from American Cancer Society, in 2019, around 268,600 new breast cancer cases is projected to occur in the United States resulting in 41,760 deaths [1]. These facts make breast cancer the most frequently diagnosed and the third most lethal type of cancer among women [2]. However, death rates have been decreasing since 1989, which is attributed to advancements in treatment and earlier detection through screening. Mammography is still the best choice for general population screening. Assessment of mammographic findings is highly dependent on the expertise of the radiologist, leading to a high number of false readings and additional interventions [3].
The role of computer-aided diagnosis (CAD) systems is already established in providing assistance in the decision-making process of radiologists. There are multiple interconnected stages in the pipeline of a typical CAD system, usually starting from a detection stage (CADe), aiming at the localization of suspicious lesions in a mammogram. Subsequently, the pipeline may include a segmentation stage, where the specific boundaries of a lesion are outlined, to differentiate it from background tissue. Finally, distinguishing malignant and benign lesion status, i.e. diagnosis, is implemented by a stage (CADx) based on classifier training to calculate either a probability or a binary decision regarding the pathology of an unknown lesion.
Prior to the emergence of deep learning, the pathology classification stage was based on hand-crafted features. Researchers designed features to capture both textural and morphological characteristics and relied on accurate segmentation [4], [5], implemented as either an automatic or a manual stage. This is attributed to the fact that the likelihood of malignancy has been found to be highly correlated to the morphology of a lesion [6]. Thus, the incorporation of segmentation can significantly facilitate the extraction of representative feature towards this end. In our previous work [7], a diagnosis pipeline was proposed, utilizing a set of texture and shape features of mammographic masses, computed from segmentation maps, acquired by expert radiologists using a semi-automatic segmentation method [8]. During experimentation, it was shown that shape and texture features had a complementary role.
Deep learning has been increasingly applied in mammographic imaging [9], with convolutional neural networks (CNN) being the most popular type of architecture. Typically, in mammographic mass diagnosis, CNNs process directly the Region Of Interest (ROI), performing consecutive convolution and pooling operations and ultimately calculate a probability for malignancy. Segmentation of mammographic masses may have an assisting role in a CNN pipeline for automatic diagnosis, as it can be a source of shape-related characteristics and a means to provide accurate localization.
The contribution of this paper is a novel methodology for integrating segmentation information into a CNN, aiming to improve breast cancer diagnosis. Unlike previous approaches, that use the segmentation maps as input in a network, in the proposed approach, the segmentation maps are made available to the convolutional layers of the CNN, that are modified, resulting in a new architecture, that improves performance in diagnosis. Furthermore, during the training phase, for each layer a new spatially-aware loss term is introduced, aiming to penalize activations based on the mass location.
The remainder of the paper is organized as follows: Section 2 presents an analysis of the current state-of-the art. Section 3 presents a detailed description of the proposed methodology. The experimental results and discussion are presented in Sections 4 and 5, respectively and conclusions are drawn in Section 6.
Section snippets
Related work
Carneiro et al. [10] used a pretrained CNN, after fine-tuning it on segmented lesions. For multiple views, they extracted CNN features both from the segmentation map and the ROI, fusing the two paths in a subsequent stage, to produce a diagnosis for the case. Arevalo et al. [11] used a CNN for extracting features from mammographic masses and a linear SVM for classification. They compared the performance achieved with hand-crafted texture and shape descriptors with the corresponding performance
Proposed methodology
The proposed scheme is based on a typical classification pipeline, where a CNN classifier directly processes the ROI and produces a diagnosis. In addition, the network is modified in a particular way, to adopt the segmentation maps as an additional input. The employed network, as well as the modifications that have been applied, are described in Sections 3.2 and 3.3, respectively. During the training phase, a spatially-aware loss term is introduced (Section 3.3.3) that, given the segmentation
Datasets
For the performance evaluation of the proposed methodology, two datasets have been used: DDSM-400 This dataset contains 400 mammograms (196 benign and 204 malignant), with masses selected from the Digital Database for Screening Mammography (DDSM) [28] and was developed in our previous work [7]. The dataset was further processed by expert radiologists, to acquire accurate contour delineations, utilizing a semi-automatic segmentation method, described in [8]. The evaluation using this dataset is
Discussion
A preliminary step of our experimental work aims to assess the effect on diagnosis performance of different ROI selection strategies, as presented in Section 4.2.1, since they affect the amount of contextual information included in the ROI. Contextual information refers to data outside the specific region of the object that is being classified. In the case of mammograms, it refers to the adjacent normal tissue that does not belong to the lesion. It can be argued that there is a trade-off in
Conclusions
In this paper, a novel method for the integration of segmentation information into a CNN is proposed. Improved diagnosis performance of a CNN classifier was achieved, applying the method using both ground-truth and automatically produced segmentation maps. This demonstrates that the incorporation of a segmentation stage into a deep learning pipeline can lead to increased diagnosis performance, compared to the typical approach of a standalone CNN classifier.
As segmentation quality is shown to
Declaration of Competing Interest
The authors declare no conflict of interest.
Acknowledgements
We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan Xp GPU used for this research.
References (35)
- et al.
Computer-aided detection/diagnosis of breast cancer in mammography and ultrasound: a review
Clin Imaging
(2013) - et al.
Computer-aided diagnosis of mammographic masses based on a supervised content-based image retrieval approach
Pattern Recognit
(2017) - et al.
Deep learning in mammography and breast histology, an overview and future trends
Med Image Anal
(2018) - et al.
Representation learning for mammography mass lesion classification with convolutional neural networks
Comput Methods Programs Biomed
(2016) - et al.
A deep feature based framework for breast masses classification
Neurocomputing
(2016) - et al.
Large scale deep learning for computer aided detection of mammographic lesions
Med Image Anal
(2017) - et al.
A deep learning approach for the analysis of masses in mammograms with minimal user intervention
Med Image Anal
(2017) - et al.
A fully integrated computer-aided diagnosis system for digital x-ray mammograms via deep learning detection, segmentation, and classification
Int J Med Inform
(2018) - et al.
Simultaneous detection and classification of breast masses in digital mammograms via a deep learning YOLO-based CAD system
Comput Methods Programs Biomed
(2018) - et al.
Convolutional neural network improvement for breast cancer classification
Expert Syst Appl
(2019)
An introduction to ROC analysis
Pattern Recognit Lett
Cancer statistics, 2019
CA Cancer J Clin
Breast cancer facts & figures
Cumulative probability of false-positive recall or biopsy recommendation after 10 years of screening mammography: a cohort study
Ann. Intern. Med.
Breast image analysis for risk assessment, detection, diagnosis, and treatment of cancer
Annu Rev Biomed Eng
Artificial intelligence in cancer imaging: clinical challenges and applications
CA Cancer J Clin
Shortest paths of mass contour estimates in mammography
Proceedings of the International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), Workshop on Breast Image Analysis (BIA)
Cited by (63)
A multi-task fusion model based on a residual–Multi-layer perceptron network for mammographic breast cancer screening
2024, Computer Methods and Programs in BiomedicineBreast cancer classification using deep learned features boosted with handcrafted features
2023, Biomedical Signal Processing and ControlBreast cancer detection and diagnosis using hybrid deep learning architecture
2023, Biomedical Signal Processing and ControlBreast Cancer Identification using Hybrid Algorithm
2024, International Journal of Intelligent Systems and Applications in Engineering