Mass detection in mammograms by bilateral analysis using convolution neural network

doi:10.1016/j.cmpb.2020.105518

Computer Methods and Programs in Biomedicine

Volume 195, October 2020, 105518

https://doi.org/10.1016/j.cmpb.2020.105518 Get rights and content

Highlight

•
The self-supervised learning network is designed for bilateral mammogram registration. It directly estimates spatial transformation by maximizing an image-wise similarity metric, which solves the problem of corresponding points labeling.
•
A Siamese-Faster-RCNN consisting of RPN and Siamese-FC network is designed and employed for mass detection. It combines single-image mass detection and bilateral comparison into a whole, which solves the problem of separating single-image mass detection and bilateral comparison apart.

Abstract

Background and objective

Automatic detection of the masses in mammograms is a big challenge and plays a crucial role to assist radiologists for accurate diagnosis. In this paper, a bilateral image analysis method based on Convolution Neural Network (CNN) is developed for mass detection in mammograms.

Methods

The proposed bilateral mass detection method consists of two networks: a registration network for registering bilateral mammograms and a Siamese-Faster-RCNN network for mass detection using a pair of registered mammograms. In the first step, self-supervised learning network is built to learn the spatial transformation between bilateral mammograms. This network can directly estimate spatial transformation by maximizing an image-wise similarity metric and corresponding points labeling is not needed. In the second step, an end-to-end network combining the Region Proposal Network (RPN) and a Siamese Fully Connected (Siamese-FC) network is designed. Different from existing methods, the designed network integrates mass detection on single image with registered bilateral images comparison.

Results

The proposed method is evaluated on three datasets (publicly available dataset INbreast and private dataset BCPKUPH and TXMD). For INbreast dataset, the proposed method achieves 0.88 true positive rate (TPR) with 1.12 false positives per image (FPs/I). For BCPKUPH dataset, the proposed method achieves 0.85 TPR with 1.86 FPs/I. For TXMD dataset, the proposed method achieves 0.85 TPR with 2.70 FPs/I.

Conclusions

Registration experimental result shows that the proposed method is suitable for bilateral mass detection. Mass detection experimental results show that the proposed method performs better than unilateral mass detection method, different bilateral connection schemes and image level fusion bilateral schemes.

Introduction

Breast cancer is the leading type of cancer which affects women's health in developed and developing countries. According to statistics, breast cancer accounts for 22.9% of all newly diagnosed cancers, causing 13.7% cancer related deaths all over the world [1]. In spite of that, if breast cancer can be early diagnosed, it has a high chance to be cured [2]. Based on this scenario, world governments together with health professionals have mobilized various campaigns to alert the female population to the risks that cause this disease [3].

Among many imaging techniques, mammography is currently considered to be the most widely used manner [4]. Mass is the most common type of breast cancer. Compared with normal tissues, a mass usually exhibits in the form of a high intensity region in the mammogram. Mass detection is an important step in mammogram interpreting. Due to the low signal-to-noise ratio of mass visualization and the lack of consistent patterns in shape, size, appearance and location [5], it is a difficult task for radiologists to distinguish between normal tissue and mass. For example, glandular tissues are compressed in a mammogram exam, presenting similar features with a mass. In such case, they may be falsely detected as masses with high probability, which are called false positives (FPs). Studies have shown that manual analysis has a specificity of 91% and a sensitivity of 84% by single radiologist in the task of diagnosing breast cancer [6]. The performance can be improved using independent readings by two radiologists [6]. However, the double reading manner increases the burden of radiologists and reduces work efficiency. In the last several decades, computer vision technologies have been developed for the purpose of a second view [7]. Existing researches have shown that detection sensitivity can be improved in clinical practice with the use of computer vision aided diagnosis that acts as a second view.

Region of interests (ROIs) detected on single mammogram may contain several FP regions. It is noteworthy that bilateral breasts of the same patient tend to have a high degree symmetry [8]. Radiologists may remove FPs by comparing the left breast and the right breast with the same view when interpreting mammograms. This practice is commonly known as bilateral analysis. The main idea of bilateral analysis is that the similarity between corresponding regions of a pair of mammograms in the same view is high for normal tissue and low for a mass [8].

In this paper, a bilateral mass detection method is proposed combining self-supervised learning network and Siamese-Faster-RCNN. A bilateral mammogram registration scheme based on self-supervised learning network is designed to make the bilateral mammograms aligned. As breast is compressed during mammogram imaging, it is even hard for a radiologist to find exact corresponding locations in the bilateral mammogram. The designed registration scheme can directly estimate the spatial transformation by maximizing an image-wise similarity metric and no corresponding points labeling is needed. To detect the mass using bilateral analysis, Siamese-Faster-RCNN network is designed by integrating Siamese Fully Connected (Siamese-FC) network into Region Proposal Network (RPN). For existing bilateral based mass detection method, single-image mass detection and bilateral comparison are separated. It is a sub-optimal problem. The designed Siamese-Faster-RCNN network is trained by an end-to-end manner, which might be the first attempt to combine mass detection and bilateral images comparison into one network.

The rest of the paper is organized as follows. Section 2 presents a review of related works. Section 3 describes the proposed method including image registration, mass detection and bilateral comparison. Section 4 shows the experiments and results. Section 5 and 6 give the discussion and conclusion.

Section snippets

Related works

There are two important issues in bilateral mass detection. One is how to register bilateral mammograms. The other is how to analyze a pair of images.

Methods

The proposed mass detection method consists of three stages: breast segmentation, registration between left breast and right breast, mass detection using bilateral comparison. The whole method is shown in Fig. 1. Pair images after breast segmentation pass through the registration network and the deformed image is obtained. Then the right image and the deformed left image are input to the Siamese-Faster-RCNN network to get the mass detection result for the right image. For mass detection of the

Dataset and experiment implementation

Experiments are performed on three datasets: one is public dataset-INbreast [36], and the other two are private dataset-Breast Center of Peking University People's Hospital (BCPKUPH) [20] and TXMD. The INbreast dataset includes 115 cases (410 images). Each case contains one left CC view, one left MLO view, one right CC view and one right MLO view. Image sizes are 3328 × 4084 and 2560 × 3328. The pixel width is 16-bits. The BCPKUPH dataset includes 100 cases and four images like INbreast are

Discussion

Examples of detection results by unilateral analysis and the proposed bilateral analysis are shown in Fig. 13. True label is indicated by the blue rectangle and the detection result is indicated by the red rectangle. The unilateral results are shown in Fig. 13(a)-(c). It can be seen that some glandular tissues are falsely detected as mass. The proposed bilateral analysis results are shown in Fig. 13(d)-(f). All FP regions are removed by bilateral analysis.

There are some shortages in the

Conclusions

In this paper, a bilateral mass detection method is proposed for mammogram combining self-supervised learning network and Siamese-Faster-RCNN. Registration experimental result shows that the proposed method does not distort glandular structure and the computation efficiency is high. It is suitable for bilateral mass detection. For mass detection, four observations can be concluded. Bilateral analysis outperforms mass detection on single mammogram, especially for dense breast. Using single

Declaration of Competing Interest

None Declared.

Acknowledgments

The work was supported in part by the National Nature Science Foundation of China61872030, and Major Science and Technology Innovation Project of Shandong Province2019TSLH0206.

References (36)

Z.C. Jiao et al.
A parasitic metric learning net for breast mass classification based on mammography
Pattern Recognit.
(2018)
P. Shi et al.
A hierarchical pipeline for breast boundary segmentation and calcification detection in mammograms
Comput. Biol. Med.
(2018)
J. Wang et al.
A context-sensitive deep learning approach for microcalcification detection in mammograms
Pattern Recognit.
(2018)
N. Dhungel et al.
A deep learning approach for the analysis of masses in mammograms with minimal user intervention
Med. Image Anal.
(2017)
Y.F. Li et al.
A bilateral analysis scheme for false positive reduction in mammogram mass detection
Comput. Biol. Med.
(2015)
M.A. Al-masni et al.
Simultaneous detection and classification of breast masses in digital mammograms via a deep learning YOLO-based CAD system
Comput. Methods Programs Biomed.
(2018)
I.C. Moreira et al.
Inbreast: toward a full-field digital mammographic database
Acad. Radiol.
(2012)
J.O.B. Diniz et al.
Detection of mass regions in mammograms by bilateral analysis adapted to breast density using similarity indexes and convolutional neural networks
Comput. Methods Programs Biomed.
(2018)
Y.F. Li et al.
Pectoral muscle segmentation in mammograms based on homogenous texture and intensity deviation
Pattern Recognit.
(2012)
F.S.S. de Oliveira et al.
Classification of breast regions as mass and non-mass based on digital mammograms using taxonomic indexes and SVM
Comput. Biol. Med.
(2015)

D. Scutt et al.

Breast asymmetry and predisposition to breast cancer

Breast Cancer Res.

(2006)

K. Bovis et al.

Detection of masses in mammograms using texture features

A. Kelder et al.

A new computer‐aided detection approach based on analysis of local and global mammographic feature asymmetry

Med. Phys.

(2018)

J.M. Celaya-Padilla et al.

Bilateral image subtraction features for multivariate automated classification of breast cancer risk

J.M. Celaya-Padilla et al.

Bilateral image subtraction and multivariate models for the automated triaging of screening mammograms

Biomed. Res. Int.

(2015)

S. Xu et al.

Bilateral asymmetry detection in mammograms using non-rigid registraion and pseudo-color coding

Z. Yang et al.

Multi-temporal remote sensing image registration using deep convolutional features

IEEE Access

(2018)

X. Han et al.

MatchNet: unifying feature and metric learning for patch-based matching

Cited by (20)

A deep learning architecture with an object-detection algorithm and a convolutional neural network for breast mass detection and visualization
2023, Healthcare Analytics
This study presents an integrated deep learning architecture with an object-detection algorithm and a convolutional neural network (CNN) for breast mass detection and visualization. Mammograms are analyzed to identify and localize breast mass lesions to aid clinician review. Two complementary forms of deep learning are used to identify the regions of interest (ROIs). An object-detection algorithm, YOLO v5, analyzes the entire mammogram to identify discrete image regions likely to represent masses. Object detections exhibit high precision, but the object-detection stage alone has insufficient overall accuracy for a clinical application. A CNN independently analyzes the mammogram after it has been decomposed into subregion tiles and is trained to emphasize sensitivity (recall). The ROIs identified by each analysis are highlighted in different colors to facilitate an efficient staged review. The CNN stage nearly always detects tumor masses when present but typically occupies a larger area of the image. By inspecting the high-precision regions followed by the high-sensitivity regions, clinicians can quickly identify likely lesions before completing the review of the full mammogram. On average, the ROIs occupy less than 20% of the tissue in the mammograms, even without removing pectoral muscle from the analysis. As a result, the proposed system helps clinicians review mammograms with greater accuracy and efficiency.
Early detection and classification of abnormality in prior mammograms using image-to-image translation and YOLO techniques
2022, Computer Methods and Programs in Biomedicine
Citation Excerpt :
The study yielded a detection accuracy of up to 90% and a classification accuracy of 93.5% on the DDSM dataset. In another work by Li et al. [25], a bilateral mass detection method was introduced using two networks: a registration network between left and right breasts and a Siamese-Faster-RCNN network to detect masses from pairs of registered mammograms. They reported results of a true positive rate of 0.88 on the INbreast dataset and 0.85 on a private dataset.
Computer-aided-detection (CAD) systems have been developed to assist radiologists on finding suspicious lesions in mammogram. Deep Learning technology have recently succeeded to increase the chance of recognizing abnormality at an early stage in order to avoid unnecessary biopsies and decrease the mortality rate. In this study, we investigated the effectiveness of an end-to-end fusion model based on You-Only-Look-Once (YOLO) architecture, to simultaneously detect and classify suspicious breast lesions on digital mammograms. Four categories of cases were included: Mass, Calcification, Architectural Distortions, and Normal from a private digital mammographic database including 413 cases. For all cases, Prior mammograms (typically scanned 1 year before) were all reported as Normal, while Current mammograms were diagnosed as cancerous (confirmed by biopsies) or healthy.
We propose to apply the YOLO-based fusion model to the Current mammograms for breast lesions detection and classification. Then apply the same model retrospectively to synthetic mammograms for an early cancer prediction, where the synthetic mammograms were generated from the Prior mammograms by using the image-to-image translation models, CycleGAN and Pix2Pix.
Evaluation results showed that our methodology could significantly detect and classify breast lesions on Current mammograms with a highest rate of 93% ± 0.118 for Mass lesions, 88% ± 0.09 for Calcification lesions, and 95% ± 0.06 for Architectural Distortion lesions. In addition, we reported evaluation results on Prior mammograms with a highest rate of 36% ± 0.01 for Mass lesions, 14% ± 0.01 for Calcification lesions, and 50% ± 0.02 for Architectural Distortion lesions. Normal mammograms were accordingly classified with an accuracy rate of 92% ± 0.09 and 90% ± 0.06 respectively on Current and Prior exams.
Our proposed framework was first developed to help detecting and identifying suspicious breast lesions in X-ray mammograms on their Current screening. The work was also suggested to reduce the temporal changes between pairs of Prior and follow-up screenings for early predicting the location and type of abnormalities in Prior mammogram screening. The paper presented a CAD method to assist doctors and experts to identify the risk of breast cancer presence. Overall, the proposed CAD method incorporates the advances of image processing, deep learning and image-to-image translation for a biomedical application.
Simultaneous detection and diagnosis of mammogram mass using bilateral analysis and soft label based metric learning
2022, Biocybernetics and Biomedical Engineering
Citation Excerpt :
For example, MatchNet architecture was introduced in [21], where the deep features and robust feature comparison network were jointly learned for a pair of breast images. Li et al. [8] built a self-supervised learning network to learn the spatial transformation between bilateral mammograms and it is no need to previously label corresponding points. After obtaining the detected ROI and its contralateral region, the similarity level between them need to be analyzed to discriminate the mass-to-normal pairs and normal-to-normal pairs.
In the clinics, mammogram masses appear as asymmetric structures between the left and right breasts. In this paper, we design a bilateral image analysis method based on convolutional neural network which can detect and classify breast mass regions simultaneously. It mainly consists of three parts: a feature similarity based region matching technique, mass region of interest (ROI) selection step and a deep metric learning based classifier. Firstly, discriminative score maps are calculated relied on the deep features extracted from bilateral left and right mammograms respectively in global or local spatial image domain. The contralateral correspondences are determined by minimum discriminative scores. Secondly, to select the mass candidate ROIs and further remove false positive mass-to-normal pairs, we propose a dynamic histogram weighting mechanism with three new constrains imposed on the distribution of discriminative score histogram. In addition, a novel soft label based deep metric learning regularization is designed for mass ROI classifier to tackle the large variation of masses in shape, size, texture and breast density. We apply it to the open dataset Digital Database for Screening Mammography. Compared with other state-of-the-art approaches, the proposed scheme gives competitive results in classification and localization tasks for mammographic lesions.
MommiNet-v2: Mammographic multi-view mass identification networks
2021, Medical Image Analysis
Citation Excerpt :
Recently, multi-view based approaches are attracting an increasing attention. In (Diniz et al., 2018; Li et al., 2020; Liu et al., 2019), bilateral analysis has been incorporated in DNN-based approaches. Some other DNN-based methods consider information of ipsilateral mammograms (Carneiro et al., 2017; Perek et al., 2018; Ren et al., 2019).
Many existing approaches for mammogram analysis are based on single view. Some recent DNN-based multi-view approaches can perform either bilateral or ipsilateral analysis, while in practice, radiologists use both to achieve the best clinical outcome. MommiNet is the first DNN-based tri-view mass identification approach, which can simultaneously perform bilateral and ipsilateral analysis of mammographic images, and in turn, can fully emulate the radiologists’ reading practice. In this paper, we present MommiNet-v2, with improved network architecture and performance. Novel high-resolution network (HRNet)-based architectures are proposed to learn the symmetry and geometry constraints, to fully aggregate the information from all views for accurate mass detection. A multi-task learning scheme is adopted to incorporate both Breast Imaging-Reporting and Data System (BI-RADS) and biopsy information to train a mass malignancy classification network. Extensive experiments have been conducted on the public DDSM (Digital Database for Screening Mammography) dataset and our in-house dataset, and state-of-the-art results have been achieved in terms of mass detection accuracy. Satisfactory mass malignancy classification result has also been obtained on our in-house dataset.
Survey on deep learning in multimodal medical imaging for cancer detection
2023, arXiv
An evidential deep learning framework for assessment of mammograms
2023, Research Square

View all citing articles on Scopus

View full text

Mass detection in mammograms by bilateral analysis using convolution neural network

Highlight

Abstract

Background and objective

Methods

Results

Conclusions

Introduction

Section snippets

Related works

Methods

Dataset and experiment implementation

Discussion

Conclusions

Declaration of Competing Interest

Acknowledgments

Pattern Recognit.

Comput. Biol. Med.

Pattern Recognit.

Med. Image Anal.

Comput. Biol. Med.

Comput. Methods Programs Biomed.

Acad. Radiol.

Detection of mass regions in mammograms by bilateral analysis adapted to breast density using similarity indexes and convolutional neural networks

Comput. Methods Programs Biomed.

Pectoral muscle segmentation in mammograms based on homogenous texture and intensity deviation

Pattern Recognit.

Classification of breast regions as mass and non-mass based on digital mammograms using taxonomic indexes and SVM

Comput. Biol. Med.

Breast asymmetry and predisposition to breast cancer

Breast Cancer Res.

Detection of masses in mammograms using texture features

A new computer‐aided detection approach based on analysis of local and global mammographic feature asymmetry

Med. Phys.

Bilateral image subtraction features for multivariate automated classification of breast cancer risk

Bilateral image subtraction and multivariate models for the automated triaging of screening mammograms

Biomed. Res. Int.

Bilateral asymmetry detection in mammograms using non-rigid registraion and pseudo-color coding

Multi-temporal remote sensing image registration using deep convolutional features

IEEE Access

MatchNet: unifying feature and metric learning for patch-based matching