Computer‐aided diagnosis of breast ultrasound images using ensemble learning from convolutional neural networks

Highlights

• A computer-aided diagnosis (CAD) system was proposed to diagnose breast cancer in ultrasound images.

• In this study, we propose a CAD system for tumor diagnosis using an image fusion method combined with different image content representations and ensemble different CNN architectures on US images.

• The results of our CAD system in the SNUH dataset show that the accuracy, sensitivity, specificity, precision, F1 score, and the AUC of the proposed method were 91.10%, 85.14%, 95.77%, 94.03%, 89.36%, and 0.9697, respectively. The results of our CAD system in the open dataset (BUSI) show that the accuracy, sensitivity, specificity, precision, F1 score, and the AUC of the proposed method were 94.62%, 92.31%, 95.60%, 90%, 91.14%, and 0.9711, respectively.

Abstract

Breast ultrasound and computer aided diagnosis (CAD) has been used to classify tumors into benignancy or malignancy. However, conventional CAD software has some problems (such as handcrafted features are hard to design; conventional CAD systems are difficult to confirm overfitting problems, etc.). In our study, we propose a CAD system for tumor diagnosis using an image fusion method combined with different image content representations and ensemble different CNN architectures on US images. The CNN-based method proposed in this study includes VGGNet, ResNet, and DenseNet. In our private dataset, there was a total of 1687 tumors that including 953 benign and 734 malignant tumors. The accuracy, sensitivity, specificity, precision, F1 score and the AUC of the proposed method were 91.10%, 85.14%, 95.77%, 94.03%, 89.36%, and 0.9697 respectively. In the open dataset (BUSI), there was a total of 697 tumors that including 437 benign lesions, 210 malignant tumors, and 133 normal images. The accuracy, sensitivity, specificity, precision, F1 score, and the AUC of the proposed method were 94.62%, 92.31%, 95.60%, 90%, 91.14%, and 0.9711. In conclusion, the results indicated different image content representations that affect the prediction performance of the CAD system, more image information improves the prediction performance, and the tumor shape feature can improve the diagnostic effect.

Introduction

Ultrasound (US) is a useful way for the detection and diagnosis of breast cancer [1] because they are non-invasive, non-radioactive, real-time imaging, and high image resolution. However, to read US images requires well-trained and experienced radiologists. Even a well-trained expert might have a high inter-observer variation rate on tumor diagnosis [2]. Hence, computer-aided diagnosis (CAD) could be used to assist radiologists in breast cancer classification and detection [3], [4], [5], [6]. Recently, several studies [7], [8], [9], [10] have discussed the automatic breast cancer diagnosis method to classify benign and malignant tumor in US images.

Convolutional neural network (CNN) approaches have been proven to be very effective in a wide range of computer vision applications [11], [12], [13], [14]. In addition, CNN can recognize visual patterns directly from pixel images with minimal preprocessing and automate the whole feature extraction process. Furthermore, CNN has been employed broadly in medical image analysis, such as segmentation [15], classification [16], and detection [17]. In recent years, the usages of CNN models in ultrasound of breast cancers are shown significant development. Byra et al. [18] proposed a color conversion method that transfers the grayscale ultrasound images to 3-channel (RGB) images, which enhanced the classification performance. Yap et al. [19] proposed an end-to-end deep learning model in automated breast ultrasound lesions recognition; they are the first to implement semantic segmentation on BUS images and compared the performance between different CNN models. Yap et al. [20] proposed an automatic detection system of breast ultrasound lesions using CNN models, which compared three different CNN models of CAD systems that reduced the operator-dependent problem. Even if the CNN method was widely used in medical image fields for segmentation and diagnosis, but we want to understand the impact of different image content descriptions and CNN architectures on the various US diagnostic system.

In our study, we propose a CAD system for tumor diagnosis using an image fusion method combined with different image content representations and ensemble different CNN architectures on US images. First, we manually extract the region of interest (ROI), which covers the whole tumor and the ROI boundary close to the tumor margin. Then, the expert manually extracts the tumor region and the tumor shape image (TSI). In addition, we employed an image fusion method to enhance the diagnostic performance of our CAD system. Finally, we employed the ensemble method to combine multiple CNN results.