Abstract
To train a convolutional neural network (CNN) from scratch is not suitable for medical image tasks with insufficient data. Benefiting from the transfer learning, the pre-trained CNN model can provide a reliable initial solution for model optimization of medical image classification. A key concern in breast cancer histology classification is that the model should cover the multi-scale features including nuclei-scale, nuclei organization, and structure-scale features. Inspired by these conjectures, we proposed a novel fusion convolutional neural network (FCNN) based on pre-trained VGG19. The FCNN fuses the shallow, intermediate abstract, and abstract layers to approximately cover the multi-scale features. In order to improve the sensitivity of carcinoma classes, the prediction priority is introduced to enable the lesions can be detected as early as possible. Experimental results show that the proposed FCNN can approximately cover the nuclei-scale, nuclei organization, and structure-scale features. Accuracies of 85%, 75%, and 80.56% are achieved in Initial, Extended, and Overall test set, respectively. The source code for this research is available at https://github.com/yxchspring/breasthistolgoy.











Similar content being viewed by others
References
Abadi M, Barham P, Chen J, et al. (2016) Tensorflow: A system for large-scale machine learning[C]. 12th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 16). 265–283
Albarqouni S, Baur C, Achilles F, Belagiannis V, Demirci S, Navab N (2016) AggNet: deep Learning from crowds for mitosis detection in breast Cancer histology images[J]. IEEE Trans Med Imaging 35(5):1313–1321
Araújo T, Aresta G, Castro E, Rouco J, Aguiar P, Eloy C, Polónia A, Campilho A (2017) Classification of breast cancer histology images using convolutional neural networks[J]. PLoS One 12(6):e0177544
Beam AL, Kohane IS (2018) Big data and machine learning in health care[J]. JAMA 319(13):1317–1318
Berg E (2015) Breast cancer[J]. CMAJ Can Med Assoc J 75(S1):257–269
Bioimaging Challenge (2015) Breast Histology Dataset: Available at https://rdm.inesctec.pt/dataset/nis-2017-003. Accessed 18 July 2019
Che D, Pan W, Ma Y et al (2019) Tire tread pattern recognition based on non-linear activated aggregation residual neural network[J]. J Jiangxi Univ Sci Technol 40(05):80–85
Ciresan DC, Giusti A, Gambardella LM et al (2013) Mitosis detection in breast Cancer histology images with deep neural networks[J]. Med Image Comput Comput Assist Interv 16(Pt 2):411–418
DeSantis CE, Ma J, Goding Sauer A et al (2017) Breast cancer statistics, 2017, racial disparity in mortality by state[J]. CA Cancer J Clin 67(6):439–448
Doi K (2007) Computer-aided diagnosis in medical imaging: historical review, current status and future potential[J]. Comput Med Imaging Graph 31(4–5):198–211
Golatkar A, Anand D, Sethi A (2018) Classification of breast cancer histology using deep learning[C]. International Conference Image Analysis and Recognition. Springer: Cham, 837–844
Gudmundsson E, Straus CM, Armato SG III (2019) Pre-trained deep convolutional neural networks for the segmentation of malignant pleural mesothelioma tumor on CT scans[C]. Medical Imaging 2019: computer-aided diagnosis. Int Soc Opt Photon 10950:109503J
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition[C]. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778
Huang JB, Yang MH (2010) Fast sparse representation with prototypes[C]. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, 3618–3625
Jaworek-Korjakowska J, Kleczek P, Gorgon M (2019) Melanoma thickness prediction based on convolutional neural Network with VGG-19 model Transfer Learning[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops: 0–0
Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories[C]. 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR'06). IEEE, 2:2169–2178
LeCun Y, Bengio Y, Hinton G (2015) Deep learning[J]. Nature 521(7553):436–444
Li B, Sun F, Zhang Y (2019) Building recognition using Gist feature based on locality sensitive histograms of oriented gradients[J]. Pattern Recognit Image Anal 29(2):258–267
McGuire S (2015) World cancer report 2014. Geneva, Switzerland: World Health Organization, international agency for research on cancer, WHO Press, [J]. 2016:418–419
Nawaz W, Ahmed S, Tahir A, et al. (2018) Classification of breast cancer histology images using alexnet[C]. International Conference Image Analysis and Recognition. Springer: Cham, 869–876
Oliva A, Torralba A (2001) Modeling the shape of the scene: A holistic representation of the spatial envelope[J]. Int J Comput Vis 42(3):145–175
Rajaraman S, Antani SK, Poostchi M, Silamut K, Hossain MA, Maude RJ, Jaeger S, Thoma GR (2018) Pre-trained convolutional neural networks as feature extractors toward improved malaria parasite detection in thin blood smear images[J]. PeerJ 6:e4568
Reinhard E, Adhikhmin M, Gooch B, Shirley P (2001) Color transfer between images[J]. IEEE Comput Graph Appl 21(5):34–41
Shima Y, Nakashima Y, Yasuda M (2018) Classifying for a mixture of object images and character patterns by using CNN pre-trained for large-scale object image dataset[C]. 2018 13th IEEE Conference on Industrial Electronics and Applications (ICIEA). IEEE, 2360–2365
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition[J]. arXiv preprint arXiv:1409.1556
Sivic J, Zisserman A (2003) Video Google: A text retrieval approach to object matching in videos[C]. Proceedings Ninth IEEE International Conference on Computer Vision, Nice, France, 1470–1477
Stain Normalisation Toolbox (n.d.): Available at https://warwick.ac.uk/fac/sci/dcs/research/tia/software/sntoolbox. Accessed 17 July 2019
Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision[C]. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2818–2826
Szeliski R (2010) Computer vision: algorithms and applications[M]. Springer Science & Business Media, Washington
Upadhyay PK, Chandra S (2019) Salient bag of feature for skin lesion recognition[J]. Int J Performability Eng 15(4):1083–1093
Wen L, Li X, Li X, et al. (2019) A New Transfer Learning Based on VGG-19 Network for Fault Diagnosis[C]. 2019 IEEE 23rd international conference on computer supported cooperative work in design (CSCWD). IEEE, 205–209
Yang J, Ma J (2019) Feed-forward neural network training using sparse representation[J]. Expert Syst Appl 116:255–264
Yang X, Yeo SY, Hong JM et al (2016) A deep learning approach for tumor tissue image classification[J]. IASTED Biomedical Engineering
Yang G, Xu N, Li F et al (2018) Research on deep learning classification for nonlinear activation function[J]. J Jiangxi Univ Sci Technol 39(03):76–83
Yu X, Yu Z, Pang W et al (2018) An improved EMD-Based dissimilarity metric for unsupervised linear subspace Learning[J]. Complexity 2018:1–24
Yu X, Yu Z, Wu L et al (2019) Data-driven two-layer visual dictionary structure learning[J]. J Electron Imaging 28(2):023006
Yu X, Cao S, Zhou Y, Yu Z, Xu Y (2020) Co-expression based cancer staging and application[J]. Sci Rep 10(1):10624
Yu X, Zhang Z, Wu L et al (2020) Deep ensemble Learning for human action recognition in still images[J]. Complexity 2020(1):1–23
Zhang J, Wang L, Wen X (2019) Combination of GIST and PHOG features for calligraphy styles classification[C]. In Proceedings of the 2019 4th International Conference on Multimedia Systems and Signal Processing, 21–24
Zhang Y, Zhang H, Yu M et al (2019) Sparse representation-based video quality assessment for synthesized 3D videos[J]. IEEE Transactions on Image Processing 29:509–524
Availability of data and material
The breast histology image data used to support the findings of this study are included within the article.
Funding
This research was funded by 1) Doctoral Scientific Research Foundation of Jiangxi University of Science and Technology, grant number jxxjbs19029, jxxjbs19006, jxxjbs19012. 2) National Natural Science Foundation of China, grant number 61901198, 61902145.
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of interest
The authors declare no conflict of interest.
Code availability
The source code for this research is available at https://github.com/yxchspring/breasthistolgoy.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Yu, X., Chen, H., Liang, M. et al. A transfer learning-based novel fusion convolutional neural network for breast cancer histology classification. Multimed Tools Appl 81, 11949–11963 (2022). https://doi.org/10.1007/s11042-020-09977-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-020-09977-1