ABSTRACT
The current challenge in breast cancer segmentation on hematoxylin and eosin-stained images lies in accurately capturing histologic phenotypes associated with cancer biomarkers. Many researchers have studied histologic patterns of cancerous regions of breast cancer images for a better understanding of diagnosis and treatment, yet it is not fully investigated because of its variability, complexity, and large data volumes. Comprehensive experiments on breast cancer segmentation can address this challenge by identifying heterogeneous cell regions in the tumor microenvironment and investigating a new methodology of segmenting histologic images by using advanced deep learning architectures. In this paper, we present findings in three experiments on breast cancer segmentation, exploring the effectiveness of a convolutional neural network called UNet identifying multiple heterogeneous cells such as blood, blood vessels, fat, glandular secretions, necrosis, and plasma cells in a tumor microenvironment, investigating the performance of UNet architecture in different sizes of cancerous regions such as a tumor, and tumor-infiltrating lymphocytes, and proposing a new methodology of a neural-style guided data augmentation focusing on the image segmentation of breast tumor-related nuclei in hematoxylin and eosin-stained images. The experiment results show that a modified UNet performs well on fat identification. Another experiment shows that the smaller input size of the tumor, stroma, and tumor-infiltrating lymphocyte provides better performance than the larger input size. Moreover, we demonstrate that the proposed method outperforms the traditional deep neural network models.
- Ayat Abedalla, Malak Abdullah, Mahmoud Al-Ayyoub, and Elhadj Benkhelifa. 2020. The 2ST-UNet for Pneumothorax Segmentation in Chest X-Rays using ResNet34 as a Backbone for U-Net. arXiv preprint arXiv:2009.02805 (2020).Google Scholar
- Mohamed Amgad, Habiba Elfandy, Hagar Hussein, Lamees A Atteya, Mai AT Elsebaie, Lamia S Abo Elnasr, Rokia A Sakr, Hazem SE Salem, Ahmed F Ismail, Anas M Saad, et al. 2019. Structured Crowdsourcing Enables Convolutional Segmentation of Histology Images. Bioinformatics 35, 18 (2019), 3461--3467.Google ScholarCross Ref
- Vickie E Baracos, Tony Reiman, Marina Mourtzakis, Ioannis Gioulbasanis, and Sami Antoun. 2010. Body Composition in Patients with Non-Small Cell Lung Cancer: a Contemporary View of Cancer Cachexia with the Use of Computed Tomography Image Analysis. The American journal of clinical nutrition 91, 4 (2010), 1133S-1137S.Google Scholar
- Bhupender S Chhikara and Keykavous Parang. 2023. Global Cancer Statistics 2022: the Trends Projection Analysis. Chemical Biology Letters 10, 1 (2023), 451--451.Google Scholar
- Antonia Creswell, Tom White, Vincent Dumoulin, Kai Arulkumaran, Biswa Sengupta, and Anil A Bharath. 2018. Generative Adversarial Networks: An Overview. IEEE signal processing magazine 35, 1 (2018), 53--65.Google Scholar
- Angel Cruz-Roa, Hannah Gilmore, Ajay Basavanhally, Michael Feldman, Shridar Ganesan, Natalie NC Shih, John Tomaszewski, Fabio A González, and Anant Madabhushi. 2017. Accurate and Reproducible Invasive Breast Cancer Detection in Whole-Slide Images: A Deep Learning Approach for Quantifying Tumor Extent. Scientific reports 7, 1 (2017), 46450.Google Scholar
- Yuxin Cui, Guiying Zhang, Zhonghao Liu, Zheng Xiong, and Jianjun Hu. 2019. A Deep Learning Algorithm for One-Step Contour Aware Nuclei Segmentation of Histopathology Images. Medical & biological engineering & computing 57 (2019), 2027--2043.Google Scholar
- Sergio Damas, Oscar Cordón, and Jose Santamaría. 2011. Medical Image Registration using Evolutionary Computation: An Experimental Survey. IEEE Computational Intelligence Magazine 6, 4 (2011), 26--42.Google ScholarDigital Library
- Andre Esteva, Katherine Chou, Serena Yeung, Nikhil Naik, Ali Madani, Ali Mottaghi, Yun Liu, Eric Topol, Jeff Dean, and Richard Socher. 2021. Deep Learning-Enabled Medical Computer Vision. NPJ digital medicine 4, 1 (2021), 5.Google Scholar
- Arthur Francisco Araújo Fernandes, João Ricardo Rebouças Dórea, and Guilherme Jordão de Magalhães Rosa. 2020. Image Analysis and Computer Vision Applications in Animal Sciences: an Overview. Frontiers in Veterinary Science 7 (2020), 551269.Google ScholarCross Ref
- Sourodip Ghosh, Aunkit Chaki, and KC Santosh. 2021. Improved U-Net Architecture with VGG-16 for Brain Tumor Segmentation. Physical and Engineering Sciences in Medicine 44, 3 (2021), 703--712.Google ScholarCross Ref
- Intisar Rizwan I Haque and Jeremiah Neubert. 2020. Deep Learning Approaches to Biomedical Image Segmentation. Informatics in Medicine Unlocked 18 (2020), 100297.Google ScholarCross Ref
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. Las Vegas, NV, USA, 770--778.Google ScholarCross Ref
- Andreas Heinrich, Felix V Güttler, Sebastian Schenkl, Rebecca Wagner, and Ulf K-M Teichgräber. 2020. Automatic Human Identification based on Dental X-ray Radiographs using Computer Vision. Scientific reports 10, 1 (2020), 3801.Google Scholar
- Kirsi Holli, Anna-Leena Lääperi, Lara Harrison, Tiina Luukkaala, Terttu Toivonen, Pertti Ryymin, Prasun Dastidar, Seppo Soimakallio, and Hannu Eskola. 2010. Characterization of Breast Cancer Types by Texture Analysis of Magnetic Resonance Images. Academic radiology 17, 2 (2010), 135--141.Google ScholarCross Ref
- Minui Hong, Jinwoo Choi, and Gunhee Kim. 2021. Stylemix: Separating Content and Style for Enhanced Data Augmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. Nashville, TN, USA, 14862--14870.Google ScholarCross Ref
- Le Hou, Vu Nguyen, Ariel B Kanevsky, Dimitris Samaras, Tahsin M Kurc, Tianhao Zhao, Rajarsi R Gupta, Yi Gao, Wenjin Chen, David Foran, et al. 2019. Sparse Autoencoder for Unsupervised Nucleus Detection and Representation in Histopathology Images. Pattern recognition 86 (2019), 188--200.Google Scholar
- Aiyue Huang, Li Jiang, Jiangshan Zhang, and Qing Wang. 2022. Attention-VGG16-UNet: a Novel Deep Learning Approach for Automatic Segmentation of the Median Nerve in Ultrasound Images. Quantitative Imaging in Medicine and Surgery 12, 6 (2022), 3138.Google ScholarCross Ref
- Philip TG Jackson, Amir Atapour Abarghouei, Stephen Bonner, Toby P Breckon, and Boguslaw Obara. 2019. Style Augmentation: Data Augmentation via Style Randomization. In CVPR workshops, Vol. 6. Long Beach, CA, USA, 10--11.Google Scholar
- Varun Jampani, Ujjwal, Jayanthi Sivaswamy, and Vivek Vaidya. 2012. Assessment of Computational Visual Attention Models on Medical Images. In Proceedings of the Eighth Indian Conference on Computer Vision, Graphics and Image Processing. Mumbai, India, 1--8.Google ScholarDigital Library
- Neeraj Kumar, Ruchika Verma, Sanuj Sharma, Surabhi Bhargava, Abhishek Vahadane, and Amit Sethi. 2017. A Dataset and a Technique for Generalized Nuclear Segmentation for Computational Pathology. IEEE transactions on medical imaging 36, 7 (2017), 1550--1560.Google ScholarCross Ref
- Sanghoon Lee, Mohamed Amgad, Pooya Mobadersany, Matt McCormick, Brian P Pollack, Habiba Elfandy, Hagar Hussein, David A Gutman, and Lee AD Cooper. 2021. Interactive Classification of Whole-Slide Imaging Data for Cancer Researchers. Cancer research 81, 4 (2021), 1171--1177.Google Scholar
- Sanghoon Lee, Sunho Park, Jae-Ho Cheong, Sam C Wang, Matthew R Porembka, In-Ho Kim, Sung Hak Lee, and Tae Hyun Hwang. 2023. Spatial Image Analysis of Tumor-Infiltrating Lymphocytes on Gastric Cancer to Predict Anti-PD1 Inhibitor Response. Cancer Research 83, 7_Supplement (2023), 1360--1360.Google Scholar
- Shuying Liu and Weihong Deng. 2015. Very Deep Convolutional Neural Network based Image Classification using Small Training Sample Size. In 2015 3rd IAPR Asian conference on pattern recognition (ACPR). IEEE, Kuala Lumpur, Malaysia, 730--734.Google ScholarCross Ref
- Anant Madabhushi and George Lee. 2016. Image Analysis and Machine Learning in Digital Pathology: Challenges and Opportunities. Medical image analysis 33 (2016), 170--175.Google Scholar
- Domingo Mery, Erick Svec, Marco Arias, Vladimir Riffo, Jose M Saavedra, and Sandipan Banerjee. 2016. Modern Computer Vision Techniques for X-ray Testing in Baggage Inspection. IEEE Transactions on Systems, Man, and Cybernetics: Systems 47, 4 (2016), 682--692.Google ScholarCross Ref
- Michael Nalisnik, Mohamed Amgad, Sanghoon Lee, Sameer H Halani, Jose Enrique Velazquez Vega, Daniel J Brat, David A Gutman, and Lee AD Cooper. 2017. Interactive Phenotyping of Large-scale Histology Imaging Data with HistomicsML. Scientific reports 7, 1 (2017), 14588.Google Scholar
- Peter Naylor, Marick Laé, Fabien Reyal, and Thomas Walter. 2018. Segmentation of Nuclei in Histopathology Images by Deep Regression of the Distance Map. IEEE transactions on medical imaging 38, 2 (2018), 448--459.Google Scholar
- Ozan Oktay, Jo Schlemper, Loic Le Folgoc, Matthew Lee, Mattias Heinrich, Kazunari Misawa, Kensaku Mori, Steven McDonagh, Nils Y Hammerla, Bernhard Kainz, et al. 2018. Attention U-net: Learning where to Look for the Pancreas. arXiv preprint arXiv:1804.03999 (2018).Google Scholar
- Anindya Apriliyanti Pravitasari, Nur Iriawan, Mawanda Almuhayar, Taufik Azmi, Irhamah Irhamah, Kartika Fithriasari, Santi Wulan Purnami, and Widiana Ferriastuti. 2020. UNet-VGG16 with Transfer Learning for MRI-based Brain Tumor Segmentation. TELKOMNIKA (Telecommunication Computing Electronics and Control) 18, 3 (2020), 1310--1318.Google ScholarCross Ref
- Anindya Apriliyanti Pravitasari, Nur Iriawan, Ulfa Siti Nuraini, and Dwilaksana Abdullah Rasyid. 2022. On Comparing Optimizer of UNet-VGG16 Architecture for Brain Tumor Image Segmentation. In Brain Tumor MRI Image Segmentation Using Deep Learning Techniques, Jyotismita Chaki (Ed.). Academic Press, 197--215.Google Scholar
- Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. Springer, Munich, Germany, 234--241.Google Scholar
- Devvi Sarwinda, Radifa Hilya Paradisa, Alhadi Bustamam, and Pinkie Anggia. 2021. Deep Learning in Image Classification using Residual Network (ResNet) Variants for Detection of Colorectal Cancer. Procedia Computer Science 179 (2021), 423--431.Google ScholarCross Ref
- Dinggang Shen, Guorong Wu, and Heung-Il Suk. 2017. Deep Learning in Medical Image Analysis. Annual review of biomedical engineering 19 (2017), 221--248.Google Scholar
- Nahian Siddique, Sidike Paheding, Colin P Elkin, and Vijay Devabhaktuni. 2021. U-net and its Variants for Medical Image Segmentation: A Review of Theory and Applications. IEEE Access 9 (2021), 82031--82057.Google ScholarCross Ref
- Khaoula Belhaj Soulami, Naima Kaabouch, Mohamed Nabil Saidi, and Ahmed Tamtaoui. 2021. Breast Cancer: One-Stage Automated Detection, Segmentation, and Classification of Digital Mammograms using UNet Model based-Semantic Segmentation. Biomedical Signal Processing and Control 66 (2021), 102481.Google ScholarCross Ref
- Yi-Sheng Sun, Zhao Zhao, Zhang-Nv Yang, Fang Xu, Hang-Jing Lu, Zhi-Yong Zhu, Wen Shi, Jianmin Jiang, Ping-Ping Yao, and Han-Ping Zhu. 2017. Risk Factors and Preventions of Breast Cancer. International journal of biological sciences 13, 11 (2017), 1387.Google ScholarCross Ref
- Hyuna Sung, Jacques Ferlay, Rebecca L Siegel, Mathieu Laversanne, Isabelle Soerjomataram, Ahmedin Jemal, and Freddie Bray. 2021. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA A Cancer Journal for Clinicians 71, 3 (2021), 209--249.Google ScholarCross Ref
- Bram Van Ginneken, Arnaud AA Setio, Colin Jacobs, and Francesco Ciompi. 2015. Off-the-shelf Convolutional Neural Network Features for Pulmonary Nodule Detection in Computed Tomography Scans. In 2015 IEEE 12th International symposium on biomedical imaging (ISBI). IEEE, Brooklyn, NY, USA, 286--289.Google Scholar
- Zhenqin Wu, Alexandro E Trevino, Eric Wu, Kyle Swanson, Honesty J Kim, H Blaize D'Angio, Ryan Preska, Gregory W Charville, Piero D Dalerba, Ann Marie Egloff, et al. 2022. Graph Deep Learning for the Characterization of Tumour Microenvironments from Spatial Protein Profiles in Tissue Specimens. Nature Biomedical Engineering 6, 12 (2022), 1435--1448.Google ScholarCross Ref
- Yin Xu, Yan Li, and Byeong-Seok Shin. 2020. Medical Image Processing with Contextual Style Transfer. Human-centric Computing and Information Sciences 10, 1 (2020), 1--16.Google ScholarDigital Library
- Qiao Zhang, Zhipeng Cui, Xiaoguang Niu, Shijie Geng, and Yu Qiao. 2017. Image Segmentation with Pyramid Dilated Convolution based on ResNet and U-Net. In Neural Information Processing: 24th International Conference, ICONIP 2017, Guangzhou, China, November 14-18, 2017, Proceedings, Part II 24. Springer, Guangzhou, China, 364--372.Google ScholarCross Ref
Index Terms
- Comprehensive Experiments on Breast Cancer Hematoxylin and Eosin-stained Images Using UNet
Recommendations
Classification of Hematoxylin and Eosin-Stained Breast Cancer Histology Microscopy Images Using Transfer Learning with EfficientNets
Breast cancer is a fatal disease and is a leading cause of death in women worldwide. The process of diagnosis based on biopsy tissue is nontrivial, time-consuming, and prone to human error, and there may be conflict about the final diagnosis due to ...
Detection of breast cancer via deep convolution neural networks using MRI images
AbstractBreast cancer is the type of cancer that develops from cells in the breast tissue. It is the leading cancer in women. Early detection of the breast cancer tumor is crucial in the treatment process. Mammography is a valuable tool for identifying ...
Segmentation of Microscopic Breast Cancer Images for Cancer Detection
ICSCA '19: Proceedings of the 2019 8th International Conference on Software and Computer ApplicationsBreast cancer is one of serious diseases that affect mainly woman and late diagnosis can lead to death. However early diagnosis increases survivability significantly, therefore making it very important. There are different diagnosis techniques for early ...
Comments