skip to main content
10.1145/3603287.3651207acmconferencesArticle/Chapter ViewAbstractPublication Pagesacm-seConference Proceedingsconference-collections
research-article
Open Access
Best Paper

Comprehensive Experiments on Breast Cancer Hematoxylin and Eosin-stained Images Using UNet

Published:27 April 2024Publication History

ABSTRACT

The current challenge in breast cancer segmentation on hematoxylin and eosin-stained images lies in accurately capturing histologic phenotypes associated with cancer biomarkers. Many researchers have studied histologic patterns of cancerous regions of breast cancer images for a better understanding of diagnosis and treatment, yet it is not fully investigated because of its variability, complexity, and large data volumes. Comprehensive experiments on breast cancer segmentation can address this challenge by identifying heterogeneous cell regions in the tumor microenvironment and investigating a new methodology of segmenting histologic images by using advanced deep learning architectures. In this paper, we present findings in three experiments on breast cancer segmentation, exploring the effectiveness of a convolutional neural network called UNet identifying multiple heterogeneous cells such as blood, blood vessels, fat, glandular secretions, necrosis, and plasma cells in a tumor microenvironment, investigating the performance of UNet architecture in different sizes of cancerous regions such as a tumor, and tumor-infiltrating lymphocytes, and proposing a new methodology of a neural-style guided data augmentation focusing on the image segmentation of breast tumor-related nuclei in hematoxylin and eosin-stained images. The experiment results show that a modified UNet performs well on fat identification. Another experiment shows that the smaller input size of the tumor, stroma, and tumor-infiltrating lymphocyte provides better performance than the larger input size. Moreover, we demonstrate that the proposed method outperforms the traditional deep neural network models.

References

  1. Ayat Abedalla, Malak Abdullah, Mahmoud Al-Ayyoub, and Elhadj Benkhelifa. 2020. The 2ST-UNet for Pneumothorax Segmentation in Chest X-Rays using ResNet34 as a Backbone for U-Net. arXiv preprint arXiv:2009.02805 (2020).Google ScholarGoogle Scholar
  2. Mohamed Amgad, Habiba Elfandy, Hagar Hussein, Lamees A Atteya, Mai AT Elsebaie, Lamia S Abo Elnasr, Rokia A Sakr, Hazem SE Salem, Ahmed F Ismail, Anas M Saad, et al. 2019. Structured Crowdsourcing Enables Convolutional Segmentation of Histology Images. Bioinformatics 35, 18 (2019), 3461--3467.Google ScholarGoogle ScholarCross RefCross Ref
  3. Vickie E Baracos, Tony Reiman, Marina Mourtzakis, Ioannis Gioulbasanis, and Sami Antoun. 2010. Body Composition in Patients with Non-Small Cell Lung Cancer: a Contemporary View of Cancer Cachexia with the Use of Computed Tomography Image Analysis. The American journal of clinical nutrition 91, 4 (2010), 1133S-1137S.Google ScholarGoogle Scholar
  4. Bhupender S Chhikara and Keykavous Parang. 2023. Global Cancer Statistics 2022: the Trends Projection Analysis. Chemical Biology Letters 10, 1 (2023), 451--451.Google ScholarGoogle Scholar
  5. Antonia Creswell, Tom White, Vincent Dumoulin, Kai Arulkumaran, Biswa Sengupta, and Anil A Bharath. 2018. Generative Adversarial Networks: An Overview. IEEE signal processing magazine 35, 1 (2018), 53--65.Google ScholarGoogle Scholar
  6. Angel Cruz-Roa, Hannah Gilmore, Ajay Basavanhally, Michael Feldman, Shridar Ganesan, Natalie NC Shih, John Tomaszewski, Fabio A González, and Anant Madabhushi. 2017. Accurate and Reproducible Invasive Breast Cancer Detection in Whole-Slide Images: A Deep Learning Approach for Quantifying Tumor Extent. Scientific reports 7, 1 (2017), 46450.Google ScholarGoogle Scholar
  7. Yuxin Cui, Guiying Zhang, Zhonghao Liu, Zheng Xiong, and Jianjun Hu. 2019. A Deep Learning Algorithm for One-Step Contour Aware Nuclei Segmentation of Histopathology Images. Medical & biological engineering & computing 57 (2019), 2027--2043.Google ScholarGoogle Scholar
  8. Sergio Damas, Oscar Cordón, and Jose Santamaría. 2011. Medical Image Registration using Evolutionary Computation: An Experimental Survey. IEEE Computational Intelligence Magazine 6, 4 (2011), 26--42.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Andre Esteva, Katherine Chou, Serena Yeung, Nikhil Naik, Ali Madani, Ali Mottaghi, Yun Liu, Eric Topol, Jeff Dean, and Richard Socher. 2021. Deep Learning-Enabled Medical Computer Vision. NPJ digital medicine 4, 1 (2021), 5.Google ScholarGoogle Scholar
  10. Arthur Francisco Araújo Fernandes, João Ricardo Rebouças Dórea, and Guilherme Jordão de Magalhães Rosa. 2020. Image Analysis and Computer Vision Applications in Animal Sciences: an Overview. Frontiers in Veterinary Science 7 (2020), 551269.Google ScholarGoogle ScholarCross RefCross Ref
  11. Sourodip Ghosh, Aunkit Chaki, and KC Santosh. 2021. Improved U-Net Architecture with VGG-16 for Brain Tumor Segmentation. Physical and Engineering Sciences in Medicine 44, 3 (2021), 703--712.Google ScholarGoogle ScholarCross RefCross Ref
  12. Intisar Rizwan I Haque and Jeremiah Neubert. 2020. Deep Learning Approaches to Biomedical Image Segmentation. Informatics in Medicine Unlocked 18 (2020), 100297.Google ScholarGoogle ScholarCross RefCross Ref
  13. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. Las Vegas, NV, USA, 770--778.Google ScholarGoogle ScholarCross RefCross Ref
  14. Andreas Heinrich, Felix V Güttler, Sebastian Schenkl, Rebecca Wagner, and Ulf K-M Teichgräber. 2020. Automatic Human Identification based on Dental X-ray Radiographs using Computer Vision. Scientific reports 10, 1 (2020), 3801.Google ScholarGoogle Scholar
  15. Kirsi Holli, Anna-Leena Lääperi, Lara Harrison, Tiina Luukkaala, Terttu Toivonen, Pertti Ryymin, Prasun Dastidar, Seppo Soimakallio, and Hannu Eskola. 2010. Characterization of Breast Cancer Types by Texture Analysis of Magnetic Resonance Images. Academic radiology 17, 2 (2010), 135--141.Google ScholarGoogle ScholarCross RefCross Ref
  16. Minui Hong, Jinwoo Choi, and Gunhee Kim. 2021. Stylemix: Separating Content and Style for Enhanced Data Augmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. Nashville, TN, USA, 14862--14870.Google ScholarGoogle ScholarCross RefCross Ref
  17. Le Hou, Vu Nguyen, Ariel B Kanevsky, Dimitris Samaras, Tahsin M Kurc, Tianhao Zhao, Rajarsi R Gupta, Yi Gao, Wenjin Chen, David Foran, et al. 2019. Sparse Autoencoder for Unsupervised Nucleus Detection and Representation in Histopathology Images. Pattern recognition 86 (2019), 188--200.Google ScholarGoogle Scholar
  18. Aiyue Huang, Li Jiang, Jiangshan Zhang, and Qing Wang. 2022. Attention-VGG16-UNet: a Novel Deep Learning Approach for Automatic Segmentation of the Median Nerve in Ultrasound Images. Quantitative Imaging in Medicine and Surgery 12, 6 (2022), 3138.Google ScholarGoogle ScholarCross RefCross Ref
  19. Philip TG Jackson, Amir Atapour Abarghouei, Stephen Bonner, Toby P Breckon, and Boguslaw Obara. 2019. Style Augmentation: Data Augmentation via Style Randomization. In CVPR workshops, Vol. 6. Long Beach, CA, USA, 10--11.Google ScholarGoogle Scholar
  20. Varun Jampani, Ujjwal, Jayanthi Sivaswamy, and Vivek Vaidya. 2012. Assessment of Computational Visual Attention Models on Medical Images. In Proceedings of the Eighth Indian Conference on Computer Vision, Graphics and Image Processing. Mumbai, India, 1--8.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Neeraj Kumar, Ruchika Verma, Sanuj Sharma, Surabhi Bhargava, Abhishek Vahadane, and Amit Sethi. 2017. A Dataset and a Technique for Generalized Nuclear Segmentation for Computational Pathology. IEEE transactions on medical imaging 36, 7 (2017), 1550--1560.Google ScholarGoogle ScholarCross RefCross Ref
  22. Sanghoon Lee, Mohamed Amgad, Pooya Mobadersany, Matt McCormick, Brian P Pollack, Habiba Elfandy, Hagar Hussein, David A Gutman, and Lee AD Cooper. 2021. Interactive Classification of Whole-Slide Imaging Data for Cancer Researchers. Cancer research 81, 4 (2021), 1171--1177.Google ScholarGoogle Scholar
  23. Sanghoon Lee, Sunho Park, Jae-Ho Cheong, Sam C Wang, Matthew R Porembka, In-Ho Kim, Sung Hak Lee, and Tae Hyun Hwang. 2023. Spatial Image Analysis of Tumor-Infiltrating Lymphocytes on Gastric Cancer to Predict Anti-PD1 Inhibitor Response. Cancer Research 83, 7_Supplement (2023), 1360--1360.Google ScholarGoogle Scholar
  24. Shuying Liu and Weihong Deng. 2015. Very Deep Convolutional Neural Network based Image Classification using Small Training Sample Size. In 2015 3rd IAPR Asian conference on pattern recognition (ACPR). IEEE, Kuala Lumpur, Malaysia, 730--734.Google ScholarGoogle ScholarCross RefCross Ref
  25. Anant Madabhushi and George Lee. 2016. Image Analysis and Machine Learning in Digital Pathology: Challenges and Opportunities. Medical image analysis 33 (2016), 170--175.Google ScholarGoogle Scholar
  26. Domingo Mery, Erick Svec, Marco Arias, Vladimir Riffo, Jose M Saavedra, and Sandipan Banerjee. 2016. Modern Computer Vision Techniques for X-ray Testing in Baggage Inspection. IEEE Transactions on Systems, Man, and Cybernetics: Systems 47, 4 (2016), 682--692.Google ScholarGoogle ScholarCross RefCross Ref
  27. Michael Nalisnik, Mohamed Amgad, Sanghoon Lee, Sameer H Halani, Jose Enrique Velazquez Vega, Daniel J Brat, David A Gutman, and Lee AD Cooper. 2017. Interactive Phenotyping of Large-scale Histology Imaging Data with HistomicsML. Scientific reports 7, 1 (2017), 14588.Google ScholarGoogle Scholar
  28. Peter Naylor, Marick Laé, Fabien Reyal, and Thomas Walter. 2018. Segmentation of Nuclei in Histopathology Images by Deep Regression of the Distance Map. IEEE transactions on medical imaging 38, 2 (2018), 448--459.Google ScholarGoogle Scholar
  29. Ozan Oktay, Jo Schlemper, Loic Le Folgoc, Matthew Lee, Mattias Heinrich, Kazunari Misawa, Kensaku Mori, Steven McDonagh, Nils Y Hammerla, Bernhard Kainz, et al. 2018. Attention U-net: Learning where to Look for the Pancreas. arXiv preprint arXiv:1804.03999 (2018).Google ScholarGoogle Scholar
  30. Anindya Apriliyanti Pravitasari, Nur Iriawan, Mawanda Almuhayar, Taufik Azmi, Irhamah Irhamah, Kartika Fithriasari, Santi Wulan Purnami, and Widiana Ferriastuti. 2020. UNet-VGG16 with Transfer Learning for MRI-based Brain Tumor Segmentation. TELKOMNIKA (Telecommunication Computing Electronics and Control) 18, 3 (2020), 1310--1318.Google ScholarGoogle ScholarCross RefCross Ref
  31. Anindya Apriliyanti Pravitasari, Nur Iriawan, Ulfa Siti Nuraini, and Dwilaksana Abdullah Rasyid. 2022. On Comparing Optimizer of UNet-VGG16 Architecture for Brain Tumor Image Segmentation. In Brain Tumor MRI Image Segmentation Using Deep Learning Techniques, Jyotismita Chaki (Ed.). Academic Press, 197--215.Google ScholarGoogle Scholar
  32. Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. Springer, Munich, Germany, 234--241.Google ScholarGoogle Scholar
  33. Devvi Sarwinda, Radifa Hilya Paradisa, Alhadi Bustamam, and Pinkie Anggia. 2021. Deep Learning in Image Classification using Residual Network (ResNet) Variants for Detection of Colorectal Cancer. Procedia Computer Science 179 (2021), 423--431.Google ScholarGoogle ScholarCross RefCross Ref
  34. Dinggang Shen, Guorong Wu, and Heung-Il Suk. 2017. Deep Learning in Medical Image Analysis. Annual review of biomedical engineering 19 (2017), 221--248.Google ScholarGoogle Scholar
  35. Nahian Siddique, Sidike Paheding, Colin P Elkin, and Vijay Devabhaktuni. 2021. U-net and its Variants for Medical Image Segmentation: A Review of Theory and Applications. IEEE Access 9 (2021), 82031--82057.Google ScholarGoogle ScholarCross RefCross Ref
  36. Khaoula Belhaj Soulami, Naima Kaabouch, Mohamed Nabil Saidi, and Ahmed Tamtaoui. 2021. Breast Cancer: One-Stage Automated Detection, Segmentation, and Classification of Digital Mammograms using UNet Model based-Semantic Segmentation. Biomedical Signal Processing and Control 66 (2021), 102481.Google ScholarGoogle ScholarCross RefCross Ref
  37. Yi-Sheng Sun, Zhao Zhao, Zhang-Nv Yang, Fang Xu, Hang-Jing Lu, Zhi-Yong Zhu, Wen Shi, Jianmin Jiang, Ping-Ping Yao, and Han-Ping Zhu. 2017. Risk Factors and Preventions of Breast Cancer. International journal of biological sciences 13, 11 (2017), 1387.Google ScholarGoogle ScholarCross RefCross Ref
  38. Hyuna Sung, Jacques Ferlay, Rebecca L Siegel, Mathieu Laversanne, Isabelle Soerjomataram, Ahmedin Jemal, and Freddie Bray. 2021. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA A Cancer Journal for Clinicians 71, 3 (2021), 209--249.Google ScholarGoogle ScholarCross RefCross Ref
  39. Bram Van Ginneken, Arnaud AA Setio, Colin Jacobs, and Francesco Ciompi. 2015. Off-the-shelf Convolutional Neural Network Features for Pulmonary Nodule Detection in Computed Tomography Scans. In 2015 IEEE 12th International symposium on biomedical imaging (ISBI). IEEE, Brooklyn, NY, USA, 286--289.Google ScholarGoogle Scholar
  40. Zhenqin Wu, Alexandro E Trevino, Eric Wu, Kyle Swanson, Honesty J Kim, H Blaize D'Angio, Ryan Preska, Gregory W Charville, Piero D Dalerba, Ann Marie Egloff, et al. 2022. Graph Deep Learning for the Characterization of Tumour Microenvironments from Spatial Protein Profiles in Tissue Specimens. Nature Biomedical Engineering 6, 12 (2022), 1435--1448.Google ScholarGoogle ScholarCross RefCross Ref
  41. Yin Xu, Yan Li, and Byeong-Seok Shin. 2020. Medical Image Processing with Contextual Style Transfer. Human-centric Computing and Information Sciences 10, 1 (2020), 1--16.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Qiao Zhang, Zhipeng Cui, Xiaoguang Niu, Shijie Geng, and Yu Qiao. 2017. Image Segmentation with Pyramid Dilated Convolution based on ResNet and U-Net. In Neural Information Processing: 24th International Conference, ICONIP 2017, Guangzhou, China, November 14-18, 2017, Proceedings, Part II 24. Springer, Guangzhou, China, 364--372.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Comprehensive Experiments on Breast Cancer Hematoxylin and Eosin-stained Images Using UNet

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      ACM SE '24: Proceedings of the 2024 ACM Southeast Conference
      April 2024
      337 pages
      ISBN:9798400702372
      DOI:10.1145/3603287

      Copyright © 2024 Owner/Author

      This work is licensed under a Creative Commons Attribution International 4.0 License.

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 27 April 2024

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed limited

      Acceptance Rates

      ACM SE '24 Paper Acceptance Rate44of137submissions,32%Overall Acceptance Rate178of377submissions,47%
    • Article Metrics

      • Downloads (Last 12 months)31
      • Downloads (Last 6 weeks)31

      Other Metrics

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader