Skip to main content
SpringerLink
Log in

DRI-UNet: dense residual-inception UNet for nuclei identification in microscopy cell images

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Nuclei segmentation has great significance in biomedical applications as the preliminary step for disease diagnosis and treatment analysis. In this study, we propose a model for automated nuclei identification of varying cell shapes and types from microscopy images. Identifying nuclei helps to understand the underlying mechanism of various diseases in their early stages and provides solutions to enable faster cures. The foremost aim of the study is to develop a lightweight model, capable of segmenting varied shapes and sizes. The proposed architecture exploits multi-scale low-level features following dense high-level feature extraction with multi-feature fusion and special skip connections resulting in enhanced learning capability. The multi-scale feature extractor module extracts low-level information which is further processed using attention-based dense connections to extract semantically meaningful information. The special short-skip residual connections replacing long-skip connections reduced the semantic gap between encoder–decoder features. Moreover, the context encoder module extracts higher-level contextual information of different receptive fields using dilated convolutions making the model robust to different shapes and sizes. The higher-level feature maps propagate upward the decoder connections following the shared attention mechanism of an encoder to decoder features to reconstruct a better segmentation map. Moreover, the evaluation scheme following the proposed test-time augmentation operations improved the mean segmentation performance. The experiments on KDSB18, Synthetic cells, Triple-negative breast cancer (TNBC), MoNuSeg, CryoNuSeg, and BUS datasets demonstrate the suitability of the model for the nuclei segmentation tasks. The DRI-UNet model holds good segmentation performance outperforming baseline architecture by 8.12%, 4.71%, 10.19%, 2.46%, 3.14%, 8.91%, and 9.32% on KDSB18, synthetic cells, TNBC, MoNuSeg, CryoNuSeg, CVC-ClinicDB, and BUS datasets, respectively. We further conducted generalization tests of the proposed model for cross-dataset validation, and two independent MIS datasets confirm model effectiveness for nuclei cell and biomedical image segmentation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Data availability

The data and source code associated with the manuscript will be made available after the reasonable request.

Notes

  1. https://github.com/kannyjyk/Nested-UNet.

  2. https://github.com/ozan-oktay/Attention-Gated-Networks

  3. https://github.com/yingkaisha/keras-unet-collection

  4. https://github.com/DebeshJha/2020-CBMS-DoubleU-Net

  5. https://github.com/nspunn1993/Inception-U-Net-model

  6. https://github.com/NoviceMAn-prog/MSRF-Net

  7. https://github.com/nikhilroxtomar/FANet

References

  1. Anwar SM, Majid M, Qayyum A, Awais M, Alnowami M, Khan MK (2018) Medical image analysis using convolutional neural networks: a review. J Med Syst 42(11):1–13

    Article  Google Scholar 

  2. Mohapatra S, Patra D, Satpathy S (2014) An ensemble classifier system for early diagnosis of acute lymphoblastic leukemia in blood microscopic images. Neural Comput Appl 24(7):1887–1904

    Article  Google Scholar 

  3. Morid MA, Borjali A, Del Fiol G (2021) A scoping review of transfer learning research on medical image analysis using imagenet. Comput Biol Med 128:104115

    Article  Google Scholar 

  4. Al-Masni MA, Kim DH (2021) CMM-Net: contextual multi-scale multi-level network for efficient biomedical image segmentation. Sci Rep 11(1):1–18

    Article  Google Scholar 

  5. Caicedo JC, Goodman A, Karhohs KW, Cimini BA, Ackerman J, Haghighi M, Heng C, Becker T, Doan M, McQuin C, Rohban M (2019) Nucleus segmentation across imaging experiments: the 2018 data science bowl. Nat Methods 16(12):1247–1253

    Article  Google Scholar 

  6. Long F (2020) Microscopy cell nuclei segmentation with enhanced U-Net. BMC Bioinform 21(1):1–12

    Article  Google Scholar 

  7. Kaggle (2018) Kaggle data science bowl challenge—KDSB. Retrieved Jan 10, 2022, from https://www.kaggle.com/c/data-science-bowl-2018

  8. Krishnadas P, Chadaga K, Sampathila N, Rao S, Prabhu S (2022) Classification of malaria using object detection models. Informatics 9(4):76

    Article  Google Scholar 

  9. Punn N S, & Agarwal S (2020) Inception u-net architecture for semantic segmentation to identify nuclei in microscopy cell images. ACM Trans Multimed Comput Commun Appl (TOMM) 16(1): 1–15

  10. Zhang Z, Wu C, Coleman S, Kerr D (2020) DENSE-INception U-net for medical image segmentation. Comput Methods Progr Biomed 192:105395

    Article  Google Scholar 

  11. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, & Rabinovich A (2015) Going deeper with convolutions. In: proceedings of the IEEE conference on computer vision and pattern recognition (pp 1–9)

  12. Huang G, Liu Z, Van Der Maaten L, & Weinberger K Q (2017) Densely connected convolutional networks. In: proceedings of the IEEE conference on computer vision and pattern recognition (pp 4700–4708)

  13. He K, Zhang X, Ren S, & Sun J (2016) Deep residual learning for image recognition. In: proceedings of the IEEE conference on computer vision and pattern recognition (pp 770–778)

  14. Sharma A, Mishra PK (2022) Covid-MANet: multi-task attention network for explainable diagnosis and severity assessment of COVID-19 from CXR images. Pattern Recognit 1(131):108826

    Article  Google Scholar 

  15. Ibtehaz N, Rahman MS (2020) MultiResUNet: rethinking the U-Net architecture for multimodal biomedical image segmentation. Neural Netw 121:74–87

    Article  Google Scholar 

  16. Long J, Shelhamer E, & Darrell T (2015). Fully convolutional networks for semantic segmentation. In: proceedings of the IEEE conference on computer vision and pattern recognition (pp 3431–3440)

  17. Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495

    Article  Google Scholar 

  18. Ronneberger O, Fischer P, & Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: international conference on medical image computing and computer-assisted intervention (pp 234–241). Springer, Cham

  19. Oktay O, Schlemper J, Folgoc L L, Lee M, Heinrich M, Misawa K, & Rueckert D (2018) Attention u-net: Learning where to look for the pancreas. arXiv preprint arXiv:1804.03999

  20. Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J (2019) Unet++: Redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans Med Imaging 39(6):1856–1867

    Article  Google Scholar 

  21. Alom MZ, Yakopcic C, Hasan M, Taha TM, Asari VK (2019) Recurrent residual U-Net for medical image segmentation. J Med Imaging 6(1):014006

    Article  Google Scholar 

  22. Zhang Z, Liu Q, Wang Y (2018) Road extraction by deep residual u-net. IEEE Geosci Remote Sens Lett 15(5):749–753

    Article  Google Scholar 

  23. Lou A, Guan S, Loew M (2021) DC-UNet: rethinking the U-Net architecture with dual channel efficient CNN for medical image segmentation. In: medical imaging 2021: image processing 11596:758–768 SPIE

  24. Srivastava A, Jha D, Chanda S, Pal U, Johansen HD, Johansen D, Riegler MA, Ali S, Halvorsen P (2021) Msrf-net: a multi-scale residual fusion network for biomedical image segmentation. IEEE J Biomed Health Inform 26(5):2252–2263

    Article  Google Scholar 

  25. Tomar N K, Jha D, Riegler M A, Johansen H D, Johansen D, Rittscher J & Ali S (2022) Fanet: a feedback attention network for improved biomedical image segmentation. IEEE Trans Neural Net Learn Syst

  26. Lou A, Guan S, Loew M (2023) Cfpnet-m: a light-weight encoder-decoder based network for multimodal biomedical image real-time segmentation. Comput Biol Med 154:106579

    Article  Google Scholar 

  27. Suman S, Tiwari AK, Singh K (2023) Computer-aided diagnostic system for hypertensive retinopathy: a review. Comput Methods Programs Biomed 240:107627

  28. Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2017) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848

    Article  Google Scholar 

  29. He K, Zhang X, Ren S, Sun J (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37(9):1904–1916

    Article  Google Scholar 

  30. Zhao H, Shi J, Qi X, Wang X, & Jia J (2017) Pyramid scene parsing network. In: proceedings of the IEEE conference on computer vision and pattern recognition (pp 2881–2890)

  31. Milletari F, Navab N, & Ahmadi S A (2016) V-net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 fourth international conference on 3D vision (3DV) (pp 565–571). IEEE

  32. Diakogiannis FI, Waldner F, Caccetta P, Wu C (2020) ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed data. ISPRS J Photogramm Remote Sens 162:94–114

    Article  Google Scholar 

  33. Jha D, Smedsrud P H, Riegler M A, Johansen D, De Lange T, Halvorsen P & Johansen H D (2019) Resunet++: an advanced architecture for medical image segmentation. In: 2019 IEEE international symposium on multimedia (ISM) (pp 225–2255). IEEE

  34. Zeng Z, Xie W, Zhang Y, Lu Y (2019) RIC-Unet: an improved neural network based on Unet for nuclei segmentation in histology images. Ieee Access 7:21420–21428

    Article  Google Scholar 

  35. Zhang J, Jin Y, Xu J, Xu X, & Zhang Y (2018). Mdu-net: multi-scale densely connected u-net for biomedical image segmentation. arXiv preprint arXiv:1812.00352

  36. Jha D, Riegler M A, Johansen D, Halvorsen P, & Johansen H D (2020) Doubleu-net: a deep convolutional neural network for medical image segmentation. In: 2020 IEEE 33rd international symposium on computer-based medical systems (CBMS) (pp 558–564). IEEE

  37. Gu Z, Cheng J, Fu H, Zhou K, Hao H, Zhao Y, Zhang T, Gao S, Liu J (2019) Ce-net: context encoder network for 2d medical image segmentation. IEEE Trans Med Imaging 38(10):2281–2292

    Article  Google Scholar 

  38. Dolz J, Gopinath K, Yuan J, Lombaert H, Desrosiers C, Ayed IB (2018) HyperDense-Net: a hyper-densely connected CNN for multi-modal image segmentation. IEEE Trans Med Imaging 38(5):1116–1126

    Article  Google Scholar 

  39. Punn NS, Agarwal S (2022) RCA-IUnet: a residual cross-spatial attention-guided inception U-Net model for tumor segmentation in breast ultrasound imaging. Mach Vis Appl 33(2):1–10

    Article  Google Scholar 

  40. Yu F, Wang D, Shelhamer E, & Darrell T (2018) Deep layer aggregation. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp 2403–2412)

  41. Yadavendra, Chand S (2022) Semantic segmentation of human cell nucleus using deep U-Net and other versions of U-Net models. Net Comput Neural Syst 1–20

  42. Azad R, Asadi-Aghbolaghi M, Fathy M, & Escalera S (2019) Bi-directional ConvLSTM U-Net with densley connected convolutions. In: proceedings of the IEEE/CVF international conference on computer vision workshops (pp 0-0)

  43. Gudhe NR, Behravan H, Sudah M, Okuma H, Vanninen R, Kosma VM, Mannermaa A (2021) Multi-level dilated residual network for biomedical image segmentation. Sci Rep 11(1):14105

    Article  Google Scholar 

  44. Ashraf H, Waris A, Ghafoor MF, Gilani SO, Niazi IK (2022) Melanoma segmentation using deep learning with test-time augmentations and conditional random fields. Sci Rep 12(1):1–16

    Article  Google Scholar 

  45. Moshkov N, Mathe B, Kertesz-Farkas A, Hollandi R, Horvath P (2020) Test-time augmentation for deep learning-based cell segmentation on microscopy images. Sci Rep 10(1):1–7

    Article  Google Scholar 

  46. Senapati MR, Panda G, Dash PK (2014) Hybrid approach using KPSO and RLS for RBFNN design for breast cancer detection. Neural Comput Appl 24(3):745–753

    Article  Google Scholar 

  47. Chadaga K, Prabhu S, Bhat KV, Umakanth S, Sampathila N (2022) Medical diagnosis of COVID-19 using blood tests and machine learning. J Phys Conf Ser 2161(1):012017

    Article  Google Scholar 

  48. Sharma A, Mishra PK (2022) Image enhancement techniques on deep learning approaches for automated diagnosis of COVID-19 features using CXR images. Multimed Tools Appl 81(29):1–42

    Article  Google Scholar 

  49. Lehmussola A, Ruusuvuori P, Selinummi J, Huttunen H, Yli-Harja O (2007) Computational framework for simulating fluorescence microscope images with cell populations. IEEE Trans Med Imaging 26(7):1010–1016

    Article  Google Scholar 

  50. Kaur A, Kaur L, Singh A (2021) GA-UNet: UNet-based framework for segmentation of 2D and 3D medical images applicable on heterogeneous datasets. Neural Comput Appl 33(21):14991–15025

    Article  Google Scholar 

  51. Hubel DH, Wiesel TN (1962) Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J Physiol 160(1):106

    Article  Google Scholar 

  52. Luo P, Ren J, Peng Z, Zhang R, & Li J (2018) Differentiable learning-to-normalize via switchable normalization. arXiv preprint arXiv:1806.10779

  53. Kolařík M, Burget R, Uher V, Říha K, Dutta MK (2019) Optimized high resolution 3d dense-u-net network for brain and spine segmentation. Appl Sci 9(3):404

    Article  Google Scholar 

  54. Karim M, Rahman A, Jares JB, Decker S, Beyan O (2020) A snapshot neural ensemble method for cancer-type prediction based on copy number variations. Neural Comput Appl 32(19):15281–15299

    Article  Google Scholar 

  55. Naylor P, Laé M, Reyal F, Walter T (2018) Segmentation of nuclei in histopathology images by deep regression of the distance map. IEEE Trans Med Imaging 38(2):448–459

    Article  Google Scholar 

  56. Mahbod A, Schaefer G, Bancher B, Löw C, Dorffner G, Ecker R, Ellinger I (2021) CryoNuSeg: a dataset for nuclei instance segmentation of cryosectioned H&E-stained histological images. Comput Biol Med 132:104349

    Article  Google Scholar 

  57. Kumar N, Verma R, Anand D, Zhou Y, Onder OF, Tsougenis E, Chen H, Heng PA, Li J, Hu Z, Wang Y (2019) A multi-organ nucleus segmentation challenge. IEEE Trans Med Imaging 39(5):1380–1391

    Article  Google Scholar 

  58. Sharma A, Mishra PK (2021) Performance analysis of machine learning based optimized feature selection approaches for breast cancer diagnosis. Int J Inf Tech 1–12

  59. Bernal J, Sánchez FJ, Fernández-Esparrach G, Gil D, Rodríguez C, Vilariño F (2015) WM-DOVA maps for accurate polyp highlighting in colonoscopy: validation vs. saliency maps from physicians. Comput Med Imaging Gr 43:99–111

    Article  Google Scholar 

Download references

Acknowledgements

We show our sincere gratitude to the University Grant Commission, Government of India, for supporting with a Senior Research Fellowship. The corresponding author greatly acknowledges the IoE grant of Banaras Hindu University.

Author information

Authors and Affiliations

  1. Department of Computer Science, Institute of Science, Banaras Hindu University, Varanasi, 221005, India

    Ajay Sharma & Pramod Kumar Mishra

Authors
  1. Ajay Sharma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pramod Kumar Mishra
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pramod Kumar Mishra.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sharma, A., Mishra, P.K. DRI-UNet: dense residual-inception UNet for nuclei identification in microscopy cell images. Neural Comput & Applic 35, 19187–19220 (2023). https://doi.org/10.1007/s00521-023-08729-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-08729-0

Keywords

Search

Navigation