Skip to main content

Advertisement

Springer Nature Link
Log in

Macaque neuron instance segmentation only with point annotations based on multiscale fully convolutional regression neural network

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In the field of biomedicine, instance segmentation / individualization is important in analyzing the number, the morphology and the distribution of neurons for the whole slide images. Traditionally, biologists apply the stereology technique to manually count the number of neurons in the regions of interest and estimate the number in anatomical regions or the entire brain. This is very tedious and time-consuming. In this paper, we propose a multiscale fully convolutional regression neural network combined with a competitive region growing technique to individualize size-varying and touching neurons in the major anatomical regions of the macaque brain. Given that neuron instance or contour annotations are infeasible to obtain in certain regions, such as the dentate gyrus where thousands of touching neurons are present, we ask an expert to perform point annotations in the center location of neurons (noted as neuron centroids) for training. Thanks to the multiscale resolution achieved by parallel multiple receptive fields and different network depths, our proposed network succeeds in detecting the centroids of size-varying and touching neurons. Competitive region growing is then applied on these centroids to achieve neuron instance segmentation. Experiments on the macaque brain data suggest that our proposed method outperform the state-of-the-art methods in terms of neuron instance segmentation performance. To our knowledge, this is the first deep learning research work to individualize size-varying and touching neurons only using point annotations in major anatomical regions of the macaque brain.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

Data availability

The datasets applied in this paper come from MIRCen CEA France (Molecular Imaging Research Center, The French Alternative Energies and Atomic Energy Commission). The data will be available later in the form of an article and an access because it corresponds to a large amount of information.

References

  1. Adams R, Bischof L (1994) Seeded region growing. IEEE Trans Pattern Anal Mach Intell 16:641–647. https://doi.org/10.1109/34.295913

    Article  Google Scholar 

  2. Al-Kofahi Y, Lassoued W, Lee W, Roysam B (2010) Improved automatic detection and segmentation of cell nuclei in histopathology images. IEEE Trans Biomed Eng 57:841–852. https://doi.org/10.1109/TBME.2009.2035102

    Article  Google Scholar 

  3. Andersen P, Morris R, Amaral D, Bliss T, O’Keefe J (2006) The Hippocampus Book. Oxford University Press, England

    Book  Google Scholar 

  4. Andrey P, Kiêu K, Kress C, Lehmann G, Tirichine L, Liu Z, Biot E, Adenot P-G, Hue-Beauvais C, Houba-Hérin N, Duranthon V, Devinoy E, Beaujean N, Gaudin V, Maurin Y, Debey P (2010) Statistical analysis of 3D images detects regular spatial distributions of centromeres and chromocenters in animal and plant nuclei. PLOS Comput Biol 6:e1000853. https://doi.org/10.1371/journal.pcbi.1000853

    Article  MathSciNet  Google Scholar 

  5. Bai X, Sun C, Zhou F (2008) Touching Cells Splitting by Using Concave Points and Ellipse Fitting, in: Digital Image Computing: Techniques and Applications (DICTA) Presented at the Digital Image Computing: Techniques and Applications (DICTA), https://doi.org/10.1109/DICTA.2008.11

  6. Brabandere BD, Neven D, Gool LV (2017) Semantic Instance Segmentation for Autonomous Driving, In: 2017 IEEE conference on computer vision and pattern recognition workshops (CVPRW). Presented at the 2017 IEEE Conference on computer vision and pattern recognition workshops (CVPRW), pp. 478–480. https://doi.org/10.1109/CVPRW.2017.66

  7. Cheng HD, Jiang XH, Sun Y, Wang JL (2001) Color image segmentation: advances and prospects. Pattern Recognit 34:2259–2281

    Article  Google Scholar 

  8. Cousty J, Bertrand G, Najman L, Couprie M (2009) Watershed cuts: minimum spanning forests and the drop of water principle. IEEE Trans Pattern Anal Mach Intell 31:1362–1374. https://doi.org/10.1109/TPAMI.2008.173

    Article  Google Scholar 

  9. Daněk O, Matula P, Ortiz-de-Solórzano C, Muñoz-Barrutia A, Maška M, Kozubek M (2009) Segmentation of Touching Cell Nuclei Using a Two-Stage Graph Cut Model, in: Image Analysis, Lecture Notes in Computer Science. Presented at the Scandinavian Conference on Image Analysis, Springer, Berlin, Heidelberg, pp. 410–419. https://doi.org/10.1007/978-3-642-02230-2_42

  10. Falk T, Mai D, Bensch R, Çiçek Ö, Abdulkadir A, Marrakchi Y, Böhm A, Deubner J, Jäckel Z, Seiwald K, Dovzhenko A, Tietz O, Dal Bosco C, Walsh S, Saltukoglu D, Tay TL, Prinz M, Palme K, Simons M, Diester I, Brox T, Ronneberger O (2019) U-Net: deep learning for cell counting, detection, and morphometry. Nat Methods 16:67–70. https://doi.org/10.1038/s41592-018-0261-2

    Article  Google Scholar 

  11. Gundersen HJ (1986) Stereology of arbitrary particles. A review of unbiased number and size estimators and the presentation of some new ones, in memory of William R. Thompson J Microsc 143:3–45

    Article  Google Scholar 

  12. He K, Zhang X, Ren S, Sun J (2016) Deep Residual Learning for Image Recognition, in: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Presented at the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778. https://doi.org/10.1109/CVPR.2016.90

  13. He Y, Gong H, Xiong B, Xu X, Li A, Jiang T, Sun Q, Wang S, Luo Q, Chen S (2015) iCut: an integrative cut algorithm enables accurate segmentation of touching cells. Sci Rep. https://doi.org/10.1038/srep12089

    Article  Google Scholar 

  14. Huang Y, Zhu H, Wang P, Dong D (2019) Segmentation of Overlapping Cervical Smear Cells Based on U-Net and Improved Level Set, In: 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC). Presented at the 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), pp. 3031–3035. https://doi.org/10.1109/SMC.2019.8914625

  15. Jelsing J, Nielsen R, Olsen AK, Grand N, Hemmingsen R, Pakkenberg B (2006) The postnatal development of neocortical neurons and glial cells in the Göttingen minipig and the domestic pig brain. J Exp Biol 209:1454–1462. https://doi.org/10.1242/jeb.02141

    Article  Google Scholar 

  16. Kainz P, Urschler M, Schulter S, Wohlhart P, Lepetit V (2015) You should use regression to detect cells, in: medical image computing and computer-assisted intervention–MICCAI 2015, Lecture notes in computer science. presented at the international conference on medical image computing and computer-assisted intervention, Springer, Cham, pp. 276–283. https://doi.org/10.1007/978-3-319-24574-4_33

  17. Karlsen AS, Pakkenberg B (2011) Total numbers of neurons and glial cells in cortex and basal ganglia of aged brains with Down syndrome–a stereological study. Cereb Cortex N Y N 1991(21):2519–2524. https://doi.org/10.1093/cercor/bhr033

    Article  Google Scholar 

  18. Kothari S, Chaudry Q, Wang MD (2009) Automated cell counting and cluster segmentation using concavity detection and ellipse fitting techniques, In: IEEE International symposium on biomedical imaging: from nano to macro, 2009. ISBI ’09. Presented at the IEEE international symposium on biomedical imaging: From Nano to Macro, 2009. ISBI ’09, pp. 795–798. https://doi.org/10.1109/ISBI.2009.5193169

  19. Larsen CC, Bonde Larsen K, Bogdanovic N, Laursen H, Graem N, Samuelsen GB, Pakkenberg B (2006) Total number of cells in the human newborn telencephalic wall. Neuroscience 139:999–1003. https://doi.org/10.1016/j.neuroscience.2006.01.005

    Article  Google Scholar 

  20. Li C, Wang X, Liu W, Latecki LJ, Wang B, Huang J (2019) Weakly supervised mitosis detection in breast histopathology images using concentric loss. Med Image Anal 53:165–178. https://doi.org/10.1016/j.media.2019.01.013

    Article  Google Scholar 

  21. Lin TY, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection, in: 2017 ieee conference on computer vision and pattern recognition (CVPR). Presented at the 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp. 936–944. https://doi.org/10.1109/CVPR.2017.106

  22. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation, In: 2015 IEEE conference on computer vision and pattern recognition (CVPR). Presented at the 2015 IEEE conference on computer vision and pattern recognition (CVPR), pp. 3431–3440. https://doi.org/10.1109/CVPR.2015.7298965

  23. Lou X, Koethe U, Wittbrodt J, Hamprecht FA (2012) Learning to segment dense cell nuclei with shape prior, in: 2012 IEEE conference on computer vision and pattern recognition. Presented at the 2012 IEEE conference on computer vision and pattern recognition, pp. 1012–1018. https://doi.org/10.1109/CVPR.2012.6247778

  24. Ma, H., Beiter, R., Gaultier, A., Acton, S.T., Lin, Z., 2018. OSLO: Automatic Cell Counting and Segmentation for Oligodendrocyte Progenitor Cells, in: 2018 25th IEEE International Conference on Image Processing (ICIP). Presented at the 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 2431–2435. https://doi.org/10.1109/ICIP.2018.8451410

  25. Mullen RJ, Buck CR, Smith AM (1992) NeuN, a neuronal specific nuclear protein in vertebrates. Dev Camb Engl 116:201–211

    Google Scholar 

  26. Naylor P, Laé M, Reyal F, Walter T (2019) Segmentation of Nuclei in Histopathology Images by Deep Regression of the Distance Map. IEEE Trans Med Imaging 38:448–459. https://doi.org/10.1109/TMI.2018.2865709

    Article  Google Scholar 

  27. Nedzved A, Ablameyko S, Pitas I (2000) Morphological segmentation of histology cell images, In: 15th International conference on pattern recognition. pp. 500–503. https://doi.org/10.1109/ICPR.2000.905385

  28. Pakkenberg B, Gundersen HJ (1997) Neocortical neuron number in humans: effect of sex and age. J Comp Neurol 384:312–320

    Article  Google Scholar 

  29. Passingham R (2009) How good is the macaque monkey model of the human brain? Curr Opin Neurobiol 19:6–11. https://doi.org/10.1016/j.conb.2009.01.002

    Article  Google Scholar 

  30. Pelvig DP, Pakkenberg H, Stark AK, Pakkenberg B (2008) Neocortical glial cell numbers in human brains. Neurobiol Aging 29:1754–1762. https://doi.org/10.1016/j.neurobiolaging.2007.04.013

    Article  Google Scholar 

  31. Qi J (2014) Dense nuclei segmentation based on graph cut and convexity-concavity analysis. J Microsc 253:42–53. https://doi.org/10.1111/jmi.12096

    Article  Google Scholar 

  32. Rad RM, Saeedi P, Au J, Havelock J (2018) Multi-resolutional ensemble of stacked dilated U-net for inner cell mass segmentation in human embryonic images, In: 2018 25th IEEE International conference on image processing (ICIP). Presented at the 2018 25th IEEE International conference on image processing (ICIP), pp. 3518–3522. https://doi.org/10.1109/ICIP.2018.8451750

  33. Raza SEA, AbdulJabbar K, Jamal-Hanjani M, Veeriah S, Quesne JL, Swanton C, Yuan Y 2019 Deconvolving convolutional neural network for cell detection, In: 2019 IEEE 16th International symposium on biomedical imaging (ISBI 2019). Presented at the 2019 IEEE 16th International symposium on biomedical imaging (ISBI 2019), pp. 891–894. https://doi.org/10.1109/ISBI.2019.8759333

  34. Renier N, Wu Z, Simon DJ, Yang J, Ariel P, Tessier-Lavigne M (2014) iDISCO: a simple, rapid method to immunolabel large tissue samples for volume imaging. Cell 159:896–910. https://doi.org/10.1016/j.cell.2014.10.010

    Article  Google Scholar 

  35. Riccio D, Brancati N, Frucci M, Gragnaniello D (2019) A new unsupervised approach for segmenting and counting cells in high-throughput microscopy image sets. IEEE J Biomed Health Inform 23:437–448. https://doi.org/10.1109/JBHI.2018.2817485

    Article  Google Scholar 

  36. Ronneberger O, Fischer P, Brox T (2015) U-Net Convolutional Networks for Biomedical Image Segmentation. In: Navab N, Hornegger J, Wells WM, Frangi AF (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Springer, Berlin

    Google Scholar 

  37. Shu J, Fu H, Qiu G, Kaye P, Ilyas M (2013) Segmenting overlapping cell nuclei in digital histopathology images. Conf Proc Annu Int Conf IEEE Eng. Med. Biol. Soc. IEEE Eng. Med. Biol. Soc. Annu. Conf. 2013, 5445–5448. https://doi.org/10.1109/EMBC.2013.6610781

  38. Sofiiuk K, Barinova O, Konushin A (2019) AdaptIS: Adaptive instance selection network, In: 2019 IEEE international conference on computer vision (ICCV). Presented at the 2019 IEEE international conference on computer vision (ICCV), pp. 7355–7363.

  39. Thu DC, Oorschot DE, Tippett LJ, Nana AL, Hogg VM, Synek BJ, Luthicarter R, Waldvogel HJ, Faull RL (2010) Cell loss in the motor and cingulate cortex correlates with symptomatology in Huntington’s disease. Brain J Neurol 133:1094–1110

    Article  Google Scholar 

  40. Ulman V, Maška M, Magnusson KEG, Ronneberger O, Haubold C, Harder N, Matula P, Matula P, Svoboda D, Radojevic M, Smal I, Rohr K, Jaldén J, Blau HM, Dzyubachyk O, Lelieveldt B, Xiao P, Li Y, Cho S-Y, Dufour AC, Olivo-Marin J-C, Reyes-Aldasoro CC, Solis-Lemus JA, Bensch R, Brox T, Stegmaier J, Mikut R, Wolf S, Hamprecht FA, Esteves T, Quelhas P, Demirel Ö, Malmström L, Jug F, Tomancak P, Meijering E, Muñoz-Barrutia A, Kozubek M, Ortiz-de-Solorzano C (2017) An objective comparison of cell-tracking algorithms. Nat Methods 14:1141–1152. https://doi.org/10.1038/nmeth.4473

    Article  Google Scholar 

  41. Vermazeren J, Eekelen Van L, Meesters L, Looijen-Salamon M, Vos S, Munari E, Mercan C, Ciompi F (2021) µPEN: multi-class PseudoEdgeNet for PD-L1 assessment Presented at the Medical Imaging with Deep Learning 2021, p. 3

  42. Vicar T, Raudenska M, Gumulec J, Balvan J (2020) The quantitative-phase dynamics of apoptosis and lytic cell death. Sci Rep. https://doi.org/10.1038/s41598-020-58474-w

    Article  Google Scholar 

  43. Vuola AO, Akram SU, Kannala J (2019) Mask-RCNN and U-Net Ensembled for nuclei segmentation, In: 2019 IEEE 16th international symposium on biomedical imaging (ISBI 2019). Presented at the 2019 IEEE 16th international symposium on biomedical imaging (ISBI 2019), pp. 208–212. https://doi.org/10.1109/ISBI.2019.8759574

  44. Waldvogel HJ, Kim EH, Tippett LJ, Vonsattel JPG, Faull RL (2015) The neuropathology of huntington’s disease. Curr Top Behav Neurosci 22:33

    Article  Google Scholar 

  45. Walløe S, Pakkenberg B, Fabricius K (2014) Stereological estimation of total cell numbers in the human cerebral and cerebellar cortex. Front Hum Neurosci 8:508. https://doi.org/10.3389/fnhum.2014.00508

    Article  Google Scholar 

  46. Wang X, Kong T, Shen C, Jiang Y, Li L (2020) SOLO: Segmenting Objects by Locations. ArXiv191204488 Cs.

  47. West MJ, Slomianka L, Gundersen HJ (1991) Unbiased stereological estimation of the total number of neurons in the subdivisions of the rat hippocampus using the optical fractionator. Anat Rec 231:482–497. https://doi.org/10.1002/ar.1092310411

    Article  Google Scholar 

  48. Williams RW, Herrup K (1988) The control of neuron number. Annu Rev Neurosci 11:423–453. https://doi.org/10.1146/annurev.ne.11.030188.002231

    Article  Google Scholar 

  49. Xie W, Noble JA, Zisserman A (2016) Microscopy cell counting and detection with fully convolutional regression networks. Comput Methods Biomech Biomed Eng Imaging Vis 6:283–292

    Article  Google Scholar 

  50. Xue Y, Ray N, Hugh J, Bigras G (2017) A novel framework to integrate convolutional neural network with compressed sensing for cell detection, In: 2017 IEEE International conference on image processing (ICIP). Presented at the 2017 IEEE International conference on image processing (ICIP), pp. 2319–2323. https://doi.org/10.1109/ICIP.2017.8296696

  51. Yang X, Li H, Zhou X (2006) Nuclei segmentation using marker-controlled watershed, tracking using mean-shift, and kalman filter in time-lapse microscopy. IEEE Trans Circuits Syst Regul Pap 53:2405–2414. https://doi.org/10.1109/TCSI.2006.884469

    Article  Google Scholar 

  52. Yoo I, Yoo D, Paeng K (2019) PseudoEdgeNet Nuclei Segmentation only with Point Annotations. In: Shen D, Liu T, Peters TM, Staib LH, Essert C, Zhou S, Yap P-T, Khan A (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2019 Lecture Notes in Computer Science. Springer, Cham

    Google Scholar 

  53. You Z, Balbastre Y, Bouvier C, Hérard A-S, Gipchtein P, Hantraye P, Jan C, Souedet N, Delzescaux T (2019) Automated individualization of size-varying and touching neurons in macaque cerebral microscopic images. Front Neuroanat. https://doi.org/10.3389/fnana.2019.00098

    Article  Google Scholar 

  54. You Z, Jiang M, Shi Z, Shi C, Du S, Liang J, Hérard AS, Jan C, Souedet N, Delzescaux T (2020) Automated detection of highly aggregated neurons in microscopic images of macaque brain, In: 2020 IEEE international conference on image processing (ICIP). Presented at the 2020 IEEE International conference on image processing (ICIP), pp. 315–319. https://doi.org/10.1109/ICIP40778.2020.9190747

  55. Zhang C, Sun C, Pham TD (2013) Segmentation of clustered nuclei based on concave curve expansion. J Microsc 251:57–67. https://doi.org/10.1111/jmi.12043

    Article  Google Scholar 

  56. Zhang J, Hu Z, Han G, He X (2016) Segmentation of overlapping cells in cervical smears based on spatial relationship and overlapping translucency light transmission model. Pattern Recognit 60:286–295. https://doi.org/10.1016/j.patcog.2016.04.021

    Article  Google Scholar 

  57. Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J (2020) UNet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans Med Imaging 39:1856–1867. https://doi.org/10.1109/TMI.2019.2959609

    Article  Google Scholar 

  58. Zucker SW (1976) Region growing: childhood and adolescence. Comput Graph Image Process 5:382–399

    Article  Google Scholar 

Download references

Funding

This work was supported by Natural Science Foundation of Shaanxi Province of China [2020JQ-652]; Natural Science Foundation of Shaanxi Provincial Department of Education of China [20JK0795]; Fund of Doctoral Start-up of Xi’an University of Technology [112/256081811]; French national funds (PIA2’ program) under contract No. P112331-3422142 (3D NeuroSecure project); General Program of National Natural Science Foundation of China [62076198]; Key Program of Natural Science Foundation of Shaanxi Province of China [2020GXLH-Y005].

Author information

Authors and Affiliations

  1. Shaanxi Key Laboratory for Network Computing and Security Technology, School of Computer Science and Engineering, Xi’an University of Technology, Xi’an, China

    Zhenzhen You, Zhenghao Shi, Cheng Shi & Shuangli Du

  2. CEA-CNRS-UMR 9199, Laboratoire des Maladies Neurodégénératives, MIRCen, Fontenay-aux-Roses, Université Paris-Saclay, Gif-sur-Yvette, France

    Zhenzhen You, Anne-Sophie Hérard, Caroline Jan, Nicolas Souedet & Thierry Delzescaux

  3. National Laboratory of Radar Signal Processing, Xidian University, Xi’an, China

    Ming Jiang

  4. School of Electronic Engineering, Xidian University, Xi’an, China

    Jimin Liang

Authors
  1. Zhenzhen You
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Ming Jiang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Zhenghao Shi
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Cheng Shi
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Shuangli Du
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Jimin Liang
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Anne-Sophie Hérard
    View author publications

    You can also search for this author inPubMed Google Scholar

  8. Caroline Jan
    View author publications

    You can also search for this author inPubMed Google Scholar

  9. Nicolas Souedet
    View author publications

    You can also search for this author inPubMed Google Scholar

  10. Thierry Delzescaux
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Zhenzhen You.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflict of interest.

Ethical approval

The animal study was reviewed and approved by the Comité d’éthique agréé par le MESR dont relève l’EU: CETEA DSV –Comité n◦44.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

You, Z., Jiang, M., Shi, Z. et al. Macaque neuron instance segmentation only with point annotations based on multiscale fully convolutional regression neural network. Neural Comput & Applic 34, 2925–2938 (2022). https://doi.org/10.1007/s00521-021-06574-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06574-7

Keywords