Skip to main content
SpringerLink
Log in

An end-to-end CNN and LSTM network with 3D anchors for mitotic cell detection in 4D microscopic images and its parallel implementation on multiple GPUs

  • Advances in Parallel and Distributed Computing for Neural Computing
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The detection and observation of mitotic event are the key to studying the behavior of the cell and used to examine various diseases. The existing cell detection methods are performed on two-dimensional images with time sequence. However, the complexity of mitotic and normal cells and the orientation of the mitosis generate high false positive when using 2D methods. On the other hand, 3D methods can perform higher performance than 2D methods but also face the problem of overfitting due to the limit of training data. With those problems, we propose a 2.5-dimensional convolutional neural network with convolutional long short-term memory to extract the information time sequence and combined with 3D anchors to gather the spatial information for final mitotic detection. Furthermore, we also propose the method with a parallel model on multi-GPUs to speed up the detection time. Compared with state-of-the-art methods, our method can reach high precision and also recall rate with detection time is speed up about 1.9 times by the use of the parallel model on 4GPUs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Ipponjima S, Hibi T, Nemoto T (2016) Three-Dimensional Analysis of Cell Division Orientation in Epidermal Basal Layer Using Intravital Two-Photon Microscopy, PLOS one

  2. Hsu YC, Li L, Fuchs E (2014) Emerging interactions between skin stem cells and their niches. Nat Med 20(8):847–856

    Google Scholar 

  3. Jones P, Simons BD (2008) Epidermal homeostasis: do committed progenitors work while stem cells sleep? Nat Rev Mol Cell Biol 9(1):82–88

    Google Scholar 

  4. Watt FM (2014) Mammalian skin cell biology: at the interface between laboratory and clini. Science 346(6212):937–940

    Google Scholar 

  5. van Diest PJ, Baak JPA, Matze-Cok P, Wisse-Brekelmans ECM, van Galen CM, Kurver PHJ et al (1992) Reproducibility of mitosis counting in 2,469 breast cancer specimens: Results from the Multicenter Morphometric Mammary Carcinoma Project. Human Pathology 23(6):603–607

    Google Scholar 

  6. Konno D, Shioi G, Shitamukai A, Mori A, Kiyonari H, Miyata T et al (2008) Neuroepithelial progenitors undergo LGN-dependent planar divisions to maintain self-renewability during mammalian neurogenesis. Nat Cell Biol. 93–101

  7. Wu M, Smith CL, Hall JA, Lee I, Luby-Phelps K, Tallquist MD (2010) Epicardial spindle orientation controls cell entry into the myocardium. Dev Cell. 114–125

  8. Mao Y, Yin Z (2016) A hierarchical convolutional neural network for mitosis detection in phase-contrast microscopy images. Medical Image Computing and Computer-Assisted Intervention - MICCAI 685–692

  9. Mao Y, Yin Z (2017) Two-Stream Bidirectional Long Short-Term Memory for Mitosis Event Detection and Stage Localization in Phase-Contrast Microscopy Images, Medical Image Computing and Computer-Assisted Intervention - MICCAI

  10. Yang W, Li K, Mo Z, Li K (2015) Performance Optimization Using Partitioned SpMV on GPUs and Multicore CPUs. IEEE Transaction on Computers 64(9):2623–2636

    MathSciNet  MATH  Google Scholar 

  11. Xiao G, Li K CASpMV: A Customized and Accelerative SpMV Framework for the Sunway TaihuLight, IEEE Transactions on Parallel and Distributed Systems, https://doi.org/10.1109/TPDS.2019.2907537

  12. Chen Y, Li K (2019) Performance-Aware Model for Sparse Matrix-Matrix Multiplication on the Sunway TaihuLight Supercomputer. IEEE Transactions on Parallel and Distributed Systems 30(4):923–938

    Google Scholar 

  13. Chen J, Li K, Bilal K, Zhou X, Li K, Yu PS (2019) A Bi-layered Parallel Training Architecture for Large-scale Convolutional Neural Networks. IEEE Transactions on Parallel and Distributed Systems 30(5):965–976

    Google Scholar 

  14. Chen J, Li K, Deng Q, Li K, Yu PS (2019) Distributed Deep Learning Model for Intelligent Video Surveillance Systems with Edge Computing. IEEE Transactions on Industrial Informatics. https://doi.org/10.1109/TII.2019.2909473

  15. Kitrungrotsakul T, Han XH, Iwamoto Y, Takemoto S, Yokota H, Ipponjima S, Nemoto T, Wei X, Chen YW (2018) A 2.5D Cascaded Convolutional Neural Network with Temporal Information for Automatic Mitotic Cell Detection in 4D Microscopic Images, Proc. of ICNC-FSKD2018

  16. Hegde V, Usmani S (2016) Parallel and Distributed Deep Learning

  17. Ben-Nun T, Hoefler T (2018) Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis, arXiv:1802.09941

  18. Xiao G, Li K, Zhou X, Li K (2017) Efficient monochromatic and bichromatic probabilistic reverse top-k query processing for uncertain big data. Journal of Computer and System Sciences 89:92–113

    MathSciNet  MATH  Google Scholar 

  19. Xiao G, Li K, Li K (2017) Reporting l Most Influential Objects in Uncertain Databases Based on Probabilistic Reverse Top-k Queries. Information Sciences 405:207–226

    Google Scholar 

  20. Xiao G, Li K, Li K, Zhou X (2015) Efficient Top-(k, l) Range Query Processing for Uncertain Data Based on Multicore Architectures. Distributed and Parallel Databases 33(3):381–483

    Google Scholar 

  21. Xiao G, Li K, Zhou X, Li K (2016) Queueing Analysis of Continuous Queries for Uncertain Data Streams Over Sliding Windows. International Journal of Pattern Recognition and Artificial Intelligence 30(9):1–16

    Google Scholar 

  22. Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks, NIPS

  23. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Alexander C (2016) SSD: Single Shot MultiBox Detector, ECCV

  24. Huh S, Ker DF, Bise R, Chen M, Kanade T (2011) Automated mitosis detection of stem cell populations in phase contrast microscopy image. IEEE Trans. Med. Imag. 30(3):586–596

    Google Scholar 

  25. Suykens J, Vandewalle J (1999) least squares support vector machine classifiers. Neural Process. Nett 9(3):293–300

    Google Scholar 

  26. Lin TY, Dollar P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature Pyramid Networks for Object Detection. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 936–944:

  27. Simonyan K, Zisserman A (2014) Very Deep Convolutional Networks for Large-Scale Image Recognition, arXiv technical report

  28. He K, Zhang X, Ren S, Sun J (2015) Deep Residual Learning for Image Recognition, arXiv preprint arXiv:1512.03385

  29. Sauvola J, Pietikainen M (2000) Adaptive document image binarization. Pattern Recognition 33(2):225–236

    Google Scholar 

  30. Erik M (2012) Cell segmentation: 50 years down the road [life sciences]

  31. Shkolyar A, Gefen A, Benayahu D, Greenspan H (2015) Automatic detection of cell divisions (mitosis) in live-imaging microscopy images using Convolutional Neural Networks. Engineering in Medicine and Biology Society (EMBC) 743–746

  32. Xingjian S, Chen Z, Wang H, Yeung DY, Wong WK, Woo WC (2015) Convolutational LSTM Network: A Machine Learning Approach for Precipitation Nowcasting. NIPS 802–810

  33. TC-IAIP AIA2017, http://www.tc-iaip.org/index-e.shtml

  34. Chollet F et al (2015) Keras, https://github.com/fchollet/keras

  35. Abadi M et al. (2015) TensorFlow: large-scale machine learning on heterogeneous systems. http://tensorflow.org

  36. Sugano J (2017) Mitotic cell division event detection using classification of temporal feature histogram, ViEW 2017 visual inspection algorithm competition

  37. Kitrungrotsakul T, Iwamoto Y, Han XH, Takemoto S, Yokota H, Ipponjima S, Nemoto T, Xiong W, Chen YW A (2019) Cascade of 2.5D CNN and Bidirectional CLSTM Network for Mitotic Cell Detection in 4D Microscopy Image, IEEE/ACM Trans. Computational Biology and Bioinformatics, in press

  38. Kitrungrotsakul T, Iwamoto Y, Han XH, Takemoto S, Yokota H, Ipponjima S, Nemoto T, Xiong W, Chen YW (2019) A Cascade of CNN and LSTM Network with 3D Anchors for Mitotic Cell Detection in 4D Microscopy Image, Proc. of IEEE ICASSP, pp.1239-1243

Download references

Acknowledgements

This work is supported in part by Japan Society for Promotion of Science (JSPS) under Grant Nos. 16J09596 and KAKEN under the Grant Nos. 18H04747, 16H01436, 15H05954, 15H05953; in part by Zhejiang Lab Program under the Grant No. 2018DG0ZX01 and in part by A*STAR Research Attachment Programme.

Author information

Authors and Affiliations

  1. Graduate School of Information Science and Engineering, Ritsumeikan University, Shiga, Japan

    Titinunt Kitrungrotsakul, Yutaro Iwamoto & Yen-Wei Chen

  2. Faculty of Science, Yamaguchi University, Yamaguchi, Japan

    Xian-Hua Han

  3. Center for Advanced Photonics, RIKEN, Wako, Japan

    Satoko Takemoto & Hideo Yokota

  4. Research Institute for Electronic Science, Hokkaido University, Sapporo, Japan

    Sari Ipponjima & Tomomi Nemoto

  5. Institute for Infocomm Research, A*Star, Singapore, Singapore

    Wei Xiong

  6. The Zhejiang Lab, Hangzhou, China

    Yen-Wei Chen

Authors
  1. Titinunt Kitrungrotsakul
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xian-Hua Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yutaro Iwamoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Satoko Takemoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hideo Yokota
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Sari Ipponjima
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Tomomi Nemoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Wei Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Yen-Wei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yen-Wei Chen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kitrungrotsakul, T., Han, XH., Iwamoto, Y. et al. An end-to-end CNN and LSTM network with 3D anchors for mitotic cell detection in 4D microscopic images and its parallel implementation on multiple GPUs. Neural Comput & Applic 32, 5669–5679 (2020). https://doi.org/10.1007/s00521-019-04374-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04374-8

Keywords

Search

Navigation