Skip to main content
SpringerLink
Log in

Automated cell division classification in early mouse and human embryos using convolutional neural networks

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

During in vitro fertilization (IVF), the timing of cell divisions in early human embryos is a key predictor of embryo viability. Recent developments in time-lapse microscopy (TLM) have allowed us to observe cell divisions in much greater detail than previously possible. However, it is a time-consuming process that relies on a highly trained staff and subjective observations. We describe an automated method based on a convolutional neural network to detect and classify cell divisions from original (unprocessed) TLM images. Here, we used two embryo TLM image datasets to evaluate our method: a public dataset with mouse embryos up to the 4-cell stage and a private dataset with human embryos up to the 8-cell stage. Compared to embryologists’ annotations, our results were almost 100% accurate for the mouse embryo images and accurate within five frames in 93.9% of cell stage transitions for the human embryos. Our approach can be used to improve the consistency and quality of the existing annotations or as part of a platform for fully automated embryo assessment. The code is available at http://github.com/JonasEMalmsten/CellDivision.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Code availability

https://github.com/JonasEMalmsten/CellDivision

References

  1. cdc.gov (2018) Center for disease control and prevention— ART success rates for 2015. http://www.cdc.gov/art/reports/. Accessed 2018

  2. sartcorsonline.com (2019) SART National summary report. https://www.sartcorsonline.com/rptCSR_PublicMultYear.aspx?ClinicPKID=0. Accessed: 2019. [cited 2019]

  3. Land J, Evers J (2003) Risks and complications in assisted reproduction techniques: report of an ESHRE consensus meeting. Human Reprod 18(2):455–457

    Article  Google Scholar 

  4. Braude P, Bolton V, Moore S (1988) Human gene expression first occurs between the four-and eight-cell stages of preimplantation development. Nature 332(6163):459–461

    Article  Google Scholar 

  5. Lundin K, Bergh C, Hardarson T (2001) Early embryo cleavage is a strong indicator of embryo quality in human IVF. Hum Reprod 16(12):2652–2657

    Article  Google Scholar 

  6. Veeck, L.I. and N. Zaninovic, Atlas of Human Blastocysts. 2007: CRC Press

  7. Meseguer M et al (2011) The use of morphokinetics as a predictor of embryo implantation. Hum Reprod 26(10):2658–2671

    Article  Google Scholar 

  8. Wong CC et al (2010) Non-invasive imaging of human embryos before embryonic genome activation predicts development to the blastocyst stage. Nat Biotech 28(10):1115–1121

    Article  Google Scholar 

  9. Rai R, Regan L (2006) Recurrent miscarriage. The Lancet 368(9535):601–611

    Article  Google Scholar 

  10. Khosravi P et al. (2018) Robust automated assessment of human blastocyst quality using deep learning. bioRxiv, 2018: p. 394882

  11. Giusti A et al. (2009) Segmentation of human zygotes in hoffman modulation contrast images. In: Proc. of MIUA

  12. Giusti A et al. (2010) Blastomere segmentation and 3d morphology measurements of early embryos from hoffman modulation contrast image stacks. In: Biomedical imaging: from nano to macro, 2010 IEEE International Symposium on. 2010. IEEE

  13. Moussavi F et al (2014) A unified graphical models framework for automated mitosis detection in human embryos. IEEE Trans Med Imaging 33(7):1551–1562

    Article  Google Scholar 

  14. LeCun Y et al (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  15. Szegedy C et al. (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  16. He K et al. (2016) Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition

  17. Khan A, Gould S, Salzmann M (2016) Deep convolutional neural networks for human embryonic cell counting. In: European conference on computer vision. 2016. Springer

  18. Gingold J et al (2018) Predicting embryo morphokinetic annotations from time-lapse videos using convolutional neural networks. Fertil Steril 110(4):e220

    Article  Google Scholar 

  19. Malmsten J et al (2018) Automatic prediction of embryo cell stages using artificial intelligence convolutional neural network. Fertil Steril 110(4):e360

    Article  Google Scholar 

  20. Lau T et al. (2019) Embryo staging with weakly-supervised region selection and dynamically-decoded predictions. arXiv preprint arXiv:1904.04419

  21. Tran A et al (2018) Artificial intelligence as a novel approach for embryo selection. Fertil Steril 110(4):e430

    Article  Google Scholar 

  22. Malmsten J et al. (2019) Automated cell stage predictions in early mouse and human embryos using convolutional neural networks. In: 2019 IEEE EMBS international conference on biomedical & health informatics (BHI). 2019. IEEE

  23. Szegedy C et al. (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  24. Kingma D, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

  25. Chollet F et al (2015) Keras. https://keras.io

  26. Cicconet M et al (2014) Label free cell-tracking and division detection based on 2d time-lapse images for lineage analysis of early embryo development. Comput Biol Med 51:24–34

    Article  Google Scholar 

  27. Khan AS, Gould, Salzmann M (2015) Automated monitoring of human embryonic cells up to the 5-cell stage in time-lapse microscopy images. In: Biomedical imaging (ISBI), 2015 IEEE 12th international symposium on. 2015. IEEE

Download references

Funding

None

Author information

Authors and Affiliations

  1. Ronald O. Perelman and Claudia Cohen Center for Reproductive Medicine, Weill Cornell Medicine, New York, USA

    Jonas Malmsten, Nikica Zaninovic, Qiansheng Zhan & Zev Rosenwaks

  2. Department of Computer Science, Seidenberg School of CSIS, Pace University, New York, USA

    Jonas Malmsten & Juan Shan

Authors
  1. Jonas Malmsten
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nikica Zaninovic
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qiansheng Zhan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zev Rosenwaks
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Juan Shan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jonas Malmsten.

Ethics declarations

Conflict of interest

Authors declare that they have no conflict of interest.

Ethical approval

IRB 19-04,020,098

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Malmsten, J., Zaninovic, N., Zhan, Q. et al. Automated cell division classification in early mouse and human embryos using convolutional neural networks. Neural Comput & Applic 33, 2217–2228 (2021). https://doi.org/10.1007/s00521-020-05127-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-05127-8

Keywords

Search

Navigation