Abstract
During in vitro fertilization (IVF), the timing of cell divisions in early human embryos is a key predictor of embryo viability. Recent developments in time-lapse microscopy (TLM) have allowed us to observe cell divisions in much greater detail than previously possible. However, it is a time-consuming process that relies on a highly trained staff and subjective observations. We describe an automated method based on a convolutional neural network to detect and classify cell divisions from original (unprocessed) TLM images. Here, we used two embryo TLM image datasets to evaluate our method: a public dataset with mouse embryos up to the 4-cell stage and a private dataset with human embryos up to the 8-cell stage. Compared to embryologists’ annotations, our results were almost 100% accurate for the mouse embryo images and accurate within five frames in 93.9% of cell stage transitions for the human embryos. Our approach can be used to improve the consistency and quality of the existing annotations or as part of a platform for fully automated embryo assessment. The code is available at http://github.com/JonasEMalmsten/CellDivision.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles and news from researchers in related subjects, suggested using machine learning.Code availability
References
cdc.gov (2018) Center for disease control and prevention— ART success rates for 2015. http://www.cdc.gov/art/reports/. Accessed 2018
sartcorsonline.com (2019) SART National summary report. https://www.sartcorsonline.com/rptCSR_PublicMultYear.aspx?ClinicPKID=0. Accessed: 2019. [cited 2019]
Land J, Evers J (2003) Risks and complications in assisted reproduction techniques: report of an ESHRE consensus meeting. Human Reprod 18(2):455–457
Braude P, Bolton V, Moore S (1988) Human gene expression first occurs between the four-and eight-cell stages of preimplantation development. Nature 332(6163):459–461
Lundin K, Bergh C, Hardarson T (2001) Early embryo cleavage is a strong indicator of embryo quality in human IVF. Hum Reprod 16(12):2652–2657
Veeck, L.I. and N. Zaninovic, Atlas of Human Blastocysts. 2007: CRC Press
Meseguer M et al (2011) The use of morphokinetics as a predictor of embryo implantation. Hum Reprod 26(10):2658–2671
Wong CC et al (2010) Non-invasive imaging of human embryos before embryonic genome activation predicts development to the blastocyst stage. Nat Biotech 28(10):1115–1121
Rai R, Regan L (2006) Recurrent miscarriage. The Lancet 368(9535):601–611
Khosravi P et al. (2018) Robust automated assessment of human blastocyst quality using deep learning. bioRxiv, 2018: p. 394882
Giusti A et al. (2009) Segmentation of human zygotes in hoffman modulation contrast images. In: Proc. of MIUA
Giusti A et al. (2010) Blastomere segmentation and 3d morphology measurements of early embryos from hoffman modulation contrast image stacks. In: Biomedical imaging: from nano to macro, 2010 IEEE International Symposium on. 2010. IEEE
Moussavi F et al (2014) A unified graphical models framework for automated mitosis detection in human embryos. IEEE Trans Med Imaging 33(7):1551–1562
LeCun Y et al (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324
Szegedy C et al. (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition
He K et al. (2016) Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition
Khan A, Gould S, Salzmann M (2016) Deep convolutional neural networks for human embryonic cell counting. In: European conference on computer vision. 2016. Springer
Gingold J et al (2018) Predicting embryo morphokinetic annotations from time-lapse videos using convolutional neural networks. Fertil Steril 110(4):e220
Malmsten J et al (2018) Automatic prediction of embryo cell stages using artificial intelligence convolutional neural network. Fertil Steril 110(4):e360
Lau T et al. (2019) Embryo staging with weakly-supervised region selection and dynamically-decoded predictions. arXiv preprint arXiv:1904.04419
Tran A et al (2018) Artificial intelligence as a novel approach for embryo selection. Fertil Steril 110(4):e430
Malmsten J et al. (2019) Automated cell stage predictions in early mouse and human embryos using convolutional neural networks. In: 2019 IEEE EMBS international conference on biomedical & health informatics (BHI). 2019. IEEE
Szegedy C et al. (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition
Kingma D, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980
Chollet F et al (2015) Keras. https://keras.io
Cicconet M et al (2014) Label free cell-tracking and division detection based on 2d time-lapse images for lineage analysis of early embryo development. Comput Biol Med 51:24–34
Khan AS, Gould, Salzmann M (2015) Automated monitoring of human embryonic cells up to the 5-cell stage in time-lapse microscopy images. In: Biomedical imaging (ISBI), 2015 IEEE 12th international symposium on. 2015. IEEE
Funding
None
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
Authors declare that they have no conflict of interest.
Ethical approval
IRB 19-04,020,098
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Malmsten, J., Zaninovic, N., Zhan, Q. et al. Automated cell division classification in early mouse and human embryos using convolutional neural networks. Neural Comput & Applic 33, 2217–2228 (2021). https://doi.org/10.1007/s00521-020-05127-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-020-05127-8