Skip to main content
Springer Nature Link
Log in

Coupling cell detection and tracking by temporal feedback

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

The tracking-by-detection strategy is the backbone of many methods for tracking living cells in time-lapse microscopy. An object detector is first applied to the input images, and the resulting detection candidates are then linked by a data association module. The performance of such methods strongly depends on the quality of the detector because detection errors propagate to the linking step. To tackle this issue, we propose a joint model for segmentation, detection and tracking. The model is defined implicitly as limiting distribution of a Markov chain Monte Carlo algorithm and contains a temporal feedback, which allows to dynamically alter detector parameters using hints given by neighboring frames and, in this way, correct detection errors. The proposed method can integrate any detector and is therefore not restricted to a specific domain. The parameters of the model are learned using an objective based on empirical risk minimization. We use our method to conduct large-scale experiments for confluent cultures of endothelial cells and evaluate its performance in the ISBI Cell Tracking Challenge, where it consistently scored among the best three methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. http://www.celltrackingchallenge.net/.

References

  1. Akram, S.U., Kannala, J., Eklund, L., Heikkilä, J.: Joint cell segmentation and tracking using cell proposals. In: 13th IEEE International Symposium on Biomedical Imaging, ISBI 2016, Prague, Czech Republic, April 13–16, 2016, pp. 920–924. IEEE (2016). https://doi.org/10.1109/ISBI.2016.7493415

  2. Bise, R., Sato, Y.: Cell detection from redundant candidate regions under nonoverlapping constraints. IEEE Trans. Med. Imaging 34(7), 1417–1427 (2015). https://doi.org/10.1109/TMI.2015.2391095

    Article  Google Scholar 

  3. Cao, J., Ehling, M., März, S., Seebach, J., Tarbashevich, K., Sixta, T., Pitulescu, M.E., Werner, A.C., Flach, B., Montanez, E., Raz, E., Adams, R.H., Schnittler, H.: Polarized actin and ve-cadherin dynamics regulate junctional remodelling and cell migration during sprouting angiogenesis. Nat. Commun. 8(1), 2210–2230 (2017). https://doi.org/10.1038/s41467-017-02373-8

    Article  Google Scholar 

  4. Chakraborty, A., Roy-Chowdhury, A.K.: A conditional random field model for tracking in densely packed cell structures. In: 2014 IEEE International Conference on Image Processing, ICIP 2014, Paris, France, October 27–30, 2014, pp. 451–455. IEEE (2014). https://doi.org/10.1109/ICIP.2014.7025090

  5. Dahlhaus, E., Johnson, D.S., Papadimitriou, C.H., Seymour, P.D., Yannakakis, M.: The complexity of multiway cuts (extended abstract). In: Proceedings of the Twenty-fourth Annual ACM Symposium on Theory of Computing, STOC ’92, pp. 241–251. ACM, New York, NY, USA (1992). https://doi.org/10.1145/129712.129736

  6. Fiaschi, L., Diego, F., Gregor, K., Schiegg, M., Koethe, U., Zlatic, M., Hamprecht, F.A.: Tracking indistinguishable translucent objects over time using weakly supervised structured learning. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014)

  7. Harder, N., Batra, R., Diessl, N., Gogolin, S., Eils, R., Westermann, F., König, R., Rohr, K.: Large-scale tracking and classification for automatic analysis of cell migration and proliferation, and experimental optimization of high-throughput screens of neuroblastoma cells. Cytom. Part A 87(6), 524–540 (2015). https://doi.org/10.1002/cyto.a.22632

    Article  Google Scholar 

  8. Jug, F., Pietzsch, T., Kainmüller, D., Funke, J., Kaiser, M., van Nimwegen, E., Rother, C., Myers, G.: Optimal Joint Segmentation and Tracking of Escherichia coli in the Mother Machine, pp. 25–36. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-12289-2_3

    Book  Google Scholar 

  9. Li, F., Zhou, X., Ma, J., Wong, S.T.C.: Multiple nuclei tracking using integer programming for quantitative cancer cell cycle analysis. IEEE Trans. Med. Imaging 29(1), 96–105 (2010). https://doi.org/10.1109/TMI.2009.2027813

    Article  Google Scholar 

  10. Lou, X., Schiegg, M., Hamprecht, F.A.: Active structured learning for cell tracking: algorithm, framework, and usability. IEEE Trans. Med. Imaging 33(4), 849–860 (2014). https://doi.org/10.1109/TMI.2013.2296937

    Article  Google Scholar 

  11. Luo, W., Zhao, X., Kim, T.: Multiple object tracking: A review. CoRR arXiv:1409.7618 (2014)

  12. Magnusson, K.E.G., Jaldén, J., Gilbert, P.M., Blau, H.M.: Global linking of cell tracks using the viterbi algorithm. IEEE Trans. Med. Imaging 34(4), 911–929 (2015). https://doi.org/10.1109/TMI.2014.2370951

    Article  Google Scholar 

  13. Maška, M., Ulman, V., Svoboda, D., Matula, P., Matula, P., Ederra, C., Urbiola, A., España, T., Venkatesan, S., Balak, D.M., Karas, P., Bolcková, T., Štreitová, M., Carthel, C., Coraluppi, S., Harder, N., Rohr, K., Magnusson, K.E.G., Jaldén, J., Blau, H.M., Dzyubachyk, O., Křížek, P., Hagen, G.M., Pastor-Escuredo, D., Jimenez-Carretero, D., Ledesma-Carbayo, M.J., Muñoz-Barrutia, A., Meijering, E., Kozubek, M., Ortiz-de Solorzano, C.: A benchmark for comparison of cell tracking algorithms. Bioinformatics 30(11), 1609 (2014). https://doi.org/10.1093/bioinformatics/btu080

  14. Perner, P.: Tracking living cells in microscopic images and description of the kinetics of the cells. Procedia Comput. Sci. 60(Complete), 352–361 (2015). https://doi.org/10.1016/j.procs.2015.08.141

    Article  Google Scholar 

  15. Reid, D.B.: An algorithm for tracking multiple targets. IEEE Trans. Autom. Control 24, 843–854 (1979)

    Article  Google Scholar 

  16. Rosin, P.L.: Measuring shape: ellipticity, rectangularity, and triangularity. Mach. Vis. Appl. 14(3), 172–184 (2003). https://doi.org/10.1007/s00138-002-0118-6

    Article  Google Scholar 

  17. Schiegg, M., Hanslovsky, P., Haubold, C., Koethe, U., Hufnagel, L., Hamprecht, F.A.: Graphical model for joint segmentation and tracking of multiple dividing cells. Bioinformatics 31(6), 948 (2015). https://doi.org/10.1093/bioinformatics/btu764

    Article  Google Scholar 

  18. Simard, P.Y., Steinkraus, D., Platt, J.C.: Best practices for convolutional neural networks applied to visual document analysis. In: Proceedings of the Seventh International Conference on Document Analysis and Recognition—Volume 2, ICDAR ’03, pp. 958. IEEE Computer Society, Washington, DC, USA (2003). http://dl.acm.org/citation.cfm?id=938980.939477

  19. Thirusittampalam, K., Hossain, M.J., Ghita, O., Whelan, P.F.: A novel framework for cellular tracking and mitosis detection in dense phase contrast microscopy images. IEEE J. Biomed. Health Inform. 17(3), 642–653 (2013). https://doi.org/10.1109/TITB.2012.2228663

    Article  Google Scholar 

  20. Türetken, E., Wang, X., Becker, C.J., Haubold, C., Fua, P.: Network flow integer programming to track elliptical cells in time-lapse sequences. IEEE Trans. Med. Imaging 36(4), 942–951 (2017). https://doi.org/10.1109/TMI.2016.2640859

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Cybernetics, Center for Machine Perception, Faculty of Electrical Engineering, Czech Technical University in Prague, Prague, Czech Republic

    Tomáš Sixta & Boris Flach

  2. Institute of Anatomy and Vascular Biology, University of Münster, Münster, Germany

    Jiahui Cao, Jochen Seebach & Hans Schnittler

  3. Cells-in-Motion Cluster of Excellence (EXC 1003 – CiM), University of Münster, Münster, Germany

    Jiahui Cao, Jochen Seebach & Hans Schnittler

Authors
  1. Tomáš Sixta
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Jiahui Cao
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Jochen Seebach
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Hans Schnittler
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Boris Flach
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Boris Flach.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported by the Czech Science Foundation project 16-05872S and by the Graduate School of the Cells-in-Motion Cluster of Excellence (EXC 1003 - CiM), WWU Münster and International Max Planck Research School – Molecular Biomedicine, Münster. HS acknowledges grants SCHN 43076-2 and DFG INST 2105/24-1 of the German Research Council and grants 03ZZ0902D and 03ZZ0906E from the BMBF. The supports by the Excellence Cluster Cells In Motion (CIM) flexible fund to J.S (FF-2016-15) and to HS (FF-2014-15) are also greatly acknowledged. BF gratefully acknowledges support by the Czech OP VVV project “Research Center for Informatics” (CZ.02.1.01/0.0/0.0/16 019/0000765).

A Experiments with endothelial cells

A Experiments with endothelial cells

  1. E1:

    Cells treated with 50 ng/ml vascular endothelial growth factor (VEGF) versus cells treated with phosphate-buffered saline (control)

    Group 1: 4 sequences, 1646 cells

    Group 2: 4 sequences, 1701 cells

    Duration: 6 h (73 frames)

    Objective: \(10\times \) (\(1.02~\upmu \)m per pixel)

  2. E2:

    Cells transfected with adenoN17Rac virus (adenoN17Rac) \(+\) VEGF versus cells transfected with adeno-empty virus \(+\) VEGF (control)

    Group 1: 3 sequences, on average 364 cells in each sequence

    Group 2: 3 sequences, 377 cells

    Duration: 26 h (79 frames)

    Objective: \(20\times \) (\(0.215~\upmu \)m per pixel)

  3. E3:

    Cells treated with the EHT1864 inhibitor of Rac activity (EHT1864) \(+\) VEGF versus cells treated with dimethyl sulfoxide \(+\) VEGF (control)

    Group 1: 2 sequences, 1700 cells

    Group 2: 3 sequences, 1520 cells

    Duration: 19 h (77 frames)

    Objective: \(10\times \) (\(1.02~\upmu \)m per pixel)

  4. E4:

    Cells transfected with Nrp siRNA (siNrp) \(+\) VEGF versus cells transfected with non-targeting siRNA \(+\) VEGF (control)

    Group 1: 3 sequences, 1221 cells

    Group 2: 3 sequences, 1272 cells

    Duration: 24.5 h (50 frames)

    Objective: \(10\times \) (\(1.02~\upmu \)m per pixel)

  5. E5:

    Cells transfected with vascular endothelial growth factor receptor 2 siRNA (siVEGFR2) \(+\) VEGF versus cells transfected with non targeting siRNA \(+\) VEGF (control)

    Group 1: 3 sequences, 1132 cells

    Group 2: 3 sequences, 1390 cells

    Duration: 22.5 h (46 frames)

    Objective: \(10\times \) (\(1.02~\upmu \)m per pixel)

  6. E6:

    Cells transfected with VE-cadherin-GFP adenovirus

    (VEcadGFP) \(+\) VEGF versus cells transfected with GFP adenovirus \(+\) VEGF (control)

    Group 1: 3 sequences, 387 cells

    Group 2: 3 sequences, 397 cells

    Duration: 26 h (157 frames)

    Objective: \(20\times \) (\(0.215~\upmu \)m per pixel)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sixta, T., Cao, J., Seebach, J. et al. Coupling cell detection and tracking by temporal feedback. Machine Vision and Applications 31, 24 (2020). https://doi.org/10.1007/s00138-020-01072-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-020-01072-7

Keywords