Skip to main content
SpringerLink
Log in

Detecting and tracking leukocytes in intravital video microscopy using a Hessian-based spatiotemporal approach

  • Published:
Multidimensional Systems and Signal Processing Aims and scope Submit manuscript

Abstract

The leukocyte recruitment analysis is an important step to understand the interactions between leukocytes and endothelial cells in the microcirculation of living animals. Performed preferably by the intravital video microscopy technique, this procedure usually requires an expert visual analysis, which is prone to the inter- and intra-observer variability. Such problem claims, therefore, an automated method to detect and track these cells. To this end, we developed an approach that combines two different analyses: in the first (2D), all video frames are individually processed by using a blob-like structure detector to find the leukocyte centroids, while in the second (2D + t), a spatiotemporal image (created by stacking all video frames) is processed by a tubular-like structure detector, which is used to determine the leukocyte trajectories over time. For both analyses, the detectors are based on the relationship between Hessian matrix eigenvalues locally obtained from image sequences. Evaluation of the proposed approach was conducted by comparing our technique to the manual annotations using precision, recall and \(F_{1}\)-score measures in two video sequences. The average results for these measures were, respectively, 0.84, 0.64, and 0.72 for the first video, and 0.84, 0.87, and 0.86 for the second. These results suggested that our proposed approach is comparable with manual annotations performed by the experts and has an excellent potential for use in real circumstances. Moreover, it can reduce the observer variabilities and the burden for visual analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. Although the name vesselness makes little sense in this study, we decided to use the same name as proposed in Frangi et al. (1998) to define the measure responsible to enhance the leukocyte trajectories.

  2. Each voxel \({\varvec{x}}\) has three types of neighbors among its 26 closest neighbors; 6 face-, 12 edge-, and 8 point-neighbors, that share a face, an edge, and a point with \({\varvec{x}}\), respectively.

References

  • Acton, S. T., Wethmar, K., & Ley, K. (2002). Automatic tracking of rolling leukocytes in vivo. Microvascular Research, 63(1), 139–148.

    Article  Google Scholar 

  • Addison, W. (1843). Experimental and practical researches on the structure and function of blood corpuscles; on inflammation, and on the origin and nature of tubercles in the lungs. Transactions of the Provincial Medical and Surgical Association, London, United Kingdom, 11, 223–306.

    Google Scholar 

  • Bartko, J. J. (1991). Measurement and reliability: Statistical thinking considerations. Schizophrenia Bulletin, 17(3), 483–489.

    Article  Google Scholar 

  • Bose, P. (2000). The encoding and fourier descriptors of arbitrary curves in 3-dimensional space. Master’s thesis, University of Florida, Gainesville.

  • Cui, J., Acton, S. T., & Lin, Z. (2006). A Monte Carlo approach to rolling leukocyte tracking in vivo. Medical Image Analysis, 10(4), 598–610.

    Article  Google Scholar 

  • Deriche, R. (1990). Fast algorithms for low-level vision. IEEE Transactions on Pattern Analysis Machine Intelligence, 12(1), 78–87.

    Article  Google Scholar 

  • Deriche, R. (1993). Recursively implementating the Gaussian and its derivatives. Research report RR-1893, INRIA.

  • Dong, G., Ray, N., & Acton, S. T. (2005). Intravital leukocyte detection using the gradient inverse coefficient of variation. IEEE Transaction on Medical Imaging, 24(7), 910–924.

    Article  Google Scholar 

  • dos Santos, A. C., Roffe, E., Arantes, R. M. E., Juliano, L., Pesquero, J. L., Pesquero, J. B., et al. (2008). Kinin B2 receptor regulates chemokines CCL2 and CCL5 expression and modulates leukocyte recruitment and pathology in experimental autoimmune encephalomyelitis (EAE) in mice. Journal of Neuroinflammation, 5, 49–58.

    Article  Google Scholar 

  • Dufour, A., Tankyevych, O., Naegel, B., Talbot, H., Ronse, C., Dokládal, Baruthio P. J., et al. (2013). Filtering and segmentation of 3D angiographic data: Advances based on mathematical morphology. Medical Image Analysis, 17(2), 147–164.

    Article  Google Scholar 

  • Dzyubak, O. P., & Ritman, E. L. (2011). Automation of hessian-based tubularity measure response function in 3D biomedical images. International Journal of Biomedical Imaging, 2011(Article ID 920401), 16 pages.

  • Ebrahimdoost, Y., Qanadli, S. D., Nikravanshalmani, A., Ellis, T. J., Shojaee, Z. S., & Dehmeshki, J. (2011). Automatic segmentation of pulmonary artery (PA) in 3D pulmonary CTA images. In 17th International conference on digital signal processing (DSP) (pp. 1–5). Corfu: IEEE.

  • Eden, E., Waisman, D., Rudzsky, M., Bitterman, H., Brod, V., & Rivlin, E. (2005). An automated method for analysis of flow characteristics of circulating particles from in vivo video microscopy. IEEE Transactions on Medical Imaging, 12(8), 1011–1024.

    Article  Google Scholar 

  • Frangi, A. F., Niessen, W. J., Vincken, K. L., & Viergever, M. A. (1998). Multiscale vessel enhancement filtering. In Medical image computing and computer-assisted interventation—MICCAI (Vol. 1496, pp. 130–137). Cambridge, MA: Springer.

  • Freeman, H. (1961). On the encoding of arbitrary geometric configurations. Institute of Radio Engineers, trans on Electronic Computers EC-10:260–268.

  • Goutte, C., & Gaussier, E. (2005). A probabilistic interpretation of precision, recall and F-score, with implication for evaluation. In Advances in information retrieval (Vol. 3408, pp. 345–359). Santiago de Compostela: Springer.

  • Gregório da Silva, B. C., Carvalho-Tavares, J., & Ferrari, R. J. (2015). Detection of leukocytes in intravital video microscopy based on the analysis of Hessian matrix eigenvalues. In 28th Conference on graphics, patterns and images, SIBGRAPI, Salvador, BA (pp. 345–352).

  • Huang, Y., Liu, Z., Shi, Y., Li, N., An, X., & Gou, X. (2013). Quantitative analysis of lymphocytes morphology and motion in intravital microscopic images. In 35th Annual international conference of the IEEE engineering in medicine and biology society (EMBC) (pp. 3686–3689). Osaka: IEEE.

  • Jimenez-Carretero, D., Santos, A., Kerkstra, S., Rudyanto, R. D., & Ledesma-Carbayo, M. J. (2013). 3D frangi-based lung vessel enhancement filter penalizing airways. In IEEE 10th international symposium on biomedical imaging (pp. 926–929). San Francisco, CA: IEEE.

  • Joy, K. I. (1999). Bresenham’s algorithm, on-line computer graphics notes. Davis: Computer Science Department, University of California.

    Google Scholar 

  • Kilarski, W. W., Güç, E., Teo, J. C. M., Oliver, S. R., Lund, A. W., & Swartz, M. A. (2013). Intravital immunofluorescence for visualizing the microcirculatory and immune microenvironments in the mouse ear dermis. PLos ONE, 8(2), e57135.

    Article  Google Scholar 

  • Koller, T. M., Gerig, G., Szekely, G., & Dettwiler, D. (1995). Multiscale detection of curvilinear structures in 2-D and 3-D image data. In Fifth international conference on computer vision (pp. 864–869). Cambridge, Massachusetts: IEEE.

  • Lee, T. C., Kashyap, R. L., & Chu, C. N. (1994). Building skeleton models via 3-D medial surface axis thinning algorithms. CVGIP: Graphical Models and Image Processing, 56(6), 462–478.

    Google Scholar 

  • Li, B., & Acton, S. T. (2007). Active contour external force using vector field convolution for image segmentation. IEEE Transactions on Image Processing, 16(8), 2096–2106.

    Article  MathSciNet  Google Scholar 

  • Li, F., Yin, Z., Jin, G., Zhao, H., & Wong, S. T. C. (2013). Bioimage informatics for systems pharmacology. PLoS Computational Biology, 9(4), e1003043.

    Article  Google Scholar 

  • Lia, K., Chenb, M., Kanadea, T., Millera, E. D., Weissa, L. E., & Campbella, P. G. (2008). Cell population tracking and lineage construction with spatiotemporal context. Medical Image Analysis, 12(5), 546–566.

    Article  Google Scholar 

  • Lindeberg, T. (1998). Feature detection with automatic scale selection. International Journal of Computer Vision, 30(2), 79–116.

    Article  Google Scholar 

  • Liu, X., Lin, Z., & Acton, S. T. (2012). A grid-based Bayesian approach to robust visual tracking. Digital Signal Processing, 22(1), 54–65.

    Article  MathSciNet  Google Scholar 

  • Lorenz, C., Carlsen, I. C., Buzug, T. M., Fassnacht, C., & Weese, J. (1997). Multi-scale line segmentation with automatic estimation of width, contrast and tangential direction in 2D and 3D medical images. In Proceedings of the First joint conference on computer vision, virtual reality and robotics in medicine and medial robotics and computer-assisted surgery (pp. 233–242) London: Springer.

  • Majer, P. (2001). The influence of the gamma-parameter on feature detection with automatic scale selection. In Proceedings of the third international conference on scale-space and morphology in computer vision (pp. 245–254). Vancouver: Springer.

  • Maška, M., Ulman, V., Svoboda, D., Matula, P., Matula, P., Ederra, C., et al. (2014). A benchmark for comparison of cell tracking algorithms. Bioinformatics, 30(11), 1609–1617.

    Article  Google Scholar 

  • Meijering, E., Dzyubachyk, O., & Smal, I. (2012). Methods for cell and particle tracking (Vol. 504, pp. 183–200). Amsterdam: Elsevier.

    Google Scholar 

  • Morgenthaler, D. G. (1980). Three-dimensional digital topology: The genus. Technical Report TR-980 46, Maryland University. College Park. Computer Vision Lab., Maryland University. College Park. Computer Vision Lab. USA.

  • Mukherjee, D. P., Ray, N., & Acton, S. T. (2004). Level set analysis for leukocyte detection and tracking. IEEE Transactions on Image Processing, 13(4), 562–572.

    Article  Google Scholar 

  • Öksüz, I., Ünay, D., & Kadipaşaoğlu, K. (2012). A hybrid method for coronary artery stenoses detection and quantification in CTA images. In MICCAI workshop 3D cardiovascular imaging: A MICCAI segmentation, Nice.

  • Ortiz-de Solórzano, C., Kozubek, M., Meijering, E., Muñoz Barrutia, A., & Tomančák, P. (2014). ISBI cell tracking challenge. http://www.codesolorzano.com/Challenges/CTC/Welcome.html. Accessed 1 June 2017.

  • Otsu, N. (1979). A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man and Cybernetics, 9(1), 62–66.

    Article  Google Scholar 

  • Pinho, V., Coelho, F. M., Menezes, G. B., & Cara, D. C. (2011). Intravital microscopy to study leukocyte recruitment in vivo. In H. Chiarini-Garcia & R. C. N. Melo (Eds.), Light microscopy: Methods and protocols (Chap. 6, Vol. 689, pp. 81–90). New York: Humana Press.

  • Ray, N. (2010). A concave cost formulation for parametric curve fitting: Detection of leukocytes from intravital microscopy images. In Proceedings of the international conference on image processing (pp. 53–56). Hong Kong: IEEE.

  • Ray, N., & Acton, S. T. (2004). Motion gradient vector flow: An external force for tracking rolling leukocytes with shape and size constrained active contours. IEEE Transactions on Medical Imaging, 23(12), 1466–1478.

    Article  Google Scholar 

  • Ray, N., Acton, S. T., & Ley, K. (2002). Tracking leukocytes in vivo with shape and size constrained active contours. IEEE Transactions on Medical Imaging, 21(10), 1222–1235.

    Article  Google Scholar 

  • Sato, Y., Chen, J., Yamamoto, S., Tamura, S., Harada, N., Shiga, T., et al. (1995). Measuring microcirculation using spatiotemporal image analysis. In N. Ayache (Ed.), Computer vision, virtual reality and robotics in medicine (Vol. 905, pp. 302–308). Nice: Springer.

    Chapter  Google Scholar 

  • Sato, Y., Chen, J., Zoroofi, R. A., Harada, N., Tamura, S., & Shiga, T. (1997). Automatic extraction and measurement of leukocyte motion in microvessels using spatiotemporal image analysis. IEEE Transactions on Biomedical Engineering, 44(4), 225–236.

    Article  Google Scholar 

  • Sato, Y., Nakajima, S., Shiraga, N., Atsumi, H., Yoshida, S., Koller, T., et al. (1998). Three-dimensional multi-scale line filter for segmentation and visualization of curvilinear structures in medical images. Medical Image Analysis, 2(2), 143–168.

    Article  Google Scholar 

  • Türetken, E., Wang, X., Becker, C. J., Haubold, C., & Fua, P. (2017). Network flow integer programming to track elliptical cells in time-lapse sequences. IEEE Transactions on Medical Imaging, 36(4), 942–951.

    Article  Google Scholar 

  • Zijdenbos, A. P., Dawant, B. M., Margolin, R. A., & Palmer, A. C. (1994). Morphometric analysis of white matter lesions in MR images: Method and validation. IEEE Transactions on Medical Imaging, 13(4), 716–724.

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP) (Grant Nos. 2013/26171-6 and 2015/02232-1); the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES); and the Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq).

Author information

Authors and Affiliations

  1. Department of Computing, Federal University of São Carlos, São Carlos, SP, Brazil

    Bruno C. Gregório da Silva & Ricardo J. Ferrari

  2. Department of Physiology and Biophysics, Federal University of Minas Gerais, Belo Horizonte, MG, Brazil

    Juliana Carvalho-Tavares

Authors
  1. Bruno C. Gregório da Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Juliana Carvalho-Tavares
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ricardo J. Ferrari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ricardo J. Ferrari.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gregório da Silva, B.C., Carvalho-Tavares, J. & Ferrari, R.J. Detecting and tracking leukocytes in intravital video microscopy using a Hessian-based spatiotemporal approach. Multidim Syst Sign Process 30, 815–839 (2019). https://doi.org/10.1007/s11045-018-0581-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11045-018-0581-5

Keywords

Search

Navigation