Abstract
Continuous identification of objects with identical appearance is crucial to analyze the behavior of laboratory animals. Most existing methods attempt to avoid this problem by excluding direct social interactions or facilitating it by implants or artificial markers. Unfortunately, these techniques may distort the results as they can affect the behavior of the observed animals. In this paper, we present a simple, deep learning-based approach that can overcome these problems by providing reliable segmentation and tracking of similar instances. Recognition of frames where the system could not reliably locate the objects and mark them suggests human supervision is central to the system since there should be no mistake in instance tracking. Manual annotation of these data improves tracking and decreases annotation needs quickly. The proposed method achieves higher segmentation accuracy and more stable tracking than previous methods despite requiring only a small set of manually annotated data.
The research has been supported by the European Institute of Innovation and Technology. Á.D., Á.F., D.K. and A.L. were supported by Thematic Excellence Programme 2020 (TKP 2020-IKA-05) of the National Research, Development and Innovation Fund of Hungary. A.L. was supported by the Thematic Excellence Programme (Project no. ED_18-1-2019-0030 titled Application-specific highly reliable IT solutions) by the same Fund. V.V. was supported by the Ministry of Innovation and Technology NRDI Office within the framework of the Artificial Intelligence National Laboratory Program. The authors thank to Robert Bosch, Ltd. Budapest, Hungary for their generous support to the Department of Artificial Intelligence.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Our implementation is available here https://github.com/lkopi/rat_tracking.
- 2.
References
Henry, J., Wlodkowic, D.: High-throughput animal tracking in chemobehavioral phenotyping: current limitations and future perspectives. Behav. Process., 104226 (2020)
Catarinucci, L., et al.: An animal tracking system for behavior analysis using radio frequency identification. Lab Anim. 43(9), 321–327 (2014)
Porcelli, S., et al.: Social dysfunction in mood disorders and schizophrenia: clinical modulators in four independent samples. Progress Neuro Psychopharmacol. Biol. Psychiatry 99, 109835 (2020)
Toth, I., Neumann, I.D.: Animal models of social avoidance and social fear. Cell Tissue Res. 354(1), 107–118 (2013)
de Chaumont, F., et al.: Real-time analysis of the behaviour of groups of mice via a depth-sensing camera and machine learning. Nat. Biomed. Eng. 3(11), 930–942 (2019)
Mitchell, E.J., Brett, R.R., Armstrong, J.D., Sillito, R.R., Pratt, J.A.: Temporal dissociation of phencyclidine: Induced locomotor and social alterations in rats using an automated homecage monitoring system-implications for the 3Rs and preclinical drug discovery. J. Psychopharmacol., 709–715 (2020)
Peleh, T., Bai, X., Kas, M.J., Hengerer, B.: RFID-supported video tracking for automated analysis of social behaviour in groups of mice. J. Neurosci. Methods 325, 108323 (2019)
Sturman, O., et al.: Deep learning-based behavioral analysis reaches human accuracy and is capable of outperforming commercial solutions. Neuropsychopharmacology 45(11), 1942–1952 (2020)
Mathis, A.: DeepLabCut: markerless pose estimation of user-defined body parts with deep learning. Nat. Neurosci., 1281–1289 (2018)
Charpiat, G., Girard, N., Felardos, L., Tarabalka, Y.: Input similarity from the neural network perspective. In: Advances in Neural Information Processing Systems, vol. 32, pp. 1–10. Curran Associates Inc. (2019)
Kang, D., Emmons, J., Abuzaid, F., Bailis, P., Zaharia, M.: NoScope: optimizing neural network queries over video at scale. Proc. VLDB Endow. 10(11), 1586 (2017)
Bergmann, P., Meinhardt, T., Leal-Taixe, L.: Tracking without bells and whistles. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 941–951 (2019)
Fóthi, Á., Faragó, K.B., Kopácsi, L., Milacski, Z.Á., Varga, V., Lőrincz, A.: Multi object tracking for similar instances: a hybrid architecture. In: Yang, H., Pasupa, K., Leung, A.C.-S., Kwok, J.T., Chan, J.H., King, I. (eds.) ICONIP 2020. LNCS, vol. 12532, pp. 436–447. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-63830-6_37
Peters, S.M., Pinter, I.J., Pothuizen, H.H., de Heer, R.C., van der Harst, J.E., Spruijt, B.M.: Novel approach to automatically classify rat social behavior using a video tracking system. J. Neurosci. Methods 268, 163–170 (2016)
He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2961–2969 (2017)
Pont-Tuset, J., Perazzi, F., Caelles, S., Arbeláez, P., Sorkine-Hornung, A., Van Gool, L.: The 2017 DAVIS challenge on video object segmentation. arXiv preprint arXiv:1704.00675, pp. 1–6 (2017)
Kopácsi, L.: A self-supervised method for body part segmentation and keypoint detection of rat images. In: 13th Joint Conference on Mathematics and Informatics Collection of Abstracts, pp. 102–103 (2020)
Perazzi, F., Khoreva, A., Benenson, R., Schiele, B., Sorkine-Hornung, A.: Learning video object segmentation from static images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2663–2672 (2017)
Khoreva, A., Benenson, R., Ilg, E., Brox, T., Schiele, B.: Lucid data dreaming for video object segmentation. Int. J. Comput. Vision 127(9), 1175–1197 (2019)
Bookstein, F.L.: Principal warps: thin-plate splines and the decomposition of deformations. IEEE Trans. Pattern Anal. Mach. Intell. 11(6), 567–585 (1989)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2016)
Bewley, A., Ge, Z., Ott, L., Ramos, F., Upcroft, B.: Simple online and realtime tracking. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 3464–3468. IEEE (2016)
Perazzi, F., Pont-Tuset, J., McWilliams, B., Van Gool, L., Gross, M., Sorkine-Hornung, A.: A benchmark dataset and evaluation methodology for video object segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 724–732 (2016)
Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48
Dutta, A., Zisserman, A.: The VIA annotation software for images, audio and video. In: Proceedings of the 27th ACM International Conference on Multimedia, MM 2019, p. 2276. ACM (2019)
Author information
Authors and Affiliations
Contributions
L.K. and A.L. conceived and designed the research, L.K. and V.V. designed and combined augmentation procedures, L.K. and Á.F. performed computational analyses, Á.D. and D.K. provided the dataset.
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Kopácsi, L., Dobolyi, Á., Fóthi, Á., Keller, D., Varga, V., Lőrincz, A. (2021). RATS: Robust Automated Tracking and Segmentation of Similar Instances. In: Farkaš, I., Masulli, P., Otte, S., Wermter, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2021. ICANN 2021. Lecture Notes in Computer Science(), vol 12893. Springer, Cham. https://doi.org/10.1007/978-3-030-86365-4_41
Download citation
DOI: https://doi.org/10.1007/978-3-030-86365-4_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-86364-7
Online ISBN: 978-3-030-86365-4
eBook Packages: Computer ScienceComputer Science (R0)