Abstract
Animal tracking is a crucial aspect of animal phenotyping, and industries are using computer vision-based methods to enhance their products. In this paper, we adopt the tracking-by-detection approach and propose a self-supervised framework for multiple animal tracking. Self-supervised learning techniques have recently been employed to train models using unlabeled data and have demonstrated improved accuracy on benchmark datasets. Our proposed framework utilizes an EfficientDet detector that was pre-trained with self-supervised learning using a modified Barlow twins method. The detected animals are associated with tracks using our proposed variant of Deepsort, which utilizes appearance information to improve the detection-to-track association. We trained and tested the framework on a customized dataset from a Norwegian pig farm, which consisted of four test and four train sequences, as well as a detection dataset containing 1674 labelled frames and 3000 unlabeled images for self-supervised learning. To evaluate the performance of our framework, we used standard tracking metrics such as HOTA (Higher order tracking accuracy), MOTA (Multiple object tracking accuracy), and IDF1 (Identification metrics). The implementation of our framework is publicly available at https://github.com/DeVcB13d/Animal_tracking_with_ssl.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Herlin, A., Brunberg, E., Hultgren, J., Högberg, N., Rydberg, A., Skarin, A.: Animal welfare implications of digital tools for monitoring and management of cattle and sheep on pasture. Animals 11(3), 829 (2021)
Afridi, H., Ullah, M., Nordbø, Ø., Alaya Cheikh, F., Guro Larsgard, A.: Optimized deep-learning-based method for cattle udder traits classification. Mathematics 10(17), 3097 (2022)
Pham-Duc, T., et al.: Improvement on mechanics attention deep learning model for classification ear-tag of swine. In; 2022 9th NAFOSTED Conference on Information and Computer Science (NICS), pp. 345–350. IEEE (2022)
Xue, X., Henderson, T.C.: Video-based animal behavior analysis from multiple cameras. In: 2006 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, pp. 335–340 (2006)
Okinda, C., et al.: A review on computer vision systems in monitoring of poultry: A welfare perspective. Artif. Intell. Agricult. 4, 184–208 (2020)
Kays, R., Crofoot, M.C., Jetz, W., Wikelski, M.: Terrestrial animal tracking as an eye on life and planet. Science, 348(6240):aaa2478 (2015)
Kresovic, M., Nguyen, T., Ullah, M., Afridi, H., Alaya Cheikh, F.: Pigpose: A realtime framework for farm animal pose estimation and tracking. In: Artificial Intelligence Applications and Innovations: 18th IFIP WG 12.5 International Conference, AIAI 2022, Hersonissos, Crete, June 17–20, 2022, Proceedings, Part I, pp. 204–215. Springer (2022). https://doi.org/10.1007/978-3-031-08333-4_17
Daud Khan, S., et al.: An efficient deep learning framework for face mask detection in complex scenes. In: Artificial Intelligence Applications and Innovations, pp. 159–169. Springer (2022). https://doi.org/10.1007/978-3-031-08333-4_13
Mamadou, K., Ullah, M., Nordbø, Ø., Alaya Cheikh, F.: Multi-encoder convolution block attention model for binary segmentation. In: 2022 International Conference on Frontiers of Information Technology (FIT), pp. 183–188. IEEE (2022)
Wojke, N., Bewley, A., Paulus, D.: Simple online and realtime tracking with a deep association metric. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 3645–3649. IEEE (2017)
Ullah, M., Alaya Cheikh, F.: A directed sparse graphical model for multi-target tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1816–1823 (2018)
Bewley, A., Ge, Z., Ott, L., Ramos, F., Upcroft, B.: Simple online and realtime tracking. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 3464–3468. IEEE (2016)
Ullah, M., Alaya Cheikh, F.: Deep feature based end-to-end transportation network for multi-target tracking. In: 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 3738–3742. IEEE (2018)
Zhang, Y., et al.: Bytetrack: Multi-object tracking by associating every detection box. In: Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXII, pp. 1–21. Springer (2022). https://doi.org/10.1007/978-3-031-20047-2_1
Quddus Khan, A., Khan, S., Ullah, M., Alaya Cheikh, F.: A bottom-up approach for pig skeleton extraction using RGB data. In: International Conference on Image and Signal Processing, pp. 54–61. Springer (2020). https://doi.org/10.1007/978-3-030-51935-3_6
Jia, Y., et al.: Selfee, self-supervised features extraction of animal behaviors. Elife 11, e76218 (2022)
Ullah, M., Shagdar, Z., Ullah, H., Alaya Cheikh, F.: Semi-supervised principal neighbourhood aggregation model for SAR image classification. In: 2022 16th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS), pp. 211–217. IEEE (2022)
Chen, B., Li, P., Chen, X., Wang, B., Zhang, L., Hua, X.-S.: Dense learning based semi-supervised object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4815–4824 (2022)
Tang, P., Ramaiah, C., Wang, Y., Xu, R., Xiong, C.: Proposal learning for semi-supervised object detection. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 2291–2301 (2021)
Xu, M., et al.: End-to-end semi-supervised object detection with soft teacher. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 3060–3069 (2021)
Zbontar, J., Jing, L., Misra, I., LeCun, Y., Deny, S.: Barlow twins: Self-supervised learning via redundancy reduction. In: International Conference on Machine Learning, pp. 12310–12320. PMLR (2021)
Wojke, N., Bewley, A.: Deep cosine metric learning for person re-identification. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 748–756. IEEE (2018)
Tan, M., Pang, R., Le, Q.V.: Efficientdet: Scalable and efficient object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10781–10790 (2020)
Ullah, E., Ullah, M., Sajjad, M., Alaya Cheikh, F.: Deep learning based wheat ears count in robot images for wheat phenotyping. Electron. Imag. 34, 1–6 (2022)
Tan, M., Le, Q.V.: Efficientnet: Rethinking model scaling for convolutional neural networks. arXiv preprint arXiv:1905.11946 (2019)
Bodla, N., Singh, B., Chellappa, R., Davis, L.S.: Soft-NMS-improving object detection with one line of code. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5561–5569 (2017)
Kalman, R.E. A new approach to linear filtering and prediction problems (1960)
Dendorfer, P., et al.: Mot20: A benchmark for multi object tracking in crowded scenes. arXiv preprint arXiv:2003.09003 (2020)
CVAT.ai Corporation. Computer Vision Annotation Tool (CVAT) 9 (2022)
Milan, A., et al.: Mot16: A benchmark for multi-object tracking. arXiv preprint arXiv:1603.00831 (2016)
Darwin V7 Labs. https://darwin.v7labs.com/
Brooks, J.: COCO Annotator. https://github.com/jsbroks/coco-annotator/ (2019)
Bernardin, K., Stiefelhagen, R.: Evaluating multiple object tracking performance: The clear mot metrics. EURASIP J. Image Video Process. 1–10, 2008 (2008)
Luiten, J., et al.: Hota: A higher order metric for evaluating multi-object tracking. Int. J. Comput. Vis. 129(2), 548–578 (2021)
Paszke, et al.: Pytorch: An imperative style, high-performance deep learning library. In: Advances in Neural Information Processing Systems, vol. 32, pp. 8024–8035. Curran Associates Inc. (2019)
Bradski, G.: The OpenCV Library. Dr. Dobb’s Journal of Software Tools (2000)
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Li, F.-F.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009)
Tan, M., Le, Q.V.: Efficientnet: Rethinking model scaling for convolutional neural networks. arXiv preprint arxiv:1905.11946 (2019)
Acknowledgment
We would like to thank Norsvin SA for sharing data and the Research Council of Norway for funding this study, within the BIONÆR program, project numbers 282252 and 321409. In special, we would also like to thank Rune Sagevik, Norsvin SA for the image acquisition.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 IFIP International Federation for Information Processing
About this paper
Cite this paper
Dev Narayan, C.B. et al. (2023). Tracking-by-Self Detection: A Self-supervised Framework for Multiple Animal Tracking. In: Maglogiannis, I., Iliadis, L., MacIntyre, J., Dominguez, M. (eds) Artificial Intelligence Applications and Innovations. AIAI 2023. IFIP Advances in Information and Communication Technology, vol 675. Springer, Cham. https://doi.org/10.1007/978-3-031-34111-3_47
Download citation
DOI: https://doi.org/10.1007/978-3-031-34111-3_47
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-34110-6
Online ISBN: 978-3-031-34111-3
eBook Packages: Computer ScienceComputer Science (R0)
