Abstract
MotionInsights facilitates object detection and tracking from multiple video streams in real-time. The system models video processing as a stream processing pipeline. Each video frame is split into smaller blocks, which are dispatched to be processed by a number of Flink operators. Each block undergoes background subtraction and component labeling. The connected components from each frame are merged into objects. In the last stage of the pipeline, all objects from each frame are concentrated to produce the trajectory of each object. The Flink application is deployed as a Kubernetes cluster in the Google Cloud Platform. Experimenting on a 7-machine Flink cluster revealed that MotionInsights achieves up to 6x speedup compared to a non-parallel implementation while providing accurate trajectory patterns. The highest (i.e., up to 6x) speedup was observed with the highest resolution video streams.
This is a preview of subscription content, log in via an institution to check access.
References
Antonakakis, M., et al.: Real-time object detection using an ultra-high-resolution camera on embedded systems. In: IEEE International Conference on Imaging Systems and Techniques (IST 2022), pp. 1–6. Kaohsiung, Taiwan (Jun 2022). https://ieeexplore.ieee.org/document/9827742
Banelas, D.: Motion and object detection from streamingvideo on Apache Flink. Diploma thesis, Technical University of Crete (TUC), Chania, Crete, Greece (Nov 2023). https://dias.library.tuc.gr/view/97992
Carbone, P., Katsifodimos, A., Ewen, S., Markl, V., Haridi, S., Tzoumas, K.: Apache Flink: stream and Batch Processing in a Single Engine. IEEE Data Eng. Bull. 38(4), 28–38 (2015). https://api.semanticscholar.org/CorpusID:3519738
Huang, Q., et al.: SVE: distributed Video Processing at Facebook Scale. In: ACM Symposium on Operating Systems Principles (SOSP 2017), pp. 87–103. Shanghai, China (Oct 2017). https://doi.org/10.1145/3132747.3132775
Huang, Q., et al.: SVE: distributed video processing at Facebook scale (SOSP 2017). In: Proceedings of 26th Symposium on Operating Systems Principles, pp. 87–103. New York, NY, USA (Oct 2017). https://doi.org/10.1145/3132747.3132775
Kastrinakis, D., Petrakis, E.G.M.: Video2Flink: Real-Time Video Partitioning in Apache Flink and the Cloud. Machine Vision and Applications 34(3) (Apr 2023). https://doi.org/10.1007/s00138-023-01391-5
Kim, Y.K., Kim, Y., Jeong, C.S.: RIDE: real-time massive image processing platform on distributed environment. EURASIP J. Image Video Process. 39(1) (2018). https://doi.org/10.1186/s13640-018-0279-5
Kubernetes: Kubernetes - Production-Grade Container Orchestration (Jun 2022). https://kubernetes.io
Narkhede, N., Shapira, G., Palino, T.: Kafka The Definitive Guide, Real Time Data and Stream Processing at Scale. O’Reilly Media (2017). https://spark.apache.org/
Pawar, D.: GPU based background subtraction using CUDA: state of the art. In: International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET 2017), pp. 1201–1204. Chennai, India (Mar 2017). https://ieeexplore.ieee.org/document/8299953
Seinstra, F.J., Geusebroek, J.M., Koelma, D., Snoek, C.G., Worring, M., Smeulders, A.W.: High-performance distributed video content analysis with parallel-horus. IEEE MultiMedia 14(4), 64–75 (2007). https://ieeexplore.ieee.org/document/4354159
Spark: Apache Spark - Unified Engine for Large-Scale Sata Analytics (2022). https://flink.apache.org/, the Apache Software Foundation
Stauffer, C., Grimson, W.: Adaptive background mixture models for real-time tracking. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 1999), pp. 246–252. Fort Collins, Colorado, USA (Jun 1999). https://ieeexplore.ieee.org/document/784637
Uddin, M.A., Alam, A., Tu, N.A., Islam, M.S., Lee, Y.K.: SIAT: a distributed video analytics framework for intelligent video surveillance. Symmetry 11(7) (2019). https://doi.org/10.3390/sym11070911
Zafeirakopoulos, A.N., Petrakis, E.G.M.: HYAS: hybrid autoscaler agent for apache Flink. In: Web Engineering: 23rd International Conference (ICWE 2023), pp. 34–48. Alicante, Spain (Jun 2023). https://doi.org/10.1007/978-3-031-34444-2_3
Acknowledgment
We thank Prof. Nik Giatrakos of TUC for insightful comments on Apache Flink performance and to Dimitris Kastrinakis who provided us with the sources of the Video2Flink system. We are also grateful to Google for the Google Cloud Platform Education Grants program. The work has received funding from the European Union’s Horizon 2020 - Research and Innovation Framework Programme H2020-SU-SEC-2019, under Grant Agreement No 883272- BorderUAS.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Banelas, D., Petrakis, E.G.M. (2024). MotionInsights: Object Tracking in Streaming Video with Apache Flink. In: Barolli, L. (eds) Advanced Information Networking and Applications. AINA 2024. Lecture Notes on Data Engineering and Communications Technologies, vol 199. Springer, Cham. https://doi.org/10.1007/978-3-031-57840-3_37
Download citation
DOI: https://doi.org/10.1007/978-3-031-57840-3_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-57839-7
Online ISBN: 978-3-031-57840-3
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)