Abstract
Video situation monitoring is important for many applications such as infrastructure surveillance, traffic monitoring, etc. Currently, situations are monitored either manually using human-in-the-loop or custom algorithms. Manual approach was applicable for short videos. Monitoring situations in hours of long videos manually is difficult and subject to human error. On the other hand, custom algorithms are designed for specific situations and video types. A new algorithm or software package must be written for every new situation type. Both of the above approaches cannot monitor situations automatically. In this paper, we propose an alternative to the above two approaches to facilitate automated situation monitoring by posing situations as queries. The proposed approach minimizes or avoids human involvement and avoids writing new software packages or algorithms for every new situation type. The proposed framework extracts video contents only once using existing video content extraction algorithms. Appropriate data models and new operators and algorithms for efficient situation analysis are required to perform ad-hoc and what if querying on the extracted contents for situation monitoring. This paper extends the traditional relational model with support for representing various extracted content types. The Continuous Query Language (CQL) is also extended with new operators for posing situations as continuous queries. Backward compatibility, ease of use, primitive new operators (including spatial and temporal), and algorithms for efficient execution are discussed in this paper. Finally, query correctness with manual ground truth, efficiency, and the robustness of the algorithms are demonstrated.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
MavVid Dataset (2023). itlab.uta.edu/downloads/mavVid-datasets/MavVid_v2.zip
MavVidR Dataset (2023). itlab.uta.edu/downloads/mavVid-datasets/MavVidR_v2.zip
Aved, A.J., Hua, K.A.: An informatics-based approach to object tracking for distributed live video computing. Multimedia Tools Appl. 68(1), 111–133 (2014)
Bastani, F., et al.: MIRIS: fast object track queries in video. In: ACM SIGMOD, pp. 1907–1921 (2020)
Baumann, P., Misev, D., Merticariu, V., Huu, B.P.: Array databases: concepts, standards, implementations. J. Big Data 8(1), 1–61 (2021)
Chao, D., Koudas, N., Xarchakos, I.: SVQ++: querying for object interactions in video streams. In: 2020 ACM SIGMOD, pp. 2769–2772 (2020)
Dönderler, M.E., Şaykol, E., Arslan, U., Ulusoy, Ö., Güdükbay, U.: BilVideo: design and implementation of a video database management system. Multimedia Tools Appl. 27(1), 79–104 (2005)
Jiang, Q., Adaikkalavan, R., Chakravarthy, S.: MavEStream: synergistic integration of stream and event processing. In: ICDT, p. 29 (2007)
Kang, D., Bailis, P., Zaharia, M.: BlazeIt: optimizing declarative aggregation and limit queries for neural network-based video analytics. VLDB 13(4), 533–546 (2019)
Kang, D., Emmons, J., Abuzaid, F., Bailis, P., Zaharia, M.: NoScope: optimizing deep CNN-based queries over video streams at scale. PVLDB 10(11), 1586–1597 (2017)
Lerner, A., Shasha, D.: AQuery: Query language for ordered data, optimization techniques, and experiments. In: PVLDB, pp. 345–356 (2003)
Lu, C., Liu, M., Wu, Z.: SVQL: a SQL extended query language for video databases. Int. J. Database Theory Appl. 8(3), 235–248 (2015)
Mathew, S., Varia, J.: Overview of Amazon web services. Amazon Whitepapers 105, 1–22 (2014)
Redmon, J., Farhadi, A.: YOLOv3: an incremental improvement. arXiv preprint arXiv:1804.02767 (2018)
Wojke, N., Bewley, A., Paulus, D.: Simple online and realtime tracking with a deep association metric. In: IEEE ICIP, pp. 3645–3649 (2017)
Xiong, P., Zhan, H., Wang, X., Sinha, B., Wu, Y.: Visual query answering by entity-attribute graph matching and reasoning. In: CVPR, pp. 8357–8366 (2019)
Yadav, P., Curry, E.: VidCEP: complex event processing framework to detect spatiotemporal patterns in video streams. In: IEEE BigData, pp. 2513–2522. IEEE (2019)
Zhang, E., Daum, M., He, D., Haynes, B., Krishna, R., Balazinska, M.: EQUI-VOCAL: synthesizing queries for compositional video events from limited user interactions. VLDB 16(11), 2714–2727 (2023)
Zhang, S., Staudt, E., Faltemier, T., Roy-Chowdhury, A.K.: A camera network tracking (CamNeT) dataset and performance baseline. In: IEEE WACV, pp. 365–372 (2015)
Acknowledgements
This work was supported by NSF award #1955798 and #1916084. The authors would like to thank Dr. Abhishek Santra for his constructive suggestions.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Billah, H., Chakravarthy, S. (2024). Video Situation Monitoring Using Continuous Queries. In: Strauss, C., Amagasa, T., Manco, G., Kotsis, G., Tjoa, A.M., Khalil, I. (eds) Database and Expert Systems Applications. DEXA 2024. Lecture Notes in Computer Science, vol 14911. Springer, Cham. https://doi.org/10.1007/978-3-031-68312-1_10
Download citation
DOI: https://doi.org/10.1007/978-3-031-68312-1_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-68311-4
Online ISBN: 978-3-031-68312-1
eBook Packages: Computer ScienceComputer Science (R0)