research-article

Interaction preservation based Multi-feature fusion for video synopsis generation

Authors:

Pei WangAuthors Info & Claims

AIPR '22: Proceedings of the 2022 5th International Conference on Artificial Intelligence and Pattern Recognition

Pages 517 - 523

https://doi.org/10.1145/3573942.3574057

Published: 16 May 2023 Publication History

Abstract

Faced with a large number of surveillance videos, it will cause users to spend a lot of time browsing and retrieving videos, Video synopsis technology solves this problem, which condenses the spatiotemporally redundant long video into a compact summary video and preserves the activity information of all objects. However, the spatial and temporal displacement of objects during the synopsis process destroys the interaction between objects. This paper proposes a video synopsis method that preserves the interaction between objects. First, the depth prediction of the objects in the video is performed, and then the surveillance video is subjected to inverse perspective Mapping (IPM) to calculate the distance between objects, Finally, the depth and distance are used as the factors for judging the interaction relationship of objects, and the interaction relationship between the objects in the original video and the synopsis video is kept consistent by optimizing the interaction energy cost term. Experimental results show that the proposed method effectively preserves the interaction between objects in the synopsis.

References

[1]

B. M. Wildemuth, "How fast is too fast? Evaluating fast forward surrogates for digital video," in Proc. Joint Conf. Digit. Libraries, May 2003, pp. 221–230. https://doi.org/10.1109/JCDL.2003.1204866

[2]

A. Rav-Acha, Y. Pritch, and S. Peleg, "Making a long video short: Dynamic video synopsis," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., Jun. 2006, pp. 435–441. https://doi.org/10.1109/CVPR.2006.179

Digital Library

[3]

X. Li, Z. Wang, X. Lu, "Surveillance Video Synopsis via Scaling Down Objects," IEEE Transactions on Image Processing, vol. 25, pp. 740-755, 2016. https://doi.org/10.1109/TIP.2015.2507942

Digital Library

[4]

Y. Nie, "Collision-Free Video Synopsis Incorporating Object Speed and Size Changes," IEEE Transactions on Image Processing, vol.PP, pp. 1-1, 2019. https://doi.org/10.1109/TIP.2019.2942543

[5]

Y. Nie, C. Xiao, H. Sun, P. Li, "Compact Video Synopsis via Global Spatiotemporal Optimization," IEEE transactions on visualization and computer graphics, vol. 19, pp. 1664-1676, 2013. https://doi.org/10.1109/TVCG.2012.176

[6]

X. Li, Z. Wang, X. Lu, "Video Synopsis in Complex Situations," IEEETrans Image Process, vol. 27, pp. 3798-3812, 2018. https://doi.org/10.1109/TIP.2018.2823420

[7]

K. Namitha, A. Narayanan, "Preserving interactions among moving objects in surveillance video synopsis," Multimedia Tools and Applications, vol. pp. 1-30, 2020. https://doi.org/10.1007/s11042-020-09493-2

Digital Library

[8]

Tian, Y., Zheng, H., Chen, Q., Wang, D., Lin, R.: Surveillance video synopsis generation method via keeping important relationship among objects. IET Comput. Vis. 10, 868–872 (2016). https:// doi.org/10.1049/iet-cvi.2016.0128

Digital Library

[9]

Shariq Farooq Bhat, Ibraheem Alhashim, and Peter Wonka. Adabins: Depth estimation using adaptive bins. In CVPR, pages 4009–4018, June 2021. https://doi.org/10.1109/CVPR46437.2021.00400

[10]

Q. Zhang, K. Dang, J.-H. Lai, Z. Feng, X. Xie, "Modeling 3D Layout For Group Re-Identification," 2022. https://openaccess.thecvf.com

[11]

E. T. Hall, "A system for the notation of proxemic behavior," Amer. Anthropologist, vol. 65, no. 5, pp. 1003–1026, Oct. 1963. http://garfield.library.upenn.edu/classics1989/

[12]

M. Rezaei, M. A. J. A. Sciences," DeepSOCIAL: Social Distancing Monitoring and Infection Risk Assessment in COVID-19 Pandemic," vol. pp. 2020. https://doi.org/10.3390/app10217514

[13]

Branch HOSD (2006) Imagery library for intelligent detection systems (i-lids). In: 2006 IET Conference on Crime and Security. IET, pp 445–448

[14]

Z. Zhang, " Multi-View Video Synopsis via Simultaneous Object-Shifting and View-Switching Optimization," IEEE Transactions onImage Processing, vol. 29, pp. 971-985, 2020. https://doi.org/10.1109/TIP.2019.2938086

Digital Library

[15]

Y. Pritch, A. Rav-Acha, S. Peleg, " Nonchronological Video Synopsis and Indexing," IEEE Transactions on Pattern Analysis and MachineIntelligence, vol. 30, pp. 1971-1984, 2008. https://doi.org/10.1109/TPAMI.2008.29

Digital Library

Cited By

Wang CYang KWang PZhu ZZhang LWang F(2024)Video synopsis method based on interaction determination using human pose estimationJournal of Electronic Imaging10.1117/1.JEI.33.1.01305333:01Online publication date: 1-Jan-2024
https://doi.org/10.1117/1.JEI.33.1.013053
Sood VRay SMahato KV SNamitha K(2024)A Graph-Based Approach for Estimating Object Interactions in Surveillance Video Synopsis2024 Second International Conference on Emerging Trends in Information Technology and Engineering (ICETITE)10.1109/ic-ETITE58242.2024.10493348(1-6)Online publication date: 22-Feb-2024
https://doi.org/10.1109/ic-ETITE58242.2024.10493348

Index Terms

Interaction preservation based Multi-feature fusion for video synopsis generation
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
        Video summarization

Recommendations

Object interaction-based surveillance video synopsis
Abstract
With the exponential growth of surveillance videos, quickly retrieving, browse and storing videos has become an urgent problem to be settled down. Video synopsis technology is an effective way to solve this problem. However, the existing video ...
Interactive visualization-based surveillance video synopsis
Abstract
Video synopsis is an effective technique for the efficient analysis of long videos in a short time. To generate a compact video, multiple tracks of moving objects, which we call as tubes are displayed simultaneously by rearranging them along the ...
Surveillance video synopsis framework base on tube set
Abstract
Video synopsis technology can shorten the length of the video, which has attracted wide attention. However, due to the limitation of object extraction technology and the difficulty of preserving interactivity, synopsis videos will lose the ...
Highlights
- We combine Yolov4 and Kalman Filter for object tracking to generate object tubes.
- We use dynamic threshold and moving direction to preserve interactivity.
- We propose a tube set mapping model for rearranging tube sets.

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

AIPR '22: Proceedings of the 2022 5th International Conference on Artificial Intelligence and Pattern Recognition

September 2022

1221 pages

ISBN:9781450396899

DOI:10.1145/3573942

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 May 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

AIPR 2022

AIPR 2022: 2022 5th International Conference on Artificial Intelligence and Pattern Recognition

September 23 - 25, 2022

Xiamen, China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
31
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)2

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang CYang KWang PZhu ZZhang LWang F(2024)Video synopsis method based on interaction determination using human pose estimationJournal of Electronic Imaging10.1117/1.JEI.33.1.01305333:01Online publication date: 1-Jan-2024
https://doi.org/10.1117/1.JEI.33.1.013053
Sood VRay SMahato KV SNamitha K(2024)A Graph-Based Approach for Estimating Object Interactions in Surveillance Video Synopsis2024 Second International Conference on Emerging Trends in Information Technology and Engineering (ICETITE)10.1109/ic-ETITE58242.2024.10493348(1-6)Online publication date: 22-Feb-2024
https://doi.org/10.1109/ic-ETITE58242.2024.10493348

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten