Abstract
Automated video analysis lacks reliability when searching for unknown events in video data. The practical approach is to watch all the recorded video data, if applicable in fast-forward mode. In this paper we present a method to adapt the playback velocity of the video to the temporal information density, so that the users can explore the video under controlled cognitive load. The proposed approach can cope with static changes and is robust to video noise. First, we formulate temporal information as symmetrized Rényi divergence, deriving this measure from signal coding theory. Further, we discuss the animated visualization of accelerated video sequences and propose a physiologically motivated blending approach to cope with arbitrary playback velocities. Finally, we compare the proposed method with the current approaches in this field by experiments and a qualitative user study, and show its advantages over motion-based measures.
Similar content being viewed by others
Notes
IEEE Symposium on Visual Analytics Science and Technology 2009 challenge, http://hcil.cs.umd.edu/localphp/hcil/vast/index.php.
“1,000 CCTV cameras to solve just one crime, Met Police admits”, 08/25/2009, www.telegraph.co.uk/news/uknews/crime/6082530/1000-CCTV-cameras-to-solve-just-one-crime-Met-Police-admits.html.
Imagery library for intelligent detection systems (i-LIDS). http://scienceandresearch.homeoffice.gov.uk/hosdb/cctv-imaging-technology/video-based-detection-systems/i-lids/ilids-datasets-pricing/545840.
Imagery library for intelligent detection systems (i-LIDS).
References
Basseville M (1988) Distance measures for signal processing and pattern recognition. Technical Report 899, Institut de Recherche En Informatique Et Systemes Aleatoires (IRISA)
Blyth S (1994) Local divergence and association. Biometrika 81(3):579–584
Cheng K, Luo S, Chen B, Chu H (2009) Smartplayer: user-centric video fast-forwarding. In: Proceedings of the 27th international conference on human factors in computing systems (CHI). ACM, New York, pp 789–798
Cover T, Thomas J (2006) Elements of information theory. Wiley-Interscience
Fairchild MD (2005) Color appearance models. Wiley-IS&T, Chichester
Ghazal M, Amer A, Ghrayeb A (2007) A real-time technique for spatio-temporal video noise estimation. IEEE Trans Circuits Syst Video Technol 17(12):1690–1699
Gill M, Spriggs A (2005) Assessing the impact of CCTV. Home office research, development and statistics directorate. Home Office Research Study 292
Hero A, Ma B, Michel O, Gorman J (2002) Alpha-divergence for classification, indexing and retrieval. Technical Report CSPL-328, Communication and Signal Processing Laboratory, The University of Michigan
Horn B, Schunck B (1981) Determining optical flow. Computer Vision 17:185–203
Kang H, Chen X, Matsushita Y, Tang X (2006) Space-time video montage. In: 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR), vol 2, pp 1331–1338
Kolb H, Fernandez E, Nelson R (2007) Webvision: the organization of the retina and visual system. Part IX. Psychophysics of vision. National Library of Medicine (US), NCBI. Available from: http://www.ncbi.nlm.nih.gov/books/bv.fcgi?rid=webvision.part.4145
Kullback S (1959) Information theory and statistics. Wiley Publication in Mathematical Statistics
Lin J (1991) Divergence measures based on the Shannon entropy. IEEE Trans Inf Theory 37(1):145–151
Neemuchwala H, Hero A, Carson P (2005) Image matching using alpha-entropy measures and entropic graphs. Signal Process 85(2):277–296
Peker K, Divakaran A (2004) Adaptive fast playback-based video skimming using a compressed-domain visual complexity measure. In: 2004 IEEE international conference on multimedia and expo, 2004. ICME’04, vol 3, pp 2055–2058
Peker K, Divakaran A, Sun H (2001) Constant pace skimming and temporal sub-sampling of video using motion activity. In: Proc. IEEE international conference on image processing (ICIP), vol 3. Thessaloniki, pp 414–417
Petrovic N, Jojic N, Huang T (2005) Adaptive video fast forward. Multimed Tools Appl 26(3):327–344
Poynton C (2003) Digital video and HDTV: algorithms and interfaces. Morgan Kaufmann, San Francisco
Pritch Y, Rav-Acha A, Gutman A, Peleg S (2007) Webcam synopsis: peeking around the world. In: Proc. ICCV, pp 1–8
Pritch Y, Rav-Acha A, Peleg S (2008) Nonchronological video synopsis and indexing. IEEE Trans Pattern Anal Mach Intell 30(11):1971–1984
Punchihewa A, Bailey D (2002) Artefacts in image and video systems; classification and mitigation. In: Proceedings of image and vision computing New Zealand, pp 197–202
Ramos G, Balakrishnan R (2003) Fluid interaction techniques for the control and annotation of digital video. In: Proceedings of the 16th annual ACM symposium on user interface software and technology (UIST). ACM, New York, pp 105–114
Rav-Acha A, Pritch Y, Peleg S (2006) Making a long video short: dynamic video synopsis. In: IEEE computer society conference on computer vision and pattern recognition (CVPR), vol 1, pp 435–441
Renyi A (1961) On measures of entropy and information. In: Proceedings of the 4th Berkeley symposium on mathematical statistics and probability, vol 1, pp 547–561
Schoeffmann K, Boeszoermenyi L (2009) Video browsing using interactive navigation summaries. In: International workshop on content-based multimedia indexing, vol 7, pp 243–248
Scott-Brown K, Cronin P (2007) An instinct for detection: Psychological perspectives on CCTV surveillance. Police J 80(4):287–305
Song B, Chun K (2003) Motion-compensated noise estimation for efficient pre-filtering in a video encoder. In: IEEE international conference on image processing (ICIP), vol 2, pp 211–214
Truong BT, Venkatesh S (2007) Video abstraction: a systematic review and classification. ACM Trans Multimed Comput Comm Appl 3(1):3
Acknowledgements
We’d like to thank Michael Wörner for proofreading this paper. This work was funded by German Research Foundation (DFG) as part of the Priority Program “Scalable Visual Analytics” (SPP 1335).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Höferlin, B., Höferlin, M., Weiskopf, D. et al. Information-based adaptive fast-forward for visual surveillance. Multimed Tools Appl 55, 127–150 (2011). https://doi.org/10.1007/s11042-010-0606-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-010-0606-z