Skip to main content
SpringerLink
Log in

Information-based adaptive fast-forward for visual surveillance

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Automated video analysis lacks reliability when searching for unknown events in video data. The practical approach is to watch all the recorded video data, if applicable in fast-forward mode. In this paper we present a method to adapt the playback velocity of the video to the temporal information density, so that the users can explore the video under controlled cognitive load. The proposed approach can cope with static changes and is robust to video noise. First, we formulate temporal information as symmetrized Rényi divergence, deriving this measure from signal coding theory. Further, we discuss the animated visualization of accelerated video sequences and propose a physiologically motivated blending approach to cope with arbitrary playback velocities. Finally, we compare the proposed method with the current approaches in this field by experiments and a qualitative user study, and show its advantages over motion-based measures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. IEEE Symposium on Visual Analytics Science and Technology 2009 challenge, http://hcil.cs.umd.edu/localphp/hcil/vast/index.php.

  2. “1,000 CCTV cameras to solve just one crime, Met Police admits”, 08/25/2009, www.telegraph.co.uk/news/uknews/crime/6082530/1000-CCTV-cameras-to-solve-just-one-crime-Met-Police-admits.html.

  3. www.vis.uni-stuttgart.de/~hoeferbn/AdaptiveFastForward/index.html.

  4. Imagery library for intelligent detection systems (i-LIDS). http://scienceandresearch.homeoffice.gov.uk/hosdb/cctv-imaging-technology/video-based-detection-systems/i-lids/ilids-datasets-pricing/545840.

  5. Imagery library for intelligent detection systems (i-LIDS).

References

  1. Basseville M (1988) Distance measures for signal processing and pattern recognition. Technical Report 899, Institut de Recherche En Informatique Et Systemes Aleatoires (IRISA)

  2. Blyth S (1994) Local divergence and association. Biometrika 81(3):579–584

    Article  MathSciNet  MATH  Google Scholar 

  3. Cheng K, Luo S, Chen B, Chu H (2009) Smartplayer: user-centric video fast-forwarding. In: Proceedings of the 27th international conference on human factors in computing systems (CHI). ACM, New York, pp 789–798

    Chapter  Google Scholar 

  4. Cover T, Thomas J (2006) Elements of information theory. Wiley-Interscience

  5. Fairchild MD (2005) Color appearance models. Wiley-IS&T, Chichester

    Google Scholar 

  6. Ghazal M, Amer A, Ghrayeb A (2007) A real-time technique for spatio-temporal video noise estimation. IEEE Trans Circuits Syst Video Technol 17(12):1690–1699

    Article  Google Scholar 

  7. Gill M, Spriggs A (2005) Assessing the impact of CCTV. Home office research, development and statistics directorate. Home Office Research Study 292

  8. Hero A, Ma B, Michel O, Gorman J (2002) Alpha-divergence for classification, indexing and retrieval. Technical Report CSPL-328, Communication and Signal Processing Laboratory, The University of Michigan

  9. Horn B, Schunck B (1981) Determining optical flow. Computer Vision 17:185–203

    Google Scholar 

  10. Kang H, Chen X, Matsushita Y, Tang X (2006) Space-time video montage. In: 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR), vol 2, pp 1331–1338

  11. Kolb H, Fernandez E, Nelson R (2007) Webvision: the organization of the retina and visual system. Part IX. Psychophysics of vision. National Library of Medicine (US), NCBI. Available from: http://www.ncbi.nlm.nih.gov/books/bv.fcgi?rid=webvision.part.4145

  12. Kullback S (1959) Information theory and statistics. Wiley Publication in Mathematical Statistics

  13. Lin J (1991) Divergence measures based on the Shannon entropy. IEEE Trans Inf Theory 37(1):145–151

    Article  MATH  Google Scholar 

  14. Neemuchwala H, Hero A, Carson P (2005) Image matching using alpha-entropy measures and entropic graphs. Signal Process 85(2):277–296

    Article  MATH  Google Scholar 

  15. Peker K, Divakaran A (2004) Adaptive fast playback-based video skimming using a compressed-domain visual complexity measure. In: 2004 IEEE international conference on multimedia and expo, 2004. ICME’04, vol 3, pp 2055–2058

  16. Peker K, Divakaran A, Sun H (2001) Constant pace skimming and temporal sub-sampling of video using motion activity. In: Proc. IEEE international conference on image processing (ICIP), vol 3. Thessaloniki, pp 414–417

  17. Petrovic N, Jojic N, Huang T (2005) Adaptive video fast forward. Multimed Tools Appl 26(3):327–344

    Article  Google Scholar 

  18. Poynton C (2003) Digital video and HDTV: algorithms and interfaces. Morgan Kaufmann, San Francisco

    Google Scholar 

  19. Pritch Y, Rav-Acha A, Gutman A, Peleg S (2007) Webcam synopsis: peeking around the world. In: Proc. ICCV, pp 1–8

  20. Pritch Y, Rav-Acha A, Peleg S (2008) Nonchronological video synopsis and indexing. IEEE Trans Pattern Anal Mach Intell 30(11):1971–1984

    Article  Google Scholar 

  21. Punchihewa A, Bailey D (2002) Artefacts in image and video systems; classification and mitigation. In: Proceedings of image and vision computing New Zealand, pp 197–202

  22. Ramos G, Balakrishnan R (2003) Fluid interaction techniques for the control and annotation of digital video. In: Proceedings of the 16th annual ACM symposium on user interface software and technology (UIST). ACM, New York, pp 105–114

    Chapter  Google Scholar 

  23. Rav-Acha A, Pritch Y, Peleg S (2006) Making a long video short: dynamic video synopsis. In: IEEE computer society conference on computer vision and pattern recognition (CVPR), vol 1, pp 435–441

  24. Renyi A (1961) On measures of entropy and information. In: Proceedings of the 4th Berkeley symposium on mathematical statistics and probability, vol 1, pp 547–561

  25. Schoeffmann K, Boeszoermenyi L (2009) Video browsing using interactive navigation summaries. In: International workshop on content-based multimedia indexing, vol 7, pp 243–248

  26. Scott-Brown K, Cronin P (2007) An instinct for detection: Psychological perspectives on CCTV surveillance. Police J 80(4):287–305

    Article  Google Scholar 

  27. Song B, Chun K (2003) Motion-compensated noise estimation for efficient pre-filtering in a video encoder. In: IEEE international conference on image processing (ICIP), vol 2, pp 211–214

  28. Truong BT, Venkatesh S (2007) Video abstraction: a systematic review and classification. ACM Trans Multimed Comput Comm Appl 3(1):3

    Article  Google Scholar 

Download references

Acknowledgements

We’d like to thank Michael Wörner for proofreading this paper. This work was funded by German Research Foundation (DFG) as part of the Priority Program “Scalable Visual Analytics” (SPP 1335).

Author information

Authors and Affiliations

  1. Intelligent Systems Group, Universität Stuttgart, Stuttgart, Germany

    Benjamin Höferlin & Gunther Heidemann

  2. VISUS, Universität Stuttgart, Stuttgart, Germany

    Markus Höferlin & Daniel Weiskopf

Authors
  1. Benjamin Höferlin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Markus Höferlin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Weiskopf
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gunther Heidemann
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Benjamin Höferlin.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Höferlin, B., Höferlin, M., Weiskopf, D. et al. Information-based adaptive fast-forward for visual surveillance. Multimed Tools Appl 55, 127–150 (2011). https://doi.org/10.1007/s11042-010-0606-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-010-0606-z

Keywords

Search

Navigation