Skip to main content
SpringerLink
Log in

Extracting representative motion flows for effective video retrieval

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this paper, we propose a novel motion-based video retrieval approach to find desired videos from video databases through trajectory matching. The main component of our approach is to extract representative motion features from the video, which could be broken down to the following three steps. First, we extract the motion vectors from each frame of videos and utilize Harris corner points to compensate the effect of the camera motion. Second, we find interesting motion flows from frames using sliding window mechanism and a clustering algorithm. Third, we merge the generated motion flows and select representative ones to capture the motion features of videos. Furthermore, we design a symbolic based trajectory matching method for effective video retrieval. The experimental results show that our algorithm is capable to effectively extract motion flows with high accuracy and outperforms existing approaches for video retrieval.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Youtube. http://www.youtube.com/

  2. Bashir FI, Khokhar AA, Schonfeld D (2007) Object trajectory-based activity classification and recognition using hidden markov models. IEEE Trans Image Process 16(7):1912–1919

    Article  MathSciNet  Google Scholar 

  3. Bashir FI, Khokhar AA, Schonfeld D (2007) Real-time motion trajectory-based indexing and retrieval of video sequences. IEEE Trans Multimedia 9(1):58–65

    Article  Google Scholar 

  4. Chang SF, Chen W, Meng HJ, Sundaram H, Zhong D (1998) A fully automated content-based video search engine supporting spatiotemporal queries. IEEE Trans Circuits Syst Video Technol 8(3):602–615

    Article  Google Scholar 

  5. Chen L, Özsu MT, Oria V (2004) Symbolic representation and retrieval of moving object trajectories. In: the 6th ACM multimedia workshop on MIR, pp 227–234

  6. Chen L, Özsu MT, Oria V (2005) Robust and fast similarity search for moving object trajectories. In: Proc. of ACM SIGMOD conference, pp 491–502

  7. Cucchiara R, Grana C, Piccardi M, Prati A (2003) Detecting moving objects, ghosts, and shadows in video streams. IEEE Trans Pattern Anal Mach Intell 25(10):1337–1342

    Article  Google Scholar 

  8. Dagtas S, Al-Khatib W, Ghafoor A, Kashyap RL (2000) Models for motion-based video indexing and retrieval. IEEE Trans Image Process 9(1):88–101

    Article  Google Scholar 

  9. Deng Y, Manjunath BS (1998) Netra-V: toward an object-based video representation. IEEE Trans Circuits Syst Video Technol 8(5):616–627

    Article  Google Scholar 

  10. Fablet R, Bouthemy P, Perez P (2002) Nonparametric motion characterization using causal probabilistic models for video indexing and retrieval. IEEE Trans Image Process 11(4):393–407

    Article  Google Scholar 

  11. Flickner M, Niblack HW, Ashley J, Huang Q, Dom B, Gorkani M, Hafner J, Lee D, Petkovic D, Steele D, Yanker P (1995) Query by image and video content: the QBIC system. Comput 28(9):23–32

    Article  Google Scholar 

  12. Hamrapur A, Gupta A, Horowitz B, Shu CF, Fuller C, Bach J, Gorkani M, Jain R (1997) Virage video engine. In: SPIE proc. storage and retrieval for image and video databases V, pp 188–197

  13. Harris CG, Stephens MJ (1988) A combined corner and edge detector. In: Proc. of 4th Alvey vision conference, pp 147–151

  14. Hsieh J-W, Yu S-L, Chen Y-S (2006) Motion-based video retrieval by trajectory matching. IEEE Trans Circuits Syst Video Technol 16:396–409

    Article  Google Scholar 

  15. Keogh E, Chu S, Hart D, Pazzani M (2004) Segmenting time series: a survey and novel approach. In: Data mining in time series databases. World Scientific

  16. Keogh EJ, Chu S, Hart D, Pazzani MJ (2001) An online algorithm for segmenting time series. In: Proc. of ICDM conference, pp 289–296

  17. Le T-L, Boucher A, Thonnat M (2007) Subtrajectory-based video indexing and retrieval. In: Proc. of MMM conference, pp 418–427

  18. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  19. Ma Y-F, Zhang H-J (2002) Motion texture: a new motion based video representation. In: Proc. of ICPR conference, pp 548–551

  20. Manjunath BS, Salembier P, Sikora T (2002) Introduction to mpeg-7: multimedia content description interface. In: Proc. of ICPR conference, pp 548–551

  21. Rath GB, Makur A (1999) Iterative least squares and compression based estimations for a four-parameter linear global motion model and global motion compensation. IEEE Trans Circuits Syst Video Technol 9:1075–1099

    Article  Google Scholar 

  22. Sivic J, Schaffalitzky F, Zisserman A (2004) Object level grouping for video shots. In: Proc. of ECCV conference, pp 85–98

  23. Su C-W, Liao H-Y, Tyan H-R, Lin C-W, Chen D-Y, Fan K-C (2007) Motion flow-based video retrieval. IEEE Trans Multimedia 9(6):1193–1201

    Article  Google Scholar 

  24. Tomasi C, Kanade T (1991) Detection and tracking of point features. Carnegie Mellon University Technical Report, pp 864–975

  25. Tsaig Y, Averbuch A (2002) Automatic segmentation of moving objects in video sequences: a region labeling approach. IEEE Trans Circuits Syst Video Technol 12(7):597–612

    Article  Google Scholar 

  26. Wang F, Jiang Y-G, Ngo C-W (2008) Video event detection using motion relativity and visual relatedness. In: Proc. of ACM MM conference, pp 239–248

  27. Wu X, Takimoto M, Satoh S, Adachi J (2008) Scene duplicate detection based on the pattern of discontinuities in feature point trajectories. In: Proc. of ACM MM conference, pp 51–60

  28. Yilmaz A, Javed O, Shah M (2006) Object tracking: a survey. ACM Comput Surv 38(4):13

    Article  Google Scholar 

  29. Zhu G, Liang D, Liu Y, Huang Q, Gao W (2005) Improving particle filter with support vector regression for efficient visual tracking. In: Proc. of ICIP conference, pp 422–425

  30. Avrithis YS, Doulamis AD, Doulamis ND, Kollias SD (1999) A stochastic framework for optimal key frame extraction from mpeg video databases. Comput Vis Image Underst 75:3–24

    Article  Google Scholar 

  31. Lertrusdachakul T, Aoki T, Yasuda H (2005) Camera motion characterization through image feature analysis. In: Proc. of ICCIMA conference

  32. Yeo B-L, Liu B (1995) Rapid scene analysis on compressed video. IEEE Trans Circuits Syst Video Technol 5:533–544

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by the National Natural Science foundation of China under Grant No.60933004, 60811120098 and 61073019, and Grant SKLSDE-2010KF-03.

Author information

Authors and Affiliations

  1. State Key Laboratory of Software Development Environment & Department of Computer Science, Peking University, Beijing, China

    Zhe Zhao & Bin Cui

  2. Nanyang Technological University, Nanyang, Singapore

    Gao Cong

  3. The University of Queensland, Queensland, Australia

    Zi Huang & Heng Tao Shen

Authors
  1. Zhe Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bin Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gao Cong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zi Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Heng Tao Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bin Cui.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhao, Z., Cui, B., Cong, G. et al. Extracting representative motion flows for effective video retrieval. Multimed Tools Appl 58, 687–711 (2012). https://doi.org/10.1007/s11042-011-0763-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-011-0763-8

Keywords

Search

Navigation