Skip to main content
SpringerLink
Log in

Motion Trajectory-Based Video Retrieval, Classification, and Summarization

  • Chapter
Book cover Video Search and Mining

Part of the book series: Studies in Computational Intelligence ((SCI,volume 287))

Abstract

This chapter provides an overview of various methods for motion trajectory-based video contentmodeling, retrieval and classification. The techniques discussed form the foundation for content-based video indexing and retrieval (CBVIR) systems. We focus on view-invariant representations of single and multiple motion trajectories based on null-space invariants that allows for video retrieval and classification from unknown and moving camera views. We introduce methods based on matrix and tensor decomposition for efficient storage and retrieval of single and multiple motion trajectories, respectively. We subsequently explore the use of one- and multi-dimensional hidden Markov models for video classification and recognition based on single and multiple motion trajectories. We summarize the basic concepts and present computer simulation results to demonstrate the fundamental notions introduced throughout the chapter.We finally discuss several open problems in the field of motion trajectory analysis and future trends in content-based video modeling, retrieval and classification.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Yuan, J., Wang, H., Zheng, W., Li, J., Lin, F., Zhang, B.: A Formal Study of Shot Boundary Detection. IEEE Transactions on Circuit and Systems for Video Technology, 168–186 (2007)

    Google Scholar 

  2. Hanjalic, A.: Shot-boundary detection: unraveled or resolved? IEEE Transactions on Circuit and Systems for Video Technoligy 12(2), 90–105 (2002)

    Article  Google Scholar 

  3. Lienhart, R.: Reliable Transition Detection in Videos: A Survey and Practitioner’s Guide. International Journal of Image and Graphics 1, 469–486 (2001)

    Article  Google Scholar 

  4. Lelescu, D., Schonfeld, D.: Statistical sequential analysis for real-time scene hange detection on compressed multimedia bitstream. IEEE Transactions on Multimedia 5, 106–117 (2003)

    Article  Google Scholar 

  5. Johansson, G.: Visual Perception of Biological Motion and a Model for its Analysis. Perception and Psychophysics 14(2), 201–211 (1973)

    Google Scholar 

  6. Yilmaz, A., Javed, O., Shah, M.: Object tracking: A survey. ACM Comput. Surv. 38(4), 45 (2006)

    Article  Google Scholar 

  7. Zhao, T., Nevatia, R.: Tracking multiple humans in crowded environment. In: Proc. IEEE Int. Conf. Compt. Vision and Pattern Recognit., vol. 2, pp. 406–413 (2004)

    Google Scholar 

  8. Chang, C., Ansari, R., Khokhar, A.: Multiple Object Tracking with Kernal Particle Filter. In: Proc. IEEE Int. Conf. Compt. Vision and Pattern Recognit., vol. 1, pp. 566–573 (2005)

    Google Scholar 

  9. Schonfeld, D., Lelescu, D.: VORTEX: Video retrieval and tracking from compressed multimedia databases-multiple object tracking from MPEG-2 bitstream. Journal of Visual Communications and Image Representation, Special Issue on Multimedia Database Management 11, 154–182 (2000) (invited paper)

    Google Scholar 

  10. Hariharakrishnan, K., Schonfeld, D.: Fast object tracking using adaptive block matching. IEEE Transactions on Multimedia 7(5), 853–859 (2005)

    Article  Google Scholar 

  11. Isard, M., Blake, A.: Condensation-Conditional Density Propagation for Visual Tracking. International Journal of Computer Vision 29(1) (1998)

    Google Scholar 

  12. Qu, W., Schonfeld, D., Mohamed, M.: Real-time distributed multi-object tracking using multiple interactive trackers and a magnetic-inertia potential model. IEEE Transactions on Multimedia 9, 511–519 (2007)

    Article  Google Scholar 

  13. Chang, S.F., Chen, W., Meng, H.J., Sundaram, H., Zhong, D.: A Fully Automated Content-Based Video Search Engine Supporting Spatiotemporal Queries. IEEE Trans. Circuits Syst. Video Technol. 8(5), 602–615 (1998)

    Article  Google Scholar 

  14. AbouGhazaleh, N., Gamal, Y.E.: Compressed Video Indexing Based on Object’s Motion. In: Int. Conf. Visual Communication and Image Processing, VCIP 2000, Perth, Australia, pp. 986–993 (2000)

    Google Scholar 

  15. Katz, B., Lin, J., Stauffer, C., Grimson, E.: Answering questions about moving objects in surveillance videos. In: Proceedings of 2003 AAAI Spring Symp. New Directions in Question Answering, Palo Alto, CA (2003)

    Google Scholar 

  16. Rea, N., Dahyot, R., Kokaram, A.: Semantic Event Detection in Sports Through Motion Understanding. In: Enser, P.G.B., Kompatsiaris, Y., O’Connor, N.E., Smeaton, A., Smeulders, A.W.M. (eds.) CIVR 2004. LNCS, vol. 3115, pp. 21–23. Springer, Heidelberg (2004)

    Google Scholar 

  17. Bashir, F.I., Khokhar, A.A., Schonfeld, D.: A Hybrid System for Affine- Invariant Trajectory Retrieval. In: Proceedings of the 6th ACM SIGMM International Workshop on Multimedia Information Retrieval, New York (2004)

    Google Scholar 

  18. Bashir, F.I., Khokhar, A.A., Schonfeld, D.: Object trajectory-based activity classification and recognition using hidden Markov models. IEEE Transactions on Image Processing 16, 1912–1919 (2007)

    Article  MathSciNet  Google Scholar 

  19. Bashir, F.I., Khokhar, A.A., Schonfeld, D.: Real-time motion trajectory-based indexing and retrieval of video sequences. IEEE Transactions on Multimedia 9, 58–65 (2007)

    Article  Google Scholar 

  20. Bashir, F.I., Khokhar, A.A., Schonfeld, D.: Real-time affine-invariant motion trajectory-based retrieval and classification of video sequences from arbitrary camera view. ACM Multimedia Systems Journal, Special Issue on Machine Learning Approaches to Multimedia Information Retrieval 12, 45–54 (2006)

    Google Scholar 

  21. The University of California at Irvine Knowledge Discovery in Databases (KDD) archive, http://kdd.ics.uci.edu

  22. Fawcett, T.: Roc graphs: Notes and practical considerations for researchers, Technical Report, HP Labs, HPL-2003-4 (2004)

    Google Scholar 

  23. Vaswani, N., Chellappa, R.: Principal Components Null Space Analysis for Image and Video Classification. IEEE Trans. Image Processing (July 2006)

    Google Scholar 

  24. Chen, X., Schonfeld, D., Khokhar, A.: Robust null space representation and sampling for view invariant motion trajectory analysis. In: IEEE Conference on Computer Vision and Pattern Recognition (2008)

    Google Scholar 

  25. Chen, X., Schonfeld, D., Khokhar, A.: Localized Null Space Representation for Dynamic Updating and Downdating in Image and Video Databases. In: IEEE International Conference on Image Processing, ICIP (2009)

    Google Scholar 

  26. Sahouria, E., Zakhor, A.: A Trajectory Based Video Indexing System For Street Surveillance. In: IEEE Int. Conf. on Image Processing (ICIP), pp. 24–28 (1999)

    Google Scholar 

  27. Chen, W., Chang, S.F.: Motion Trajectory Matching of Video Objects. In: IS&T/ SPIE, pp. 544–553 (2000)

    Google Scholar 

  28. Bashir, F.I., Khokhar, A.A., Schonfeld, D.: Segmented trajectory based indexing and retrieval of video data. In: Proc. IEEE Int. Conf. Image Processing, pp. 623–626 (2003)

    Google Scholar 

  29. Ma, X., Bashir, F., Knokhar, A., Schonfeld, D.: Event Analysis Based on Multiple Interactive Motion Trajectories. IEEE Trans. on Circuits and Syst. for Video Technology 19(3) (2009)

    Google Scholar 

  30. Ma, X., Bashir, F., Knokhar, A., Schonfeld, D.: Tensor-based Multiple Object Trajectory Indexing and Retrieval. In: Proc. IEEE Int. Conf. on Multimedia and Expo. (ICME), Toronto, Canada, pp. 341–344 (2006)

    Google Scholar 

  31. Lathauwer, L., Moor, B.D., Vandewalle, J.: A multilinear singular value decomposition. SIAM Journal on Matrix Analysis and Applicat. (SIMAX) 21(4), 1253–1278 (2000)

    Google Scholar 

  32. Lathauwer, L.D., Moor, B.D.: From Matrix to Tensor: Multilinear Algebra and Signal Processing. In: Proc. 4th IMA Int. Conf. Mathmatics in Signal Process., pp. 1–15 (1996)

    Google Scholar 

  33. Harshman, R.A.: Foundations of the PARAFAC procedure: Model and Conditions for an “explanatory” multi-mode factor analysis. UCLA Working Papers in Phonetics, pp.1-84 (1970)

    Google Scholar 

  34. The Context Aware Vision using Image-based Active Recognition (CAVIAR) dataset, http://homepages.inf.ed.ac.uk/rbf/CAVIAR/

  35. Brezeale, D., Cook, D.J.: Automatic Video Classification: A Survey of the Literature. IEEE Transactions on Systems, Man and Cybernetics, Part C: Applications and Reviews 38(3) (2008)

    Google Scholar 

  36. Rabiner, L.R.: A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE 77, 257–286 (1989)

    Article  Google Scholar 

  37. Starner, T., Pentland, A.: Real-Time American Sign Language Recognition From Video Using Hidden Markov Models, Technical Report, MIT Media Lab, Perceptual Computing Group, vol. 375 (1995)

    Google Scholar 

  38. Raphael, C.: Automatic Segmentation of Acoustic Musical Signals Using Hidden Markov Models. IEEE Transactions on Pattern Analysis and Machine Intelligence 21(4), 360–370 (1999)

    Article  MathSciNet  Google Scholar 

  39. Bashir, F.I., Khokhar, A.A., Schonfeld, D.: HMM based motion recognitionsystem using segmented pca. In: IEEE International Conference on Image Processing (ICIP 2005), vol. 3, pp. 1288–1291 (2005)

    Google Scholar 

  40. Ma, X., Schonfeld, D., Khokhar, A.: Distributed multidimensional hidden Markov Model: theory and application in multiple-object trajectory classification and recognition. In: SPIE International Conference on Multimedia Content Access: Algorithms and Systems, San Jose, California (2008)

    Google Scholar 

  41. Ma, X., Schonfeld, D., Khokhar, A.: Image segmentation and classification based on a 2D distributed hidden Markov model. In: SPIE International Conference on Visual Communications and Image Processing (VCIP 2008), San Jose, California (2008)

    Google Scholar 

  42. Ma, X., Schonfeld, D., Khokhar, A.: Distributed Multi-dimensional Hidden Markov Models for Image and Trajectory-Based Video Classification. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2008), Las Vegas, Nevada (2008)

    Google Scholar 

  43. Ma, X., Schonfeld, D., Khokhar, A.: Video Event Classification and Image Segmentation Based on Non-Causal Multi-Dimensional Hidden Markov Models. IEEE Transactions on Image Processing, T-IP (May 2009) (to appear)

    Google Scholar 

  44. Baum, L.E., Petrie, T., Soules, G., Weiss, N.: A maximization technique occuring in the statistical analysis of probabilistic functions of markov chains. Ann. Math. Stat. 1, 164–171 (1970)

    Article  MathSciNet  Google Scholar 

  45. Li, J., Najmi, A., Gray, R.M.: Image classification by a two-dimensional hidden markov model. IEEE Trans. on Signal Processing 48, 517–533 (2000)

    Article  Google Scholar 

  46. Schonfeld, D., Bouaynaya, N.: A new method for multidimensional optimization and its application in image and video processing. IEEE Signal Processing Letters 13, 485–488 (2006)

    Article  Google Scholar 

  47. Ma, X., Khokhar, A., Schonfeld, D.: Robust video mining based on local similarity alignment of motion trajectories. In: IEEE Conference on Image Processing (ICIP 2009), Cairo, Egypt (2009)

    Google Scholar 

  48. Ma, X., Schonfeld, D., Khokhar, A.: Dynamic updating and downdating matrix SVD and tensor HOSVD for adaptive indexing and retrieval of motion trajectories. In: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2009), Taipei, Taiwan (2009)

    Google Scholar 

Download references

Authors
  1. Xiang Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ashfaq Khokhar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dan Schonfeld
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Multimedia Communications Laboratory Department of Electrical & Computer Engineering, University of Illinois at Chicago, Room 1020 SEO (M/C 154), 851 South Morgan Street, 60607-7053, Chicago, IL, USA

    Dan Schonfeld

  2. Philips Research, High-Tech Campus 36, 5656, Eindhoven, AE, The Netherlands

    Caifeng Shan

  3. Department of Computing, Hong Kong Polytechnic University, 7/F, Building P, Hung Hom, PQ704, Kowloon,Hong Kong, China

    Dacheng Tao

  4. Department of Computer Science, University of Bath, BA2 7AY, United Kingdom

    Liang Wang

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Ma, X., Chen, X., Khokhar, A., Schonfeld, D. (2010). Motion Trajectory-Based Video Retrieval, Classification, and Summarization. In: Schonfeld, D., Shan, C., Tao, D., Wang, L. (eds) Video Search and Mining. Studies in Computational Intelligence, vol 287. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12900-1_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-12900-1_3

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-12899-8

  • Online ISBN: 978-3-642-12900-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation