Skip to main content
SpringerLink
Log in

Trajectory clustering in CCTV traffic videos using probability product kernels with hidden Markov models

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

This article presents a method for clustering the trajectories obtained by tracking vehicles in traffic videos, recorded from CCTV cameras in public spaces. The proposed method employs a model-based approach in which (1) each trajectory (position and velocity) is modelled using a hidden Markov model (HMM), and (2) the distance between two trajectories is computed as the probabilistic similarity between HMMs, by means of the probability product kernel. Experiments on a set of real traffic video sequences reveal very good results of the proposed approach, outperforming two non-trivial baselines. To the best of the authors’ knowledge, this approach is novel for trajectory grouping in traffic videos.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. http://wwwl.cs.Columbia.edu/~jebara/code/.

  2. http://lear.inrialpes.fr/~verbeek/software.php.

  3. Available at http://ngsim.camsys.com/.

References

  1. Kastrinaki V, Zervakis M, Kalaitzakis K (2003) A survey of video processing techniques for traffic applications. Image Vis Comput 21(4):359–381

    Article  Google Scholar 

  2. Kamijo S, Matsushita Y, Ikeuchi K, Sakauchi M (2000) Traffic monitoring and accident detection at intersections. IEEE Trans Intell Transp Syst 1(2):108–118

    Article  Google Scholar 

  3. Beymer D, Mclauchlan P, Coifman B, Malik J (1997) A real-time computer vision system for measuring traffic parameters. In: Proceedings IEEE Computer Society Conference on computer vision and pattern recognition, pp 495–501

  4. Brand M, Kettnaker V (2000) Discovery and segmentation of activities in video. IEEE Trans Pattern Anal Mach Intell 22(8):844–851

    Article  Google Scholar 

  5. Hu W, Tan T, Wang L, Maybank S (2004) A survey on visual surveillance of object motion and behaviors. IEEE Trans Syst Man Cybern Part C 34(3):334–352

    Article  Google Scholar 

  6. Hu W, Xie D, Tan T (2004) A hierarchical self-organizing approach for learning the patterns of motion trajectories. IEEE Trans Neural Netw 15(1):135–144

    Article  Google Scholar 

  7. Kilger M (1992) A shadow handler in a video-based real-time traffic monitoring system. In: Proceedings IEEE Workshop on applications of computer vision, pp 11–18

  8. Yilmaz A, Javed O, Shah M (2006) Object tracking: a survey. ACM Comput Surv 38(4), article no. 13

  9. Makris D (2002) Path detection in video surveillance. Image Vis Comput 20(12):895–903

    Article  Google Scholar 

  10. Makris D, Ellis T, Black J (2008) Intelligent visual surveillance: towards cognitive vision systems. Open Cybern and Syst J 2:219–229

    Article  Google Scholar 

  11. Stauffer C, Grimson WEL (2000) Learning patterns of activity using realtime tracking. Trans Pattern Anal Mach Intell 22(8):747–757

    Article  Google Scholar 

  12. Li C, Biswas G (2000) A bayesian approach to temporal data clustering using hidden markov models. In: ICML’00: Proceedings of the Seventeenth International Conference on Machine Learning, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, pp 543–550

  13. Smyth P (1997) Clustering sequences with hidden markov models. Adv Neural Inf Process Syst 9:648–654

    MathSciNet  Google Scholar 

  14. Fu Z, Hu W, Tan T (2005) Similarity based vehicle trajectory clustering and anomaly detection, In: IEEE International Conference on image processing. ICIP 2005, vol 2, pp II-602–II-605

  15. Izo T, Grimson WEL (2007) Unsupervised modeling of object tracks for fast anomaly detection. In: IEEE International Conference on image processing ICIP 2007, vol 4, pp IV-529–IV-532

  16. Dubuisson MP, Jain AK (1994) A modified hausdorff distance for object matching. Pattern Recogn 1:566–568

    Google Scholar 

  17. Khalid S, Naftel (2005) Evaluation of matching metrics for trajectory-based indexing and retrieval of video clips. In: Seventh IEEE Workshops on application of computer vision. WACV/MOTIONS’05, vol 1, pp. 242–249

  18. Bashir F, Khokhar A, Schonfeld D (2003) Segmented trajectory based indexing and retrieval of video data. In: International Conference on image processing, vol 2, pp 623–626

  19. Ng A, Jordan M, Weiss Y (2002) Neural Information Processing Systems

  20. Khalid S, Naftel A (2005) Classifying spatiotemporal object trajectories using unsupervised learning of basis function coefficients. In: VSSN’05: Proceedings of the third ACM international workshop on Video surveillance and sensor networks, ACM, New York, NY, USA, pp 45–52

  21. Cooley JW, Tukey JW (1965) An algorithm for the machine calculation of complex fourier series. Math Comput 19:297–301

    Article  MathSciNet  MATH  Google Scholar 

  22. Vlachos M, Kollios G, Gunopulos D (2002) Discovering similar multidimensional trajectories. In: Proceedings of 18th International Conference on data engineering, pp 673–684

  23. Buzan D, Sclaroff S, Kollios G (2004) Extraction and clustering of motion trajectories in video. In: Proceedings of the 17th International Conference on pattern recognition, ICPR 2004, vol 2, pp 521–524

  24. Pusiol G, Patino L, Bremond F, Thonnant M, Suresh S (2008) Optimizing trajectories clustering for activity recognition. In: The 1st International workshop on machine learning for vision-based motion analysis—MLVMA08

  25. Johnson S (1967) Hierarchical clustering schemes. Psychometrika 32(3):241–254

    Article  Google Scholar 

  26. Kohonen T (1997) Self-organizing maps. Springer, New York

  27. Rabiner LR (1989) A tutorial on hidden Markov models and selected applications in speech recognition. Proc IEEE 77:257–286

    Article  Google Scholar 

  28. Bashir F, Khokhar A, Schonfeld D (2007) Object trajectory-based activity classification and recognition using hidden Markov models. IEEE Trans Image Process 16(7):1912–1919

    Article  MathSciNet  Google Scholar 

  29. Porikli F, Haga T (2004) Event detection by eigenvector decomposition using object and frame features. In: Proceedings of the 2004 Conference on computer vision and pattern recognition workshop (CVPRW’04), vol 7, p 114

  30. Suzuki N, Hirasawa K, Tanaka K, Kobayashi Y, Sato Y, Fujino Y (2007) Learning motion patterns and anomaly detection by human trajectory analysis, In: IEEE International Conference on Systems, Man and Cybernetics, ISIC, 2007, pp 498–503

  31. Oates T, Firoiu L, Cohen P (2001) Using dynamic time warping to bootstrap hmm-based clustering of time series, pp 35–52. http://dx.doi.org/10.1007/3-540-44565-X\_3

  32. Duda RO, Hart PE, Stork DG (2000) Nonparametric techniques, pattern classification, Ch. 4. Wiley-Interscience Publication, New York

  33. Alon J, Sclaroff S, Kollios G, Pavlovic V (2003) Discovering clusters in motion time-series data. In: IEEE Computer Society Conference on computer vision and pattern recognition, vol 1, IEEE Computer Society, Los Alamitos, CA, USA, p 375

  34. Moon T (1996) The expectation–maximization algorithm. IEEE Signal Process Mag 13:47–60

    Article  Google Scholar 

  35. Noceti N, Santoro M, Odone F (2008) Unsupervised learning of behavioural patterns for video-surveillance. In: Proceedings of the 1st International Workshop on Machine Learning for Vision-based Motion Analysis, MLVMA08 (2008)

  36. Jebara T, Song Y, Thadani K (2007) Spectral clustering and embedding with hidden Markov models. In: Proceedings of European Conference on Machine Learning

  37. Stauffer C, Grimson WEL (1999) Adaptive background mixture models for real-time tracking. In: IEEE conference on computer vision and pattern recognition, pp 246–252

  38. Baum LE, Petrie T, Soules G, Weiss N (1970) A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Ann Math Stat 41(1):164–171

    Article  MathSciNet  MATH  Google Scholar 

  39. Bahlmann C, Burkhardt H (2001) Measuring HMM similarity with the Bayes probability of error and its application to online handwriting recognition. In: Proceedings of the 6th International Conference on document analysis and recognition

  40. Hershey JR, Olsen PA, Rennie SJ (2007) Variational Kullback–Leibler divergence for hidden Markov models. In: Proceedings of the 10th IEEE workshop on automatic speech recognition and understanding, pp 323–328

  41. Huo Q, Li W (2006) A DTW-based dissimilarity measure for left-to-right hidden Markov models and its application to word confusability analysis. In: INTERSPEECH 2006, pp 2338–2341

  42. Likas A, Vlassis N, Verbeek JJ (2003) The global k-means clustering algorithm. Pattern Recogn 36(2):451–461

    Article  Google Scholar 

  43. Sakoe H, Chiba S (1978) Dynamic programming algorithm optimization for spoken word recognition. IEEE Trans Acoust Speech Signal Process 26(1):159–165

    Google Scholar 

  44. Rath TM, Manmatha R (2003) Word image matching using dynamic time warping. In: Proceedings of the 2003 IEEE Conference on computer vision and pattern recognition, pp 521–527

  45. Kim Z, Gomes G, Hranac R, Skabardonis A (2005) A machine vision system for generating vehicle trajectories over extended freeway segments. In: 12th World Congress on Intelligent Transportation Systems

  46. Bezdek JC, Pal NR (1998) Some new indexes of cluster validity. IEEE Trans Syst Man Cybern 28(3):301–315

    Article  Google Scholar 

Download references

Acknowledgments

This research is undertaken by Loughborough University as part of the FREELOW Project. FREEFLOW is a Research and Development Project formed by a consortium of industrial companies, academia and local authorities, aiming to improve transport performance by turning “data into intelligence”. It is part funded by the Technology Strategy Board, the Department for Transport (DfT) and the Engineering and Physical Sciences Research Council (EPSRC). The remaining funding comes from the partners themselves. Freeflow has specific objectives of: (1) exploiting new and existing data sources for traffic management that helps meet network managers’ objectives of balancing the network and for providing traveller information to individual and commercial users; (2) understanding real world user requirements for “Intelligent Decision Support” (IDS) using military “Situational Awareness” tools; (3) researching, demonstrating and evaluating the benefits of these joined up tools for both network managers and the travelling public; and (4) developing services, products and tools for both UK and global markets. The CCTV images were provided and are reproduced by kind permission of Kent County Council.

Author information

Authors and Affiliations

  1. Department of Computer Science, Research School of Informatics, Loughborough University, Loughborough, LE11 3TU, UK

    José A. Rodríguez-Serrano & Sameer Singh

Authors
  1. José A. Rodríguez-Serrano
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sameer Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to José A. Rodríguez-Serrano.

Additional information

This work was done while the first author was at Loughborough University.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rodríguez-Serrano, J.A., Singh, S. Trajectory clustering in CCTV traffic videos using probability product kernels with hidden Markov models. Pattern Anal Applic 15, 415–426 (2012). https://doi.org/10.1007/s10044-012-0269-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-012-0269-7

Keywords

Search

Navigation