Skip to main content
Log in

Hierarchical visual event pattern mining and its applications

Data Mining and Knowledge Discovery Aims and scope Submit manuscript

Abstract

In this paper, we propose a hierarchical visual event pattern mining approach and utilize the patterns to address the key problems in video mining and understanding field. We classify events into primitive events (PEs) and compound events (CEs), where PEs are the units of CEs, and CEs serve as smooth priors and rules for PEs. We first propose a tensor-based video representation and Joint Matrix Factorization (JMF) for unsupervised primitive event categorization. Then we apply frequent pattern mining techniques to discover compound event pattern structures. After that, we utilize the two kinds of event patterns to address the applications of event recognition and anomaly detection. First we extend the Sequential Monte Carlo (SMC) method to recognition of live, sequential visual events. To accomplish this task we present a scheme that alternatively recognizes primitive and compound events in one framework. Then, we categorize the anomalies into abnormal events (never seen events) and abnormal contexts (rule breakers), and the two kinds of anomalies are detected simultaneously by embedding a deviation criterion into the SMC framework. Extensive experiments have been conducted which demonstrate that the proposed approach is effective as compared to other major approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  • Blank B, Gorelick L, Shechtman E, Irani M, Basri R (2005) Actions as space-time shapes. Internatinal conference on computer vision

  • Boiman O, Irani M (2005) Detecting irregularities in images and in video. Internatinal conference on computer vision

  • Chan MT, Hoogs A, Schmiederer J, Petersen M (2004) Detecting rare events in video using semantic primitives with HMM. International conference on pattern recognition

  • Chris D, Li T, Peng W, Park H (2006) Orthogonal nonnegative matrix tri-factorizations for clustering. International conference on knowledge discovery and data mining

  • Cui P, Sun LF, Liu ZQ, Yang SQ (2007) A sequential monte carlo approach to anomaly detection in tracking visual events. International conference on computer vision and pattern recognition

  • Davis JW, Taylor SR (2002) Analysis and recognition of walking movements. International conference on pattern recognition

  • Doucet A, Freitas N, Gordon N (2001) Sequential monte carlo methods in practice. Springer, New York

    MATH  Google Scholar 

  • Du YT, Chen F, Xu WL, Li YB (2006) Recognizing interaction activities using dynamic bayesian network. International conference on pattern recognition

  • Duong TV, Bui HH, Phung DQ, Venkatesh S (2005) Activity recognition and abnormality Detection with the switching hidden semi-markov model. International conference on computer vision and pattern recognition

  • Efros AA, Berg AC, Mori G, Malik J (2003) Recognizing action at a distance. International conference on computer vision

  • Gilbert A, Illingworth J, Bowden R (2009) Fast realistic multi-action recognition using mined dense spatio-temporal features. International conference on computer vision

  • Hakeem A, Shah M (2005) Multiple agent event detection and representation in videos. National conference on artificial intelligence

  • Haritaoglu I, Harwood D, Davis LS (2000) W4 real-time surveillance of people and their activities. IEEE Trans Pattern Anal Mach Intell 22(8): 809–830

    Article  Google Scholar 

  • Ivanov YA, Bobick AF (2000) Recognition of visual activities and interactions by stochastic parsing. IEEE Trans Pattern Anal Mach Intell 22: 852–872

    Article  Google Scholar 

  • Kawanaka D, Okatani T, Deguchi K (2006) HHMM based recognition of human activity. IEICE transactions on information and systems E89-D(7): 2180–2185

    Article  Google Scholar 

  • Ke Y, Sukthankar R, Hebert M (2005) Efficient visual event detection using volumetric features. International conference on computer vision

  • Li T, Chris D (2006) The relationships among various nonnegative matrix factorization methods for clustering. International conference on data mining

  • Lin Z, Jiang ZL, Davis LS (2009) Recognizing actions by shape-motion prototype trees. International conference on computer vision

  • Ling HB, Soatto S (2007) Proximity distribution kernels for geometric context in category recognition. International conference on computer vision

  • Liu HW, Feris R, Krueger V, Sun MT (2010) Unsupervised action classification using space-time link analysis. EURASIP J Image Video Process

  • Makris D, Ellis T (2002) Spatial and probabilistic modeling of pedestrian behavior. British machine vision conference

  • Mannila H, Toivonen H, Verkamo AI (1997) Discovery of frequent episodes in event sequences. Data Min Knowl Discov 1(3): 259–289

    Article  Google Scholar 

  • Morup M, Hansen LK, Arnfred SM (2006) Decomposing the time-frequency representation of EEG using nonnegative matrix and multi-way factorization. Technical report, Institute for Mathematical Modeling, Technical University of Denmark

  • Niebles JC, Wang HC, Fei-Fei L (2008) Unsupervised learning of human action categories using spatial-temporal words. Int J Comput Vis 79(3): 299–318

    Article  Google Scholar 

  • Robertson N, Reid I (2006) A general method for human activity recognition in video. J Comput Vis Image Underst 104(2): 232–248

    Article  Google Scholar 

  • Stauffer C, Grimson W (2000) Learning patterns of activity using real-time tracking. IEEE Trans Pattern Anal Mach Intell 22(8): 747–758

    Article  Google Scholar 

  • Sun XH, Chen MY, Hauptmann A (2009) Action recognition via local descriptors and holistic features. International conference on computer vision and pattern recognition

  • Tao DC, Li XL, Wu XD, Hu WM, Maybank SJ (2007a) Supervised tensor learning. Knowl Inf Syst 13: 1–42

    Article  Google Scholar 

  • Tao DC, Li XL, Wu XD, Maybank SJ (2007b) General tensor discriminant analysis and gabor features for gait recognition. IEEE Trans Pattern Anal Mach Intell 29: 1700–1715

    Article  Google Scholar 

  • Tao D, Li X, Wu X, Maybank S (2008a) Tensor rank one discriminant analysis—a convergent method for discriminative multilinear subspace selection. Neurocomputing 71: 1866–1882

    Article  Google Scholar 

  • Tao D, Song M, Li X, Shen J, Sun J, Wu X, Faloutsos C (2008b) Bayesian tensor approach for 3-D face modeling. IEEE Trans Circ Syst Video Tech 18(10): 1397–1410

    Article  Google Scholar 

  • Vaswani N, Chowdhury A, Chellappa R (2003) Activity recognition using the dynamics of the configuration of interacting objects. International conference on computer vision and pattern recognition

  • Wang F, Li T (2007) Gene Selection via Matrix Factorization. International symposium on bioinformatics and bioengineering

  • Wang F, Li T, Zhang CS (2008) Semi-supervised clustering via matrix factorization. SIAM conference on data mining

  • Xiang T, Gong S (2005) Video behaviour profiling and abnormality detection without manual labelling. International conference on computer vision

  • Xiang T, Gong S (2006) Beyond tracking: modelling action and understanding behavior. Int J Comput Vis 67(1): 21–51

    Article  Google Scholar 

  • Xu W, Liu X, Gong YH (2003) Document clustering based on non-negative matrix factorization. SIGIR conference on research and development in informaion retrieval

  • Yamamoto M, Mitomi H, Fujiwara F, Sato T (2006) Bayesian classification of task-oriented actions based on stochastic contextfree grammar. International conference on automatic face and gesture recognition

  • Yilmaz A, Shah M (2005) Actions sketch: a novel action representation. International conference on computer vision and pattern recognition

  • Zelnik-Manor L, Irani M (2001) Event-based video analysis. International conference on computer vision and pattern recognition

  • Zhong H, Shi J, Visontai M (2004) Detecting unusual activity in video. International conference on computer vision and pattern recognition

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Peng Cui.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Cui, P., Liu, ZQ., Sun, LF. et al. Hierarchical visual event pattern mining and its applications. Data Min Knowl Disc 22, 467–492 (2011). https://doi.org/10.1007/s10618-010-0195-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10618-010-0195-5

Keywords

Navigation