Skip to main content
SpringerLink
Log in

Trajectory Analysis and Semantic Region Modeling Using Nonparametric Hierarchical Bayesian Models

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

We propose a novel framework of using a nonparametric Bayesian model, called Dual Hierarchical Dirichlet Processes (Dual-HDP) (Wang et al. in IEEE Trans. Pattern Anal. Mach. Intell. 31:539–555, 2009), for unsupervised trajectory analysis and semantic region modeling in surveillance settings. In our approach, trajectories are treated as documents and observations of an object on a trajectory are treated as words in a document. Trajectories are clustered into different activities. Abnormal trajectories are detected as samples with low likelihoods. The semantic regions, which are subsets of paths commonly taken by objects and are related to activities in the scene, are also modeled. Under Dual-HDP, both the number of activity categories and the number of semantic regions are automatically learnt from data. In this paper, we further extend Dual-HDP to a Dynamic Dual-HDP model which allows dynamic update of activity models and online detection of normal/abnormal activities. Experiments are evaluated on a simulated data set and two real data sets, which include 8,478 radar tracks collected from a maritime port and 40,453 visual tracks collected from a parking lot.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Anjum, N., & Cavallaro, A. (2008). Multifeature object trajectory clustering for video analysis. IEEE Transactions on Circuits and Systems for Video Technology, 18, 1555–1564.

    Article  Google Scholar 

  • Asuncion, A., Smyth, P., & Welling, M. (2008). Asynchronous distributed learning of topic models. In Proc. of neural information processing systems conference.

    Google Scholar 

  • Blackman, S., & Popoli, R. (1999). Design and analysis of modern tracking systems. Norwood: Artech House.

    MATH  Google Scholar 

  • Blei, D. M., & Lafferty, J. D. (2006). Dynamic topic models. In Proc. of international conference on machine learning.

    Google Scholar 

  • Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent Dirichlet allocation. Journal of Machine Learning Research, 3, 993–1022.

    MATH  Google Scholar 

  • Brand, M., & Kettnaker, V. (2000). Discovery and segmentation of activities in video. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 844–851.

    Article  Google Scholar 

  • Brostow, G., & Cippola, R. (2006). Unsupervised Bayesian detection of independent motion in crowds. In Proc. of IEEE int’l conf. computer vision and pattern recognition.

    Google Scholar 

  • Caron, F., Davy, M., & Doucet, A. (2007). Generalized Polya urn for time-varying Dirichlet process mixtures. In Proc. of uncertainty in artificial intelligence.

    Google Scholar 

  • Dunson, D. B., Pillai, N., & Park, J. H. (2006). Bayesian density regression. Journal of the Royal Statistical Society. Series B, Statistical Methodology, 69, 163–183.

    MathSciNet  Google Scholar 

  • Fei-Fei, L., & Perona, P. (2005). A bayesian hierarchical model for learning natural scene categories. In Proc. of IEEE int’l conf. computer vision and pattern recognition.

    Google Scholar 

  • Ferguson, T. S. (1973). A bayesian analysis of some nonparametric problems. Annals of Statistics, 1, 209–230.

    Article  MathSciNet  MATH  Google Scholar 

  • Fernyhough, J., Cohn, A., & Hogg, D. (1996). Generation of semantic regions from image sequences. In Proc. of European conf. computer vision.

    Google Scholar 

  • Fowlkes, C., Belongie, S., Chung, F., & Malik, J. (2004). Spectral grouping using the Nystrom method. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26, 214–225.

    Article  Google Scholar 

  • Fox, E. B., Choi, D. S., & Willsky, A. S. (2006). Nonparameteric bayesian methods for large scale multi-target tracking. In Proceedings of Asilomar conference on signals, systems, and computers.

    Google Scholar 

  • Fu, Z., Hu, W., & Tan, T. (2005). Similarity based vehicle trajectory clustering and anomaly detection. In Proc. of IEEE int’l conf. image processing.

    Google Scholar 

  • Gelman, A., Stern, H. S., & Rubin, H. S. (2004). Bayesian data analysis. Boca Raton: CRC Press.

    MATH  Google Scholar 

  • Gennari, G., & Hager, G. D. (2004). Probabilistic data association methods in visual tracking of groups. In Proc. of IEEE int’l conf. computer vision and pattern recognition.

    Google Scholar 

  • Griffin, J. E., & Steel, M. F. J. (2006). Order-based dependent Dirichlet processes. Journal of the American Statistical Association, 101, 179–194.

    Article  MathSciNet  MATH  Google Scholar 

  • Haritaoglu, I., Harwood, D., & Davis, L. S. (2000). W4: Real-time surveillance of people and their activities. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 809–830.

    Article  Google Scholar 

  • Hofmann, T. (1999). Probabilistic latent semantic analysis. In Proc. of uncertainty in artificial intelligence.

    Google Scholar 

  • Honggeng, S., & Nevatia, R. (2001). Multi-agent event recognition. In Proc. of IEEE int’l conf. computer vision.

    Google Scholar 

  • Hu, W., Xiao, X., Fu, Z., Xie, D., Tan, T., & Maybank, S. (2006). A system for learning statistical motion patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28, 1450–1464.

    Article  Google Scholar 

  • Hu, W., Xie, D., Fu, Z., Zeng, W., & Mayband, S. (2007). Semantic-based surveillance video retrieval. IEEE Transactions on Image Processing, 16, 1168–1181.

    Article  MathSciNet  Google Scholar 

  • Johnson, N., & Hogg, D. (1995). Learning the distribution of object trajectories for event recognition. In Proc. of British machine vision conference.

    Google Scholar 

  • Junejo, I., & Foroosh, H. (2007). Trajectory rectification and path modeling for video surveillance. In Proc. of IEEE int’l conf. computer vision.

    Google Scholar 

  • Junejo, I., Javed, O., & Shah, M. (2004). Multi feature path modeling for video surveillance. In Proc. of IEEE int’l conf. pattern recognition.

    Google Scholar 

  • Kaucic, R., Perera, A., Brooksby, G., Kaufhold, J., & Hoogs, A. (2005). A unified framework for tracking through occlusions and across sensor gaps. In Proc. of IEEE int’l conf. computer vision and pattern recognition.

    Google Scholar 

  • Keogh, E., & Pazzani, M. (2000). Scaling up dynamic time scaling up dynamic time. In Proc. of ACM SIGKDD.

    Google Scholar 

  • Li, X., Hu, W., & Hu, W. (2006). A coarse-to-fine strategy for vehicle motion trajectory clustering. In Proc. int’l conf. pattern recognition.

    Google Scholar 

  • MacEachern, S., Kottas, A., & Gelfand, A. (2001). Spatial nonparametric bayesian models (Technical Report). Institute of Statistics and Decision Sciences, Duke University.

  • Makris, D., & Ellis, T. (2002). Path detection in video surveillance. Image and Vision Computing, 20, 859–903.

    Article  Google Scholar 

  • Makris, D., & Ellis, T. (2003). Automatic learning of an activity-based semantic scene model. In Proc. of AVSBS.

    Google Scholar 

  • Medioni, G., Cohen, I., BreAmond, F., Hongeng, S., & Nevatia, R. (2001). Event detection and analysis from video streams. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23, 873–889.

    Article  Google Scholar 

  • Moberts, B., Vilanova, A., & Jake, J. W. (2005). Evaluation of fiber clustering methods for diffusion tensor imaging. In Proc. of IEEE visualization.

    Google Scholar 

  • Morris, B., & Trivedi, M. (2009). Learning trajectory patterns by clustering: experimental studies and comparative evaluation. In Proc. of IEEE int’l conf. computer vision and pattern recognition.

    Google Scholar 

  • Ng, A. Y., Jordan, M. I., & Weiss, Y. (2002). On spectral clustering: analysis and an algorithm. In Proc. neural information processing systems conf.

    Google Scholar 

  • Niebles, J. C., & Fei-Fei, L. (2007). A hierarchical model of shape and appearance for human action classification. In Proc. of IEEE int’l conf. computer vision and pattern recognition.

    Google Scholar 

  • Niebles, J. C., Wang, H., & Fei-Fei, L. (2006). Unsupervised learning of human action categories using spatial-temporal words. In Proc. of British machine vision conference.

    Google Scholar 

  • Nillius, P., Sullivan, J., & Carlsson, S. (2006). Multi-target tracking—linking identities using bayesian network inference. In Proc. of IEEE int’l conf. computer vision and pattern recognition.

    Google Scholar 

  • Oliver, N., Rosario, B., & Pentland, A. (2000). A bayesian computer vision system for modeling human interactions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 831–843.

    Article  Google Scholar 

  • Pang, S. K., Li, J., & Godsill, S. J. (2008). Models and algorithms for detection and tracking of coordinated groups. In Proceedings of aerospace conference.

    Google Scholar 

  • Rao, S., Tron, R., Vidal, R., & Ma, Y. (2010). Motion segmentation in the presence of outlying, incomplete, or corrupted trajectories. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32, 1832–1845.

    Article  Google Scholar 

  • Ren, L., Dunson, D. B., & Carin, L. (2008). The dynamic hierarchical Dirichlet process. In Proc. of international conference on machine learning.

    Google Scholar 

  • Rittscher, J., Tu, P., & Krahnstoever, N. (2005). Simultaneous estimation of segmentation and shape. In Proc. of IEEE int’l conf. computer vision and pattern recognition.

    Google Scholar 

  • Rodriguez, A., Dunson, D. B., & Gelfand, A. E. (2006). The nested Dirichlet process (Technical report, Working Paper 2006-19). Duke Institute of Statistics and Decision Sciences.

  • Saleemi, I., Shafique, K., & Shah, M. (2009). Probabilistic modeling of scene dynamics for applications in visual surveillance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31, 1472–1485.

    Article  Google Scholar 

  • Sethuraman, J. (1994). A constructive definition of Dirichlet priors. Statistica Sinica, 4, 639–650.

    MathSciNet  MATH  Google Scholar 

  • Sivic, J., Russell, B. C., Efros, A. A., Zisserman, A., & Freeman, W. T. (2005). Discovering object categories in image collections. In Proc. of IEEE int’l conf. computer vision.

    Google Scholar 

  • Smith, P., Lobo, N., & Shah, M. (2005). Temporalboost for event recognition. In Proc. of IEEE int’l conf. computer vision.

    Google Scholar 

  • Srebro, N. & Roweis, S. (2005). Time-varying topic models using dependent Dirichlet process (Technical Report). Department of Computer Science, University of Toronto.

  • Stauffer, C., & Grimson, E. (2000). Learning patterns of activity using real-time tracking. In IEEE Trans. on pattern analysis and machine intelligence.

    Google Scholar 

  • Sudderth, E. B., Torralba, A., Freeman, W. T., & Willsky, A. S. (2005). Describing visual scenes using transformed Dirichlet processes. In Proc. of neural information processing systems conference.

    Google Scholar 

  • Sudderth, E. B., Torralba, A., Freeman, W. T., & Willsky, A. S. (2007). Describing visual scenes using transformed objects and parts. International Journal of Computer Vision, 77, 291–330.

    Article  Google Scholar 

  • Teh, Y. W. (2010). Dirichlet processes. In Encyclopedia of machine learning.

    Google Scholar 

  • Teh, Y. W., Jordan, M. I., Beal, M. J., & Blei, D. M. (2006). Hierarchical Dirichlet process. Journal of the American Statistical Association.

  • Truyen, T. T., Phung, D. Q., Bui, H. H., & Venkatesh, S. (2006). Adaboost.mrf: Boosted Markov random forests and application to multilevel activity recognition. In Proc. of IEEE Int’l conf. computer vision and pattern recognition.

    Google Scholar 

  • Vidal, R., & Hartley, R. (2004). Motion segmentation with missing data using powerfactorization and GPCA. In Proc. of IEEE int’l conf. computer vision and pattern recognition.

    Google Scholar 

  • Vlachos, M., Kollios, G., & Gunopulos, D. (2002). Discovering similar multidimensional trajectories. In Proc. IEEE conf. data engineering.

    Google Scholar 

  • Wang, X., & Grimson, E. (2007). Spatial latent Dirichlet allocation. In Proc. of neural information processing systems conference.

    Google Scholar 

  • Wang, Y., Jiang, T., Drew, M. S., Li, Z., & Mori, G. (2006a). Unsupervised discovery of action classes. In Proc. of IEEE int’l conf. computer vision and pattern recognition.

    Google Scholar 

  • Wang, X., Tieu, K., & Grimson, E. (2006b). Learning semantic scene models by trajectory analysis. In Proc. of European conf. computer vision.

    Google Scholar 

  • Wang, X., Ma, X., & Grimson, E. (2007). Unsupervised activity perception by hierarchical bayesian models. In Proc. of IEEE int’l conf. computer vision and pattern recognition.

    Google Scholar 

  • Wang, X., Ma, K. T., Ng, G., & Grimson, E. (2008). Trajectory analysis and semantic region modeling using a nonparameteric hierarchical bayesian model. In Proc. of IEEE int’l conf. computer vision and pattern recognition.

    Google Scholar 

  • Wang, X., Ma, X., & Grimson, W. E. L. (2009). Unsupervised activity perception in crowded and complicated scenes using hierarchical bayesian models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31, 539–555.

    Article  Google Scholar 

  • Xiang, T., & Gong, S. (2005). Video behaviour profiling and abnormality detection without manual labelling. In Proc. of IEEE int’l conf. computer vision.

    Google Scholar 

  • Xiang, T., & Gong, S. (2006). Beyond tracking: Modelling activity and understanding behaviour. International Journal of Computer Vision, 67, 21–51.

    Article  Google Scholar 

  • Zelnik-Manor, L., & Irani, M. (2001). Event-based analysis of video. In Proc. of IEEE int’l conf. computer vision and pattern recognition.

    Google Scholar 

  • Zhang, Z., Huang, K., Tan, T., & Wang, L. (2007). Trajectory series analysis based event rule induction for visual surveillance. In Proc. of IEEE int’l conf. computer vision and pattern recognition.

    Google Scholar 

  • Zhang, T., Lu, H., & Li, S. Z. (2009). Learning semantic scene models by object classification and trajectory clustering. In Proc. of IEEE int’l conf. computer vision and pattern recognition.

    Google Scholar 

  • Zhong, H., Shi, J., & Visontai, M. (2004). Detecting unusual activity. In Proc. of IEEE int’l conf. computer vision and pattern recognition.

    Google Scholar 

  • Zhu, X., Ghahramani, Z., & Lafferty, J. (2005). Time-sensitive Dirichlet process mixture model (Technical Report). School of Computer Science, Carnegie Mellon University.

Download references

Author information

Authors and Affiliations

  1. Department of Electronic Engineering, Chinese University of Hong Kong, Shatin, N.T., Hong Kong

    Xiaogang Wang

  2. National University of Singapore, Singapore, Singapore

    Keng Teck Ma

  3. DSO National Laboratories, Singapore, Singapore

    Gee-Wah Ng

  4. Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA

    W. Eric L. Grimson

Authors
  1. Xiaogang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Keng Teck Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gee-Wah Ng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. W. Eric L. Grimson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaogang Wang.

Additional information

Part of this work was published in Wang et al. (2008).

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wang, X., Ma, K.T., Ng, GW. et al. Trajectory Analysis and Semantic Region Modeling Using Nonparametric Hierarchical Bayesian Models. Int J Comput Vis 95, 287–312 (2011). https://doi.org/10.1007/s11263-011-0459-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-011-0459-6

Keywords

Search

Navigation