Skip to main content
SpringerLink
Log in

Behaviour recognition using multivariate m-mediod based modelling of motion trajectories

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

The importance of behaviour analysis and activity recognition systems continue to increase with the increasing demand and deployment of video surveillance systems. Motion trajectories provide rich spatio-temporal information about an object’s activity. In this article, we present a supervised feature extraction and multivariate modelling approach for motion-based behaviour recognition and anomaly detection. In the proposed motion learning system, trajectories are treated as time series and modelled using modified DFT-based coefficient feature space representation. We employ supervised dimensionality reduction using Local Fisher Discriminant Analysis to enhance the feature space representation of trajectories. A modelling approach, referred to as multivariate m-mediods, is proposed that can cater for the presence of multivariate distribution of samples within a given motion pattern. A hierarchical indexing of mediods and retrieval approach is presented to improve the efficiency of proposed classifier. Our proposed techniques are validated using variety of simulated and complex real-life trajectory datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. Aggarwal, J.K., Ryoo, M.S.: Human activity analysis: a review. ACM Comput. Surv. 43(3), 16 (2011)

    Article  Google Scholar 

  2. Ke, S.-R., Thuc, H.L.U., Lee, Y.-J., Hwang, J.-N., Yoo, J.-H., Choi, K.-H.: A review on video-based human activity recognition. Computers 2(2), 88–131 (2013)

    Article  Google Scholar 

  3. Aghbari, Z., Kaneko, K., Makinouchi, A.: Content-trajectory approach for searching video databases. IEEE Trans. Multimed. 5(4), 516–531 (2003)

    Article  Google Scholar 

  4. Chang, S.F., Chen, W., Horace, J.M., Sundaram, H., Zhong, D.: A fully automated content based video search engine supporting spatiotemporal queries. IEEE Trans. Circuits Syst. Video Technol. 8(5), 602–615 (1998)

    Article  Google Scholar 

  5. Dagtas, S., Ali-Khatib, W., Ghafor, A., Kashyap, R.L.: Models for motion-based video indexing and retrieval. IEEE Trans. Image Process. 9(1), 88–101 (2000)

    Article  Google Scholar 

  6. Hsu, C.T., Teng, S.J.: Motion trajectory based video indexing and retrieval. IEEE Int. Conf. Image Process. 1, 605–608 (2002)

    Article  Google Scholar 

  7. Jin, Y., Mokhtarian, F.: Efficient video retrieval by motion trajectory. In: Proceedings of British Machine Vision Conference, Kingston, pp. 667–676 (2004)

  8. Khalid, S., Naftel, A.: Evaluation of matching metrics for trajectory based indexing and retrieval of video clips. In: Proceedings of IEEE WACV, Colorado, pp. 242–249 (2005)

  9. Shim, C., Chang, J.: Trajectory based video retrieval for multimedia information systems. In: Proceedings of ADVIS, pp. 372–382 (2004)

  10. Johnson, N., Hogg, D.: Learning the distribution of object trajectories for event recognition. In: Proceedings of British Conference on Machine Vision, pp. 582–592 (1995)

  11. Hu, W., Xiao, X., Xie, D., Tan, T., Maybank, S.: Traffic accident prediction using 3-D model based vehicle tracking. IEEE Trans. Veh. Tech. 53(3), 677–694 (2004)

    Article  Google Scholar 

  12. Owens, J., Hunter, A.: Application of the self-organising map for trajectory classification. In: Proceedings of 3rd IEEE International Workshop on Visual Surveillance, Dublin, pp 77 (2000)

  13. Bashir, F.I., Khokhar, A.A., Schonfeld, D.: View-invariant motion trajectory based activity classification and recognition. In: ACM Multimedia Systems, special issue on machine learning approaches to multimedia information retrieval, pp. 45–54 (2006)

  14. Bashir, F.I., Khokhar, A.A., Schonfeld, D.: HMM based motion recognition system using segmented PCA. In: IEEE International Conference on Image Processing, Genova, 11–14 September 2005, pp. 1288–1291 (2005)

  15. Hu, W., Xie, D., Tan, T., Maybank, S.: Learning activity patterns using fuzzy self-organizing neural networks. IEEE Trans. Syst. Man Cybern. 34(3), 1618–1626 (2004)

    Article  Google Scholar 

  16. Brotherton, T., Johnson, T., Chadderdon, G.: Classification and novelty detection using linear models and a class dependent elliptical basis function neural network. In: Proceedings of IJCNN Conference, vol. 2, pp. 876–879 (1998)

  17. Roberts, S., Tarassenko, L.: A probabilistic resource allocating network for novelty detection. Neural Comput. 6, 270–284 (1994)

    Article  Google Scholar 

  18. Yeung, D.Y., Chow, C.: Parzen window network intrusion detectors. IEEE International Conference on Pattern Recognition, Canada, pp. 385−388 (2002)

  19. Xiang, T., Gong. S.: Video behaviour profiling and abnormality detection without manual labelling. In: IEEE International Conference on Computer Vision, vol 2, pp. 1238–1245, London (2005)

  20. Xiang, T., Gong, S.: Incremental visual behaviour modelling. In: 6th IEEE Workshop on Visual Surveillance, Graz, pp. 65−72 (2006)

  21. Zhang, D., Gatica-Perez, D., Bengio, S., McCowan, I.: Semi-supervised adapted HMMs for unusual event detection. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, pp. 611–618 (2005)

  22. Yacoob, Y., Black, M.J.: Parameterized modeling and recognition of activities. Comput. Vis. Image Underst. 73(2), 232–247 (1999)

    Article  Google Scholar 

  23. Bashir, F.I., Khokhar, A.A., Schonfeld, D.: Object trajectory-based activity classification and recognition using hidden Markov models. IEEE Trans. Image Process. 16(7), 1912–1919 (2007)

    Article  MathSciNet  Google Scholar 

  24. Rea, N., Dahyot, R., Kokaram, A.: Semantic event detection in sports through motion understanding. In: Proceedings of Conference on Image and Video Retrieval, Dublin (2004)

  25. Yang, Y., Cheng, N., Zhang, M.: Research on activity recognition method based on human motion trajectory features. J. Converg. Inf. Technol. 7(1), 79 (2012)

    Article  Google Scholar 

  26. Gao, Q., Sun, S.: Trajectory-based human activity recognition with hierarchical dirichlet process hidden markov models. In: IEEE International Conference on Signal and Information Processing, Beijing, pp. 456–460 (2013)

  27. Gao, Q., Sun, S.: Trajectory-based human activity recognition using hidden conditional random fields. Int. Conf. Mach. Learn. Cybern. 15–17, 1091–1097 (2012)

    Google Scholar 

  28. Hu, W., Xiao, X., Fu, Z., Xie, D., Tan, T., Maybank, S.: A system for learning statistical motion patterns. IEEE Trans. Pattern Anal. Mach. Learn. 28(9), 1450–1464 (2006)

    Article  Google Scholar 

  29. Hu, W., Xie, D., Fu, Z., Zeng, W., Maybank, S.: Semantic based surveillance video retrieval. In: IEEE Transactions on Image Processing, pp. 11681181 (2007)

  30. Khalid, S., Naftel, A.: Classifying spatiotemporal object trajectories using unsupervised learning in the coefficient feature space. Multimed. Syst. 12(3), 227–238 (2006)

    Article  Google Scholar 

  31. Sun, S., Xu, X.: Variational inference for infinite mixtures of gaussian processes with applications to traffic flow prediction. In: IEEE Transactions on Intelligent Transportation Systems, pp. 466–475 (2011)

  32. Sengto, A., Leauhatong, T.: Human falling detection algorithm using back propagation neural network. In: Proceedings of IEEE Biomedical Engineering International Conference (BMEiCON), Ubon Ratchathani, vol. 57, pp. 15 (2012)

  33. Saleemi, I., Shafique, K., Shah, M.: Probabilistic modeling of scene dynamics for applications in vIsual surveillance. IEEE TPAMI 31(8) (2009)

  34. Varadarajan, J., Emonet, R., Odobez, J.-M.: Bridging the past, present and future: modeling scene activities from event relationships and global rules. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence (2012)

  35. Chawla, S., Hand, D.J.: Vasant Dhar, outlier detection special issue. Data Min. Knowl. Discov. 20(2), 189–190 (2010)

    Article  MathSciNet  Google Scholar 

  36. Silla Jr., C.N., Freitas, A.A.: A survey of hierarchical classification across different application domains. Data Min. Knowl. Discov. 22(1–2), 31–72 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  37. Jargalsaikhan, I., Little, S., Direkoglu, C., OConnor, N.E.: Action recognition based on sparse motion trajectories. In: 20th International Conference on Image Processing, Melbourne, pp. 3982–3985 (2013)

  38. Seo, H.J., Milanfar, P.: Action recognition from one example. IEEE T-PAMI 33(5), 867882 (2011)

    Google Scholar 

  39. Wang, H., Yuan, C., Hu, W., Sun, C.: Supervised classspecific dictionary learning for sparse modeling in action recognition. Pattern Recogn. 45(11), 39023911 (2012)

    Google Scholar 

  40. Khalid, S.: Motion based behaviour learning, profiling and classification in the presence on anomalies. Pattern Recogn. doi:10.1016/j.patcog.2009.04.025 (2009)

  41. Khalid, S.: Activity classification and anomaly detection using m-mediods based modeling of motion patterns. Pattern Recogn. 43(10), 3636–3647 (2010)

    Article  MATH  Google Scholar 

  42. Yu, J.: Localized Fisher discriminant analysis based complex chemical process monitoring (2010). doi:10.1002/aic.12392

  43. Khalid, S., Naftel, A.: Automatic motion learning in the presence of anomalies using coefficient feature space representation of trajectories. Acta Autom. Sin. 36(5) (2010)

  44. Bashir, F.I., Khokhar, A.A., Schonfield, D.: Automatic object trajectory based motion recognition using gaussian mixture models. In: IEEE International Conference on Multimedia and Expo, The Netherlands, pp. 1532–1535 (2005)

  45. Tax, D.M.J.: One-class classification. PhD thesis, Delft University of Technology. http://ict.ewi.tudelft.nl/davidt/thesis (2001)

  46. Marsland, S., Shapiro, J., Nehmzow, U.: A self-organising network that grows when required. Neural Netw. 15(8–9), 1041–1058 (2002)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Bahria University, Islamabad, 44000, Pakistan

    Shehzad Khalid & Shahid Razzaq

  2. Department of Computer Engineering, College of Electrical & Mechanical Engineering, National University of Science and Technology, Rawalpindi, 44000, Pakistan

    Usman Akram

Authors
  1. Shehzad Khalid
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Usman Akram
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shahid Razzaq
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shehzad Khalid.

Additional information

Communicated by Q. Tian.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khalid, S., Akram, U. & Razzaq, S. Behaviour recognition using multivariate m-mediod based modelling of motion trajectories. Multimedia Systems 21, 485–505 (2015). https://doi.org/10.1007/s00530-014-0413-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-014-0413-x

Keywords

Search

Navigation