Skip to main content
SpringerLink
Log in

Real-time spatio-temporal analysis of dynamic scenes

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

We propose a set of tools for spatio-temporal real-time analysis of dynamic scenes. It is designed to improve the grounding situation of autonomous agents in (simulated) physical domains. We introduce a knowledge processing pipeline ranging from relevance-driven compilation of a qualitative scene description to a knowledge-based detection of complex event and action sequences, conceived as a spatio-temporal pattern-matching problem. A methodology for the formalization of motion patterns and their inner composition is introduced and applied to capture human expertise about domain-specific motion situations. We present extensive experimental results from a challenging environment: 3D soccer simulation. It substantiates real-time applicability of our approach under tournament conditions, based on a 5-Hz (a) precise and (b) noisy/incomplete perception. The approach is not limited to robot soccer. Instead, it can also be applied in other fields such as experimental biology and logistic processes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aggarwal CC (2009) On classification and segmentation of massive audio data streams. Knowl Inf Syst 20(2): 137–156

    Article  MathSciNet  Google Scholar 

  2. Allen J (1983) Maintaining knowledge about temporal intervals. Commun ACM 26(11): 832–843

    Article  MATH  Google Scholar 

  3. Allen JF (1984) Towards a general theory of action and time. Artif Intell 23(2): 123–154

    Article  MATH  Google Scholar 

  4. Allen JF, Hayes PJ (1990) Moments and points in an interval-based temporal logic. Comput Intell 5(4): 225–238

    Google Scholar 

  5. Allen J, Ferguson G (1994) Actions and events in interval temporal logic. J Logic Comput 4(5): 531–579

    Article  MathSciNet  MATH  Google Scholar 

  6. Baker C, Dolan J (2008) Traffic interaction in the urban challenge: putting Boss on its best behavior. In: International conference on intelligent robots and systems (IROS 2008), pp 1752–1758

  7. Balch T, Dellaert F, Feldman A et al (2006) How multirobot systems research will accelerate our understanding of social animal behavior. Proc IEEE 94(7): 1445–1463

    Article  Google Scholar 

  8. Beetz M, Kirchlechner B, Lames M (2005) Computerized real-time analysis of football games. IEEE Perv Comput 4(3): 33–39

    Article  Google Scholar 

  9. Beetz M, Hoyningen-Huene N, Bandouch J et al (2006) Camera-based Observation of football games for analyzing multi-agent activities. In: ‘AAMAS ’06: proceedings of the fifth international joint conference on autonomous agents and multiagent systems. ACM, New York, NY, USA, pp 42–49

  10. Bertrand O, Carle P, Choppy C (2009) Modelling chronicle recognition for distributed simulation processing with coloured petri nets. In: ‘Simutools ’09: proceedings of the 2nd international conference on simulation tools and techniques’, ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering). ICST, Brussels, Belgium, Belgium, pp 1–2

  11. Choppy C, Bertrand O, Carle P (2009) Coloured petri nets for chronicle recognition. In: Ada-Europe ’09: proceedings of the 14th Ada-Europe international conference on reliable software technologies. Springer, Berlin, pp 266–281

  12. Divakaran A, Peker K, Radharkishnan R et al (2003) Video summarization using MPEG- 7 motion activity and audio descriptors. Video Mining, p 91

  13. Dousson C, Le Maigat P (2007) Chronicle recognition improvement using temporal focusing and hierarchization. In: IJCAI’07: proceedings of the 20th international joint conference on artifical intelligence, pp 324–329. Morgan Kaufmann, San Francisco

  14. Feldman A, Balch T (2004) Representing honey bee behavior for recognition using human trainable models. Adapt Behav 12(3–4): 241–250

    Article  Google Scholar 

  15. Ferman AM, Tekalp AM (2002) Efficient filtering and clustering methods for temporal video segmentation and visual summarization. J Vis Commun Image Represent 9(4): 336–351

    Article  Google Scholar 

  16. Fox D, Thrun S, Burgard W et al (2001) Particle filters for mobile robot localization. In: Doucet A, Freitas N, Gordon N (eds) Sequential Monte Carlo methods in practice. Springer, New York, pp 470–498

    Google Scholar 

  17. Fraser G, Steinbauer G et al (2004) Application of qualitative reasoning to robotic soccer. In: 18th International workshop on qualitative reasoning, USA

  18. Freksa C (1992) Temporal reasoning based on semi-intervals. In: Artificial intelligence, vol 54, pp 199–227

  19. Gehrke JD (2005) Qualitative Szenenrepräsentation für intelligente Fahrzeuge. Master’s thesis, Universität Bremen

  20. Ghallab M (1996) On chronicles: representation, on-line recognition and learning. In: KR, pp 597–606

  21. Gottfried B (2007) Representing short-term observations of moving objects by a simple visual language. J Vis Lang Comput 19(3): 321–342

    Article  Google Scholar 

  22. Gregorčič G, Lightbody G (2009) Gaussian process approach for modelling of nonlinear systems. Eng Appl Artif Intell 22(4–5): 522–533

    Google Scholar 

  23. Heintz F, Doherty P (2004) DyKnow: a framework for processing dynamic knowledge and object structures in autonomous systems. In: Proceedings of the international workshop on monitoring, security, and rescue techniques in multiagent systems (MSRAS)

  24. Herzog G (1995) Utilizing interval-based event representations for incremental high level scene analysis. Technical report 91, Universität des Saarlandes

  25. Intille S, Bobick A (2001) Recognizing planned, multiperson action. Comput Vis Image Understand 81(3): 414–445

    Article  MATH  Google Scholar 

  26. Jeannin S, Divakaran A (2001) MPEG-7 visual motion descriptors. IEEE Trans Circuits Syst Video Tech 11(6): 720–724

    Article  Google Scholar 

  27. Jenkins OC, Matarić MJ (2004) A spatio-temporal extension to isomap nonlinear dimension reduction. In: The international conference on machine learning (ICML 2004), Banff, AB, Canada, pp 441–448

  28. Lafferty J, McCallum A, Pereira F (2001) Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of 18th international conference on machine learning, Morgan Kaufmann, San Francisco, CA, pp 282–289

  29. Lattner AD (2007) Temporal pattern mining in dynamic environments. Dissertation, Universität Bremen, number 309 in ‘DISKI’, Akademische Verlagsgesellschaft Aka GmbH, Berlin, Germany

  30. Lattner AD, Visser U, Warden T et al (2006) Virtual Werder 3D team documentation. Technical report 36, TZI - Center for Computing Technologies, Universität Bremen

  31. Miene A (2004) Räumlich-zeitliche analyse von dynamischen Szenen. Dissertation, Universität Bremen, number 279 in ‘DISKI’, Akademische Verlagsgesellschaft Aka GmbH, Berlin, Germany

  32. Miene A, Visser U, Herzog O (2004) Recognition and prediction of motion situations based on a qualitative motion description. In: Polani D, Browning B, Bonarini A, Yoshida K (eds) RoboCup 2003: robot soccer world cup VII, vol 3020 of LNCS. Springer, Berlin, pp 77–88

    Chapter  Google Scholar 

  33. Mohanty S (2009) Gaussian process time series model for life prognosis of metallic structures. J Intell Mater Syst Struct 20(8): 887–896

    Article  Google Scholar 

  34. Müller UT (2002) Beschreiben und Erkennen von Verhaltensmustern beim simulierten Fußballspiel. Master’s thesis, Humboldt-Universität zu Berlin, Berlin, Germany

  35. Musto A (2000) Qualitative Repräsentation von Bewegungsverläufen. PhD thesis, Technische Universität München

  36. Nair R, Milind T et al (2004) Automated assistants for analyzing team behaviors. J Autonom Agents Multi-Agent Syst 8(1): 69–111

    Article  Google Scholar 

  37. Neumann B, Möller R (2006) On scene interpretation with description logics. In: Christensen H, Nagel H-H (eds) Cognitive vision systems: sampling the spectrum of approaches, number 3948. Lect Notes Comput Sci, Springer, Berlin, pp 247–248

  38. Obeid N, Rao R (2010) On integrating event definition and event detection. Knowl Inf Syst 22: 129–158

    Article  Google Scholar 

  39. Papapetrou P, Kollios G, Sclaroff S, Gunopulos D (2009) Mining frequent arrangements of temporal intervals. Knowl Inf Syst 21(2): 133–171

    Article  Google Scholar 

  40. Pokrajac D, Hoskinson RL, Obradovic Z (2003) Modeling spatial-temporal data with a short observation history. Knowl Inf Syst 5(3): 368–386

    Article  Google Scholar 

  41. Rabiner LR (1990) A tutorial on hidden Markov models and selected applications in speech recognition, pp 267–296

  42. Rasmussen CE, Williams C (2006) Gaussian processes for machine learning. MIT Press, Cambridge

    MATH  Google Scholar 

  43. Stanton C, Williams M-A et al (2004) Grounding robot sensory and symbolic information using the semantic web. In: Polani D (eds) RoboCup 2003: robot soccer world cup VII. Springer, Berlin, pp 757–764

    Chapter  Google Scholar 

  44. Steinbauer G, Weber J, Wotawa F (2005) From real-world to its qualitative representation—practical lessons learned. In: 18th International workshop on qualitative reasoning, Graz, pp 186–191

  45. Thrun S, Montemerlo M, Becker J et al (2008) Junior: the stanford entry in the urban challenge. J Field Robot 25(9): 569–597

    Article  Google Scholar 

  46. Vail DL, Veloso MM, Lafferty JD (2007) Conditional random fields for activity recognition. In: AAMAS ’07: proceedings of the 6th international joint conference on autonomous agents and multiagent systems. ACM, New York, NY, USA, pp 1–8

  47. Visser U, Burkhard H-D (2007) RoboCup: 10 years of achievements and future challenges. AI Mag 28(2): 115–132

    Google Scholar 

  48. Warden T (2007) Spatio-temporal real-time analysis of dynamic scenes in the RoboCup 3D soccer simulation league. Master’s thesis, Universität Bremen

  49. Wendler J (2003) Automatisches Modellieren von Agenten-Verhalten: Erkennen, Verstehen und Vorhersagen von Verhalten in komplexen Multi-Agenten-Systemen. PhD thesis, Humboldt-Universität zu Berlin

  50. Wendler J, Bach J et al (2004) Recognizing and Predicting agent behavior with case based reasoning. In: Polani D (eds) RoboCup 2003: Robot Soccer World Cup VII’, LNAI. Springer, Berlin, pp 729–738

    Chapter  Google Scholar 

  51. Zhou G., Dong N, Wang Y (2009) Non-linear dynamic texture analysis and synthesis using constrained Gaussian process latent variable model. Pacific-Asia conference on circuits, communications and systems, pp 27–30

Download references

Author information

Authors and Affiliations

  1. Center for Computing and Communication Technologies, University of Bremen, 28359, Bremen, Germany

    Tobias Warden

  2. Department of Computer Science, University of Miami, Coral Gables, FL, 33146-4245, USA

    Ubbo Visser

Authors
  1. Tobias Warden
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ubbo Visser
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tobias Warden.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Warden, T., Visser, U. Real-time spatio-temporal analysis of dynamic scenes. Knowl Inf Syst 32, 243–279 (2012). https://doi.org/10.1007/s10115-011-0422-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-011-0422-4

Keywords

Search

Navigation