Skip to main content
SpringerLink
Log in

Logic-based interpretation of geometrically observable changes occurring in dynamic scenes

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The work presented here is about employing a theory of updates to study geometrically observable changes that occur in spatial information about image sequences of a dynamic scene. The logical framework consists of a formalism for specifying the geometrical content of a scene, as well as the changes that occur in this geometry, and an algorithm for constructing a description for such changes from logical deductions. In this approach, a database state represents the available sensor data at a particular time instant. Transitions in sensor data are modeled by changes in the database and interpreted based on axioms encoding commonsense spatial reasoning. The main contribution of this work is that it provides the theoretical foundations for symbolically interpreting long sequences of sensor data transitions. For testing the framework and its implementation, the problem of interpreting rotational movements of objects in a sequence of images was used. Our experiments show that the system correctly interprets rotational movements for objects of different colors and provides satisfactory results for interpreting such movements from perceptually indistinguishable objects.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Randell D, Cui Z, Cohn A (1992) A spatial logic based on regions and connection. In: Proceedings of KR, Cambridge, pp 165–176

  2. Moratz R, Renz J, Wolter D (2000) Qualitative spatial reasoning about line segments. In: ECAI, pp 234–238

  3. Schlieder C (1996) Qualitative shape representation. In: Burrough PA, Frank AU (eds) Geographic objects with indeterminate boundaries. Taylor & Francis, London, pp 123–140

    Google Scholar 

  4. Freksa C (1992) Using orientation information for qualitative spatial reasoning. In: Theories and methods of spatial-temporal reasoning in geographic space. Lecture notes in computer science, vol 629. Springer, Berlin

    Google Scholar 

  5. Ligozat G (1998) Reasoning about cardinal directions. J Vis Lang Comput 9(1):23–44

    Article  Google Scholar 

  6. Cohn AG, Hazarika SM (2001) Qualitative spatial representation and reasoning: an overview. Fundam Inform 46(1–2):1–29

    MATH  MathSciNet  Google Scholar 

  7. Stock O (ed) (1997) Spatial and temporal reasoning. Kluwer Academic, Dordrecht

    Google Scholar 

  8. Randell DA, Cohn AG, Cui Z (1992) Computing transitivity tables: a challenge for automated theorem provers. In: Kapur D (ed) Proceedings of CADE, Saratoga Springs. Lecture notes in computer science. Springer, Berlin, pp 786–790

    Google Scholar 

  9. Cohn AG, Bennett B, Gooday J, Gotts N (1997) Representing and reasoning with qualitative spatial relations about regions. In: Stock O (ed) Spatial and temporal reasoning. Kluwer Academic, Dordrecht, pp 97–134

    Chapter  Google Scholar 

  10. Cui Z, Cohn A, Randell D (1992) Qualitative simulation based on a logic of space and time. In: Proceedings of AAAI, California, pp 679–684

  11. Gotts N (1994) How far can we ‘C’? Defining a ‘doughnut’ using connection alone. In: Proceedings of KR, Bon, Germany, pp 246–257

  12. Wolter F, Zakharyaschev M (2000) Spatio-temporal representation and reasoning based on RCC-8. In: Proceedings of KR, San Francisco, pp 3–14

  13. Muller P (2002) Topological spatio-temporal reasoning and representation. Comput Intell 18(3):420–450

    Article  Google Scholar 

  14. Randell D, Witkowski M (2002) Building large composition tables via axiomatic theories. In: Proceedings of KR, Toulouse, France, pp 26–35

  15. Köhler C (2002) The occlusion calculus. In: Proceedings of cognitive vision workshop, Zürich, Switzerland

  16. Reinz J, Nebel B (1999) On the complexity of qualitative spatial reasoning: a maximal tractable fragment of the region connection calculus. Artif Intell 108:69–123

    Article  Google Scholar 

  17. Galton A (1994) Lines of sight. In: Proceedings of the seventh annual conference of AI and cognitive science, Dublin, Ireland, pp 103–113

  18. Randell D, Witkowski M, Shanahan M (2001) From images to bodies: Modeling and exploiting spatial occlusion and motion parallax. In: Proceedings of IJCAI, Seattle, pp 57–63

  19. Santos PE (2007) Reasoning about depth and motion from an observer’s viewpoint. Spat Cogn Comput 7(2):133–178

    Google Scholar 

  20. Galton A (2000) Qualitative spatial change. Oxford University Press, Oxford

    Google Scholar 

  21. Galton A (1995) Towards a qualitative theory of movement. In: Spatial information theory, pp 377–396

  22. Erwig M, Schneider M (2002) Spatio-temporal predicates. IEEE Trans Knowl Data Eng 14(4):881–901

    Article  Google Scholar 

  23. Erwig M (2004) Toward Spatiotemporal Patterns. In: Spatio-temporal databases. Springer, Berlin, pp 29–54

    Google Scholar 

  24. Santos P, Shanahan M (2002) Hypothesising object relations from image transitions. In: van Harmelen F (ed) Proceedings of ECAI, Lyon, France, pp 292–296

  25. Reiter R, Mackworth A (1989) A logical framework for depiction and image interpretation. Artif Intell 41(2):125–155

    Article  MATH  MathSciNet  Google Scholar 

  26. Matsuyama T, Hwang VS (1990) SIGMA: a knowledge-based image understanding system. Plenum, New York

    Google Scholar 

  27. Poole D, Goebel R, Aleliunas R (1987) Theorist: a logical reasoning system for defaults and diagnosis. In: Cercone N, McCalla G (eds) The knowledge frontier—essays in the representation of knowledge. Springer, Berlin, pp 331–352

    Google Scholar 

  28. Schroeder C, Neumann B (1996) On the logics of image interpretation: model construction in a formal knowledge representation framework. In: International conference on image processing, Switzerland, vol 2, pp 785–788

  29. Neumann B, Möller R (2008) On scene interpretation with description logics. Image Vis Comput 26(1):82–101

    Article  Google Scholar 

  30. Shanahan M (1996) Robotics and the common sense informatic situation. In: Proceedings of ECAI, Budapest, Hungary, pp 684–688

  31. Santos P, Shanahan M (2003) A logic-based algorithm for image sequence interpretation and anchoring. In: Proceedings of IJCAI, Acapulco, Mexico, pp 1408–1410

  32. Hazarika SM, Cohn AG (2002) Abducing qualitative spatio-temporal histories from partial observations. In: Proceedings of KR, Toulouse, France, pp 14–25

  33. Fernyhough J, Cohn AG, Hogg DC (2000) Constructing qualitative event models automatically from video input. Image Vis Comput 18:81–103

    Article  Google Scholar 

  34. Bennett B, Cohn A, Magee D (2005) Enforcing global spatio-temporal consistency to enhance reliability of moving object tracking and classification. Künstl Intell 2:32–35

    Google Scholar 

  35. Nagel H-H (1977) Analysing sequences of tv-frames: System design considerations. In: Proceedings of IJCAI, Cambridge, p 626

  36. Tsotsos JK, Mylopoulos J, Covvey HD, Zucker SW (1980) A framework for visual motion understanding. IEEE Trans Pattern Anal Mach Intell 2(6):563–573, Special Issue on Computer Analysis of Time-Varying Imagery

    Google Scholar 

  37. Tsotsos JK (1985) Knowledge organization and its role in representation and interpretation for time-varying data: the ALVEN system. Comput Intell 1:16–32

    Article  Google Scholar 

  38. Herzog G (1995) From visual input to verbal output in the visual translator. Technical Report 124, Universitat des Saarlandes

  39. Herzog G, Wazinski P (1994) Visual Translator: linking perceptions and natural language descriptions. Artif Intell Rev 8(2–3):175–187

    Article  Google Scholar 

  40. Gerber R, Nagel H-H, Schreiber H (2002) Deriving textual descriptions of road traffic queues from video sequences. In: Proceedings of ECAI, Lyon, France, pp 736–740

  41. Nagel H-H (2000) Image sequence evaluation: 30 years and still going strong. In: Proceedings of ICPR, Barcelona, Spain, pp 1149–1158

  42. Nagel H-H (1988) From image sequences towards conceptual descriptions. Image Vis Comput 6(2):59–74

    Article  Google Scholar 

  43. Bouthemy P, François E (1993) Motion segmentation qualitative dynamic scene analysis from an image sequence. Int J Comput Vis 10(2):157–182

    Article  Google Scholar 

  44. Mitiche A, Bouthemy P (1996) Computation and analysis of image motion: a synopsis of current problems and methods. Int J Comput Vis 19(1):29–55

    Article  Google Scholar 

  45. Buxton H (2002) Learning and understanding dynamic scenes activity: a review. Image Vis Comput 21(1):125–136

    Article  Google Scholar 

  46. Frank T, Haag M, Kollnig H, Nagel H-H (1996) Characterization of occlusion situations occurring in real-world traffic scenes. In: Proceedings of the workshop on conceptual descriptions from images, ECCV, Cambridge, UK, pp 43–57

  47. Brand M (1997) Physics-based visual understanding. Comput Vis Image Underst 65(2):192–205

    Article  MathSciNet  Google Scholar 

  48. Brand M, Birnbaum L, Cooper P (1993) Sensible scenes: visual understanding of complex structures through causal analysis. In: Proceedings of AAAI, Washington, DC, pp 588–593

  49. Brand M (1996) Understanding manipulation in video. In: Proceedings of the 2nd international conference on face and gesture recognition, pp 94–99

  50. Siskind JM (1995) Grounding language in perception. Artif Intell Rev 8(5–6):371–391

    Article  Google Scholar 

  51. Mann R, Jepson A, Siskind JM (1997) The computational perception of scene dynamics. Comput Vis Image Underst 65(2):113–128

    Article  Google Scholar 

  52. Siskind JM (2000) Visual event classification via force dynamics. In: Proceedings of AAAI, Austin, pp 149–155

  53. Hayes PJ (1984) The second naïve physics manifesto. In: Hobbs J, Moore RC (eds) Formal theories of the common sense world. Ablex, Norwood

    Google Scholar 

  54. Gärdenfors P (2000) Conceptual Spaces: the geometry of thought. MIT Press, Cambridge

    Google Scholar 

  55. Chella A, Frixione M, Gaglio S (2000) Understanding dynamic scenes. Artif Intell 123(1–2):89–132

    Article  MATH  Google Scholar 

  56. Reiter R (2002) Knowledge in action. MIT Press, Cambridge

    Google Scholar 

  57. Bonner A, Kifer M (1993) Transaction logic programming. In: Proceedings of the tenth international conference on logic programming (ICLP). MIT Press, Cambridge, pp 257–279

    Google Scholar 

  58. Bonner A, Kifer M (1998) A logic for programming database transactions. In: Logics for databases and information systems. Kluwer Academic, Dordrecht

    Google Scholar 

  59. Ballard DH, Brown C (1982) Computer vision. Prentice Hall, Englewood Cliffs

    Google Scholar 

  60. Marr D (1982) Vision: a computational investigation into the human representation and processing of visual information. Freeman, San Francisco

    Google Scholar 

  61. Huang C (1990) Contour generation and shape restoration of the straight homogeneous generalized cylinder. Int Conf Pattern Recognit A 90:409–413

    Google Scholar 

  62. Cabalar P, Santos P (2006) Strings and holes: an exercise on spatial reasoning. In: Sichman J (ed) Proceedings of SBIA–IBERAMIA. Lecture notes in artificial intelligence, vol 4140. Springer, Berlin, pp 419–429

    Google Scholar 

  63. Santos P, Cabalar P (2007) Holes, knots and shapes: a spatial ontology of a puzzle. In: 8th international symposium on logical formalizations of commonsense reasoning (Commonsense’07), Stanford, CA

  64. Anger F, Rodriguez R, Guesgen H, van Benthem J (1996) Space, time, and computation: trends and problems. Appl Intell 6:5–9

    Article  Google Scholar 

  65. Harel D, Kozen D, Parikh R (1982) Process logic: expressiveness, decidability, completeness. J Comput Syst Sci 2(25):144–170

    Article  MathSciNet  Google Scholar 

  66. Guesgen H (2002) Reasoning about distance based on fuzzy sets. Appl Intell 17(3):265–270

    Article  MATH  Google Scholar 

  67. Hernández D, Clementini E, di Felice P (1995) Qualitative distances. In: Spatial information theory. Lecture notes in computer science, vol 988. Springer, Berlin, pp 45–57

    Google Scholar 

  68. Newell A (1982) The knowledge level. Artif Intell 18(1):87–127

    Article  Google Scholar 

  69. Freksa C (1991) Conceptual neighbourhood and its role in temporal and spatial reasoning. In: Decision support systems and qualitative reasoning. Elsevier Science, Amsterdam, pp 181–193

    Google Scholar 

  70. Kuipers B (1994) Qualitative reasoning: modelling and simulation with incomplete knowledge. MIT Press, Cambridge

    Google Scholar 

  71. Bonner A, Kifer M (1995) Transaction logic programming (or a logic of declarative and procedural knowledge). Tech. Rep. CSRI-323, University of Toronto, November 1995. http://www.cs.toronto.edu/~bonner/transaction-logic.html

  72. Santore J, Shapiro S (2002) Identifying perceptually indistinguishable objects: Is that the same one you saw before? In: AAAI workshop on cognitive robotics, Edmonton, Canada, pp 96–102

  73. Shanahan M (1999) What sort of computation mediates between perception and action? In: Logical foundations for cognitive agents: contributions in honor of Ray Reiter. Springer, Berlin, pp 352–369

    Google Scholar 

  74. Needham C, Santos P, Magee D, Devin V, Hogg D, Cohn A (2005) Protocols from perceptual observations. Artif Intell J 167:103–136

    Article  Google Scholar 

  75. Santos P, Magee D, Cohn A, Hogg D (2004) Combining multiple answers for learning mathematical structures from visual observation. In: Proceedings of the 16th European conference on artificial intelligence (ECAI-04), Valencia, Spain

  76. Dambreville S, Rathi Y, Tannen A (2006) Shape-based approach to robust image segmentation using kernel PCA. In: CVPR ’06: proceedings of the 2006 IEEE computer society conference on computer vision and pattern recognition, Washington, DC. IEEE Computer Society, Los Alamitos, pp 977–984

    Google Scholar 

  77. Chen YB, Chen OT-C (2006) Robust image segmentation using modified edge-following scheme with automatically-determined thresholds. In: Proceedings of the first conference on innovative computing, information and control, vol 3, pp 292–295

Download references

Author information

Authors and Affiliations

  1. Ryerson University, 350 Victoria Street, Toronto, Ontario, M5B 2K3, Canada

    M. V. dos Santos

  2. Centro Universitario da FEI, Sao Paulo, Brazil

    R. C. de Brito & P. Santos

  3. Chung-Ang University, Seoul, South Korea

    H.-H. Park

Authors
  1. M. V. dos Santos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. R. C. de Brito
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. H.-H. Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. P. Santos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. V. dos Santos.

Rights and permissions

Reprints and permissions

About this article

Cite this article

dos Santos, M.V., de Brito, R.C., Park, HH. et al. Logic-based interpretation of geometrically observable changes occurring in dynamic scenes. Appl Intell 31, 161–179 (2009). https://doi.org/10.1007/s10489-008-0120-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-008-0120-4

Keywords

Search

Navigation