Abstract
The work presented here is about employing a theory of updates to study geometrically observable changes that occur in spatial information about image sequences of a dynamic scene. The logical framework consists of a formalism for specifying the geometrical content of a scene, as well as the changes that occur in this geometry, and an algorithm for constructing a description for such changes from logical deductions. In this approach, a database state represents the available sensor data at a particular time instant. Transitions in sensor data are modeled by changes in the database and interpreted based on axioms encoding commonsense spatial reasoning. The main contribution of this work is that it provides the theoretical foundations for symbolically interpreting long sequences of sensor data transitions. For testing the framework and its implementation, the problem of interpreting rotational movements of objects in a sequence of images was used. Our experiments show that the system correctly interprets rotational movements for objects of different colors and provides satisfactory results for interpreting such movements from perceptually indistinguishable objects.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Randell D, Cui Z, Cohn A (1992) A spatial logic based on regions and connection. In: Proceedings of KR, Cambridge, pp 165–176
Moratz R, Renz J, Wolter D (2000) Qualitative spatial reasoning about line segments. In: ECAI, pp 234–238
Schlieder C (1996) Qualitative shape representation. In: Burrough PA, Frank AU (eds) Geographic objects with indeterminate boundaries. Taylor & Francis, London, pp 123–140
Freksa C (1992) Using orientation information for qualitative spatial reasoning. In: Theories and methods of spatial-temporal reasoning in geographic space. Lecture notes in computer science, vol 629. Springer, Berlin
Ligozat G (1998) Reasoning about cardinal directions. J Vis Lang Comput 9(1):23–44
Cohn AG, Hazarika SM (2001) Qualitative spatial representation and reasoning: an overview. Fundam Inform 46(1–2):1–29
Stock O (ed) (1997) Spatial and temporal reasoning. Kluwer Academic, Dordrecht
Randell DA, Cohn AG, Cui Z (1992) Computing transitivity tables: a challenge for automated theorem provers. In: Kapur D (ed) Proceedings of CADE, Saratoga Springs. Lecture notes in computer science. Springer, Berlin, pp 786–790
Cohn AG, Bennett B, Gooday J, Gotts N (1997) Representing and reasoning with qualitative spatial relations about regions. In: Stock O (ed) Spatial and temporal reasoning. Kluwer Academic, Dordrecht, pp 97–134
Cui Z, Cohn A, Randell D (1992) Qualitative simulation based on a logic of space and time. In: Proceedings of AAAI, California, pp 679–684
Gotts N (1994) How far can we ‘C’? Defining a ‘doughnut’ using connection alone. In: Proceedings of KR, Bon, Germany, pp 246–257
Wolter F, Zakharyaschev M (2000) Spatio-temporal representation and reasoning based on RCC-8. In: Proceedings of KR, San Francisco, pp 3–14
Muller P (2002) Topological spatio-temporal reasoning and representation. Comput Intell 18(3):420–450
Randell D, Witkowski M (2002) Building large composition tables via axiomatic theories. In: Proceedings of KR, Toulouse, France, pp 26–35
Köhler C (2002) The occlusion calculus. In: Proceedings of cognitive vision workshop, Zürich, Switzerland
Reinz J, Nebel B (1999) On the complexity of qualitative spatial reasoning: a maximal tractable fragment of the region connection calculus. Artif Intell 108:69–123
Galton A (1994) Lines of sight. In: Proceedings of the seventh annual conference of AI and cognitive science, Dublin, Ireland, pp 103–113
Randell D, Witkowski M, Shanahan M (2001) From images to bodies: Modeling and exploiting spatial occlusion and motion parallax. In: Proceedings of IJCAI, Seattle, pp 57–63
Santos PE (2007) Reasoning about depth and motion from an observer’s viewpoint. Spat Cogn Comput 7(2):133–178
Galton A (2000) Qualitative spatial change. Oxford University Press, Oxford
Galton A (1995) Towards a qualitative theory of movement. In: Spatial information theory, pp 377–396
Erwig M, Schneider M (2002) Spatio-temporal predicates. IEEE Trans Knowl Data Eng 14(4):881–901
Erwig M (2004) Toward Spatiotemporal Patterns. In: Spatio-temporal databases. Springer, Berlin, pp 29–54
Santos P, Shanahan M (2002) Hypothesising object relations from image transitions. In: van Harmelen F (ed) Proceedings of ECAI, Lyon, France, pp 292–296
Reiter R, Mackworth A (1989) A logical framework for depiction and image interpretation. Artif Intell 41(2):125–155
Matsuyama T, Hwang VS (1990) SIGMA: a knowledge-based image understanding system. Plenum, New York
Poole D, Goebel R, Aleliunas R (1987) Theorist: a logical reasoning system for defaults and diagnosis. In: Cercone N, McCalla G (eds) The knowledge frontier—essays in the representation of knowledge. Springer, Berlin, pp 331–352
Schroeder C, Neumann B (1996) On the logics of image interpretation: model construction in a formal knowledge representation framework. In: International conference on image processing, Switzerland, vol 2, pp 785–788
Neumann B, Möller R (2008) On scene interpretation with description logics. Image Vis Comput 26(1):82–101
Shanahan M (1996) Robotics and the common sense informatic situation. In: Proceedings of ECAI, Budapest, Hungary, pp 684–688
Santos P, Shanahan M (2003) A logic-based algorithm for image sequence interpretation and anchoring. In: Proceedings of IJCAI, Acapulco, Mexico, pp 1408–1410
Hazarika SM, Cohn AG (2002) Abducing qualitative spatio-temporal histories from partial observations. In: Proceedings of KR, Toulouse, France, pp 14–25
Fernyhough J, Cohn AG, Hogg DC (2000) Constructing qualitative event models automatically from video input. Image Vis Comput 18:81–103
Bennett B, Cohn A, Magee D (2005) Enforcing global spatio-temporal consistency to enhance reliability of moving object tracking and classification. Künstl Intell 2:32–35
Nagel H-H (1977) Analysing sequences of tv-frames: System design considerations. In: Proceedings of IJCAI, Cambridge, p 626
Tsotsos JK, Mylopoulos J, Covvey HD, Zucker SW (1980) A framework for visual motion understanding. IEEE Trans Pattern Anal Mach Intell 2(6):563–573, Special Issue on Computer Analysis of Time-Varying Imagery
Tsotsos JK (1985) Knowledge organization and its role in representation and interpretation for time-varying data: the ALVEN system. Comput Intell 1:16–32
Herzog G (1995) From visual input to verbal output in the visual translator. Technical Report 124, Universitat des Saarlandes
Herzog G, Wazinski P (1994) Visual Translator: linking perceptions and natural language descriptions. Artif Intell Rev 8(2–3):175–187
Gerber R, Nagel H-H, Schreiber H (2002) Deriving textual descriptions of road traffic queues from video sequences. In: Proceedings of ECAI, Lyon, France, pp 736–740
Nagel H-H (2000) Image sequence evaluation: 30 years and still going strong. In: Proceedings of ICPR, Barcelona, Spain, pp 1149–1158
Nagel H-H (1988) From image sequences towards conceptual descriptions. Image Vis Comput 6(2):59–74
Bouthemy P, François E (1993) Motion segmentation qualitative dynamic scene analysis from an image sequence. Int J Comput Vis 10(2):157–182
Mitiche A, Bouthemy P (1996) Computation and analysis of image motion: a synopsis of current problems and methods. Int J Comput Vis 19(1):29–55
Buxton H (2002) Learning and understanding dynamic scenes activity: a review. Image Vis Comput 21(1):125–136
Frank T, Haag M, Kollnig H, Nagel H-H (1996) Characterization of occlusion situations occurring in real-world traffic scenes. In: Proceedings of the workshop on conceptual descriptions from images, ECCV, Cambridge, UK, pp 43–57
Brand M (1997) Physics-based visual understanding. Comput Vis Image Underst 65(2):192–205
Brand M, Birnbaum L, Cooper P (1993) Sensible scenes: visual understanding of complex structures through causal analysis. In: Proceedings of AAAI, Washington, DC, pp 588–593
Brand M (1996) Understanding manipulation in video. In: Proceedings of the 2nd international conference on face and gesture recognition, pp 94–99
Siskind JM (1995) Grounding language in perception. Artif Intell Rev 8(5–6):371–391
Mann R, Jepson A, Siskind JM (1997) The computational perception of scene dynamics. Comput Vis Image Underst 65(2):113–128
Siskind JM (2000) Visual event classification via force dynamics. In: Proceedings of AAAI, Austin, pp 149–155
Hayes PJ (1984) The second naïve physics manifesto. In: Hobbs J, Moore RC (eds) Formal theories of the common sense world. Ablex, Norwood
Gärdenfors P (2000) Conceptual Spaces: the geometry of thought. MIT Press, Cambridge
Chella A, Frixione M, Gaglio S (2000) Understanding dynamic scenes. Artif Intell 123(1–2):89–132
Reiter R (2002) Knowledge in action. MIT Press, Cambridge
Bonner A, Kifer M (1993) Transaction logic programming. In: Proceedings of the tenth international conference on logic programming (ICLP). MIT Press, Cambridge, pp 257–279
Bonner A, Kifer M (1998) A logic for programming database transactions. In: Logics for databases and information systems. Kluwer Academic, Dordrecht
Ballard DH, Brown C (1982) Computer vision. Prentice Hall, Englewood Cliffs
Marr D (1982) Vision: a computational investigation into the human representation and processing of visual information. Freeman, San Francisco
Huang C (1990) Contour generation and shape restoration of the straight homogeneous generalized cylinder. Int Conf Pattern Recognit A 90:409–413
Cabalar P, Santos P (2006) Strings and holes: an exercise on spatial reasoning. In: Sichman J (ed) Proceedings of SBIA–IBERAMIA. Lecture notes in artificial intelligence, vol 4140. Springer, Berlin, pp 419–429
Santos P, Cabalar P (2007) Holes, knots and shapes: a spatial ontology of a puzzle. In: 8th international symposium on logical formalizations of commonsense reasoning (Commonsense’07), Stanford, CA
Anger F, Rodriguez R, Guesgen H, van Benthem J (1996) Space, time, and computation: trends and problems. Appl Intell 6:5–9
Harel D, Kozen D, Parikh R (1982) Process logic: expressiveness, decidability, completeness. J Comput Syst Sci 2(25):144–170
Guesgen H (2002) Reasoning about distance based on fuzzy sets. Appl Intell 17(3):265–270
Hernández D, Clementini E, di Felice P (1995) Qualitative distances. In: Spatial information theory. Lecture notes in computer science, vol 988. Springer, Berlin, pp 45–57
Newell A (1982) The knowledge level. Artif Intell 18(1):87–127
Freksa C (1991) Conceptual neighbourhood and its role in temporal and spatial reasoning. In: Decision support systems and qualitative reasoning. Elsevier Science, Amsterdam, pp 181–193
Kuipers B (1994) Qualitative reasoning: modelling and simulation with incomplete knowledge. MIT Press, Cambridge
Bonner A, Kifer M (1995) Transaction logic programming (or a logic of declarative and procedural knowledge). Tech. Rep. CSRI-323, University of Toronto, November 1995. http://www.cs.toronto.edu/~bonner/transaction-logic.html
Santore J, Shapiro S (2002) Identifying perceptually indistinguishable objects: Is that the same one you saw before? In: AAAI workshop on cognitive robotics, Edmonton, Canada, pp 96–102
Shanahan M (1999) What sort of computation mediates between perception and action? In: Logical foundations for cognitive agents: contributions in honor of Ray Reiter. Springer, Berlin, pp 352–369
Needham C, Santos P, Magee D, Devin V, Hogg D, Cohn A (2005) Protocols from perceptual observations. Artif Intell J 167:103–136
Santos P, Magee D, Cohn A, Hogg D (2004) Combining multiple answers for learning mathematical structures from visual observation. In: Proceedings of the 16th European conference on artificial intelligence (ECAI-04), Valencia, Spain
Dambreville S, Rathi Y, Tannen A (2006) Shape-based approach to robust image segmentation using kernel PCA. In: CVPR ’06: proceedings of the 2006 IEEE computer society conference on computer vision and pattern recognition, Washington, DC. IEEE Computer Society, Los Alamitos, pp 977–984
Chen YB, Chen OT-C (2006) Robust image segmentation using modified edge-following scheme with automatically-determined thresholds. In: Proceedings of the first conference on innovative computing, information and control, vol 3, pp 292–295
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
dos Santos, M.V., de Brito, R.C., Park, HH. et al. Logic-based interpretation of geometrically observable changes occurring in dynamic scenes. Appl Intell 31, 161–179 (2009). https://doi.org/10.1007/s10489-008-0120-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-008-0120-4