Skip to main content
SpringerLink
Log in

The effect of occlusion on the semantics of projective spatial terms: a case study in grounding language in perception

  • Research Report
  • Published:
Cognitive Processing Aims and scope Submit manuscript

Abstract

Although data-driven spatial template models provide a practical and cognitively motivated mechanism for characterizing spatial term meaning, the influence of perceptual rather than solely geometric and functional properties has yet to be systematically investigated. In the light of this, in this paper, we investigate the effects of the perceptual phenomenon of object occlusion on the semantics of projective terms. We did this by conducting a study to test whether object occlusion had a noticeable effect on the acceptance values assigned to projective terms with respect to a 2.5-dimensional visual stimulus. Based on the data collected, a regression model was constructed and presented. Subsequent analysis showed that the regression model that included the occlusion factor outperformed an adaptation of Regier & Carlson’s well-regarded AVS model for that same spatial configuration.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Notes

  1. See Mukerjee (1998) for an overview of seminal approaches.

  2. In this paper, we use Logan & Sadler’s term spatial template to describe both Logan and Sadler’s (1996) psychological account of how people apprehend spatial relations using spatial templates and a set of computational models that are primarily based on geometric factors, such as distance between the trajector and landmark, e.g., (Gapp, 1994; Regier and Carlson 2001; Kelleher and Costello 2009). From a functional perspective, there is no distinction between these two groups. However, from a psychological perspective, they make different predictions: Logan & Sadler’s theory proposes that people apply a template over all of their visual field at once, meaning that factors such as distance between trajector and landmark should not play a role, whereas the computational models predict that these factors do play a role. Notwithstanding this distinction, we use the term spatial template to describe both accounts because fundamentally they both use a template-based semantic representation.

  3. It should be noted that Logan & Sadler analysed a number of spatial relations in a similar fashion and that the general form of the spatial template presented in Fig. 1 is not specific to the preposition above or to a particular landmark shape (Carlson-Radvansky and Logan 1997; Logan and Sadler 1996).

  4. See (Logan 1994; Logan 1995).

  5. https://www.mturk.com.

  6. The condition 1 pairs where both trajectors occluded the landmark.

  7. The condition 2 pairs where only the first trajector occluded the landmark

  8. Similarly, it is interesting to note that for acceptability judgments for “to the right” and “to the left”, we see that landmark occlusion seems to have an effect on term acceptability. Namely, taking groups YL2–YL5 and YR2–YR5 for the case of “to the right” and “to the left”, respectively, the results indicate that occlusion of the landmark by the trajector results in an accelerated drop in acceptance values. For “to the right” |YR3–YR4| = 0.75 and |YR4–YR5| = 1.714, while for “to the left” |YL4–YL5| = 1.143 and |YL3–YL4| = 1.000.

  9. In all of the correlation tests reported below, associated p-values of <0.001 were returned.

References

  • Brenner M, Hawes N, Kelleher J, Wyatt J (2007) Mediating between qualitative and quantitative representations for task-orientated human-robot interaction. In: Proceedings of the twentieth international joint conference on artificial intelligence (IJCAI), Hyderabad, India

  • Carlson-Radvansky LA, Logan GD (1997) The influence of reference frame selection on spatial template construction. J Mem Lang 37:411–437

    Article  Google Scholar 

  • Carlson-Radvansky LA, Radvansky GA (1996) The influence of functional relations on spatial term selection. Psychol Sci 7:56–60

    Article  Google Scholar 

  • Coventry KR, Garrod SC (2004) Saying, seeing and acting. The psychological semantics of spatial prepositions. Essays in cognitive psychology series. Psychology Press, Hove

    Google Scholar 

  • Coventry KR, Prat-Sala M, Richards LV (2001) The interplay between geometry and function in the comprehension of ‘over’, ‘under’, ‘above’ and ‘below’. J Mem Lang 44:376–398

    Article  Google Scholar 

  • Fuhr T, Socher G, Scheering C, Sagerer G (1998) A three-dimensional spatial model for the interpretation of image data. In: Olivier P, Gapp K (eds) Representation and processing of spatial expressions, Lawrence Erlbaum Associates, New Jersey, pp 103–118

  • Gapp K (1994a) Basic meanings of spatial relations: computation and evaluation in 3D space. In: Proceedings of AAAI-94. Seattle, Washington, pp 1393–1398

  • Gapp KP (1994b) Basic meanings of spatial relations: computation and evaluation in 3d space. In: AAAI’94, pp 1393–1398

  • Gorniak P, Roy D (2004) Grounded semantic composition for visual scenes. J Artif Intell Res 21:429–470

    Google Scholar 

  • Heer J, Bostock M (2010) Crowdsourcing graphical perception: using mechanical turk to assess visualization design. In: Mynatt ED, Schoner D, Fitzpatrick G, Hudson SE, Edwards K, Rodden T (eds) CHI. ACM, pp 203–212, http://doi.acm.org/10.1145/1753326.1753357

  • Jackendoff R (1999) The architecture of the linguistic-spatial interface. In: Bloom P, Peterson MA, Nadel L, Garrett MF (eds) Language and space. MIT Press, Cambride, MA, pp 1–30

    Google Scholar 

  • Jackendoff R, Landau B (1992) Spatial language and spatial cognition. In: Jackendoff R (eds) Languages of the mind. MIT Press, Cambride, MA, pp 99–125

    Google Scholar 

  • Kelleher J (2003) A perceptually based computational framework for the interpretation of spatial language. PhD thesis, Dublin City University

  • Kelleher J, Costello F (2005) Cognitive representations of projective prepositions. In: Kordoni V, Valencia A (eds) In Proceedings of the 2nd ACL-Sigsem workshop on the linguistic dimensions of prepositions and their use in computational linguistics formalisms and applications

  • Kelleher J, Kruijff GJ, Costello F (2006) Proximity in context: an empirically grounded computational model of proximity for processing topological spatial expression. In: Proceedings of Coling-ACL ’06, Sydney Australia

  • Kelleher JD, Costello FJ (2009) Applying computational models of spatial prepositions to visually situated dialog. Comput Linguist 35(2):119–149, http://aclweb.org/anthology-new/J/J09/J09-2005.pdf

    Google Scholar 

  • Langacker RW (1987) Foundations in cognitive grammar, vol 1. Theoretical prerequisites. Stanford University Press, Stanford, CA

    Google Scholar 

  • Logan G (1994) Spatial attention and the apprehension of spatial relations. J Exp Psychol Hum Percept Perform 20:1015–1036

    Article  CAS  PubMed  Google Scholar 

  • Logan G (1995) Linguistic and conceptual control of visual spatial attention. Cognit Psychol 12:523–533

    Article  Google Scholar 

  • Logan GD, Sadler DD (1996) A computational analysis of the apprehension of spatial relations. In: Bloom P, Peterson MA, Nadel L, Garrett MF (eds) Language and space. MIT Press, Cambride, MA, pp 493–530

    Google Scholar 

  • Mukerjee A (1998) Neat versus scruffy: a review of computational models for spatial expressions. In: Olivier P, Gapp KP (eds) Representation and processing of spatial expressions. Lawrence Erlbaum Associates, Mahwah, NJ, pp 1–36, http://citeseer.ist.psu.edu/250615.html

  • Olivier P, Tsujii J (1994) Quantitative perceptual representation of prepositional semantics. Artif Int Rev 8:147–158

    Google Scholar 

  • Regier T, Carlson LA (2001) Grounding spatial language in perception: an empirical and computational investigation. J Exp Psychol Gen 130(2):273–298

    Article  CAS  PubMed  Google Scholar 

  • Ross RJ (2009) Situated dialogue systems: agency & spatial meaning in task-oriented dialogue. PhD thesis, Universitat Bremen

  • Roy D, Reiter E (2005) Connecting language to the world. Artif Int 167(1–2):1–12

    Article  Google Scholar 

  • Talmy L (2000) Towards a cognitive semantics. A bradford book. MIT Press, Cambridge, MA

    Google Scholar 

  • Vandeloise C (1991) Spatial prepositions: a case study from French. The University of Chicago Press, Chicago

    Google Scholar 

  • Yamada A (1993) Studies in spatial descriptions understanding based on geometric constraints satisfaction. PhD thesis, University of Kyoto

Download references

Acknowledgments

The authors would like to thank our anonymous reviewers for valuable comments and feedback. We would also like to thank Ms. Aoife D’Arcy for her advice on which statistical techniques to use in our tests. Finally, the second author wishes to thank the Irish Research Council for Science Engineering and Technology for their generous financial support through an Empower Government of Ireland Research Fellowship.

Author information

Authors and Affiliations

  1. Applied Intelligence Research Centre, Dublin Institute of Technology, Dublin, Ireland

    John D. Kelleher, Robert J. Ross, Colm Sloan & Brian Mac Namee

Authors
  1. John D. Kelleher
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robert J. Ross
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Colm Sloan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Brian Mac Namee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to John D. Kelleher.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kelleher, J.D., Ross, R.J., Sloan, C. et al. The effect of occlusion on the semantics of projective spatial terms: a case study in grounding language in perception. Cogn Process 12, 95–108 (2011). https://doi.org/10.1007/s10339-010-0380-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10339-010-0380-x

Keywords

Search

Navigation