Nelson Wong
ABSTRACT
Deictic reference -- pointing at things during conversation -- is ubiquitous in human communication, and should also be an important tool in distributed collaborative virtual environments (CVEs). Pointing gestures can be complex and subtle, however, and pointing is much more difficult in the virtual world. In order to improve the richness of interaction in CVEs, it is important to provide better support for pointing and deictic reference, and a first step in this support is to determine how well people can interpret the direction that another person is pointing. To investigate this question, we carried out two studies. The first identified several ways that people point towards distant targets, and established that not all pointing requires high accuracy. This suggested that natural CVE pointing could potentially be successful; but no knowledge is available about whether even moderate accuracy is possible in CVEs. Therefore, our second study looked more closely at how accurately people can produce and interpret the direction of pointing gestures in CVEs. We found that although people are more accurate in the real world, the differences are smaller than expected; our results show that deixis can be successful in CVEs for many pointing situations, and provide a foundation for more comprehensive support of deictic pointing.
- Bangerter, A. 2004. Using Pointing and Describing to Achieve Joint Focus of Attention in Dialogue, Psychological Science, 15, 6, 415--418.Google Scholar
Cross Ref
- Bekker, M., Olson, J.S., and Olson, G.M. 1995. Analysis of gestures face-to-face design teams provides guidance for how to use groupware design. Proc. DIS'95, 157--166. Google ScholarDigital Library
- Benford, S., Bowers, J., Lennart, E.F., Greenhalgh, C., and Snowdon, D. 1995. User embodiment collaborative virtual environments. Proc. CHI'95, 242--249. Google ScholarDigital Library
- Bowers, J., Pycock, J., and O'Brien, J. 1996. Talk and embodiment collaborative virtual environments. Proc. CHI '96, 58--65. Google ScholarDigital Library
- Bowers, J., O'Brien, J., and Pycock, J. 1996. Practically accomplishing immersion: Cooperation and for virtual environments. Proc. CSCW '96, 380--389. Google ScholarDigital Library
- Clark, H.H. 2003. Pointing and Placing. S. Kita (Ed.), Pointing. Where language, culture, and cognition meet. Hillsdale NJ: Erlbaum. 243--268.Google Scholar
- Clark, H.H., and Brennan, S.E. Grounding Communication. 1991. L.B. Resnick, R.M. Levine, & S.D. Teasley (Eds.). Perspectives on socially shared cognition. Washington, DC: APA. 127--149.Google Scholar
- Clark, H., and Wilkes-Gibbes, D., Referring as a Collaborative Process, Cognition, 22, 1986, 1--39.Google Scholar
- Duchowski, A., Cournia, N., Cumming, B., McCallum, D., Gramopadhye, A., Greenstein, J., Sadasivan, S., and Tyrrell, R. 2004. Visual deictic reference a collaborative virtual environment. Proc. ETRA, 35--40. Google ScholarDigital Library
- Fussell, S.R., Setlock, L.D., Yang, J., Ou, J., Mauer, E.M., and Kramer, A. 2004. Gestures over video streams to support remote collaboration on physical tasks. Human-Computer Interaction, 19, 273--309. Google ScholarDigital Library
- Fraser, M., and Benford, S. 2002. Interaction effects of virtual structures. Proc. CVE 2002, 128--134. Google ScholarDigital Library
- Fraser, M., Benford, S., Hindmarsh, J., and Heath, C. 1999. Supporting awareness and interaction through collaborative virtual interfaces. Proc. UIST'99, 27--36. Google ScholarDigital Library
- Goodwin, C. 2003. Pointing as Situated Practice. S. Kita (Ed.), Pointing. Where language, culture, and cognition meet. Hillsdale NJ: Erlbaum. 217--241.Google Scholar
- Greenberg, S., Gutwin, C., and Roseman, M., 1996. Semantic Telepointers for Groupware. Proc. OzCHI '96, 54--61. Google ScholarDigital Library
- Gutwin, C., and Penner, R. 2002. Improving Interpretation of Remote Gestures with Telepointer Traces, Proc. CSCW'02, 49--57. Google ScholarDigital Library
- Hindmarsh, J., Fraser, M., Heath, C., Benford, S., and Greenhalgh, C. 1998. Fragmented Interaction: establishing mutual orientation virtual environments. Proc. CSCW'98, 217--226. Google ScholarDigital Library
- Hindmarsh, J., Fraser, M., Heath, C., Benford, S., and Greenhalgh, C. 2000. Object-Focused Interaction in Collaborative Virtual Environments, ToCHI, 7, 4, 477--509. Google ScholarDigital Library
- Hindmarsh, J., and Heath, C. 2000. Embodied reference: A study of deixis workplace interaction. J. Prag. 32, 12, 1855--1878.Google ScholarCross Ref
- Kelly, J., Bell, A., and Loomis, J. 2004. Perception of Shared Visual Space: Establishing Common Ground in Real and Virtual Environments, Presence, 13, 4, 442--450. Google ScholarDigital Library
- Kirk, D., Rodden, T., and Fraser, D.S. 2007. Turn it this way: grounding collaborative action with remote gestures. Proc CHI'07, 1039--1048. Google ScholarDigital Library
- Kita, S. (Ed.). 2003. Pointing: Where language, culture and cognition meet. Mahwah, NJ: Erlbaum.Google ScholarCross Ref
- Krauss, R.M., and Fussell, S.R. 1990. Mutual knowledge and communicative effectiveness. In, J. Galegher, R.E. Kraut & C. Egido (Eds.), Intellectual Teamwork: Social and Technical Bases of Collaborative Work. Hillsdale, NJ: Erlbaum. 111--145. Google ScholarDigital Library
- McNeill, D., Cassell, J., and Levy, E.T. 1993. Abstract deixis. Semiotica, 95(1/2), 5--19.Google Scholar
- Moore, R., Ducheneaut, N., and Nickell, E. 2007. Doing Virtually Nothing: Awareness and Accountability in Massively Multiplayer Online Worlds, JCSCW, 16, 3, 265--305. Google ScholarDigital Library
- Pechmann, T., and Deutsch W. 1982. The Development of Verbal and Nonverbal Devices for Reference. Journal of Experimental Child Psychology 34, 330--341.Google ScholarCross Ref
- Tang, A., Boyle, M., and Greenberg, S. 2005. Display and Presence Disparity Mixed Presence Groupware. JRPIT, 37, 2, 71--88.Google Scholar
- Tang, J. 1991. Findings from Observational Studies of Collaborative Work, IJMMS, 34, 2, 143--160. Google ScholarDigital Library
- Tang, J.C., and Minneman, S.L. 1990. VideoDraw: a video interface for collaborative drawing, Proc. CHI'90, 313--320. Google ScholarDigital Library
- Tang, J., and Minneman, S. 1991. VideoWhiteboard: video shadows to support remote collaboration, Proc. CHI'91, 315--322. Google ScholarDigital Library
Index Terms
- Where are you pointing?: the accuracy of deictic pointing in CVEs
Recommendations
Support for deictic pointing in CVEs: still fragmented after all these years'
CSCW '14: Proceedings of the 17th ACM conference on Computer supported cooperative work & social computingPointing gestures -- particularly deictic references -- are ubiquitous in face-to-face communication. However, deictic pointing can be much more difficult in collaborative virtual environments (CVEs) than in everyday life -- early studies found that the ...
Accuracy of interpreting pointing gestures in egocentric view
UbiComp '16: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous ComputingCommunicating spatial information by pointing is ubiquitous in human interactions. With the growing use of head-mounted cameras for collaborative purposes, it is important to assess how accurately viewers of the resulting egocentric videos can interpret ...
Controlling an avatar's pointing gestures in desktop collaborative virtual environments
GROUP '12: Proceedings of the 2012 ACM International Conference on Supporting Group WorkCollaborative Virtual Environments (CVEs) allow people to interact with each other in virtual worlds through computer-generated avatars. Avatars are much less expressive than real bodies, and one main limitation is their lack of support for non-verbal ...
Comments