skip to main content
10.1145/2535948.2535957acmconferencesArticle/Chapter ViewAbstractPublication Pagesicmi-mlmiConference Proceedingsconference-collections
research-article

Finding the timings for a guide agent to interveneinter-user conversation in considering their gazebehaviors

Published: 13 December 2013 Publication History

Abstract

As the advance of embodied conversational agent (ECA) technologies, there are more and more real-world deployed applications of ECA's like the guides in museums or exhibitions. However, in those situations, the agent systems are usually used by groups of visitors rather than individuals. In such multi-user situation which is much more complex than single user one, specific features are required. One of them is the ability for the agent to smoothly intervene user-user conversation. This feature is supposed to facilitate mixed-initiative human-agent conversation and more proactive service for the users. This paper presents the results of the first step of our project that aims to build an information providing the agent for collaborative decision making tasks, finding the timings for the agent to intervene user-user conversation to provide active support by focusing on the user's gaze. In order to realize this, at first, a Wizard-of- Oz (WOZ) experiment was conducted for collecting human interaction data. By analyzing the collected corpus, eight kinds of timings which allow the agent to do intervention potentially were found. Second, a method was developed to automatically identify four of the eight kinds of timings only by using nonverbal cues, gaze direction, body posture, and speech information. Although the performance of the method is moderate (F-measure 0.4), it should be able to be improved by integrating context information in the future.

References

[1]
Michael Argyle and Mark Cook. Gaze and Mutual Gaze. Cambridge University Press, 1976.
[2]
Dan Bohus and Eric Horvitz. Facilitating multiparty dialog with gaze, gesture, and speech. In In International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction, 2010.
[3]
Herbert H. Clark and Edward F. Schaefer. Contributing to discourse. Cognitive Science, Vol. 13, pp. 259--294, 1989.
[4]
Starkey Duncan. Some signals and rules for taking speaking turns in conversations. Journal of Personality and Psychology, Vol. 23, No. 2, pp. 283--292, 1972.
[5]
Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H. Witten. The weka data mining software: An update. ACM SIGKDD Explorations, Vol. 11, No. 1, pp. 11--18, 2009.
[6]
Gudny Ragna Jonsdottir and Kristinn Thorisson. Teaching computers to conduct spoken interviews: Breaking the realtime barrier with learning. In Zsofia Ruttkay, Michael Kipp, Anton Nijholt, Hannes Hogni, and Vilhjalmsson, editors, 9th International Conference on Intelligent Virtual Agents (IVA'09), Vol. 5773/2009 of LNCS, pp. 446--459, Amsterdam, Netherlands, September 2009. Springer Berlin.
[7]
Adam Kendon. some functions of gaze direction in social interaction. Acta Psychologica, Vol. 26, pp. 22--63, 1967.
[8]
Michael Kipp. Spatiotemporal coding in anvil. In Proceedings of the 6th international conference on Language Resources and Evaluation (LREC-08), 2008.
[9]
Stefan Kopp, Lars Gesellensetter, Nicole C. Kramer, and Ipke Wachsmuth. A conversational agent as museum guide - design and evaluation of a real-world application. In Proceedings of the 5th International Conference on Intelligent Virtual Agents (IVA'05), Kos, Greece, 2005.
[10]
Mikio Nakano, Kohji Dohsaka, Noboru Miyazaki, Junichi Hirasawa, Masafumi Tamoto, Masahito Kawamori, Akira Sugiyama, and Takeshi Kawabata. Handling rich turn-taking in spoken dialogue systems. In European Conference on Speech Communication and Technology (EUROSPEECH'99), 1999.
[11]
John Ross Quinlan. C4.5: Programs for Machine Learning. Machine Learning Representation and Reasoning Series. Morgan Kaufmann, 1993.
[12]
Steve Renals, Thomas Hain, and Herve Bourlard. Recognition and understanding of meetings the ami and amida projects. In IEEE Workhshop on Automatic Speech Recognition and Understanding (ASRU'07), 2007.
[13]
Harvey Sacks, Emanuel A. Schegloff, and Gail Jefferson. a simplest systematics for the organization of turn-taking for conversation. language, Vol. 50, No. 4, pp. 696--735, 1974.
[14]
Ramanathan Subramanian, Jacopo Staiano, Kyriaki Kalimeri, Nicu Sebe, and Fabio Pianesi. Putting the pieces together: multimodal analysis of social attention in meetings. In Proceedings of the international conference on Multimedia, pp. 659--662. ACM, 2010.
[15]
Yoshinao Takemae, Kazuhiro Otsuka, and Naoki Mukawa. Video cut editing rule based on participants' gaze in multiparty conversation. In 11th ACM International Conference on Multimedia, 2003.
[16]
David Traum. Issues in multiparty dialogues. In Advances in Agent Communication, International Workshop on Agent Communication Languages (ACL'03), pp. 201--211, 2003.
[17]
David Traum, Priti Aggarwal, Ron Artstein, Susan Foutz, Jillian Gerten, Athanasios Katsamanis, Anton Leuski, Dan Noren, and William Swartout. Ada and grace: Direct interaction with museum visitors. In 12th International Conference on Intelligent Virtual Agents (IVA 2012), pp. 245--251, 2012.
[18]
A. Waibel, M. Bett, M. Finke, and R. Stiefelhagen. Meeting browser: Tracking and summarizing meetings. In DARPA Broadcast News Transcription and Understanding Workshop, pp. 281--286, 1998.
[19]
Alex Waibel, Michael Bett, Florian Metze, Klaus Ries, Thomas Schaaf, Tanja Schultz, Hagen Soltau, Hua Yu, and Klaus Zechner. Advances in automatic meeting record creation and access. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing 2001 (ICASSP'01), Seattle, USA, May 2001.

Cited By

View all
  • (2022)UX Research on Conversational Human-AI Interaction: A Literature Review of the ACM Digital LibraryProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501855(1-24)Online publication date: 29-Apr-2022

Index Terms

  1. Finding the timings for a guide agent to interveneinter-user conversation in considering their gazebehaviors

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    GazeIn '13: Proceedings of the 6th workshop on Eye gaze in intelligent human machine interaction: gaze in multimodal interaction
    December 2013
    68 pages
    ISBN:9781450325639
    DOI:10.1145/2535948
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 13 December 2013

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. gaze
    2. human-agent interaction
    3. multi-party conversation

    Qualifiers

    • Research-article

    Conference

    ICMI '13
    Sponsor:

    Acceptance Rates

    GazeIn '13 Paper Acceptance Rate 11 of 13 submissions, 85%;
    Overall Acceptance Rate 19 of 21 submissions, 90%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)11
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 14 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2022)UX Research on Conversational Human-AI Interaction: A Literature Review of the ACM Digital LibraryProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501855(1-24)Online publication date: 29-Apr-2022

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media