Ron Artstein
Paul Debevec
ABSTRACT
Time-offset interaction is a new technology that allows for two-way communication with a person who is not available for conversation in real time: a large set of statements are prepared in advance, and users access these statements through natural conversation that mimics face-to-face interaction. Conversational reactions to user questions are retrieved through a statistical classifier, using technology that is similar to previous interactive systems with synthetic characters; however, all of the retrieved utterances are genuine statements by a real person. Recordings of answers, listening and idle behaviors, and blending techniques are used to create a persistent visual image of the person throughout the interaction. A proof-of-concept has been implemented using the likeness of Pinchas Gutter, a Holocaust survivor, enabling short conversations about his family, his religious views, and resistance. This proof-of-concept has been shown to dozens of people, from school children to Holocaust scholars, with many commenting on the impact of the experience and potential for this kind of interface.
- Chabert, C.-F., Einarsson, P., Jones, A., Lamond, B., Ma, A., Sylwan, S., Hawkins, T., and Debevec, P. Relighting human locomotion with fiowed refiectance fields. In SIGGRAPH 06: ACM SIGGRAPH 2006 Sketches, ACM (2006), 76. Google Scholar
Digital Library
- Clarkson, P., and Rosenfeld, R. Statistical language modeling using the Carnegie Mellon University-Cambridge toolkit. In Proc. of Eurospeech (Rhodes, Greece, 1997).Google Scholar
- Ghosh, A., Fyffe, G., Tunwattanapong, B., Busch, J., Yu, X., and Debevec, P. Multiview face capture using polarized spherical gradient illumination. In Proceedings of the 2011 SIGGRAPH Asia Conference, SA '11, ACM (New York, NY, USA, 2011), 129:1--129:10. Google ScholarDigital Library
- Gratch, J., Rickel, J., Andre, E., Cassell, J., Petajan, E., and Badler, N. Creating interactive virtual humans: Some assembly required. IEEE Intelligent Systems (2002), 54--63. Google ScholarDigital Library
- Hartholt, A., Traum, D., Marsella, S. C., Shapiro, A., Stratou, G., Leuski, A., Morency, L.-P., and Gratch, J. All together now: Introducing the virtual human toolkit. In International Conference on Intelligent Virtual Humans (Edinburgh, UK, Aug. 2013).Google ScholarCross Ref
- Leuski, A., and Traum, D. Practical language processing for virtual humans. In Proceedings of the 22nd Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-10) (2010).Google Scholar
- Maio, H., Traum, D., and Debevec, P. New dimensions in testimony. PastForward, Summer (2012), 22--26.Google Scholar
- Morbini, F., Audhkhasi, K., Sagae, K., Artstein, R., Can, D., Georgiou, P., Narayanan, S., Leuski, A., and Traum, D. Which ASR should I choose for my dialogue systemfi In Proceedings of the SIGDIAL 2013 Conference (Metz, France, August 2013), 394--403.Google Scholar
- Robinson, S., Traum, D., Ittycheriah, M., and Henderer, J. What would you ask a conversational agentfi Observations of human-agent dialogues in a museum setting. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC) (Marrakech, Morocco, 2008).Google Scholar
- Traum, D., Aggarwal, P., Artstein, R., Foutz, S., Gerten, J., Katsamanis, A., Leuski, A., Noren, D., and Swartout, W. Ada and Grace: Direct interaction with museum visitors. In Intelligent Virtual Agents: 12th International Conference, IVA 2012, Santa Cruz, CA, USA, September 12fi14, 2012 Proceedings, Y. Nakano, M. Neff, A. Paiva, and M. Walker, Eds., vol. 7502 of Lecture Notes in Artificial Intelligence, Springer (Heidelberg, September 2012), 245--251. Google ScholarDigital Library
- Vertanen, K. Baseline WSJ acoustic models for HTK and Sphinx: Training recipes and recognition experiments. Tech. rep., Cavendish Laboratory, University of Cambridge, 2006.Google Scholar
- Weide, R. The Carnegie Mellon University pronouncing dictionary, 2008.Google Scholar
Index Terms
- Time-offset interaction with a holocaust survivor
Recommendations
Digital survivor of sexual assault
IUI '19: Proceedings of the 24th International Conference on Intelligent User InterfacesThe Digital Survivor of Sexual Assault (DS2A) is an interface that allows a user to have a conversational experience with a survivor of sexual assault, using Artificial Intelligence technology and recorded videos. The application uses a statistical ...
Wizard of Oz experiments and companion dialogues
BCS '10: Proceedings of the 24th BCS Interaction Specialist Group ConferenceNovel speech systems such as the conversational agents being developed by the Companions Project (www.companions-project.org) can be simulated using the Wizard of Oz methodology. In this approach technologies that are not yet ready for testing by people ...
From vocal to multimodal dialogue management
ICMI '06: Proceedings of the 8th international conference on Multimodal interfacesMultimodal, speech-enabled systems pose different research problems when compared to unimodal, voice-only dialogue systems. One of the important issues is the question of how a multimodal interface should look like in order to make the multimodal ...
Comments