skip to main content
10.1145/1822018.1822035acmconferencesArticle/Chapter ViewAbstractPublication PageseicsConference Proceedingsconference-collections
research-article

WebWOZ: a wizard of oz prototyping framework

Published: 19 June 2010 Publication History

Abstract

Language Technology (LT) based applications become more popular as technology improves. Prototyping early in the design process is critical for the development of high quality applications. It is difficult, however, to do low-fidelity prototyping (e.g. paper prototyping) of applications based on LT. One technique that has been used for this kind of prototyping is Wizard of Oz (WOZ). However, this generally involves the development of one-off user and wizard interfaces. A tool that facilitates the flexible integration of LT components into WOZ experiments is desirable. In this paper we explore the requirements for such a tool, drawing from the literature and a first WOZ experiment in which different wizards were observed and their behaviour was analysed.

References

[1]
D. Bohus, A. Raux, T. K. Harris, M. Eskenazi, and A. I. Rudnicky. Olympus: an open-source framework for conversational spoken language interface research. In NAACL-HLT: Proceedings of the Workshop on Bridging the Gap: Academic and Industrial Research in Dialog Technology, pages 32--39, 2007.
[2]
J. Bradley, O. Mival, and D. Benyon. Wizard of oz experiments for companions. In Proceedings of the British Computer Society Conference on HCI, pages 313--317, 2009.
[3]
B. Buxton. Sketching user experiences. Morgan Kaufman, 2007.
[4]
C. L. Chen and T. V. Raman. AxsJAX: a talking translation bot using google im: bringing web-2.0 applications to life. In Proceedings of W4A, pages 54--56, 2008.
[5]
D. Coyle, G. Doherty, M. Matthews, and J. Sharry. Computers in talk-based mental health interventions. Interacting with Computers, 19:545--562, 2007.
[6]
N. Dahlback, A. Jonsson, and L. Ahrenberg. Wizard of Oz studies: why and how. In Proceedings of IUI, pages 193--200, 1993.
[7]
R. L. Erdmann and A. S. Nea. Laboratory vs. field experimentation in human factors an evaluation of an experimental self-service airline ticket vendor. Human Factors, 13:521--531, 1971.
[8]
C. Fugen, M. Kolss, D. Bernreuther, M. Paulik, S. Stucker, S. Vogel, and A. Waibel. Open domain speech recognition and translation: lecture notes & speeches. In Proceedings of ICASSP, 2006.
[9]
P. Geutner, F. Steffens, and D. Manstetten. Design of the VICO spoken dialogue system: Evaluation of user expectations by wizard-of-oz experiments. In Proceedings of LREC, 2002.
[10]
J. D. Gould and C. Lewis. Designing for usability: key principles and what designers think. Communications of ACM, 28(3):300--311, 1985.
[11]
A. C. Graesser, K. VanLehn, C. P. Rose, P. W. Jordan, and D. Harter. Intelligent tutoring systems with conversational dialogue. AI Magazine, 22(4):39--51, 2001.
[12]
D. Jurafsky and J. H. Martin. Speech and Language Processing. Prentice Hall, second edition, 2008.
[13]
A. Karpov, A. Ronzhin, and A. Leontyeva. A semi-automatic wizard of oz technique for Let's Fly spoken dialogue system. In Lecture Notes in Computer Science: Text, Speech and Dialogue, pages 585--592. Springer, 2008.
[14]
C. Keskin, K. Balci, O. Aran, B. Sankur, and L. Akarun. A multimodal 3D healthcare communication system. In 3D Conference, 2007.
[15]
D. G. Kidd, D. M. Cades, D. J. Horvath, S. M. Jones, M. J. Pitone, and C. A. Monk. Listen up! Do voice recognition systems help drivers focus on the road? User Experience, 7(4):10--12, 2008.
[16]
S. R. Klemmer, A. K. Sinha, J. Chen, J. A. Landay, N. Aboobaker, and A. Wang. SUEDE: a wizard of oz prototyping tool for speech user interfaces. In Proceedings of UIST, pages 1--10, 2000.
[17]
L. Lamel, S. Rosset, J. L. Gauvain, and S. Bennacef. The LIMSI arise system for train travel information. In Proceedings of ICASSP, pages 501--504, 1999.
[18]
D. J. Litman and S. Pan. Predicting and adapting to poor speech recognition in a spoken dialogue system. In Proceedings of AAAI, pages 100--111, 2000.
[19]
Y. Liu and P. Fung. Multi-accent chinese speech recognition. In Proceedings of INTERSPEECH, pages 133--136, 2006.
[20]
K. Makela, E.-P. Salonen, M. Turunen, J. Hakulinen, and R. Raisamo. Conducting a Wizard of Oz experiment on a ubiquitous computing system doorman. In Proceedings of the International Workshop on Information Presentation and Natural Multimodal Dialogue, pages 115--119, 2001.
[21]
M. Paul. Overview of the IWSLT 2009 Evaluation Campaign. In Proceedings of IWSLT, 2009.
[22]
M. Rajman, M. Ailomaa, A. Lisowska, M. Melchiar, and S. Armstron. Extending the wizard of oz methodology for language-enabled multimodal systems. In Proceedings of LREC, pages 2531--2536, 2006.
[23]
A. Raux, D. Bohus, B. Langner, A. W. Black, and M. Eskenazi. Doing research on a deployed spoken dialogue system: One year of Lets Go! experience. In Proceedings of INTERSPEECH, pages 65--68, 2006.
[24]
D. Salber and J. Coutaz. A wizard of oz platform for the study of multimodal systems. In Proceedings of CHI, pages 95--96, 1993.
[25]
H. Somers. Language Engineering and the Pathway to Healthcare: A user-oriented view. In NAACL Workshop on Medical Speech Translation, pages 32--39, 2006.
[26]
S. Stucker, C. Fugen, F. Kraft, and M. Wolfel. The ISL 2007 english speech transcription system for european parliament speeches. In Proceedings of INTERSPEECH, 2007.
[27]
S. Sutton, R. Cole, J. de Vielliers, J. Schalkwyk, P. Vermeulen, M. Macon, Y. Yan, E. Kaiser, B. Rundle, K. Shobaki, P. Hosom, A. Kain, J. Wouters, D. Massaro, and M. Cohen. Universal speech tools: the CSLU toolkit, 1998.
[28]
M. Turunen and J. Hakulinen. Jaspis -- a framework for multilingual adaptive speech applications. In Proceedings of ICSLP, pages 719--722, 2000.
[29]
M. Turunen, J. Hakulinen, E.--P. Salonen, A. Kainulainen, and L. Helin. Spoken and multimodal bus timetable systems: design, development and evaluation. In Proceedings of SPECOM, pages 389--392, 2005.
[30]
W. Wahlster. Verbmobil: foundations of speech-to-speech translation. Springer, 2000.
[31]
M. A. Walker, A. Rudnicky, R. Prasad, J. Aberdeen, E. O. Bratt, J. Garofolo, H. Hastie, A. Le, B. Pellom, A. Potamianos, R. Passoneau, S. Roukos, G. Sanders, S. Seneff, and D. Stallard. DARPA communicator: cross-system results for the 2001 evaluation. In Proceedings of ICSLP, pages 269--272, 2002.

Cited By

View all
  • (2024)CounterSludge in Alcohol Purchasing on Online Grocery Shopping PlatformsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661595(1741-1759)Online publication date: 1-Jul-2024
  • (2023)Wizundry: A Cooperative Wizard of Oz Platform for Simulating Future Speech-based Interfaces with Multiple WizardsProceedings of the ACM on Human-Computer Interaction10.1145/35795917:CSCW1(1-34)Online publication date: 16-Apr-2023
  • (2022)A Multilingual Neural Coaching Model with Enhanced Long-term Dialogue StructureACM Transactions on Interactive Intelligent Systems10.1145/348706612:2(1-47)Online publication date: 12-Jul-2022
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
EICS '10: Proceedings of the 2nd ACM SIGCHI symposium on Engineering interactive computing systems
June 2010
390 pages
ISBN:9781450300834
DOI:10.1145/1822018
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 June 2010

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. language technology
  2. prototyping
  3. wizard of oz

Qualifiers

  • Research-article

Conference

EICS '10
Sponsor:

Acceptance Rates

Overall Acceptance Rate 73 of 299 submissions, 24%

Upcoming Conference

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)34
  • Downloads (Last 6 weeks)3
Reflects downloads up to 27 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2024)CounterSludge in Alcohol Purchasing on Online Grocery Shopping PlatformsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661595(1741-1759)Online publication date: 1-Jul-2024
  • (2023)Wizundry: A Cooperative Wizard of Oz Platform for Simulating Future Speech-based Interfaces with Multiple WizardsProceedings of the ACM on Human-Computer Interaction10.1145/35795917:CSCW1(1-34)Online publication date: 16-Apr-2023
  • (2022)A Multilingual Neural Coaching Model with Enhanced Long-term Dialogue StructureACM Transactions on Interactive Intelligent Systems10.1145/348706612:2(1-47)Online publication date: 12-Jul-2022
  • (2022)Wizards in the Middle: An Approach to Comparing Humans and Robots*2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)10.1109/RO-MAN53752.2022.9900655(329-336)Online publication date: 29-Aug-2022
  • (2020)Analysis of the interaction between elderly people and a simulated virtual coachJournal of Ambient Intelligence and Humanized Computing10.1007/s12652-020-01983-3Online publication date: 22-May-2020
  • (2020)A Spanish Corpus for Talking to the ElderlyConversational Dialogue Systems for the Next Decade10.1007/978-981-15-8395-7_13(183-192)Online publication date: 25-Oct-2020
  • (2017)Wizard’s Apprentice: Cognitive Suggestion Support for Wizard-of-Oz Question AnsweringArtificial Intelligence in Education10.1007/978-3-319-61425-0_79(630-635)Online publication date: 23-Jun-2017
  • (2015)ConWIZ: The contextual Wizard of OzJournal of Ambient Intelligence and Smart Environments10.3233/AIS-1503507:6(719-744)Online publication date: 20-Nov-2015
  • (2015)WozARdAdvances in Human-Computer Interaction10.1155/2015/2712312015(3-3)Online publication date: 1-Jan-2015
  • (2014)Wizard of Oz Experimentation for Language Technology Applications: Challenges and ToolsInteracting with Computers10.1093/iwc/iwu01627:6(592-615)Online publication date: 9-May-2014
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media