John Sören Pettersson
ABSTRACT
The Wizard-of-Oz method has been around for decades, allowing researchers and practitioners to conduct prototyping without programming. An extensive literature review conducted by the authors revealed, however, that the re-usable tools supporting the method did not seem to last more than a few years. While generic systems start to appear around the turn of the millennium, most have already fallen out of use.
Our interest in undertaking this review was inspired by the ongoing re-development of our own Wizard-of-Oz tool, the Ozlab, into a system based on web technology.
We found three factors that arguably explain why Ozlab is still in use after 15 years instead of the two-three years lifetime of other generic systems: the general approach used from its inception; its inclusion in introductory HCI curricula, and the flexible and situation-dependent design of the wizard's user interface.
- Alce, G., Wallergård, M. and Hermodsson, K. WozARd: A Wizard of Oz Method for Wearable Augmented Reality Interaction---A Pilot Study. Advances in Human-Computer Interaction, vol. 2015, article ID 271231. Google Scholar
Digital Library
- Baum, L. F. The Wonderful Wizard of Oz. With illustrations by W. W. Denslow. Georg M. Hill Company, Chicago (1900).Google Scholar
- Bellucci, A., Bottoni, P. and Levialdi, S. WOEB: Rapid Setting of Wizard of Oz Experiments and Reuse for Deployed Applications. Proceedings of the IUI'09 Workshop on Model Driven Development of Advanced User Interfaces, MDDAUI'09. CEUR Workshop Proceedings (2009), Vol 439.Google Scholar
- Beaudouin-Lafon, M. and Mackay, W. Prototyping tools and techniques. In The Human-Computer Interaction Handbook. Third Edition, J. A. Jacko, Ed. CRC Press (2012), 1081--1104.Google Scholar
- Carter, A. S. and Hundhausen, C. D. How is user interface prototyping really done in practice? A survey of user interface designers. In Proc. 2010 IEEE Symposium on Visual Languages and Human-Centric Computing, 207--211. IEEE Computer Society (2010). Google ScholarDigital Library
- Coutaz, J., Nigay, L. and Salber, D. Multimodality from the User and System Perspectives. ERCIM'95 Workshop on Multimedia Multimodal User Interfaces (1995).Google Scholar
- Grill, T., Polacek, O., and Tscheligi, M. ConWIZ: A tool supporting contextual Wizard of Oz simulation. Proc. 11th Int. Conf. on Mobile and Ubiquitous Multimedia, MUM '12. Ulm, Germany (2012), 21--28. Google ScholarDigital Library
- Hundhausen, C. D., Balkar, A., Nuur, M. and Trent, S. WOZ pro: a pen-based low fidelity prototyping environment to support Wizard of Oz studies. In Proc. CHI 2007. ACM Press (207), 2453--2458. Google ScholarDigital Library
- Kelley, J. F. An empirical methodology for writing user-friendly natural language computer applications. In Proc. CHI 1983, ACM Press (1983), 193--196. Google ScholarDigital Library
- Li, X. Improving the reliability and validity of Wizard-of-Oz methods. PhD Thesis. School of Computing and Engineering, University of Huddersfield (2012), eprints.hud.ac.ukGoogle Scholar
- Molin, L. Wizard-of-Oz prototyping for cooperative interaction design of graphical user interfaces. In Proc. NordiCHI 2004, ACM (2004), 425--428. Google ScholarDigital Library
- Pettersson, J. S. Simulating interactive graphic user interfaces. In Proc. IT-lyftet 2001. Blekinge Tekniska Högskola (2001), 55--61.Google Scholar
- Pettersson, J. S. Visualising interactive graphics design for testing with users. Digital Creativity, 13, 3 (2002), 144--156.Google ScholarCross Ref
- Pettersson, J. S., and Wik, M. Perspectives on Ozlab in the cloud. A literature review of tools supporting Wizard-of-Oz experimentation. Karlstad University, Karlstad, Working paper, urn:nbn:se:kau:diva-33617 (2014).Google Scholar
- Schlögl, S., Doherty, G., and Luz, S. Wizard of Oz experimentation for language technology applications: Challenges and tools. Interacting with Computers, 27 (2015), 592--615.Google ScholarCross Ref
- Venters, C. C., Lau, L., Griffiths, M. K., Holmes, V., Ward, R. R., Jay, C., Dibsdale, and C. E., Xu, J. The blind men and the elephant: Towards an empirical evaluation framework for software sustainability. J. of Open Research Software 2, 1 (2014), e8 1--6.Google ScholarCross Ref
Index Terms
- The longevity of general purpose Wizard-of-Oz tools
Recommendations
Wizard of Oz studies in HRI: a systematic review and new reporting guidelines
Inaugural Special Issue: Intersection of Systems Sciences and Human SciencesMany researchers use Wizard of Oz (WoZ) as an experimental technique, but there are methodological concerns over its use, and no comprehensive criteria on how to best employ it. We systematically review 54 WoZ experiments published in the primary HRI ...
The oz of wizard: simulating the human for interaction research
HRI '09: Proceedings of the 4th ACM/IEEE international conference on Human robot interactionThe Wizard of Oz experiment method has a long tradition of acceptance and use within the field of human-robot interaction. The community has traditionally downplayed the importance of interaction evaluations run with the inverse model: the human ...
Wizard of Oz experiments and companion dialogues
BCS '10: Proceedings of the 24th BCS Interaction Specialist Group ConferenceNovel speech systems such as the conversational agents being developed by the Companions Project (www.companions-project.org) can be simulated using the Wizard of Oz methodology. In this approach technologies that are not yet ready for testing by people ...
Comments