Jean-Yves Lionel Lawson
Jean Vanderdonckt
ABSTRACT
In this paper we present an extensible software workbench for supporting the effective and dynamic prototyping of multimodal interactive systems. We hypothesize the construction of such applications to be based on the assembly of several components, namely various and sometimes interchangeable modalities at the input, fusion-fission components, and also several modalities at the output. Successful realization of advanced interactions can benefit from early prototyping and the iterative implementation of design requires the easy integration, combination, replacement, or upgrade of components. We have designed and implemented a thin integration platform able to manage these key elements, and thus provide the research community a tool to bridge the gap of the current support for multimodal applications implementation. The platform is included within a workbench offering visual editors, non-intrusive tools, components and techniques to assemble various modalities provided in different implementation technologies, while keeping a high level of performance of the integrated system.
- Benoit, A., Bonnaud, L., Caplier, A., Damousis, I., Tzovaras, D., Jourde, E., Nigay, L., Serrano M. and Lawson, J-Y. 2006. Multimodal Signal Processing and Interaction for a Driving Simulation: Component-based Architecture. Journal on Multimodal User Interfaces,1, 1,49--58.Google Scholar
Cross Ref
- Bouchet, J., Nigay, L. (2004). ICARE: A Component-Based Approach for the Design and Development of Multimodal Interfaces, Extended Abstracts CHI'04, ACM, pp. 1325--1328. Google ScholarDigital Library
- Camurri, A., Ricchetti, M., and Trocca, R. 1999. EyesWeb -- toward gesture and affect recognition in dance/music interactive systems, Proc. IEEE Multimedia Systems '99, Firenze, Italy, June 1999. Google ScholarDigital Library
- Cohen, P. R., Johnston, M., McGee, D., Oviatt, S., Pittman, J., Smith, I., Chen, L., and Clow, J. 1997. QuickSet: multimodal interaction for distributed applications, Proc. of MULTIMEDIA '97. ACM Press, pp. 31--40. Google ScholarDigital Library
- Coutaz, J. Nigay, L. Salber, D. Blandford, A. May, J. and Young, R. (1995), Four Easy Pieces for Assessing the Usability of Multimodal in Interaction the CARE Properties, Proc.of Interact' 95, pp. 115--120.Google Scholar
- Dragicevic P. and Fekete J-D. Input Device Selection and Interaction Configuration with ICON: Joint proc. of IHM'01-HCI'01. Springer Verlag, pp 543--558.Google Scholar
- eNTERFACE Workshops, http://www.enterface.net/Google Scholar
- Gray, P., Ramsay, A., Serrano, M. A demonstration of the OpenInterface Interaction Development Environment. UIST'07 Adj. Proc.Google Scholar
- H. Sakoe, S. Chiba, Dynamic programming algorithm optimization for spoken word recognition. In IEEE Transactions on Acoustics, Speech, and Signal Processing 26 (1) (1978) 43--49.Google Scholar
- Hartmann, B., Abdulla, L., Mittal, M., and Klemmer, S. R. 2007. Authoring sensor-based interactions by demonstration with direct manipulation and pattern recognition. Proc. of CHI '07. ACM Press, 145--154. Google ScholarDigital Library
- Hartmann, B., Klemmer, S. R., Bernstein, M., Abdulla, L., Burr, B., Robinson-Mosher, A., and Gee, J. 2006. Reflective physical prototyping through integrated design, test, and analysis. Proc. of UIST '06. Google ScholarDigital Library
- Kato, H., Billinghurst, M. (1999) Marker Tracking and HMD Calibration for a video-based Augmented Reality Conferencing System. Proc. of IWAR'99. Google ScholarDigital Library
- Lawson, J.-Y., Vanderdonckt, J., Macq, B. Rapid Prototyping of Multimodal Interactive Applications Based on Off-The-Shelf Heterogeneous Components. UIST'08 Adjunct. Proc.Google Scholar
- Lawson, J.-Y., 2006. OpenInterface Description Languages Specification. Technical report, 2006: http://www.openinterface.org/platform/documentationGoogle Scholar
- Li, Y. and Landay, J. A. 2005. Informal prototyping of continuous graphical interactions by demonstration, Proc. of UIST '05. ACM Press, pp. 221--230. Google ScholarDigital Library
- Martin, J. C., TYCOON: Theoretical Framework and Software Tools for Multimodal Interfaces, Intelligence and Multimodality in Multimedia Interfaces, AAAI Press, 1997.Google Scholar
- Max/MSP, http://www.cycling74.comGoogle Scholar
- Maynes-Aminzade, D., Winograd, T., and Igarashi, T. (2007). Eyepatch: prototyping camera-based interaction through Examples, Proc. of UIST'07, ACM Press, pp. 33--42. Google ScholarDigital Library
- Mehta, N. R., Medvidovic, N., and Phadke, S. 2000. Towards a taxonomy of software connectors. Proc. of ICSE '00. ACM Press, pp. 178--187. Google ScholarDigital Library
- Merrill, D. and Paradiso, J. A Personalization, Expressivity, and Learnability of an Implicit Mapping Strategy for Physical Interfaces, Proc. of CHI'05 ACM Press, pp. 2152--2161.Google Scholar
- Puckette, M., 1996. "Pure Data: another integrated computer music environment.", Proc. of the Second Intercollege Computer Music Concerts, Tachikawa, pp. 37--41Google Scholar
- Schedlbauer, M. J. An extensible platform for the interactive exploration of Fitts' Law and related movement time models. In CHI '07 Extended Abstracts on Human Factors in Computing Systems. ACM Press, pp. 2633--2638. Google Scholar
- Shaw, M. and Garlan, D. Software Architecture: Perspectives on an Emerging Discipline. Prentice-Hall, 1996. Google ScholarDigital Library
- Sinha, A. K. and Landay, J. A. 2003. Capturing user tests in a multimodal, multidevice informal prototyping tool. Proc. of ICMI '03, ACM Press, pp. 117--124. Google ScholarDigital Library
- Trevisan, Daniela G.; Nicolas, Vincent; Macq, Benoit; NEDEL, Luciana P. MedicalStudio: a medical component-based framework, Proc of.WIM '07.Google Scholar
- Vybornova, O., Mendonça, H., Lawson, J.Y., Macq, B. High Level Data Fusion on a Multimodal Interactive Applications Platform. Proc of ISM'08. Google ScholarDigital Library
- Web Services Description Language (WSDL), http://www.w3.org/TR/wsdl.Google Scholar
Index Terms
- An open source workbench for prototyping multimodal interactions based on off-the-shelf heterogeneous components
Recommendations
Component-based high fidelity interactive prototyping of post-WIMP interactions
ICMI-MLMI '10: International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal InteractionIn order to support interactive high-fidelity prototyping of post-WIMP user interactions, we propose a multi-fidelity design method based on a unifying component-based model and supported by an advanced tool suite, the OpenInterface Platform Workbench. ...
Multimodal slideshow: demonstration of the openinterface interaction development environment
ICMI '08: Proceedings of the 10th international conference on Multimodal interfacesIn this paper, we illustrate the OpenInterface Interaction Development Environment (OIDE) that addresses the design and development of multimodal interfaces. Multimodal interaction software development presents a particular challenge because of the ever ...
Gesture and voice prototyping for early evaluations of social acceptability in multimodal interfaces
ICMI-MLMI '10: International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal InteractionInteraction techniques that require users to adopt new behaviors mean that designers must take into account social acceptability and user experience otherwise the techniques may be rejected by users as they are too embarrassing to do in public. This ...
Comments