Fredy Cuenca
Davy Vanacken
Kris Luyten
ABSTRACT
Choosing an appropriate toolkit for creating a multimodal interface is a cumbersome task. Several specialized toolkits include fusion and fission engines that allow developers to combine and decompose modalities to capture multimodal input and provide multimodal output. Unfortunately, the extent to which these toolkits can facilitate the creation of a multimodal interface is hard or impossible to estimate, due to the absence of a scale where the toolkit's capabilities can be measured on. In this paper, we propose a measurement scale, which allows the assessment of specialized toolkits without need for time-consuming testing or source code analysis. This scale is used to measure and compare the capabilities of three toolkits: CoGenIVE, HephaisTK and ICon.
- Bolt, R. Put-that-there: Voice and gesture at the graphics interface. In SIGGRAPH'80 Proc. of the 7th annual conference on computer graphics and interactive techniques, ACM (1980). Google Scholar
Digital Library
- Coutaz, J., Nigay, L., Salber, D., Blandford, A., May, J., and R., Y. Four easy pieces for assessing the usability of multimodal interaction: The care properties. In Proc. of INTERACT'95 (1995).Google ScholarCross Ref
- De Boeck, J., Raymaekers, C., and Coninx, K. Comparing nimmit and data-driven notations for describing multimodal interaction. In TAMODIA' 06 Proc. of the fifth International Conference on Task Models and Diagrams for User Interaction Design, Springer Verlag (2007). Google ScholarDigital Library
- De Boeck, J., Vanacken, D., Raymaekers, C., and Coninx, K. High level modeling of multimodal interaction techniques using NiMMiT. Journal of Virtual Reality and Broadcasting 4, 2 (2007).Google Scholar
- Dragicevic, P., and Fekete, J. Support for input adaptability in the icon toolkit. In ICMI'04 Proc. of the 6th International Conference on Multimodal Interfaces, ACM (2004). Google ScholarDigital Library
- Dumas, B. Frameworks, Description Languages and Fusion Engines for Multimodal Interactive Systems. PhD thesis, University of Fribourg, 2010.Google Scholar
- Dumas, B., Lalanne, D., and Oviatt, S. Multimodal interfaces: A survey of principles, models and frameworks. In Human Machine Interaction, Springer Verlag (2009). Google ScholarDigital Library
- Dumas, B., Signer, B., and Lalanne, D. A graphical uidl editor for multimodal interaction design based on smuiml. In Proc. of the Workshop on Software Support for User Interface Description Language, WISE publication (2011).Google Scholar
- Navarre, D., Palanque, P., Ladry, J., and Barboni, E. ICOs: A Model-Based User Interface Description Technique dedicated to Interactive Systems Addressing Usability, Reliability and Scalability. ACM Transactions on Computer-Human Interaction 16, 4 (2009). Google ScholarDigital Library
- Nigay, L., and Coutaz, J. A design space for multimodal systems: Concurrent processing and data fusion. In Proc. of INTERACT'93, ACM (1993). Google ScholarDigital Library
- Vanacken, D. Touch-based interaction and collaboration in walk-up-and-use and multi-user environments. PhD thesis, Universiteit Hasselt, 2012.Google Scholar
- Wahlster, W., Reithinger, N., and Blocher, A. Smartkom: Multimodal communication with a life-like character. In Proc. of the 7th European Conference on Speech Communication and Technology, DKFI (2001).Google Scholar
- Werner, K., Raedle, R., and Harald, R. Interactive Design of Multimodal User Interfaces - Reducing technical and visual complexity. Journal on Multimodal User Interfaces 3, 3 (2010).Google Scholar
Index Terms
- Assessing the support provided by a toolkit for rapid prototyping of multimodal systems
Recommendations
SwingStates: adding state machines to Java and the Swing toolkit
This article describes SwingStates, a Java toolkit designed to facilitate the development of graphical user interfaces and bring advanced interaction techniques to the Java platform. SwingStates is based on the use of finite-state machines specified ...
The CoGenIVE concept revisited: a toolkit for prototyping multimodal systems
EICS '13: Proceedings of the 5th ACM SIGCHI symposium on Engineering interactive computing systemsMany specialized toolkits have been developed with the purpose of facilitating the creation of multimodal systems. They allow their users to specify certain tasks of their intended systems by means of a visual language instead of programming code. One ...
From vocal to multimodal dialogue management
ICMI '06: Proceedings of the 8th international conference on Multimodal interfacesMultimodal, speech-enabled systems pose different research problems when compared to unimodal, voice-only dialogue systems. One of the important issues is the question of how a multimodal interface should look like in order to make the multimodal ...
Comments