Skip to main content

Advertisement

Springer Nature Link
Log in

Assessment of a multimodal interaction and rendering system against established design principles

  • Original Paper
  • Published:
Journal on Multimodal User Interfaces Aims and scope Submit manuscript

Abstract

To provide user interfaces for a rich set of devices and interaction modalities, we follow a model-based development methodology. We devised an architecture which deploys user interfaces specified as dialogue models with abstract interaction objects and allows context-based adaptations by means of an external transcoding process. For the validation of the applicability of this methodology for developing usable multimodal multi-device systems, we present two case studies based on proof-of-concept implementations and assessed them with a large set of established design principles and different types of modality cooperation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

References

  1. Ducatel K, Bogdanow M, Scapolo F, Laijten J, Burgelman J-C (2001) Scenarios for ambient intelligence in 2010. IST Advisory Group

  2. Kim SH, Kim SW, Park HM (2003) Usability challenges in ubicomp environment. In: Proceeding of IEA (International Ergonomics Association)

  3. Gram C, Cockton G (1996) Design principles for interactive software. Chapman & Hall, Menlo Park

    MATH  Google Scholar 

  4. Abrams M, Phanouriou C, Batongbacal AL, Williams SM, Shuster J (1999) Uiml: an appliance-independent xml user interface language. Comput Netw 31:1

    Article  Google Scholar 

  5. Schaefer R, Bleul S, Mueller W (2006) Dialog modelling for multiple devices and multiple interaction modalities. In: Proceedings of the 5th international workshop on task models and diagrams for UI design (TAMODIA’2006)

  6. Bleul S, Schaefer R, Mueller W (2004) Multimodal dialog description for mobile devices. In: Luyten K, Abrams M, Vanderdonckt J, Limbourg Q (eds) Developing user interfaces with XML: advances on user interface description languages, satellite workshop of advanced visual interfaces (AVI)

  7. Green M (1986) A survey of three dialog models. ACM Trans Graph 5(3):244–275

    Article  Google Scholar 

  8. Ali MF, Abrams M (2001) Simplifying construction of multi-platform user interfaces using uiml. In: Proceedings of the user interface markup language conference

  9. Sandor C, Reicher T (2001) Cuiml: a language for generating multimodal human-computer interfaces. In: Proceedings of the European UIML conference

  10. Boyer JM, Landwehr D, Merrick R, Raman TV, Dubinko M, Klotz LL (2006) XForms 1.0 second edition, W3C recommendation. World Wide Web Consortium, March 2006

  11. OMA (2001) Wireless markup language version 2.0, wireless application protocol, wap-238-wml-20010911-a

  12. Palanque P, Bastide R (1994) Petri net based design of user-driven interfaces using interactive cooperative object formalism. In: Proceedings of the 1st Eurographics workshop on design, specification and verification of interactive systems, DSVIS

  13. d’Ausbourg B, Durrieu G, Roché P (1996) Deriving a formal model of an interactive system from its uil description in order to verify and test its behaviour. In: Design, specification and verification of interactive systems’96, proceedings of the third international Eurographics workshop, pp 105–122

  14. Janssen C (1993) Dialogue nets for the description of dialogue flows in graphical interactive systems. In: Proceedings of Software-Ergonomie ’93. Teubner, Stuttgart

    Google Scholar 

  15. Curry MB, Monk AF (1995) Dialogue modelling of graphical user interfaces with a production system. Behav Inf Technol 14(1):41–55

    Article  Google Scholar 

  16. Harel D (1987) State charts: a visual formalism for complex systems. Sci Comput Program 8

  17. Ali MF, Pérez-Quiñones MA, Abrams M (2004) Building multi-platform user interfaces with uiml’. In: Seffah A, Javahery H (eds) Multiple user interfaces—cross-platform applications and context-aware interfaces. Wiley, New York, pp 95–118

    Google Scholar 

  18. Vanderdonckt J, Bodart F (1993) Encapsulating knowledge for intelligent automatic interaction objects selection. In: ACM conference on human aspects in computing systems, INTERCHI’93

  19. Plomp J, Mayora-Ibarra O, Yli-Nikkola H (2001) Graphical and speech-driven user interface generation from a single source format. In: Proceedings of the first annual VoiceXML forum user group meeting (AVIOS 2001)

  20. Plomp J, Mayora-Ibarra O (2002) A generic widget vocabulary for the generation of graphical and speech driven user interfaces. Int J Speech Technol 5(1):39–47

    Article  MATH  Google Scholar 

  21. Tarasewich P Campbell CS, Xia T, Dideles M (2003) Evaluation of visual notification cues for ubiquitous computing. In: Dey AK, Schmidt A, McCarthy JF (eds) Proceedings of UbiComp 2003: ubiquitous computing, 5th international conference. Lecture notes in computer science, vol 2864. Springer, Berlin, pp 349–366

    Google Scholar 

  22. Luk J, Pasquero J, Little S, MacLean K, Lévesque V, Hayward V (2006) A role for haptics in mobile interaction: Initial design using a handheld tactile display prototype. In: Proceedings of Conference on Human Factors in Computing Systems (CHI 2006). ACM, New York, pp 171–180

    Chapter  Google Scholar 

  23. Brewster SA, McGookin DK, Miller CA (2006) Olfoto: designing a smell-based interaction. In: Proceedings of conference on human factors in computing systems (CHI 2006). ACM, New York, pp 653–662

    Chapter  Google Scholar 

  24. Mäntyjärvi J, Kallio S, Korpipää P, Kela J, Plomp J (2005) Gesture interaction for small handheld devices to support multimedia applications. J Mobile Multimed 1:92–111

    Google Scholar 

  25. Jacob RJK (1991) The use of eye movements in human-computer interaction techniques: what you look at is what you get. ACM Trans Inf Syst 9:152–169

    Article  MathSciNet  Google Scholar 

  26. Blankertz B, Dornhege G, Krauledat M, Müller K-R, Kunzmann V, Losch F, Curio G (2006) The Berlin brain-computer interface: EEG-based communication without subject training. IEEE Trans Neural Syst Rehabil Eng 14:147–152

    Article  Google Scholar 

  27. Schaefer R, Mueller W, López AM, Sánches DD (2007) Using smart cards for secure and device independent user interfaces. In: Proceedings of the mobility conference

  28. Szwillus G (1993) Object oriented dialogue specification with odsn. In: Proceedings of software-ergonomie ’93. Teubner, Stuttgart

    Google Scholar 

  29. Schaefer R, Bleul S (2006) Towards object oriented, uiml-based interface descriptions for mobile devices. In: Calvary G, Pribeanu C, Santucci G, Vanderdonckt J (eds) Computer-aided design of user interfaces V, Proceedings of 6th international conference on computer-aided design of user interfaces CADUI’2006. Information systems series, pp. 24–35. Springer, Berlin

    Google Scholar 

  30. Zabel H, Rettberg A (2006) Prototyping an ambient light system—a case study. In: Proceedings of the 5th IFIP working conference on distributed and parallel embedded systems (DIPES 2006)

  31. Loeser C, Mueller W, Berger F, Eikerling H-J (2003) Peer to peer networks for virtual home environments. In: Proceedings of the 36th Hawaii international conference on system sciences (HICSS-36)

  32. Neumann T (2003) Development of a java-based real-time controller for the aj-100 processor. Master’s thesis, Paderborn University

  33. Schaefer R, Mueller W, Deimann R, Kleinjohann B (2007) An affordable positioning system for location-aware applications in smart homes. In: Workshop on mobile spatial interaction (CHI 2007)

  34. Kamada T (1998) Compact HTML for small information appliances. W3CNote. World Wide Web Consortium, February 1998

  35. Tuulari E, Ylisaukko-oja A (2002) Soapbox: a platform for ubiquitous computing research and applications. In: Proceedings of pervasive, Zurich, Switzerland, August 2002

  36. Coutaz J, Nigay L, Salber D, Blandford A, May J, Young RM (1995) Four easy pieces for assessing the usability of multimodal interaction: the care properties. In: Arnesen SA, Gilmore D (eds) Proceedings of the INTERACT’95 conference. Chapman & Hall, London, pp 115–120

    Google Scholar 

  37. Martin JC (1998). Tycoon: Theoretical framework and software tools for multimodal interfaces. In: Lee J (ed) Intelligence and Multimodality in Multimedia interfaces. AAAI Press, Menlo Park

    Google Scholar 

  38. Maes SH, Saraswat V (2003). Multimodal interaction requirements. W3C NOTE. World Wide Web Consortium, January 2003

  39. Larson JA, Raman T, Raggett D (2003). W3C multimodal interaction framework. W3C NOTE. World Wide Web Consortium, May 2003

  40. Plomp J, Schaefer R, Mueller W (2002). Comparing transcoding tools for use with a generic user interface format. In: Proceedings of extreme markup languages, August 2002

  41. Schaefer R, Mueller W, Groppe J (2006). Profile processing and evolution for smart environments. In: Proceedings of 3rd international conference on ubiquitous intelligence and computing (UIC-06)

  42. Mueller W, Schaefer R, Bleul S (2004). Interactive multimodal user interfaces for mobile devices. In: Proceedings of Hawaii international conference on system sciences (HICSS 37)

Download references

Author information

Authors and Affiliations

  1. C-LAB, Paderborn University, Paderborn, Germany

    Robbie Schaefer & Wolfgang Mueller

Authors
  1. Robbie Schaefer
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Wolfgang Mueller
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Robbie Schaefer.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Schaefer, R., Mueller, W. Assessment of a multimodal interaction and rendering system against established design principles. J Multimodal User Interfaces 2, 25–41 (2008). https://doi.org/10.1007/s12193-008-0003-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12193-008-0003-3

Keywords