Article

Migratory MultiModal interfaces in MultiDevice environments

Authors:

Fabio PaternòAuthors Info & Claims

ICMI '05: Proceedings of the 7th international conference on Multimodal interfaces

Pages 92 - 99

https://doi.org/10.1145/1088463.1088481

Published: 04 October 2005 Publication History

Abstract

This paper describes an environment able to support migratory multimodal interfaces in multidevice environments. We introduce the software architecture and the device-independent languages used by our tool, which provides services enabling users to freely move about, change device and continue the current task from the point where they left off in the previous device. Our environment currently supports interaction with applications through graphical and vocal modalities, either separately or together. Such applications are implemented in Web-based languages. We discuss how the features of the device at hand, desktop or mobile, are considered when generating the multimodal user interface.

References

[1]

Abrams, M., Phanouriou, C., Batongbacal, A., Williams, S., Shuster, J., 1994. UIML: An Appliance-Independent XML User Interface Language, Proceedings of the 8th WWW conference.

Digital Library

[2]

Balme, L. Demeure, A., Barralon, N., Coutaz, J., Calvary, G. CAMELEON-RT: a Software Architecture Reference Model for Distributed, Migratable, and Plastic User Interfaces. In Proceedings the Second European Symposium on Ambient Intelligence (EUSAI '04), LNCS 3295, Markopoulos et al. Springer-Verlag, Berlin Heidelberg, 2004, 291--302.

[3]

Bandelloni, R., Berti, S., Paternò F., "Mixed-Initiative, Trans-Modal Interface Migration", Proceedings Mobile HCI 2004, Glasgow, September 2004, Lecture Notes Computer Science 3160, pp.216--227, Sprinter Verlag.

[4]

Bouchet J., Nigay L., ICARE: A Component-Based Approach for the Design and Development of Multimodal Interfaces, Adjunct Proceedings CHI 2004, pp.1325--1328.

Digital Library

[5]

Calvary, G., Coutaz, J., Thevenin, D., Limbourg, Q., Bouillon, L., Vanderdonckt, J. A Unifying Reference Framework for Multi-Target User Interfaces. Interacting with Computers. Vol. 15, No. 3, June 2003, pp. 289--308.

[6]

Coutaz J., Nigay L., Salber D.,.Blandford A, May J., Young R., 1995. Four Easy Pieces for Assessing the Usability of Multmodal Interaction: the CARE properties. Proceedings INTERACT 1995, pp.115--120

[7]

Channarukul S., McRoy S. W.,. Ali S. S. MULTIFACE: Multimodal Content Adaptations for Heterogeneous Devices.

[8]

Garlan, D., Siewiorek, D., Smailagic, A., Steenkiste, P. Project Aura: Toward Distraction-Free Pervasive Computing. IEEE Pervasive Computing, Vol 21, No 2 (April-June 2002), 22--31.

Digital Library

[9]

Limbourg, Q., Vanderdonckt, J., UsiXML: A User Interface Description Language Supporting Multiple Levels of Independence, in Matera, M., Comai, S. (Eds.), 'Engineering Advanced Web Applications', Rinton Press, Paramus, 2004.

[10]

Bandelloni, R., Paternò, F. Migratory User Interfaces Able to Adapt to Various Interaction Platforms. In International Journal of Human Computer Studies 60, pp. 621--639, Elsevier, 2004.

[11]

Nichols, J. Myers B. A., Higgins M., Hughes J., Harris T. K., Rosenfeld R., Pignol M., 2002. "Generating remote control interfaces for complex appliances". Proceedings ACM UIST'02, pp.161--170.

Digital Library

[12]

Obrenovic, Z., Starcevic D., Selic B., A Model-Driven Approach to Content Repurposing, IEEE Mutimedia, Januray March 2004, pp.62--71.

Digital Library

[13]

Paternò, F., "Model-Based Design and Evaluation of Interactive Application". Springer Verlag, ISBN 1-85233-155-0, 1999.

Digital Library

[14]

Pérez-Quiñoes, M, A., Capra, R, G., Shao, Z. The Ears Have It: A Task by Information Structure Taxonomy for Voice Access to Web Pages. In Proceedings of INTERACT 2003, IOS Press.

[15]

Puerta, A., Eisenstein, XIML: A Common Representation for Interaction Data, Proceedings ACM IUI'01, pp.214--215.

Digital Library

[16]

Szekely, P., 1996. Retrospective and Challenges for Model-Based Interface Development. 2nd International Workshop on Computer-Aided Design of User Interfaces, Namur, Namur University Press.

Cited By

Yuan YRiche NMarquardt NNicholas MSeyed TRomat HLee BPahud MGoldstein JVishkaie RHolz CHinckley K(2022)Understanding Multi-Device Usage Patterns: Physical Device Configurations and Fragmented WorkflowsProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517702(1-22)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517702
Teixeira AAlmeida NKetsmur MSilva S(2020)Effective natural interaction with our sensorized smart homesAdvances in Ubiquitous Computing10.1016/B978-0-12-816801-1.00006-2(185-222)Online publication date: 2020
https://doi.org/10.1016/B978-0-12-816801-1.00006-2
Brudy FHolz CRädle RWu CHouben SKlokmose CMarquardt NBrewster SFitzpatrick GCox AKostakos V(2019)Cross-Device TaxonomyProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300792(1-28)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300792
Show More Cited By

Index Terms

Migratory MultiModal interfaces in MultiDevice environments
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
  2. Interaction design
    1. Interaction design process and methods
      1. Interface design prototyping
2. Software and its engineering
  1. Software notations and tools
    1. Development frameworks and environments
      1. Integrated and visual development environments

Recommendations

Dynamic generation of web migratory interfaces
MobileHCI '05: Proceedings of the 7th international conference on Human computer interaction with mobile devices & services

In this paper, we present a solution for dynamic generation of Web user interfaces that can dynamically migrate among different platforms. The solution is based on a migration/proxy server able to automatically convert a desktop service into a service ...
Towards multimodal user interfaces composition based on UsiXML and MBD principles
HCI'07: Proceedings of the 12th international conference on Human-computer interaction: intelligent multimodal interaction environments

In software design, the reuse issue brings the increasing of web services, components and others techniques. These techniques allow reusing code associated to technical aspect (as software component). With the development of business components which ...
Usability of user interfaces: from monomodal to multimodal
BCS-HCI '07: Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI...but not as we know it - Volume 2

This workshop is aimed at reviewing and comparing existing Usability Evaluation Methods (UEMs) which are applicable to monomodal and multimodal applications, whether they are web-oriented or not. It addresses the problem on how to assess the usability ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMI '05: Proceedings of the 7th international conference on Multimodal interfaces

October 2005

344 pages

ISBN:1595930280

DOI:10.1145/1088463

General Chairs:
Gianni Lazzari
ITC-irst, Trento (Italy)
,
Fabio Pianesi
ITC-irst, Trento (Italy)
,
Program Chairs:
James Crowley
I.N.P. Grenoble (France)
,
Kenji Mase
Nagoya University (Japan)
,
Sharon Oviatt
Oregon Health & Sciences University

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 October 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ICMI05

Sponsor:

ICMI05: Seventh International Conference on Multimodal Interfaces 2005

October 4 - 6, 2005

Torento, Italy

Acceptance Rates

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
528
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yuan YRiche NMarquardt NNicholas MSeyed TRomat HLee BPahud MGoldstein JVishkaie RHolz CHinckley K(2022)Understanding Multi-Device Usage Patterns: Physical Device Configurations and Fragmented WorkflowsProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517702(1-22)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517702
Teixeira AAlmeida NKetsmur MSilva S(2020)Effective natural interaction with our sensorized smart homesAdvances in Ubiquitous Computing10.1016/B978-0-12-816801-1.00006-2(185-222)Online publication date: 2020
https://doi.org/10.1016/B978-0-12-816801-1.00006-2
Brudy FHolz CRädle RWu CHouben SKlokmose CMarquardt NBrewster SFitzpatrick GCox AKostakos V(2019)Cross-Device TaxonomyProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300792(1-28)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300792
Almeida NSilva SSantos BTeixeira A(2016)Interactive, Multi-device Visualization Supported by a Multimodal Interaction Framework: Proof of ConceptHuman Aspects of IT for the Aged Population. Design for Aging10.1007/978-3-319-39943-0_27(279-289)Online publication date: 21-Jun-2016
https://doi.org/10.1007/978-3-319-39943-0_27
Pruvost GHeinroth TBellik YMinker W(2016)User Interaction Adaptation Within Ambient EnvironmentsNext Generation Intelligent Environments10.1007/978-3-319-23452-6_6(221-263)Online publication date: 2016
https://doi.org/10.1007/978-3-319-23452-6_6
Pruvost GHeinroth TBellik YMinker W(2011)User Interaction Adaptation within Ambient EnvironmentsNext Generation Intelligent Environments10.1007/978-1-4614-1299-1_5(153-194)Online publication date: 21-Aug-2011
https://doi.org/10.1007/978-1-4614-1299-1_5
Blumendorf MRoscher DAlbayrak SGao WLee CYang JChen XEskenazi MZhang Z(2010)Dynamic user interface distribution for flexible multimodal interactionInternational Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction10.1145/1891903.1891930(1-8)Online publication date: 8-Nov-2010
https://dl.acm.org/doi/10.1145/1891903.1891930
Coutaz JSukaviriya NVanderdonckt JHarrison M(2010)User interface plasticityProceedings of the 2nd ACM SIGCHI symposium on Engineering interactive computing systems10.1145/1822018.1822019(1-8)Online publication date: 19-Jun-2010
https://dl.acm.org/doi/10.1145/1822018.1822019
Lardon JGravier CFayolle J(2010)Towards a unified device characteristics retrieval and propagation: Context proxy presentation2010 IEEE International Conference on Multimedia and Expo10.1109/ICME.2010.5583283(1342-1346)Online publication date: Jul-2010
https://doi.org/10.1109/ICME.2010.5583283
Alapetite A(2010)Dynamic 2D-barcodes for multi-device Web session migration including mobile phonesPersonal and Ubiquitous Computing10.1007/s00779-009-0228-514:1(45-52)Online publication date: 1-Jan-2010
https://dl.acm.org/doi/10.1007/s00779-009-0228-5
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten