skip to main content
10.1145/2414536.2414575acmotherconferencesArticle/Chapter ViewAbstractPublication PagesozchiConference Proceedingsconference-collections
research-article

Adaptive probabilistic fission for multimodal systems

Published: 26 November 2012 Publication History

Abstract

Human beings continuously adapt their way of communication to their surroundings and their communication partner. Although context-aware ubiquitous systems gather a lot of information to maximize their functionality, they predominantly use static ways to communicate. In order to fulfill the user's communication needs and demands, the sensor's diverse and sometimes uncertain information must also be used to dynamically adapt the user interface. In this article we present ProFi, a system for Probabilistic Fission, designed to reason on adaptive and multimodal output based on uncertain or ambiguous data. In addition, we present a system architecture as well as a new meta model for multimodal interactive systems. Based on this meta model we describe ProFi's process of multimodal fission along with our current implementation.

References

[1]
M. Blumendorf, D. Roscher, and S. Albayrak. Dynamic user interface distribution for flexible multimodal interaction. In International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction, ICMI-MLMI '10, pages 20:1--20:8, New York, NY, USA, November 2010. ACM.
[2]
G. Calvary, J. Coutaz, D. Thevenin, Q. Limbourg, L. Bouillon, and J. Vanderdonckt. A unifying reference framework for multi-target user interfaces. Interacting with Computers, 15(3):289--308, 2003.
[3]
D. Costa and C. Duarte. Adapting multimodal fission to user's abilities. In Proceedings of the 6th international conference on Universal access in human-computer interaction: design for all and eInclusion - Volume Part I, UAHCI'11, pages 347--356, Berlin, Heidelberg, 2011. Springer-Verlag.
[4]
A. K. Dey and G. D. Abowd. Towards a better understanding of context and context-awareness. In In HUC'99: Proceedings of the 1st international symposium on Handheld and Ubiquitous Computing, pages 304--307. Springer-Verlag, 1999.
[5]
B. Dumas, D. Lalanne, and S. Oviatt. Multimodal Interfaces: A Survey of Principles, Models and Frameworks. In D. Lalanne and J. Kohlas, editors, Human Machine Interaction -- Research Results of the MMI Program, volume 5440/2009 of Lecture Notes in Computer Science, chapter 1, pages 3--26. Springer-Verlag, Berlin, Heidelberg, March 2009.
[6]
M. E. Foster. State of the art review: Multimodal fission. Public Deliverable 6.1, University of Edinburgh, September 2002. COMIC Project.
[7]
M. E. Foster and M. White. Assessing the impact of adaptive generation in the comic multimodal dialogue system. In In Proceedings of the IJCAI 2005 Workshop on Knowledge and Reasoning in Practical Dialogue Systems, 2005.
[8]
C. Gram and G. Cockton. Design principles for interactive software. Chapman & Hall, Ltd., London, UK, 1997.
[9]
M. D. Hina, C. Tadj, A. Ramdane-Cherif, and N. Levy. A Multi-Agent based Multimodal System Adaptive to the User's Interaction Context. In Multi-Agent Systems -- Modeling, Interactions, Simulations and Case Studies, chapter 2, pages 29--56. InTech, 2011.
[10]
F. Honold, M. Poguntke, F. Schüssel, and M. Weber. Adaptive Dialogue Management and UIDL-based Interactive Applications. In A. Coyette, D. Faure, J. Gonzalez, and J. Vanderdonckt, editors, Proceedings of the International Workshop on Software Support for User Interface Description Language (UIDL 2011), page 4, Paris, September 2011. Thales Research and Technology France. Interact 2011, Workshop.
[11]
F. Honold, F. Schüssel, M. Weber, G. Bertrand, F. Nothdurft, and W. Minker. Ein goal-basierter ansatz für adaptive multimodale systeme. In M. Eibl, editor, Mensch & Computer 2011: üüberMEDIEN\ÜBERmorgen, pages 357--360, München, 2011. Oldenbourg Verlag.
[12]
M. Horchani, L. Nigay, and F. Panaget. A platform for output dialogic strategies in natural multimodal dialogue systems. In IUI '07: Proceedings of the 12th international conference on Intelligent user interfaces, pages 206--215, New York, NY, USA, 2007. ACM.
[13]
C. Kohrs, N. Angenstein, H. Scheich, and A. Brechmann. Human striatum is differentially activated by delayed, omitted, and immediate registering feedback. Frontiers in Human Neuroscience, 6(00243), 2012.
[14]
D. Koller and N. Friedman. Probabilistic graphical models: Principles and techniques. The MIT Press, 2009.
[15]
L. Nigay and J. Coutaz. Multifeature systems: The care properties and their impact on software design. In Multimedia Interfaces: Research and Applications, chapter 9. AAAI Press, 1995.
[16]
S. Oviatt. Multimodal Interfaces. In A. Sears and J. A. Jacko, editors, The Human-Computer Interaction Handbook, pages 413--432. CRC Press, 2007.
[17]
R. Paramythis and S. Weibelzahl. A Decomposition Model for the Layered Evaluation of Interactive Adaptive Systems. In Proceedings of the 10th International Conference on User Modeling, LNAI 3538, pages 438--442. Springer, 2005.
[18]
C. Rousseau, Y. Bellik, F. Vernier, and D. Bazalgette. A framework for the intelligent multimodal presentation of information. Signal Process., 86(12):3696--3713, 2006.
[19]
T. Strang and C. Linnhoff-Popien. A Context Modeling Survey. In Workshop on Advanced Context Modelling, Reasoning and Management, UbiComp 2004 -- The Sixth Int. Conf. on Ubiquitous Computing, Nottingham/England, page 8, 2004.
[20]
O. Strnad, A. Felic, and A. Schmidt. Context Management for Self-adaptive User Interfaces in the Project MyUI, pages 261--272. VDE, Springer, 2012.
[21]
P. Watzlawick, J. B. Bavelas, and D. D. Jackson. Pragmatics of Human Communication: A Study of Interactional Patterns, Pathologies and Paradoxes. Norton, New York, 1967. Some Tentative Axioms of Communication.
[22]
A. Wendemuth and S. Biundo. A companion technology for cognitive technical systems. In A. Esposito, A. Vinciarelli, R. Hoffman, and V. C. Müller, editors, Proceedings of the EUCogII-SSPNET-COST2102 International Conference (2011), LNCS Proceedings on Cognitive Behavioural Systems, Dresden, 2012. Springer LNCS {in press or to appear 2012}.
[23]
S. Wolff, C. Kohrs, H. Scheich, and A. Brechmann. Temporal contingency and prosodic modulation of feedback in human-computer interaction: Effects on brain activation and performance in cognitive tasks. In Informatik 2011, GI-Jahrestagung 2011, Berlin, volume 192 of LNI, page 238, 2011.

Cited By

View all
  • (2019)Software platforms and toolkits for building multimodal systems and applicationsThe Handbook of Multimodal-Multisensor Interfaces10.1145/3233795.3233801(145-190)Online publication date: 1-Jul-2019
  • (2018)Combining Design-time Generation of Web-pages with Responsive Design for Improving Low-vision AccessibilityProceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems10.1145/3220134.3220141(1-7)Online publication date: 19-Jun-2018
  • (2018)Context-Based Multimodal Output for Human-Robot Collaboration2018 11th International Conference on Human System Interaction (HSI)10.1109/HSI.2018.8431264(450-456)Online publication date: Jul-2018
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
OzCHI '12: Proceedings of the 24th Australian Computer-Human Interaction Conference
November 2012
692 pages
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

  • New Zealand Chapter of ACM SIGCHI
  • Human Factors & Ergonomics Soc: Human Factors & Ergonomics Soc

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 November 2012

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. adaptive user interface
  2. modality arbitration
  3. multimodal interaction
  4. probabilistic multimodal fission

Qualifiers

  • Research-article

Conference

OzCHI '12
Sponsor:
  • Human Factors & Ergonomics Soc

Acceptance Rates

Overall Acceptance Rate 362 of 729 submissions, 50%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)4
  • Downloads (Last 6 weeks)1
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2019)Software platforms and toolkits for building multimodal systems and applicationsThe Handbook of Multimodal-Multisensor Interfaces10.1145/3233795.3233801(145-190)Online publication date: 1-Jul-2019
  • (2018)Combining Design-time Generation of Web-pages with Responsive Design for Improving Low-vision AccessibilityProceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems10.1145/3220134.3220141(1-7)Online publication date: 19-Jun-2018
  • (2018)Context-Based Multimodal Output for Human-Robot Collaboration2018 11th International Conference on Human System Interaction (HSI)10.1109/HSI.2018.8431264(450-456)Online publication date: Jul-2018
  • (2017)To Plan for the User Is to Plan with the User: Integrating User Interaction into the Planning ProcessCompanion Technology10.1007/978-3-319-43665-4_7(123-144)Online publication date: 5-Dec-2017
  • (2017)User-Centered PlanningCompanion Technology10.1007/978-3-319-43665-4_5(79-100)Online publication date: 5-Dec-2017
  • (2017)Advanced User Assistance for Setting Up a Home TheaterCompanion Technology10.1007/978-3-319-43665-4_24(485-491)Online publication date: 5-Dec-2017
  • (2017)Companion-Systems: A Reference ArchitectureCompanion Technology10.1007/978-3-319-43665-4_22(449-469)Online publication date: 5-Dec-2017
  • (2017)Management of Multimodal User Interaction in Companion-SystemsCompanion Technology10.1007/978-3-319-43665-4_10(187-207)Online publication date: 5-Dec-2017
  • (2016)User Expertise in Multimodal HCIProceedings of the European Conference on Cognitive Ergonomics10.1145/2970930.2970941(1-6)Online publication date: 5-Sep-2016
  • (2016)Measuring effects of user-specific behaviour on selection tasks in HCI2016 SAI Computing Conference (SAI)10.1109/SAI.2016.7556010(380-387)Online publication date: Jul-2016
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media