Abstract
One of the main challenges of Human Computer Interaction researches is to improve naturalness of the user’s interaction process. Currently two widely investigated directions are the adaptivity and the multimodality of interaction. Starting from the adaptivity concept, the paper provides an analysis of methods that make multimodal interaction adaptive respect to the final users and evolutionary over time. A comparative analysis between the concepts of adaptivity and evolution, given in literature, is provided, highlighting their similarities and differences and an original definition of evolutionary multimodal interaction is provided. Moreover, artificial intelligence techniques, quantum computing concepts and evolutionary computation applied to multimodal interaction are discussed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Thomsen, J., Vanrompay, Y., Berbers, Y.: Evolution of context-aware user profiles. In: ICUMT 2009, vol. 1(6) (2009)
Maybury, M., Wahlster, W.: Intelligent User Interfaces: An Introduction. In: Readings in Intelligent User Interfaces, pp. 1–13. Morgan Kaufmann, San Francisco (1998)
Caschera, M.C., Ferri, F., Grifoni, P.: Multimodal interaction systems: information and time features. International Journal of Web and Grid Services (IJWGS) 3(1), 82–99 (2007)
D’Ulizia, A., Ferri, F., Grifoni, P.: Generating Multimodal Grammars for Multimodal Dialogue Processing. IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans 40(6), 1130–1145 (2010)
D’Ulizia, A.: Exploring Multimodal Input Fusion Strategies. In: The Handbook of Research on Multimodal Human Computer Interaction and Pervasive Services: Evolutionary Techniques for Improving Accessibility, pp. 34–57. IGI Publishing (2009)
Foster, M. E.: State of the art review: multimodal fission. Public deliverable 6.1, COMIC project (2002)
Grifoni, P.: Multimodal fission. In: The Handbook of Research on Multimodal Human Computer Interaction and Pervasive Services: Evolutionary Techniques for Improving Accessibility, pp. 103–120. IGI Publishing (2009)
Harper, M.P., Shriberg, E.: Multimodal model integration for sentence unit detection. In: ICMI 2004, pp. 121–128 (2004)
Caschera, M.C.: Interpretation methods and ambiguity management in multimodal systems. In: Grifoni, P. (ed.) Handbook of Research on Multimodal Human Computer Interaction and Pervasive Services: Evolutionary Techniques for Improving Accessibility, pp. 87–102. IGI Global, USA (2009)
Bowman, D.A., Kruijff, E., LaViola, J.J., Poupyrev, I.: 3D User Interfaces. In: Theory and Practice. Addison-Wesley (2005)
Renny Octavia, J., Raymaekers, C., Coninx, K.: Adaptation in virtual environments: conceptual framework and user models. Multimedia Tools Appl. 54(1), 121–142 (2011)
Ehlert P.A.M.: Intelligent user interfaces: introduction and survey. Research Report DKS03-01 / ICE 01 Data and Knowledge Systems group Faculty of Information Technology and Systems Delft University of Technology (2003)
Au Yeung, C.M., Gibbins, N., Shadbolt, N.: A Study of User Profile Generation from Folksonomies. In: Social Web and Knowledge Management, Social Web 2008 Workshop at WWW 2008, Beijing, China, April 21-25 (2008)
Carrillo-Ramos, A., Villanova-Oliver, M., Gensel, J., Martin, H.: Contextual User Profile for Adapting Information in Nomadic Environments. In: Weske, M., Hacid, M.-S., Godart, C. (eds.) WISE Workshops 2007. LNCS, vol. 4832, pp. 337–349. Springer, Heidelberg (2007)
Rousseau, C., Bellik, Y., Vernier, F.: Architecture Framework for Output Multimodal Systems Design. In: Proceedings of OZCHI 2004, November 22-24. ACM Press (2004)
McBryan, T., McGee-Lennon, M.R., Gray, P.: An integrated approach to supporting interaction evolution in home care systems. In: 1st International Conference on Pervasive Technologies Related to Assistive Environments, Athens, Greece, July 16-18, vol. 282, p. 1. ACM Press (2008)
Goldin, D., Keil, D.: Interaction, Evolution, and Intelligence. In: Proc. CEC 2001 (Congress on Evolutionary Computation), Seoul, Korea, vol. 2, pp. 805–814. IEEE Press, Seoul (2001)
Holland, J.H.: Adaptation in Natural and Artificial Systems. The Univ. of Michigan Press (1975)
Pauplin, O., Caleb-Solly, P., Smith, J.: User-centric image segmentation using an interactive parameter adaptation tool. Pattern Recognition 43(2), 519–529 (2010)
Toney, D.: Evolutionary Reinforcement Learning of Spoken Dialogue Strategies. PhD thesis (2007), http://hdl.handle.net/1842/1769 (retrieved)
Shankar, A., Louis, S.J.: XCS for Personalizing Desktop Interfaces. IEEE Transactions on Evolutionary Computation 14, 547–560 (2010)
Miwa, Y., Wesugi, S., Ishibiki, C., Itai, S.: Embodied interface for emergence and co-share of ‘Ba’, usability evaluation and interface design. In: Proceedings of HCI Int., pp. 248–252 (2001)
Hinoshita, W., Ogata, T., Kozima, H., Kanda, H., Takahashi, T., Okuno, H.G.: Emergence of Evolutionary Interaction with Voice and Motion between Two Robots using RNN. In: IROS 2009 Proceedings of the 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4186–4192 (2009)
Althoff, F., Al-Hames, M., McGlaun, G., Lang, M.: Towards a New Approach for Integrating Multimodal User Input Based on Evolutionary Computation. In: Proceedings ICASSP 2002. IEEE Signal Processing Society, Orlando, USA (2002)
Meshoul, S., Batouche, M., Belhadj-moustefa, K.: An evolutionary framework for image data fusion based on the maximization of mutual information. In: Proceeding of the International Symposium on Software and Systems, I3S 2001 (2001)
Han, K., Kim, J.: Quantum-inspired evolutionary algorithm for a class of combinatorial optimization. IEEE Transactions on Evolutionary Computation 6(6) (2002)
Draa, A., Batouche, M., Talbi, H.: A quantum-inspired differential evolution algorithm for rigid image registration. Trans. Eng. Comput. Technol., 408–411 (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Caschera, M.C., D’Ulizia, A., Ferri, F., Grifoni, P. (2012). Towards Evolutionary Multimodal Interaction. In: Herrero, P., Panetto, H., Meersman, R., Dillon, T. (eds) On the Move to Meaningful Internet Systems: OTM 2012 Workshops. OTM 2012. Lecture Notes in Computer Science, vol 7567. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33618-8_80
Download citation
DOI: https://doi.org/10.1007/978-3-642-33618-8_80
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33617-1
Online ISBN: 978-3-642-33618-8
eBook Packages: Computer ScienceComputer Science (R0)