Article

Agent-based architecture for implementing multimodal learning environments for visually impaired children

Authors:

Jouni SaloAuthors Info & Claims

ICMI '05: Proceedings of the 7th international conference on Multimodal interfaces

Pages 309 - 316

https://doi.org/10.1145/1088463.1088516

Published: 04 October 2005 Publication History

Abstract

Visually impaired children have a great disadvantage in the modern society since their ability to use modern computer technology is limited due to inappropriate user interfaces. The aim of the work presented in this paper was to develop a multimodal software architecture and applications to support visually impaired children and to enable them to interact equally with sighted children in learning situations. The architecture is based on software agents, and has specific support for visual, auditory and haptic interaction. It has been used successfully with different groups of 7-8-year-old and 12-year-old visually impaired children. In this paper we focus on the enabling software technology and interaction techniques aimed to realize our goal.

References

[1]

3Dconnexion. http://www.3dconnexion.com/

[2]

BASS audio library. http://www.un4seen.com/

[3]

Cheyer, A., Julia, L., InfoWiz: An Animated Voice Interactive Information System. In Third International Conference on Autonomous Agents (Agents'99), Communicative Agents Workshop. 1999. http://www.ai.sri.com/~cheyer/pubs.html

[4]

Cheyer, A., Julia, L., Multimodal Maps: An Agent-based Approach. Multimodal Human-Computer Communication, Lecture Notes in Artificial Intelligence 1374, Springer, 1998, 111--121.

Digital Library

[5]

CLIPS: A Tool for Building Expert Systems. http://www.ghg.net/clips/CLIPS.html

[6]

Computer GRaphics Access for Blind people through a haptic virtual environment. http://www.grab-eu.com

[7]

Coutaz, J. PAC, an Object Oriented Model for Dialog Design. In Proceedings of Interact '87. North Holland, 1987, 431--436.

[8]

Foundation for Intelligent Physical Agents. http://www.fipa.org/

[9]

GNU Common C++ library. http://www.gnu.org/software/commoncpp/

[10]

Hietala, P. and Niemirepo T., A framework for building agent-based learning environments. In Proceedings of AI-ED 95: Artificial intelligence in education. AACE, 1995, 578.

[11]

Jansson, G. and Billberger,K., The PHANToM Used without Visual Guidance. In The First Phantom Users Research Symposium (PURS 99). http://mbi.dkfz-heidelberg.de/purs99/

[12]

Kangassalo, M. Exploratory Learning in PICCO Environment. In, Proceedings of the 10th European - Japanese Conference on Information Modelling and Knowledge Bases, Pori School of Technology and Economics, Series A, number 28, Pori, 2000, 259--261.

[13]

Magnusson, C., Rassmus-Gröhn, K., Sjöström, C. and Danielsson, H., Navigation and recognition in complex haptic virtual environments - reports from an extensive study with blind users. In Proceedings of Eurohaptics 2002, University of Edinburgh, 2002. http://www.eurohaptics.vision.ee.ethz.ch/2002.shtml

[14]

Moore, R., Dowding, J., Bratt, H., Gawron, J. M., Gorfu, Y. and Cheyer A., CommandTalk: A Spoken-Language Interface for Battlefield Simulations. In Proceedings of the Fifth Conference on Applied Natural Language Processing. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1997, 1--7.

Digital Library

[15]

Moran, D. B., Cheyer, A. J., Julia, L. E., Martin, D. L. and Park, S., Multimodal User Interfaces in the Open Agent Architecture. In Proceedings of the 2nd international conference on intelligent user interfaces. ACM Press, 1997, 61--68.

Digital Library

[16]

Nigay, L. and Coutaz, J. A Generic Platform for Addressing the Multimodal Challenge. In Proceedings of ACM CHI'95. ACM Press, 1995, 98--105.

Digital Library

[17]

Patomäki, S., Raisamo, R., Salo, J., Pasto, V.and Hippula, A., Experiences on Haptic interfaces for visually impaired young children. In Proceedings of Sixth International Conference on Multimodal Interfaces (ICMI'04). ACM Press, 2004, 281--288.

Digital Library

[18]

Reachin Technologies AB. http://www.reachin.se/

[19]

SensAble Technologies Inc. http://www.sensable.com/

[20]

Sjöström, C., Designing haptic computer interfaces for blind people. In Proceedings of Sixth International Symposium on Signal Processing and its Applications (ISSPA 2001). IEEE, 2001, 68--71.

[21]

Sjöström, C., Non-visual haptic interaction design: Guidelines and applications. Doctoral dissertation, Certec, Lund Institute of Technology, 2002.

[22]

SQLite database library. http://www.sqlite.org/

[23]

Wood, J., Magennis, M., Arias, E., Gutierrez, T., Graupp, H. and Bergamasco, M., The Design and Evaluation of a Computer Game for the Blind in the GRAB Haptic Audio Virtual Environment. In Proceedings of Eurohaptics 2003. http://www.eurohaptics.vision.ee.ethz.ch/2003.shtml

[24]

Xerces C++ library. http://xml.apache.org/xerces-c/

Cited By

Szymczak DRassmus-Gröhn KHedvall PMagnusson C(2019)Dynamic multimodal drawing in school: Exploring technology support of drawing skills development in children with visual impairmentsTechnology and Disability10.3233/TAD-19022431:3(83-99)Online publication date: 1-Aug-2019
https://doi.org/10.3233/TAD-190224
Tuominen EKangassalo MHietala PRaisamo RPeltola K(2018)Proactive agents to assist multimodal explorative learning of astronomical phenomenaAdvances in Human-Computer Interaction10.1155/2008/3870762008(1-13)Online publication date: 13-Dec-2018
https://dl.acm.org/doi/10.1155/2008/387076
Pun TRoth PBologna GMoustakas KTzovaras D(2018)Image and video processing for visually handicapped peopleJournal on Image and Video Processing10.1155/2007/252142007:5(1-12)Online publication date: 13-Dec-2018
https://dl.acm.org/doi/10.1155/2007/25214
Show More Cited By

Index Terms

Agent-based architecture for implementing multimodal learning environments for visually impaired children
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Haptic devices
      2. Touch screens
    2. Interaction techniques
      1. Auditory feedback
  2. Interaction design
    1. Interaction design process and methods
      1. User centered design

Recommendations

Supporting visually impaired children with software agents in a multimodal learning environment

Visually impaired children have a great disadvantage in the modern society since their ability to use modern computer technology is limited due to inappropriate user interfaces. The aim of the work presented in this paper was to develop a multimodal ...
Testing Usability of Multimodal Applications with Visually Impaired Children

Testing multimodal applications with visually impaired children requires specific testing methods andprocedures. In this article, we used a SensAble Phantom device to produce haptic feedback together withstereo sound and visual feedback. We also ...
Supporting Collaboration between Visually Impaired and Sighted Children in a Multimodal Learning Environment
HAID '08: Proceedings of the 3rd international workshop on Haptic and Audio Interaction Design

Visually impaired pupils are a group that teachers need to pay attention to especially when planning group work. The need for supporting collaboration between visually impaired and sighted people has been pointed out but still there are few evaluations ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMI '05: Proceedings of the 7th international conference on Multimodal interfaces

October 2005

344 pages

ISBN:1595930280

DOI:10.1145/1088463

General Chairs:
Gianni Lazzari
ITC-irst, Trento (Italy)
,
Fabio Pianesi
ITC-irst, Trento (Italy)
,
Program Chairs:
James Crowley
I.N.P. Grenoble (France)
,
Kenji Mase
Nagoya University (Japan)
,
Sharon Oviatt
Oregon Health & Sciences University

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 October 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ICMI05

Sponsor:

ICMI05: Seventh International Conference on Multimodal Interfaces 2005

October 4 - 6, 2005

Torento, Italy

Acceptance Rates

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
763
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)3

Reflects downloads up to 27 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Szymczak DRassmus-Gröhn KHedvall PMagnusson C(2019)Dynamic multimodal drawing in school: Exploring technology support of drawing skills development in children with visual impairmentsTechnology and Disability10.3233/TAD-19022431:3(83-99)Online publication date: 1-Aug-2019
https://doi.org/10.3233/TAD-190224
Tuominen EKangassalo MHietala PRaisamo RPeltola K(2018)Proactive agents to assist multimodal explorative learning of astronomical phenomenaAdvances in Human-Computer Interaction10.1155/2008/3870762008(1-13)Online publication date: 13-Dec-2018
https://dl.acm.org/doi/10.1155/2008/387076
Pun TRoth PBologna GMoustakas KTzovaras D(2018)Image and video processing for visually handicapped peopleJournal on Image and Video Processing10.1155/2007/252142007:5(1-12)Online publication date: 13-Dec-2018
https://dl.acm.org/doi/10.1155/2007/25214
Moll JHuang YSallnäs E(2010)Audio makes a difference in haptic collaborative virtual environmentsInteracting with Computers10.1016/j.intcom.2010.06.00122:6(544-555)Online publication date: 1-Nov-2010
https://dl.acm.org/doi/10.1016/j.intcom.2010.06.001
Junaini SChui L(2009)Learning Sarawak Local Malay Dialect Using Pedagogical AgentProceedings of the 2009 International Conference on Computer Technology and Development - Volume 0110.1109/ICCTD.2009.81(463-467)Online publication date: 13-Nov-2009
https://dl.acm.org/doi/10.1109/ICCTD.2009.81
Tanhua-Piiroinen EPasto VRaisamo RSallnäs E(2008)Supporting Collaboration between Visually Impaired and Sighted Children in a Multimodal Learning EnvironmentHaptic and Audio Interaction Design10.1007/978-3-540-87883-4_2(11-20)Online publication date: 2008
https://doi.org/10.1007/978-3-540-87883-4_2
Pietrzak TMartin BPecci ISaarinen RRaisamo RJärvi JMase KMassaro DTakeda KRoy DPotamianos A(2007)The micole architectureProceedings of the 9th international conference on Multimodal interfaces10.1145/1322192.1322227(193-200)Online publication date: 12-Nov-2007
https://dl.acm.org/doi/10.1145/1322192.1322227
Raisamo RHippula APatomaki STuominen EPasto VHasu M(2006)Testing Usability of Multimodal Applications with Visually Impaired ChildrenIEEE MultiMedia10.1109/MMUL.2006.6813:3(70-76)Online publication date: 1-Jul-2006
https://dl.acm.org/doi/10.1109/MMUL.2006.68

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten