Skip to main content
SpringerLink
Log in

Supporting visually impaired children with software agents in a multimodal learning environment

  • Original Article
  • Published:
Virtual Reality Aims and scope Submit manuscript

Abstract

Visually impaired children have a great disadvantage in the modern society since their ability to use modern computer technology is limited due to inappropriate user interfaces. The aim of the work presented in this paper was to develop a multimodal software architecture and applications to support learning of visually impaired children. The software architecture is based on software agents, and has specific support for visual, auditory and haptic interaction. It has been used successfully with different groups of 7-8 year-old and 12 year-old visually impaired children. In this paper we discuss the enabling software technology and interaction techniques aimed to realize our goal and our experiences in the actual use of the system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Patomäki S, Raisamo R, Salo J, Pasto V, Hippula A (2004) Experiences on haptic interfaces for visually impaired young children. In: Proceedings of sixth international conference on multimodal interfaces (ICMI’04): ACM Press, pp 281-288

  2. SensAble Technologies Inc. http://www.sensable.com

  3. Reachin Technologies AB. http://www.reachin.se

  4. 3Dconnexion. http://www.3dconnexion.com/

  5. Kangassalo M (1997) The Formation of Children’s Conceptual Models Concerning a Particular Natural Phenomenon Using PICCO, a Pictorial Computer Simulation. Doctoral dissertation. Acta Universitatis Tamperensis 559. University of Tampere. Tampere, p 188

  6. Kangassalo M (1991/1999) PICCO -kuvallinen tietokonesimulaatio [PICCO, Pictorial Computer Simulation]. CD-ROM 951-98035-0-5

  7. Kangassalo M (1992) The pictorial computer-based simulation in natural sciences for children’s use. In: Ohsuga S, Kangassalo H, Jaakkola H, Hori K, Yonezaki N (eds) Information modelling and knowledge bases 3rd Foundations, theory and applications. IOS Press, Amsterdam, pp 511-524

    Google Scholar 

  8. Jansson G, Billberger K (1999) The PHANToM used without visual guidance. In: The first phantom users research symposium (PURS 99). http://mbi.dkfz-heidelberg.de/purs99

  9. Magnusson C, Rassmus-Gröhn K, Sjöström C, Danielsson H (2002) Navigation and recognition in complex haptic virtual environments—reports from an extensive study with blind users. In: Proceedings of Eurohaptics 2002

  10. Sjöström C (2001) Designing haptic computer interfaces for blind people. In: proceedings of sixth international symposium on signal processing and its applications (ISSPA 2001). IEEE, pp68-71

  11. Sjöström C (2002) Non-visual haptic interaction design: guidelines and applications. Doctoral dissertation, Certec, Lund Institute of Technology

  12. Computer graphics access for Blind people through a haptic virtual environment. http://www.grab-eu.com

  13. Wood J, Magennis M, Arias E, Gutierrez T, Graupp H, Bergamasco M (2003) The design and evaluation of a computer game for the blind in the GRAB haptic audio virtual environment. In: Proceedings of Eurohaptics 2003

  14. Moran D-B, Cheyer AJ, Julia LE, Martin DL, Park S (1997) Multimodal user interfaces in the open agent architecture. In: Proceedings of the 2nd international conference on intelligent user interfaces. ACM Press, pp61-68

  15. Cheyer A, Julia L (1999) InfoWiz: An animated voice interactive information system. In: Third international conference on autonomous agents (Agents’99), communicative agents workshop

  16. Moore R, Dowding J, Bratt H, Gawron J M, Gorfu Y, Cheyer A (1997) CommandTalk: A spoken-language interface for battlefield simulations. In: Proceedings of the fifth conference on applied natural language processing. morgan kaufmann publishers inc. San Francisco, CA, USA, pp1-7

  17. Cheyer A, Julia L (1998) Multimodal Maps: An agent-based approach. multimodal human-computer communication. lecture notes in artificial intelligence 1374. Springer, pp111-121

  18. Coutaz J (1987) PAC, an object oriented model for dialog design. in: proceedings of interact ‘87. North-Holland, pp431-436

  19. Nigay L, Coutaz J (1995) A generic platform for addressing the multimodal challenge. In: Proceedings of ACM CHI’95. ACM press, pp98-105

  20. Hietala P, Niemirepo T (1995) A framework for building agent-based learning environments. In: Proceedings of AI-ED 95: Artificial intelligence in education. AACE, p 578

  21. Foundation for Intelligent Physical Agents. http://www.fipa.org/

  22. CLIPS: A Tool for Building Expert Systems. http://www.ghg.net/clips/CLIPS.html

  23. SQLite. http://www.sqlite.org/

  24. Kangassalo M (1998) Modelling a natural phenomenon for a pictorial computer-based simulation. In: Kangassalo H, Charrel PJ, Jaakkola H (eds) Information modeling and knowledge bases IX. IOS press, Amsterdam, pp 239-254

    Google Scholar 

  25. Kangassalo M (1996) PICCO as a Cognitive Tool. In: Tanaka Y, Kangassalo H, Jaakkola H, Yamamoto A (eds) Information modelling and knowledge bases VII. IOS press, Amsterdam, pp 344-357

    Google Scholar 

  26. Lonka K, Hakkarainen K, Sintonen M (2000) Progressive inquiry learning for children–Experiences, Possibilities, Limitations. Eur Early Childhood Educ Res J 8:7-23

    Google Scholar 

  27. Kangassalo M, Raisamo R, Hietala P, Järvi J, Peltola K, Saarinen R, Tuominen E, Hippula A (2005) Proactive agents that support children’s exploratory learning. In: Kiyoki Y, Wangler B, Jaakkola H, Kangassalo H (eds) Information modelling and knowledge bases XVI. IOS press, Amsterdam, pp 123-133

    Google Scholar 

Download references

Acknowledgments

This research was funded by Academy of Finland, Proactive Computing Research Program (grants 202179 and 202180).

Author information

Authors and Affiliations

  1. Tampere Unit for Computer-Human Interaction (TAUCHI), Department of Computer Sciences, University of Tampere, 33014, Tampere, Finland

    Rami Saarinen, Janne Järvi, Roope Raisamo & Jouni Salo

  2. Department of Teacher Education, Early Childhood Education, University of Tampere, 33014, Tampere, Finland

    Eva Tuominen, Marjatta Kangassalo & Kari Peltola

Authors
  1. Rami Saarinen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Janne Järvi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Roope Raisamo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Eva Tuominen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Marjatta Kangassalo
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kari Peltola
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jouni Salo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Janne Järvi.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Saarinen, R., Järvi, J., Raisamo, R. et al. Supporting visually impaired children with software agents in a multimodal learning environment. Virtual Reality 9, 108–117 (2006). https://doi.org/10.1007/s10055-005-0011-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10055-005-0011-5

Keywords

Search

Navigation