Skip to main content
SpringerLink
Log in

An Arduino-based device for visually impaired people to play videogames

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Blind players have many difficulties to access video games since most of them rely on impressive graphics and immersive visual experiences. To overcome this limitation, we propose a device designed for visually impaired people to interact with virtual scenes of video games. The device has been designed considering usability, economic cost, and adaptability as main features. To ensure usability, we considered the white cane paradigm since this is the most used device by the blind community. Our device supports left to right movements and collision detection as well as actions to manipulate scene objects such as drag and drop. To enhance realism, it also integrates a library with sounds of different materials to reproduce object collision. To reduce the economic cost, we used Arduino as the basis of our development. Finally, to ensure adaptability, we created an application programming interface that supports the connection with different games engines and different scenarios. To test the acceptance of the device 12 blind participants were considered (6 males and 6 females). In addition, we created three mini-games in Unity3D that require navigation and walking as principal actions. After playing, participants filled a questionnaire related to usability and suitability to interact with games, among others. They scored well in all features without distinction among player gender and being blind from birth. The relationship between device responsiveness and user interaction has been considered satisfactory. Despite our small test sample, our main goal has been accomplished, the proposed device prototype seems to be useful to visually impaired people.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Abdel-Wahab AG, El-Masry AAA (2011) Mobile information communication technologies adoption in developing countries: effects and implications. Hershey, IGI Global

    Book  Google Scholar 

  2. Archambault D (2004) The TIM project: overview of results. In: Proc. Computers helping people with special needs, pp 248–256

  3. Archambault D, Ossmann R, Gaudy T, Miesenberger K Computer games and visually impaired people (https://cedric.cnam.fr/fichiers/RC1204.pdf)

  4. Arduino What is Arduino. Available online: https://www.arduino.cc/en/Guide/Introduction. Accessed 22 May 2017

  5. Bach-y-Rita P, Collins CC, Saunders FA, White B, Scadden L (1969) Vision substitution by tactile image projection. Nature 221:963–964

    Article  Google Scholar 

  6. Balan O, Moldoveanu A, Moldoveanu F (2015) Navigational audio games: an effective approach toward improving spatial contextual learning for blind people. Int J Disabil Hum Dev 14(2):109–118

    Article  Google Scholar 

  7. Brewster SA (1998) Using non-speech sounds to provide navigation cues. ACM Trans Comput-Human Interact 5:224–259

    Article  Google Scholar 

  8. Card SK, Robertson GG, Mackinlay JD (1991) The information visualizer: an information workspace. In: Proc. ACM CHI’91 Conf, 181–188

  9. Csapó G, Nagy H, Stockman T (2015) A survey of assistive technologies and applications for blind users on mobile platforms: a review and foundation for research. J Multimodal User Interf 9(4):275–286

    Article  Google Scholar 

  10. Cui J, Liu Y, Xu Y, Zhao H, Zha H (2013) Tracking generic human motion via fusion of low- and high-dimensional approaches. IEEE Trans Syst Man Cybern Syst 43(4):996–1002

    Article  Google Scholar 

  11. Cuturi LF, Aggius-Vella E, Campus C, Parmiggiani A, Goria M (2016) From science to technology: orientation and mobility in blind children and adults. Neurosci Biobehav Rev 71:240–251

    Article  Google Scholar 

  12. Dabrowski J, Munson EV (2011) 40 Years of searching for the best computer system response time. Interact Comput 23:555–564

    Article  Google Scholar 

  13. Damaschini R, Legras R, Leroux R, Farcy R (2005) Electronic travel aid for blind people. Assistive Technol Virtual Realit 16(1):251–255

    Google Scholar 

  14. Farcy R, Leroux R, Damaschini R, Legras R, Bellik Y, Jacquet C, Pardo P (2003) Laser telemetry to improve the mobility of blind people: report of the 6 month training course. In: Proc. 1st International conference on smart homes and health telematics12, pp 113–115

  15. Friberg J, Gärdenfors D (2004) Audio games: new perspectives on game audio. In: Proc. Int. ACM conference on advances in computer entertainment technology, pp 148–154

  16. Ghali NI, Soluiman O, El-Bendary N, Nassef TM, Ahmed SA, Elbarawy YM, Hassanien AE (2012) Virtual reality technology for blind and visual impaired people: reviews and recent advances. In: Gulrez T, Hassanien A E (eds) Advances in robotics and virtual reality, ISRL 26, pp 363–385

  17. Gori M, Cappagli G, Tonelli A, Baud-Bovy G, Finocchietti S (2016) Devices for visually impaired people: High technological devices with low user acceptance and no adaptability for children. Neurosci Biobehav Rev 69:79–88

    Article  Google Scholar 

  18. Gutschmidt R, Schiewe M, Zinke F, Jürgensen H (2010) Haptic emulation of games: haptic Sudoku for the blind. In: Proc. Int. Conf. on pervasive technologies related to assistive environments. Article 2. ACM

  19. Hakobyan L, Lumsden J, OSullivan D, Bartlett H (2013) Mobile assistive technologies for the visually impaired. Surv Opthalmol 58(6):513–528

    Article  Google Scholar 

  20. Haptics rendering and applications Edited by Dr. Abdulmotaleb El Saddik, 978-953- 307-897-7 (2012)

  21. Heuten W, Henze N, Boll S, Klante P (2007) Auditorypong, playing pong in the dark. In: Audio mostly- proc. conference on interaction with sound

  22. Hossain E, Khan R, Muhida R, Ali A (2011) State of the art review on walking support system for visually impaired people. Int J Biomechatron Biomed Robot 1(4):232–251

    Article  Google Scholar 

  23. Kajimoto H, Inami M, Kawakami N, Tachi S (2003) Smart Touch-augmentation of skin sensation with electro cutaneous display, haptic interfaces for virtual environment and teleoperator systems. HAPTICS 2003. In: Proc. Symposium on IEEE, pp 40–46

  24. Kay L (2000) Auditory perception of objects by blind persons, using a bioacoustic high resolution air sonar. J Acoust Soc Am 107:3266–3275

    Article  Google Scholar 

  25. Kim J, Ricaurte J (2011) TapBeats: accessible and mobile casual gaming. In: Proc. ACM Conference on computers and accessibility. ACM, pp 285–286

  26. Kim S, Lee K, Nam T (2016) Sonic-badminton: audio-augmented badminton game for blind people. In: Proc. Conference on human factors in computing systems. ACM, pp 1922–1929

  27. Lacey G, Dawson-Howe KM, Vernon D (1995) Personal autonomous mobility aid for the frail and elderly blind (Technical Report, No. TCD-CS-95-18). Trinity College, Dublin

    Google Scholar 

  28. Leonard R (2002) Statistics on vision impairment: a resource manual, 5th edn. Arlene Gordon Research Institute of Lighthouse International, New York

    Google Scholar 

  29. Liu Y, Zhang X, Cui J (2010) Visual analysis of child-adult interactive behaviors in video sequences. Int Conf Virt Syst Multimed

  30. Liu Y, Cui J, Zhao H, Zha H (2012) Fusion of low-and high-dimensional approaches by trackers sampling for generic human motion tracking. Int Conf Pattern Recogn 898–901

  31. Liu Y, Nie L, Han L, Zhang L, Rosenblum DS (2015) Action2Activity: recognizing complex activities from sensor data. In: Yang Q, Wooldridge M (eds) Proc. int. conference on artificial intelligence. AAAI Press, pp 1617–1623

  32. Liu L, Cheng L, Liu Y, Jia Y, Rosenblum DS (2016) Recognizing complex activities by a probabilistic interval-based model. Int Conf Artific Intell, 1266–1272

  33. Liu Y, Zhang L, Nie L, Yan Y, Rosenblum DS (2016) tFortune teller: predicting your career path. Int Conf Artif Intell, 201–207

  34. Liu Y, Nie L, Liu L, Rosenblum D S (2016) From action to activity: sensor-based activity recognition. Neurocomputing 181(2016):108–115

    Article  Google Scholar 

  35. Lu Y, Wei Y, Liu L, Zhong J, Sun L, Liu Y (2017) Towards unsupervised physical activity recognition using smartphone accelerometers. Multimed Tools 76(8):10701–10719

    Article  Google Scholar 

  36. Maidenbaum S, Levy-Tzedek S, Chebat D, NamerFurstenberg R, Amedi A (2014) The effect of expanded sensory range via the eyecane sensory substitution device on the characteristics of visionless virtual navigation,MSR. ISSN 2213–4794

  37. Manduchi R, Coughlan JM (2012) Computer vision without sight. Commun ACM 55(1):96–104

    Article  Google Scholar 

  38. Manduchi R, Kurniawan S (2011) Mobility-related accidents experienced by people with visual impairment. Res Pract Vis Impairment Blindness 4(2):44–54

    Google Scholar 

  39. Manduchi R, Kurniawan S (2012) Assistive technology for blindness and low vision. CRC Press, ISBN 9781439871539

  40. Magnusson C, Rassmus-Grhn K, Sjstrm C, Danielsson H (2002) Navigation and recognition in complex haptic virtual environments? Reports from an extensive study with blind users? In: Proc. Eurohaptics 2002. Edinburgh

  41. Mau S, Melchior NA, Makatchev M, Steinfeld A (2008) BlindAid: an electronic travel aid for the blind (Technical Report, No. CMU-RI-TR-07-39). The Robotics Institute at Carnegie Mellon University, Pittsburgh

    Google Scholar 

  42. Miller RB (1968) Response time in man-computer conversational transactions. Proc AFIPS Fall Joint Comput Conf 33:267–277

    Google Scholar 

  43. Miller D, Parecki A, Douglas S (2007) Finger dance: a sound game for blind people. In: Proc. Int. ACM conference on computers and accessibility, pp 253–254

  44. Milne L, Bennett C, Ladner R, Azenkot S (2014) Brailleplay: educational smartphone games for blind children. In: Proc. Int. conference on computers and accessibility, pp 137–144

  45. Morelli T, Foley J, Columna L, Lieberman L, Folmer E (2010) VI-Tennis a vibrotactile / audio exergame for players who are visually impaired categories and subject descriptors. In: Proc. Int. ACM conference on the foundations of digital games, pp 147–154

  46. Morelli T, Foley J, Folmer E (2010) Vi-bowling: a tactile spatial exergame for individuals with visual impairments. In: Proc. International conference on computers and accessibility, pp 179–186

  47. Nagarajan R, Yaacob S, Sainarayanan G (2003) Role of object identification in sonification system for visually impaired. In: IEEE Tencon (IEEE region 10 conference on convergent technologies for the AsiaPacific). Bangalore, pp 15–17

  48. Neilson J (1993) Usability engineering

  49. Nikolakis G, Tzovaras D, Moustakidis S, Strintzis M (2004) Cybergrasp and phantom integration: Enhanced haptic access for visually impaired users. Conf Speech Comput 20–22

  50. Papa Sangre (2013) http://www.papasangre.com/

  51. Rempel J (2012) Glasses that alert travelers to objects through vibration? An evaluation of iGlasses by RNIB and AmbuTech. AFB Access World Mag 13:9

    Google Scholar 

  52. Revuelta P, Ruiz B, Sánchez JM, Bruce N (2014) Scenes and images into sounds: a taxonomy of image sonification methods for mobility applications. J Audio Eng Soc 62(3):161–171

    Article  Google Scholar 

  53. Rodriguez A, Garcia RJ, Garcia JM, Magdics M, Sbert M (2013) Implementation of a videogame: legends of girona actas del primer simposio espaol de entretenimiento digital. In: Gonzlez P A, Gmez, M A (eds) pp 96–107

  54. Rodriguez-Sancheza MC, Moreno-Alvareza MA, Martin E, Borromeoa S, Hernandez-Tamamesa JA (2014) Accessible smartphones for blind users: a case study for a way finding system. Expert Syst Appl 41(16):7210–7222

    Article  Google Scholar 

  55. Roentgen UR, Gelderblom GJ, Soede M, de Witte L (2008) Inventory of electronic mobility aids for persons with visual impairments: a literature review. J Vis Impairment Blindness 102(11):702– 724

    Google Scholar 

  56. Sakhardande J, Pattanayak P, Bhowmick M (1163) Arduino based mobility caneinternational. J Sci Eng Res 4:4

    Google Scholar 

  57. Sánchez J, Sáenz M, Ripoll M (2009) Usability of a multimodal videogame to improve navigation skills for blind children. In: Proc. ACM Computers and accessibility, pp 35–42

  58. Sánchez J, Sáenz J, Garrido JM (2010) Usability of a multimodal video game to improve navigation skills for blind children. TACCESS 3:2010

    Article  Google Scholar 

  59. Sánchez J, Campos M, Espinoza M, Merabet LB (2014) Audio haptic video gaming for developing way finding skills in learners who are blind. In: Proc. International conference on intelligent user interfaces. ACM, pp 199–208

  60. Savidis A, Stamou A, Stephanidis C (2007) An accessible multimodal pong game space. Universal Access Ambient Intell Environ 405–418

  61. Shoval S, Borenstein J, Koren Y (1998) The Navbelt-A computerized travel aid for the blind based on mobile robotics technology. IEEE Trans Biomed Eng 45:1376–1386

    Article  Google Scholar 

  62. Sohl-Dickstein J, Teng S, Gaub BM, Rodgers CC, Li C, DeWeese MR, Harper NS (2015) A device for human ultrasonic echolocation. IEEE Trans Biomed Eng 62:1526–1534

    Article  Google Scholar 

  63. Sudhanthiradevi M, Suganya Devi M, Roshini R, Sathya T (2016) Arduino based walking stick for visually impaired. Int J Adv Res Trends Eng Technol 5:4

    Google Scholar 

  64. Sung-Yeon K, Kwangsu C (2013) Usability and design guidelines of smart canes for users with visual impairments. Int J Des 7:1

    Google Scholar 

  65. Torres-Gil MA, Casanova-Gonzalez O, Gonzalez-Mora JL (2010) Applications of virtual reality for visually impaired people. WSEAS Trans Comput 2(9):184–193

    Google Scholar 

  66. Tzovaras D, Moustakas K, Nikolakis G, Strinzis M (2009) Interactive mixed reality white cane simulation for the training of the blind and the visually impaired. Person Ubiq Comput 13(1):51–58

    Article  Google Scholar 

  67. Velzquez R (2010) Wearable assistive devices for the blind. Wearable and autonomous biomedical devices and systems for smart environment. Lect Notes Electr Eng 75:331–349

    Article  Google Scholar 

  68. Vorderer P, Bryant J (2006) Playing video games-motives, responses and consequences, Mahwab, NJ:LEA, ISBN: 978-0805853223

  69. Wood J, Magennis M, Cano Arias EF, Gutierrez T, Graupp H, Bergamasco M (2003) The design and evaluation of a computer game for the blind in the GRAB haptic audio virtual environment eurohaptics 2003. Dublin

  70. www.theesa.com/. Accessed on May 2017

  71. Yuan B, Folmer E (2008) Blind hero: enabling guitar hero for the visually impaired. In: Proc. International ACM conference on computers and accessibility, pp 169–176

  72. Yuan B, Folmer E, Harris FC (2011) Game accessibility: a survey. Univers Access Inf Soc 10(1):81–100

    Article  Google Scholar 

  73. Zhao S, Yao H, Gao Y, Ji R, Xie W, Jiang X, Chua TS (2016) Predicting personalized emotion perceptions of social images. Proc ACM Multimed Conf 1385–1394

Download references

Acknowledgments

This work was supported by the Catalan Government (Grant No. 2014-SGR-1232) and by the Spanish Government (Grant No. TIN2016-75866-C3-3-R).

Author information

Authors and Affiliations

  1. Graphics and Imaging Laboratory, University of Girona, Girona, Spain

    Antonio Rodríguez, Imma Boada & Mateu Sbert

  2. Department of Computer Science and Technology, Tianjin, China

    Mateu Sbert

Authors
  1. Antonio Rodríguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Imma Boada
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mateu Sbert
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Imma Boada.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rodríguez, A., Boada, I. & Sbert, M. An Arduino-based device for visually impaired people to play videogames. Multimed Tools Appl 77, 19591–19613 (2018). https://doi.org/10.1007/s11042-017-5415-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5415-1

Keywords

Search

Navigation