Skip to main content
SpringerLink
Log in

Picture superiority effect in authentication systems for the blind and visually impaired on a smartphone platform

  • Long Paper
  • Published:
Universal Access in the Information Society Aims and scope Submit manuscript

Abstract

Pictures are more likely to be remembered than words or text. For smartphone authentication, graphical password interfaces employing both visual objects and auditory cues are more memorable than textual password interfaces among sighted people because the graphical interface evokes visual imagery in the brain. However, interfaces employing visual imagery have not been studied for the blind and visually impaired. The objective of this research is to demonstrate that graphical password interfaces, designed to evoke visual imagery among blind and visually impaired users, improve the ease of use of smartphone authentication systems. We developed and tested two graphical password systems, BlindLoginV2, which employs object picture superiority effect and AudioBlindLogin, which employs auditory cues to enrich the picture superiority effect. We collected quantitative metrics measuring login speed, configuration time and failure rates immediately after training, 1 h later, 1 day later and 1 week later and qualitative evidence through face-to-face interviews. This study shows that blind and visually impaired users benefit from the picture superiority effect and passwords are more memorable, quicker to key in with greater accuracy as compared to 4-character textual password interfaces. Using the authentication system as an example, we demonstrate that visual imagery can be evoked in blind and visually impaired users through careful design of smartphone interfaces and when paired with additional sensory cues such as audio, can significantly improve the ease-of-use and thereby enhance access among visually impaired users to the rich array of security features available in smartphones.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Khan, A., Khusro, S.: An insight into smartphone-based assistive solutions for visually impaired and blind people – issues, challenges and opportunities. Univ. Access Inf. Soc. 19(3), 1–25 (2021)

    Google Scholar 

  2. Azenkot, S.: Eyes-free input on mobile devices. ProQuest Dissertations and Theses, 148. (2014). https://digital.lib.washington.edu/researchworks/bitstream/handle/1773/27220/Azenkot_washington_0250E_13766.pdf?sequence=1

  3. Alsuhibany, S.A.: Vibration-based pattern password approach for visually impaired people. Comput. Syst. Sci. Eng. 40(1), 341–356 (2022). https://doi.org/10.32604/CSSE.2022.018563

    Article  Google Scholar 

  4. Al-Razgan, M., Almoaiqel, S., Alrajhi, N., Alhumegani, A., Alshehri, A., Alnefaie, B., AlKhamiss, R., Rushdi, S.: A systematic literature review on the usability of mobile applications for visually impaired users. PeerJ Comput. Sci. 7, e771 (2021). https://doi.org/10.7717/PEERJ-CS.771

    Article  PubMed  PubMed Central  Google Scholar 

  5. Dosono, B., Hayes, J., Wang, Y.: “ I’m stuck !”: A contextual inquiry of people with visual impairments in authentication. 151–168 (2015)

  6. Khan, A., Khusro, S., Alam, I.: BlindSense: an accessibility-inclusive universal user interface for blind people engineering. Technol Appl Sci Res 8(2), 2775–2784 (2018). https://doi.org/10.48084/etasr.1895

    Article  Google Scholar 

  7. Belanger, K:. Type in Braille with the TalkBack Braille Keyboard for your android Device (2020) https://nfb.org/blog/type-braille-talkback-braille-keyboard-your-android-device

  8. Olofsson, S.: Designing interfaces for the visually impaired: contextual information and analysis of user needs. Master’s Thesis, Ume˚a University (2017)

  9. Alnfiai, M. Accessible Tools on Touchscreen Devices FOR. June (2018)

  10. Raz, N., Striem, E., Pundak, G., Orlov, T., Zohary, E.: superior serial memory in the blind: a case of cognitive compensatory adjustment, current biology, 17(13). ISSN 1129–1133, 0960–9822 (2007). https://doi.org/10.1016/j.cub.2007.05.060

    Article  CAS  Google Scholar 

  11. Kamel, H.M., Landay, J.A.: A study of blind drawing practice. In: Proceedings of the Fourth International ACM Conference on Assistive Technologies - Assets ’00, 34–41 (2002). https://doi.org/10.1145/354324.354334

  12. Davis, D., Monrose, F., & Reiter, M.: On user choice in graphical password schemes. In Proceedings of the 13th Conference on USENIX Security Symposium-Volume 13, 11. (2004). https://doi.org/10.1016/j.apenergy.2011.11.075

  13. Gloede, M.E., Paulauskas, E.E., Gregg, M.K.: Experience and information loss in auditory and visual memory. 70(7), 1344–1352 (2017). https://doi.org/10.1080/17470218.2016. 1183686

  14. Kaski, D.: Revision:is visual perception a requisite for visual imagery? Perception 31(6), 717–731 (2002)

    Article  PubMed  Google Scholar 

  15. van den Hurk, J., Van Baelen, M., Op de Beeck, H.P.: Development of visual category selectivity in ventral visual cortex does not require visual experience. Proc. Natl Acad. Sci. 114(22), 4501–4510 (2017). https://doi.org/10.1073/pnas.1612862114

    Article  CAS  Google Scholar 

  16. Bértolo, H. Visual imagery without visual perception? 173–188 (2005)

  17. Gehring, R.E., Toglia, M.P., Kimble, G.A.: Recognition memory for words and pictures at short and long retention intervals. Mem. Cognit. 4(3), 256–260 (1976). https://doi.org/10.3758/BF03213172

    Article  CAS  PubMed  Google Scholar 

  18. De Angeli, A., Coventry, L., Johnson, G., Renaud, K.: Is a picture really worth a thousand words? exploring the feasibility of graphical authentication systems. Int. J. Hum. Comput. Stud. 63(1–2), 128–152 (2005). https://doi.org/10.1016/j.ijhcs.2005.04.020

    Article  Google Scholar 

  19. Anne, M., Russo, R., Louise, P.: The picture superiority effect in recognition memory: a developmental study using the response signal procedure. Cognit. Dev. 24(2009), 265–273 (2014). https://doi.org/10.1016/j.cogdev.2009.05.002

    Article  Google Scholar 

  20. Crutcher, R.J., Beer, J.M.: An auditory analog of the picture superiority effect. Mem. Cognit. (2011). https://doi.org/10.3758/s13421-010-0015-6

    Article  PubMed  Google Scholar 

  21. Defeyter, M.A., Russo, R., McPartlin, P.L.: The picture superiority effect in recognition memory: a developmental study using the response signal procedure. Cognit. Dev. 24(3), 265–273 (2009). https://doi.org/10.1016/j.cogdev.2009.05.002

    Article  Google Scholar 

  22. Oates, J. M., Reder, L. M. (2011). Memory for pictures: sometimes a picture is not worth a single word. Successful Remembering and Successful Forgetting: A Festschrift in Honor of Robert A. Bjork, 447–462. https://doi.org/10.4324/9780203842539

  23. Lopes da Silva, F.H.: Visual dreams in the congenitally blind ? Trends Cognit. Sci. 7(8), 328–330 (2003). https://doi.org/10.1016/S1364-6613(03)00163-3

    Article  Google Scholar 

  24. Paivio, A., Rogers, T.B., Smythe, P.C.: Why are pictures easier to recall than words? Psychon. Sci. 11(4), 137–138 (1968). https://doi.org/10.3758/BF03331011

    Article  Google Scholar 

  25. Thorpe, J., Van Oorschot, P.C.: (2004). Towards secure design choices for implementing graphical passwords. In: Proceedings—Annual Computer Security Applications Conference, ACSAC, 50–60. https://doi.org/10.1109/CSAC.2004.44

  26. Suo, X.: A design and analysis of graphical password. Inf. Control (2006). https://doi.org/10.1109/ICICIC.2009.158

    Article  Google Scholar 

  27. Lin, P.L., Weng, L.T., Huang, P.W.: Graphical passwords using images with random tracks of geometric shapes. In: Proceedings—1st International Congress on Image and Signal Processing, CISP, 3, 27–31. (2008). https://doi.org/10.1109/CISP.2008.603

  28. Khan, W.Z., Aalsalem, M.Y., Xiang, Y.: A graphical password based system for small mobile devices. Int. J. Comput. Sci. Issues 8(5), 145–154 (2011). https://doi.org/10.1145/2078827.2078835

    Article  Google Scholar 

  29. Ray, P.P.: Ray’s scheme: graphical password based hybrid authentication system for smart hand held devices. J. Inf. Eng. Appl. 2(2), 1–12 (2012)

    Google Scholar 

  30. Stobert, E., Biddle, R.: Memory retrieval and graphical passwords. In: Proceedings of the Ninth Symposium on Usable Privacy and Security - SOUPS ’13, 1. (2013). https://doi.org/10.1145/2501604.2501619

  31. Bértolo, H.: Visual imagery without visual perception?: Lessons from blind subjects. Second International Conference on Applications of Optics and Photonic, 9286l (2014). https://doi.org/10.1117/12.2066101

  32. Said, K., Kuber, R., Murphy, E.: AudioAuth: exploring the design and usability of a sound-based authentication system. Int. J. Mobile Hum. Comput. Interact. 7(4), 16–34 (2015)

    Article  Google Scholar 

  33. Zahid, S., Shahzad, M., Khayam, S.A., Farooq, M.: Keystroke-based user identification on smart phones. In: International Workshop on Recent Advances in Intrusion Detection (RAID 2009), pp. 224–243. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  34. Kang, J.: Procedia computer science mobile iris recognition systems : an emerging biometric technology. Procedia Comput. Sci. 1(1), 475–484 (2012). https://doi.org/10.1016/j.procs.2010.04.051

    Article  MathSciNet  Google Scholar 

  35. Haque, M. M., Zawoad, S., Hasan, R. Secure techniques and methods for authenticating visually impaired mobile phone users. In: IEEE International Conference on Technologies for Homeland Security, HST 2013, 735–740 (2013). https://doi.org/10.1109/THS.2013.6699095

  36. Esther, R.D., Jose, J.J.R.: Fingerprint based biometric authentication. IJCSMC 5(9), 6–15 (2016)

    Google Scholar 

  37. von Graevenitz, G.A.: About speaker recognition techology. Bergdata Biometrics GmbH, Bonn, Germany. (2008)

  38. Yuan, X., Li, G., Han, J., Wang, D., Tiankai, Z.: Overview of the development of speaker recognition. J. Phys. Conf. Ser. 1827(1), 012125 (2021). https://doi.org/10.1088/1742-6596/1827/1/012125

    Article  Google Scholar 

  39. Brunet, K., Taam, K., Cherrier, E., Faye, N., Rosenberger, C.: Speaker recognition for mobile user authentication: an android solution. In: 8ème Conférence Sur La Sécurité Des Architectures Réseaux et Systèmes d’Information (SAR SSI), September, 10 (2013)

  40. Chen, Y., Sun, J., Jin, X., Li, T., Zhang, R., Zhang, Y.: Your face your heart : secure mobile face authentication with photoplethysmograms. In: IEEE Conference on Computer Communications (INFOCOM 2017). 1–9 (2017)

  41. Chiasson, S., Forget, A., Biddle, R.: Accessibility and graphical passwords. symposium on accessible privacy and security (SOAPS). (2008)

  42. Wobbrock, J.O.: TapSongs : tapping rhythm-based passwords on a single binary sensor. UIST’09, 93–96 (2009)

  43. Rathanavel, V., Mali, S.: Graphical password as an OTP. Int. J. Eng. Comput. Sci. 6(1), 1–6 (2017). https://doi.org/10.18535/ijecs/v6i1.41

    Article  Google Scholar 

  44. Kuber, R., Sharma, S.: Toward tactile authentication for blind users. In: International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS’10), 289-290 (2010)

  45. Kuber, R. , Sharma, S Developing an extension to an existing tactile authentication mechanism to support non-visual interaction. In: IASTED Conference on Human-Computer Interaction, 190–198 (2012).

  46. Sun, J., Zhang, R., Zhang, J., and Zhang, Y.: TouchIn : sightless two-factor authentication on multi-touch mobile devices. In: IEEE Conference on Communications and Network Security, 436–444 (2014)

  47. Marques, D., Guerreiro, T., Duarte, L., Carrico, L.: Under the table: tap authentication for smartphones. BCS HCI (2013). https://doi.org/10.14236/EWIC/HCI2013.42

    Article  Google Scholar 

  48. Chen, Y., Sun, J., Zhang, R., Zhang, Y.: Your song your way : rhythm-based two-factor authentication for multi-touch mobile devices. 2686–2694 (2015)

  49. Gurary, J.: Multi-dimensional and analog authentication (2018)

  50. Qinghai, G.: A Preliminary study of fake fingerprints. Int. J. Comput. Netw. Inf. Secur. 6(12), 1–8 (2014). https://doi.org/10.5815/ijcnis.2014.12.01

    Article  Google Scholar 

  51. Vaughan, C.E., Schroeder, F.K.: Social and cultural perspectives on blindness: barriers to community integration (2nd Ed.). Charles C Thomas, 93 (2018)

  52. Mcgookin, D., Brewster, S., Jiang, W.: Investigating touchscreen accessibility for people with visual impairments. NordiCHI, 298–307 (2008)

  53. Ahmed, T. Privacy concerns and behaviors of people with visual impairments (2015)

  54. Wobbrock, J.O., Findlater, L., Gergle, D., Higgins, J.J.: The aligned rank transform for nonparametric factorial analyses using only anova procedures. In: Conference on Human Factors in Computing Systems—Proceedings, May 2018, 143–146 (2011). https://doi.org/10.1145/1978942.1978963

  55. Oh, U.: Accessible on-body interaction for people with visual impairments. PhD Thesis. University of Maryland, College Park. (2016)

  56. Bonner, M., Brudvik, J., Abowd, G., Keith Edwards, W.: No-look notes: accessible eyes-free multi-touch text entry. (2010). http://matt-bonner.net/pdf/papers/pervasive2010-nolooknotes.pdf

  57. Maetz, Y., Heen, O., Irisa, I. Story graphical password system. 15–16 (2009)

  58. Cowan, N. The magical number 4 in short-term memory : a reconsideration of mental storage capacity. 4, 87–185 (2000)

  59. Mandler, G.: Organization and memory. In: The psychology of learning and motivation, vol. 1, ed. K. W. Spence & J. T. Spence. Academic Press. (1967)

  60. Dubey, R., Peterson, J., Khosla, A., Yang, M., Ghanem, B. What makes an object memorable ? (2015)

  61. Ma, W.J., Husain, M., Bays, P.M.: Review changing concepts of working memory. Nat. Publ. Group 17(3), 347–356 (2014). https://doi.org/10.1038/nn.3655

    Article  CAS  Google Scholar 

  62. Bouchacourt, F.: A flexible model of working memory. 1–41 (2018)

  63. Daniels A.: The memory after-image and attention. The American Journal of Psychology, 6(4) 558–564 (1895). https://www.jstor.org/stable/141119

  64. Brown, J.: Some tests of the decay theory of immediate memory. Quarterly J. Exp. Psychol 10(1), 12–21 (1958)

    Article  Google Scholar 

  65. Peterson, L.R., Peterson, M.J.: Short-term retention of individual verbal items. J. Exp. Psychol. 58(3), 193 (1959)

    Article  CAS  PubMed  Google Scholar 

  66. Muter, P.: Very rapid forgetting. Mem. Cognit. 8(2), 174–179 (1980)

    Article  CAS  PubMed  Google Scholar 

  67. Solso, R.L.: Cognitive psychology, 4th edn. Allyn and Bacon (1995)

    Google Scholar 

  68. Marsh, R.L., Sebrechts, M.M., Hicks, J.L., Landau, J.D.: Processing strategies and secondary memory in very rapid forgetting. Mem. Cognit. 25(2), 173–181 (1997)

    Article  CAS  PubMed  Google Scholar 

  69. Alt, F., Mikusz, M., Schneegass, S., Bulling, A.: Memorability of cued-recall graphical passwords with saliency masks. In: ACM International Conference Proceeding Series, December, 191–200 (2016). https://doi.org/10.1145/3012709.3012730t

  70. Baddeley, A.: Your memory: a user’s guide. Firefly Books. (2004)

  71. Tanviruzzaman, M.: Towards usable end-user authentication. PhD Dissertation. (2014). http://gradworks.umi.com/36/23/3623762.html

  72. Kalayev, E.: Logon Duration – What Can Be Learned from 2 Million Logons?. Control Up. https://www.controlup.com/resources/blog/entry/logon-duration- what-can-be-learned-from-2-million-logons/. (2016)

  73. Lawrence, D.M., Banks, W.P.: Accuracy of recognition memory for common sounds. Bull. Psychon. Soc. 1(5-A), 298–300 (1973)

    Article  Google Scholar 

  74. Grudzinska, A.: An auditory analog of the picture superiority effect in typically developing children. University of West London. (Unpublished). (2018)

  75. National federation of the blind (2019). Blindness statistics. https://www.nfb.org/resources/blindness-statistics

  76. Paivio, A., Csapo, K.: Picture superiority in free recall: imagery or dual coding? Cogn. Psychol. 5(2), 176–206 (1973). https://doi.org/10.1016/0010-0285(73)90032-7

    Article  Google Scholar 

  77. Huss, M.T., Weaver, K.A.: Effect of modality in earwitness identification: memory for verbal and nonverbal auditory stimuli presented in two contexts. J. Gen. Psychol. 123(4), 277–287 (1996)

    Article  CAS  PubMed  Google Scholar 

  78. Atkinson, R.C., Shiffrin, R.M.: Human memory: a proposed system and its control processes. In: Spence, K., Spence, J. (eds.) The Psychology of Learning and Motivation, pp. 89–195. Elseiver, Armsertdam (1968)

    Google Scholar 

  79. Braz, C.: Integrating a usable security protocol for user authentication into the requirements and design process. Dissertation (2011)

  80. Norris, D.: Short-term memory and long-term memory are still different. Psychol. Bull. 143(9), 992–1009 (2017)

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Information Science and Technology, Multimedia University, Jalan Ayer Keroh Lama, Hang Tuah Jaya 75450, Melaka, Malaysia

    Yean Li Ho & Siong Hoe Lau

  2. Taylor’s University, 1, Jalan Taylors, 47500, Subang Jaya, Selangor, Malaysia

    Afizan Azman

Authors
  1. Yean Li Ho
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Siong Hoe Lau
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Afizan Azman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yean Li Ho.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

1.1 Comments on difficulty in using textual keyboard

Difficult to type in the keyboard. It causes more error when selecting the password from the keyboard, and another problem is hard to delete the wrong password, user need to find the delete button and delete it one by one…Many errors when typing from the keyboard because hard to find the location of the key. User 6 (age 23, Blind)

It is troublesome because need to find the location of the key one by one by swiping on the keyboard. It makes it slower to enter the password.

User 4 (age 22, Blind).

The weakness is that it is difficult to find the characters. For example: If we want to find ‘wf’, it will go to ‘wd’ and other keys instead.

User 7 (age 41, Blind).

The weakness is the keyboard system and the comfort of a person using a phone because different phones have different keyboard layouts and different keyboards have different settings. Sometimes, it depends on an individual, their fingers and the keyboard.

User 10 (age 27, Visually Impaired).

Sometimes we cannot log in successfully because we need to practice using the keyboard first, it is hard for us...The keyboard is very hard, we need to be very familiar with the keyboard and know where the keyboard is.

User 1 (age 53, Blind).

(The virtual keyboard is) not suitable totally for blind user because it is hard to use. Sometime the keyboard words are big but sometimes it is small. Because the font size in the phone is too small, so it is more difficult for visually impaired user, so the user cannot know what they had entered.

(User 5, age 21, Visually Impaired).

Maybe to the youngsters yes but then to the elderly blind maybe it’s abit difficult for them to play around with the invisible keyboard because some of them their fingers may not be that sensitive.

User 15 (age 69, Visually Impaired)

But for those who are the adult blind and who are not so sensitive and maybe those who are new phone users, it’s harder for them to confirm what they have chosen is correct or not. Because after we have chosen, it does not say. For example, I chose ‘I’, it doesn’t say ‘I’, it says bullet, right? So, it’s harder for them to confirm if it is right or not.

User 16 (age 36, blind)

It’s easier for people like us because sometimes it’s hard for us to type and fingerprint also fails to detect for some of us. However, this apps seems much easier. It is easier to access your password because most of the handicapped people like us have difficulties typing or using the fingerprint because the fingerprint cannot be detected. So, this is another good way.

User 11 (age 20, Visually Impaired).

1.2 Comments on memorability of environmental sounds/sound effects

For me, the most attractive to try is AudioBlindLogin because it uses sound so we blind people find sound interesting. And its only one sound, we immediately double tap. If it is wrong, the input won’t be accepted yet. For me that is interesting

User 10 (age 27, Visually Impaired)

Easy because it is acceptable sounds for blind people. Faster to login

User 4 (age 22, Blind).

Easy because the sounds are commonly heard.

User 6 (age 23, Blind).

We can choose the sounds that we like. If we feel that the sound is weird or funny and we choose it then it’s easier for us to remember

User 16 (age 36, Blind).

The sound system is easier to remember. The sound is distinct, it’s easily distinguishable.

User 15 (age 69, Visually Impaired).

The sounds are clear and it is easier than BlindLoginV2. Its login process is also easy for the blind to remember. I like it because I like sound effects better because it is easy to remember and is more accessible to the blind.

User 8 (age 22, Blind)

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ho, Y.L., Lau, S.H. & Azman, A. Picture superiority effect in authentication systems for the blind and visually impaired on a smartphone platform. Univ Access Inf Soc 23, 179–189 (2024). https://doi.org/10.1007/s10209-022-00928-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10209-022-00928-1

Keywords

Search

Navigation