We present research leading toward an understanding of the optimal audio-visual representation for illustrating concepts for illiterate and semi-literate users of computers. In our user study, which to our knowledge is the first of its kind, we presented to 200 illiterate subjects each of 13 different health symptoms in one representation randomly selected among the following ten: text, static drawings, staticphotographs, hand-drawn animations, and video, each with and without voice annotation. The goal was to see how comprehensible these representation types were for an illiterate audience. We used a methodology for generating each of the representations tested in a way that fairly stacks one representational type against the others.
Our main results are that (1) voice annotation generally helps in speed of comprehension, but bimodal audio-visual information can be confusing for the target population; (2) richer information is not necessarily better understood overall; (3) the relative value of dynamic imagery versus static imagery depends on various factors. Analysis of these statistically significant results and additional detailed results are also provided.
References
[1]
Bery, S. National Council of Applied Economic Research, 2006. http://www.rediff.com/money/2004/mar/09shining.htm
Blattner, M. M., Sumikawa, D. A. and Greenberg, R. M. Earcons and icons: Their structure and common design principles. Human-Computer Interaction, 1989. 4, 11--44.
Bolek, J. and Cowgill, J. Development of a symbol system for use in the health care industry. Proceedings of the 2005 Conference on Designing for User eXperience, November 2005.
Byrne, M. D., Catrambone, R. and Stasko, J. T. Evaluating animations as student aids in learning computer algorithms. Computers & Education, 33, 253--278. (1999).
Frost, J., and Smith, B. K. Focusing on user-to-product relationships: Visualizing health: imagery in diabetes education, Proceedings of the 2003 conference on Designing for User Experiences. June 2003
Grisedale, S., Graves, M and Grünsteidl, A. Designing a graphical user interface for healthcare workers in rural India, Proceedings of the SIGCHI conference on Human factors in computing systems, p. 471--478, March 22-27, 1997, Atlanta, Georgia, United States
Huenerfauth, M. Developing design recommendations for computer interfaces accessible to illiterate users. Master's thesis, University College Dublin, (2002).
Khai, N. Truong, Gillian R. Hayes, Gregory D. Abowd Interaction design methods 1: Storyboarding: an empirical determination of best practices and effective guidelines June 2006 Proceedings of the 6th ACM conference on Designing Interactive systems DIS '06
Kieras, D. E. Diagrammatic displays for engineered systems: effects on human performance in interacting with malfunctioning systems. International Journal of Man-Machine Studies, 36, 861--895. (1992).
Large, A., Beheshti, J., Breuleux, A. & Renaud, A. The effect of animation in enhancing descriptive and procedural texts in a multimedia learning environment. Journal of the American Society for Information Science, 47, 437--448. 1996
Marcus, A. Smilonich, N. and Thompson, L. The cross-GUI handbook for multiplatform user interface design, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1994
Mitra. S. Self organizing systems for mass computer literacy: Findings from the hole in the wall experiments. International Journal of Development Issues, Vol. 4, No. 1 (2005) 71-81.
Medhi, I., Sagar A., and Toyama K. Text-Free User Interfaces for Illiterate and Semi-Literate Users. International Conference on Information and Communication Technologies and Development (Berkeley, USA), May 2006.
Morrison, J. B. and Tversky, B. The (in) effectiveness of animation in instruction. In J. Jacko & A. Sears, Eds. Extended Abstracts of the ACM Conference on Human Factors in Computing Systems, pp. 377--378. Seattle: ACM. (2001).
Nathan, M. J., Kintsch, W. & Young, E. A theory of algebra-word-problem comprehension and its implications for the design of learning environments. Cognition and Instruction, 9, 329--389. (1992).
Palmiter, S. L., Elkerton, J. & Baggett, P. Animated demonstrations vs. written instructions for learning procedural tasks: a preliminary investigation. International Journal of Man-Machine Studies, 34, 687--701. (1991).
Parikh, T. Ghosh K. and Chavan, A. Design Considerations for a Financial Management System for Rural, Semi-literate Users. ACM Conference on Computer-Human Interaction, (2003).
Parikh, T. Ghosh K. and Chavan, A. Design Studies for a Financial Management System for Micro-credit Groups in Rural India. ACM Conference on Universal Usability, (2003).
Park, O. -C. & Gittelman, S. S. Selective use of animation and feedback in computerbased instruction. Educational Technology, Research, and Development, 40, 27--38. (1992).
Pane, J. F., Corbett, A. T. & John, B. E. Assessing dynamics in computer-based instruction. In M. J. Tauber, Ed. Proceedings of the ACM Conference on Human Factors in Computing Systems, pp. 797--804. Vancouver: ACM. (1996).
Ratnam, B. V., Reddy P. K., and Reddy, G. S. eSagu: An IT based personalized agricultural extension system prototype - analysis of 51 Farmers' case studies International Journal of Education and Development using Information and Communication Technology (IJEDICT), 2005, Vol. 2, Issue 1, pp. 79--94.
Rieber, L. P. & Hannafin, M. J. Effects of textual and animated orienting activities and practice on learning from computer-based instruction. Computers in the Schools, 5, 77--89. (1988).
Rieber, L. P. The effects of computer animated elaboration strategies and practice on factual and application learning in an elementary science lesson. Journal of Educational Computing Research, 5, 431--444. (1989).
Rieber, L. P., Boyce, M. J. & Assad, C. The effects of computer animation on adult learning and retrieval tasks. Journal of Computer-Based Instruction, 17, 46--52. (1990).
Schnotz, W. & Grzondziel, H. Individual and co-operative learning with interactive animated pictures. European Journal of Psychology of Education, 14, 245--265. (1999).
Tversky, B., Morrison, J. B. & Betrancourt, M. Animation: Can it facilitate? International Journal of Human-Computer Studies. 57(4). 2002. pp. 247--262
The DIL team, Innovative ICT Tools for Information Provision in Agricultural Extension International Conference on Information and Communication Technologies and Development (Berkeley, USA), May 2006.
Thompson, S. V. & Riding, R. J. The effect of animated diagrams on the understanding of a mathematical demonstration in 11- to 14-year-old pupils. British Journal of Educational Psychology, 60, 93--98. (1990).
Teran LMelo GSilva ISalles RRocha TMota M(2024)A Set of Professional Tools to Support the Design and Evaluation of Real-Time Payment Systems and Emergent UsersJournal on Interactive Systems10.5753/jis.2024.420715:1(790-809)Online publication date: 1-Aug-2024
Saif AZakir MRaza ANaseem M(2024)EvolveUI: User Interfaces that Evolve with User ProficiencyProceedings of the 7th ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies10.1145/3674829.3675078(230-237)Online publication date: 8-Jul-2024
CHI '09: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Multimodal interfaces with little or no text have been shown to be useful for users with low literacy. However, this research has not differentiated between the needs of the fully illiterate and semiliterate - those who have basic literacy but cannot ...
MS '15: Proceedings of the 2015 IEEE International Conference on Mobile Services
Existing mobile money platforms have text based interfaces and target literate people. Illiterate people, without the assistance of literate individuals, cannot use such platforms. Applying user-centered requirements gathered in an Ethiopian context, ...
Many recent studies show that Augmented Reality (AR) and Automatic Speech Recognition (ASR) technologies can be used to help people with disabilities. Many of these studies have been performed only in their specialized field. Audio-Visual Speech ...
How does one design user interfaces for an illiterate or semiliterate population__?__ How can computing technology be used to support activities for an unlettered population who needs help, but who doesn't know how to use a computer with a standard user interface__?__ This was the challenge presented to the authors of this paper.
The authors narrowed their task to learning how health information could be collected in a region where the ratio of health professionals to patients is very low, and the work of collecting health information is very time consuming. Their decision was to design a system that would allow patients to fill out forms themselves. Subsequently, the forms would be saved and distributed to the appropriate medical practitioners and public health facilities.
Although considerable research has been done on designing user interfaces for illiterate and semiliterate populations using visuals accompanied by voice, the authors found very few that described optimal representation formats for nonreaders. Their goal was to determine from among five different kinds of representations, each with or without voice annotation, which was the most easily understood by the potential user.
The study was conducted with 200 illiterate or semiliterate subjects and a set number of predefined visual representations. Working with a government-run hospital and a nongovernment organization (NGO) in Bangalore, India, subjects were chosen from five urban slum communities. Since most men worked outside the area, the subjects were mostly women between the ages of 25 and 55, who were either semiliterate or illiterate. Interestingly, all were comfortable reading and writing numbers and doing arithmetic. Subjects were presented with several different health-related problems, which they would identify by choosing one of the visual representations.
The researchers tested visual representations of common symptoms, such as headache, sore throat, skin infection, and foot swelling. To ensure that these depictions were subject appropriate, the authors worked with a design group made up of members of the same community. The method the researchers chose for designating the best visuals was the use of paper dolls whose limbs could be manipulated. Using the dolls, the design group was asked to act out the symptoms silently, using gestures and other visual indicators. The results were videotaped. In addition, designers verbally asked how symptoms might best be depicted in order to elicit more information. Responses across the design group were consistent.
Five representational types were tested, each with and without audio: static drawing, photo, animation, video, and text. The results showed that in most cases a static hand-drawn representation worked best to portray the symptoms. In others, such as for a swollen foot, a time-based animation was preferred. In cases of skin disease, a photograph best showed the color and texture relating to the ailment. In all cases, text accompanying the visual was preferred to a visual without text. Based on the results of the study, the authors came to three conclusions. First, though voice annotation could accelerate understanding, dual communication channels were confusing to some members of the population. Second, "richer" information, as in photography, was not necessarily better since it may contain confusing extraneous features. Third, whether the use of video or animation is more effective than using static imagery depends on the nature of the symptom being portrayed. Results were shown in a matrix chart that listed symptoms for several ailments along with the five visual representations used for each (static drawing, photo, animation, video, and text). The paper includes a very extensive bibliography.
A significant part of the global population depends on information that can best be delivered via computer. However, many populations cannot read or are semiliterate. User interface designers will find that the research methods described in this well-thought-out paper can add significantly to their ability to design for these groups.
Online Computing Reviews Service
Access critical reviews of Computing literature here
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Teran LMelo GSilva ISalles RRocha TMota M(2024)A Set of Professional Tools to Support the Design and Evaluation of Real-Time Payment Systems and Emergent UsersJournal on Interactive Systems10.5753/jis.2024.420715:1(790-809)Online publication date: 1-Aug-2024
Saif AZakir MRaza ANaseem M(2024)EvolveUI: User Interfaces that Evolve with User ProficiencyProceedings of the 7th ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies10.1145/3674829.3675078(230-237)Online publication date: 8-Jul-2024
Islam MKhan NInan TSarker I(2023)Designing User Interfaces for Illiterate and Semi-Literate Users: A Systematic Review and Future Research AgendaSage Open10.1177/2158244023117274113:2Online publication date: 16-May-2023
Teran LMota M(2023)Design Considerations for Real-Time Payment Systems and Emerging UsersProceedings of the XXII Brazilian Symposium on Human Factors in Computing Systems10.1145/3638067.3638107(1-11)Online publication date: 16-Oct-2023
Han MGautam A(2023)A Case For Supporting Translanguaging in TechnologyCompanion Publication of the 2023 ACM Designing Interactive Systems Conference10.1145/3563703.3596627(228-231)Online publication date: 10-Jul-2023
Rayed MElahi TArefin Shimon SAhmed N(2023)MFS Design in Appstore-enabled Smart Featurephones for Low-literate, Marginalized CommunitiesProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580661(1-19)Online publication date: 19-Apr-2023
Georgiou TBaillie LNetto GPaterson S(2023)Investigating Technology Concepts to Support Rohingya Refugees in MalaysiaInteracting with Computers10.1093/iwc/iwad02635:3(481-509)Online publication date: 9-Mar-2023
Kaliprasad SBose SJithin Gangadharan KNagpal RSingh PGangwar V(2023)Utilizing Artificial Intelligence to Successfully Communicate and Equip Real Estate Construction Workers with Ergonomic Footwear for Their HealthRenewable Energy Optimization, Planning and Control10.1007/978-981-19-8963-6_36(399-409)Online publication date: 8-Mar-2023
da Silva TFurtado VFurtado EMendes MAlmeida VSales L(2022)How Do Illiterate People Interact with an Intelligent Voice Assistant?International Journal of Human–Computer Interaction10.1080/10447318.2022.212121940:3(584-602)Online publication date: 22-Sep-2022