Keywords

1 Introduction

Hearing loss affects around 32 million children worldwide and the vast majority live in low and middle income countries [1]. Taking into account that the deaf community has sign language as their mother tongue (L1), it is of great importance for them to acquire a written language (L2) through the development of literacy skills [2].

For these children the learning of reading and writing is a challenge, due to their auditory deficit and teaching strategies must be different from those used with hearing children, so it has restrictions on language proficiency at lexical, syntactic and semantic level [3].

One way to address this challenge is to use a different communication method such as sign language, as an alternative to spoken language [4]. This type of language favors the teaching-learning processes of deaf children, since they will be able to develop a complete language in an early stage that will allow them to think, plan, hypothesize, etc. In addition, it can serve as a linguistic basis for the acquisition of the written language because it facilitates its learning as a second language [3, 7].

On the other hand, the technological advance in mobile devices is creating a digital paradigm shift, especially in the way we communicate. Having a mobile device has become a necessity to communicate within a world in constant movement, so you can find a growing trend of communities and mobile social networks [6]. However, there are few studies conducted on this type of communities and networks for disabled users, particularly for the deaf community in a developing country. Taking into account the above-mentioned arguments, there is a need to establish a GUI design guide to support the development of mobile applications aimed at deaf children.

2 Background

Information and communication technologies (ICT) are a great resource for people working with deaf children, because it has characteristics such as high memory capacity, visualization capabilities, sensors, as well as sophisticated artificial intelligence techniques that can be exploited to build educational tools capable of meeting the needs of deaf children in an effective way [7]. Unfortunately, ICT tools base their content not only on images but also on text, which for reasons previously mentioned hinders the use of these by deaf children who have not yet mastered their second language (L2).

Currently, the existing educational software (national or imported) has been developed by expert programmers or systems engineers who have been mainly concerned with the technical aspects neglecting almost completely the characteristics of the user and fundamentally the pedagogical and didactic aspects of the educational goal of the developed tool [8]. However, this should not be happening on these days. The interface of a well-designed software should provide a link or relationship with users, leading them during the learning process and letting them enjoy what they are doing. It is fundamental for developers to base their designs on a set of general principles and guidelines for the design of user interfaces, agreed upon by most experts in the field [9], because when a product follows the standards and design conventions, it is more likely to direct the attention of its users and achieve their objectives while at the same time it can be innovative and attractive [10].

Before the advent of mobile devices, visual language and printed text in an interactive and visually rich format was not possible, since children with hearing problems had to look at a book and then alternate between looking at a screen (television or computer) to watch the video [11]. Now, by integrating videos and texts on the same screen will greatly enhance their learning processes.

In recent years, the design of electronic tools (e-tools) for children has increased and guidelines have been proposed for the design of this type of tools. The existing guidelines differentiate children according to their age or sex, and not according to other cognitive characteristics such as the ability to understand text, visual attention and memory skills. However, these and other skills turn out to be crucial in designing usable and accessible electronic tools for deaf people. For instance, reading skills of deaf people are lagging behind compared to their hearing peers, while visual-perceptual skills of deaf people are generally considered as equal or even more developed than their hearing peers. These differences require guidelines for the design of electronic tools that are usable and accessible for deaf people in general, and for deaf children in particular [12]. That is why the need arises to carry out the present research that proposes a guide to the design of graphical user interfaces for mobile applications aimed at deaf children.

3 Related Works

Table 1 lists some studies that contain different guidelines for designing games for deaf children, methodologies used for teaching to deaf children, applications aimed at the deaf population and the process of how those applications were developed.

Table 1. Related works
Full size table

4 Design of the Guide

Designing interfaces that are easy to use for deaf children is not a simple task. Common interfaces are designed so that the use of audio helps users to interact easily with it, but when deaf users interact with these types of interfaces, they may face some difficulties and may need more time to perform tasks This is why the developers of applications for deaf children must understand their characteristics, capacities and needs when designing the interfaces, considering the general guidelines for their development and taking into account the specific guidelines and characteristics for these children [16]. During the process of research and construction of the GUI design guide for mobile applications aimed at deaf children, the process model of usability and accessibility engineering was integrated (MPIu+a) [17], which allowed a thorough research of important aspects of the deaf children population and thus form a set of guidelines that are reflected in the aforementioned design guide.

The research was carried out in collaboration with La Pamba Educational Institution in the city of Popayán (Colombia). This Institution has a basic classroom for the deaf, where support is provided in the educational process to children and young deaf people through the learning and use of their mother tongue. For specific reasons to the project, the case study consisted of 5 children (3 boys and 2 girls) from the first grades of school, aged between 7 and 14 years. The group of children are profound deaf.

When integrating the MPIu+a model, which is an iterative model, the phases used were: Software engineering (requirements and design analysis), prototyping and evaluation. This model is based on the user, and it is the user who decides whether a design or development is friendly or not. This model allowed the construction of the design guide, the prototype based on such guide and its validation.

4.1 Requirements Analysis

The bibliography referring to the design of interfaces for the construction of mobile applications for children was reviewed. Meetings were held with specialists and teachers responsible for teaching deaf children to obtain pedagogical recommendations that are applicable in the construction of mobile applications for this population. In addition, field work was done using the observation technique to identify the different activities that specialists, in therapies for deaf children, use in communication. In the same way, the abilities and difficulties of children when interacting with the environment were taken into account during the case study. In this way, different aspects were determined that may affect in one way or another the design of the interface for a deaf child. Based on the above, the guidelines that were part of the guide of design were presented and validated through a prototype.

4.2 Design + Prototype

The design and development of a mobile application based on the guidelines of the design guide in the previous phase was carried out. For this process, some of the previously identified activities were taken as a basis.

In the first two iterations, a prototype of low fidelity was built through the use of paper that allowed to have a first approximation of the design. In the following iterations, a functional prototype was developed in order to carry out a continuous improvement of the guidelines of the design guide. All this was done taking into account the opinions and recommendations of the experts and the case study who were involved throughout the process.

4.3 Evaluation + Prototype

In this phase, the guidelines of the design guide were validated through the prototype with the case study and the collaboration of experts in the education of deaf children, designers and developers of mobile applications.

In the process of construction of the prototype, the experts made a descriptive evaluation of each applied guideline, giving their opinions, recommendations and suggestions, and rated each guideline between 1 and 10 to determine the degree of importance as a guideline, being 1 the lowest rating and 10 the highest rating. The guidelines valued between 1 and 4 were not part of the design guide, those valued between 5 and 7 were reviewed and rethought for a second review and finally the guidelines valued between 8 and 10 were included in the design guide.

The prototype was tested in the case study so that it was observed and recorded the interaction and reactions that occurred during this process to later perform an analysis with the help of experts.

4.4 GUI Design Guide

Based on the results of the different evaluations, the design guide was divided into 6 categories which are style, components, patterns, learning, content and general. Due to the extension of all the guidelines, to have access to the final version of the guide, go to the website: https://www.guiaappssordos.com.

5 Development and Validation of the Design Guide

The process of construction and validation of the design guide was carried out in three iterations that are briefly described below.

In the first iteration through the review of the literature, suggestions from experts and related works, a set of guidelines were compiled, which would be the basis of the design guide. Three prototypes were also developed based on the above guidelines. The first prototype was on paper and was based on an existing application called ‘cocosigna’. The second version of the prototype, also on paper, contains the suggestions received and this second version of the paper prototype led to a first functional prototype for Android devices. It is noteworthy that the prototype activities were taken based on the pedagogical material called Programa de desarrollo de habilidades básicasFootnote 1[18] level 1 to 4. The functional prototype had 20 activities which were divided into two levels.

Among the main features of the design of the prototype are that it has a start screen, indications within the activities with videos in sign language represented by an avatar, in the same way it has the stimulation for achievements at the end of an activity and a trophy reward system. It was also added that, when finishing a task correctly, congratulate the children through the avatar.

In the second iteration the design guide was refined based on the suggestions, evaluation of the guidelines and observation of the deaf children’s interaction with the application that was made in the previous iteration and a new version of the prototype was made functional, where the main changes made were the addition of a welcome screen to attract the attention of children and generate curiosity and interest, with the aim of continuing to interact with it. Indications and labels were also added to the main elements of the activities. In addition, a context interface was added, which aims to show the word and image in a specific context so that the child makes connections between the word and its meaning. In the same way, for the different activities, visual effects were added to feed the children after performing an action.

For the first iteration, the design guide was refined based on the evaluation of guide-lines in the previous iteration and the recommendations and observation of the inter-action of the application with deaf children. Based on this guide a final prototype is built to which new levels and activities were added, thus allowing to apply most of the guidelines of the guide, in the same way videos were included in sign language recorded by a certified interpreter and a linguistic model.

The evaluation of the guide and validation through the prototype, in each one of the iterations was carried out by means of inspection techniques: cognitive walkthrough and heuristic evaluation, which were chosen taking into account the profiles of the users that would evaluate the guidelines of the guide and the prototype. These are:

  • Specialists in children with hearing disabilities.

  • Children with hearing disability.

  • Designers and software developers

5.1 Definition of Metrics

The selected metrics that allowed to objectively measure the results that were obtained from the evaluations carried out were:

Score Guidelines:

It consisted of giving a score to each of the guidelines based on the specific usability criteria taken into account to perform the heuristic evaluation, these criteria are: Identity, design, accessibility, navigation and operation. This metric is used for the heuristic evaluation.

The score for each guideline is between 1 and 10, to determine the degree of importance as a guideline, with 1 being the lowest rating and 10 the highest qualification.

The ranges proposed below are analyzed, reviewed and approved by the different experts who proposed that the first rank has greater breadth in comparison with the other ranges so that the guidelines that will be part of the design guide are filtered. The two remaining ranges are distributed equally. The ranges are posed as follows:

  • The guidelines valued between 1 and 4 will not be part of the design guide.

  • The guidelines valued between 5 and 7 will be reviewed and restated for a second evaluation.

  • The guidelines valued between 8 and 10 will be included in the design guide.

Metrics: Number of Errors - Activities Completed.

The metrics of number of errors and completed activities were selected because they are the most relevant measures when interacting with the application, since they provide important information for the validation of the usability. Therefore, completing the activities satisfactorily and a smaller number of errors can indicate the understanding of the tasks to be performed within the application. Table 2 shows the metrics with their respective description.

Table 2. Description of metrics for the evaluation of cognitive walkthrough.
Full size table

5.2 Metric Normalization

Once the measurements associated with each metric have been obtained, the values are normalized using the formula presented in the image (see Fig. 1) in order to better analyze the data. For the formula, the maximum and minimum values of the sample must be obtained and X is the measurement obtained during the evaluation associated with the metric.

Fig. 1.
figure 1

Palettes used in the evaluation smiley meter.

Full size image
$$ {\text{Z}} = \frac{{{\text{x}} - { \hbox{min} }}}{{{ \hbox{max} } - { \hbox{min} }}} $$

5.3 Smiley Meter Evaluation

For the evaluation using the Smiley Meter scale [19], 3 palettes were used indicating the degree of satisfaction of the children with the application (see Fig. 1). With the help of the teacher and the linguistic model, the children were asked how they felt when interacting with each prototype in each of the iterations, and in turn, the children explained to the teacher how they felt when interacting with each using sign language and the teacher translated the responses of the children to us.

5.4 Results of the Evaluation

Because there are two types of evaluation techniques, the result of the evaluation will be shown in each iteration of the research organized by user profile.

User Profiles: Specialists in Children with Hearing Disabilities, Designers and Developers.

Cognitive Walkthrough Analisys.

The evaluation of the different prototypes in each iteration is carried out, applying the technique of cognitive walkthrough to the specialist teacher in children with auditory disability, to the designers and developers, where they are assigned a series of tasks that must be performed interacting with the different prototypes.

After completing the tasks, the evaluators make different observations and recommendations among which it is very important that all the instructions are accompanied by images or videos in sign language, since this is the mother tongue of deaf children. In the same way, they indicated that the most important information should be highlighted. On the other hand, the expert teacher recommends that when an element of an activity is shown, the image should be used accompanied by its corresponding word and a phrase that puts the word in context, since day-to-day words are used in complete sentences (e.g. “I saw a big beautiful mountain”). Designers and developers recommend making a welcome screen to capture the child’s attention and make the application more eye-catching. The use of animals is proposed, since these attract the attention of the child.

The specialist teacher considered it necessary to change the avatar for a person to guide the children through the different activities in the videos shown in the prototype, since the gestures performed by the avatar are not natural compared to the gestures made by a person. In addition, the use of interpreters trained in Colombian sign language is recommended.

Heuristic Evaluation.

For the heuristic evaluation, we counted on the collaboration of 5 evaluators, among them experts in children with hearing disabilities, designers and developers, who are presented with a form to fill in which each of the guidelines must be qualified based on compliance with the criteria of specific usability: design, accessibility, navigation, operation.

The heuristic evaluation of the guidelines is carried out taking into account the functional prototype built in the design phase of the second iteration and analyzed during the cognitive walkthrough, since this prototype allows to see most of the guidelines applied.

With the result of the different evaluations, a comparative table is generated for each category and subcategory to average the results and thus obtain the guidelines that will be part of the design guide, review the guidelines that obtained scores between 5 and 7 and eliminate the guidelines that are not considered necessary or do not apply for the design guide, which were rated with scores between 1 and 4.

Likewise, it is suggested by some evaluators that the guide should be restructured in terms of categories and subcategories, since some guidelines are found in categories that do not correspond and there are very extensive categories that can be divided into subcategories. On the other hand, the evaluators make recommendations on the guidelines in terms of wording and the terminology used, thus avoiding technical words and drafting guidelines that are understandable for both expert designers and developers and for inexperienced people who wish to make an application for deaf children.

User Profile: Children with Hearing Impairment

Cognitive Walkthrough Analisys.

For the analysis of cognitive walkthrough tasks are assigned to deaf children with the help of the teacher and the linguistic model. Each of the deaf children performed the different activities with the functional prototype and at the same time they took note of the different difficulties they presented during the interaction with it. In the same way, the details that caught their attention and the different elements that allowed the child to interact without problems with the prototype were observed. It was noted that the elements of stimulation for each activity successfully completed and the feedback elements within the prototype caught the attention of the children and made it easier for them to understand what happened when performing an action. In the same way, the stimuli and rewards managed to capture the child’s attention.

On the other hand, the videos in sign language made by trained interpreters were clear and understandable for the children, which allowed them to carry out some of the activities without asking for help from the teacher. The number of errors made within an activity decreased markedly, since they paid attention to the indication given by the interpreter in the video and then carried out the activity.

Once the cognitive walkthrough analysis is completed, the children are evaluated on the prototype by means of the Smiley Meter survey, which consisted of asking questions about aspects of the prototype with the help of the linguistic model and the teacher. The children answered with palettes indicating how they felt using the prototype of the application or if they liked or not each of the activities, colors, images, etc.

Results of the Cognitive Walkthrough for Each Iteration.

Second iteration.

The prototype had the following 5 activities:

  • ACT1: Find the images that look like the model

  • ACT2: Find the repeated image

  • ACT3: Find the differences between 2 images

  • ACT4: Search for the image without a match

  • ACT5: Search for specific objects

The results of the Smiley meter survey are shown below in Table 3. Even though 5 children participated in the evaluation of the prototypes, only 4 were part of the

Table 3. Survey results Smiley meter first functional prototype
Full size table

During the execution of the cognitive walkthrough technique, the measures associated with the metrics are taken, number of errors (E) and number of times they requested help (A) and the normalization of the data obtained for each of the activities of the prototype (see Table 4) where (En) represents the normalized value of the number of errors made by each child and (An) (see Table 5) that represents each of the times the children requested help. The NAV column represents

Table 4. Errors made with the first functional prototype
Full size table
Table 5. Activities in which they requested help with the first functional prototype.
Full size table

In this first evaluation of the functional prototype it can be seen that the high values presented in Tables 4 and 5 are mainly due to the fact that it was the first time they interacted with the application, so it was difficult for them to understand from the beginning the dynamics of each one of the activities and had the need to ask for help to understand them.

On the other hand, the children are just beginning to familiarize themselves with the written word, therefore the texts were not very helpful to carry out the activity.

Table 4 shows that the activities in which most of the children made most of the errors were the activity of finding the 4 differences and finding the image without a match. The errors made in the activity find the 4 differences are due to the fact that the model image and the image they had to interact with to select the differences were not clear. On the other hand, the activity of finding the image without a match generates confusion in the child since the concept “Without a match” is not easy to explain and the signs given by the avatar were not clear to the child. For the rest of the activities the children asked a few times and with the explanation given to them, they carried out the activities. The amount of errors in general of all the activities is also due to the lack of response of the application in front of an incorrect action of the child within the activities, for instance, in the activity to find the images equal to the model, if the child played an element that did not correspond to the model image, the system did not take any action to inform the child that his response was incorrect, however when he selected the correct answer it was hidden implying that the option was chosen.

For the navigation within the application there were few errors made by 3 children, who used the back button of the device without having an answer, then they realized that there was an arrow that took them back inside the application.

Third Iteration.

The children were asked how they felt with the application and everyone answered with the happy face palette. In the same way, they were asked if it was easy to use, all the children answered with the happy face palette except for one of the girls who put her face down and indicated that it was difficult for her to perform two of the activities of level 1 and 2 (see Table 6).

Table 6. Evaluation results with smiley meter scale of the second functional prototype
Full size table

During this process, the different measures associated with the metrics of number of errors (E) and number of times they requested help (A) are taken and the data obtained in each of the activities of the prototype are normalized.

It can be seen that, for this second evaluation of the functional prototype with children, the values dropped considerably compared to the first evaluation made in the previous iteration, this is because they had already interacted one time with the application, so the number of times they asked for help was less. In addition, the feedback provided in the activities was meaningful for the children and did not need help to understand that they had selected a right or wrong answer.

The different videos with the explanations given by an avatar within the activities were not taken into account by the children, so the number of questions on the indications of the activities Find the differences or find the image without a match was high.

At the end of the activities, the children observed the context interface where they can understand the meaning of the element that is shown in each activity and associate it with a real life context, relying on the sign and the written word, which attracts a lot of attention and it generates emotion when they see it.

Final Evaluation.

During the Smiley Meter survey with the support of the expert teacher and the linguistic model, the children were asked how they felt with the application and everyone answered raising the palette with the happy face and gave their opinion. Similarly, they were asked if they found it easy to use (see Table 7). Three children were part of this final evaluation.

Table 7. Survey results smiley meter with the final prototype
Full size table

Expert Reviews.

Regarding the opinions given by the experts about the final prototype, they concluded that the videos with people trained in sign language are better than the avatar that was used previously and the improvement in the ease of interaction can be clearly observed with the prototype. In the same way, the help presented is clear and easy to understand.

Similarly, they believe that the guide can be applied to different types of content and promote different types of perceptual concepts that the deaf child should know as a first instance, for example, reading and writing, concepts of quantity, many, few, more, less, long, short.

6 Conclusions

The existing mobile apps reviewed and the existing gaps identified during the document review process, show that most of the applications that are aimed at deaf children have been developed without taking into account the specific needs of them, in addition to the content that is shown, it is not oriented to teaching or reinforcement of learning, since they mostly show images and text, without this representing any meaning for the children.

The qualitative research process was carried out to identify the needs of deaf children in terms of the GUI design needs of mobile applications aimed at these children. Through the recommendations of experts, observation of the interaction of the case study with mobile applications and review of related works, the needs of deaf children are identified and the design of inclusive tools is necessary, this can be seen in the notable lack of availability of mobile applications aimed at deaf children.

In the educational field, applications can be potentially significant in the process of acquiring new knowledge as long as the content is adequate and its graphic interfaces are designed to break the communication barriers between the child and the application.

There is a need for an artifact for developers and designers of mobile applications who wish to develop applications/games for deaf children, which allows them to cover the needs of deaf children, for which the design guide proposed in this research is oriented to cover this need.

A prototype of an application was designed and developed based on the design guide proposed for the learning or reinforcement of some subjects such as animals or objects of the house, through mini games. The validation of the prototype carried out with the case study yielded very positive approval concepts, as well as the evaluation of the experts.

The validation of the design guide was carried out along with the teachers and students, considering criteria of identity, design, accessibility, navigation and operation. From the results obtained, it can be affirmed that there was acceptance of all the aspects related to the evaluated criteria. This allows us to conclude that the GUI design for mobile applications aimed at deaf children, is a tool that can help developers and designers of mobile applications to create new tools that consequently allow the inclusion of this population in the use of these new technologies.