1 Introduction

Information and communication technology (ICT) has been heavily utilized in education sector of the developed world. However in the developing countries, the utilization of ICT in education, specifically in primary education, is still evolving. Usability plays a crucial role for the success of an ICT system [2]. Measuring usability is difficult when it involves children [3]. However, Read and MacFarlane state that usability in this age group can be measured in terms of fun [4]. Read et al. have worked on evaluating usability of computer application, school trip and website designing [3]. They have introduced Fun Toolkit scale for measuring the children’s self-reported experience of fun [3]. Read et al. have explored the Fun Toolkit as measuring tool specially designed for children aged from 5 to 10 years. They have discussed about the relationship between fun and usability [5].

Several researchers have worked on child-computer interaction and have assessed usability or fun for various technologies, i.e., fun games for children [6], mobile applications for educational purpose [7], educational software for learning English [8], the educational app for improving teaching-learning methods [9], measuring fun for Sindhi to English mobile learning App using Fun Toolkit [10], Interactive Learning System [11], measuring entertainment products [12], Child e-Learning Programs [13], and computer applications [14].

There is not much work carried out in the developing countries, specifically, in Pakistan, on evaluating the usability of the learning mobile Apps. To the best of our knowledge, no work has been carried out on the usability assessment of the Urdu learning mobile Apps for children. Therefore, this study tries to fill this gap. In this paper, the results of the usability study regarding the Urdu learning mobile Apps for children are presented.

1.1 Selection of the Urdu Learning Mobile Apps

From the Urdu learning mobile Apps available at Google Play store, four Apps were selected on the basis of their rankings, number of downloads, ratings and the reviews. Moreover, these Apps had similar activities for Urdu learning. These selected four Apps are shown in Fig. 1.

Fig. 1.
figure 1

Selected Urdu learning mobile Apps [1, 15]

Full size image

2 Methodology

The usability study was conducted at the primary schools of District Shaheed Benazirabad, Pakistan. A total of 232 students participated in the study. The demographic questionnaire, Fun Toolkit based post-test questionnaire and the usability tasks were developed. The questionnaires were filled by the participating children with the help of their teachers and the researchers. The following three usability tasks, which were performed by the children on each of the Apps on Tablet-PC, are shown in Fig. 2.

Fig. 2.
figure 2

Usability tasks [1]

Full size image

In the current study, two instruments from the Fun Toolkit have been used: Smileyometer and Again-Again table. In Smileyometer, five faces with meaningful labels have been given as choices of different expressions to be selected by children. These faces express children’s feeling from “Awful” to “Brilliant” as shown in Fig. 3 [3].

Fig. 3.
figure 3

The Smileyometer [3]

Full size image

The Again–Again table measures engagement of children [3]. This measure is based on the knowledge that people like to do fun things again. Children use the Again-Again table to answer the question if they would like to do the activities again. Figure 4 shows the Again-Again table for all the Apps used in the current study.

Fig. 4.
figure 4

The Again-Again table [1, 3]

Full size image

3 Results and Discussions

3.1 Demographic

In this study, from the test participating children, 57.33% were boys and 42.67% were girls. Among these participating children, 46.60% students were in 5–6 years age group, 23.72% students were in 7–8 years age group and 29.68% students were in 9 years or above age group.

3.2 Usability Test

In this study, the efficiency of the Apps was measured through the task completion time. The three tasks were given to the participants one by one and time was recorded for each task by the stop watch. The effectiveness of the Apps was measured by recording the tasks completion rate. Immediately after the usability test, the participating children’s experience of fun was measured through the Fun Toolkit by getting their opinions about the Apps. Two instruments from the Fun Toolkit were used: Smileyometer and Again-Again table.

3.3 Tasks Completion Time

The average tasks completion time in seconds for all the Urdu learning mobile Apps was recorded. On average, the participants had completed all the tasks of the App1 in 11.95 seconds, of the App2 in 12.11 seconds, of the App3 in 9.41 seconds and they took 16.58 seconds for completing all the tasks of App4. The participants have taken minimum time in the App3 which was 9.41 seconds, as shown in Fig. 5. Hence, the App3 was more efficient App in terms of tasks completion time.

Fig. 5.
figure 5

Average tasks completion time for each App

Full size image

Table 1 shows group differences of the Apps regarding tasks completion time. The data were analyzed using SPSS software.

Table 1. Descriptive statistics regarding tasks completion time
Full size table

To determine whether there was a difference in the Apps in terms of tasks completion time, one-way ANOVA was used, F (3,180) = 23.201, p < 0.001 as shown in Table 2. The results show that there is statistical significant difference among the Apps in terms of tasks completion time. In Table 3, tukey post hoc pairwise statistical differences among the Apps show that there is statistically significant difference between App3 and App1 (P < 0.05), between App3 and App2 (P < 0.05), between App3 and App4 (P < 0.05). Since mean tasks completion time of the App3 was less comparing to all other Apps and the results were statistically significant, thus it can be concluded that the App3 was the most efficient App as the participants completed all the tasks in lesser time than the other Apps.

Table 2. One-way ANOVA
Full size table
Table 3. Multiple comparisons
Full size table

3.4 Tasks Completion Rate

Figure 6 shows number of the participants in percentage in two categories: “Tasks Completed” and “Tasks Not Completed”. Only 78.40% children have completed all the tasks for the App1 while 74.27% children have completed all the tasks for the App2. However, all the children have completed all the tasks for the App3 while only 71.59% children have completed all the tasks for the App4. Hence, the most effective App was the App3 in terms of tasks completion rate.

Fig. 6.
figure 6

Tasks completion rate of all the tasks for each App

Full size image

3.5 Again-Again Table

Using Again-Again table, the participants were asked to report regarding each of the Apps: “Would you like to use this App again?” Figure 7 shows responses of the participants. For the App1, 78.22% participants’ reported response was “Yes”. For the App2, 81.49% participants responded as “Yes”. The App3 was most rated as 95.22% of the participants rated it as “Yes”. For the App4, 73.66% of the participants’ response was “Yes”. According to the results of the Again-Again table, most of the children wanted to use the App3 again and again.

Fig. 7.
figure 7

Results of Again-Again table for all the Apps

Full size image

The Again-Again results were coded as 3 for “Yes”, 2 for “Maybe” and 1 for “No”. Table 4 shows descriptive statistics of the Again-Again table responses.

Table 4. Descriptive statistics of Again-Again table
Full size table

To determine whether there was a difference in the Apps in terms of the responses of Again-Again table, one-way ANOVA was used, F (3,924) = 15.319, p < 0.001 as shown in Table 5. The results show that there is statistical significant difference among the Apps in terms of Again-Again table responses. Tukey post hoc pairwise statistical differences among the Apps show that there is statistically significant difference between App3 and App1 (P < 0.05), between App3 and App2 (P < 0.05), between App3 and App4 (P < 0.05). Since mean Again-Again table responses of the App3 was less comparing to all other Apps and the results were statistically significant; hence the App3 was the App which most of the children wanted to use again and again.

Table 5. One-way ANOVA
Full size table

3.6 Responses of Smileyometer

Using Smileyometer, the participants were asked to report regarding each of the Apps: “How much fun was it to use the App?” Figure 8 shows Smileyometer responses of the participants. For the App1, 57.56% of the participants’ response was “Brilliant” and 30.56% participants’ response was “Really Good”. For the App2, 54.31% of the participants rated “Brilliant” and 34% rated “Really Good”. 71.45% of the participants responded as “Brilliant” and 24.44% responded as “Really Good” for the App3. Finally for the App4, 46.77% of the participants’ response was “Brilliant” and 25.47% of the participants’ response was “Really Good”. According to the Smileyometer results, the maximum positive response was reported for the App3.

Fig. 8.
figure 8

Smileyometer results for each App

Full size image

The Smileyometer responses were coded from 1 to 5 where 1 was used for “Awful” and 5 was used for “Brilliant”. Table 6 shows descriptive statistics of the Smileyometer responses.

Table 6. Descriptive statistics of Smileyometer
Full size table

To determine whether there was a difference in the Apps in terms of the responses of Smileyometer, one-way ANOVA was used, F (3,924) = 18.769, p < 0.001 as shown in Table 7. The results show that there is statistical significant difference among the Apps in terms of Smileyometer responses. Tukey post hoc pairwise statistical differences among the Apps show that there is statistically significant difference between App3 and App1 (P < 0.05), between App3 and App2 (P < 0.05), between App3 and App4 (P < 0.05). Since mean Smileyometer responses of the App3 was less comparing to all other Apps and the results were statistically significant; hence the App3 was the App which most of the children reported as having most fun.

Table 7. One-way ANOVA
Full size table

4 Conclusions

In this research four Urdu learning mobile Apps were compared from the usability perspective. According to the tasks completion time results, the App3 was the most efficient App as the participants completed all the tasks in lesser time than the other Apps. The results were statistically significant. The most effective App was the App3 in terms of tasks completion rate and the results were also statistically significant. Since mean Again-Again table responses of the App3 was less comparing to all other Apps and the results were statistically significant; hence the App3 was the App which most of the children wanted to use again and again. Since mean Smileyometer responses of the App3 was less comparing to all other Apps and the results were statistically significant; hence the App3 was the App which most of the children reported as having most fun. According to the usability perspective, the App3 is the best App. It is recommended that the App3 should be complemented with the existing traditional classroom learning in the primary schools of District Shaheed Benazirabad and other cities of the country for enhancing the learning outcomes of children. In future, more usability studies would be conducted in various schools throughout the country for getting more feedback. An experiment would also be conducted to measure the learning outcomes of the students in the primary school by introducing the App3.