Keywords

1 Introduction

There are several interactive software products, which involve the intervention of real users, being these required in different areas, which has given them importance in society. This implies that software products are in constant challenge, which focuses on improving their quality. Usability is one of the main characteristics of the quality of a software product. To evaluate the usability, software developers have the possibility of using the well-known usability evaluation methods, which are diverse and combinations of these can be used [4]. In addition, it is possible to complement said usability evaluation methods with the use of UX evaluation methods.

According to Holzinger, usability evaluation methods are quite diverse, which can be classified into inspection methods and test methods [4]. The present paper focuses on a particular test method and a single variant of it, together with the use of a UX method, which is the use of usability testing process in its “one-to-one” variant, along with the use of usability metrics of the ISO/IEC 9126 standard and Emocards. Usability testing, in its variant “one to one”, consist of a process that employs people as test participants, who are representative of the target audience, which are observed, each by a moderator/observer, in order to evaluate the degree to which a software product meets specific usability criteria, such as usability metrics [1].

According to the information gathered from interviews conducted with experts with experience in usability evaluation methods, the traditional usability testing process with users, which includes the use of usability metrics and Emocards, brings some problems due to this process is carried out manually. High rates of time to plan a usability testing session, large volume of material and/or templates, human errors in the calculation for the preparation of reports, significant reduction of the performance of the experts and delays in presenting the results reports to the clients, were the most outstanding problems detected in this research.

In addition, the literature offers descriptions at a high level about the usability testing process [1], usability metrics [2, 5,6,7] and Emocards [3]. However, the description of the process offered in the literature cannot be applied to the letter, since there are different contexts with different limitations, such as time, the amount of investment available, the number of participating users and experts, among others. This forces test teams to design usability tests with users under different variants. Therefore, in this paper it was necessary to contrast and find points of similarity between the existing literature and cases of real test teams that apply a usability testing process with users adjusted to their context. In this way, we can model a process that can be flexible with the reality of the usability testing teams with users from different environments and contexts.

Several of the usability evaluation and UX evaluation methods can be executed and/or supported by software. In particular, the usability testing process with users is already being supported by various commercial tools and tools developed in the scientific field, but, as we will see in this research, they have many limitations. In this sense, this paper presents the tools UTWebSystem and UTMobileApp, a web system and a mobile application, which aim to support the execution of the activities that make up the usability testing process with users, which pretend to provide a solution to the previously mentioned problems.

2 Related Works

There are commercial tools that try to support the usability testing process; such is the case of Morae [8]. Morae is a software tool developed by Techsmith corporation, which allows users to capture, analyze, edit and play videos of usability tests sessions remotely or in person. Although this tool provides a way to cover certain steps of the usability testing process, it has a number of limitations that are important to highlight. Firstly, the results of the final reports are based on graphs that are only composed of the combination of three metrics: effectiveness, efficiency and satisfaction. Secondly, there is a limitation when adding new metrics to include them in the usability evaluation, since these can only be based on the movement of the mouse, the clicks that the user gives to the interface, the time to perform a task and the results of the satisfaction questionnaires. Finally, the use of recordings implies an extra effort when performing the usability evaluation, since it implies that usability experts have the task of repeatedly watching the recording of sessions that can take hours.

There are also other commercial tools such as Chalkmark [9], ClickHeat [10], ClickTale [11], Open Hallway [12], Silverback [13] and a variety of software services for usability evaluation, such as Usertesting.com, Userzoom.com, Loop11.com, among others. While these tools are capable of recording usability tests, which allows you to perform the test remotely and produce reports based on cursor tracking, clicks and heat maps, most of these do not allow, at all, develop questionnaires or reports with the most usability metrics based on ISO/IEC 9126 or other standards. All the commercial tools previously presented do not contemplate the combination of usability evaluation methods with the particular user experience evaluation method used by Emocards, which is a limitation that the solution proposed in this paper aims to solve.

With respect to sources of research and/or scientific work, there are software tools, which were found through a systematic search defining a search strategy based on PICOC, according to the guidelines established by Kitchenham and Charters [14]. A search chain (1) was developed considering synonyms and whose publications date from 2013 onwards to analyze the status of the solutions. The search process was carried out using two widely recognized databases to search for primary studies: SCOPUS and IEEE. Gray literature was excluded since it is not peer reviewed.

C: (software OR “support system” OR “support application” OR “collaborative system” OR “collaborative application” OR “support tool” OR “collaborative tool”) AND (manual OR “manual process” OR steps OR “manual steps” OR phases OR procedures OR “manual procedures”) AND (metrics OR “usability metrics” OR measurements) AND (“usability Testing” OR “usability evaluation” OR “user testing”) AND (publication year > 2013) (1).

One of the tools found, through systematic search, that is important to highlight is ECUSI [15], which allows us to plan projects for usability evaluation methods (MEU) according to combinations of more than one MEU, among which are the tests of usability with traditional users. This is based on the use of document templates created in Google Drive for their collaborative work, which, although this tool provides a novel way to solve the problems of disorganization, loss of information and rate of error in the results, this has limitations with respect to the quantity of usability metrics that it contemplates. Added to this, the elaborations of the final reports must be carried out by means of manual calculations, which indicates that there is still the effort and extra time, besides leaving open the possibility of human errors in the calculations. In addition to ECUSI, there are other studies related to the application of usability tests with users remotely, such as, for instance, the use of Surveymonkey [16] and other collaborative tools for recording test sessions, which highlights the effectiveness of these, have to evaluate software products.

In general, and taking into account the current state of literature, the tools that support the traditional usability testing process are lacking. Of the few that were found, it was discovered that they have several limitations, which does not allow us to solve the main problems previously exposed, which are derived from the execution of a traditional usability testing process. Added to the above, there is literature that describes in detail the usability testing process with users, such as the Handbook of Usability Testing [1]; however, this cannot be applied to the letter, since there are different contexts with different limitations, such as time, the amount of investment, the number of participants and experts, among others. This forces test teams to design usability tests with users under different variants. Therefore, it was necessary to contrast and find points of similarity between the existing literature and cases of real test teams that apply a usability testing process with users adjusted to their context.

3 UTWebSystem and UTMobileApp as Tools to Support the Usability Testing Process

Having identified the problems that arise when executing the traditional usability testing process based on metrics of ISO/IEC 9126 and Emocards, the UTWebSystem and UTMobileApp tools were proposed as a solution, which provide support to the usability testing process based on usability metrics of ISO/IEC 9126, leaving the possibility to manage new metrics, and various Emocards proposals. Likewise, UTWebSystem and UTMobileApp are oriented to a process that was modeled as a result of the analysis of the information offered by the literature on usability testing process, usability metrics, Emocards and, on the other hand, the information gathered from the interviews that were conducted as part of the investigation. The objective of the development of these tools is to offer possible benefits for both the evaluators and for the effectiveness of the evaluations, but it should also be noted that there were limitations with respect to the diversity of variants that they can support.

3.1 Supported Process

As will be described later in Sect. 4, interviews were conducted with various usability experts, who have experience performing this type of testing method, in order to contrast the information obtained from the testimony of these experts with information from Literature [1, 17] on the traditional usability testing process.

Having looked for similarities and differences between the information coming from the literature and the information obtained from the interviews with the usability experts, a process was modeled that considers the alternative flows that were identified with the information retrieved from the interviews. The new usability testing process based on usability metrics of the ISO/IEC 9126 standard and Emocards contemplates the stages shown in Fig. 1, from the perspective of the evaluators and the users. Subsequently, a new process model was created in which the same stages are contemplated, but also UTWebSystem and UTMobileApp as tools that support it and allow to overcome the problems that arise when the process is traditionally carried out, which were mentioned in Sect. 1.

Fig. 1.
figure 1

Adapted from [18]

Stages of usability testing process.

Full size image

3.2 Functionalities

According to the results of the interviews, 100% of the interviewees agreed that it is not a good practice for a usability expert to be involved in more than one stage of the process, except for the Analysis and Synthesis stages because they can be covered by only one role. That is why UTWebSystem has different functionalities that are segregated according to different roles that are based on the stages of the usability testing process. The role “planner” has all the permissions with respect to the tasks of the Planning stage, while the “observer” role and the “participant” role contemplate tasks related to the Evaluation stage. Finally, the “expert-editor” role has all the permissions related to the Analysis and Synthesis stages. It should be noted that there is the role “administrator”, which has all the existing permissions, but only had demonstrative purposes on the web system.

UTMobileApp is focused on the evaluation stage from the evaluator’s perspective. Consequently, it has functionalities aimed at improving the performance of the evaluators during the observation and the registration of the data collected from the participants.

Planning Stage.

In UTWebSystem, users registered in the system with a role “planner” can manage work teams, which can have one or more associated usability testing sessions (see Fig. 2). This means that users have the possibility to configure the test data, select the participants registered in the system according to their profiles and select the evaluators and experts responsible for the analysis and synthesis of results. In addition, users with a “planner” role can manage the Pre-Test and Post-Test questionnaires, the list of tasks for the participants, the list of usability metrics that will be considered in the evaluation and the Emocards. UTWebSystem is designed to reduce the duration of the planning stage, so it allows reusing questionnaires, list of tasks, usability metrics, Emocards or designs of total tests of usability testing projects previously created to be able to create new sessions according to the needs.

Fig. 2.
figure 2

Interface corresponding to management of usability testing sessions.

Full size image

Evaluation Stage.

In this stage, on the UTWebSystem side, the interaction of the participant with the software product is contemplated. That is why we have the role “participant”, which can answer the programmed questionnaires, see the assigned task list and register the user experience through the Emocards. The latter are synchronized according to the progress of the task list.

On the side of UTMobileApp, the corresponding tasks of the observer/moderator are contemplated. UTMobileApp aims to provide functionalities to improve the performance of the observer/moderator in the Evaluation stage (see Fig. 3). That is why only users with observer role can enter the mobile application, which will provide the necessary information for the observer on the dynamics to follow, the metrics to be evaluated and instructions on how to evaluate them. The most outstanding feature of UTMobileApp is that it allows registering, in real time, the input fields of the usability metrics when observing the participants.

Fig. 3.
figure 3

Interfaces corresponding to the main functionalities of UTMobileApp.

Full size image

Analysis and Synthesis Stages.

In the Analysis stage, UTWebSystem will automatically make reports based on the configuration of the test in the Planning stage and the data collected in the Evaluation stage. These reports are known as preliminary reports. Users registered in UTWebSystem who have the role “expert-editor” may consult these preliminary reports. The reports have summaries and statistical graphs of the results of the participants of the test. Its main objective is to provide quick information for the team that developed the software product evaluated to redesign it, if that is necessary.

The report editor of UTWebSystem allows to the usability experts to create detailed final reports. Registered users with the role “expert-editor” have the results to be analyzed (see Fig. 4) and can emit observations on positive and negative aspects. The sections that comprise the final report are editable, allowing you to customize according to the requirements of the clients. Finally, the usability expert can issue a conclusion supported by the results and observations previously detailed in the report.

Fig. 4.
figure 4

Interface corresponding to the sample of results for the analysis stage.

Full size image

4 Development Process

The software development process that was followed to develop the Web system “UTWebSystem” and the mobile application “UTMobileApp” was to use an agile methodology, since it had the collaboration of three usability experts, who took the role of client in all the process. The agile methodology chosen was the XP methodology (Extreme Programming) [19]. This involved going through the stages of Extreme Programming methodology, which are Planning, Design, Codification and Testing.

In the Planning stage, a questionnaire was previously prepared to carry out structured interviews with five usability experts from different contexts. They have the experience of carrying out usability tests in a traditional way. The interviews were useful to collect relevant information about the usability testing process, which allowed us to contrast the theory of the Handbook [1] and it was also possible to abstract generalities to model two processes, under the BPMN standard [20]. On the one hand, the usability testing process with users carried out in a traditional way and, on the other hand, the user usability testing process supported with the solution proposed in this paper.

In the Design stage, an architecture document based on RUP was previously elaborated, which allowed to define the architecture components that will be necessary for the deployment of the applications. In addition, a catalog of user stories was produced because of interviews with customers. It should be noted that these user stories changed throughout the software development process due to the nature of the agile methodology used. Added to the above, prototypes were developed, which the clients in the weekly meetings validated. On the one hand, prototypes were developed with the Mockups tool [21] from Balzamiq for the case of the Web System. On the other hand, prototypes were developed with the Marvel Apps tool [22] for the case of the Mobile Application.

In the Codification stage, the web and mobile application system was developed with the use of two frameworks. On the one hand, the web system was developed with the Laravel framework [23], which allows managing the view-controller model, previously defined in the architecture document. On the other hand, the mobile application was developed with the IONIC framework [24], which allows to build hybrid applications with a high-end aesthetic, in addition, through the Cordova Apache project, it allows building mobile applications for the IOS and Android operating system, which are the most used operating systems to date.

Finally, in the testing stage, a test plan of functionality was carried out based on the user stories that were established for both the web system and the mobile application. The objective of the test plan is to verify the correct functioning of the user stories implemented, in addition to checking whether they meet the requirements of the clients. Based on XP, the test plan was presented in each iteration based on the results obtained. That is, the tests were conducted on a weekly basis through acceptance tests with clients at the end of the presentation of each iteration.

5 Validation Process

Once the solution proposals were implemented, one of the most important questions of the investigation arose. This question was how to validate the ease of use and especially the importance that UTWebSystem and UTMobileApp have in the usability testing process with users based on the usability metrics of the ISO/IEC 9126 and Emocards. As previously mentioned, there are several methods to validate usability, as well as there are other methods to validate the importance of software products that are in a given context.

The environment in which this project was developed was in the Department of Informatics Engineering of the Pontifical Catholic University of Peru (PUCP). Thanks to this, it was possible to count on the support of the UMetSoft research group, which develops research focused on the area of HCI and Software Engineering, in which it has as research topics usability evaluation methods, usability metrics, user experience, usability, among others. The validation process could not have been carried out without the collaboration of the researchers of the UMetSoft group, since they provided the necessary permissions to have a testing laboratory and the attendance of participants that meet the user profile of the web system and mobile application.

5.1 Execution of an Experimental Design

In the first instance, it was decided to evaluate the usability and importance of the web system by means of the research method based on the execution of an experimental design with pre-test-post-test and control group [25]. The experimental design with pre-test-post-test and control group is a type of “pure” experiment, which has two requirements to achieve control and internal validity. On the one hand, they require comparison groups that, in this case, will be two groups. On the other hand, they require that these comparison groups be equivalent; that is, they have similar characteristics. Participants are randomly assigned to the two groups that make up the experiment and pretests are applied simultaneously. Subsequently, one group receives the experimental treatment and the other group does not receive it (control group). Finally, a post-test is applied to both groups [25].

This experiment was adapted to the context of the present research in order to be able to validate later, through a statistical analysis of the results obtained, the usability and the importance that UTWebSystem has in the usability testing process based on usability metrics of ISO/IEC 9126 and Emocards.

Participants.

We counted with the collaboration of 30 students of the course of Information Systems 2, which is dictated in undergraduate of the specialty of Informatics Engineering of the Pontifical Catholic University of Peru. The participants fulfilled the profile of users of the web system, since they had the minimum knowledge of the usability testing process with users, usability metrics and user experience.

Materials and Equipment Used.

For the use of the web system, a testing laboratory was required with the necessary quantity of equipment. With the help of the UMetSoft research group, it was possible to separate a suitable environment in the facilities of the Pontifical Catholic University of Peru (PUCP).

The pre-test and post-test questionnaires were prepared previously considering the Likert scale from one to five, where one is a negative perception and five is a positive perception.

Use of Questionnaires.

On the one hand, the pre-test questionnaire obtained the scores that were used for control purposes in the experiment; that is, it will serve to know how adequate the allocation of the groups that was carried out was. On the other hand, the post-test questionnaire, which will be subject to analysis, consists of 16 items, which collect the data, to determine the perception of the participants, necessary for the validation of the Web System with respect to its ease of use and its importance.

Experiment Design.

A total of 30 participants were expected, of which 19 were attended. The design of the experiment consisted of an experimental group and a control group that is why two groups were randomly separated; that is, a group of nine participants and another group of 10 participants. There were two rounds in which, in the first round, a group became the experimental group. In other words, this was the group that supported by UTWebSystem and UTMobileApp to perform the usability testing process based on metrics and Emocards. The other group became the control group; that is, the group that performed the usability test based on metrics and Emocards in a traditional way. In the next round the same thing was done, but in an inverse manner. In short, the group in the previous round was the experimental group became the control group and the group in the previous round was the control group, became the experimental group.

At the beginning of the execution of the experimental design, the pre-test was evaluated to all the participants in order to measure the previous knowledge with the participants and to find out how adequate was the assignment of the groups that was carried out. At the end of the experiment, the post-test questionnaire was evaluated for all participants with the purpose of using the results to perform the statistical analysis.

Both groups (control and experimental) conducted a usability test for a specific software product. The software product that was evaluated was the Hoteles.com website, which offers online accommodation in various hotels around the world.

The usability testing session consisted of a traditional test using questionnaires, usability metrics of the ISO/IEC 9126 Standard, a task list developed by the participants and Emocards to evaluate the user experience. Each group had three members with the role “participant” and the rest with the role of experts (three with the role “observer”, a “planner” and an “expert-editor”).

Previously the Pre-Test and Post-Test questionnaires were elaborated, a list of predefined tasks for the participants of the experiment was defined, the metrics of the ISO/IEC 9126 standard of usability to be evaluated were selected and the Emocards were designed to collect the experience of the users.

Results of the Experiment.

The results of the questionnaires evaluated were collected from the participants who were part of the control and experimental groups, which were analyzed independently of the groups to which they belonged to evaluate the statistical results.

On the one hand, the results obtained from the pre-test allowed confirming that the participants had the prior knowledge required to perform the proposed usability test. In addition, the results allowed knowing that the allocation of the groups was adequate regardless of the rotation shifts that were carried out. On the other hand, the results obtained from the post-test (see Table 1), allowed us to perform a statistical analysis to find out if the general hypothesis is accepted. The hypothesis states that the web system allows usability testing with users and Emocards more easily and efficiently overcoming the problems described in Sect. 1. According to the Likert scale from one to five, the general hypothesis presented is taken only if through the tests that were performed in the statistical analysis it is possible to determine that the average of perceptions exceeds the neutral value 3.0.

Table 1. Results of the post-test questionnaires.
Full size table

Statistical Analysis of the Results.

Table 2 shows the descriptive statistical results obtained with the SPSS tool. On the one hand, the average represents the average assigned to UTWebSystem by the participants (3.898026) which is quite good, since it exceeds the neutral value of 3.0, taking into account that 1.0 is considered a totally negative perception and 5.0 a totally positive perception about the Web System. On the other hand, the standard deviation represents how far apart the opinions are among the participants. We find a value of 0.3506, a value below 1.0 and that tends to zero, which indicates that the opinion among the different participants is quite similar. That is, they have the same perception that the Web System would contribute to the development of tests with users and meets the requirements of ease of use.

Table 2. Descriptive statistics results. Own elaboration using the SPSS software.
Full size table

Although the average results in a value above 3.0, which is a favorable result that serves as a validation of the proposed software, it is necessary to perform a statistical analysis. This analysis has the purpose of determining if the difference between the resulting value (3.898026) and 3.0 is significant; that is, it can be said that the perception of the software product is positive and does not tend to be neutral. Therefore, we proceeded to find out if the distribution of the data is a normal distribution, with which we propose two hypotheses, the null hypothesis (H0) and the alternative hypothesis (H1), which are described below:

  • H0: The data come from a normal distribution.

  • H1: The data do not come from a normal distribution.

To find out which hypothesis is the one that is true, we proceeded to perform the Kolmogorov-Smirnov test [26].

The SPSS software was employed and the result of the Kolmogorov-Smirnov test was obtained, which is detailed in Table 3.

Table 3. Results of Kolmogorov-Smirnov test. Own elaboration using the SPSS software.
Full size table

The value of the Kolmogorov-Smirnov statistic is 0.20. Under a confidence level of 95% and a significance of 5%, it was not possible to refuse the null hypothesis, so it was concluded that the data come from a normal distribution.

Finally, due to the reason that the data follow a normal distribution, the most appropriate statistical test to determine if there is a significant difference with 3.0 is the T-Student of a single sample [27]. Again, two hypotheses were proposed, the null hypothesis (H0) and the alternative hypothesis (H1), which are described below:

  • H0: The average of the perceptions is equal to 3.0

  • H1: The average of the perceptions is greater than 3.0.

The comparative value used in the T-Student test of a single sample was 3.0. Then, in Table 4, the results of the T-Student test are shown.

Table 4. T-Student test results from a single sample. Own elaboration using the SPSS software.
Full size table

As can be seen in Table 5, the value of the T-Student statistic is 0.000. Under a 95% confidence level and a significance of 5%, the null hypothesis was refused. This is because 0.000 is less than 0.05 (5%). Therefore, it was concluded that the average of the perceptions of the participants is greater than 3.0. In other words, the participants of the experiment had a positive and significant perception in relation to the neutral value (3.0).

Table 5. Expert judgment results regarding the ease of use of UTMobileApp.
Full size table

Finally, the general hypothesis is true. Therefore, it is stated that the Web System allows usability experts to perform usability tests based on metrics of ISO/IEC 9126 and Emocards more easily and efficiently overcoming the problems described in Sect. 1.

5.2 Expert Judgment

In the second instance, it was decided to jointly evaluate the web system with the mobile application, giving emphasis to the latter in order to validate the usability and the importance it has in the usability testing process based on usability metrics of ISO/IEC 9126 and Emocards. For this reason, the expert judgment method was used, which consists of an informed opinion of people who have experience in the topic of interest, who are recognized by others as qualified experts, and who can provide information, evidence, judgments and assessments [28].

The use of this method opened two important questions to solve. The first question was to know the profile that had the experts who were part of the evaluation. The second question was to know the amount of experts to use. In the context of the present research, factors such as experience and/or publications made in topics related to usability and specifically to methods of usability evaluation were considered for the selection of experts. In addition, it was considered three usability experts who were part of the test. This number of experts was considered adequate because it exceeds the minimum amount required, which is two experts according to the author McGarland et al. [29].

The steps for the execution of the expert judgment method that were considered in the validation of UTMobileApp were based on the Guide for conducting expert judgment of Escobar-Pérez and Cuervo-Martínez [28]. Firstly, the objective of the expert judgment was defined, which was to validate the usability and importance of UTMobileApp, which supports the evaluation stage of the usability testing process with users based on usability metrics of the ISO/IEC 9126 standard and Emocards. Secondly, the experts were contacted, who were given the objective of expert judgment and validation. Thirdly, a list of items was specified, which were separated into two categories. The items related to the ease of use and the items related to the importance of UTMobileApp in the usability testing process with users based on usability metrics of the ISO/IEC 9126 standard and Emocards. With this, a template was developed, which served as a validation instrument since it meets all the characteristics described in the steps previously described. Fourthly, the meeting with the judges was held to carry out the validation by expert judgment. In this meeting, each judge separately used the mobile application together with the web system and qualified the items that were previously specified. The results for the ease-of-use and importance categories of UTMobileApp are shown in Tables 5 and 6 respectively.

Table 6. Expert judgment results regarding the importance of UTMobileApp.
Full size table

Having elaborated the items based on the Likert scale from one to five, where one is the perception in total disagreement and five in total agreement, we obtained means greater than the neutral value 3.0, which is quite positive. To evaluate the concordance of judges, Kappa statistic was used, which had confirmed that agreement in the opinion of the three judges. The judges provided a set of minimum observations and recommendations that, although they did not affect in a decisive way in the validation and in the conclusion of the experts, they served to improve the graphic user interface of UTMobileApp. Finally, the judges came to the mutual agreement that UTMobileApp is easy to use and is important in the usability testing process based on the ISO/IEC 9126 standard and Emocards.

6 Conclusions and Future Works

UTWebSystem and UTMobileApp are tools developed in order to provide a solution to the frequent problems that arise when executing the known usability tests with users along with the use of usability metrics and Emocards. In this sense, they pretend to be tools of frequent use in the field of evaluation of usability and user experience. With the development of these tools, it has also been planned to increase the productivity of the usability experts and/or usability practitioners, since within the agile development methodology that was used, the usability expert has been considered as the client and the development of these tools has focused on these.

Taking into account the results of the systematic search of related works and similar tools in the commercial field, these have many limitations and do not solve most or none of the problems detected in the interviews conducted in this research. In this sense, having developed the solution proposals and having carried out experiments that obtained positive results, encouraging conclusions emerge regarding the UTWebSystem and UTMobileApp tools. With respect to UTWebSystem, it can be concluded that UTWebSystem supports the usability testing process based on usability metrics of ISO/IEC 9126 and Emocards. Firstly, because the experts can have the information centralized in a database, which avoids loss or confusion. Secondly, because it allows collaboration and communication between different usability experts. Thirdly, the web system provides support because it manages to automate the generation of reports, which is no longer manual. Finally, the web system allows working new usability testing sessions based on previous sessions, which greatly facilitates the work of the usability test planner. With respect to the UTMobileApp, it can be concluded that it supports the evaluation stage of the usability testing process with users due to three reasons. Firstly, because it makes the task of the observer simple when entering the observation data to the system. In other words, the well-known observation and compliance sheets in physical format are no longer needed. Secondly, the mobile application was designed to take care of the comfort of the user, since having implemented this module, as a web part would have involved using a larger and heavier equipment. This would make the work of the observer more uncomfortable during the test. Finally, the Mobile Application allows the observer to do his work intuitively without needing to pay all his attention in the handling of this and focus more on the actions of the participants.

Due to the encouraging results, a set of future activities has been proposed. In the first place, since there are several variants of usability tests with users and this solution proposal only addresses the particular case of the “one-to-one” variant, it is proposed as a future work to be able to generalize this solution for all variants of possible usability testing process. This would involve broadening the scope of the research and conducting more interviews with collaborators from different contexts in which traditional usability testing, usability metrics and Emocards are applied. Secondly, another future work to consider is to integrate these tools with the S.I. of a company and evaluate the results obtained. Due to the fact that within the scope of the proposed solution it is contemplated to support usability tests of software products in process or in preliminary versions. This would greatly help to measure the usability of software products giving it a commercial focus. Finally, the last future works to be considered are to refine the graphical user interface of both the web system and the mobile application and, in addition, to be able to control versions of usability testing sessions and apply audit to a greater degree than already It is managed. This will further reduce the risks of human error, which is the main source of problems encountered in the traditional usability testing process.