UUX Method Selection

1 Introduction

The human-centered design (HCD) of software solutions as a competitive advantage has gained acceptance within small and medium-sized enterprises (SME). However, many methods exist that aim to increase the system’s usability and user experience (UUX). Some methods are done quickly with less effort, others require more time as well as resources (e. g. staff, software tools). SME – mostly the persons in charge of UUX – have to choose the most appropriate method or set of methods depending on their individual project and product parameters (e. g. experiences and knowledge of the people executing the method, context of use). As it is challenging for SME to draw up a shortlist, lots of potentials are still unused. In addition, the assumption about high costs and complexity of improving the UUX inhibit the use of adequate methods.

Several different approaches for the selection of UUX methods exist, which differ primarily in terms of the used selection criteria. Along with static method catalogues (e. g. [1, 4, 6, 7]), there are also dynamic catalogues in terms of online tools, e. g. ‘UsabilityNet’ [2], ‘Usability Planner’ [3], ISO / TR 16982 [5] or ‘UCD Toolbox’ [8]. These approaches are mostly useful to select single, decoupled methods for a project. However, UUX goals are achieved in using a set of multiple methods. Therefore, it is necessary to provide methods that act together and take the correlations and interrelations of methods into account.

A workshop during the ‘Mensch und Computer 2015’ conference was organized to analyze which criteria are used to select UUX methods in practice. Furthermore, we aimed to explore how these criteria match or differ from those elicited within the approaches listed above as well as in the research initiative ‘Einfach intuitiv – Usability für den Mittelstand’ (en. Usability for SME) by the German Federal Ministry for Economic Affairs and Energy (BMWi). A scenario with three different variations of information and tasks was prepared. These variations were assigned to different groups of randomly chosen participants.

In the following, the case study will be described in more detail. Afterwards, the consolidated results of the working groups as well as of the discussion will be presented. The results are structured along the three topics selecting a method, necessary UUX expertise and the cooperation within an overall development process.

2 Case Study

The case study was conducted as part of the Usability Professionals practitioners track within the “Mensch und Computer 2015” conference. Overall, around 35 people attended the workshop what leads to six randomly formed working groups with about five to six participants each. All workshop participants were confronted with the same case, but a different task was assigned to the different working groups. More particular, participants were presented the following information about the setup of an online platform by a private company: The platform should enable associations (e. g. sports clubs or chess clubs) to perform administration tasks online instead of using a stationary software solution on their desktop computers. Therefore, the platform should be developed as a web service as well as an app. Because the company is highly aware of the importance of a good user experience for product success, they planned on setting up a development process that has a big focus on the user itself. The development process should be implemented as agile software development with a sprint duration of two weeks and multiple developers involved.

Based on this general case description, the groups were confronted with three different tasks: Group 1 and group 2 were asked to answer an evaluative task. They were told that the features of the product are fixed and now should be implemented step by step during the next sprints. It should be the task of the groups to find out whether the implementation of the features themselves leads to a good user experience. Their working capacity for evaluation was limited to five days a week. Therefore, they were asked to choose wisely which methods would work best.

Group 3 and group 4 were given an exploratory task. In their task description their main goal was to find out, which features of the former products (the desktop solution) were really valued by the users and therefore should be implemented into the online relaunch. It should be the tasks of those two groups to find out which are the features that should be implemented and how that should be done. Again, they only had five days a week working capacity and therefore had to pick methods wisely.

Group 5 and group 6 were confronted with a mixture of both tasks. These groups should decide which features should be integrated into the platform, how they should be integrated and how the success of the development process and the according evaluation can be monitored throughout the whole process. Again, their efforts were restricted to five days a week.

Each of the six groups were asked to define their methods and decide on a process that should be presented on a flipchart afterwards. Additionally, they were asked to justify their choice for a certain method. Each group had about 30 minutes for the selection of the methods and the preparation of the flipchart as well as five minutes for the presentation of their work.

3 Results

Each group presented their solution in about five minutes to the other participants and had a few minutes to discuss and to answer open questions (see Figure 1). For the evaluation of the group results, the authors of this paper took pictures of the flipcharts and compared the used methods between the different tasks. Additionally, the questions of the other participants and their answers as well as the overall discussion of the whole group was summarized and documented by the authors. There were three main topics discussed: the process of choosing a method itself, the expertise of the persons who applies the method and the cooperation between different people involved into the development process. Therefore, the results are structured along these three topics.

Figure 1

Impressions of the workshop.

3.1 Method Selection

In the current literature about usability methods more than 200 more or less different methods can be found. During our workshop overall ten methods were named in total, whereby each working group used a compilation of about seven different methods (see Figure 2). As can be seen in this numbers, many methods were similar between the different groups. It turned out that the variation of selected methods between the groups with the same task (e. g. both groups should evaluate features) is similar to the variation between the different tasks (evaluative, exploratory or both). The methods “card sorting”, “usability test with low fidelity prototype”, “interviews” and “focus groups” were selected for each of the three tasks. The methods “personas” and “questionnaire (in this special case the AttrakDiff)” was used for the exploration and the combined task, whereas “remote usability test” was named for the evaluation and the mixed task. For the evaluation task “field test” and “participatory design” were additionally mentioned and “A / B-Test” for the mixed task. Overall, the number of known and used methods is quite restricted and the selection despite some exceptions fairly independent of the task. As justification the experts mentioned multiple reasons:

1. knowing a method theoretically is not sufficient for applying it – there is a lack of practical knowledge, which makes the experts feel insecure about new methods

2. time pressure leads to the selection of well-known methods

3. the added value of methods – especially of the more expensive ones – is not clear enough for practitioners

Figure 2

Discussed method sets.

Additionally, the usability experts mentioned that they usually do not apply any method in its purest form but instead always combine it with other methods (e. g. usability testing with thinking aloud). Also, the selection itself is often independent from the actual development process. Therefore, methods are selected due to the experience of the expert which makes very common options ‘cheaper’ than unknown ones.

4 Conclusion

The selection of appropriate UUX methods is one essential task to establish human-centered design within software development processes. A workshop during the ‘Mensch und Computer 2015’ conference was organized to analyze which selection criteria are used in practice. The results of this workshop were presented within this report.

UUX professionals often argue the importance of the context of use – users, tasks, equipment, social and physical environment – within the specific domain of their project (e. g. health, agriculture, production). But UUX methods are selected focusing only on the project constraints (e. g. time, budget, level of expertise). Furthermore, UUX professionals unfortunately often neglect the importance of the context of use in terms of the organizational structure of the project itself. Issues to be addressed can be for example: How does the business analyst specify requirements? How does the software developer structure his work? How does the software tester perform the quality assurance? Which tools do they use to collaborate? Maybe results of a usability test can be presented for example not as a report with 80 pages to read, but as issues within an existing ticket system or on a Kanban board. Thus, the selection of a UUX method or a set of methods also depends on the people UUX professionals are working with or the people UUX professionals supply with data (social environment) in a project. As a UUX professional it is important to ask yourself what the objectives and expectations of this people are (users, users’ characteristics and tasks) and what tools or working documents have a major influence on their work (physical environment). Having this in mind, the selection of UUX methods could be more focused and the results of a UUX professional’s activities will have a deeper impact as well as an increased acceptance within a project.