Keywords

1 Introduction

Currently software users are increasingly demanding and not only have a clearer understanding of their needs but, upon using an application, require a certain degree of usability. Usability is defined as the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use [18].

Agile methodologies have allowed, as its name emphasizes, to speed up the process of building software. However, different authors agree that, despite facilitating the development and adjusting easily to usability techniques, more industrial applications must be developed to validate the proper way to integrate both philosophies [38].

The Agile manifesto [10] consists of four values:

  • Individuals and interactions over processes and tools.

  • Working software over comprehensive documentation.

  • Customer collaboration over contract negotiation.

  • Responding to changes over following a plan.

These values and its twelve principles are funded through incremental, cooperative, and adaptive methods [2].

The main agile methods are: Extreme Programming (XP), Scrum, DSDM (Dynamic Software Development Method), AM (Agile Modeling), ASD (Adaptive Software Development), Crystal series, FDD (Feature Driven Development [2]. The first three are the most used as seen in the results of the review.

On the other hand we have user-centered design (UCD) whose life cycle is very similar to that of agile methodologies. UCD is a set of techniques, methods, procedures and processes that places the user at the centre of the development process. The ISO 13407 defines UCD as an iterative process composed of four basic design activities [18]:

  • Understand and Specify Context of Use. Know the user, the environment of use, and the tasks that he or she uses the product for.

  • Specify the User and Organizational Requirements. Determine the success criteria of usability for the product in terms of user tasks, e.g. how quickly a typical user should be able to complete a task with the product.

  • Determine the design guidelines and constraints. Produce Design Solutions. Incorporate HCI knowledge (of visual design, interaction design, usability) into design solutions.

  • Evaluate Designs against Requirements. The usability of designs is evaluated against user tasks.

Based on the significant similarity of the objectives sought by the UCD and agile methodologies, it would seem that the problem could be reduced to developing soft-ware using an agile methodology and UCD in the design stage, in order to ensure a usable software product.

For Salah, Paige and Cairns [36], Agile and User Centered Design Integration gained increased interest due to three reasons:

  • The reported advantages of UCD on the developed software as it enables developers to understand the needs of the potential users of their software, and how their goals and activities can be best supported by the software thus leading to improved usability and user satisfaction.

  • The Agile community hardly discusses user needs and user interface design. Moreover, none of the major Agile processes explicitly include guidance for how to develop usable software.

  • There are differences between Agile methods and UCD in: focus, evaluation method, culture and documentation, that suggest that their integration will be fundamentally challenging.

Could it occur that a software product designed using UCD fulfills the requirements proposed by users but does not have a high degree of usability? These and other questions are discussed in this systematic review about integrating agile methodologies and techniques of usability as a complete process of software development to fulfill the requirements of the user through an easy to use tool.

2 Systematic Review

The systematic review is a research method widely used in medical sciences and adapted by Kitchenham to software engineering. In contrast to traditional reviews, systematic reviews allow us to find the best available evidence by methods to identify, evaluate and synthesize relevant studies on a research topic. The methods used are pre-defined and documented, so that other researchers can evaluate them critically and can be replicated later [20].

2.1 Research Questions

The definition of research questions is the most important part during a systematic review, because they will appropriately focus the research. In order to determine these questions, criterion called “PICOC” (Population, Intervention, Comparison, Outcome and Context) was used.

As the objective of this systematic review is not to find comparisons between usability techniques and agile methodologies, the Comparison criteria of PICOC was not used. Table 1 shows the results obtained by applying the other criterion.

Table 1. PICOC results
Full size table

These terms were compared to systematic reviews found in this review as they cover similar issues but with different objectives.

The research questions that were defined for the review were:

  • Question 1: What usability methods are integrated into software development methodologies?

  • Question 2: What agile methodologies have integrated usability techniques throughout the complete software development process?

  • Question 3: What new frameworks or methods have been proposed for the integration of agile processes and usability engineering?

2.2 Search Strategy and Selection

In order to perform the search and selection of papers, a strategy was defined. First, the search terms were defined in the databases; then the phases of the search; and finally, the criteria that would be used to include or exclude the resulting papers.

Search Terms.

Based on information obtained by applying the PICOC criterion, search terms were defined and thus the following search string was derived.

Search Process.

The search was performed using three online databases that contained references for scientific articles and journals, conference proceedings and technical papers: Scopus, ISI Web of Science, and IEEE Xplore.

Criterion of Inclusion and Exclusion.

We defined a set of precise selection criteria (inclusive and exclusive) to select the most appropriate studies for the systematic review. The inclusion of studies was determined by the following criteria: the study presents integration of usability methods/techniques and agile methodologies.

However, studies that fulfilled either of the following criteria were excluded:

  • The study only presents usability techniques applied in post production

  • The study is not written in English or Spanish

  • The study is a book or conference review

Additionally, the studies published before 2010 were excluded because this review includes four previous systematic reviews [19, 36–38] that cover earlier years.

2.3 Search

The search was conducted on December 19th, 2014, which allowed finding 2015’s papers, but after applying the criteria for exclusion were not considered.

The search string found 693 papers. After reviewing the title, keywords and abstract, 96 articles were obtained and finally after the full reading 37 were selected. These results are shown in Tables 2 and 3.

Table 2. Amount of papers classified by database
Full size table
Table 3. Complete list of seleted studies
Full size table

3 Results

In order to answer each of the research questions, the selected items were classified according to the method used for the study, the agile methodology employed and the usability techniques applied. This classification is shown in Table 4.

Table 4. Classifcation of studies
Full size table

Systematic reviews found [19, 36–38] were used for tuning the review process applied to define the years that were considered and check if the search string was adequate.

Exploratory studies [9, 11, 16, 23, 24, 34] are the outcomes of observation or applying questionnaires or interviews to development teams about using agile methodologies integrated with usability. They allow us to understand the point of view of developers, designers and usability experts who every day face the construction of software.

The methods and frameworks [2–5, 8, 12, 14, 15, 21, 26, 28–31, 35, 39, 41] were set forth by the authors as proposals of specific procedures to follow according to a determined type of software development. In all of these studies the proposed or adapted method or framework is described, and include at least one case study where its application is shown.

The selected case studies [1, 6, 7, 13, 17, 25, 27, 32, 33] include the intent of integrating usability techniques and methods to agile methodologies. Case studies that used usability techniques in an isolated manner with regard to the methodology without forming a unified process were excluded.

It was also observed, as can be appreciated in Fig. 1, that the concern for integrating Usability and Agile Methodologies in a unified process, that would allow the development of a usable software product, has grown considerably in the last two years.

Fig. 1.
figure 1figure 1

Papers per year

Full size image

3.1 Integration of Usability and Agile Methodologies

The objective of Isa et al. in [17] is to demonstrate the applicability of agile user-centered design in software applications developed in a rural community. In said article, it is stated that particular issues regarding information use exist, and that these are influenced by the culture and environment in this scenario. Using Extreme Programming (XP), interviews, prototypes and feedbacks in the two proposed iterations, software was developed for the commercialization of handcrafts. Lastly, the satisfaction with the fulfillment of functional and non-functional requirements was verified, but no usability evaluations were applied that could demonstrate that this integration aided the construction of a usable software.

On the other hand, Lizano et al. [27] propose applying Remote Synchronous User Testing (RS) to projects developed using Scrum. The case study included RS in the different development stages as a tool to evaluate usability, with the principal contribution being the feedback obtained in these evaluations. The authors propose that RS should be applied by developers. This last statement can be related to [32] where it is mentioned that the low cohesion between designers and developers must be mitigated. This presents the challenge of verifying the technical viability of the design proposed by the designers and the prioritization of the agile methodology proposed by the developers.

Despite being the integration of UCD and Scrum similar to the previously mentioned case studies, [13] obtains additional results specific to the type of software developed. Being an educational tool, besides being usable it must also be accessible, and thus includes tests with kids affected by Down Syndrome. This case study again emphasizes the cohesion among the different actors that participate in software creation and the importance of the communication among them. In [33] a tool is also developed that must be usable and accessible, but aimed at elder adults. Additionally the authors observed that Agile Methodologies are more flexible than others because they allow reevaluating the requirements in the different software development stages, allowing the system to be adapted.

Finally, the exploratory studies [9, 11, 16, 23, 24, 34] confirm what has been said in the different case studies: the integration of Agile Methodologies and Usability is convenient and totally possible. In order to realize that integration the usability experts (designers and testers) must share a common objective: to satisfy the needs of the user. The diverse stages of Agile Methods complicate the labor of the designers, since they may lose the general vision of the software product due to the incremental and iterative process. This situation can be mitigated by the adequate formation of future professionals in both design and computing, in such a way that both disciplines find common grounds. These studies also reveal the need of a unified and clear process that allows the team members (analysts, designers, programmers and testers) to follow precise steps and techniques. In the following subsection the different proposed methods and frameworks are discussed.

3.2 Frameworks and Methods

Bertholdo et al. [5] present a series of usability patterns applied during agile development. Examples and recommendations are given to clearly identify the situations in which to apply each pattern and solution example. As in [3, 21, 30] these methods propose applying different usability techniques in the early stages of software development to establish the software requirements.

Methods that describe in detail the integrated usability and software engineering activities and techniques applied throughout the software development process exist for the benefit of development teams. Papers [1, 2, 4, 8, 15, 26, 29, 31, 35, 39, 41] propose well defined and detailed methods that integrate agile methodologies and usability in the different stages unlike [14, 28] where the emphasis is placed on the use of different types of usability evaluations to obtain user feedback in the iterations.

Some of the frameworks, such as those proposed in [2, 4] were developed to be applied in specific types of software.

4 Conclusions and Future Works

Agile methodologies are the most accepted in current development teams due to their level of flexibility and speed but must be integrated with different techniques and methods that will permit validating and verifying requirements compliance and the generation of usable software.

Designers, usability experts and developers must improve their communication and cohesion, acknowledging themselves as actors in the same process, and participating in different stages.

A different professional education may be required so that the different actors of the software development process see usability design as not only an aesthetic issue, but the development of a pleasant user interface, natural interaction and simple processes that aid the user in the fulfillment of his objectives.

Emphasis must be made in designing strategies that permit combining the general vision of the software product, realized during the design stage, with the modular vision produced when applying Agile Methodologies.

Although the methods and frameworks that propose the integration of usability and agile methodologies in a single unified process are facilitators for the work of development teams, more case studies of their use is needed to validate their applicability in the industry.

Finally, the systematic review is a powerful tool for the visualization of a subject, however the vast difference in the paper formats demands a significant time investment during their selection. Hopefully this issue will eventually be overcome with initiatives such as those of ELSEVIER to include specific elements in the abstracts like: goals, methods and results, among others.