Abstract
Small and medium-sized enterprises (SMEs) increasingly rely on agile software development. However, the majority of established usability methods have been developed with traditional software engineering principles in mind. So, one might assume that these methods and tools might not be applicable to agile development projects. In this paper, a possible approach to systematically adapting traditional usability methods to application in agile projects is introduced and documented as best practices. These best practices can be quickly and dynamically employed by agile development team members and can thus contribute to higher quality of software development outputs. The approach described in this paper was developed in the context of a German research project funded by the German Federal Ministry of Education and Research (BMBF) and conducted by one research institute and three software-developing enterprises and will be evaluated throughout the further course of the project. For this purpose, a number of best practices have been adapted to the context of agile software development and described in detail in order to allow inexperienced software developers of small and medium-sized enterprises to successfully apply these best practices. As two examples of these best practices, we illustrate the best practices ‘Contextual Inquiry’ and ‘Template-Based UI Design’ in this paper.
1 Usability Engineering in Agile Development Methods
Today, agile development methods such as pair programming, test-driven development, and continuous refactoring have gained wide popularity (Komus 2015). Particularly small and medium-sized software companies tend to switch to agile methods as they expect a greater degree of flexibility, high-resolution interaction with customers, as well as improved dynamic reaction to changing requirements. Customer centricity, consideration of customer feedback, and increased orientation towards customer satisfaction are substantial aspects of both agile development and usability engineering. Hence, the domains of agility and usability engineering share commonalities, which should facilitate the integration of both methods. Nevertheless, both approaches seem to contravene at the current stage.
1.1 Usability Flaws in Agile Projects
When assessing qualitative aspects in agile software projects, the focus is primarily placed on functional completeness and correctness, while non-functional aspects such as usability and user experience take a backseat. Moreover, there is a potential conflict between the iterative cycles in agile projects and the coherent, integral perspectives of usability engineering. The most commonly used agile framework Scrum (Sutherland and Schwaber 2015), for instance, offers software vendors excellent support for IT project management and implementation activities. However, Scrum does not provide any information on how non-functional quality aspects can be achieved systematically. This commonly results in usability flaws when applying agile development methods, which in some circumstances become visible only at later stages, when modifications can only be made with high effort. The negative implications caused by these usability flaws can directly affect existing customer relationships, sales opportunities, as well as the market position of the concerned company. Many SMEs that opt for the introduction of agile development methods therefore need additional assistance in order to realize enhanced and more reliable usability and user experience for their products.
1.2 Problem-Solving Approach of the PQ4Agile Project
A possible approach for addressing the above-mentioned challenges is to systematically adapt existing usability methods, which have been scientifically validated and proven to work in practice for years, to application in agile projects. For this purpose, these methods can be designed in a more lightweight fashion and – if required – in separately executable components, so as to allow agile development team members to flexibly apply them on demand. The research project “PQ4Agile – Product quality for agile software development” aims precisely at bridging this gap between agility and empirically underpinned usability engineering (Schmitt 2015). In the collaborative project PQ4Agile, systematic support for development team members is being elaborated, aimed at promoting broader and more successful application of usability methods in SMEs using agile development. One of the core concepts within the scope of the project is to enrich agile development processes with established activities and practices from the usability and user experience engineering domains. This offers a holistic quality assessment perspective for agile software development. Hereinafter, the results of the PQ4Agile project will be outlined with regard to their contribution to enhanced usability and predictable high quality in agile development projects.
2 Methodology
As a basis for the work carried out in PQ4Agile, the project consortium has generated a quality model that covers all known aspects in the areas of usability and user experience. However, in order to facilitate software vendors in holistic quality assessment, both functional requirements and the complete range of non-functional requirements have to be taken into consideration. Furthermore, the quality of software products is also influenced by development process activities as well as by the company structure, incl. the available human resources and the technical infrastructure. Therefore, the quality model has been enhanced to cover the three overriding quality areas, namely, software quality, process quality, and structure quality. This model follows the common hierarchical subdivision into quality criteria and sub-criteria. The quality criteria are consolidated into two or three criteria groups per quality area. In the area of software quality, the criteria groups comprise product quality (i. a. with the criterion fitness for use) and usage quality (i. a. with the criteria effectivity, efficiency, and satisfaction). In the area of process quality, the criteria groups comprise process structure quality and process application quality. Finally, in the area of structure quality, the criteria groups comprise quality of human resources, quality of infrastructure, and quality of work environment. For the definition of the criteria and sub-criteria, an existing quality model served as a basis for each quality area. For the software quality area, the ISO 25010 (ISO 2011) model was used. For the process quality area, the “Gokyo Ri” (Kneuper 2015) model was used; and for the structure quality area, the ISO 9001 (ISO 2008) model was used.
Subsequently, suitable linkage points were identified in order to integrate the best practices into individual development processes of different companies. Software companies rely on agile processes and methods in numerous variations. Usually, different approaches are tested, customized, and partly discarded again. As a consequence, development processes emerge that do not follow specific reference frameworks but are rather modified according to the specific givens of the companies. In order to integrate the developed best practices into the individualized processes of different companies, common and universal linkage points had to be identified. For this purpose, the development processes of the software companies involved in the PQ4Agile projects were analyzed in a first step: Which activities are being carried out? Which artifacts and tools are being used? Which roles participate in the development process? Which rituals are prevalent (e. g., meetings)? Based on these process analyses, the project consortium modeled a reference process that consolidates the commonalities of the different as-is processes. To avoid missing essential process components, the consortium additionally analyzed the Scrum framework, the CMMI-DEV-Model (CMMI Product Team 2010), as well as different processes from well-known software engineering literature. Due to the fact that the surveyed roles, rituals, artifacts, and tools are highly specific and barely generalizable, the consortium exclusively adduced activities for the modeling of the reference process. These are classified into six sub-areas: requirements, planning and design, evaluation, realization, controlling, as well as project planning and project governance (see Figure 1). These areas are not to be understood as temporally subsequent project phases. The described activities taking place during development processes are rather of an iterative nature. An iteration is usually triggered by a specific event, such as an alteration of customer requirements or the beginning of a new Sprint.
Reference Process.
3 Development of Usability Best Practices
In order to address all qualitative aspects of usability and user experience within an agile development process, a number of different best practices are required. The goal of the PQ4Agile project therefore is to generate a compendium of best practices that can be used efficiently by developers. On the one hand, it is intended to be as lucid as possible; on the other hand, the compendium should offer comprehensive support for the optimization of all relevant quality areas. The consortium has investigated established methods and practices (e. g., personas, RITE method) that can be systematically adapted for integration into agile processes based on requirements. Additionally, promising new approaches have also been taken into consideration, e. g., in-house practices of the project partners. Also included were contemporary recommendations from the usability community that are suitable for making a contribution to the enhancement of software quality in agile development projects.
3.1 Selection of Suitable Best Practices
In order to ensure that the usability best practices fit the parameters of agile development, seven principles were defined for their selection and generation, respectively:
Minimalism / Simplicity: The scope of the performance of a software engineering activity and the documentation of its results shall only comprise what is necessary for the given project situation in order to allow efficient further processing and maintenance.
Closeness to People: It shall be possible to leverage the availability of and the close collaboration with customer stakeholders and among team members through software engineering activities.
Closeness to Artifacts: Software engineering artifacts shall be produced and made available in such a way that they can be used as efficiently as possible by subsequent activities and that they are as close as possible to the target product.
Timing: Software engineering activities shall always be performed exactly (the latest) at the point in time when this is most beneficial and efficient for the project’s progress.
Scalability: If project factors such as stakeholders, system complexity, or team size increase, it must still be possible to perform software engineering activities in such a way that they do not contradict the basic principles of agile processes and still continue to provide enough support to achieve quality attributes.
Change Affinity: Agile processes consider frequent changes to be an essential part of the performance of a project. Software engineering activities shall also comprehend changes and shall allow efficient changes affecting the areas of requirements engineering, software architecture, and user experience engineering, while providing support for the stakeholders.
Integratability: The software engineering activities must be such that they will allow integration into existing agile development processes as seamlessly as possible.
During the selection of the best practices, the consortium preferred methods and practices that avoid redundancy and that are renowned for their efficiency, i. e., their thriftiness in terms of required human resources, time, and other resources. Ideally, the selected best practices should directly benefit the enhancement of quality. In addition, the best practices should be applied in development processes as early as possible in order to appropriately eliminate and / or remedy quality deficits at an early stage. Constructive measures that add to quality enhancement “by design” were chosen over analytical measures that require later improvements. Furthermore, the best practices should be easy to learn, preferably without expert knowledge. For beginners, if required, a short introduction given by an experienced team member should be sufficient. Also, the best practices have to be lightweight in order not to discourage beginners with excessive complexity. The target groups of the best practices are neither usability experts nor expert teams but rather common agile development team members in SMEs. These developers are usually all-rounders, albeit possibly having a specific focus and expertise in domains such as requirements analysis, software architecture, usability engineering, or UI design.
3.2 Example “Contextual Inquiry”
According to ISO 9241, usability is defined as “the effectiveness, efficiency and satisfaction with which specified users achieve specified goals in particular environments”. This directly implies that developers have to consider how specific users perform their tasks in a specific context. “Contextual inquiry” (Holtzblatt, Wendell and Wood 2004) was selected as a best practice due to the fact that this method is rarely known among non-usability experts (Ross 2015). Hence, it potentially provides new insights and values for most agile developers. Contextual inquiry combines semi-structured interviews with participatory observation of the (prospective) user in a real-life use case context. Within the scope of contextual inquiry, data is collected on how users carry out certain processes and which tools they draw on during the processes. In this way, detailed information can be obtained about users, processes, selected tools, as well as about the organizational, social, technical, and physical environment. With a little preparation and practice, any random agile development team member can conduct such an interview. Additionally, contextual inquiry is highly compliant with the above-listed principles, particularly with the principle of “Closeness to People”. Contextual inquiry efficiently combines the activities of multiple usability methods (user interview and task analysis) and is therefore exceptionally effective. The outputs of this best practice can be further processed in the form of personas, functional and non-functional requirements, or as-is state descriptions. A convenient side effect of contextual inquiry is the fact that a development team member is personally present on the customer’s premises. This allows the team member to get a clearer picture of the customer’s circumstances. Moreover, the face-to-face interaction means that the development team is no longer an anonymous group for the customer.
3.3 Example “Template-Based UI Design”
The best practice “Template-based UI-design” (see Figures 2 and 3) serves the purpose of generating and using templates for the design of user interfaces. These templates define the base frame of an application through instantiable UI elements that are used within the application, e. g., buttons, tables, header and navigation menus. For this purpose, a hierarchical structure of these elements is defined in order to identify the relationships between single elements and elements at higher levels on the one hand. On the other hand, it is used to define and identify, a priori, influences caused by changes in specific higher-level elements on low-level and dependent elements. Furthermore, all attributes of each element are specified, so as to avoid under-specified or non-specified aspects during their implementation. All dialogs of the developed application are designed exclusively on the basis of these templates. Similar to a style guide, it is therefore possible to specify the elements for a single project, a specific product line, or for the entire company. This practice does not only facilitate the generation of consistent prototypes but also enables traceability of prototypically instantiated UI elements for executable code.
Template-based UI design – Template.
Template-based UI design – App.
3.4 Documentation of the Best Practices
Due to the fact that the best practices are structured differently, the consortium had to find a way to document them on a comparable level of description and in a similar form. For this purpose, a generic template was developed that comprises all relevant information concerning best practices for developers: objectives and short descriptions of the best practices, motivation and conditions for their application, detailed approach (incl. preparation and follow-up), as well as required inputs and generated outputs. By mapping the best practices to the artifacts described in chapter 2, the best practices can be integrated into the agile reference process and the PQ4Agile quality model. Recommendations for the best practice applicant as well as information concerning quality criteria and risks (e. g., common mistakes and potential negative implications through other activities) round out the description template. With the help of the description template, the best practices are concretized in a way that any agile team member is able to immediately understand and, ideally, apply them. Simultaneously, the description is abstract and generic enough to enable the application of the best practices independent of the given development process and to allow their transfer to different domains. In order to keep the applicability barrier as low as possible, all best practices are designed in a way that requires neither complex process modifications nor the introduction of new process models.
3.5 Evaluation of the Best Practices
In order to evaluate the applicability and the usefulness of the selected and documented best practices in agile software development projects, an appropriate evaluation concept was developed in the PQ4Agile project (see Figure 4). This evaluation concept follows the foundations of empirical software engineering and allows for systematic evaluation. It does so by conducting a controlled experiment that results in traceable and reliable propositions regarding the application of the best practices in agile software development projects. The evaluation concept focuses on the coupling of the best practice documentations and the PQ4Agile quality model. In the best practice documentations, generic goals that are achieved by applying a particular best practice are listed. For example, one goal of the best practice “Conduct a Contextual User Interview” is “Identify environmental influences on the interaction with the product”. These generic goals are concretized in a given project context. For this purpose, generic goals are broken down into individual project sub-goals. For each goal, several sub-goals can be specified. Each sub-goal must address exactly one particular criterion of the PQ4Agile quality model that is supposed to be positively influenced by the application of the particular best practice in the given project. To facilitate the formulation of sub-goals, user stories (Cohn 2004) are used. User stories are widely used in agile software development projects. They are composed of the following parts: “As a <role>, I want <goal> so that <benefit>”. The element <goal> is a generic goal from the best practice documentation, but instantiated to the particular project context. The element <benefit> refers to a single particular criterion from the PQ4Agile quality model. So, the benefits to be achieved (a positive influence of a quality criterion) with the help of the application of a best practice bridges the gap between the best practice documentations and the PQ4Agile quality model.
Best Practices and Quality Model.
For instance, the aforementioned example goal can be broken down into the following two user stories that address the quality criteria ‘Context Completeness’ and ‘Flexibility’:
As an interaction designer, I want to identify the impact of changing usage environments on the interaction with the product so that I can assure the usage of the product in an office, an open space office, a car, a plane, and a train (context completeness).
As an interaction designer, I want to identify the impact of changing usage environments on the interaction with the product so that I can assure the product usage in contexts beyond the currently specified ones (flexibility).
The goals of best practices are broken down into sub-goals that specify which quality criteria are supposed to be positively influenced by the application of the best practices. This must take place before the best practices are applied. This way, a reliable proposition regarding the influence of the application of best practices to quality criteria can be ensured. In general, best practices can be applied to positively influence several quality criteria. By defining which particular quality criteria are supposed to be positively influenced by the application of which best practice, an effect on a quality criterion becomes traceable to the application of a particular best practice.
The actual evaluation of the best practices is conducted with the help of newly created detailed questionnaires for end users, developers, project managers, and team leaders. Participants are asked to answer tailored questions regarding the use of the best practices from their particular points of view. The questionnaire is based on the PQ4Agile quality model and covers a balanced number of statements regarding the fulfillment of each criterion of the quality model in particular situations. The statements are assessed via 5-point Likert scales by the persons who fill in the questionnaire. With this approach, it is possible to judge the actual impact of the application of particular best practices on particular criteria of the PQ4Agile quality model. In addition, an evaluation of the applicability of the best practices and their integratability into existing software development processes is performed on a regular basis. For this purpose, software developers who apply the best practices are asked to give feedback on their individual experiences with the selection, usage, and integration of the best practices in their daily business. The feedback regarding the process applied by the software developers to select appropriate best practices is meant to form the basis for the future provision of a best practice for selecting best practices. This will facilitate this process and support software developers in making their decisions.
Currently, we imagine both a top-down approach and a bottom-up approach. With the top-down approach, the software developer starts with the coarse goals provided with the best practice documentations and breaks them down into sub-goals addressing particular quality criteria. With the bottom-up approach, the software developer defines the quality criteria he wants to positively influence and then looks for best practices that are appropriate for this purpose according to their coarse goals.
4 Outlook
During the PQ4Agile project, best practices are being integrated into the software development processes of multiple SMEs and applied to reference projects. The perceived improvements according to the defined dimensions of the quality model are measured empirically, covering the developers’ perspective as well as product and process quality from the customers’ perspective. This feedback cycle enables continuous refinement of best practices, processes, and presentation. But the core mission of PQ4Agile is to make the methodology and the best practices accessible to other developers at software SMEs. This is accomplished through the creation of a knowledge platform as a free-of-charge reference repository combining intuitiveness with structure as well as with advanced filtering and search capability. Semantic MediaWiki (SMW) is a commonly known and intuitive open source knowledge management framework based on standard mediawiki technology. Offering a variety of semantic extensions, SMW allows describing the quality model, the process structure, and the relations of best practices explicitly through semantic properties, attributes, and classes. Hence, methods can be filtered and grouped more effectively, e. g. by specifying the activity, role, effort, and standard (sub-) criteria of the quality model. While mediawiki leaves the knowledge structure and presentation to the user alone, SMW supports this task with semantic forms and templates. New best practices are automatically classified and presented in a consistent form. Features of mediawiki such as discussion and feedback are available in SMW and turn it into a platform where developers can easily describe their experiences and suggest enhancements. This makes SMW an ideal starting point for designing a knowledge portal for PQ4Agile. The only drawback of SMW lies in its standard representation style, which is wiki-like. Therefore, PQ4Agile SMW will be enhanced by a visualization overlay to produce dynamic process charts as navigational assistance and by UX enhancement to make it a comprehensive and convenient knowledge portal. The PQ4Agile knowledge repository will be offered directly via the project homepage.
About the authors
Hartmut Schmitt is research project coordinator at HK Business Solutions GmbH (Sulzbach / Saar, Germany), a provider of business software and hardware solutions for SMEs. Since 2006 he has been working on research projects in the fields of human-computer interaction, usability, user experience, and requirements engineering, including project supervision for several BMBF-funded joint projects. At present, he is leading and participating in the research project “PQ4Agile – Product quality for agile software development”.
Dominik Magin studied Computer Science at the University of Kaiserslautern with a focus on software engineering. Since 2013, he has been a researcher at the Fraunhofer Institute for Experimental Software Engineering IESE in Kaiserslautern, Germany. He is working in the area of user experience with a strong focus on the mobile domain. His particular interests are wearables and the conception of mobile applications as well as their respective development.
Andreas Maier studied Computational Linguistics, Language Science and Technology, and Philosophy at Saarland University in Saarbrücken, Germany. Since 2008, he has been working first as a scientific assistant, then as a researcher at the Fraunhofer Institute for Experimental Software Engineering IESE. His particular areas of interest are human-computer interaction, user experience and user requirements engineering, and multimodal and multimedia-based interaction. Regarding the latter, he is especially interested in the evocation of positive user emotions and is currently working on conceptual models of human-computer interaction and user experience.
Richard Wacker studied Information Engineering and Management at the Karlsruhe Institute of Technology (KIT), Germany, and worked for 5 years at the Centre of Applied Legal Studies with a focus on data protection, data security, and semantic technologies. He was a project manager at Ontoprise GmbH and Volz Innovation GmbH. In July 2013, he joined YellowMap in the position of a research project manager and is now Head of Research, responsible for YellowMap’s national and European research activities and projects.
Josh Wang holds degrees in International Business Management from Konstanz University of Applied Sciences, Germany, and in Business Information Systems from Ferdinand Porsche University of Applied Sciences, Vienna, Austria. He gained experience in international project management, marketing & relationship management in various global enterprises, including Mercedes-Benz China, the German Chamber of Commerce and Industry, and SAP. Since 2009, he has worked for CAS Software AG in the Innovation & Business Design department.
Acknowledgment
The PQ4Agile project is funded by the Federal Ministry of Education and Research (BMBF) under project number 01IS13032.
References
CMMI Product Team 2010. CMMI for Development, Version 1.3. Pittsburgh: Carnegie Mellon University.Search in Google Scholar
Cohn, M. 2004. User stories applied for agile software development. Boston: Addison-Wesley.Search in Google Scholar
Holtzblatt, K., J.B. Wendell and S. Wood. 2004. Rapid contextual design: a how-to guide to key techniques for user-centered design. San Francisco: Morgan Kaufmann.10.1145/1066348.1066325Search in Google Scholar
ISO – International Organization for Standardization. 2008. ISO 9001:2008 Quality management systems – Requirements.Search in Google Scholar
ISO – International Organization for Standardization. 2011. ISO / IEC 25010:2011 Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – System and software quality models.Search in Google Scholar
Kneuper, R. Gokyo Ri: Messung und Bewertung von Prozessqualität, retrieved April 16, 2015 from http://www.kneuper.de/GokyoRi/.Search in Google Scholar
Komus, A. Status Quo Agile, retrieved April 16, 2015 from http://www.hs-koblenz.de/rmc/fachbereiche/wirtschaft/forschung-projekte/forschungsprojekte/status-quo-agile/.Search in Google Scholar
Ross, J. Why Are Contextual Inquiries So Difficult?, retrieved April 16, 2015 from http://www.uxmatters.com/mt/archives/2012/06/why-are-contextual-inquiries-so-difficult.php.Search in Google Scholar
Schmitt, H.: PQ4Agile – Produktqualität für Agile Softwareentwicklung, retrieved April 16, 2015 from http://www.pq4agile.de/.Search in Google Scholar
Sutherland, J.; Schwaber, K.: Scrum Guides, retrieved April 16, 2015 from http://www.scrumguides.org/.Search in Google Scholar
© 2015 Walter de Gruyter GmbH, Berlin/Boston
