A controlled experiment to evaluate the understandability of KAOS and i* for modeling Teleo-Reactive systems

Highlights

• A novel method to model software requirements for Teleo-Reactive systems using i*.

• A novel method to model software requirements for Teleo-Reactive systems using KAOS.

• i* notation has higher understandability level for modeling Teleo-Reactive systems than KAOS.

Abstract

Context

Teleo-Reactive (TR) specifications allow engineers to define the behavior of reactive systems while taking into account goals and changes in the state of the environment.

Objective

This article evaluates two different Goal Oriented Requirements Engineering notations, i* and KAOS, to determine their understandability level for specifying TR systems.

Method

A controlled experiment was performed by two groups of Bachelor students. Each group first analyzed a requirements model of a mobile robotic system, specified using one of the evaluated languages, and then they filled in a questionnaire to evaluate its understandability. Afterwards, each group proceeded similarly with the model of another system specified with the second language.

Results

The statistical analysis of the data obtained by means of the experiment showed that the understandability of i* is higher than that of KAOS when modeling TR systems.

Conclusion

Both languages are suitable for specifying TR systems although their notations should be specialized to maximize the understandability attribute. i* surpasses KAOS due to two main reasons: i* models represent dependencies between agents and goals or tasks; and notational differences between tasks and goals in i* are more evident than those between goals and requirements in KAOS.

Introduction

Reactive systems, as event-driven systems that react to external/internal stimuli, possess specific characteristics that make them particularly sensitive to the architectural decisions that are made during the course of their construction. Issues such as task planning, concurrency management, fault tolerance, and distributed communication need to be addressed as far as possible through the solutions that the literature has shown to be most suited to systems of this kind.

Some authors (Lamsweerde, 2001, Yu and Mylopoulos, 1998) argue that software development for this kind of systems should benefit from an approach driven by what the user wishes to obtain from it, i.e. its goals, rather than how it is constructed. In other words, goals should drive the development process. We opted for the Teleo-Reactive (TR) paradigm (Nilsson, 1993), a goal-oriented approach for modeling systems that manages their actions, outputs and status in response to stimuli coming from either inside or outside the system. The TR approach thus offers engineers a high-level goal-oriented formalism for the development of reactive systems that allows them to define the system behavior while taking into account goals and changes in the state of the environment. As detailed below in Section 3, a TR specification is defined as a set of prioritized condition/action rules. Conditions are a set of inputs either sensory or coming from a model of the world, while actions change the world or the model of the world. A TR specification interpreter should constantly re-evaluate the triggering condition set for each rule, and should execute the action corresponding to the highest-priority rule with a satisfied condition. Actions may also be TR specifications themselves, thus allowing the definition of hierarchical TR programs. This kind of nested TR specifications can be considered as system subgoals. The root of the hierarchy of a TR program is called the goal of the system-to-be.

One representative instance of reactive systems is the mobile robotics domain, where the TR paradigm has provided very interesting results (Ranjan et al., 2010, Gubisch et al., 2008, Broda et al., 2000), since TR-based systems inherently recover from errors and unexpected events whilst proceeding toward their goals. This domain embraces applications in which robotic vehicles work autonomously in non-structured environments. The reuse of existing solutions in current robotic systems is mostly focused on algorithms, and has not usually taken place at a higher abstraction level. The growing sophistication of robotic applications has led to the need for more sophisticated languages and platforms with which to specify, implement and validate them. To overcome this situation, we introduced a Model-Driven Software Development approach (Atkinson and Kühne, 2003, Bézivin, 2005) that separated the component-based description of real-time applications from their possible implementations on different platforms. This separation of concerns was supported by the automatic integration of the code obtained from the input architectural models (basically, UML component diagrams) into executable object-oriented frameworks. More recently, we introduced a systematic approach (Sánchez et al., 2012) that made it possible to automatically derive architectural models (including both structural and behavioral descriptions) from TR specifications. Thus, the development of reactive systems benefited significantly from the integration of the TR and architectural approaches for the design of component-based software. This integration was demonstrated with a development environment that supports the generation of code via model transformations, opening up new possibilities in the development of reactive systems.

In spite of the suitability of the TR formalism for specifying reactive systems, and more specifically robotic systems, developers perceive the system mainly as a set of prioritized condition/action rules. The development of these rules is an error prone task because it relies on mostly the expertise of the developers and their understanding of the requirements. When constructing TR specifications, the main causes of errors are the following: (1) minimal changes in the specification of the priorities produce huge execution errors, (2) it is not straightforward to ensure that each condition of a rule is a regression of some higher (priority) condition, that will ultimately lead the system to achieve its goal, and (3) it is difficult to observe the system encapsulating details or subgoals because the textual representation of the specification cannot be easily fragmented. Other inherent limitations of the TR formalism is that reuse cannot be easily achieved, since the paradigm does not offer clear resources for supporting both ‘design for reuse’ (for example, by means of catalogs of goals defined at different levels of abstraction), and ‘design from reuse’ (by using the previous catalog to identify which goals can be adapted to the behavior of the software-to-be), closer to the problem domain. All these limitations lead us to conclude that an approach that facilitates a Requirements Engineering (RE) approach for eliciting, specifying, and analyzing goals for TR specifications, i.e., that provides mechanisms for reasoning about the specifications, is needed. It is also needed for facilitating the process of evaluating alternatives of the system-to-be (Navarro et al., 2007, Chung et al., 2000).

Considering the definition of the TR formalism, it is evident that the most adequate RE technique for modeling TR systems is the Goal-Oriented approach (Hawthorne et al., 2011). Goal-oriented approaches mainly focus on the question “why” of the system-to-be. They do not only consider the deterministic aspects of the system-to-be, but also the non-deterministic aspects, such as non-functional requirements or evaluation of different alternatives for achieving a goal. In this article, we focus on the deterministic part of GORE approaches, leaving the non-deterministic part for future work.

Different modeling languages and methodologies have been defined for eliciting, specifying, and analyzing goals (see Kavakli and Loucopoulos, 2005, Pohl, 2010 for exhaustive surveys), although they all share concepts such as goal, agent and many refinement relationships (AND, OR, XOR, etc.). KAOS (Lamsweerde, 2001) and i* (Yu, 1997) are the two most well known proposals in the field. KAOS comprises six complementary submodels (goal, obstacle, object, agent, operation, and behavior model), interrelated via traceability links. i* defines a software development process that guides developers throughout several phases from early requirements to detailed design.

The objective of this work is to evaluate which of the considered requirements languages, KAOS or i*, has a better understandability level when modeling requirements for TR-based systems. This evaluation has been performed by means of a family of experiments conducted according to Kitchenham et al.’s (2002) guidelines (Kitchenham et al., 2002), as a controlled experiment with a 2 × 2 factorial design, involving students of Industrial Engineering and Telecommunications Engineering from the Technical University of Cartagena (Spain). The students were asked to fill in two understandability questionnaires just after analyzing the requirements models of two mobile robotic systems, similarly to the approach followed by Teruel et al., 2012, Teruel et al., 2011a. Both requirements models were specified using KAOS and i*.

Understandability has been the criteria analyzed in the controlled experiment because several international standards have recognized its importance in the software development process. The International Organization for Standardization (ISO) defines understandability as “the capability of the software product to enable the user understand whether the software is suitable, and how it can be used for particular tasks and conditions of use” (ISO/IEC, 2001). Besides, the IEEE830-1998 standard (IEEE Computer Society, 1998) considers an understandable specification as one of the means to decrease later redesign, recoding, retesting, etc., being one of the five quality characteristics that a requirements model must have. Therefore, in the context of TRs, we are evaluating which language, KAOS or i*, offers higher capability to enable engineers understand (i) which inputs (either sensory or coming from a model of the world) are inputs for the TR system, and (ii) which actions are to be executed to achieve the goals of the TR system. Consequently, a TR specification that is easily understandable by all stakeholders will help in developing the right software and the software right. As a side effect of the experiment we have identified some additional concepts that should be considered for extending both KAOS and i* in order to better specify TR programs.

This article is structured as follows. Section 2 presents several works related to TR systems development and Software Engineering techniques. Next, Section 3 introduces the TR paradigm along with an illustrative example. In Section 4, KAOS and i* are briefly introduced and some directives for using them to specify TR systems are given. Section 5 describes how the controlled experiment has been carried out as well as the main results, while the threats to the validity of the experiment are analyzed in Section 6. Finally, our main conclusions and future work are described in Section 7.