The inconsistency between theory and practice in managing inconsistency in requirements engineering

Abstract

The problem of inconsistency in requirements engineering has been in the spotlight of the RE community from the early 1990s. In the early years, inconsistency was perceived in the literature as a problem that needs to be eliminated on sight. More recently, it has become recognized that maintaining consistency at all times is not only infeasible, but also even counterproductive. Based on this recognition, paradigms and tools have been proposed in the RE literature for managing inconsistency. However, over the same period, inconsistency as perceived and managed in practice has not received much attention. Our research aims to better understand the phenomenon of inconsistency and the strategies to address it in RE practice. This paper describes an empirical study investigating practitioners’ perceptions of inconsistency manifestations in RE, their attitudes towards these manifestations, and strategies they apply to address them. The findings of this research led to the two contributions: (a) an explanation of how the ideas of the RE field about managing RE inconsistency are reflected in practitioners’ perceptions of the inconsistency that they encounter in their daily work, and (b) the identification of some barriers that appear to be hindering practitioners’ adoption of the RE field’s inconsistency management strategies, together with possible reasons underlying these barriers.

Appendices

Appendix: A Illustrative Examples

This appendix gives some examples of using the ZJVF to manage inconsistencies in the RE for three easily described CBSs.

A.1 Traffic Light

This example shows the ZJVF interplay applied to how a traffic light at a 4-way, 2-perpendicular-street intersection can help prevent perpendicular collisions among vehicles being driven through the intersection. The interplay is apparent even when very high level assertions are used for D, S, and R. We start with R and S in the hopes that just S is enough to entail R:

R::

There are no perpendicular collisions at a 4-way intersection.

S::

The 4-way traffic light above the 4-way intersection ensures that at no time does it show green to perpendicular directions. ⁷

Unfortunately, S by itself is not strong enough to entail R. What is missing? We have to have domain assumptions D that say something about the behavior of drivers and of their vehicles.

D::

All drivers obey the traffic light and all vehicles obey their drivers.

Now D, S together are strong enough to entail R.

However, D is in fact not true. An occasional driver does not obey a traffic light. Independently, an occasional vehicle does not do what its driver has commanded it to do, perhaps because it is broken. Fortunately, the probability that D is not true is very low, say ε, because an overwhelming majority of drivers obey traffic lights, and an even larger majority of vehicles do what their drivers command them to do. Because of this very low probability, we as a society have decided to accept the risk of disobedient drivers and disobedient vehicles to obtain the benefit of smooth intersectional traffic flow that traffic lights provide while preventing most but not all perpendicular collisions. So, we permanently tolerate the inconsistency between D and the real world, the D that is a lie.

For an R that includes There are no rear-end and head-on collisions., S is really incapable of having anything to do with entailing R, and it falls entirely to domain assumptions that would include Drivers are careful enough to avoid hitting vehicles going in the same or the opposite direction. Also, probably the very existence of a red light in any direction increases the probability of rear-end collisions among the vehicles that are facing the red light.

Getting back to the original R and S, is there anyway that the D that is a lie can be made unnecessary? There are two possible ways:

• 1. We could decide that all perpendicular collisions are acceptable, i.e., weaken R so that it no longer requires no perpendicular collisions. Then D, and in fact, S are unnecessary.

• 2. We could try to strengthen S, to make it unnecessary for drivers to obey traffic lights and vehicles to obey drivers.

The first approach is clearly not acceptable to society. So all that is left is the second approach. There are a number of ways that S can be strengthened, two of which are:

• (a) Have the traffic light pop a steel wall out of the street at the stop lines that face a red light.

• (b) Have the traffic light send a signal to all vehicles on the road that face a red light, a signal that causes the vehicles to stop.

Approach (a) will work, but has some other side effects that make it not a good idea. Among these side effects are that vehicles will collide with the steel wall and there will be more rear-end collisions. Approach (b) will work, but then there are some other assumptions needed about the domain:

D^′::

Vehicles obey traffic lights.

That D^′ is a lie would have to be permanently tolerated, just as for the the original D. As a society, we would probably prefer D^′ to the original D, because we know that properly constructed self-driving vehicles are in fewer accidents than are human drivers. However, we need to ensure that we are able to build vehicles that can be controlled by traffic lights. Of course, unless the vehicles are quite smart, and know how to slow down without hitting other vehicles, there will be an increase in rear-end collisions.

2.1 A.2 Aircraft

This example shows the ZJVF interplay applied to how the rare, but usually catastrophic collision between flying aircraft and flying birds is managed in aviation. Here too, the interplay is apparent even when very high level assertions are used for D, S, and R. We start with R and S in the hopes that just S is enough to entail R:

R::

A flying aircraft does not crash when it collides with birds in the air.

S::

An aircraft, made of aluminum, is able to achieve sufficient speed that it lifts off the ground and flies through the air.

Here too S is not strong enough to entail R, mainly because an aircraft built with aluminum and flying at flying speeds will break into pieces if it hits a bird in the air.

A domain assumption D that would allow D, S to entail R is:

D::

There are no birds in the air anywhere near any aircraft.

This D is not true, although the probability of its not being true is observably quite low, say ε.

If we try strengthening S to

S^′::

An aircraft is built with a thick enough fuselage that the aircraft will not break up in a collision with a bird.,

then S^′ will not entail the flying part of R. The aircraft will be too heavy to fly. So, if we insist that an aircraft both fly and not crash when it collides with a bird, and we cannot change the material of the aircraft, we are stuck. There is no way that D, S can entail R.

We could decide to permanently tolerate that D is not true, because the probability of its not being true is low enough. This permanent toleration is logically equivalent to changing R to state that an aircraft collides with a bird and crashes with probability less than ε, where ε is the low probability that D is not true. This toleration is exactly the state of aircraft building today. We as a society have decided to accept the risk of crashes for the benefits of flying.

If neither the change in R nor the toleration of the falseness of D is acceptable, another alternative is to find a metal that is super light and super strong so that an aircraft built with it both flies and does not break up when it hits a bird while flying.

2.2 A.3 Sluice gates

A more complete operationalization was proposed by Hayes et al. (2003) and Jones et al. (2007) in which the three elements, D, S, and R are played against each other to derive the specification S of a system that will meet a set of suitable requirements R in the context of the real-world environment that meets the domain assumptions D well enough. Here “suitable”, “real-world”ness, and “meet … well enough” are decided by the stakeholders of the system with respect to the real world as they see it. As the reader is able to consult the original reports, their operationalization is only summarized here.

Hayes, Jackson, and Jones’s idea is to take each requirement R1, and if R1 does not already hold in the environment, then to find a feature, specified by S1, combined with an assumption about the environment D1, such that D1, S1 ⊩ R1. This balancing act may require adjusting any of D1, S1, R1, and the part of the world that is considered to be the environment, to allow D1, S1 ⊩ R1 to hold. The balancing act may require adjusting any of D1, S1, and R1 also to allow each to be a conjunct in the global D, S, and R, respectively, for which D, S ⊩ R holds. The authors demonstrate their method by carrying it out to derive D, S, and R for a system to control the opening and closing of sluice gates (Hayes et al. 2003; Jones et al. 2007).

B Interview guide and questionnaire questions

B.1 Interview guiding questions

• 1. What is inconsistency?

• 2. Please provide several (at least 3) examples of inconsistencies.

• 3. What is inconsistency in requirements engineering?

• 4. Please provide several (at least 3) examples of inconsistencies in requirements engineering.

• 5. How do you feel about the inconsistencies you described in (2)?

• 6. How do you feel about the inconsistencies you described in (3)?

• 7. What do you think should be done in each example?

• 8. Imagine there is a case in which there is an inconsistency in requirements so that in rare cases the system will behave inconsistently, however resolving this situation would be of high cost. What would you do?

Interviewee’s Background Questions:

• 1. Education

• 2. Profession/occupation

• 3. Years of experience (if in different occupations, detail accordingly)

• 4. Current position

• 5. Experience with Agile development (y/n)

B.2 Questionnaire questions

• 1. What is inconsistency in requirements engineering?

• 2. Have you encountered inconsistency in your work? If so, give an example.

• 3. How do you feel about the inconsistency you described in (2)?

• 4. Give an example of a very severe inconsistency in RE.

• 5. Give an example of a less severe inconsistency in RE.

• 6. Give an example of a inconsistency in RE that can be tolerated (does not have to resolved).

Respondent’s Background Questions:

• 1. Education

• 2. Years of experience

• 3. Current position

• 4. Experience with Agile development (y/n)

C Participant Demographics

Tables 1 and 2 provide the demographics of the interview and questionnaire participants respectively.

Table 1 Interviewees

Table 2 Questionnaire respondents