Elsevier

Knowledge-Based Systems

Volume 13, Issue 4, 10 June 2000, Pages 215-224
Knowledge-Based Systems

Notation and nature of task in comprehending design rationale

https://doi.org/10.1016/S0950-7051(00)00072-1Get rights and content

Abstract

This paper provides the elements of a ground theory on how design rationale can be effectively re-used across design generations. In particular, we explore the role of notation, argumentative structure and task conditions under which comprehension in that context can be facilitated. An experiment is set up in order to investigate whether and under which task requirements visual formalisms outperform text and whether Vessey's ‘cognitive fit’ argument stands true in decision-making activities related to the design of interactive systems. Results confirm that visual formalisms enhance the comprehension of design argumentation, and in particular, tables facilitate quicker access to meaningful information. There are also indications that complex argumentation schemata that are depicted graphically are acquired quicker by novices. We then lay out a framework of the notational and task conditions under which comprehension of design rationale by novices is optimised.

Introduction

Our work is concerned with assisting the software design process by looking at ways of transferring expertise among design stakeholders. Considering the large flow of personnel and the need for more efficient project management, it is a common belief that communication of design material and better decision-making are two processes of high importance for the software industry. Design rationale (DR) has been around for some time, and its suitability for that role has been advocated before [1]. However, successful stories of its application to real-world problems have been scarce. One reason is that there has been no underlying theory or adequacy of examples of the use of DR in practical design problems. In this paper, we present a first attempt to set out the laws that underlie the comprehension of DR. We hope to increase understanding on the matter and provide guidance and encouragement to practitioners for trying out complementary techniques to communicate design material.

In loose terms, DR refers to the argumentation that underlies the decisions made during design. For the purposes of this paper we adopt part of Moran and Carroll's collective definition [1] “Documentation of (a) the reasons for the design of an artefact, (b) the stages or steps of the design process, (c) the history of the design and its context”. In this paper, we have chosen the QOC formalism [2] as a platform on which to test different representational aspects of DR. QOC is a DR language whose basic elements are Questions, Options and Criteria. Questions stand for issues that come up during the design task, Options stand for alternative answers to the question at hand and Criteria are meant to be the objectives under which each option is evaluated for fitness. Criteria can be assessed either positively or negatively, leading to a decision. The choice of QOC is based on the QOC's appropriateness to user interface design and related tasks as illustrated in Ref. [2].

On the one hand, Pennington's work [3] as well as the extensions by Navarro-Prieto and Canas [4] and on the other hand, Detienne [5] have established a sound theoretical background for the comprehension of the code by experienced programmers. In particular, Pennington has developed a two-stage theory of program comprehension based on mental models. According to that theory, initially expert programmers build a mental representation based on control flow relationships between code segments; when and if the task requires so, they build a data flow mental model of the functions of the program. In computer programs, the flow of data corresponds to domain knowledge or the purpose a program is meant to fulfil. In our case, we are looking for similar abstractions that underlie the comprehension of DR as well as the factors that determine their construction.

There have been numerous studies that have examined the role of external representation and task in problem solving performance—a good example from the study of programming can be found in Ref. [6]. The right combination of problem representation and task conditions can yield optimum performance. In terms of appropriateness of problem representation, Navarro-Prieto and Canas have showed that visual languages facilitate the construction of data flow models even in easier tasks. On the comprehension task front, Detienne has elaborated on the tasks that relate to program comprehension, by differentiating between “read-to-recall” and “read-to-do” tasks and establishing their influence on the generated mental models. To that end, Vessey [7] has noted the importance of the fitness between the problem representation and the nature of task in problem solving. In aid of that argument, she provides plenty of information that can help one to distinguish between types of tasks in order to spot the “fitting” problem representation and problem solving strategy.

These theories are used as a basis on which to build a similar framework for the comprehension of DR. We envisage the extension of these theories to accommodate the specifics of argumentation and the decision making process in design, alongside control and data flow abstractions that are employed when programmers read the code.

A piece of program code or specification includes information of the design product whereas DR provides information on the design process. The benefits from understanding DR comprehension in relation to program comprehension is to find ways of structuring the material to be read in an efficient way so that assimilation and subsequent problem-solving performance can be improved. As a starting point to our pursuit we investigate the role of notation and structure in the comprehension of DR by novice analysts/designers. In particular, we are putting together a task scenario of DR re-use—focussing on the comprehension part—and we are concerned with: (a) the type of notation that is most suitable to a typical design problem, and (b) the role of the structure of the argumentation material in such a context. Can we gather any information out of it about the structure of reasoning that has taken place?

Section snippets

DR notation

DR comes in different forms. Most approaches have adopted a graphical form, starting off from the IBIS paradigm [8]. Buckingham Shum and Hammond [9], among others, have put forward tabular forms, whereas others including Fischer [10] often use a narrative one. In most current systems, which are predominantly based on hypertext or hypermedia, different representations are combined. Although the majority of those systems provide a high-level graphical map with a zooming facility, and a link to

Task materials

Subjects were asked to read DR fragments and carry out tasks related to the re-use of these fragments. The stimulus material was drawn out of the Fast Automatic Teller Machine (FATM) problem included originally in Ref. [2]—a sample is shown in Fig. 1(a)–(c).

A set of eight issues were given, of which five were related—two were siblings implied by a third one, and another one was a child of a fifth. The eight DR fragments were laid out on two sides of an A4 paper, and subjects had to browse

Representation

As no significant effect was caused on the percentage of correct answers by any type of representation, we are thoughtful of the sensitivity of correct answers as a dependent variable. Its high reliance to the phrasing of task questions makes it intricate and the importance of how the questions are put in this type of tests is stretched in this paper. Vessey [7] also reports inconclusive results for the same reason, in a comprehension experiment. However, there is a consistent dominance of

Related work

In this section, we discuss DR creation and volume of argumentation. They are two issues not directly covered by this experiment, though very interesting and of high practical value. Before reviewing certain empirical studies, we briefly point out the key features of the main DR approaches from the point of view of creation or task, in general. Out of those studies, we discuss issues of volume of argumentation. Note that all DR approaches have been designed to serve specific purposes and with

Conclusions and future work

In this paper we have outlined a set of task conditions that facilitate the comprehension of DR. The implications of these results can have a major impact on design practice in the software industry. By setting out the task framework in this paper, we attempt to encourage practitioners to use additional documentation, like DR, for tasks where knowledge transfer is the key issue. Firstly, by establishing the representational attributes that underlie comprehension of DR, we can focus on certain

References (19)

  • N. Pennington

    Stimulus structures and mental representations in expert comprehension of computer programs

    Cognitive Psychology

    (1987)
  • D. McKerlie et al.

    Reasoning with design rationale: practical experience with design space analysis

    Design Studies

    (1994)
  • T.P. Moran et al.

    Overview of Design Rationale

  • A. MacLean et al.

    Questions, options, and criteria: elements of design space analysis

    Human-Computer Interaction

    (1991)
  • R. Navarro-Prieto, J.J. Canas, Mental representation and imagery in program comprehension, Proceedings of the 11th...
  • F. Detienne, What model(s) for program understanding?, Proceedings of UCIS'96, Pointers, France, September...
  • D. Gilmore

    Methodological considerations in the study of programming

  • I. Vessey

    Cognitive fit: a theory-based analysis of the graphs versus tables literature

    Decision Sciences

    (1991)
  • The IBIS manual: a short course in IBIS methodology, 1996, at...
There are more references available in the full text version of this article.

Cited by (5)

  • Knowledge evaluation in product lifecycle design and support

    2014, Knowledge-Based Systems
    Citation Excerpt :

    These tools are useful to study the traceability of product design and foresee future developments [8]. Design rationale can be effectively re-used across design generations [17]. Tang et al. [27] have introduced a rationale-based architecture model (AREL) to capture design rationale and help people understand architecture design.

  • THE THEORY OF COGNITIVE FIT: One Aspect of a General Theory of Problem Solving?

    2015, Human-Computer Interaction and Management Information Systems: Foundations
  • Thinking and representing in design

    2005, Design Process Improvement: A Review of Current Practice
  • Interoperability for GIS document management in environmental planning

    2005, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
View full text