Search and similarity based selection of use case scenarios: An empirical study

Abstract

Use case modeling is a well-known requirements specification method and has been widely applied in practice. Use case scenarios of use case models are input elements for requirements inspection and analysis, requirements-based testing, and other downstream activities. It is, however, a practical challenge to inspect all use case scenarios that can be obtained from any non-trivial use case model, as such an inspection activity is often performed manually by domain experts. Therefore, it is needed to propose an automated solution for selecting a subset of use case scenarios with the ultimate aim of enabling cost-effective requirements (use case) inspection, analysis, and other relevant activities. Our solution is built on a natural language based, restricted use case modeling methodology (named as RUCM), in the sense that requirements specifications are specified as RUCM use case models. Use case scenarios can be automatically derived from RUCM use case models with the already established Zen-RUCM framework. In this paper, we propose a search-based and similarity-based approach called S³RCUM, through an empirical study, to select most diverse use case scenarios to enable cost-effective use case inspections. The empirical study was designed to evaluate the performance of three search algorithms together with eight similarity functions, through one real-world case study and six case studies from literature. Results show that (1+1) Evolutionary Algorithm together with Needleman-Wunsch similarity function significantly outperformed the other 31 combinations of the search algorithms and similarity functions. The combination managed to select 50% of all the generated RUCM use case scenarios for all the case studies to detect all the seeded defects.

Appendices

Appendix 1

Table 14 provides all the abbreviations used in the paper.

Table 14 Abbreviations

Appendix 2

The encoding mechanism of the RUCM mutation operators (Zhang et al. 2016):

RUCM mutation operator = Operation, −, Element, −, {Defect}-;

Operation = ADD | DEL | SWAP | REP | ICR | DEC;

Element = UC | AR | INC | EXD | ASSO | UCS | UCN | ARN | PAR | SAR | BD | GA | GUC | SenBD | PreC | SenPreC | AF | SenAF | BF | SenBF | PostC | SenPostC | AS | RFS | ABORT | IFELSE | DO | VLD | MW |RFSsi | RFSflow | toABT | RES | toRES | IFELSE | IFELSEcs | IFELSEas | DOcs | DOas;

Defect = Defect Prefix, Defect ID

Defect Prefix = Defect Category, Defect Element;

Defect Category = C1 | C2 | C3 | C4 | C5 | C6 | C7 | C8 | C9;

Defect Element = AR | UC | R | H | F | UCM | UCS;

Defect ID = ‘1’|‘2’|‘3’|‘4’|‘5’|‘6’|‘7’|‘8’|‘9’;

UC = Use Case; AR = Actor; INC = Include; EXD = Extend;

ASSO = Association; PAR = primary actor; SAR = secondary actor;

UCS = use case specification; UCN = use case name;

UCM = use case model; BD = use case brief description;

ARN = Actor name; GAR = generalization between actors;

GUC = generalization between use cases;

SenBD = sentence in BD;

PreC = pre-condition; SenPreC = sentence in PreC;

BF = Basic Flow; SenBF = sentence in BF;

AF = Alternative Flow; SenAF = sentence in AF;

PostC = post-condition; SenPostC = sentence in PostC;

AS = Action step; VLD = VALIDATES THAT; MW = MEANWHILE;

RFS = sentence RFS; RFSsi = step index of RFS;

RFSflow = reference flow name of RFS;

ABORT = sentence ABORT; toRES = changing ABORT to RESUME STEP;

RES = RESUME STEP; toABT = changing RES to ABORT;

IFELSE = IF-SELSE-ENDIF; IFELSEcs = condition sentence in IF-ELSE-ENDIF; IFELSEas = Action Step in IF-ELSE-ENDIF;

DO = DO-UNTIL; DOcs = condition sentence in DO-UNITL;

DOas = Action step in DO-UNITL;

Appendix 3

In Table 15, we report, for each mutated use case, all the seeded defects and the number of use case scenarios generated from the mutated use case specifications. For example, as shown in the 2nd row of Table 3, MO1 (1) indicates that the mutation operator MO1 (i.e., ‘DEL-SenBF-C1F1’, Table 3) was used once to seed a defect to use case UC1 (Section 4.3.1.1) and 202 use case scenarios were automatically generated when all the defects listed in the 2nd column were seeded into the mutated use case specification of UC1.

Table 15 All manually seeded defects in the mutated use case specifications

Appendix 4

In this section, we provide detailed statistical test results for related RQs.

4.1 Results for RQ1

To answer RQ1, for each similarity function, we compared every selected search algorithm with RS in terms of FV and DDR respectively. In Table 16, we report the results for RQ1. The A>B category denotes the number of the optimization problems that one search algorithm performed significantly better than RS (denoted as A>B); A<B means vice versa; and A=B means the number of optimization problems for which there was no significant difference since p-value > 0.05.

Table 16 Vargha and Delaney statistic and Wilcoxon rank sum test in conjunction with Bonferroni Correction for RQ1 (at significance level of 0.05)

4.2 Results for RQ2

To answer RQ2, for each similarity function, we compared each pair of selected search algorithm in terms of DDR (Table 17) and FV (Table 18), respectively. Similar to RQ1, A>B denotes the total number of optimization problems that one search algorithm performed significantly better than the other; A<B means vice versa; and A=B means the number of optimization problems, for which there was no significant difference since p-value > 0.05.

Table 17 Vargha and Delaney statistic and Wilcoxon rank sum test in conjunction with Bonferroni Correction for DDR of RQ2 (at significance level of 0.05)

Table 18 Vargha and Delaney statistic and Wilcoxon rank sum test in conjunction with Bonferroni Correction for FV of RQ2 (at significance level of 0.05)

4.3 Results for RQ6

Table 19 presents the average running time for each combination of the similarity functions and search algorithms for each NUCS value.

Table 19 Average running time for each CSA (time unit: second)