Empirically evaluating OCL and Java for specifying constraints on UML models

Abstract

The Object Constraint Language (OCL) has been applied, along with UML models, for various purposes such as supporting model-based testing, code generation, and automated consistency checking of UML models. However, a lot of challenges have been raised in the literature regarding its applicability in industry such as extensive training, slow learning curve, and significant effort to use OCL due to lack of familiarity of practitioners. To confirm these challenges, empirical evidence is needed, which is severely lacking in the literature. To build such preliminary evidence, we report a controlled experiment that was designed to evaluate OCL by comparing it with Java; a programming language that has also been used to specify constraints on UML models. Results show that the participants using OCL perform as good as the participants working with Java in terms of three objective quality metrics (i.e., completeness, conformance and redundancy) and two subjective metrics (i.e., applicability and confidence level). In addition, the participants using OCL performed consistently well for all the constraints of varying complexity, while fluctuating results were obtained for the participants using Java for the same constraints. Based on the empirical evidence, we can conclude that it does not make much difference to use OCL or Java for specifying constraints on UML models. However, the participants working with OCL performed consistently well on specifying constraints of varying complexity suggesting that OCL can be used to model complicated constraints (commonly observed in industrial applications) with the same quality as for simpler constraints. Moreover, additional analyses on the constraints when using Java and OCL tools revealed that tools are needed to specify fully correct constraints that can be used to support automation.

Appendices

Appendix A: Answer sheet of the Banking System case study

In this Appendix, we present the answer sheet that was provided to the students during the controlled experiment for Banking System, including the instruction, the system description, the class diagram that OCL constraints should be applied on, and the 10 constraints written in English.

1.1 A.1 Instruction for specifying constraints using OCL

Check the provided system description and the class diagram, and specify the given constraints using OCL with the following format:

$\mathtt{\textit{context Bank} }$

            $\mathtt{\textit{inv:} }$

1.2 A.2 Instruction for specifying constraints using Java

Check the provided system description and the class diagram, and specify the given constraints using Java. All the attributes in the class diagram are public, which means that they can be directly accessed with objects. Please provide the specification of each constraint as the body of the following function:

$\mathtt{\textit{public boolean constraint (Bank b)} }$ {

}

1.3 A.3 System description

This case study is an extended version of Banking System case study from the OCL 2.2 specification. A bank has several employees and customers. Each customer can have at most two accounts in a bank: One is saving account and the other is current account. A customer must be employed in a company or owns a company in order to have a bank account. An employee of a bank can also be its customer having accounts in the bank.

1.4 A.4 Class diagram

Figure 3 shows a class diagram modeling the Banking System. Table 13 provides the description of each attribute of a class in the class diagram.

Fig. 3

Class diagram for Banking System

Table 13 Description of each attribute

1.5 A.5 Constraints

• 1. All customers of the bank must be employed.

• 2. All customers and employees of the bank must be 18 years or older.

• 3. If all the accounts of a customer have balance less than or equal to 0, then these accounts should be all closed.

• 4. A customer is either employed in a company or owns his/her company.

• 5. All accounts in the bank must have unique account numbers and each account must be linked to exactly one customer.

• 6. Each customer of the bank either owns at least a company or work in a company that has more than one employee.

• 7. The bank does not allow their customers to withdraw money from their saving accounts.

• 8. In the bank, there should be gender equality in its employees, i.e., the number of male employees should be equal to the number of female employees.

• 9. An employee of the bank must not work in another company, but may own a company with exact one employee.

• 10. A customer of the bank can only have a saving account when he/she has a current account but not vice versa.

Appendix B: Answer sheet of the Video Conferencing System (VCS) case study

In this Appendix, we present the answer sheet that was provided to the students during the controlled experiment for Video Conferencing System, including the instruction, the system description, the class diagram that OCL constraints should be applied on, and the 10 constraints written in English.

1.1 B.1 Instruction for specifying constraints using OCL

Check the provided system description and the class diagram, and specify the given constraints using OCL with the following format:

$\mathtt{\textit{context Bank} }$

            $\mathtt{\textit{inv:} }$

1.2 B.2 Instruction for specifying constraints using Java

Check the provided system description and the class diagram, and specify the given constraints using Java. All the attributes in the class diagram are public, which means that they can be directly accessed with objects. Please provide the specification of each constraint as the body of the following function:

$\mathtt{\textit{public boolean constraint (Saturn s)} }$ {

}

1.3 B.3 System description

Our case study is part of a project aiming at supporting automated, model-based testing of a core subsystem of a video conferencing system (VCS) called Saturn. The core functionality to be modeled manages the sending and receiving of multimedia streams. Audio and video signals are sent through separate channels, and there is also a possibility of transmitting presentations in parallel with audio and video. Only one conference participant can send presentations at a time and all others receive it.

1.4 B.4 Class diagram

The functional behavior of Saturn consists of a set of class diagrams and a set of UML state machines. An excerpt of class diagram for Saturn is provided in Fig. 4. The UML class diagram is meant to capture information about APIs and system (state) variables, which are required to generate executable test cases in our application context. In this figure, however, we do not show APIs, since we do not need them in this context. Figure 5 is also a class diagram for Saturn capturing various configuration parameters. The Saturn class in Figs. 4 and 5 is the same, and we present two separate class diagrams for the purpose of clarity. Table 14 shows the description of each attribute.

Fig. 4

Class diagram for Saturn (Part I)

Fig. 5

Class diagram for Saturn (Part II)

Table 14 Description of each attribute

1.5 B.5 Constraints

• 1. Saturn should be either in SIP mode or H323, but not in both modes at the same time.

• 2. Number of active calls for Saturn ranges from 0 to 3.

• 3. When Saturn is presenting, the presenter’s ID should be a non-negative integer.

• 4. For each call, call item should be a non-negative integer and call number shouldn’t be an empty string (“”).

• 5. For each call, call item and number should be unique.

• 6. In H323 mode, rate adaption and forward error correct mode should be enabled at the same time.

• 7. When Saturn is in SIP mode then all the information related to SIP protocol shouldn’t be empty.

• 8. When Saturn is presenting, protocols of all the outgoing presentation channels should not be off.

• 9. When Saturn is receiving presentation, exactly one of input presentation channels should have protocol not equal to off.

• 10. When Saturn’s presenting or receiving presentation, exactly one of the presentation channels should have protocol not equal to “Off”.

Appendix C: Post-questionnaire

Please put a $(\surd )$ in the corresponding column. You are strongly encouraged to refer to the list of constraints you were provided with. (The constraint numbers in the following tables match the numbers provided in the list of restriction specifications).

A: Completely agree; B: Generally agree; C: Generally disagree; D: Completely disagree

Appendix D: Pre-questionnaire

Levels of agreement: Strongly agree, Agree, Neither agree nor disagree, Disagree, and Strongly disagree

Likert Scale Questions:

• 1. I have good knowledge on UML class diagram modeling.

• 2. I have good knowledge on writing OCL expressions.

• 3. I have good knowledge on programming using Java.

Open Questions:

• 4. How many courses have you taken that taught UML? What are these courses?

• 5. How many courses have you taken that taught OCL? What are these courses?

• 6. How many Java-programming projects have you conducted in the past? What are they?

• 7. How many OCL and UML related courses have you conducted in the past? What are they?

• 8. What other kinds training on UML, OCL, or Java have you received in the past?

Appendix E: Examples of OCL and Java constraints

In this Appendix, we provide some examples of OCL and Java constraints taken from the answer sheets submitted by the students during the controlled experiment. We also provide the evaluation results based on the metrics defined in Sect. 2.4.1.

1.1 E.1 OCL, Banking System

Banking System:

Constraint A: All customers of the bank must be employed.

Student A:

Context Bank

inv: self.customer-$>$ iterate(c:Customer ${\vert }$

c.isEmployed = true)

Completeness = 1, Conformance = 1, and Redundancy = 0

Student B:

Context Bank

inv: for each self.employee.isEmployed = true

Completeness = 1, Conformance = 0.67, and Redundancy = 0

Constraint B: All accounts in the bank must have unique account numbers and each account must be linked to exactly one customer.

Student A:

Context Bank

inv: self.amount-$>$ isUnique(account Number) and self.account.customer-$>$ size() =1

Completeness = 1, Conformance = 0.83, and Redundancy = 0

Student B:

Context Bank

inv: if for all self.account.account Number is unique and for each self. account.account

Completeness = 0.5, Conformance = 0.25, and Redundancy = 0

Constraint C: A customer of the bank can only have a saving account when he/she has a current account but not vice versa.

Student A:

Context Customer::Saving:Saving

inv: self.current-$>$ size() $>$ =1

Completeness = 0.25, Conformance = 0.097, and Redundancy = 0

Student B:

Context Customer

inv: if self.current.size() $>$ 0 then self.saving.size() $>$ 0

Completeness = 0.5, Conformance = 0.11, and Redundancy = 0

1.2 E.2 OCL, Video Conferencing Systems

Constraint D: Saturn should be either in SIP model or H323, but not in both modes at the same time.

Student C:

Context Saturn

inv: (self.networkService.SIP_Mode = On and self.networkService.H323_Mode = Off) or (self.networkService.SIP_mode = Off and self.networkService.H323_Mode = On)

Completeness = 1, Conformance = 0.75, and Redundancy = 0

Student D:

Context Saturn

inv: self.h323-$>$ size() + self.sip-$>$ size() =1

Completeness = 1, Conformance = 1, and Redundancy = 0

Constraint E: For each call, call item and number should be unique.

Student C:

Context Saturn

inv: self.conference.calls-$>$ isUnique (callItem) and self.conference.calls-$>$ isUnique(number)

Completeness = 1, Conformance = 0.33, and Redundancy = 0

Student D:

Context Saturn

inv: self.conference.calls-$>$ isUnique (CallItem) and self.conference.calls-$>$ isUnique(number)

Completeness = 1, Conformance = 0.33, and Redundancy = 0

Constraint F: When Saturn’s presenting or receiving presentation, exactly one of the presentation channels should have protocol not equal to “Off”.

Student C:

Context Saturn

inv: OutgoingPresentationChannel:: Protocol $\qquad <> \qquad$ Off or Incoming PresentationChannel::Protocol $<>$ Off

Completeness = 0.5, Conformance = 0.28, and Redundancy = 0

Student D:

Completeness = 1, Conformance = 1, and Redundancy = 0

1.3 E.3 Java, Banking System

Constraint A: All customers of the bank must be employed.

Student C:

Completeness = 1, Conformance = 1, and Redundancy = 0

Student D:

Completeness = 1, Conformance = 0.67, and Redundancy = 0

Student D:

Completeness = 1, Conformance = 1, and Redundancy = 0

1.4 E.4 Java, Video Conferencing System

Constraint D: Saturn should be either in SIP model or H323, but not in both modes at the same time.

Student A:

Completeness = 1, Conformance = 1, and Redundancy = 0.5

Student B:

Completeness = 1, Conformance = 1, and Redundancy = 0

Constraint E: For each call, call item and number should be unique.

Student A:

Completeness = 1, Conformance = 0.83, and Redundancy = 0

Student B:

Completeness = 1, Conformance = 0.83, and Redundancy = 0

Constraint F: When Saturn’s presenting or receiving presentation, exactly one of the presentation channels should have protocol not equal to “Off”.

Student A:

Completeness = 0.5, Conformance = 0.5, and Redundancy = 0.5

Student B:

Completeness = 1, Conformance = 1, and Redundancy = 0.5