Keywords

1 Introduction

Before an idea becomes a system or even a list of requirement specifications, an acting requirements engineer captures an initial description as set forth by the stakeholder. A stakeholder is defined as an individual that either affects or affects the system success [1]. Starting with the initial set of raw requirements, the engineer begins the long process of requirements analysis by modeling eliciting further requirements. It is important to emphasize the importance of this step in the development process, as a system cannot just be built with just an initial description. Implementing a system without a thorough analysis of the priorities, wants and needs of the stakeholder, will lead to the development team spending time building an incorrect system that the stakeholder will not accept.

Certain challenging issues are associated with requirements engineering. Requirements engineers may not have the necessary domain knowledge to understand what the stakeholder wants. Stakeholders may make implicit domain assumptions and fail to communicate those to the requirements engineer. The stakeholder may not fully understand the desired behavior of the system. If there are multiple stakeholders, an engineer may run into the problem of conflicting priorities.

Research has shown that it is more costly to correct defects later in the lifecycle of a system than eliciting and capturing the correct requirements earlier [2]. Before writing the requirements, a requirements engineer will model the behavior of the system to lead to a comprehensive understanding of the system in question. These models can come in all many different forms including a use case diagram, state chart, data flow diagram, etc., all serving their own role in defining different perspectives of a system. Currently, the only research done with eye tracking measures and UML diagrams has been a couple of studies using people at different levels of experience and measuring their workload as they interact with an existing diagram. These studies have showed that the layout of the UML diagrams can have an impact on readability thus making it less time consuming and cheaper to work with [3].

After creating a model of the system, engineers must verify the model for correctness by either having another engineer review it or build a prototype and validate with a stakeholder. This stage of development is where it is important for models to be in a readable layout so that other engineers can easily understand what is being captured. However, research has shown that this form of verification can be ineffective because looking at an existing model can be suggestive and stump the development of new ideas as suggestive memory has been proven especially in the field of law [4]. Therefore, the use of an objective measure can be more comprehensive. Thus, the purpose of this study is to define a new approach to verify use case scenarios in a way that is objective (i.e. non-human and automatic) and a technique that has proven to provide useful non-subjective information to user experience experts has been eye tracking.

This paper provides a technique that can be used as an unbiased review of use case scenarios. Once a requirements engineer creates a use case diagram modelling the relationships of different entities that interact with a system, the use case scenarios are built detailing every step that entities can make and the response the system gives. Based off the scenarios and information from other models, a prototype is built with low or high fidelity. Using the prototype and a scenario, a stakeholder or non-member of the design team can be guided through the use case model demonstrating where they expect to find the next step. Instead of measuring how many mistakes the individual makes or how long it takes them to complete the scenario, the purpose of going through the scenario with the prototype can be to capture where the individual is actually looking. Analysis of that information can be used to further elicit requirements such as identifying steps that should have alternative sequences or determining problems with the flow of the steps or major problems with the design of the prototype from the very beginning. The proposed approach can be used to both facilitate the further elicitation of requirements using eye-tracking techniques and validate prototypes in tandem with use case scenarios.

2 Background

Eye tracking is a method that has been around for over 100 years. The beginning of eye tracking started with a standard camera with users going frame by frame to document the movements of the eye. Nowadays, eye trackers have adapted to capture a high rate of frames and automatically analyze data and present it in charts.

2.1 Eye Tracker Data Collection

An eye tracker collects information about where a participant is looking in an area by tracking the scanpath, movement and fixation. A scanpath for the purpose of this study is defined as the sequence of eye movements across a page, and a fixation is defined as “when a gaze remains within a small area for a given time” [5]. The method by which it collects this information depends on the type of eye tracker used, but the information that it collects is standard across all devices. The raw data that an eye tracker collects consists of the eye movement of the participant as well as the size of the pupil. The eye movement is translated to an x/y coordinate on the screen while the fidelity of the eye trackers determines how much movement is collected [6]. The higher the fidelity of the eye tracker the higher the sample rate by which the eye tracker collects. Typical sampling rate ranges from 30 Hz – 250 Hz although the range of 50 Hz to 60 Hz is the norm for usability studies [6]. It is important to determine which sampling rate will be used for a study, as higher sampling rates will out put more data and require more data reduction when reviewing participant data. The data reduction and post processing of the data depends on the eye tracker used, data can be visualized as heat or focus maps, gaze plots, and aggregated gaze plots. Heat maps compile groups of eye tracking gaze plots and visualize them into a visual light spectrum from red, representing a heavy traffic gaze area, to blue, representing a light traffic gaze area [7]. Gaze plots are a visualization of a scanpath for a user [8], and aggregated gaze plots are scan patterns aggregated from many users viewing the same visual stimulus [9]. Each visualization provides specific in-formation about a scanpath and fixation. However, these visualizations are mean-ingless unless backed with a specific research question.

2.2 Usability Studies Usage

Eye tracking is a popular method to supplement usability testing, however it should only be used when appropriate. The Neilson Norman group recommends that one should only use eye tracking after about one hundred rounds of regular usability testing [10]. The reasoning behind this belief is that eye tracking requires specific research questions to gather meaningful data and requires supplemental usability testing methods to make sense of the gaze plot data. In addition, the financial, labor [11], and time obligations are significantly greater than the majority of the usability testing methods. The two most popular usability testing methods to supplement eye tracking in usability studies are think aloud protocols, where a participant is asked to “verbalize whatever crosses their mind during the task performance” [12].

The approach presented in the paper uses the video-based combined corneal reflection eye tracking method supplemented by a retrospective think aloud protocol. The video-based combined corneal reflection method is one of the most widely used because it offers point of regard tracking, where the eye movement is distinguishable from the head movement of a participant, without any intrusive head stabilizing techniques. The approach was supplemented with a retrospective interview to prevent any confirmation bias that may have occurred during data analysis as well to extrapolate possible reasons behind the participants thought process and actions.

2.3 Use Case Model and Scenarios

A use case model is a representation of the desired behavior in a system the interactions with the surrounding environment [13]. Use case models are composed of actors and use cases, surrounded by a system boundary drawn as a box around the system. Use cases represent an abstract view of the system and its interactions with the external entities. These external entities, which can be human users, organizations or other systems, are depicted as actors. Lines associated between actors and use cases represent interactions of value to the actors by the system. The use case model facilitates the elicitation of functional requirements by providing an abstraction of the main uses of a software system and the actor that will interact with the system, enabling a way for requirement engineers to analyze and identify the functional needs. This assists in the analysis of what information these actors will provide or receive from the system thus highlighting requirements about interfaces. Additional relationships are used such as includes and extends to model relationship between the use cases, which signify use case inclusion and optional extension to use cases, respectively.

A scenario describes the specific exchange of information as a flow of interactions between the actors associated with the use case and the system. The steps include alternative flows that can be taken while executing the main flow. The in-depth analysis of a use case happens when writing a scenario, the use case and actor provide a general abstraction of what the actor comes to the system to do, but the requirements of what the system must rise up from writing the scenario text. Each use case can have one or more scenarios as documentation and because they are simple text, stakeholders can use the scenarios to validate the elicited behavior. In addition, developers can use scenarios to build other analysis models, create UI prototypes, walkthrough design and implementations, and generate test cases.

3 Related Work

In addition to all the current uses of eye tracking as defined above, eye tracking is being applied across multiple disciplines to find all the different applications. For example, a lab just recently, in 2016, proposed a relationship between baseline pupil size and intelligence along with other labs that have done research with applications that include an engineer’s understanding of a technical drawing [14, 15]. Eye tracking is becoming more common even though it is not the most reliable measure mainly because eye tracking is mainly nonintrusive. Most current measures used for things like situational awareness and workload are measures that interrupt the task and require verbal or written information like the Situational Awareness Global Assessment Technique (SAGAT) and National Aeronautics and Space Administration Task Load Index (NASA-TLX) [16, 17]. There have been a couple studies done with eye tracking and UML models. However, unlike the focus of this study where the design is specific to a specific model, the previous studies were fixed on the overall design qualities to reduce workload [18,19,20,21].

3.1 Model Validation

Validating the models created is an important step in the process of defining the correct requirements for a system. Models capture the current understanding of the expected behavior of the system, thus a model needs to be as correct as possible to mitigate specifying incorrect requirements. Traditional approaches in the validation of models include the application of reviews, specifically two types of reviews can be used, inspections and walkthroughs. A review aims at examining the model created, looking for flaws, inconsistencies, or omissions that can be corrected, and usually done by requirements engineers and stakeholders [22]. An inspection is a formal review that follows a well-defined set of steps, in which each participant is assigned a specific role in the review [23]. A walkthrough is an informal type of review, i.e. it does not follow a pre-define process and participants do not have a specific role [24]. These techniques have been shown to be effective at detecting defects in requirements specifications [25,26,27,28].

4 Approach

As stated in Sect. 2, use case diagrams are a specific type of UML model that maps out the different actors and their interactions with the system. The visualization of these interactions contributes to the development of the use case scenarios by defining functionality that needs to be accessible for a user to be satisfied with the use. Use case scenarios are what drives the development of a prototype especially in a computer system and are what the design team goes through when demonstrating a prototype to a stakeholder. For this study, the scenario and prototype are based off a system that was defined and designed for a graduate-level requirements class modeled and specified. The students were offered a description for the desired program named Sharing and Discovering Semantic Use Case Scenarios (SADSUCS); below is an excerpt of that description:

figure a

The use case scenario and prototype were derived from those completed in the class. A sample scenario used for the study is shown below:

figure b
figure c

The prototype was built using the elicited use case scenario using the wireframing program, Axure [29]. The program used in this study was derived from the ones built in the class. Sample screenshots of the prototype can be found in the section below accompanied by the gaze plot data.

4.1 Eye-Tracking Study Setup

The individual selected to part of this study was an undergraduate female between the ages of 18 to 25 that did not require the use on contacts or eyeglasses. The participant sat in front of the Tobii eye tracking equipped computer at Embry-Riddle Aeronautical University’s Usability Lab. After explaining the purpose of the pilot study, to validate a framework using the eye tracking information, the eye tracker was calibrated to the participant. The participant was given the task to find a public UML diagram and allowed to navigate the prototype with little guidance. Guidance and answers were provided when the participant did not know how to continue or had a question about how to proceed; however, the participant was encouraged to navigate the system in self-guided approach.

5 Results and Observations

The eye-tracking test combined with the retrospective think aloud provided insight into what the participant was thinking and doing during their completion of the task. The home page provides little information about the site, and by reviewing the scanpath of the participant, it becomes apparent that the main area of interest is the paragraph text, contact, and about feature. The reasoning behind this scanpath is explained by the participant in the interview, where they stated to “not know what the purpose of the site is” and did not know how to continue. The task of logging in to the site is then considered a failure, as they needed assistance in continuing.

The eye tracking analysis of the dashboard page is broken up into two phases. The first is the participants natural scan for information in scan plots one through ten, and a second phase of when the participant asked to be reminded of the task in scan plots eleven through twenty. In the interview, the participant noted that they were confused as “nothing on the page said diagram.” This was deemed as a failure of the task to find public use diagrams.

The public projects page has a strong indication of an area of interest in the use cases panel. Seventeen out of the twenty-three (74%) total gaze points are focused in this area (see Fig. 1). When asked in the retrospective interview why they were focused on the area they responded with the fact that they were told to search for a use case diagram, however after being unable to interact with any of them they the participant decided to move to the search function. This is considered a pass of the task to search for a public use diagram, and a failure on part of the task for not being specific enough.

Fig. 1.
figure 1

Use case panel

Full size image

The final page the eye tracker captured was the use case diagram associated with this task. Again, an obvious area of interest was identified and is located in the use case diagram where twenty-eight of the thirty-seven (76%) of the gaze points were located. In the interview, the participant suggested they were trying to figure out if this was the end of the task or if they had to continue through the site.

5.1 Modification Based off Results

Using the information gathered from the eye tracker, the use case scenario can be revised to provide a more comprehensive use case scenario. In effect, this was a new alternative way to elicit requirements about the system. For example, the screenshot below was part of the prototype that was used along with an overlay of the gazeplot gathered by the participant. This specific screen involves steps 4 and 5 as follows (Fig. 2):

figure d
Fig. 2.
figure 2

User dashboard

Full size image

Using the data from the gazeplot above, after the system displays the diagrams, the user looks at the different use cases, the dashboard option, the create a new project option, and browsing the public projects option. Using this information, there should be 3 different alternative use cases that should be added to step 5 as followed:

figure e

As a result, the proposed approach was used to further elicit requirements using eye-tracking techniques. The participant acted as a validation for the scenario, which was implemented indirectly by the prototype. With the data gathered from the eye-tracker, this technique was successfully used to modify the use case scenario and provide requirement validation on the modeled system, identifying missing functionality. The initial results show promise that this technique can be used in use case scenario validating through eye tracking of prototypes.

6 Future Work

The purpose of this study was to provide a description of a new approach to validate requirements via scenarios and conduct a pilot study. It is important to note that only one participant was guided through the prototype and issues can be found with the data that was collected. For example, since there was only one data set, things like losses of calibration could not be accounted for. Thus, more research should to develop a framework or set of standards that a design team could use with ease without understanding completely the concept of eye tracking and user-based testing.

Mouse tracking is another usability testing method that can be used both independently and supplementary to eye tracking. Research indicates that a participant’s gaze leads the movement of their mouse movements [30], and it is a common practice to use mouse tracking when eye tracking is not a viable option. These methods are most useful when used in combination, as in usability the click and clickstream of a user’s navigation through an interface [31]. The incorporation of mouse tracking into the proposed approach could provide more accurate information, however it might provide an average user with too much information and thus possibly making it.