Developing comprehensive acceptance tests from use cases and robustness diagrams

Abstract

In agile development processes, the rewards from acceptance testing are maximized by using the practice to drive the development process. Traditionally, User Stories are used in agile projects to describe a system’s usage scenarios and are utilized as a basis for developing acceptance tests. This paper introduces a technique that aims to achieve the benefits of acceptance testing within large-scale development projects that deploy a V-model development process, specifically those that utilize use case models. The approach is based on utilizing a number of artifacts: use case models supported by robustness diagrams and domain models. The feasibility of the proposed approach is demonstrated by applying it to a real-world system—the RestoMapper system. The results show that a comprehensive set of acceptance tests can be developed based upon use case models.

Appendices

Appendix 1: syntax for creating ColumnFixtures and RowFixtures

A ColumnFixture is most suitable for checking rules and calculations—basically input/output relationships in the absence of state information. For a given business logic, a series of input values are provided and the resulting outputs are checked accordingly. The fixture shown in Table 14 tests the return on investment calculation, which is used to test the behavior described by the “Calculate Investments” UC.

Table 14 ColumnFixture example of calculating the return on one year investments

RowFixtures are more suitable for checking sets of data. Rows in the test data are compared to the contents of objects under test. Test data are used to examine an object under test for any missing or surplus (unexpected) data sets. The test shown in Table 15 is also relative to the behavior described by the “Perform Transactions” UC and it evaluates the transaction log after the transactions performed in the test presented by the ActionFixture shown in Table 16. FIT uses the fixtures created to generate code skeletons for later implementation.

Table 15 RowFixture example of checking account activities

Table 16 ActionFixture example of performing transactions

When fixtures run, the data returned by the software under test are compared against the values provided in the tables. Test results are indicated by color-coding the cells containing the expected data (cell containing output). For example, running the RowFixture shown in Table 15 will result in coloring the data fields of “Balance” column. The following is the list of color codes for test results as defined in [19]:

• Green: The software returned an expected value.

• Red: The software returned an incorrect value.

• Yellow: The software caused an exception to be thrown.

• Grey background: The cell was ignored for some reason.

• Grey text: When cell is left intentionally blank by the user, FIT will fill in the answer from your software.

Appendix 2: detailed robustness analysis of the featured UCs

2.1 Traced paths through the robustness diagram of the “generate restaurant map for city” UC

2.1.1 Basic flow:

• 1. Get restaurants for current city with valid zoom setting

  • 1.1. The MapViewer interface is provided with the current city and the zoom setting by the “Generate City” UC or the User.

  • 1.2. MapViewer invokes “get restaurants for city” given the current city and the zoom setting

  • 1.3. “get restaurants for City” then passes these parameters to “map scale is OK?” to check the validity of the zoom setting.

  • 1.4. Given the validity of the zoom setting, “map scale is OK?” then passes on the two parameters to the RestaurantServer

  • 1.5. RestaurantServer invokes MapServer to get the restaurant layer

  • 1.6. MapServer returns the Restaurant layer to RestaurantServer

  • 1.7. RestaurantServer then return the queried restaurants to “map scale is OK?

  • 1.8. “map scale is OK?” then returns the restaurants to “get restaurants for city”

  • 1.9. “get restaurants for city” then creates the RestaurantCollection

Inputs: Current City, Zoom setting

• 2. Get map

  • 2.1. The MapViewer interface is provided with the current city and the zoom setting by the “Generate City” UC.

  • 2.2. MapViewer invokes “get map for City” and provides it with the two external parameters

  • 2.3. “get map for City” passes those two parameters to the MapServer which return the required map data that can be used to build the map

  • 2.4. “get map for City” uses the returned map data to build a Map

• 3. Add restaurant icons to map

  • 3.1. “add Restaurant icons to map” retrieves information about the Restaurants to be displayed as icons from the RestaurantCollection

  • 3.2. “add Restaurant icons to map” retrieves information about the desired display filters from the DisplayFilter

  • 3.3. “add Restaurant icons to map” add the qualifying Restaurants as icons by editing the Map entity object

Inputs: DisplayFilter

Outputs: Edited Map entity object containing Restaurant icons

2.1.2 Alternative flow: invalid zoom setting

• 1. Get restaurants for current city with invalid zoom setting

  • 1.1. The MapViewer interface is provided with the current city and the zoom setting by the “Generate City” UC or the User.

  • 1.2. MapViewer invokes “get restaurants for city” given the current city and the zoom setting

  • 1.3. “get restaurants for city” then passes these parameters to “map scale is OK?” to check the validity of the zoom setting.

  • 1.4. “map scale is OK?” determines that the given zoom setting is invalid invokes the “Show Invalid Zoom Popup” control object.

  • 1.5. The “Show Invalid Zoom Popup” object then creates an interface object “Invalid Zoom Popup” to display a message prompting the User change the zoom setting.

Inputs: Current city, zoom setting

Outputs: Invalid zoom popup

Appendix 3: analyzing UC display rollover information

The “Display Rollover Information” UC extends the “View Map” UC to provide additional information to the “User” about restaurants displayed in the map”. The additional information are presented in the form of a popup and is only presented when the “User” points to a restaurant icon or clicks on it with the mouse cursor. The UC description is shown in Table 17.

Table 17 Textual description of the display rollover information UC

3.1 Examining the UC description and creating its HLATs (Phase 1)

A common precondition for all flows is that the map is displayed. For the Basic Flow to be performed successfully, the map must contain at least one restaurant icon. In order to perform the Alternative Flow: Clicked on coordinates with multiple restaurants, where the “User” clicks on coordinates with multiple restaurant icons, it is required that the map contain at least two restaurant icons located at the same coordinates. Finally, for the Alternative Flow: Clicked on coordinates with no restaurants, where a “User” clicks on nothing, it is not necessary for the map to contain any restaurant icons. The set of HLATs created are shown in Table 18.

Table 18 HLATs for the display rollover information UC

It is important to note that the output of the Alternative Flow: Clicked on coordinates with no restaurants indicates that nothing should happen in response to a mouse click. However, how is it possible to verify that nothing happened? From a Software Testing perspective, to verify that nothing has happened, certain data values need to be checked in order to determine that the system state has not changed. Therefore, an infinite number of tests to explore an infinite number of scenarios will be required to ensure that the system behaves correctly, which is infeasible and unpractical. Therefore, it is a judgment call as to how many tests should be created, and which specific scenarios they should address.

3.2 Robustness analysis (Phase 2)

Tracing the steps in each flow indicate that MapViewer handles mouse inputs (see Appendix 2). For the Basic Flow and the Alternative Flow: Clicked on coordinates with multiple restaurants, the interface object MapTipWindow implements the display of the map tip window. The map tip window is generated in response to a mouse pointer moving over it (Basic Flow) or a mouse clicking at coordinates where multiple restaurant icons are present (Alternative Flow: Clicked on coordinates with multiple restaurants). Meanwhile, for both the Basic Flow and the Alternative Flow: Clicked on coordinates with multiple restaurants, the RestaurantInfoPopup object handles the popup window containing information about the selected restaurant. The results of performing robustness on the flows are shown in Tables 19, 20, 21, respectively.

Table 19 Results of performing robustness analysis on the basic flow

Table 20 Results of performing robustness analysis on the alternative flow: clicked on coordinates with multiple restaurants

Table 21 Results of performing robustness analysis on the alternative flow: clicked on coordinates with no restaurants

3.2.1 Basic flow:

• 1. Display Map Tip Window containing the name of the pointed to Restaurant icon

  • 1.1. The “User” rolls the mouse cursor over a Restaurant icon presented by the “MapViewer” interface.

  • 1.2. The “MapViewer” detects that the mouse cursor is over a Restaurant icon and invokes “Change cursor and highlight Restaurant icon” to change the cursor to “selection hand” and the Restaurant icon to “selected Restaurant”.

  • 1.3. “Change cursor and highlight Restaurant icon” then invokes “Get Restaurant name(s)” to get the name of the pointed to Restaurant.

  • 1.4. “Create the Tip Text” is then invoked to create tip text containing the Restaurant name.

  • 1.5. “Show the tip window” is then invoked to show the tip window.

  • 1.6. The tip window is then visually displayed and handled by the “Map Tip Window” interface.

Inputs: Coordinates of mouse pointer that occurred over a Restaurant icon.

Outputs: Map Tip Window containing the name of the Restaurant that had the mouse roll over it.

• 2. Display a popup window containing various information about the clicked on Restaurant

  • 2.1. The “MapViewer” interface is provided with the coordinates of a mouse click from the “User”.

  • 2.2. The “MapViewer” detects that the mouse was clicked on a single Restaurant icon or name in a list.

  • 2.3. The attributes of the clicked Restaurant is retrieved using “Get Restaurant Attributes”

  • 2.4. “Get Restaurant Attributes” retrieves the required Restaurant information from the “Restaurant” entity that corresponds to the clicked Restaurant.

  • 2.5. The Restaurant attributes are then used by “Show Restaurant Info Popup” to prepare them for display.

  • 2.6. After the Restaurant attributes are prepared, they are displayed and controlled by the “Restaurant Info Popup” interface.

Inputs: Coordinates of mouse click that occurred over a Restaurant icon.

Outputs: Popup window containing the information of the Restaurant that was selected.

3.2.2 Alternative flow: clicked on coordinates with multiple restaurants

• 1. Generate Pop-up of Clicked Restaurants

  • 1.1. The “MapViewer” interface is provided with the coordinates of a mouse click from the “User”.

  • 1.2. The “MapViewer” detects that the click was on multiple Restaurants and invokes “Get list of Restaurants”.

  • 1.3. The list of Restaurants are retrieved and stored in a “MultipleRestaurantCluster” object.

  • 1.4. The “MultipleRestaurantCluster” object then invokes “Get Restaurant name(s)” to get the name of each Restaurant it contains.

  • 1.5. “Get Restaurant name(s)” is then invoked to retrieve the name of each Restaurant in “MultipleRestaurantCluster”.

  • 1.6. “Create the Tip Text” is then invoked to create tip text containing the Restaurant names.

  • 1.7. “Show the tip window” is then invoked to show the tip window.

  • 1.8. The tip window is then visually displayed and handled by the “Map Tip Window” interface.

Inputs: Coordinates of a mouse click that occurred over multiple Restaurant icons.

Outputs: Map Tip Window showing a list of Restaurants corresponding to the Restaurants that were selected by the mouse click.

• 2. Display a popup window containing various information about the clicked on Restaurant→See Basic Flow

Inputs: Coordinates of a mouse click that occurred over a Restaurant name in a Map Tip Window.

Outputs: Popup window containing the information of the Restaurant that was selected.

3.2.3 Alternative Flow: Clicked on coordinates with no restaurants

• 1. Ignore mouse click

  • 1.1. The “MapViewer” interface is provided with the coordinates of a mouse click from the “User”.

Inputs: Coordinates of mouse click that occurred that do not match a Restaurant icon coordinates.

Outputs: None.

• 2. Ignore mouse click.

3.2.4 Creating low-level acceptance tests (Phase 3)

The EAT corresponding to the Basic Flow consists of four fixtures (Fig. 16):

Fig. 16

EAT of the basic flow

• An ActionFixture to simulate a mouse rollover on a single restaurant icon. The ActionFixture also checks that a tip window ( MapTipWindow ) is displayed;

• a RowFixture to check that the MapTipWindow is displaying the expected restaurant;

• an ActionFixture to simulate a mouse click on the restaurant icon. The ActionFicture checks that a pop-up window ( RestaurantInfoPopup ) is displayed; and

• a RowFixture to examine that the RestaurantInfoPopup is displaying information about the selected restaurant.

The EAT corresponding to the Alternative Flow: Clicked on coordinates with multiple restaurants consists of four fixtures (Fig. 17):

Fig. 17

EAT of the alternative flow: clicked on coordinates with multiple restaurants

• An ActionFixture to simulate a mouse click over multiple restaurant icons and that the MapTipWindow is displayed; while a subsequent RowFixture examines the MapTipWindow produced to check that the expected list of restaurant names is displayed; and

• A second ActionFixture to simulate a mouse click on a single restaurant name displayed by the MapTipWindow , which is followed by a RowFixture to check the details of the restaurant that was selected.

The EAT for the Alternative Flow: Clicked on coordinates with no restaurants (Fig. 18) consists of one ActionFixture that simulates a mouse click over an area with no restraurants and checks that RestaurantInfoPopup is not displayed.

Fig. 18

EAT of the alternative flow: clicked on coordinates with no restaurants