VR-Powered Scenario-Based Testing for Visual and Acoustic Web of Things Services

Abstract

Web of Things (WoT) services are Web services that interact with physical things in the environment. Testing of WoT services should be performed considering the physical and human factors that affect their quality. Scenario-based testing is known to be one of the most effective testing techniques by which we can test software while considering various real-world scenarios. However, applying scenario-based testing to real-world WoT testbed environments is not practical in terms of cost and reconfigurability. In this work, we utilize Virtual Reality (VR) technology to mimic real-world WoT environments for cost-effective testing over various scenarios.

1 Introduction

Recent advances in networking systems have expanded the boundaries of the traditional Web to Web of Things (WoT). WoT enables services on the Web, called WoT services, to actively interact with physical things in the environment. In particular, visual and acoustic WoT services, which produce lights and sounds respectively via physical devices, are the most common types of WoT services. To test the actuating or sensing functions of such WoT services, scenario-based testing can be performed by building real-world testbeds that are composed of various physical devices. Scenario-based testing is usually regarded as an effective technique to test software while considering various human factors that cannot be tested using a limited set of test suites. However, building real-world testbeds has a high cost of installing numerous physical devices or changing configurations.

In a recent work, Virtual Reality (VR) techniques have been used to mimic psychological experiments [4]. Utilizing Head Mount Displays (HMD), users experienced virtual environments visually and acoustically, and also interacted with services in the environment by using controllers. Further, another recent work suggested utilizing VR for user studies to experience the perspectives of elderly people who have poor eyesight [3].

In this paper, we first propose our scenario-based testing framework for WoT services to enable the systematic development of test environments for various test scenarios. Next we describe the VR-powered implementation of our framework, i.e. virtual WoT environments for scenario-based testing to reduce the cost of testing various scenarios. Obviously, developing a physical environment virtually requires a lower cost than building real-world testbeds, and it is also relatively easier to change its configurations. In the demo, we show our implementations of the framework and let users experience how scenario-based testing of WoT services can be performed based on our framework.

Fig. 1.

VR-powered Scenario-based Testing Framework for WoT Services

2 Scenario-Based Testing of WoT Services

2.1 Framework

Figure 1 shows the preliminary model of our scenario-based testing framework for WoT services. A target WoT service $s = \{\mathbb {E}, \mathbb {R}\}$ for testing consists of the effects generated by the service, $\mathbb {E}$, and the required capabilities, $\mathbb {R}$. A service can be tested on a set of scenarios $\mathbb {C}_s$, by a set of testers $\mathbb {T}$. A scenario $c = \{\mathbb {U}, env, \mathbb {I}\}$, consists of the user pattern, $\mathbb {U}$, the environment configuration, env, and the service interference, $\mathbb {I}$. In our framework, patterns for some dynamics of users such as mobility can be inserted, so the tester may follow a pre-determined mobility pattern or by simply move freely. An environment env consists of services, $\mathbb {S}$, devices, $\mathbb {D}$, and physical objects, $\mathbb {O}$, where a service may be a Web or a WoT service. According to a service’s required capability, an appropriate device is discovered and gets acquired by the service. An environment may contain other services that cause interference within the target service in terms of physical effects, and models of such interference should be included in the scenario definition.

2.2 Motivating Example

We can imagine a simple service that delivers today’s headline news to a user in the morning, a so-called news-delivery service, as shown in Fig. 2. News can be represented and delivered to the user as visual text or in voice format by using media such as a display or a speaker, respectively. Furthermore, users may be mobile, so the service should deliver news to the user by following the user’s location and locating media that is spatially closer to the user [1, 2]. While the objective of the service is to deliver news to a user, the evaluation of such a news-delivery service can be performed by examining how well the news is delivered and perceived by the user. After the developer of the news-delivery service finishes its implementation, scenario-based testing should be performed in various environments to evaluate whether or not the functions of the news-delivery service work in general.

Fig. 2.

Example News-delivery Service

Following our framework, the news-delivery service can be defined as follows: $s_{news} = \{\{light, sound\}, \{display, speaker\}\}$. To test the news-delivery service, a basic scenario is designed as $c = \{\emptyset , env_c, \emptyset \}$, where $env_c$ contains configurations for every entity of the scenario. An empty set of user patterns and interference means that there are no pre-determined user patterns or interfering services. Extended scenarios can be designed based on the basic scenario, as follows. A specific travel path would be included in a scenario: $c_{mobility} = \{\{mobility_u\}, env_c, \emptyset \}$. To test whether news is delivered to a user even when other services are also generating lights or sounds, we can extend the basic scenario by inserting the interference model: $c_{i} = \{\{mobility_u\}, env_c, \{noise, glare\}\}$. To test the news-delivery service in a different environment, the environment of a scenario can simply be replaced by another environment, $env\prime _c$: $c_{env} = \{\{mobility_u\}, env\prime _c, \{noise, glare\}\}$.

3 VR-Based Simulation of WoT Services

As shown in Fig. 3a, we implemented virtual environments in a 3-dimensional space by using Unreal Engine¹, which is the most commonly used game engine for developing VR-based simulations. Testers wore HTC Vive Pro² as shown in Fig. 3b, which is one of the most common HMD for VR, for immersion inside the virtual environment and interaction through HMD and controllers.

Fig. 3.

VR-powered Scenario-based Testing

4 Conclusion

In this paper, we proposed a physical environment aware scenario-based testing framework for visual and acoustic WoT services, and described a VR-powered implementation of the framework. By using VR, multiple testing scenarios can be efficiently developed, and testers may test the target service to reflect as many real-world cases as possible.