MSSN-Onto: An ontology-based approach for flexible event processing in Multimedia Sensor Networks

Highlights

• Ontological-based approach for modeling data and events in multimedia sensor networks.

• MSSN-Onto is validated by means of prototyping and simulation.

• Results confirm the capability of MSSN-Onto in both interoperability and performance aspects.

Abstract

Multimedia Sensor Networks (MSNs) have gained much attention in recent years from the emerging trends of Internet of Things (IoT). They can be found in different scenarios in our everyday life (e.g., smart homes, smart buildings). Sensors in MSNs can have different capacities, produce multiple kinds of outputs, and have different output encoding formats. Thus, detecting complex events, which requires the aggregation of several sensor readings, can be difficult due to the lack of a generic model that can describe: (i) sensor networks infrastructure, (ii) individual sensor specificities, as well as (iii) multimedia data, while allowing the alignment with the application domain knowledge. In this study, we propose Multimedia Semantic Sensor Network Ontology (MSSN-Onto) to ensure MSNs modeling and provide both syntactic and semantic data interoperability for defining and detecting events in various domains. To show the readiness of MSSN-Onto, we used it as the core ontology of a dedicated framework (briefly defined here). We also adopted MSSN-Onto in HIT2GAP European Project. A prototype has been implemented to conduct a set of tests. Experimental results show that MSSN-Onto can be used to: (i) effectively model MSNs and multimedia data; (ii) define complex events; and (iii) allow to build an efficient event querying engine for MSNs.

Introduction

The vertiginous advances in low-cost hardware technology, wireless communications, and digital electronics have fostered the development of multi-functional Multimedia Sensor Networks (MSNs). MSNs are networks of interconnected sensor devices able to ubiquitously sense data from the environment of various multimedia content (e.g., videos, audio streams, images, texts) [[1], [2], [3]] and detect consequently various types of events. It is worthy to note that two major types of events can be detected¹ : an atomic event using a single reading of a sensor, and a complex event using the combination of several sensor readings of several sensors.

Nowadays, MSNs become increasingly popular and important in our everyday live. They can be used for monitoring, tracking, and detecting various atomic and complex events in different scenarios (e.g., smart homes, smart buildings, smart cities) [4]. However, the success in deploying MSNs for event detection mainly relies on how efficiently data is represented, gathered, and processed by different sensors and how efficiently an event can be detected from the gathered (multimedia) data. These issues can be analyzed from two perspectives: the MSNs modeling and the event modeling. From the MSNs modeling perspective, given (i) the diversified types of multimedia sensors, (ii) the different types of sensor output (i.e., scalar, multimedia), and (iii) the possibility to dynamically reconfigure the network (e.g., plug/remove mobile sensor devices), a syntactic interoperability able to model the multimedia sensor data and the network configuration is needed. From the event modeling perspective, given the diversity of semantics (i.e., meaning, interpretation, analysis) of events and the intrinsic unreliability of real-world (multimedia) sensor data, a semantic interoperability to cope with the various facets of the events to be detected is also requested. The integration of both models would be very useful to facilitate the usage of MSNs data and applications to effectively and efficiently detect events.

Ontologies have been commonly adopted (SensorML [1], OntoSensor [5], and SSN ontology [6]) to overcome both the syntactic and semantic interoperability problems related to modeling sensor readings, events to be extracted, and sensor networks infrastructure. However, they have been designed to model scalar sensor networks which can be very restrictive for appropriate MSN representation. In essence, MSNs impose new challenges related to: (i) the flexibility of MSNs infrastructure where sensors can be heterogeneous, mobile and reconfigurable; (ii) appropriate syntactic interoperability support, due to multi-modality of multimedia data (e.g., video, audio, image) that can be encoded in different formats; (iii) domain-related semantic interoperability support, due to the difficulty of retrieving semantic information from multimedia data (e.g., what is inside a video stream, the object in the image); and (iv) the need of a dedicated event modeling and definition that can be detected from the data combination of different multimedia sensors. Therefore, in order to overcome these challenges, it becomes critical to build a full-fledged representation of MSNs to allow the syntactic and semantic interoperability, in a way that it can facilitate the event definition and detection process.

In this context, we propose a Multimedia Semantic Sensor Network Ontology, called MSSN-Onto, designed to be generic for modeling MSNs in many application domains. We present and discuss here main MSSN-Onto concepts, show how they can be used to model MSNs, and how they can be used in different application domains. We also propose MSSN-Onto as the core ontology of a dedicated framework to support the modeling of MSNs, as well as the definition and detection of events, while answering the aforementioned challenges. To show the readiness of MSSN-Onto, an online prototype of this framework has been implemented in a meeting room scenario where several atomic and complex events can be defined and detected easily. Additionally, we conducted experiments to evaluate the performance of our MSSN-Onto-based framework under high-workload scenarios. We also integrated it in HIT2GAP European Project (http://www.hit2gap.eu). Obtained results show that we can use our approach to develop an expressive event detecting engine that allows to retrieve complex events in MSNs with an acceptable accuracy despite the usage of the on-the-shelf multimedia processing techniques. The performance of our framework is also proven within the experiment to be acceptable in a high workload scenario where 500 multimedia sensors are deployed and modeled.

The remainder of this work is organized as follows. The motivation scenario is described in Section 2. Related studies are discussed in Section 3. Our MSSN-Onto and designed framework are described in details in Section 4. Section 5 describes how MSSN-Onto can be used in different application domains. Section 6 presents the prototype of the framework, the conducted experimentation, results, and discussions. Section 7 is dedicated to conclude this study and to provide several perspectives.