Elsevier

Knowledge-Based Systems

Volume 66, August 2014, Pages 46-60
Knowledge-Based Systems

A fuzzy ontology for semantic modelling and recognition of human behaviour

https://doi.org/10.1016/j.knosys.2014.04.016Get rights and content

Highlights

  • Human activity modelling ontologies for imprecise knowledge reasoning are inexistent.

  • We develop a fuzzy ontology to deal with uncertain or vague knowledge representation.

  • We show examples of use of the ontology in different scenarios within office domain.

  • We create a dataset with rules and queries and compare fuzzy and crisp reasoning.

  • We demonstrate that fuzzy ontological reasoning improves accuracy and it is scalable.

Abstract

We propose a fuzzy ontology for human activity representation, which allows us to model and reason about vague, incomplete, and uncertain knowledge. Some relevant subdomains found to be missing in previous proposed ontologies for this domain were modelled as well. The resulting fuzzy OWL 2 ontology is able to model uncertain knowledge and represent temporal relationships between activities using an underlying fuzzy state machine representation. We provide a proof of concept of the approach in work scenarios such as the office domain, and also make experiments to emphasize the benefits of our approach with respect to crisp ontologies. As a result, we demonstrate that the inclusion of fuzzy concepts and relations in the ontology provide benefits during the recognition process with respect to crisp approaches.

Introduction

Human activity study is a complex but key aspect in the development of Ambient Intelligence (AmI) systems. Different techniques have been developed for activity and user modelling, and they may be classified as data-driven [1] and knowledge-based [2] approaches. The methods in the first category are aimed at providing robust models for handling human behaviour specific features using statistical and machine learning techniques. The best strengths of these models are their ability to handle noise, uncertainty, or incomplete sensor data [3], and they have proven to be accurate in different domains where semantics are not key. However, the need for training data and the time and performance required for these models are limitations in dynamic environments and situations where context-aware data prevail. Furthermore, data-driven algorithms do not offer abstract reasoning mechanisms that allow inferring the meaning of the actions according to their semantics [4].

On the other hand, knowledge-based techniques have been applied in pervasive computing environments to improve interoperability and adaptation to different context situations. Usually, context data sources are dynamic, continuously changing depending on the environment, not always mobile, known, nor taken into account in advance. For this reason, these methods show advantages with respect to data-driven models due to the inclusion of context management tools. Further features of knowledge-based techniques that are interesting for human activity representation are the possibility of providing both the environment and the user with semantics to aid in the context definition process, facilitate the definition and comprehension of human behaviours (e.g. machine readable and easier to interpret), and consequently, ease the development of new learning and recognition models able to better understand the meaning of human actions and execute logic reasoning about future needs, situations, or actions. In addition, all this can occur considering the context information where the activity is performed. Examples of knowledge-based techniques contain logic-based approaches [5], [6], rule-based systems [7], and ontological models [8].

Despite the fact that most of the approaches about human activity recognition are focused in Ambient Assisted Living and Smart Home assistance [2], [9], [10], another emerging scenario is the office/work domain and public buildings environments. In this case, the goals are aimed at improving energy efficiency and work assistance [11], [12], [13], [14]. For instance, MOSES [15] localizes work staff and identifies the tasks they are doing at any moment, being able in this way to give advice on the remaining tasks to be carried out and warn about potential oversights or forgotten actions. Another example is iShopFloor [16], a multi-agent architecture to plan and control industry processes. However, semantic technologies have not been generally included into these models, although there are exceptions such as in intelligent meeting rooms [17] or maintenance of large buildings [16], [18]. AmI scenarios in offices or work environments focus on easing the work in groups and optimizing the office space. For example, EasyMeeting [19] is an intelligent meeting room system that builds on the design of CoBrA [20]. RFID sensors embedded in the walls and furniture detect the presence of the users’ devices and clothing. On receiving information about the user’s context and intention, the broker sharing platform allows the activation of the projector, slide downloading, and lighting control. In [18], ontology-based interoperability is applied to Smart Spaces for a context-aware maintenance of large buildings, monitoring environmental variables, automatically detecting building-related faults, and executing multi-modal interventions. An architecture for an ubiquitous group decision support system, WebMeeting [21], is able to consider the emotional factors of participants and their associated argumentation processes. The system shows available information to the participants, analyses the meeting trends, and suggests arguments to be exchanged with others. Further interesting projects regarding activity recognition in the office domain are the AIRE project [12], the SmartOffice or Monica project [14], the Interactive Room (iRoom) [22], or the NIST Smart Space and Meeting Recognition projects, which develop tools for assistance in meeting rooms [13].

The most widely used tool to integrate semantics into activity recognition systems are ontologies [4]. However, there are current limitations of ontology-based activity recognition techniques that must be tackled: they require good knowledge engineering skills to model the domain, OWL DL does not allow interval (i.e. overlapping) temporal reasoning, ontological reasoning can be computationally expensive [23], and they cannot deal with uncertainty [4]. In this work, we provide advances to solve this last limitation and propose a fuzzy ontology to give support for imprecision and uncertainty, typical of everyday life situations. For instance, a sensor can give readings with a certain degree of reliability, or work only at specific times or in certain conditions; users may perform subtle changes in the way they perform their activities, the execution of an activity may be detected with a certainty or satisfiability degree, and all this information should be taken into account into the reasoning process. Unfortunately, classic crisp ontology proposals cannot handle this type of information. In our approach, different levels of granularity are designed so that incremental context acquisition allows behaviour abstraction and a more accurate, i.e., low-level recognition. By setting a behaviour specification structure, a set of rules can define how to recognize a human behaviour out of a sequence of observations. And, since fuzzy ontologies can handle uncertainty, our approach is able to solve this limitation with respect to crisp approaches. Fuzzy logic was already proposed as an argument to “reject the maximality rule, according to which only altogether true sentences are true, and embracing instead the rule of endorsement, which means that whatever is more or less true is true” [24]. As argued in [24], positing fuzzy predicates usually simplifies theories in most scientific fields; fuzzy predicates are much more plausible and give a more cohesive world view than their crisp counterpart. In this way, classical ontologies are not suitable to deal with imprecise and vague knowledge, which is inherent to real world domains [25]. On the other hand, fuzzy ontologies have the advantage of extending information queries, allowing the search to also cover related results. This makes the decisions about relatedness based on modelled domain knowledge, i.e., instead of just offering exact matches, the search can be extended to cover also related concepts, so that precise wording is not needed to get a useful hit (as the context of a document does not have to be exactly the same one for the user to benefit from it) [26]. This results on more effective retrieval. Likewise, another advantage of fuzzy ontologies is the fuzzy semantics, as they are more flexible towards mapping between different ontologies [26].

Let us put an example to show the benefits of fuzzy ontologies versus crisp ones. Because in a fuzzy ontology we can define that the CoffeeBreak activity is recognized accounting for different weights on the actions that compose it (e.g. 0.3 TakeMug, 0.3 TakeCoffeePan, 0.4 TakeMilk), thus, when one action has been skipped due to an exception (e.g. milk run out) or a missing sensor reading, the activity can still be recognized to a lower degree. In contrast, the same activity formalized in a crisp ontology could not be recognized if any of the exclusive elements that compose it is missing.

The rest of the paper is organized as follows: the following section describes related work on ontologies for human activity recognition and introduces fuzzy ontologies as the main tool for the rest of the manuscript. After that, in Section 3, we present a novel ontology for human activity modelling and its extension to Fuzzy OWL 2 with support for the fuzzyDL reasoner. We detail concepts and relationships in the fuzzy ontology as well as Section 4.1 presents the use case on domain specific entities for the office environment. Section 4.2 describes an evaluation of the approach with respect to the crisp case, and finally, conclusions and future work are shown in Section 5.

Section snippets

Ontologies for human behaviour recognition

The literature offers a wide range of ontology definitions [27], although the most extended one is that a ontology is a “formal specification of a shared conceptualization” [28]. It offers a formalism to represent classes or concepts, individuals, relations, functions, and attributes. As providers of a format for exchanging knowledge, they promote interoperability, knowledge reuse, and information integration with automatic validation. Ontologies separate declarative and procedural knowledge,

A fuzzy ontology for human activity modelling

In this section, we first describe the design principles used to represent human behaviour in a crisp subset/core of the ontology and then, the fuzzification procedure to obtain the proposed fuzzy approach. Among the different knowledge engineering methodologies, we mostly followed NeOn Ontology engineering methodology [49] to make our approach. Some highlighted aspects of this methodology are the ontology resource reuse, requirements specification, development of required scenarios, and

Case study: office domain and public buildings

Office and public buildings are a good scenario to test our approach, since the inclusion of the designed ontology into an automatic building control system could help to improve both energy efficiency and occupants’ wellness. This section describes a proof of concept about the potential of the fuzzy ontology in this domain, and the next sections provide an evaluation and comparison with crisp ontologies to show the benefits of the approach. The main elements that are used to model knowledge in

Conclusions and future work

Knowledge-based techniques, such as ontology-based activity modelling, add a set of advantages for incremental and context-aware recognition. It is a suitable approach to achieve interoperability, abstraction, and modularity in an easier way. However, some expressiveness limitations in OWL DL are also found related with the lack of support for temporal reasoning [35] and often, an external rule engine is used to express more complex modelling. A set of ontologies for human activity modelling

Acknowledgments

This work was funded by TUCS (Turku Centre for Computer Science), Hans Bang Foundation, Finnish Cultural Foundation, Nokia Foundation, CIE BioTIC Project CEI2013-P-3, and Project TIN2012-30939 from National I + D Research Program (Spain). We also thank Fernando Bobillo and Robin Wikström for their advice with FuzzyOWL and fuzzyDL tools.

References (57)

  • N. Díaz Rodríguez et al.

    A survey on ontologies for human behavior recognition

    ACM Comput. Surv.

    (2014)
  • R. Kowalski et al.

    A logic-based calculus of events

    New Gen. Comput.

    (1986)
  • F. Doctor et al.

    A fuzzy embedded agent-based approach for realizing ambient intelligence in intelligent inhabited environments

    IEEE Trans. Syst. Man Cybern. Part A

    (2005)
  • H. Hagras et al.

    Creating an ambient-intelligence environment using embedded agents

    Intell. Syst. IEEE

    (2004)
  • L. Chen et al.

    Using event calculus for behaviour reasoning and assistance in a smart home

  • G. Villarrubia et al.

    Proximity detection prototype adapted to a work environment

  • A. Adler, R. Davis, Speech and sketching for multimodal design, in: Proceedings of the 9th International Conference on...
  • V. Stanford, Infrastructure for distributed and embedded systems and interfaces, in: Proceedings of the Embedded...
  • C.L. Gal

    Smart offices

  • M. Stoettinger, Context-Awareness in Industrial Environments, 2004....
  • W. Shen et al.

    iShopFloor: an internet-enabled agent-based intelligent shop floor

    IEEE Trans. Syst. Man Cybern. Part C

    (2005)
  • H. Chen, T. Finin, A. Joshi, Semantic web in a pervasive context-aware architecture, in: Artificial Intelligence in...
  • A. D’Elia, L. Roffia, G. Zamagni, F. Vergari, P. Bellavista, A. Toninelli, S. Mattarozzi, Smart applications for the...
  • H. Chen et al.

    Intelligent agents meet the semantic web in smart spaces

    IEEE Internet Comput.

    (2004)
  • H. Chen, T. Finin, A. Joshi, An Ontology for context-aware pervasive computing environments, in: Proceedings of the...
  • G. Marreiros, R. Santos, C. Ramos, J. Neves, P. Novais, J. Machado, J. Bulas-Cruz, Ambient intelligence in emotion...
  • A. Fox et al.

    Integrating information appliances into an interactive space

    IEEE Comput. Graph. Appl.

    (2000)
  • D. Riboni, L. Pareschi, L. Radaelli, C. Bettini, Is ontology-based activity recognition really effective?, in: PerCom...
  • Cited by (0)

    1

    Principal corresponding author. Tel.: +34 956526159.

    View full text