A framework for utilizing qualitative spatial relations between networked embedded systems
Introduction
The miniaturization and seamless embedding of computing and wireless communication technologies into things of everyday use like tools, appliances and whole rooms lead to huge quantities of heterogeneous and interconnected smart objects, which are characterized by constrained user interfaces and limited computing resources. This development is mainly driven by technological advances within the past few decades, which have made it possible to shrink sensors and actuators as well as processing and wireless communication technologies to a size that enables their integration into virtually everything. A notable example for it is the recent integration of GPS and micro-electromechanical inertial sensors in a variety of consumer appliances like mobile phones, notebook computers, digital cameras and even wrist watches. Their density is expected to increase rapidly within the next decades, which will lead to a situation where humans are no longer able to manage and explicitly control them. Instead, it will be necessary to design and implement systems which operate autonomously, i.e. with as little human attendance as possible, and interact with humans in a more implicit and unobtrusive way. Clearly, the traditional approach of instructive systems [1] with their deterministic and context-free nature appears less appropriate as an architecture for providing services by the interaction of embedded and networked devices; instead, a more autonomous system architecture [2], [3] which implements context-dependent behavior is required.
For achieving autonomy, we consider two aspects to be especially important. The first one is the ability of an embedded system to acquire and interpret information about its environment from sensor data, which enables it to become aware of its context and adapt to changing contexts at runtime, and the second aspect is ad hoc context sharing, namely the spontaneous exchange of context information among systems within communication range. There has been much research on the former [4], [5], whereas location and spatial proximity have been the most often used information in context-aware applications. By sharing their contexts, it becomes possible for networked embedded systems to collaboratively adjust their actions to one another, and thus achieve coordinated behavior as has been demonstrated in several research projects [6], [7], [8]. According to [9], essential parts of the context of technology-rich artifacts are their spatial properties position, direction and extension as well as spatial relations between them; in the following, we refer to spatial properties and relations as spatial contexts. This is because they are part of the ever-changing physical world, and thus–due to movements or manipulations–also their spatial contexts are subject to frequent changes. The scope of this article is on the issue of making networked–and potentially mobile–embedded systems aware of their spatial context, with the aim to enable an adaptation to changing spatial situations and thus facilitate their contextual interaction in space. Our focus in this regard is on algebraic relations as well as their changes over time, which we consider especially relevant because of the fact that they describe the situation of a system with respect to its surroundings; therefore, they are fundamental for the development of applications whose functionality not only depends on the individual contexts of networked embedded systems, but also on the co-existence of other systems and their relations to each other. Such spatially aware applications are executed either on a single or on multiple embedded systems. Hence, we define spatial awareness as follows:
A system is spatially aware if it is able to autonomously determine and use its spatial properties, as well as to relate these properties to the spatial properties of other systems.
Spatial awareness requires means for acquiring spatial contexts from sensors, representing and reasoning (i.e. drawing conclusions) about perceived context information and sharing it among networked embedded systems in communication range. We use generic, structured “self-descriptions” for the latter, which encode the artifacts’ spatial contexts using an XML-based metadata format, and are exchanged among their embedded systems upon coming into communication range. A key aspect is the abstraction of context for its use by context-aware applications [10], [11]; for example, an application using location information may only be interested in high-level information like rooms and buildings instead of geographical coordinates. Such symbolic representations of locations have been addressed by several researchers (e.g. [12], [13], [14]), examples of abstracted positional relations–which are represented with meaningful names such as left or near instead of numeric angles or distances–can be found in [15]. The underlying idea is to provide context information in a sensor-independent way which (i) decouples low-level details that are not relevant for the application and (ii) allows programmers and developers to use context at a higher level of abstraction.
In order to represent spatial contexts, we propose two types of abstraction in the present work. First, quantitative abstractions are used for representing the spatial properties of technology-rich artifacts, which means that in a scene of artifacts just the spatial properties of those which are relevant for a spatially aware application are represented; numerical values are used to allow for calculating spatial relations between them. A concept referred to as Zones-of-Influence has been developed for this purpose, which builds upon initial work published in [16]. A Zone-of-Influence represents a geographic region, which is of relevance for and has an influence at a particular spatially aware application, by its numerical position, direction and spatial extension. Technology-rich artifacts are associated with one or more Zones-of-Influence at a time, together representing the spatial knowledge which is distributed across them in physical space and shared by an ad hoc exchange of self-descriptions. This allows artifacts to autonomously recognize spatial relations between their Zones-of-Influence, including distance and orientation relations as well as topological relations between their spatial extensions.
Second, qualitative abstractions are used for representing such spatial relations, which means that the continuous relation values are quantized according to the required accuracy and represented with discrete systems of symbols. Qualitative representations have attracted much interest by researchers in the field of spatial reasoning [17], [18]. Compared with quantitative approaches, they are often the preferred choice when the spatial cognition of humans is involved, which is due to the fact that (i) we are used to draw conclusions from coarse abstractions [19], [20] and (ii) qualitative approaches are similar to natural language terms and thus considered cognitively more adequate [21], [22], or when systems with limited computing resources or sensing capabilities are concerned [23], [24]. Hence, it should be easier for a programmer to develop applications by using high-level qualitative spatial relations whose semantics are adjusted to the application domain, and computationally less demanding reasoning techniques can be implemented due to the separation from details that are not relevant for a certain application or in a particular context.
In this article, we present novel models of spatial abstraction as well as an according software framework which is designed to run on embedded systems and facilitate the development of spatially aware applications. It is based on the concepts for recognizing and representing qualitative spatial relations between networked embedded systems mentioned above, and uses a rule-based approach for reasoning about spatial relations as well as their temporal relations to each other, which allows to trigger predefined actions upon observing certain patterns on the recognized relations. The architecture of the framework comprises layers for (i) the acquisition of spatial contexts from sensors and via an exchange of self-descriptions, (ii) the achievement of spatial awareness by recognizing, representing and reasoning about qualitative spatial relations, as well as (iii) the provision of services for the development of spatially aware applications. An essential aspect is that the relationship abstractions are encapsulated in components which can easily be exchanged at runtime, hence enabling the systems to adapt the availability of spatial relations and their semantics to the current application demands.
The paper is structured as follows. First, Section 2 presents fundamental aspects concerning the spatial awareness of wirelessly interconnected embedded systems as well as the exchange of context-information among them. Section 3 afterwards elaborates on the used spatial abstractions, namely (i) the quantitative representation of spatial properties with Zones-of-Influence and (ii) the qualitative representation of spatial relations. The framework is presented in Section 4, whereas an overview of the architecture is given and details about the implementation are presented. In this regard, the rule-based approach for reasoning about spatial relations and their changes over time is discussed in detail. Section 5 presents an evaluation of the framework performance, and Section 6 gives a comprehensive overview on related work. Section 7 finally concludes the paper, discusses the potentials of and lessons learnt from the presented concepts for realizing real-world applications, and gives an overview of open issues for future work.
Please note that this paper is intended to provide a comprehensive and concise presentation of our research on the use of spatial relations in spatially aware applications, which we have partly addressed in previous work. A fundamental aspect is the qualitative abstraction of space discussed in Section 3.2, which is rooted in initial work [25], [26] whose subject was the recognition and transitive inference of qualitative spatial distance and orientation relations between wireless embedded systems. The Zones-of-Influence concept (Section 3.1) and the ad hoc exchange of context information with structured self-descriptions (Section 2) build on previous work described in [16], [27], [28] respectively, which has been extended and adapted for the recognition of spatial relations between wireless embedded systems. First versions of the software framework and its rule-based reasoning approach have been presented in [29], [30], and they are described in thoroughly reworked and extended versions in Section 4.
Section snippets
Context awareness and spatial contexts
Context awareness refers to the ability of a system to acquire environmental information in order to understand its situation and adapt to changes accordingly [31], [32], [14]. It is considered important especially for applications where the environment changes frequently [5], [32], which is commonly the case in pervasive computing applications and requires an adaptation to the current context of use. From the different definitions of context awareness which appeared during the last 15 years,
Quantitative abstraction with Zones-of-Influence
Zones-of-Influence represent application-specific spatial regions which are associated with technology-rich everyday artifacts and shared among them by means of self-descriptions. This concept builds on work published in [16], where Zones-of-Influence are proposed as three-dimensional spatial regions that describe the space surrounding an artifact, and they serve as an explicit proximity model insofar as they are used for limiting the interaction to those systems whose Zones-of-Influence
A framework for spatial awareness
We have developed a flexible and modular software framework, with the aim to facilitate the implementation of spatially aware applications for mobile and ad hoc networked embedded systems. Instead of having to re-implement the recognition and maintenance of spatial relations for each application in which they are required, the framework allows to reuse existing implementations and dynamically adapt the calculation and use of relations to the current application demands. It runs on each embedded
Performance evaluation
The presented framework allows to build spatially aware applications using different numbers of Zones-of-Influence, spatial relations and rules. In this section, the results of performance tests are presented, which have been carried out to show the runtime efficiency and scalability of the framework. Scalability is a key issue for building complex applications which comprise a multitude of interacting embedded systems, as the required processing time of each system increases with (i) the
Related work
This section finally gives an overview on related work addressing the use of qualitative spatial relations by networked embedded systems, which is–in contrast to location sensing–a sparsely investigated topic in the field of pervasive computing. A closely related project is the project Relate [73], [36], [15], which uses specialized USB dongles for providing mobile devices with an awareness about both quantitative and qualitative spatial relations to other co-located devices. Compared to our
Conclusion
In future computing systems, which are expected to be increasingly interconnected and embedded in the environment, operate more autonomously and with which humans will interact in a more natural and implicit way, the consideration of spatial contexts–and in particular of spatial relations between co-located systems–will play a significant role. It enables embedded systems to adapt to their spatial situation, for which purpose they must be able to sense their context, process and reason about
Acknowledgements
This work is supported by the FP7 ICT Future Enabling Technologies programme of the European Commission under grant agreement No. 225938 (OPPORTUNITY). Parts of it have been developed in a research project with Siemens AG Germany, Corporate Technology, CT SE 2.
Clemens Holzmann received master’s degrees in both Computer Science (2004) and Business Informatics (2006) from the Johannes Kepler University Linz, where he also received the doctorate in Computer Science (2008). From 2004 to 2010, he was with the Department of Pervasive Computing at the Johannes Kepler University Linz at the level of an assistant professor. During this time, he was involved in the EU project OPPORTUNITY and in several research projects funded by SIEMENS Germany. In 2010, he
References (85)
- et al.
Mediacups: Experience with design and use of computer-augmented everyday artefacts
Computer Networks
(2001) - et al.
Qualitative representation of positional information
Artificial Intelligence
(1997) - et al.
Peer-it: stick-on solutions for networks of things
Journal of Pervasive and Mobile Computing
(2008) - et al.
There is more to context than location
Computers & Graphics
(1999) On autonomous computing and cognitive processes
- et al.
Connecting the physical world with pervasive networks
IEEE Pervasive Computing
(2002) - et al.
The vision of autonomic computing
IEEE Computer
(2003) - G. Chen, D. Kotz, A survey of context-aware mobile computing research, Tech. Rep. TR2000-381, Dartmouth College,...
- A.K. Dey, Providing architectural support for building context-aware applications, Ph.D. thesis, Georgia Institute of...
- et al.
Smart-Its Friends: A technique for users to easily establish connections between smart artefacts
u-Texture: Self-organizable universal panels for creating smart surroundings
Understanding and using context
Personal and Ubiquitous Computing
The context toolkit: Aiding the development of context-enabled applications
Context-awareness for group interaction support
Sensing and visualizing spatial relations of mobile devices
Qualitative spatial representation and reasoning: an overview
Fundamenta Informaticae
Spatial representation and reasoning
Qualitative reasoning
Qualitative direction calculi with arbitrary granularity
Using orientation information for qualitative spatial reasoning
Spatiotemporal reasoning for smart homes
Inferring and distributing spatial context
Towards collective spatial awareness using binary relations
Building flexible manufacturing systems based on Peer-its
EURASIP Journal on Embedded Systems
Digital aura
Proceedings of the 4th Int’l Conference on Pervasive Computing, Pervasive 2004
Advances in Pervasive Computing
Rule-based reasoning about qualitative spatiotemporal relations
Using spatial abstractions in industrial environments
The stick-e note architecture: Extending the interface beyond the user
Context-aware computing applications
Context-aware applications: from the laboratory to the marketplace
IEEE Personal Communications
The active badge location system
ACM Transactions on Information Systems
A relative positioning system for co-located mobile devices
An intersection-based formalism for representing orientation relations in a geographic database
Spatially aware local communication in the RAUM system
Cooperative artefacts: Assessing real world situations with embedded technology
The design and applications of a context service
Mobile Computing and Communications Review
A middleware infrastructure for active spaces
IEEE Pervasive Computing
Cited by (10)
Spatial relationship recognition via heterogeneous representation: A review
2023, NeurocomputingA systematic review on the engineering of software for ubiquitous systems
2016, Journal of Systems and SoftwareFIA<inf>5</inf>: A customized Fuzzy Interval Algebra for modeling spatial relevancy in urban context-aware systems
2014, Engineering Applications of Artificial IntelligenceCitation Excerpt :Context appears as a fundamental key to enable systems to filter relevant information from what is available (Dey, 2001; Schilit et al., 1994; Siewe et al., 2009; Cook and Das, 2012), to choose relevant actions from a list of possibilities (Coronato and Pietro, 2011; Hong et al., 2009; Chedrawy and Abidi, 2006), or to determine the optimal method of information delivery (Pan et al., 2007). The major challenge of context-aware systems is the separation of the relevant from the irrelevant information (Holzmann and Ferscha, 2010; Choi et al., 2012; Raper et al., 2002; Grossmann et al., 2005; Reichenbacher, 2007; Thyagaraju and Kulkarni, 2012). Among all types of relevancies, spatial relevancy is the main type which could handle other types of relevancies such as user preferences, history, etc. (Afyouni et al., 2011).
Grounding ecologies on multiple spaces
2012, Pervasive and Mobile ComputingCitation Excerpt :In addition, our concrete framework can be used as a generic service to support context awareness [33] as organisms could utilize its features to acquire and disseminate context information. Finally, models and frameworks have been recently proposed with the aim of obtaining spatially localized information (see for instance [34,35]). In contrast with our approach, they emphasize physical spaces only and they do not propose space-aware interaction mechanisms.
Developing FIA<inf>5</inf> to FSTPR<inf>25</inf> for modeling spatio-temporal relevancy in context-aware wayfinding systems
2020, Journal of Ambient Intelligence and Humanized ComputingA framework for qualitative representation and reasoning about spatiotemporal patterns
2014, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Clemens Holzmann received master’s degrees in both Computer Science (2004) and Business Informatics (2006) from the Johannes Kepler University Linz, where he also received the doctorate in Computer Science (2008). From 2004 to 2010, he was with the Department of Pervasive Computing at the Johannes Kepler University Linz at the level of an assistant professor. During this time, he was involved in the EU project OPPORTUNITY and in several research projects funded by SIEMENS Germany. In 2010, he joined the Upper Austria University of Applied Sciences, where he is now a professor in the degree programme Mobile Computing. His research interests include Pervasive and Mobile Computing, Embedded Systems and Human–Computer Interaction.
Alois Ferscha was with the Department of Applied Computer Science at the University of Vienna at the levels of assistant and associate professor (1986–1999). In 2000 he joined the University of Linz as full professor where he heads the Excellence Initiative ”Pervasive Computing”, the department of Pervasive Computing, and the Research Studio Pervasive Computing Applications. He is active in international EU funded projects (FET FP7: PANORAMA, SOCIONICAL, OPPORTUNITY, BeyondTheHorizon, InterLink, CRUISE) as well as national (funded projects SPECTACLES, PowerSaver, WirelessCampus, MobiLearn) research, and keeps tight cooperations with industrial stakeholders (SIEMENS Project FACT, IBM Project VRIO). Currently he is focused on Pervasive and Ubiquitous Computing, Embedded Software Systems, Wireless Communication, Multiuser Cooperation, Distributed Interaction and Distributed Interactive Simulation.