The sensor internet at work: Locating everyday items using mobile phones

https://doi.org/10.1016/j.pmcj.2007.12.002Get rights and content

Abstract

We present a system for monitoring and locating everyday items using mobile phones. The system is based on phones which are enhanced with the capability to detect electronically tagged objects in their vicinity. It supports various functionalities: On the one hand, phones can store the context in which users leave registered items and thus help to locate them later on. On the other hand, object owners can search for their objects using the infrastructure of mobile phones carried by other users. We describe the design of our object location system and provide an algorithm which can be used to search for lost or misplaced items efficiently by selecting the most suitable sensors based on arbitrary domain knowledge. Furthermore, we demonstrate the practicability of such wide-area searching by means of user-held sensors in a series of simulations complemented by a real-world experiment.

Introduction

Inexpensive sensing devices are expected to play a major role in future computing systems that aim to make the daily life of their users easier by monitoring everyday physical processes and providing novel features based on the acquired data.

What currently hinders most of the conceived systems from becoming commercial applications, however, is a lack of adequate infrastructure of various types: First, a sensing infrastructure must be installed to perform the sensing task. Second, a communication infrastructure is required to distribute and aggregate sensor readings from multiple sensors. Finally, a commercial infrastructure is needed to manufacture and deploy sensing devices, and to generate revenue from the system.

The mobile phone system provides a unique opportunity to overcome these difficulties. Sensing technologies can be embedded into mobile handset devices or accessed from the handset via short-range wireless communication. Wide-area communication is a core property of the cellular network. It enables the integration of data from many sensors and the support of applications with back-end services such as data storage and dissemination. Finally, cellular network operators control an important commercial infrastructure as they can promote the large-scale production of sensor-enhanced devices and deploy such devices to a large subscriber base through their established sales channels (as we have already seen with camera phones).

With new devices and an adequate communication infrastructure connecting sensors to form a global Sensor Internet, the commercial deployment of powerful applications becomes a real possibility. Mobile phones can provide a unique sensing infrastructure for such applications: A single mobile phone, enhanced with appropriate sensors, can already provide almost full “coverage” of its owner’s activities, the routes he follows, and the places he visits. Moreover, given the prevalence of mobile phones, multiple phones can achieve virtually ubiquitous geographic coverage: at any given moment, data could be retrieved from any place where there are phone users.

Making use of these unique properties of mobile phones and the cellular network, we present a system which is concerned with monitoring and locating everyday items by means of mobile phones. Mobile phones, with integral hardware that allows them to detect the presence of electronically tagged objects in their vicinity, fulfill the dual role of object sensors and of a user interface for managing personal items at the same time. Furthermore, data from many user-held object sensors can be aggregated to locate an arbitrary missing item.

This system allowed us to identify two particular challenges that are common to many applications which make use of the large people-centric infrastructure provided by mobile phones and the cellular network. The first is to define the scope of a sensing task: due to the large scale of the network, it would be inefficient to query all available object sensors or, alternatively, to store all sensor readings in a centralized database for any subsequent queries. Rather, it is sensible to involve only a small subset of all available sensors which are likely to cover the phenomenon one is interested in. To address this issue, we present in Section 3 an algorithm that incorporates arbitrary application-specific knowledge to select a subset of sensors for a given object search query.

The second challenge is to answer the question of whether a system based on sensors carried by people can provide sufficient sensor coverage in a relatively short period of time. In an extensive evaluation, which includes a real-world experiment with our object localization prototype, we therefore analyzed the properties of the coverage obtained given a wide range of different operational parameters such as the density of participants, their mobility, the range of the sensors used, and the time intervals within which a sensing task is performed. In addition to confirming the feasibility of object localization based on mobile phones, the study can provide valuable guidelines for the design of future people-centric sensing systems in general.

Our application requires the integration into mobile phones of object sensors, which are able to detect the nearby presence of an electronically tagged item. Various technologies could be employed for this purpose. For example, passive RFID tags are expected to be attached to many consumer products in the near future, as they may realize significant cost savings in stock and supply chain management. In particular, passive UHF RFID technology [25] or active tags with a small autonomous power source [21] can provide reading ranges of a couple of meters even with small reader modules. If improved variants of today’s handheld RFID readers were integrated into mobile phones, a ubiquitous system could be deployed within a few years using the short innovation cycle established through mobile phone sales. In addition to RFID, other upcoming radio communication technologies such as Zigbee, some even compatible with the Bluetooth capability of today’s phones, could be used to identify objects in the phone’s proximity in a similar way. If small, inexpensive Bluetooth-discoverable tags can be built (e.g., based on Wibree [29] or “Ultra Low Power (ULP) Bluetooth” [27]), this would mean that a ubiquitous object-sensing infrastructure is already in place today.

For each tagging technology, there is a certain trade-off between tag costs and size, the identification range achievable, and the costs of reader hardware. Irrespective of the technology used, we assume for our scenario that suitable object sensors can be integrated into mobile phones, as is already the case today with Bluetooth and NFC. Our current system prototype requires battery-powered tags (BTnodes [3]) on objects and uses the phones’ built-in Bluetooth discovery for object sensing. While we also evaluate the benefit of an increased sensing range of possible future technology in this paper, the final choice of technology is based on costs versus range trade-offs, which remain to be explored in a concrete product’s business plan.

Our object localization application involves various use cases concerned with managing everyday items by means of mobile phones.

Remember. The remember use case allows users to set up their mobile phone to store the context in which an object leaves the phone’s local sensing range. This includes a trace of the user’s location before and after the loss event and of other people present or other personal objects carried at the time the object was left behind. As there will be numerous managed objects that users leave behind on a regular basis (for example when leaving their home), users will find it unpleasant to receive notifications each time objects go out of range. Instead, the relevant data is silently stored and can be used at a later time to help the user recall the circumstances of the loss or as a clue to the whereabouts of a lost object.

Find. In the find use case, the user can query the system for an item from the list of objects that have previously been associated with the user (Figs. 1(a) and (b)). The system will then forward the query to a set of object sensors which, based on user preferences and system settings, are presumed to be good candidates to find the object. Object search strategies can be based on various heuristics, such as querying sensors near the location where the object was last within the range of the user’s device. Once a remote sensor has located the object, the user will receive a notification containing the object’s location as shown in Fig. 1(c).

Delegate. Our system also allows users to locally delegate the care of a personal item from the personal mobile device to other object sensors installed in the environment. For instance, a smart coat hanger in a restaurant can be tasked with guarding a coat and sending an alarm if it is removed.

Gate. Finally, users may add object sensors in places where these provide a particular benefit. As an example of such functionality, we support the gate use case, which involves installing an object sensing device to the door of a facility (such as an equipment room or industry lab) in order to record which objects leave with whom. Such functionality provides unobtrusive check-in and check-out management for the equipment used in the facility — and, like the remember use case, may contribute useful information for dealing with subsequent find queries.

In the remainder of this paper we present a system that can be flexibly adapted to implement each of the above use cases, while particularly focusing on the challenging find scenario. We first overview the system architecture in Section 2 and then detail our query services, which can be used to set up each use case, together with the methods developed for defining the scope of a find query in Section 3. We then discuss the privacy implications of the system in Section 4. In Section 5, we evaluate the practicability of mobile-phone-based object localization by means of a real-world experiment complemented by a series of simulations. We survey related work in Section 6 and provide conclusions on people-centered sensing in Section 7.

Section snippets

System architecture

Fig. 2 shows an overview of the system architecture. As mentioned above, mobile phones are used to link sensing functionality to users and the back-end infrastructure. Sensing functionality on mobile phones includes sensing the presence of tagged items, the phone’s location, and other information relevant for remembering the context of an object’s loss, as detailed in Section 2.1. Furthermore, our architecture involves application-specific services, such as associating objects and their owners,

Query services

The query services form the integrating element of our system, wiring the distributed components for the required application task. The back-end infrastructure hosts the global query service, which exports a query interface allowing the desired application behavior to be set up according to the use cases remember, find, delegate, and gate. Each use case also contains some functionality that should be performed at the level of the individual mobile devices. Thus, each mobile device executes a

Privacy considerations

The query service, presented in the previous sections, makes use of a wide variety of personal data. It is therefore important to keep these data private and secure. In the following, we describe some of the privacy-enhancing features of our system.

Most prominently, tagged objects and persons can be protected from being sensed by unauthorized users with the help of a zero-knowledge authentication scheme as proposed in [8]. This protection is based on a shared secret x, which is known by an

Evaluation

In the following, we examine whether our object search system can perform well enough to be a useful application. To achieve this, we evaluate the coverage of the system and the reply time for search queries in a real-world experiment and in simulations.

Related work

Various research papers argue for the relevance of locating everyday objects, monitoring the presence of items, or avoiding their loss. Many such systems [2], [5], [24], [28], [30], however, suggest a specific pre-installed object-sensing infrastructure, which is costly to deploy and to maintain. Reminder systems [2], [24] focus on notifying users before a loss takes place. As mentioned, we generally assume that tagged objects will often be intentionally left behind and therefore avoid

Conclusion

We have presented a comprehensive system for managing and finding everyday objects relying on mobile phones as omnipresent object-sensing devices. We have discussed the architecture, design, and expected performance of this system, together with a flexible means of generating object search heuristics from application data. Based on the ubiquitous mobile network infrastructure which is already in place, wide-area searches for everyday objects become possible without incurring the high costs

Acknowledgments

The work presented in this paper was supported by DoCoMo Euro-Labs and partially by NCCR-MICS, a center funded by the Swiss National Science Foundation. We would like to thank Michael Fahrmair and Daisuke Ochi for their thoughtful comments and suggestions on draft versions of this paper, Christof Roduner and Chie Noda for their contributions to the presented work in the earlier phases of our research collaboration, and Hans-Florian Geerdes for his valuable advice on the Momentum dataset [18].

Christian Frank is a Ph.D. student at the Institute for Pervasive Computing of ETH Zurich, Switzerland. He received a Diploma (M.Sc.) degree in computer science from Technische Universität Berlin, Germany in 2003. His research interests include the configuration of wireless sensor networks, distributed algorithms, and infrastructures and middleware for distributed systems.

References (30)

  • Philipp Bolliger, Marc Langheinrich, Distributed persistence for limited devices, in: System Support for Ubiquitous...
  • Gaetano Borriello, Waylon Brunette, Matthew Hall, Carl Hartung, Cameron Tangney, Reminding about tagged objects using...
  • BTnodes. http://www.btnode.ethz.ch,...
  • J. Burke, D. Estrin, M. Hansen, A. Parker, N. Ramanathan, S. Reddy, M.B. Srivastava, Participatory sensing, in:...
  • Christian Decker, Uwe Kubach, Michael Beigl, Revealing the retail black box by interaction sensing, in: 3rd...
  • Andreas Eisenblätter et al.

    Public UMTS radio network evaluation and planning scenarios

    International Journal on Mobile Network Design and Innovation

    (2005)
  • Shane B. Eisenman, Gahng-Seop Ahn, Nicholas D. Lane, Emiliano Miluzzo, Ronald A. Peterson, Andrew T. Campbell,...
  • Stephan J. Engberg, Morten B. Harning, Christian D. Jensen, Zero-knowledge device authentication: Privacy & security...
  • ETSI. Selection procedures for the choice of radio transmission technologies of the UMTS. Technical Report 3.2.0,...
  • Lucio Ferreira, Luis M. Correia, David Xavier, Allen Vasconcelos, Erik Fledderus, Deliverable d1.4: Final report on...
  • Christian Frank, Philipp Bolliger, Christof Roduner, Wolfgang Kellerer, Objects calling home: Locating objects using...
  • Christian Frank, Christof Roduner, Philipp Bolliger, Chie Noda, Wolfgang Kellerer, A service architecture for...
  • Christian Frank, Christof Roduner, Chie Noda, Wolfgang Kellerer, Query scoping for the Sensor Internet, in: Proceedings...
  • Phillip B. Gibbons et al.

    Srinivasan Seshan, IrisNet: An architecture for a worldwide sensor web

    IEEE Pervasive Computing

    (2003)
  • Minkyong Kim, David Kotz, Songkuk Kim, Extracting a mobility model from real user traces, in: Proceedings of the 25th...
  • Cited by (0)

    Christian Frank is a Ph.D. student at the Institute for Pervasive Computing of ETH Zurich, Switzerland. He received a Diploma (M.Sc.) degree in computer science from Technische Universität Berlin, Germany in 2003. His research interests include the configuration of wireless sensor networks, distributed algorithms, and infrastructures and middleware for distributed systems.

    Philipp Bolliger received a Diploma (M.Sc.) degree in computer science from ETH Zurich, Switzerland. Since then, he is a Ph.D. student at the Institute for Pervasive Computing of ETH Zurich. His research interests include distributed location models, location oriented programming, and location information processing.

    Friedemann Mattern is a Professor of Computer Science at the Institute for Pervasive Computing of ETH Zurich, Switzerland. He received his Masters Diploma from the University of Bonn, and his Ph.D. from the University of Kaiserslautern, Germany. Before joining ETH Zurich in 1999, he was Professor at Saarbrucken and at Darmstadt, Germany. Friedemann Mattern is member of the editorial board of several scientific journals, initiated and chaired a number of international conferences, and has authored or co-authored more than 150 research articles. His main research interests are distributed systems, ubiquitous computing, and the upcoming Internet of Things.

    Wolfgang Kellerer is heading the Ubiquitous Services Platform group of NTT DoCoMo’s European Research Laboratories in Munich, Germany. His research interests include mobile service platforms and overlay networks, peer-to-peer systems, sensor networks, and cross-layer optimization. He received his Dipl.-Ing. (M.Sc.) and Dr.-Ing. degrees in electrical engineering and information technology from Munich University of Technology (TUM), Munich, Germany, in 1995 and 2002, respectively. In 2001 he was a visiting researcher in the Information Systems Lab at Stanford University. He is a member of IEEE ComSoc and VDE ITG.

    View full text