Skip to main content
SpringerLink
Log in

From devices to tasks: automatic task prediction for personalized appliance control

  • Original Article
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

One of the driving applications of ubiquitous computing is universal appliance interaction: the ability to use arbitrary mobile devices to interact with arbitrary appliances, such as TVs, printers, and lights. Because of limited screen real estate and the plethora of devices and commands available to the user, a central problem in achieving this vision is predicting which appliances and devices the user wishes to use next in order to make interfaces for those devices available. We believe that universal appliance interaction is best supported through the deployment of appliance user interfaces (UIs) that are personalized to a user’s habits and information needs. In this paper, we suggest that, in a truly ubiquitous computing environment, the user will not necessarily think of devices as separate entities; therefore, rather than focus on which device the user may want to use next, we present a method for automatically discovering the user’s common tasks (e.g., watching a movie, or surfing TV channels), predicting the task that the user wishes to engage in, and generating an appropriate interface that spans multiple devices. We have several results. We show that it is possible to discover and cluster collections of commands that represent tasks and to use history to predict the next task reliably. In fact, we show that moving from devices to tasks is not only a useful way of representing our core problem, but that it is, in fact, an easier problem to solve. Finally, we show that tasks can vary from user to user.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2a, b
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Brumitt B, Meyers B, Krumm J, Kern A, Shafer S (2000) Easy living: technologies for intelligent environments. In: Proceedings of the 2nd international symposium on handheld and ubiquitous computing (huc2k), Bristol, UK, September 2000

    Google Scholar 

  2. Chan J, Zhou S, Seneviratne A (1998) A QoS adaptive mobility prediction scheme for wireless networks. In: Proceedings of the IEEE global telecommunications conference (GlobeCom’98), Sydney, Australia, November 1998

  3. Davison B, Hirsh H (1997) Toward an adaptive command line interface. In: Proceedings of the 7th international conference on human–computer Interaction (HCI’97), San Francisco, California, August 1997. Elsevier, Amsterdam

  4. Desai N, Kaowthumrong K, Lebsack J, Shah N, Han R (2002) Automated selection of remote control user interfaces in pervasive smart spaces. In: Proceedings of the HCIC winter workshop 2002, Fraser, Colorado, January/February 2002

  5. Dewan P, Solomon M (1990) An approach to support automatic generation of user-interfaces. ACM Trans Programming Lang Sys 12(4):566–609

    Article  Google Scholar 

  6. Fox C (2001) Genarians: the pioneers of pervasive computing aren’t getting any younger. Wired, November 2001, pp 187–193

  7. Guttman E, Perkins C, Veizades J, Day M (1999) Service location protocol, version 2. RFC 2608, Internet Engineering Task Force, June 1999

  8. Han R, Perret V, Naghshineh M (2000) WebSplitter: a unified XML framework for multi-device collaborative Web browsing. In: Proceedings of the ACM conference on computer supported cooperative work (CSCW 2000), Philadelphia, Pennsylvania, December 2000, pp 221–230

  9. Hirsh H, Davison B (1997) An adaptive UNIX command-line assistant. In: Proceedings of the 1st international conference on autonomous agents (AA’97), Marina del Rey, California, February 1997

  10. Hodes T, Katz R (1999) A document-based framework for Internet application control. In: Proceedings of the 2nd USENIX symposium on Internet technologies and systems (USITS), Boulder, Colorado, October 1999, pp 59–70

  11. Hodes T, Katz R (1999) Enabling smart spaces: entity description and user-Interface generation for a heterogeneous component-based system. In: Proceedings of the DARPA/NIST smart spaces workshop, July 1999

  12. Hodes T, Newman M, McCanne S, Katz R, Landay J (1998) Shared remote control of a video conferencing application: motivation, design, and implementation. Proceedings of the SPIE multimedia computing and networking conference (MMCN), San Jose, California, January 1999. Proc SPIE 3654:17–28

    Article  Google Scholar 

  13. Horvitz E, Koch P, Kadie C, Jacobs A (2002) Coordinate: probabilistic forecasting of presence and availability. In: Proceedings of the 18th conference on uncertainty and artificial intelligence (UAI 2002), Edmonton, Canada, August 2002, pp 224–233

  14. Isbell C, Shelton C, Kearns M, Singh S, Stone P (2001) A social reinforcement learning agent. In: Proceedings of the 5th international conference on autonomous agents (AA 2001), Montreal, Canada, May/June 2001

  15. Johanson B, Ponnekanti SR, Sengupta C, Fox A (2001) Multibrowsing: moving Web content across multiple displays. In: Technical note in UBICOMP 2001, Atlanta, Georgia, September/October 2001

  16. Kaowthumrong K, Lebsack J, Han R (2002) Automated selection of the active device in interactive multi-device smart spaces. In: the spontaneity workshop at UBICOMP 2002, Göteborg, Sweden, September/October 2002

  17. Mitchell T (1997) Machine learning. McGraw-Hill, New York, ISBN 0070428077

  18. Mozer M (1998) The neural network house: an environment that adapts to its inhabitants. In: Proceedings of the AAAI spring symposium on intelligent environments, Stanford University, California, 23–25 March 1998. Technical report SS-98-02, AAAI Press, California

  19. Myers B, Stiel H, Gargiulo R (1998) Collaboration using multiple PDAs connected to a PC. In: Proceedings of the ACM conference on computer-supported cooperative work (CSCW’98), Seattle, Washington, 14–18 November 1998, pp 285–294. See also http://www.cs.cmu.edu/~pebbles/

  20. Myers B (2000) Using multiple devices simultaneously for display and control. IEEE Pers Commun 7(5):62–65

    Article  Google Scholar 

  21. Nichols J, Myers BA, Higgins M, Hughes J, Harris TK, Rosenfeld R, Pignol M (2002) Generating remote control interfaces for complex appliances. In: Proceedings of the ACM symposium on user interface software and technology (UIST’02), Paris, France, 27–30 October 2002, pp 161–170

  22. Omojokun O, Isbell C (2003) Supporting personalized agents in universal appliance interaction. In: Proceedings of the 41st ACM southeast conference, Savannah, Georgia, March 2003

  23. Omojokun O, Isbell C, Dewan P (2001) An architecture for supporting personalized agents in appliance interaction. In: Technical report of the AAAI fall symposium on personalized agents, North Falmouth, Massachusetts, November 2001. AAAI Press, California, pp 40–47

  24. Priyantha N, Miu A, Balakrishnan H, Teller S (2001) The cricket compass for context-aware applications. In: Proceedings of the ACM annual international conference on mobile computing and networking (MobiCom 2001), Rome, Italy, July 2001

  25. Rekimoto J (1998) A multiple device approach for supporting whiteboard-based interactions. In: Proceedings of the ACM SIGCHI conference on human factors in computing systems, Los Angeles, California, pp 344–351

  26. Robertson S, Wharton C, Ashworth C, Franzke M (1996) Dual device user interface design: PDAs and interactive television. In: Proceedings of the ACM SIGCHI conference on human factors in computing systems, Vancouver, Canada, April 1996, pp 79–86

  27. Salton G (ed) (1971) The SMART retrieval system: experiments in automatic document processing. Prentice-Hall, Englewood Cliffs, New Jersey

    Google Scholar 

  28. Schiffman S, Reynolds M, Young F (1981) Introduction to multidimensional scaling: theory, methods and applications. Academic Press, New York

    Google Scholar 

  29. Stanfill C, Waltz D (1986) Toward memory-based reasoning. Commun ACM 29(12):1213–1228

    Article  Google Scholar 

  30. Weiser M (1993) Some computer science problems in ubiquitous computing. Commun ACM 36(7):74–83

    Article  Google Scholar 

Download references

Acknowledgements

This research was funded in part by Microsoft and NSF grants ANI 0229998, EIA 03-03590, and IIS 0312328.

Author information

Authors and Affiliations

  1. Georgia Institute of Technology, 801 Atlantic Avenue, Atlanta, GA, USA

    Charles L. Isbell Jr & Jeffrey S. Pierce

  2. University of North Carolina/Chapel Hill, CB3175 Sitterson Hall, Chapel Hill, NC, USA

    Olufisayo Omojokun

Authors
  1. Charles L. Isbell Jr
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Olufisayo Omojokun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jeffrey S. Pierce
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Charles L. Isbell Jr.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Isbell, C.L., Omojokun, O. & Pierce, J.S. From devices to tasks: automatic task prediction for personalized appliance control. Pers Ubiquit Comput 8, 146–153 (2004). https://doi.org/10.1007/s00779-004-0273-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-004-0273-z

Keywords

Search

Navigation