Skip to main content
SpringerLink
Log in

A relational hierarchical model for decision-theoretic assistance

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

Building intelligent assistants has been a long-cherished goal of AI, and many were built and fine-tuned to specific application domains. In recent work, a domain-independent decision-theoretic model of assistance was proposed, where the task is to infer the user’s goal and take actions that minimize the expected cost of the user’s policy. In this paper, we extend this work to domains where the user’s policies have rich relational and hierarchical structure. Our results indicate that relational hierarchies allow succinct encoding of prior knowledge for the assistant, which in turn enables the assistant to start helping the user after a relatively small amount of experience.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ambite JL, Barish G, Knoblock CA, Muslea M, Oh J, Minton S (2002) Getting from here to there: interactive planning and agent execution for optimizing travel. In: IAAI, pp 862–869

  2. Bertsekas DP, Tsitsiklis JN (1996) Neuro-dynamic programming. Athena Scientific, Belmont

    MATH  Google Scholar 

  3. Bui H, Venkatesh S, West G (2002) Policy recognition in the abstract hidden Markov models. JAIR 17:451–499

    Google Scholar 

  4. Bui HH, Cesari F, Elenius D, House N, Morley D, Myers KM, Natarajan S, Saadati S, Yeh E, Yorke-Smith N (2008) CALO workflow recognition and proactive assistance. In: AAAI-08 AI video competition

  5. Boger J, Poupart P, Hoey J, Boutilier, C, Fernie G, Mihailidis A (2005) A decision-theoretic approach to task assistance for persons with dementia. In: IJCAI, pp 1293–1299

  6. De Raedt L, Frasconi P, Kersting K, Muggleton SH (2010) Probabilistic inductive logic programming, Lecture Notes in Computer Science/Lecture Notes in Artificial Intelligence

  7. Dietterich TG (2000) Hierarchical reinforcement learning with the MAXQ value function decomposition. JAIR 13: 227–303

    MathSciNet  MATH  Google Scholar 

  8. Domingos P, Richardson M (2006) Markov logic networks. Mach Learn 62(1–2): 107–136

    Google Scholar 

  9. Fern A, Natarajan S, Judah K, Tadepalli P (2007) A decision-theoretic model of assistance. In: IJCAI

  10. Fierens D, Blockeel H, Bruynooghe M, Ramon J (2005) Logical Bayesian networks and their relation to other probabilistic logical models. In: Proceedings of ILP-05

  11. Fine S, Singer Y, Tishby N (1998) The hierarchical hidden Markov model: analysis and applications. Mach Learn 32(1): 41–62

    Article  MATH  Google Scholar 

  12. Getoor L, Friedman N, Koller D, Pfeffer A (2001) Learning probabilistic relational models. In: Dzeroski S, Lavrac N (eds) Invited contribution to the book relational data mining. Springer, Berlin

  13. Getoor L, Grant J (2005) PRL: a probabilistic relational language. Mach Learn J 62:7–31

    Google Scholar 

  14. Horvitz E, Breese J, Heckerman D, Hovel D, Rommelse K (1998) The Lumiere project: Bayesian user modeling for inferring the goals and needs of software users. In: UAI, pp 256–265

  15. Hui B, Boutilier C (2006) Who’s asking for help? A Bayesian approach to intelligent assistance. In: IUI, pp 186–193

  16. Jaeger M (1997) Relational Bayesian networks. In: UAI-97

  17. Kearns MJ, Mansour Y, Ng AY (1999) A sparse sampling algorithm for near-optimal planning in large Markov decision processes. In: IJCAI

  18. Kersting K, De Raedt L (2000) Bayesian logic programs. In: ILP

  19. Kim H, Lee S (2004) An intelligent information system for organizing online text documents. KAIS 6(2): 125–149

    Google Scholar 

  20. Lafferty J, McCallum A, Pereira F (2001) Conditional Random Fields,b Probabilistic Models for Segmenting and Labeling Sequence Data. In: Proceedings of the 18th international conference on machine learning

  21. Laskey KB (2008) MEBN: a language for first-order Bayesian knowledge bases. Artif Intell 172(2–3): 140–178

    Article  MathSciNet  MATH  Google Scholar 

  22. Littman M, Cassandra AR, Kaelbling LP (2005) Learning policies for partially observable environments: scaling up. ICML

  23. Liu S, Duffy AHB, Whitfield RI, Boyle IM (2008) Integration of decision support systems to improve decision support performance. KAIS 22(31): 261–286

    Google Scholar 

  24. Muggleton S (1996) Stochastic logic programs. In: Advances in inductive logic programming

  25. Murphy K, Paskin M (2001) Linear time inference in hierarchical HMMs. In: NIPS

  26. Myers K, Berry P, Blythe J, Conleyn K, Gervasio M, McGuinness D, Morley D, Pfeffer A, Pollack M, Tambe M (2007) An intelligent personal assistant for task and time management.. AI Magazine 28(2): 47–61

    Google Scholar 

  27. Natarajan S, Tadepalli P, Altendorf E, Dietterich TG, Fern A, Restificar A (2005) Learning first-order probabilistic models with combining rules. In: Proceedings of ICML-05

  28. Ngo L, Haddawy P (1995) Probabilistic logic programming and Bayesian networks. In: Proceedings ACSC

  29. Qin B, Xia Y, Prabhakar S (2011) Rule induction for uncertain data. Knowl Inf Syst (KAIS), 1–28. doi:10.1007/s10115-010-0335-7

  30. Russell S, Norvig P (2002) Artificial Intelligence: a modern approach, 2nd edn. Prentice-Hall , New York

    Google Scholar 

  31. Sato T, Kameya Y (2001) Parameter learning of logic programs for symbolic-statistical modeling. J Artif Intell Res 15: 391–454

    MathSciNet  MATH  Google Scholar 

  32. Varakantham P, Maheswaran R, Tambe M (2005) Exploiting belief bounds: practical POMDPs for personal assistant agents. In: AAMAS

Download references

Author information

Authors and Affiliations

  1. School of EECS, Oregon State University, Corvallis, OR, USA

    Sriraam Natarajan, Prasad Tadepalli & Alan Fern

  2. Wake Forest University School of Medicine, Winston-Salem, NC, USA

    Sriraam Natarajan

Authors
  1. Sriraam Natarajan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Prasad Tadepalli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alan Fern
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sriraam Natarajan.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Natarajan, S., Tadepalli, P. & Fern, A. A relational hierarchical model for decision-theoretic assistance. Knowl Inf Syst 32, 329–349 (2012). https://doi.org/10.1007/s10115-011-0435-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-011-0435-z

Keywords

Search

Navigation