Skip to main content
SpringerLink
Log in

Symbolic Dynamic Programming

  • Reference work entry
  • First Online:
Encyclopedia of Machine Learning and Data Mining

  • 172 Accesses

Synonyms

Dynamic programming for relational domains; Relational dynamic programming; Relational value iteration; SDP

Definition

Symbolic dynamic programming (SDP) is a generalization of the dynamic programming technique for solving Markov decision processes (MDPs) that exploits the symbolic structure in the solution of relational and first-order logical MDPs through a lifted version of dynamic programming.

Motivation and Background

Decision-theoretic planning aims at constructing a policy for acting in an uncertain environment that maximizes an agent’s expected utility along a sequence of steps. For this task, Markov decision processes (MDPs) have become the standard model. However, classical dynamic programming algorithms for solving MDPs require explicit state and action enumeration, which is often impractical: the number of states and actions grows very quickly with the number of domain objects and relations. In contrast, SDP algorithms seek to avoid explicit state and action...

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 699.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 949.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Recommended Reading

  • Bellman RE (1957) Dynamic programming. Princeton University Press, Princeton

    MATH  Google Scholar 

  • Boutilier C, Reiter R, Price B (2001) Symbolic dynamic programming for first-order MDPs. In: IJCAI-01, Seattle, pp 690–697

    Google Scholar 

  • Fern A, Yoon S, Givan R (2003) Approximate policy iteration with a policy language bias. In: NIPS-2003, Vancouver

    Google Scholar 

  • Fikes RE, Nilsson NJ (1971) STRIPS: a new approach to the application of theorem proving to problem solving. Artif Intell 2:189–208

    Article  MATH  Google Scholar 

  • Gretton C, Thiebaux S (2004) Exploiting first-order regression in inductive policy selection. In: UAI-04, Banff, pp 217–225

    Google Scholar 

  • Guestrin C, Koller D, Gearhart C, Kanodia N (2003) Generalizing plans to new environments in relational MDPs. In: IJCAI-03, Acapulco

    Google Scholar 

  • Hölldobler S, Skvortsova O (2004) A logic-based approach to dynamic programming. In: AAAI-04 workshop on learning and planning in MDPs, Menlo Park, pp 31–36

    Google Scholar 

  • Karabaev E, Skvortsova O (2005) A heuristic search algorithm for solving first-order MDPs. In: UAI-2005, Edinburgh, pp 292–299

    Google Scholar 

  • Kersting K, van Otterlo M, De Raedt L (2004) Bellman goes relational. In: ICML-04. ACM Press, New York

    Google Scholar 

  • Kushmerick N, Hanks S, Weld D (1995) An algorithm for probabilistic planning. Artif Intell 76:239–286

    Article  Google Scholar 

  • Puterman ML (1994) Markov decision processes: discrete stochastic dynamic programming. Wiley, New York

    Book  MATH  Google Scholar 

  • Sanner S, Boutilier C (2005) Approximate linear programming for first-order MDPs. In: UAI-2005, Edinburgh

    Google Scholar 

  • Sanner S, Boutilier C (2006) Practical linear evaluation techniques for first-order MDPs. In: UAI-2006, Boston

    Google Scholar 

  • Sanner S, Boutilier C (2007) Approximate solution techniques for factored first-order MDPs. In: ICAPS-07, Providence, pp 288–295

    Google Scholar 

  • Wang C, Khardon R (2007) Policy iteration for relational MDPs. In: UAI, Vancouver

    Google Scholar 

  • Wang C, Joshi S, Khardon R (2007) First order decision diagrams for relational MDPs. In: IJCAI, Hyderabad

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Statistical Machine Learning Group, NICTA, Canberra, ACT, Australia

    Scott Sanner

  2. Technische Universität Dortmund, Dortmund, Germany

    Kristian Kersting

  3. Knowledge Discovery, Fraunhofer IAIS, Sankt Augustin, Germany

    Kristian Kersting

Authors
  1. Scott Sanner
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kristian Kersting
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. The University of New South Wales, Sydney, NSW, Australia

    Claude Sammut

  2. Faculty of Information Technology, Monash University, Melbourne, VIC, Australia

    Geoffrey I. Webb

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer Science+Business Media New York

About this entry

Cite this entry

Sanner, S., Kersting, K. (2017). Symbolic Dynamic Programming. In: Sammut, C., Webb, G.I. (eds) Encyclopedia of Machine Learning and Data Mining. Springer, Boston, MA. https://doi.org/10.1007/978-1-4899-7687-1_806

Download citation

Publish with us

Policies and ethics

Search

Navigation