Abstract
We present a framework for building agents that learn using SMART, a system that combines stochastic model acquisition with reinforcement learning to enable an agent to model its environment through experience and subsequently form action selection policies using the acquired model. We extend an existing algorithm for automatic creation of stochastic strips operators [9] as a preliminary method of environment modelling. We then define the process of generation of future states using these operators and an initial state and finally show the process by which the agent can use the generated states to form a policy with a standard reinforcement learning algorithm. The potential of SMART is exemplified using the well-known predator prey scenario. Results of applying SMART to this environment and directions for future work are discussed.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Boutilier, C., Dean, T., Hanks, S.: Decision-Theoretic Planning: Structural Assumptions and Computational Leverage. Journal of Artificial Intelligence Research 11, 1–94 (1999)
Dehaspe, L.: Maximum Entropy Modeling with Clausal Constraints. In: Džeroski, S., Lavrač, N. (eds.) ILP 1997. LNCS, vol. 1297, pp. 109–125. Springer, Heidelberg (1997)
Dzeroski, S., De Raedt, L., Blockeel, H.: Relational Reinforcement Learning. In: International Workshop on Inductive Logic Programming (1998)
Drescher, G.L.: Made-Up Minds, A Constructivist Approach to Artificial Intelligence. MIT Press, Cambridge (1991)
Fikes, R.E., Nilsson, N.J.: STRIPS: a new approach to the application of theorem proving to problem-solving. Artificial Intelligence 2(3-4), 189–208 (1971)
Hanks, S.: Projecting plans for uncertain worlds. Ph.D. thesis, Yale University, Department of Computer Science (1990)
Kaelbling, L.P., Littman, H.L., Moore, A.P.: Reinforcement Learning: A Survey. Journal of Artificial Intelligence Research 4, 237–285 (1996)
Muggleton, S.H.: Learning Stochastic Logic Programs. In: Getoor, L., Jensen, D. (eds.) Proceedings of the AAAI2000 Workshop on Learning Statistical Models from Relational Data. AAAI, Menlo Park (2000)
Oates, T., Schmill, M.D., Gregory, D.E., Cohen, P.R.: Detecting complex dependencies in categorical data. In: Chap. in Finding Structure in Data: Artificial Intelligence and Statistics V. Springer, Heidelberg (1995)
Oates, T., Cohen, P.R.: Learning Planning Operators with Conditional and Probabilistic Effects. In: AAAI 1996 Spring Symposium on Planning with Incomplete Information for Robot Problems. AAAI, Menlo Park (1996)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. A Bradford Book. MIT Press, Cambridge (1998)
Shen, W.: Discovery as Autonomous Learning from the Environment. Machine Learning 12, 143–165 (1993)
Tesauro, G.J.: TD-Gammon, a self-teaching backgammon program, achieves master-level play. Neural Computation 6(2), 215–219 (1994)
Varsy, R.: Extending Planning and Learning Through Reinterpretation of World Model. M.Sc. thesis, City Univesity (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Child, C., Stathis, K. (2005). SMART (Stochastic Model Acquisition with ReinforcemenT) Learning Agents: A Preliminary Report. In: Kudenko, D., Kazakov, D., Alonso, E. (eds) Adaptive Agents and Multi-Agent Systems II. AAMAS AAMAS 2004 2003. Lecture Notes in Computer Science(), vol 3394. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-32274-0_5
Download citation
DOI: https://doi.org/10.1007/978-3-540-32274-0_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-25260-3
Online ISBN: 978-3-540-32274-0
eBook Packages: Computer ScienceComputer Science (R0)