Abstract
We consider an agent which learns a relational action model in order to be able to predict the effects of his actions. The model consists of a set of STRIPS-like rules, i.e. rules predicting what has changed in the current state when applying a given action as far as a set of preconditions is satisfied by the current state. Here several rules can be associated to a given action, therefore allowing to model conditional effects. Learning is online, as examples result from actions performed by the agent, and incremental, as the current action model is revised each time it is contradicted by unexpected effects resulting from his actions. The form of the model allows using it as an input of standard planners.
In this work, the learning unit IRALe is embedded in an integrated system able to i) learn an action model ii) select its actions iii) plan to reach a goal. The agent uses the current action model to perform active learning, i.e. to select actions with the purpose of reaching states that will enforce a revision of the model, and uses its planning abilities to have a realistic evaluation of the accuracy of the model.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ade, H., Malfait, B., De Raedt, L.: Ruth: an ILP Theory Revision System. In: Raś, Z.W., Zemankova, M. (eds.) ISMIS 1994. LNCS, vol. 869, pp. 336–345. Springer, Heidelberg (1994)
Benson, S.: Inductive learning of reactive action models. In: ICML 1995, pp. 47–54 (1995)
Biba, M., Ferilli, S., Esposito, F., Di Mauro, N., Basile, T.M.A.: A fast partial memory approach to incremental learning through an advanced data storage framework. In: Proceedings of the Fifteenth Italian Symposium on Advanced Database Systems, SEBD 2007, pp. 52–63 (2007)
Croonenborghs, T., Ramon, J., Blockeel, H., Bruynooghe, M.: Online learning and exploiting relational models in reinforcement learning. In: IJCAI, pp. 726–731 (2007)
Dabney, W., McGovern, A.: Utile distinctions for relational reinforcement learning. In: IJCAI, pp. 738–743 (2007)
Driessens, K., Ramon, J., Blockeel, H.: Speeding Up Relational Reinforcement Learning through the Use of an Incremental First Order Decision Tree Learner. In: Flach, P.A., De Raedt, L. (eds.) ECML 2001. LNCS (LNAI), vol. 2167, pp. 97–108. Springer, Heidelberg (2001)
Dzeroski, S., De Raedt, L., Driessens, K.: Relational reinforcement learning. Machine Learning 43, 7–52 (2001)
Esposito, F., Ferilli, S., Fanizzi, N., Basile, T.M.A., Di Mauro, N.: Incremental learning and concept drift in inthelex. Intell. Data Anal. 8(3), 213–237 (2004)
Gil, Y.: Learning by experimentation: Incremental refinement of incomplete planning domains. In: ICML, pp. 87–95 (1994)
Hoffmann, J.: Ff: The fast-forward planning system. The AI Magazine (2001)
Jiménez, S., Fernández, F., Borrajo, D.: The pela architecture: integrating planning and learning to improve execution. In: 23rd National Conference on Artificial Intelligence, vol. 3, pp. 1294–1299. AAAI Press (2008)
Kearns, M., Singh, S.: Near-optimal reinforcement learning in polynomial time. Machine Learning 49 (2002)
Lang, T., Toussaint, M., Kersting, K.: Exploration in Relational Worlds. In: Balcázar, J.L., Bonchi, F., Gionis, A., Sebag, M. (eds.) ECML PKDD 2010. LNCS, vol. 6322, pp. 178–194. Springer, Heidelberg (2010)
Li, L., Littman, M.L., Walsh, T.J.: Knows what it knows: a framework for self-aware learning. In: ICML, pp. 568–575 (2008)
Maloof, M.A., Michalski, R.S.: Incremental learning with partial instance memory. Artif. Intell. 154(1-2), 95–126 (2004)
McDermott, D.: The 1998 ai planning systems competition. AI Magazine 21(2), 35–55 (2000)
Otero, R.P.: Induction of the Effects of Actions by Monotonic Methods. In: Horváth, T., Yamamoto, A. (eds.) ILP 2003. LNCS (LNAI), vol. 2835, pp. 299–310. Springer, Heidelberg (2003)
Otero, R.P.: Induction of the Indirect Effects of Actions by Monotonic Methods. In: Kramer, S., Pfahringer, B. (eds.) ILP 2005. LNCS (LNAI), vol. 3625, pp. 279–294. Springer, Heidelberg (2005)
Pasula, H.M., Zettlemoyer, L.S., Kaelbling, L.: Learning symbolic models of stochastic domains. Journal of Artificial Intelligence Research (JAIR) 29, 309–352 (2007)
Pasula, H.M., Zettlemoyer, L.S., Pack Kaelbling, L.: Learning probabilistic planning rules. In: ICAPS, pp. 146–163 (2004)
Richards, B.L., Mooney, R.J.: Automated refinement of first-order horn-clause domain theories. Machine Learning 19, 95–131 (1995)
Rodrigues, C., Gérard, P., Rouveirol, C., Soldano, H.: Incremental learning of relational action rules. In: ICMLA (2010)
Settles, B.: Active Learning Literature Survey. Technical Report Technical Report 1648, University of Wisconsin-Madison (2009)
Shen, W.M.: Discovery as autonomous learning from the environment. Machine Learning 12(1-3), 143–165 (1993)
Sutton, R.S.: Dyna, an integrated architecture for learning, planning, and reacting. SIGART Bull. 2, 160–163 (1991)
Walsh, T.J., Littman, M.L.: Efficient learning of action schemas and web-service descriptions. In: AAAI, pp. 714–719 (2008)
Walsh, T.J., Szita, I., Diuk, M., Littman, M.L.: Exploring compact reinforcement-learning representations with linear regression. In: UAI, pp. 714–719 (2009)
Wang, X.: Learning by observation and practice: An incremental approach for planning operator acquisition. In: ICML, pp. 549–557 (1995)
Yang, Q., Wu, K.: Y Jiang. Learning action models from plan examples using weighted max-sat. Artificial Intelligence 171(2-3), 107–143 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Rodrigues, C., Gérard, P., Rouveirol, C., Soldano, H. (2012). Active Learning of Relational Action Models. In: Muggleton, S.H., Tamaddoni-Nezhad, A., Lisi, F.A. (eds) Inductive Logic Programming. ILP 2011. Lecture Notes in Computer Science(), vol 7207. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-31951-8_26
Download citation
DOI: https://doi.org/10.1007/978-3-642-31951-8_26
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-31950-1
Online ISBN: 978-3-642-31951-8
eBook Packages: Computer ScienceComputer Science (R0)