Agents that Reason and Learn

Abstract

This talk will address the issue of designing architectures for agents that need to be able to adapt to changing circumstances during deployment. From a scientific point of view, the primary challenge is to design agent architectures that seamlessly integrate reasoning and learning capabilities. That this is indeed a challenge is largely due to the fact that reasoning and knowledge representation capabilities of agents are studied in different subfields of computer science from the subfields in which learning for agents is studied. So far there have been few attempts to integrate these two research themes. In any case, agent architectures is very much an open issue with plenty of scope for new ideas.

The research to be described is being carried out in the context of the Smart Internet Technology Cooperative Research Centre [4], a substantial 7 year Australian research initiative having the overall research goal of making interactions that people have with the Internet much simpler than they are now. One of the research programs in the CRC is concerned with building Internet agents and one project in that program is concerned with building adaptive agents, the main topic of this talk.

The first attempt in this project at an architecture involves integrating BDI agent architectures for the reasoning component and reinforcement learning for the learning component. The talk will concentrate on a particular aspect of this integration, namely, approximation of the Q-function in reinforcement learning. In seminal work on relational reinforcement learning [1,2], the TILDE decision-tree learning system was employed to approximate the Q-function in various experiments in blocks world. An extremely attractive aspect of the use of a symbolic learning system for function approximation in reinforcement learning is that the functions learned are essentially plans that can be explicitly manipulated for various purposes. In the research to be described in this talk, the learning system used to approximate the Q-function is Alkemy, a decision-tree learning system with a foundation in higher-order logic [3]. The talk will describe the agent architecture and also progress towards building practical Internet agents. Along the way, a setting for predicate construction in higher-order logic used by Alkemy and some theoretical results concerning the efficient construction of predicates will be presented.