Applying Ant Colony Optimization algorithms for high-level behavior learning and reproduction from demonstrations

Highlights

• A behavior learning method based on Ant Colony Optimization (ACO) is proposed.

• We combined ACO, Semantic Networks and Spreading Activation mechanisms.

• The method is able to learn high-level aspects of behaviors from demonstrations.

• The method answers questions of “What to imitate” and “When to imitate”.

• The method generalizes concepts while learning high-level aspects of behaviors.

Abstract

In domains where robots carry out human’s tasks, the ability to learn new behaviors easily and quickly plays an important role. Two major challenges with Learning from Demonstration (LfD) are to identify what information in a demonstrated behavior requires attention by the robot, and to generalize the learned behavior such that the robot is able to perform the same behavior in novel situations.

The main goal of this paper is to incorporate Ant Colony Optimization (ACO) algorithms into LfD in an approach that focuses on understanding tutor’s intentions and learning conditions to exhibit a behavior. The proposed method combines ACO algorithms with semantic networks and spreading activation mechanism to reason and generalize the knowledge obtained through demonstrations. The approach also provides structures for behavior reproduction under new circumstances. Finally, applicability of the system in an object shape classification scenario is evaluated.

Introduction

During the past years robot task learning has received remarkable attention and motivated the robotics community to take a deeper interest in techniques based on human skill learning from observation  [1]. In robotics, such an approach fits in the framework of Learning from Demonstration (LfD).

In the current paper, we address the questions “What to imitate” and “When to imitate” from a high-level perspective, while employing methods to learn and reproduce motor actions from demonstrations. Therefore, our focus is on learning and reproducing high-level aspects of demonstrated behaviors. For this purpose, we utilize a core Semantic Network (SN) as a model to represent behaviors by nodes and link them to a set of other nodes, which correspond to concepts and objects in the real world. The network contains concepts, objects and their properties required for learning and reproducing behaviors, and must be provided prior to the learning and reproducing process. The learned behaviors are then used as object affordances as well as preparing the ground for behavior arbitration. Our learning methods also provide techniques to learn conditions that result in the behavior and thus answer the question of “When to imitate”. These conditions can be environmental, objects to use, and concepts related to demonstrated behavior. Therefore, depending on the amount of knowledge available in the core SN, the robot may perceive enormous amounts of information during learning. In many cases when the desired behavior is very complex or is demonstrated in an ambiguous manner, the robot requires a bias in order to focus on the right aspects of the demonstration  [2]. By having a controller from a higher abstraction level to guide the robot especially during the learning phase, the robot’s attention can be directed to aspects of demonstration that are significant for learning behaviors and thereby answering question of “What to imitate”. This controller is a part of an architecture that has been proposed and developed in our previous work  [3], [4], and is employed in the current paper.

Our current work introduces, as an original contribution, the use of SNs for biasing the robot and the use of ACO algorithms to learn new behaviors and define the degree of generalization in order to exhibit the learned behaviors in situations new to the robot. Our previously developed learning method  [4] is based on one-way ANOVA test and has some observed limitations caused by the imposed statistical constraints. This makes the method less successful in noisy environments. To address this issue, our new method views the problem of behavior learning as an optimization problem and applies ACO algorithms to determine the most related elements of demonstrations. Nodes and links in the core SN represent elements of a demonstration and the goal is to find the shortest path between each element’s node and a behavior node using pheromone-laying behavior implementation of ACO.

The rest of the article is organized in the following manner: Section  2 provides background of the work, Section  3 presents principle elements of SNs for modeling the world and representing behaviors, Section  4 provides adapted formulations of ACO algorithms for the purpose of learning behaviors, Section  5 elaborates learning and reproduction of high-level representation of behaviors using ACO and SN, Section  6 presents the experimental setup and results, Section  7 gives a discussion of the approach and finally in Section  8 we make the conclusions.