Towards selective attention: generating image features by learning a visuo-motor map

Abstract

Robots require a form of visual attention to perform a wide range of tasks effectively. Existing approaches specify in advance the image features and attention control scheme required for a given robot to perform a specific task. However, to cope with different tasks in a dynamic environment, a robot should be able to construct its own attentional mechanisms. This paper presents a method that a robot can use to generating image features by learning a visuo-motor map. The robot constructs the visuo-motor map from training data, and the map constrains both the generation of image features and the estimation of state vectors. The resulting image features and state vectors are highly task-oriented. The learned mechanism is attentional in the sense that it determines what information to select from the image to perform a task. We examine robot experiments using the proposed method for indoor navigation and scoring soccer goals.

Introduction

Through billions of years of evolution, biological systems have acquired their organs and strategies to survive in hostile environments. Visual attention can be regarded as a combination of such organs and strategies: vision captures a huge amount of data about the external world, and attentional mechanisms extract information necessary for the system to achieve the mission at hand. This capability is desirable for artificial systems, and it has remained one of the most formidable issues in robotics and AI for many years.

Human beings can readily exploit attentional mechanisms in various kinds of situations, and much research focuses on the early visual processing of human beings [8], [16], [19], [20]. Some research applies Shannon’s information theory to the observed image to select the focus of attention in the view [14]. The main emphasis of this work is the analysis of human visual processing and the explanation of our own attentional mechanisms.

Some computer vision researchers focused on the viewpoint selection (i.e., where to look) problem [1], [13] in order to disambiguate the descriptions for the observed image that is obtained by matching the image with a model database. The selection criterion is based on the statistics of image data. The actions (gaze control) are intended to obtain a better observation for object recognition, but are not directly related to the physical actions needed to accomplish a given task beyond those required for recognition.

Some robot researchers focused on the attention problem of robot vision. Thrun [15] and Vlassis et al. [17] extracted image features correlated with the mobile robot’s self-localization information from the observed images based on a probabilistic method. Kröse and Bunschoten [7] decided the robot direction, that is, the camera direction, by minimizing the conditional entropy of the robot position given the observations. These methods are considered to be task-relevant visual attention but are not related to any physical actions.

As mentioned above, there are many methods to construct attentional mechanisms. However, existing methods do not generally take the robot’s (or human’s) physical actions into consideration.

Meanwhile, in biological systems, feature extracting cells in the visual cortex develop depending on visual and motor experiences. The functions of these cells are not innate, but are adaptively acquired depending on visual experiences in early development after birth [2], [5]. Furthermore, self-produced movement with its concurrent visual feedback is necessary for the development of visuo-motor coordination [4].

In light of these neurophysiological studies, the visual organs of a robot should develop depending on its visual and motor experiences. Linsker [9], [10] showed that an orientation-selective cell emerged in an artificial multilayered network using modified Hebbian learning. This result shows that the artificial system can learn a visual function similar to the one found in the brain. However, this is a closed system with respect to visual experiences.

In this paper, we focus on extracting image features as an attentional mechanism, and propose a method for image feature (e.g., edges or color regions) generation by visuo-motor map learning depending on the experience gathered by the robot while performing a task. The training data constructs the visuo-motor mapping that constrains image feature generation and state vector estimation for the selection of actions. That is, the state space is constructed so that the correlation between a state and a given instruction can be maximized. The resultant image feature and state vector are task-oriented. The method is applied to indoor navigation and scoring soccer tasks.

There are some existing methods to construct the visual state spaces through task execution (e.g. [6], [12]). These methods can construct the task-oriented state vector, but they have not focused on image features. The proposed method constructs the task-oriented visual state space and image features that are useful for selective attention.

The remainder of the paper is organized as follows. First, we describe the basic idea of image feature generation along with the learning formulation. We use the projection matrix from the extracted image feature to the state vector to determine the optimal action. Next, we give experimental results to show the validity of the proposed method. Finally, we conclude with a discussion on the attentional mechanism suggested by the current results.