Anticipatory, Goal-Directed Behavior

Abstract

David Hume may be one of the first who thought about the causes that actually enable us to act goal-directedly in our pursuit of happiness. Besides having usually an end, or goal, in mind, Hume realized that the end must elicit those means that were learned to correlate with the end. Such correlation knowledge, according to Hume (1748), was based on three types of connecting “ideas”: resemblance, contiguity in time and place, and cause and effect. Knowledge of correlations and cause-effect relations alone, though, do not directly lead to effective behavior. Thus, not only the question how we learn correlations in the environment needs to be addressed, but also how we can exploit the obtained knowledge, if learned properly. While Hume did mainly address the former question, the latter question was acknowledged by Hume only in so far that the acquired knowledge may be used to pursue our goals.

Another related line of research on causation is put forward by Kant (1998). Sloman (2006) contrasted a ‘Humean’ and a ’Kantian’ view of understanding correlations and causations in particular. The former is evidence-based, probabilistic, and statistical. The latter is structure-based and deterministic. Kant highlights the role of concepts and necessity in contrast with the Humean emphasis on observation and correlation. The Kantian notion of causation is more complex and requires understanding of spatial structures and relationships as well as the capability to reason about what happens when they change. While humans are usually seen as explorers that learn correlations in the world based on experimentation—and thus more ’Humean’ evidence-based—they rely on and inevitably detect the ’Kantian’ a priori structures in our world given due to time, space, and physical constraints.

Just like humans, all other anticipatory systems, both biological and artificial, need to learn and store knowledge about themselves and the world. Only this knowledge enables them to predict future events, gauge the consequences of one’s own actions, and finally interact competently and intelligently with objects or other agents. Thus, one of the questions addressed in this chapter is how knowledge about the world or the self may be represented.

However, regardless if we acquire knowledge based on ’Humean’ or ’Kantian’ principles, or both, just having gathered knowledge about the world does not mean per se that our decisions will be wise and our actions will be appropriate. It is not even clear how predictive knowledge may be turned into actual decisions in general. The ideomotor principle,

which dates back to the 19-th century (James 1890; Herbart 1825; cf. Hoffmann et al. 2004), suggests that actions are bi-directionally linked to the effects they usually produce. Thus, once a goal is chosen and activated, the bi-directional links point to those actions that previously caused the goal to come about. While this still does not clarify the actual mechanism of selecting the appropriate means, it implies that an inverse mechanism is necessary that stores means to achieve current goals.

With the ideomotor principle as the basic principle of goal-directed behavior in mind, this chapter analyzes related predictive and anticipatory systems that learn predictive representations of their environment and can use those to act goal-directedly. Predictive systems are systems that are able to predict sensory inputs or pre-processed, more abstracted perceptual input. From an adaptive behavioral perspective most important are systems that learn such predictive representation. These predictive capabilities are an important part of any goal-directed behavioral system that is explicitly anticipatory. However, as suggested in the comparison of the insights put forward by David Hume and the ideomotor principle, predictive capabilities are only the first step toward an anticipatory behavioral system. Thus, the second question that this chapter addresses refers to the structures and processes that enable the selection of actions or decision making, based on the acquired knowledge.

We identify two fundamental classes of approaches that realize action selection based on predictive representations of sensory-motor correlations. First, schemas form a mental (internal) predictive world model, which encodes all kinds of properties, independent of possible tasks and goals. Although the representation might correspond to an exhaustive internal world model, the schemas alone cannot be used directly for decision making or action selection. Before a decision and action is made, internal processes are required that evaluate possible means in the light of current behavioral goals and desired states. Even more so, to be able to make complex decisions or execute meaningful actions, many schemas may have to be combined. Thus, schema approaches generally build a forward model that is used inversely for action selection.

Second, inverse models—in contrast to schema approaches—encode direct connections between behavioral goals and actions. Thus, they may be directly used for action decision making without any further processing. Inverse models can be seen as the result of abstracting or aggregating schemas because they focus on a generalized, inverted representation of properties in the world. In this sense, they can also be considered a world model, which is however rather limited compared to the world models realized in schema approaches.

Accordingly, this chapter first gives an overview of several kinds of schema approaches and inverse modeling approaches. We classify each system from an anticipatory behavior perspective discussing how knowledge is represented and which processes are necessary to turn anticipatory knowledge into behavior. As we cannot review all architectures that have been proposed to date, we exemplify each class of anticipatory behavioral systems with a representative model. Examples are chosen to provide details of state-of-the-art models of goal-directed behavior and to cover a broad range of approaches, including symbolic, subsymbolic, and neural models as well as supervised, unsupervised, and reinforcement learning approaches.

In the next section, we first give a brief history of schemas and provide a definition. We then distinguish different schema system classes and give system examples. Similarly, we discuss inverse model approaches, combinations of both approaches and other advanced techniques. In the second part of the chapter,

we assess weaknesses and strengths of the architectures in learning and representing predictions and in using those predictions for the generation of anticipatory cognitive functions. Finally, we contrast the systems’ capabilities and give an outlook on potential future macroscopic organizational structures for anticipatory systems, especially highlighting hierarchical and modular structures.