Crossbar Adaptive Array: The first connectionist network that solved the delayed reinforcement learning problem

Abstract

The paper discusses important issues for reinforcement learning agents, the issue of delayed reinforcement learning (DRL). It points out that an early agent, the Crossbar Adaptive Array (CAA) architecture, not widely known in connectionist and reinforcement learning community, was the first to solve the DRL problem among connectionist agents. The work contributes toward understanding the initial neuron-like computational efforts to solve the DRL problem, giving a comparison between CAA and the well-known Actor/Critic (AC) architecture. It also points out relevant contemporary issues of autonomous agents, the issue of genetic/behavioral environment and the issue of emotion based learning architectures.