An Experimental Study of Emergence of Communication of Reinforcement Learning Agents

Abstract

Ability to use language is an essential requirement for human-level intelligence. For artificial general intelligence, the ability to learn and to create language is even more important [1]. Most previous models of learning and emergence of language took successful communication itself as the task target. However, language, or communication in general, should have evolved to improve certain fitness of the population of agents. Here we consider whether and how a population of reinforcement learning agents can learn to send signals and to respond to signals for the sake of maximizing their own rewards. We take a communication game tested in human subjects [2, 3, 6], in which the aim of the game is for two players to meet together without knowing exact location of the other. In our decentralized reinforcement learning framework with communicative and physical actions [4], we tested how the number N of usable symbols affects whether the meeting task is successfully achieved and what kind of signaling and responding are learned. Even though $N=2$ symbols are theoretically sufficient, the success rate was only 1 to 2%. With $N=3$ symbols, success rate was more than 60% and three different signaling strategies were observed. The results indicate the importance of redundancy in signaling degrees of freedom and that a variety of signaling conventions can emerge in populations of simple independent reinforcement learning agents.