Ensembles and Multiple Classifiers: A Game-Theoretic View

Aggregating Strategies

The study of multiple classifier systems is a fundamental topic in modern machine learning. However, early work on aggregation of predictors can be traced back to the Fifties, in the area of game theory. At that time, the pioneering work of James Hannan [11] and David Blackwell [2] laid down the foundations of repeated game theory. In a nutshell, a repeated game is the game-theoretic interpretation of learning. In games played once, lacking any information about the opponent, the best a player can do is to play the minimax strategy (the best strategy against the worst possible opponent). In repeated games, by examining the history of past opponent moves, the player acquires information about the opponent’s behavior and can adapt to it, in order to achieve a better payoff than that guaranteed by the minimax strategy.