Loading [a11y]/accessibility-menu.js
Lamarckian Platform: Pushing the Boundaries of Evolutionary Reinforcement Learning Toward Asynchronous Commercial Games | IEEE Journals & Magazine | IEEE Xplore

Lamarckian Platform: Pushing the Boundaries of Evolutionary Reinforcement Learning Toward Asynchronous Commercial Games


Abstract:

Despite the emerging progress of integrating evolutionary computation into reinforcement learning, the absence of a high-performance platform endowing composability and m...Show More

Abstract:

Despite the emerging progress of integrating evolutionary computation into reinforcement learning, the absence of a high-performance platform endowing composability and massive parallelism causes nontrivial difficulties for research and applications related to asynchronous commercial games. Here, we introduce Lamarckian1—an open-source platform featuring support for evolutionary reinforcement learning scalable to distributed computing resources. To improve the training speed and data efficiency, Lamarckian adopts optimized communication methods and an asynchronous evolutionary reinforcement learning workflow. To meet the demand for an asynchronous interface by commercial games and various methods, Lamarckian tailors an asynchronous Markov Decision Process interface and designs an object-oriented software architecture with decoupled modules. In comparison with the state-of-the-art RLlib, we empirically demonstrate the unique advantages of Lamarckian on benchmark tests with up to 6000 CPU cores: 1) both the sampling efficiency and training speed are doubled when running proximal policy optimization (PPO) on Google football game; 2) the training speed is 13 times faster when running PBT+PPO on Pong game. Moreover, we also present two use cases: 1) how Lamarckian is applied to generating behavior-diverse game AI; 2) how Lamarckian is applied to game balancing tests for an asynchronous commercial game.
Published in: IEEE Transactions on Games ( Volume: 16, Issue: 1, March 2024)
Page(s): 51 - 63
Date of Publication: 21 September 2022

ISSN Information:

Funding Agency:


References

References is not available for this document.