ns-3 meets OpenAI Gym: The Playground for Machine Learning in Networking Research

Recently, we have seen a boom of attempts to improve the operation of networking protocols using machine learning techniques. The proposed reinforcement learning (RL) based control solutions very often overtake traditionally designed ones in terms of performance and efficiency. However, in order to reach such a superb level, an RL control agent requires a lot of interactions with an environment to learn the best policies. Similarly, the recent advancements in image recognition area were enabled by the rise of large labeled datasets (e.g. ImageNet). This paper presents the ns3-gym - the first framework for RL research in networking. It is based on OpenAI Gym, a toolkit for RL research and ns-3 network simulator. Specifically, it allows representing an ns-3 simulation as an environment in Gym framework and exposing state and control knobs of entities from the simulation for the agent's learning purposes. Our framework is generic and can be used in various networking problems. Here, we present an illustrative example from the cognitive radio area, where a wireless node learns the channel access pattern of a periodic interferer in order to avoid collisions with it. The toolkit is provided to the community as open-source under a GPL license.