RILNET: A Reinforcement Learning Based Load Balancing Approach for Datacenter Networks

Abstract

Modern datacenter networks are facing various challenges, e.g., highly dynamic workloads, congestion, topology asymmetry. ECMP, as a traditional load balancing mechanism which is widely used in today’s datacenters, can balance load poorly and lead to congestion. Variety of load balancing schemes are proposed to address the problems of ECMP. However, these traditional schemes usually make load balancing decision only based on network knowledge for a snapshot or a short time past. In this paper, we propose a Reinforcement Learning (RL) based approach, called RILNET (ReInforcement Learning NETworking), aiming at load balancing for datacenter networks. RILNET employs RL to learn a network and control it based on the learned experience. To achieve a higher granularity of control, RILNET is constructed to route flowlet rather than flows. Moreover, RILNET makes routing decisions for aggregation flows (an aggregation flow is a flow set that includes all flows flowing from the same source edge switch to the same destination edge switch) instead of a single flow. In order to test performance of RILNET, we propose a flow-level simulation and a packet-level simulation, and the both results show that RILNET can balance traffic load much more effectively than ECMP and another load balancing solution, i.e., DRILL. Compared with DRILL, RILNET outperforms DRILL in data loss and maximal link delay. Specifically, the maximal link data loss and the maximal link delay of RILNET are 44.4% and 25.4% smaller than DRILL, respectively.