A Deep Reinforcement Learning Algorithm Using A New Graph Transformer Model for Routing Problems

Abstract

Routing problems, which belong to a classical kind of problem in combinatorial optimization, have been extensively studied for many decades by researchers from different backgrounds. In recent years, Deep Reinforcement Learning (DRL) has been applied widely in self-driving, robotics, industrial automation, video games, and other fields, showing its strong decision-making and learning ability. In this paper, we propose a new graph transformer model, based on the DRL algorithm, for minimizing the route lengths of a given routing problem. Specifically, the actor-network parameters are trained by an improved REINFORCE algorithm to effectively reduce the variance and adjust the frequency of the reward values. Further, positional encoding is used in the encoding structure to make the multiple nodes satisfy translation invariance during the embedding process and enhance the stability of the model. The aggregate operation of the graph neural network applies to transformer model decoding stage at this time, which effectively captures the topological structure of the graph and the potential relationships between nodes. We have used our model to two classical routing problems, i.e., Traveling Salesman Problem (TSP) and Capacitate Vehicle Routing Problem (CVRP). The experimental results show that the optimization effect of our model on small and medium-sized TSP and CVRP surpasses the state-of-the-art DRL-based methods and some traditional algorithms. Meanwhile, this model also provides an effective strategy for solving combinatorial optimization problems on graphs.