A Q-Learning-Based Hyper-Heuristic Evolutionary Algorithm for the Distributed Flexible Job-Shop Scheduling Problem

Abstract

As an important branch of distributed scheduling, distributed flexible job shop scheduling problem (DFJSP) has become an emerging production pattern. This article proposes a Q-learning-based hyper-heuristic evolutionary algorithm (QHHEA) to minimize the maximum completion time (i.e., makespan) of DFJSP. First, a hybrid initialization strategy is introduced to acquire a high-quality initial population with certain diversity. Second, according to DFJSP’s characteristics, we design a three-dimensional vector coding scheme, and left-shift method is embedded into the decoding stage to improve the utilization of machines. Third, six simple but effective low-level neighborhood structures are devised. Forth, a Q-learning-based high-level strategy (QHLS) is developed to automatically learn the execution order of the low-level neighborhood structure. Moreover, a novel definition of the state and a dynamic adaptive parameter mechanism are developed to avoid QHHEA trapping in the local optima. Finally, comprehensive comparisons are conducted for our QHHEA against several state-of-the-art algorithms based on 18 instances. And the statistical results demonstrate the efficiency and effectiveness of the proposed QHHEA in solving the DFJSP.