An experimental analysis of local minima to improve neighbourhood search

https://doi.org/10.1016/S0305-0548(02)00128-4Get rights and content

Abstract

The paper reports the results from a number of experiments on local search algorithms applied to job shop scheduling problems. The main aim was to get insights into the structure of the underlying configuration space. We consider the disjunctive graph representation where the objective function of job shop scheduling is equal to the length of longest paths. In particular, we analyse the number of longest paths, and our computational experiments on benchmark problems provide evidence that in most cases optimal and near optimal solutions do have a small number of longest paths. For example, our best solutions have one to five longest paths only while the maximum number is about sixty longest paths. Based on this observation, we investigate a non-uniform neighbourhood for simulated annealing procedures that gives preference to transitions where a decrease of the number of longest paths is most likely. The results indicate that the non-uniform strategy performs better than uniform methods, i.e. the non-uniform approach has a potential to find better solutions within the same number of transition steps. For example, we obtain the new upper bound 886 on the 20×20 benchmark problem YN1.

Scope and purpose

The paper reports a number of experiments with local search algorithms applied to job shop scheduling (JSS). The JSS problem is defined as follows: Given a number of l jobs, the jobs have to be processed on m machines. Each job consists of a sequence of m tasks, i.e., each task of a job is assigned to a particular machine. The tasks have to be processed during an uninterrupted time period of a fixed length on a given machine. A schedule is an allocation of the tasks to time intervals on the machines and the aim is to find a schedule that minimises the overall completion time which is called the makespan. The scheduling problem is one of the hardest combinatorial optimization problems (cf. M.R. Garey, D.S. Johnson, SIAM J. Comput. 4(4) (1975) 397. Many methods have been proposed to find good approximations of optimum solutions to job shop scheduling problems; for an overview (see E.H.L. Aarts, Local Search in Combinatorial Optimization, Wiley, New York, 1998). In our paper, the main aim is to get insights into the structure of the underlying configuration space. We consider the disjunctive graph representation where the objective function of job shop scheduling is equal to the length of longest paths. In particular, we analyse the number of longest paths, and our computational experiments on benchmark problems provide evidence that in most cases optimal and near optimal solutions do have a small number of longest paths. For example, our best solutions have one to five longest paths only while the maximum number is about sixty longest paths. Based on this observation, we investigate a non-uniform neighbourhood for simulated annealing procedures that gives preference to transitions where a decrease of the number of longest paths is most likely. The results indicate that the non-uniform strategy performs better than uniform methods, i.e., the non-uniform approach has a potential to find better solutions within the same number of transition steps. For example, we obtain the new upper bound 886 on the 20×20 benchmark problem YN1.

Introduction

Local search is often the method of choice when dealing with combinatorial optimization problems such as scheduling. Problem size or lack of insight into the problem structure may prohibit the application of true optimization algorithms. Local search provides a robust approach to obtain high quality solutions to problems of a realistic size in reasonable time.

Roughly speaking, a local search algorithm starts off with an initial solution and then continually tries to find better solutions by searching neighbourhoods. The algorithm walks through the configuration space such that each solution visited is a neighbour of the previously visited one. A solution is called a local minimum with respect to a neighbourhood function, if all its neighbours have the same or a worse value of the objective function. Clearly, if the algorithm selects always the best or at least a better-cost neighbour, the algorithm will end up in a local minimum.

Simulated annealing is an algorithmic method that is able to escape from local minima. It is a randomised local search method for two reasons: First, from the neighbourhood of a solution a neighbour is randomly selected. Secondly, in addition to better-cost neighbours, which are always accepted if they are selected, worse-cost neighbours are also accepted, although with a probability that is gradually decreased in the course of the algorithm's execution. The randomised nature enables asymptotic convergence to optimum solutions under certain mild conditions. Nevertheless, the energy landscape, which is determined by the objective function and the neighbourhood structure, may admit many and/or “deep” local minima. Therefore, avoiding local minima is a crucial part of the performance of the algorithm.

In this paper, we report an experimental analysis of the energy landscape of job shop scheduling. In this optimization problem one considers a set of l jobs consisting of tasks and a set of m machines which can handle at most one task at a time. We consider non-preemptive scheduling, i.e., the tasks of each job have to be processed during an uninterrupted time of fixed length on a given machine. A schedule is an allocation of tasks to time intervals on the machines. The goal is to find a schedule that minimizes the overall completion time which is called the makespan.

We focus specifically on the objective criterion of makespan minimization since it has received the most attention within the job shop scheduling literature. The problem of finding a minimum makespan can be reformulated as one of solving a series of different but related deadline scheduling problems. For a presentation and discussion of the importance of makespan minimization we refer to [1].

We apply simulated annealing-based heuristics to the NP-complete [2], [3] problem of job shop scheduling (for local search methods and, in particular, simulated annealing, see [4], [5], [6]). The paper is a continuation of [7], [8], where a new neighbourhood function based on the number of longest paths was introduced.

First, we analyse the algorithm's ability to escape from local minima when the new neighbourhood function is used. It is known that job shop scheduling induces a very complex energy landscape with a huge number of local minima [9]. In our paper, we are concentrating on a deeper analysis of the structure of near optimum solutions. In particular, we have analysed the number of longest paths in the disjunctive graph representation by a large number of computational experiments. These experiments were performed by using a cooling schedule with an expected run-time of O(n4+ε/m), where n=l·m and ε represents O(lnlnn/lnn) [8].

During our experiments we observed in most cases a small number of longest paths in local minima. These results lead us to the conjecture that global minima do have only a few longest paths. Based on the conjecture, one can argue that a large number of longest paths in best solutions indicates that the optimum still has not been found.

Secondly, we compare our non-uniform neighbourhood to a uniform one. The non-uniform function is determined by reversing more than a single arc on a machine path. The selection of arcs depends on the number of longest paths to which a particular arc belongs. The uniform neighbourhood relation allows to reverse a single arc on a longest path only and is described in [6]. A comparison between both neighbourhoods indicates that the non-uniform neighbourhood function has a potential to find better solutions within the same number of transition steps.

In the next section, we describe the representation of feasible schedules based on directed acyclic graphs and the neighbourhood function. In Section 3, we give a run-time estimation of the simulated annealing procedure resulting from the combination of the non-uniform neighbourhood and the utilized cooling schedule. We report some computational experiments with the procedure and compare the results with genetic algorithms and a variant of the shifting bottleneck procedure. Section 4 presents our experimental analysis of the number of longest paths near global minima. In Section 5 we concentrate on the robustness of the algorithm and in Section 6 we compare the non-uniform neighbourhood function to a uniform one.

Section snippets

Problem formulation and neighbourhood relation

The job shop scheduling problem can be formalized as follows. Given a set J of l jobs and a M set of m machines such that each job j∈J consists of a sequence T1j,T2j,…,Tmj of m tasks with Tij∈T. Each task Tij requires processing on a given machine M(Tij)∈M and Ti+1j, for i=1,…,m−1 must not start before its predecessor Tij is completed. We consider the model where preemption of the given processing times p(Tij)∈N is not allowed. Each machine can process only one task at a time and is available

Counting longest paths and global minima

The following results emerged from a more in-depth-analysis of the structure of near-optimal solutions. In particular, we analysed the number of longest paths in the disjunctive graph representation by a large number of computational experiments.

In Fig. 5, Fig. 6, Fig. 7, Fig. 8, statistics of the maximum number of longest paths are given. For this experiment we have chosen the smaller instance FT10 [11], LA38 and LA40 [12], two of the four instances YN [13], and two of the instances SWV [14].

Computational experiments on selected benchmarks

Simulated annealing algorithms are acting within a configuration space in accordance with a certain neighbourhood relation. The transitions between adjacent elements of the configuration space are governed by an objective function in conjunction with a certain cooling schedule. The objective function is given by (1), and the neighbourhood relation has been defined and analysed in 2 Problem formulation and neighbourhood relation, 3 Counting longest paths and global minima. Thus, it remains to

Robustness of the algorithm

Table 3 presents typical results out of five consecutive computational experiments with our algorithm CSSAW in comparison with the simulated annealing procedure SAL of Van Laarhoven et al. and SAL(t→∞), respectively. The algorithm SAL(t→∞) denotes a simulated annealing algorithm using the neighbourhood function of Van Laarhoven et al. but was analysed and run by Aarts et al., see [21] (no run-time is provided in the literature for these results). All reported results of SAL and SAL(t→∞) in the

An empirical comparison between neighbourhoods

Many algorithms that have been developed for scheduling are usually compared with each other in an empirical way. Beck et al. describe in [22] some dangers of the empirical testing. They unmask pitfalls which, in general, obfuscating results, blurring comparisons among scheduling algorithms and algorithm components, and therefore complicating validation of work in the literature. In our experiments, which we have just described, we tried to avoid pitfalls by using not only the CPU time as a

Concluding remarks

We have presented a number of experimental results related to the run-time behaviour of simulated annealing-based heuristics. In particular, we have investigated the structure of optimum and near optimum solutions of job shop scheduling problems. The experiments have confirmed the choice of a new, non-uniform neighbourhood relation.

The new neighbourhood is defined within the disjunctive graph model and gives preference to transitions where a larger number of longest paths is destroyed. The

Acknowledgements

Research partially supported by the Strategic Research Programme at The Chinese University of Hong Kong under Grant No. SRP 9505, by a Hong Kong Government RGC Earmarked Grant, Ref. No. CUHK 4010/98E.

The authors would like to thank the referees for their careful reading of the manuscript and helpful suggestions that resulted in an improved presentation.

Kathleen Steinhöfel is a researcher at the Institute for Computer Architecture and Software Technology of the Frauenhofer Gesellschaft in Berlin. She received her diploma in Informatics from the Technical University Leipzig and the Ph.D. degree from the Technical University Berlin. Her research interests are in stochastic algorithms and scheduling theory.

References (23)

  • Roy B, Sussmann B. Les problèmes d'Ordonnancement avec Constraints Disjonctives. Note DS No. 9 bis. SEMA,...

    • Cited by (0)

    Kathleen Steinhöfel is a researcher at the Institute for Computer Architecture and Software Technology of the Frauenhofer Gesellschaft in Berlin. She received her diploma in Informatics from the Technical University Leipzig and the Ph.D. degree from the Technical University Berlin. Her research interests are in stochastic algorithms and scheduling theory.

    Andreas Albrecht is a Senior Lecturer in the Computer Science Dept. of Hertfordshire University. He received the Diploma in Mathematics from Moscow State University, and the Ph.D. and the Habilitation degrees in Mathematics from Humboldt University at Berlin. His research interests include complexity problems of Boolean functions, combinatorial optimization, and algorithmic learning theory.

    C.K. Wong is a Professor of Computer Science and Engineering at The Chinese University of Hong Kong, on leave from IBM. He received the B.A. degree in mathematics from the University of Hong Kong, and the M.A. and Ph.D. degrees in mathematics from Columbia University. His research interests include combinatorial algorithms, and algorithms arising directly from industrial applications, such as mass storage systems, magnetic bubble memories, computer network design, satellite communications and VLSI design. He is author of a book on algorithms for mass storage system design, and coauthor of a book on VLSI physical design algorithms. He was a founding member of the Editorial Board of “Fuzzy Sets and Systems”. He is the founding Editor-in-Chief of the international journal “Algorithmica”, and a founding member of the Editorial Board of “IEEE Transactions on VLSI Systems”.

    View full text
    Copyright © 2003 Elsevier Ltd. All rights reserved.