Elsevier

Computers & Operations Research

Volume 54, February 2015, Pages 108-116
Computers & Operations Research

Model-based automatic neighborhood design by unsupervised learning

https://doi.org/10.1016/j.cor.2014.09.006Get rights and content

Abstract

The definition of a suitable neighborhood structure on the solution space is a key step when designing a heuristic for Mixed Integer Programming (MIP). In this paper, we move on from a MIP compact formulation and show how to take advantage of its features to automatically design efficient neighborhoods, without any human analysis. In particular, we use unsupervised learning to automatically identify “good” regions of the search space “around” a given feasible solution. Computational results on compact formulations of three well-known combinatorial optimization problems show that, on large instances, the neighborhoods constructed by our procedure outperform state-of-the-art domain-independent neighborhoods.

Introduction

Given a Mixed Integer Programming (MIP) model and a feasible solution, a relevant issue is whether that solution can be improved using local search techniques [1]. A challenge for the research community is to develop mechanisms that derive automatically, without any human intervention, suitable neighborhoods on the basis of the MIP model׳s features. Along this line of research, Fischetti and Lodi [2] have proposed the Local Branching (LB) algorithm which uses a generic MIP solver as a black-box “tactical” tool to explore suitable neighborhoods. Such neighborhoods are defined at a “strategic” level by branching conditions expressed through invalid linear inequalities (local branching cuts). In particular, for purely binary MIPs, a neighborhood is defined as the set of all solutions whose Hamming distance from the current solution does not exceed a given threshold. The LB paradigm has subsequently been refined and generalized in a number of ways. In Hansen et al. [3], LB is embedded within a Variable Neighborhood Search (VNS) framework. Fischetti and Lodi [4] applied the LB framework to slightly infeasible initial solutions which are first repaired to become feasible and then possibly improved. Lazić et al. [5] proposed a hybrid heuristic for solving 0–1 MIPs based on the principle of Variable Neighborhood Decomposition Search. Kiziltan et al. [6] combined the LB general-purpose neighborhood structure with constraint propagation.

Another remarkable contribution in this field is due to Danna et al. [7] who proposed two heuristics that explore a neighborhood of the current incumbent solution in the attempt to improve it. The more effective of the two, called Relaxation-Induced Neighborhood Search (RINS), formulated the neighborhood exploration as a reduced MIP model. When applied at a node of a branch-and-cut tree, the RINS method (i) fixes the variables having the same values in the incumbent and in the continuous relaxation; (ii) sets a cutoff based on the objective value of the current incumbent; (iii) solves a sub-MIP on the remaining variables. The RINS parameters are the following: how often RINS is invoked (RINS frequency); the maximum number of nodes generated in the search tree of the sub-MIP (RINS sub-MIP node limit). Danna et al. [7] showed that the RINS algorithm outperforms both default CPLEX [8] and LB with respect to several metrics. More recently, Parisini and Milano [9] presented a search strategy called Sliced Neighborhood Search (SNS) which explores randomly selected slices of a neighborhood of an incumbent solution. In a subsequent paper, Parisini and Milano [10] combined SNS with LB to take advantage of their respective characteristics.

The previously mentioned algorithms are part of a wider family of heuristics known as Very Large Scale Neighborhood Search (VLSN) algorithms [11]. These algorithms derive from the observation that searching a large neighborhood may result in local optima of high quality. On the other hand, searching a large neighborhood may be time consuming, hence various filtering techniques are customarily used to limit the search. The choice of the neighborhood is typically done by an operations research analyst on the basis of the specific features of the problem to be solved.

In this paper, we consider MIP models stated in terms of sets of entities (e.g., customers, plants, commodities) as allowed by algebraically oriented languages like GAMS [12], AMPL [13] and OPL [14]. In such models, variables and constraints (as well as parameters) are tagged by one or more entities. Entities can be either fundamental or derived. Derived set of entities is defined as subsets or cartesian products of other sets. For instance, the following constraint(i,j)Axij=1,jVis tagged by (fundamental) entities in set V while variables x are tagged by (derived) entity set A=V×V. This kind of high-level knowledge representation is key to exploiting the model structure in order to automatically design efficient neighborhoods. To further clarify how the entities are related to the variables and constraints of a problem, in the following we first report the MIP formulation of one of the test problems we use in Section 4, namely the Traveling Salesman Problem (TSP), and then we show which is the corresponding formulation in terms of the algebraic language OPL. The TSP is the problem of determining a minimum cost Hamiltonian circuit through a set of vertices. It is formulated on a complete directed graph G=(N,A), where N={1,,n} is the vertex set, and A={(i,j):i,jNj} is the set of arcs between the vertices. The value cij is the cost of traveling from i to j. We consider the Miller–Tucker–Zemlin (MTZ) formulation [15]:min(i,j)Acijxijs.t.jNxij=1,iN,iNxij=1,jN,uiuj+1(n1)(1xij),i,jN,i,j1,u1=1,2uin,iN,i1,xij{0,1},(i,j)Awhere xij, (i,j)A, takes value 1 if arc (i,j) is in the tour, 0 otherwise, and variables ui, iN, are used to exclude subtours. Table 1 reports a possible OPL implementation of such a model. Here, vertices are fundamental entities, tagging variables x and u, as well as the various constraints.

Given a current feasible solution, our approach defines the neighborhood structure in two steps:

  • 1.

    Learning: It exploits the structure of both the model and the current solution in order to group the entities originating the MIP model into a number of homogeneous clusters;

  • 2.

    Neighborhood definition: The neighborhood is obtained by considering each cluster in turn and fixing all the variables unrelated to such cluster to their current values.

The rationale of this approach is that the “best” neighbors of the current solution can be obtained by destroying a portion of the solution related to “close” entities (i.e., entities which are similar to each other with respect to either the model or the current solution itself). More specifically, the first phase is performed by using unsupervised learning with the aim to obtain homogeneous clusters of approximately the same size. In our experimentations, the neighborhood is explored by using a generic MIP solver as a black-box tool and the number of clusters is chosen in such a way that a state-of-the-art MIP solver is able to explore adequately the cluster subproblems.

The remainder of the paper is organized as follows. In Section 2, we outline the proposed automatic neighborhood design procedure. In Section 3, we show how to partition the fundamental entities on the basis of both the current solution and the model. Then, in Section 4 we present computational results on three well-known combinatorial optimization problems highlighting the improvements provided by our automatically designed neighborhoods with respect to state-of-the-art domain-independent neighborhoods. Conclusions follow in Section 5.

Section snippets

Automatic neighborhood design

In this paper, we consider a generic MIP problem of the form(P)minz=j=1ncjxjs.t.j=1naijxj=bi,i=1,,mxj{0,1},jBxj0andinteger,jGxj0,jC.Here, the variable index set J={1,,n} is partitioned into (B,G,C), where B, G and C are the index sets of the 0–1, general integer and continuous variables, respectively. Let x(h) be the current solution at iteration h of a generic neighborhood search procedure, and N(x(h)) its neighborhood.

As mentioned in Section 1, we assume that program (9), (10),

Learning phase

We now describe how the algorithm learns about the model and the current solution structure in order to partition each set Er (r=1,,p) into a number of clusters Ers (s=1,,kr) of “similar” entities. Two entities e,eEr(r=1,,p) are assigned or not to the same cluster depending on a dissimilarity measure which is computed as follows.

Computational results

We have tested our automatic neighborhood design procedure on three well-known combinatorial optimization problems for which formulations with the following properties are available: (i) the models can be expressed in terms of sets of entities (as explained in Section 2); (ii) the number of variables and constraints is polynomially bounded by the number of entities. Of course, not every MIP formulation satisfies these properties. For instance, the widely used MIPLIB instances [17] do not

Conclusions

We have introduced a procedure that automatically designs efficient neighborhoods by taking advantage of the problem structure without any human analysis. In particular, we have used unsupervised learning to automatically identify the most promising regions of the search space around a given feasible solution. Computational experiments were carried out by using a generic MIP solver as a black-box tool. Results on compact formulations of three well-known combinatorial optimization problems have

Acknowledgments

This work was partly supported by the Canadian National Sciences and Engineering Research Council under Grant 39682-10, and by the Ministero dell׳Istru-zione, dell׳Università e della Ricerca (MIUR) of Italy (PON project PON01_00878 “DIRECT FOOD”). This support is gratefully acknowledged. The authors also thank Dr. Andrea Manni for his help with programming.

References (24)

  • M. Fischetti et al.

    Local branching

    Math Prog Ser B

    (2003)
  • Z. Kiziltan et al.

    Bounding, filtering and diversification in CP-based local branching

    J Heuristics

    (2012)
  • Cited by (9)

    • MIP neighborhood synthesis through semantic feature extraction and automatic algorithm configuration

      2017, Computers and Operations Research
      Citation Excerpt :

      Subsequently, the same authors combined Sliced Neighborhood Search with Local Branching to exploit their respective characteristics (Parisini and Milano, 2011). More recently, Ghiani et al. (2015) moved on from a MIP compact formulation and showed how to take advantage of its features to automatically design efficient neighborhoods, without any human analysis. In particular, they used unsupervised learning to automatically identify “good” regions of the search space “around” a given feasible solution.

    • Automatic instantiation of a Variable Neighborhood Descent from a Mixed Integer Programming model

      2017, Operations Research Perspectives
      Citation Excerpt :

      Particularly relevant to our work are the contributions by Ghiani et al. [8] and Adamo et al. [3], that showed how to take advantage of a MIP compact formulation to automatically design efficient neighborhood structures, without any human analysis. In particular, Ghiani et al. [8] used unsupervised learning to automatically select “good” portions of the search space “around” a given feasible solution. Adamo et al. [3] proposed a procedure extracting some semantic features from a given MIP model.

    • Learning Large Neighborhood Search for Vehicle Routing in Airport Ground Handling

      2023, IEEE Transactions on Knowledge and Data Engineering
    View all citing articles on Scopus
    View full text