Keywords

1 Introduction

Providing sub-optimal solution to the variety of NP-hard optimization problems is one of the most popular research problems, which is still receiving enormous attention. The development of major metaheuristic methods, like simulated annealing, evolutionary algorithms or taboo search, with an uncountable set of variants and applications to particular problems, required tremendous effort of researchers worldwide. It is to note, that multi-criteria problems are especially interesting and difficult, because not only one solution is sought, but a number of it (called Pareto set). Other features should be also considered like even and dense coverage of the Pareto front by the solutions.

According to Wolpert and Macready “No free lunch theorems for optimization” [29] each metaheuristic algorithm must be tailored for a given problem. However not all of them were properly theoretically analyzed, as e.g. Simple Genetic Algorithm (works by Vose [28]). Therefore not only tailoring is needed, but the NFL theorem encourages the researchers to seek new metaheuristics, which can be better suited for particular problems. One should however remember that new algorithms should not be proposed for their own sake (cf. Sorensen [26]), and introducing new inspirations must be carefully justified.

One of the popular, bio-inspired method, dedicated for solving discrete optimization problems, is Ant Colony Optimization, ACO [15]. It is dedicated for solving single-criteria discrete problems, and it has proven efficiency in one of the most recognized problem of this class, the travelling salesperson problem, TSP. The natural consequence of its growing popularity was the appearance of variants dedicated for multi-criteria optimization.

Another promising inspiration from nature, which has already proven usefulness in several problems, is the socio-cognitive ACO [11]. It introduces several types of ants, which present different behaviour – use different decision function. The types of ants and their behaviours reflect the types of characters observed in humans and classified by cognitive psychologists. Previously observed features of the socio-cognitive ACO makes it a good candidate for solving complex problems of multi-criteria optimization.

In this paper a novel, socio-cognitive ACO algorithm for Multi-criteria Optimization is proposed. A set of experiments using multi-criteria TSP benchmark problems shows that the proposed method outperforms previously proposed methods in terms of Pareto set cardinality and solutions quality.

2 Swarm Intelligence in Multi-criteria Optimization

In this section, starting from giving basic information on Ant Colony Optimization, selected aspects of ACO in solving multi-criteria are presented and background in socio-cognitive metaheuristics is given, before describing the proposed algorithm in detail.

2.1 Ant Colony Optimization

Ant System, introduced in 1991 by Marco Dorigo in application to graph problems, is a progenitor of all the Ant Colony Optimization (ACO) techniques [15]. The classic ACO algorithm is an iterative process during which a certain number of agents (ants) gradually create solutions [17, 18]. The main goal of the ant is to traverse the graph is search of a path with the lowest cost (usually meaning the shortest distance, but can also be the lowest fuel consumption, etc.).

In each step any particular ant selects a subsequent component of the solution (that is, a graph edge) with certain probability. This decision may be impacted by the levels of pheromones, which may have been deposited into the environment (onto the edges of the graph) by other ants. This interaction is guided by stigmergic relations (communication among individuals by means of an environment, instead of via direct contact) according to the rules proposed in [16]. The computation is finished once a feasible solution is found thanks to the cooperative efforts of all the ants.

Let us assume that one wants to solve a combinatiorial optimization problem, such as the Traveling Salesman Problem. In the basic ACO algorithm - the Ant System (AS, [15]) - the probability of moving from component i to component j for ant k is defined as follows:

$$\begin{aligned} p_{ij}^k = {\left\{ \begin{array}{ll} \frac{ [\tau _{ij}]^\alpha \cdot [\eta _{ij}]^\beta }{ \sum _{k\in {allowed}_k} [\tau _{ik}]^\alpha \cdot [\eta _{ik}]^\beta } &{} \quad \text {if } j \in {allowed}_k\\ 0 &{} \quad \text {otherwise} \end{array}\right. } \end{aligned}$$
(1)

where \(\tau _{ij}\) is the intensity of the pheromone trail on edge (i, j) and \(\eta _{ij}(t)\) is the visibility of edge (i, j), which in case of TSP can be defined as an inverted distance between the cities. \(\alpha \) and \(\beta \) are parameters that control the relative importance of the trail versus visibility. Finally, \({allowed}_k\) is a set of possible transitions for ant k. The tour ends when a feasible solution is found. After each iteration the ants update the pheromone trails on their paths based on its constructed solution. Furthermore, the pheromone also slowly evaporates to prevent premature convergence (cf. [16]).

In the classic AS version the update is performed at the end of a single iteration following the formula:

$$\begin{aligned} \tau _{ij}' = \rho \tau _{ij} + \sum _{k=1}^{m} \varDelta \tau _{ij}^{k} \end{aligned}$$
(2)

where \(\rho \in [0,1)\) is a pheromone persistence coefficient and m is the number of the ants. The pheromone update value for each ant is defined as follows:

$$\begin{aligned} \varDelta \tau _{ij}^{k} = {\left\{ \begin{array}{ll} \frac{Q}{L_k} &{} \quad \text {if } k \text {-th ant uses edge } (i, j)\\ 0 &{} \quad \text {otherwise} \end{array}\right. } \end{aligned}$$
(3)

where \(L_k\) is the tour length of the k-th ant and Q is a constant, often with value 1.

Since the first publication describing Ant System [17] researchers have put a lot of effort into improving the optimization performance. One of the most efficient modifications is Max-Min Ant System (MMAS, [27]). The algorithm introduced three major adjustments into the AS algorithm:

  • only one ant leaves the pheromone trails after each iteration - either the best from the iteration or the best found so far by the algorithm,

  • the pheromone trials values are limited by \(\tau _{min}\) and \(\tau _{max}\),

  • the pheromone matrix is initialized with the \(\tau _{max}\) value.

The effectiveness of the MMAS algorithm was further improved by the pheromone trail smoothing mechanism. When the algorithm is very close to convergence, the pheromone matrix gets smoothed by increasing the values proportionally to their difference to \(\tau _{max}\). This mechanism can also be effectively applied to other elitist ant systems [8, 16].

2.2 ACO in Multi-criteria Optimization

ACO algorithms are designed to solve single-criterion problems in their basic versions. However, over the last years there have been more and more attempts to adapt them to solve more complex problems. Designing such algorithms requires solving several difficulties. First of all, it should be specified how the deposition of the pheromone will be managed, in particular when several ant colonies are used at the same time. It should be determined how the ants will use this information in their decisions. Then you have to decide which ants and how they will modify this value.

There are three classic methods for solving multi-criteria problems using formic algorithms: using the weight function, the distance function or the min-max formula. These approaches are based on reducing the multi-criteria to a single-criterion problem by generating an artificial objective function combining individual criteria from the input problem.

The use of multi-criteria mapping to a single-criterion problem has several significant disadvantages and undesirable consequences. Therefore, several more advanced multi-criteria formulas have been developed that do not require combining different properties into a single one. In [22] quality tests for optimisation of various multicriteria implementations of ACO, we can find information on the basic method of categorizing these algorithms (Table 1).

Table 1. A taxonomy of multiple objective ACO algorithms proposed in [22]
Full size table

The Multiple Objective ACO metaheuristic (MOACOM) [23] is an extension of the Ant System algorithm [15], which represents multiple criteria in a single distance matrix. Different criteria are aggregated into a single distance value using dedicated rules. The method introduces an order of the criteria – only the most important criteria is used for updating the pheromone matrix. The BicriterionAnt algorithm introduced two pheromone matrices, updated independently. Multiple pheromone matrices allow representing different criteria in ant’s decisions. Te approach introduced in P-ACO algorithm [14] integrates values from several pheromone matrices, using different weights generated randomly for each ant.

The Multiple ant colony system (MACS) [1], a multiobjective version of the Ant System, uses three equally important objective functions. All three objectives share the same pheromone trails, which can be updated iteratively. The authors introduce the concept of different visibilities of each objective, which differentiate the behaviour of ants. The MACS algorithm will be used as a basis for the socio-cognitive solution presented in this paper.

2.3 Socio-cognitive Metaheuristics

In cognitive psychology, the character traits of egocentrism (taking one’s own perspective) and altercentrism (taking another person’s perspective into consideration) have long been recognized to play a key role in interpersonal relationships (see, for instance, [19, 25]). Moreover, brain-imaging studies have shown that altercentricity and the strategy of perspective taking develop in parallel with brain maturation and psychosocial development during adolescence [5, 13]. Perhaps mirroring this psychological development, in recent years, artificial intelligence researchers have started to incorporate altercentricity into robots and autonomous systems [24]. We also continue with utilizing the notions of ego- and altercentrism, adapting them appropriately to use in our computing system.

Typically, perspective taking is seen as a one-dimensional ability: the degree to which an agent can take another one’s perspective. But recent research has explored a two-dimensional approach [6], where one distinguishes between the ability of an agent to handle conflict between its own and the other agent’s perspectives, and the relative priority that an agent gives to his own perspective relative to the other’s perspective. During social interactions, humans do not always share the same views. Being able to consider the other person’s point of view therefore requires putting aside one’s own perspective. This is particularly hard if one holds a strong view. Individuals endowed with good cognitive skills to manage conflicting information are therefore usually better perspective-takers [20]. In addition, however, humans also differ in terms of how much they are interested in or are willing to pay attention to others compared to themselves. Sometimes individuals focus only on their own perspective (egocentrism) while on other occasions individuals focus more on other people’s perspective (altercentrism) [19, 24].

The less a person focuses on her own perspective, the more that person will be motivated to engage in perspective taking [6]. Experimental research has suggested that these two dimensions (conflict handling and perspective priority) might be independent; and factors such as guilt or shame affect each of these dimensions individually [7]. This two-dimensional approach to perspective taking inspired us to define four types of individuals:

  • Egocentric individuals, focusing on their own perspective and becoming creative thanks to finding their own new solutions to a given task. These individuals do not pay attention to the other ones and do not get inspired by the actions of other ones (or these inspirations do not become a main factor of their work).

  • Altercentric individuals, focusing on the perspective of others and thus following the mass of others. Such individuals become less creative but they still can end up supporting good solutions by simply following them.

  • Good-at-conflict-handling individuals, getting inspired in a complex way by the actions of other individuals, considering different perspectives and choosing the one considered as the best for them.

  • Bad-at-conflict-handling individuals, acting purely randomly, following sometimes one perspective, sometimes another without any inner logic.

In this work we follow only the first three (although involving the fourth one is also feasible and will be tackled in the future work).

Based on those inspirations, novel socio-cognitive metaheuristics have been proposed, enhancing well-known swarm metaheuristics, like Ant Colony Optimization and Particle Swarm Optimization. The enhancement of these consisted in introducing different “species” of swarm individuals, inspired by egocentricity and altercentricity and making them perceive the others.

In the case of socio-cognitive ACO [11], different species of ants leaved different types of pheromones and they were able to perceive them and compute the probabilities of choosing the next edge, following the socio-cognitive inspirations. The egocentric ants focused on their own knowledge (perceived only the distance connected with the current edges to be chosen), the altercentric ants focused only on pheromone markings. Good-at-conflict-handling ants were inspired in a more complex way (they computed the attractiveness based on a weighted sum of information from distance and pheromone expressed by other species of ants). The structure of the ant population was static in the beginning (the percentage of different species). However later, going beyond these inspirations, the attractiveness was parameterized, and those parameters were explored along with considering automatic adaptation of the population structure [10].

In the case of socio-cognitive PSO [4], different species of particles were introduced into the basic PSO algorithm, making them perceive others, by modifying their cognitive abilities: the particles could perceive the best global solution gathered in the whole swarm, in their neighborhood and their own, historical best solution. In the case of socio-cognitive PSO, an effort has been made also to introduce auto-adaptation mechanism modifying the structure of population [3].

Both socio-cognitive ACO and PSO turned out to be better than classic reference algorithms when considering selected multi-dimensional benchmark functions and TSPLIB instances [9].

3 Multi-criteria Socio-cognitive ACO

In this section the reference algorithm, Multiple-Ant Colony System [2] is presented, being a base for extension towards introducing socio-cognitive mechanisms (i.e. different species of ants, similarly to [11].

3.1 Multiple-Ant Colony System

Multiple-Ant Colony System (MACS) was proposed by Barán and Schaerer in 2003 [2]. According to García-Martínez [22] this algorithm produces a very good coverage of the Pareto front. Because MACS uses one pheromone table, it is a very good candidate for extension using socio-cognitive inspirations.

The algorithm is a developed version of the MACS-VRPTW algorithm. The main difference is that it uses a single pheromone table \(\tau \) and two heuristic functions which are used to make decisions about choosing the next edge. In the classic implementation of MACS-VRPTW, the algorithm uses two colonies, each with an independent array of pheromones. One minimizes the number of vehicles and the second distance travelled. Information between colonies is exchanged by maintaining the best solution for both colonies. The authors showed that the proposed methodology improves the quality of results for the problem under investigation [21].

Fist the value \(\tau _0\), being a current starting pheromone value is computed as follows:

$$\begin{aligned} \tau _0=\frac{1}{\hat{f}^1 \cdot \hat{f}^2} \end{aligned}$$
(4)

where \(\hat{f}^1, \hat{f}^2\) are values of the criteria function computed for the current solution.

The next vertex is chosen by the ant h belonging to the set of m ants according to the following equation:

$$\begin{aligned} j = {\left\{ \begin{array}{ll} arg max_{j\in \varOmega } ( \tau _{ij} \cdot [\eta _{ij}^0]^{\beta \lambda _h}\cdot [\eta _{ij}^1]^{\beta (1-\lambda _h)}) \quad \text {if } q \le q_0 \\ \hat{i} \end{array}\right. } \end{aligned}$$
(5)

where \(\varOmega \) is the set of the vertices adjacent to the current vertext the ant is located on; \(\eta _{ij}^0, \eta _{ij}^1\) are visibility or distance perceived from the point of view of two criterion functions (0 and 1); \(\lambda _h=\frac{h-1}{m-1}\), this value is used for enforcing the ant to explore new areas of the Pareto frontier; q is a certain probability, while \(q_0=0.98\) for this algorithm. In the above equation, \(\hat{i}\) is a randomly-chosen vertex with the following probability:

$$\begin{aligned} p_{\hat{i}} = {\left\{ \begin{array}{ll} \frac{ [\sum _{k=1}^K p_k \cdot \tau _{ij}^k]^\alpha \cdot [\eta _{ij}]^\beta }{ \sum _{u\in \varOmega } [\sum _{k=1}^K p_k \cdot \tau _{iu}]^\alpha \cdot [\eta _{iu}]^\beta } &{} \quad \text {if } j \in {allowed}_k\\ 0 &{} \quad \text {otherwise} \end{array}\right. } \end{aligned}$$
(6)

The pheromone at a certain edge is updated according to the following evaporation rule:

$$\begin{aligned} \tau _{ij} = (1-\varrho ) \cdot \tau _{ij} + \varrho \tau _0 \end{aligned}$$
(7)

where \(\varrho \) is a parameter of the algorithm.

The Pareto set is used to modify the value of \(\tau _0\) during the life of the colony. The ant after finding a full, non-dominated solution adds them to the common set. Then after finding all solutions by ants in a given iteration for the first and second cost functions, their average values are calculated, which we substitute for the formula from the previous point. In this way, we get the value of \(\tau _0'\). Then, if \(\tau _0'>\tau _0\), all the pheromone values on the edges are converted into the value \(\tau _0'\). Otherwise, the total pheromone table value update operation is performed by performing the following formula for each edge and each solution from the Pareto set S (actually for each \(s \in S\)):

$$\begin{aligned} \tau _{ij}' = (1 - \varrho ) \tau _{ij} + \frac{\varrho }{f^0(s) \cdot f^1(s)} \end{aligned}$$
(8)

where s is a solution belonging to the Pareto set S. Thus the criteria function are used to constantly modify the pheromone table.

3.2 Multi-criteria Socio-cognitive Ant Colony System

The proposed multicriteria algorithm merges the sociological inspirations together with the concepts introduced in MACS algorithm. The preliminary tests showed, that three types of ants should be used: the Good-at-conflict-handling, the Egocentric and Altercentric. The algorithm uses three independent pheromone matrices, one for each type of ant. The matrices are updated after each iteration, and the information contained in them is used depending on the ant’s type. The following rules are used by the different types of ants:

  • Good-at-conflict-handling ant considers pheromone in all three matrices. Each ant undertakes the decision about choosing the next vertex according to Eqs. 5 and 6. However the pheromone is now located in three tables, and the actual decision depends on the ant type (or species). Thus, assume that the \(\tau _{ij}\) used in the above-cited equations is substituted by the \(\hat{\tau }_{ij}\) computed as follows:

    $$\begin{aligned} \hat{\tau }_{ij} = \kappa \cdot \tau _{ij}(GC) + \psi \cdot \tau _{ij}(EC) + \tau _{ij}(AC) \end{aligned}$$
    (9)

    where \(\kappa =1, \psi =5, \omega =7\) (these values were discovered experimentally) and may be further adapted; \(\tau _{ij}(GC)\) is the value of pheromone for the vertex (i, j) deposited in the pheromone table of good-at-conflict-handling ants GC, and respectively for EC and AC.

  • Altercentric ant considers only the pheromone values in all three matrices. Thus the next vertex is chosen according to the following equation:

    $$\begin{aligned} j = {\left\{ \begin{array}{ll} arg max_{j\in \varOmega } (\hat{\tau }_{ij}^\alpha ) \cdot \quad \text {if } q \le q_0 \\ \hat{i} \end{array}\right. } \end{aligned}$$
    (10)

    where the vertex \(\hat{i}\) is chosen with the probability given by Eq. 6, with \(\tau _{ij}\) substituted by \(\hat{\tau }_{ij}\):

    $$\begin{aligned} \hat{\tau }_{ij} = \tau _{ij}(GC) + \tau _{ij}(EC) + \tau _{ij}(AC) \end{aligned}$$
    (11)

  • Egocentric ant is only driven by the distance to the next vertex, ignoring the pheromone matrices, thus the next vertex is chosen as follows:

    $$\begin{aligned} j = {\left\{ \begin{array}{ll} arg max_{j\in \varOmega } ([\eta _{ij}^0]^{\beta \lambda _h}\cdot [\eta _{ij}^1]^{\beta (1-\lambda _h)}) \quad \text {if } q \le q_0 \\ \hat{i} \end{array}\right. } \end{aligned}$$
    (12)

    and \(\hat{i}\) is chosen randmoly with the following probability:

    $$\begin{aligned} p_{\hat{i}} = {\left\{ \begin{array}{ll} \frac{ [\sum _{k=1}^K p_k]^\alpha \cdot [\eta _{ij}]^\beta }{ \sum _{u\in \varOmega } [\sum _{k=1}^K p_k \cdot ]^\alpha \cdot [\eta _{iu}]^\beta } &{} \quad \text {if } j \in {allowed}_k\\ 0 &{} \quad \text {otherwise} \end{array}\right. } \end{aligned}$$
    (13)

The update of the three pheromone matrices is done after the whole iteration is finished. Only the solutions from the identified Pareto set are used. Each solution \(s \in S\) (belonging to the Pareto set) updates the matrix of the ant type, which found the solution, according to the following formula (for all \(s \in S\)):

$$\begin{aligned} \tau _{i,j}^{'}(s) = \tau _{i,j}(s) + \frac{1}{0.5f^0(s) + 0.5f^1(s)} \end{aligned}$$
(14)

this modification is realized for all the ant types (thus all the pheromone tables) separately.

Introducing of properly adapted ant species, namely egocentric, altercentric and good-at-conflict-handling follows the socio-cognitive computing paradigm proposed in [11].

4 Experimental Results

In order to verify the effectiveness of the proposed, socio-cognitive approach to the problem of multi-criteria optimization, the two-criteria travelling salesperson problem has been selected. The experiments were conducted using popular benchmark problems from the TSPLIB libraryFootnote 1. Five variants of the problem have been selected: Kro100AB and Kro100ED contain graphs composed of 100, Kro200AB with 200 cities, and two problems, euclidA100 and euclidA300, generated with the program of the DIMACS competitionFootnote 2, with 100 and 300 cities respectively. All tests have been executed on a single computer with Intel i7, 2.5 GHz CPU and 12 GB of RAM. Each of the experimental configurations was run 30 times and the results were gathered, consituting one Pareto front (either shown in the relevant figure or processed and described in a table in this section).

Table 2. Results of search for optimal population structure using the problem Kroo100AB.
Full size table

Two metrics were used to evaluate the quality of the solutions: the HyperVolume and the Cardinality of the computed Pareto front. The HyperVolume [12] value is a well-recognised metric for multi-criteria optimisation, which integrates the quality and the diversity of the Pareto front. It is computed as a volume of a n-dimensional shape surrounded by the solutions of the Pareto front, where n is the number of criteria.

The implementation of the proposed method (MSCACO) has been compared with the MACS algorithm. During the tests a constant composition of the ants types in the population of the MSCACO algorithm has been used: 40% of Good-at-conflict-handling ants, 40% of Egocentric and 20% of Altercentric. These values have been concluded from a series of preliminary tests (cf. Table 2).

We have also conducted a series of experiments in order to find an appropriate number of ants and number of iterations to get reasonable HV indicator values, finding that 100 ants and 50 iterations can be used as a good starting point for further research (see Table 3).

The most important results from the conducted experiments are collected in Table 4. In all cases a population of 100 ants were used. The algorithms executed 50 iterations.

The results clearly show the superiority of the proposed, socio-cognitive ACO algorithm over the MACS method. In all test cases the cardinality of the created Pareto Set is far greater. Also the values of the HyperVolume metrics are significantly better in all test cases, which is a valuable result.

More detailed results of the selected test cases are presented in Figs. 1 and 2, which present results from the Kro100AB and EuclidAB100 test cases. In both cases it is clearly visible, that the Pareto Front created by the MSCACO algorithm is more consistent and the particular results are better for vast majority of values.

Table 3. Results of search for optimal number of ants and iterations.
Full size table
Table 4. The comparison of the MSCACO and MACS optimisation results.
Full size table
Fig. 1.
figure 1

The computed Pareto sets for the Kro100AB problem (MACS: red, MSCACO: blue) (Color figure online)

Full size image
Fig. 2.
figure 2

The computed Pareto sets for the EuclidAB100 problem (MACS: red, MSCACO: blue) (Color figure online)

Full size image

5 Conclusions and Future Work

In this paper a new metaheuristic algorithm for solving multi-criteria optimization, belonging to the class of socio-cognitive algorithms was presented. Following our previous work on incorporation of socio-cognitive inspirations, the population of ants has been divided into three species with different cognitive abilities. This new algorithm (MSCACO) has been built on the basis of the existing MACS. Introduction of socio-cognitive inspirations lead to increasing the efficiency of the algorithm applied to selected instances of multi-criteria TSP problems connected with TSPLIB library. It was shown, that socio-cognitive versions produced more solutions on the Pareto front, and finally the Hyper Volume metrics was also better.

In the future work we are planning to tackle new problems and introduce more flexibility in building the parameters of the algorithms, thus auto-adaptation in the structure of population is to be considered.