Skip to main content

Probabilistic Multimodal Optimization

  • Chapter
  • First Online:
Metaheuristics for Finding Multiple Solutions

Part of the book series: Natural Computing Series ((NCS))

Abstract

Multimodal optimization, which aims to discover multiple satisfactory solutions simultaneously, has attracted increasing attention from researchers in the evolutionary computation community. With the aid of niching methods, evolutionary algorithms could simultaneously locate multiple satisfactory solutions in a single run. Although many multimodal evolutionary algorithms have been developed, they are confronted with two limitations: (1) in the niching stage, most niching-based multimodal methods need to compute pairwise Euclidean distances between individuals to separate the population into species, which gives rise to a high computational burden; and (2) in the optimization stage, most existing multimodal algorithms may have limitations in exploring the solution space, due to the utilization of traditional individual-based meta-heuristics, which may easily get trapped in local areas. To resolve the above issues, in this chapter, we introduce probabilistic multimodal optimization algorithms by presenting two probability-based frameworks for multimodal optimization. More specifically, we present a probability-based niching framework to accelerate the niching speed and a probability-based optimization framework to promote the optimization efficiency of multimodal algorithms, respectively. In the former, we utilize locality sensitive hashing to project individuals into buckets with probabilities to divide the population into species. Such a framework can be embedded into different niching methods to accelerate the niching speed. In the latter, we take advantage of the probability distribution of individuals to evolve the population along with a novel adaptive local search method. To instantiate these two frameworks, we customize them using two distinct approaches, respectively. More concretely, we embed the former into locally informed particle swarm optimization (LIPS) and neighborhood-based crowding differential evolution (NCDE), and customize the latter utilizing an explicit probability-based algorithm, the estimation of distribution algorithm (EDA), and an implicit probability-based algorithm, the continuous ant colony optimization algorithm (ACO), respectively. The efficiency and effectiveness of these two frameworks are carefully examined on a widely used multimodal benchmark set by means of comparing the associated customized algorithms with state-of-the-art multimodal methods. Lastly, the application of the proposed algorithms on multiple pedestrian detection problems is also presented. Experimentally, our approach is seen to perform competitively with traditional methods in this domain.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ali, M.Z., Awad, N.H., Suganthan, P.N., Reynolds, R.G.: An adaptive multipopulation differential evolution with dynamic population reduction. IEEE Trans. Cybern. 47(9), 2768–2779 (2017)

    Article  Google Scholar 

  2. Biswas, S., Kundu, S., Das, S.: An improved parent-centric mutation with normalized neighborhoods for inducing niching behavior in differential evolution. IEEE Trans. Cybern. 44(10), 1726–1737 (2014)

    Article  Google Scholar 

  3. Biswas, S., Kundu, S., Das, S.: Inducing niching behavior in differential evolution through local information sharing. IEEE Trans. Evol. Comput. 19(2), 246–263 (2015)

    Article  Google Scholar 

  4. Bosman, P.A., Thierens, D.: Expanding from discrete to continuous estimation of distribution algorithms: the idea. In: International Conference on Parallel Problem Solving from Nature. Springer, pp. 767–776 (2000)

    Google Scholar 

  5. Chen, W.N., Zhang, J.: An ant colony optimization approach to a grid workflow scheduling problem with various qos requirements. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 39(1), 29–43 (2009)

    Article  Google Scholar 

  6. Chen, W.N., Zhang, J.: Ant colony optimization for software project scheduling and staffing with an event-based scheduler. IEEE Trans. Softw. Eng. 39(1), 1–17 (2013)

    Article  Google Scholar 

  7. Chen, W.N., Zhang, J., Chung, H.S.H., Huang, R.Z., Liu, O.: Optimizing discounted cash flows in project scheduling–an ant colony optimization approach. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 40(1), 64–77 (2010)

    Article  Google Scholar 

  8. Chen, W.N., Zhang, J., Lin, Y., Chen, N., Zhan, Z.H., Chung, H.S.H., Li, Y., Shi, Y.H.: Particle swarm optimization with an aging leader and challengers. IEEE Trans. Evol. Comput. 17(2), 241–258 (2013)

    Article  Google Scholar 

  9. Chen, Z.G., Zhan, Z.H., Wang, H., Zhang, J.: Distributed individuals for multiple peaks: a novel differential evolution for multimodal optimization problems. IEEE Trans. Evolut. Comput. 24(4), 708–719 (2020)

    Google Scholar 

  10. Dalal, N., Triggs, B.: In: Histograms of oriented gradients for human detection, vol. 1, pp. 886–893. IEEE (2005)

    Google Scholar 

  11. Das, S., Konar, A., Chakraborty, U.K.: Two improved differential evolution schemes for faster global search. In: The Annual Conference on Genetic and Evolutionary Computation. ACM, pp. 991–998 (2005)

    Google Scholar 

  12. Das, S., Suganthan, P.N.: Differential evolution: a survey of the state-of-the-art. IEEE Trans. Evol. Comput. 15(1), 4–31 (2011)

    Article  Google Scholar 

  13. Datar, M., Immorlica, N., Indyk, P., Mirrokni, V.S.: Locality-sensitive hashing scheme based on p-stable distributions. In: The Annual Symposium on Computational Geometry. ACM, pp. 253–262 (2004)

    Google Scholar 

  14. De Jong, K.A.: Analysis of the Behavior of a Class of Genetic Adaptive Systems. University of Michigan (1975)

    Google Scholar 

  15. Dilettoso, E., Salerno, N.: A self-adaptive niching genetic algorithm for multimodal optimization of electromagnetic devices. IEEE Trans. Mag. 42(4), 1203–1206 (2006)

    Article  Google Scholar 

  16. Dong, W., Chen, T., Tino, P., Yao, X.: Scaling up estimation of distribution algorithms for continuous optimization. IEEE Trans. Evol. Comput. 17(6), 797–822 (2013)

    Article  Google Scholar 

  17. Dorigo, M., Stützle, T.: Ant colony optimization: overview and recent advances. In: Handbook of Metaheuristics, pp. 227–263. Springer (2010)

    Google Scholar 

  18. François, O.: An evolutionary strategy for global minimization and its markov chain analysis. IEEE Trans. Evol. Comput. 2(3), 77–90 (1998)

    Article  Google Scholar 

  19. Gao, W., Yen, G.G., Liu, S.: A cluster-based differential evolution with self-adaptive strategy for multimodal optimization. IEEE Trans. Cybern. 44(8), 1314–1327 (2014)

    Article  Google Scholar 

  20. Ge, Y.F., Yu, W.J., Lin, Y., Gong, Y.J., Zhan, Z.H., Chen, W.N., Zhang, J.: Distributed differential evolution based on adaptive mergence and split for large-scale optimization. IEEE Trans. Cybern. 48(7), 2166–2180 (2018)

    Article  Google Scholar 

  21. Geronimo, D., Lopez, A.M., Sappa, A.D., Graf, T.: Survey of pedestrian detection for advanced driver assistance systems. IEEE Trans. Pattern Anal. Mach. Intell. 32(7), 1239–1258 (2010)

    Article  Google Scholar 

  22. Gionis, A., Indyk, P., Motwani, R., et al.: Similarity search in high dimensions via hashing. VLDB 99, 518–529 (1999)

    Google Scholar 

  23. Goldberg, D.E., Richardson, J., et al.: Genetic algorithms with sharing for multimodal function optimization. In: International Conference on Genetic Algorithms and Their Applications, pp. 41–49. Lawrence Erlbaum, Hillsdale, NJ (1987)

    Google Scholar 

  24. Gong, Y.J., Zhang, J., Zhou, Y.: Learning multimodal parameters: a bare-bones niching differential evolution approach. IEEE Trans. Neural Netw. Learn. Syst. 29(7), 2944–2959 (2018)

    Google Scholar 

  25. Hauschild, M., Pelikan, M.: An introduction and survey of estimation of distribution algorithms. Swarm Evol. Comput. 1(3), 111–128 (2011)

    Article  Google Scholar 

  26. Hu, X.M., Zhang, J., Chung, H.S.H., Li, Y., Liu, O.: Samaco: variable sampling ant colony optimization algorithm for continuous optimization. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 40(6), 1555–1566 (2010)

    Article  Google Scholar 

  27. Huang, T., Gong, Y.J., Kwong, S., Wang, H., Zhang, J.: A niching memetic algorithm for multi-solution traveling salesman problem. IEEE Trans. Evolut. Comput.24(3), 508–522 (2020)

    Google Scholar 

  28. Huang, T., Gong, Y.J., Zhang, Y.H., Zhan, Z.H., Zhang, J.: Automatic planning of multiple itineraries: A niching genetic evolution approach. IEEE Trans. Intell. Transp. Syst. 21(10), 4225–4240 (2020)

    Google Scholar 

  29. Jia, Y.H., Chen, W.N., Gu, T.L., Zhang, H.X., Yuan, H.Q., Kwong, S., Zhang, J.: Distributed cooperative co-evolution with adaptive computing resource allocation for large scale optimization. IEEE Trans. Evol. Comput. 23(2), 188–202 (2019)

    Article  Google Scholar 

  30. Li, C., Yang, S.: A general framework of multipopulation methods with clustering in undetectable dynamic environments. IEEE Trans. Evol. Comput. 16(4), 556–577 (2012)

    Article  Google Scholar 

  31. Li, J.P., Balazs, M.E., Parks, G.T., Clarkson, P.J.: A species conserving genetic algorithm for multimodal function optimization. Evol. Comput. 10(3), 207–234 (2002)

    Article  Google Scholar 

  32. Li, X.: Efficient differential evolution using speciation for multimodal function optimization. In: The Annual Conference on Genetic and Evolutionary Computation. ACM, pp. 873–880 (2005)

    Google Scholar 

  33. Li, X.: A multimodal particle swarm optimizer based on fitness euclidean-distance ratio. In: the Annual Conference on Genetic and Evolutionary Computation. ACM, pp. 78–85 (2007)

    Google Scholar 

  34. Li, X.: Niching without niching parameters: particle swarm optimization using a ring topology. IEEE Trans. Evol. Comput. 14(1), 150–169 (2010)

    Article  Google Scholar 

  35. Li, X., Engelbrecht, A., Epitropakis, M.G.: Benchmark functions for cec’2013 special session and competition on niching methods for multimodal function optimization. RMIT University, Evolutionary Computation and Machine Learning Group, Australia, Technical Report (2013)

    Google Scholar 

  36. Li, X., Epitropakis, M.G., Deb, K., Engelbrecht, A.: Seeking multiple solutions: an updated survey on niching methods and their applications. IEEE Trans. Evol. Comput. 21(4), 518–538 (2017)

    Article  Google Scholar 

  37. Li, Y.K., Chen, Y.L., Zhong, J.H., Huang, Z.X.: Niching particle swarm optimization with equilibrium factor for multi-modal optimization. Inf. Sci. 494, 233–246 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  38. Li, Y.L., Zhan, Z.H., Gong, Y.J., Chen, W.N., Zhang, J., Li, Y.: Differential evolution with an evolution path: A deep evolutionary algorithm. IEEE Trans. Cybern. 45(9), 1798–1810 (2015)

    Article  Google Scholar 

  39. Liang, J.J., Suganthan, P.N.: Dynamic multi-swarm particle swarm optimizer with local search. In: IEEE Congress on Evolutionary Computation, vol. 1, pp. 522–528. IEEE (2005)

    Google Scholar 

  40. Liao, T., Socha, K., de Oca, M.A.M., Stützle, T., Dorigo, M.: Ant colony optimization for mixed-variable optimization problems. IEEE Trans. Evol. Comput. 18(4), 503–518 (2014)

    Article  Google Scholar 

  41. Liao, T., Stützle, T., de Oca, M.A.M., Dorigo, M.: A unified ant colony optimization algorithm for continuous optimization. Eur. J. Oper. Res. 234(3), 597–609 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  42. Lin, Y., Zhang, J., Chung, H.S.H., Ip, W.H., Li, Y., Shi, Y.H.: An ant colony optimization approach for maximizing the lifetime of heterogeneous wireless sensor networks. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 42(3), 408–420 (2012)

    Article  Google Scholar 

  43. Lin, Z.J., Chen, W.N., Zhang, J., Li, J.J.: In: Fast multiple human detection with neighborhood-based speciation differential evolution. In: International Conference on Information Science and Technology, pp. 200–207. IEEE (2017)

    Google Scholar 

  44. Ling, H.L., Wu, J.S., Zhou, Y., Zheng, W.S.: How many clusters? a robust pso-based local density model. Neurocomputing 207, 264–275 (2016)

    Article  Google Scholar 

  45. Mahfoud, S.W.: Niching methods for genetic algorithms. Urbana 51(95001), 62–94 (1995)

    Google Scholar 

  46. Maji, S., Berg, A.C., Malik, J.: In: Classification using intersection kernel support vector machines is efficient. IN: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2008)

    Google Scholar 

  47. Pelikan, M., Mühlenbein, H.: The bivariate marginal distribution algorithm. In: Advances in Soft Computing, pp. 521–535. Springer (1999)

    Google Scholar 

  48. Pétrowski, A.: A clearing procedure as a niching method for genetic algorithms. In: IEEEInternational Conference on Evolutionary Computation, pp. 798–803. IEEE (1996)

    Google Scholar 

  49. Qing, L., Gang, W., Qiuping, W.: Restricted evolution based multimodal function optimization in holographic grating design. In: IEEE Congress on Evolutionary Computation, vol. 1, pp. 789–794. IEEE (2005)

    Google Scholar 

  50. Qu, B.Y., Suganthan, P.N., Das, S.: A distance-based locally informed particle swarm model for multimodal optimization. IEEE Trans. Evol. Comput. 17(3), 387–402 (2013)

    Article  Google Scholar 

  51. Qu, B.Y., Suganthan, P.N., Liang, J.J.: Differential evolution with neighborhood mutation for multimodal optimization. IEEE Trans. Evol. Comput. 16(5), 601–614 (2012)

    Article  Google Scholar 

  52. Sheng, W., Swift, S., Zhang, L., Liu, X.: A weighted sum validity function for clustering with a hybrid niching genetic algorithm. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 35(6), 1156–1167 (2005)

    Article  Google Scholar 

  53. Shi, Y., Eberhart, R.: A modified particle swarm optimizer. In: IEEE Congress on Evolutionary Computation, pp. 69–73. IEEE (1998)

    Google Scholar 

  54. Socha, K., Dorigo, M.: Ant colony optimization for continuous domains. Eur. J. Oper. Res. 185(3), 1155–1173 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  55. Song, A., Chen, W.N., Gu, T.L., Yuan, H.Q., Kwong, S., Zhang, J.: Distributed virtual network embedding system with historical archives and set-based particle swarm optimization. IEEE Trans. Syst. Man Cybern.: Syst. 51(2), 927–942 (2021)

    Google Scholar 

  56. Srinivas, M., Patnaik, L.M.: Genetic algorithms: a survey. Computer 27(6), 17–26 (1994)

    Article  Google Scholar 

  57. Stoean, C., Preuss, M., Stoean, R., Dumitrescu, D.: Multimodal optimization by means of a topological species conservation algorithm. IEEE Trans. Evol. Comput. 14(6), 842–864 (2010)

    Article  Google Scholar 

  58. Suganthan, P.N.: Particle swarm optimiser with neighbourhood operator. In: IEEE Congress on Evolutionary Computation, vol. 3, pp. 1958–1962. IEEE (1999)

    Google Scholar 

  59. Tan, D.Z., Chen, W.N., Zhang, J., Yu, W.J.: Fast pedestrian detection using multimodal estimation of distribution algorithms. In: The Genetic and Evolutionary Computation Conference, pp. 1248–1255. ACM (2017)

    Google Scholar 

  60. Thomsen, R.: Multimodal optimization using crowding-based differential evolution. In: IEEE Congress on Evolutionary Computation, vol. 2, pp. 1382–1389. IEEE (2004)

    Google Scholar 

  61. Wang, Y., Li, H.X., Yen, G.G., Song, W.: Mommop: multiobjective optimization for locating multiple optimal solutions of multimodal optimization problems. IEEE Trans. Cybern. 45(4), 830–843 (2015)

    Article  Google Scholar 

  62. Wang, Z.J., Zhan, Z.H., Lin, Y., Yu, W.J., Wang, H., Kwong, S., Zhang, J.: Automatic niching differential evolution with contour prediction approach for multimodal optimization problems. IEEE Trans. Evol. Comput. 24(1), 114–128 (2020)

    Article  Google Scholar 

  63. Wang, Z.J., Zhan, Z.H., Zhang, J.: Distributed minimum spanning tree differential evolution for multimodal optimization problems. Soft. Comput. 23(24), 13339–13349 (2019)

    Article  Google Scholar 

  64. Wong, K.C., Leung, K.S., Wong, M.H.: Protein structure prediction on a lattice model via multimodal optimization techniques. In: The Annual Conference on Genetic and Evolutionary Computation, pp. 155–162. ACM (2010)

    Google Scholar 

  65. Wu, Y., Ma, W., Miao, Q., Wang, S.: Multimodal continuous ant colony optimization for multisensor remote sensing image registration with local search. Swarm Evol. Comput. 47, 89–95 (2019)

    Article  Google Scholar 

  66. Xing, L.N., Rohlfshagen, P., Chen, Y.W., Yao, X.: A hybrid ant colony optimization algorithm for the extended capacitated arc routing problem. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 41(4), 1110–1123 (2011)

    Article  Google Scholar 

  67. Yang, P., Tang, K., Lu, X.: Improving estimation of distribution algorithm on multimodal problems by detecting promising areas. IEEE Trans. Cybern. 45(8), 1438–1449 (2015)

    Article  Google Scholar 

  68. Yang, Q., Chen, W.N., Da Deng, J., Li, Y., Gu, T., Zhang, J.: A level-based learning swarm optimizer for large scale optimization. IEEE Trans. Evolut. Comput. (2017)

    Google Scholar 

  69. Yang, Q., Chen, W.N., Gu, T., Zhang, H., Deng, J.D., Li, Y., Zhang, J.: Segment-based predominant learning swarm optimizer for large-scale optimization. IEEE Trans. Cybern. 47(9), 2896–2910 (2017)

    Google Scholar 

  70. Yang, Q., Chen, W.N., Gu, T.L., Zhang, H.X., Yuan, H.Q., Kwong, S., Zhang, J.: A distributed swarm optimizer with adaptive communication for large-scale optimization. IEEE Trans. Cybern. 50(7), 3393–3408 (2020)

    Google Scholar 

  71. Yang, Q., Chen, W.N., Li, Y., Chen, C.P., Xu, X.M., Zhang, J.: Multimodal estimation of distribution algorithms. IEEE Trans. Cybern. 47(3), 636–650 (2017)

    Article  Google Scholar 

  72. Yang, Q., Chen, W.N., Yu, Z., Gu, T., Li, Y., Zhang, H., Zhang, J.: Adaptive multimodal continuous ant colony optimization. IEEE Trans. Evol. Comput. 21(2), 191–205 (2017)

    Article  Google Scholar 

  73. Yao, X., Liu, Y., Lin, G.: Evolutionary programming made faster. IEEE Trans. Evol. Comput. 3(2), 82–102 (1999)

    Article  Google Scholar 

  74. Yu, X., Chen, W.N., Gu, T.L., Yuan, H.Q., Zhang, H.X., Zhang, J.: Aco-a*: Ant colony optimization plus a* for 3-d traveling in environments with dense obstacles. IEEE Trans. Evol. Comput. 23(4), 617–631 (2019)

    Article  Google Scholar 

  75. Zhang, Y.H., Gong, Y.J., Gao, Y., Wang, H., Zhang, J.: Parameter-free voronoi neighborhood for evolutionary multimodal optimization. IEEE Trans. Evolut. Comput. 24(2), 335–349 (2020)

    Google Scholar 

  76. Zhang, Y.H., Gong, Y.J., Yuan, H.Q., Zhang, J.: A tree-structured random walking swarm optimizer for multimodal optimization. Appl. Soft Comput. 78, 94–108 (2019)

    Article  Google Scholar 

  77. Zhang, Y.H., Gong, Y.J., Zhang, H.X., Gu, T.L., Zhang, J.: Toward fast niching evolutionary algorithms: a locality sensitive hashing-based approach. IEEE Trans. Evol. Comput. 21(3), 347–362 (2017)

    Google Scholar 

  78. Zhao, H., Zhan, Z.H., Lin, Y., Chen, X.F., Luo, X.N., Zhang, J., Kwong, S., Zhang, J.: Local binary pattern-based adaptive differential evolution for multimodal optimization problems. IEEE Trans. Cybern. 50(7), 3343–3357 (2020)

    Google Scholar 

  79. Zhou, A., Sun, J., Zhang, Q.: An estimation of distribution algorithm with cheap and expensive local search methods. IEEE Trans. Evol. Comput. 19(6), 807–822 (2015)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the Key Project of Science and Technology Innovation 2030 supported by the Ministry of Science and Technology of China (Grant No. 2018AAA0101300), in part by the National Natural Science Foundation of China under Grant 61976093, 62006124, and 61873097, in part by the Science and Technology Plan Project of Guangdong Province 2018B050502006, in part by Guangdong Natural Science Foundation Research Team 2018B030312003, in part by the Natural Science Foundation of Jiangsu Province under Project BK20200811, in part by the Natural Science Foundation of the Jiangsu Higher Education Institutions of China under Grant 20KJB520006 and in part by the Startup Foundation for Introducing Talent of NUIST.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Wei-Neng Chen .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Yang, Q., Chen, WN., Zhang, J. (2021). Probabilistic Multimodal Optimization. In: Preuss, M., Epitropakis, M.G., Li, X., Fieldsend, J.E. (eds) Metaheuristics for Finding Multiple Solutions. Natural Computing Series. Springer, Cham. https://doi.org/10.1007/978-3-030-79553-5_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-79553-5_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-79552-8

  • Online ISBN: 978-3-030-79553-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics