Abstract
MapReduce is a popular cloud computing platform which has been widely applied in large-scale data-intensive fields. However, when dealing with computation extensive tasks, particularly, iterative computation, frequent loading Map and Reduce processes will lead to overhead. Resilient distributed datasets model which has been implemented in Spark, is an in-memory clustering computing which can overcome this shortcoming efficiently. In this paper, we attempt to use resilient distributed datasets model to parallelize Differential Evolution algorithm. A wide range of benchmark problems have been adopted to conduct numerical experiment, and the speedup of PDE due to use of resilient distributed datasets model is demonstrated. The results show us that resilient distributed datasets model is a potential way to parallelize evolutionary algorithm.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Store, R., Price, K.V.: Differential evolution CA simple and efficient heuristic for global optimization over continuous spaces. J. Glob. Optim. 11(4), 341–359 (1997)
Vesterstrom, J., Thomsen, R.: A comparative study of differential evolution, particle swarm optimization, and evolutionary algorithms on numerical benchmark problems, pp. 1980–1987 (2007)
Yousefi, H., Handroos, H., Soleymani, A.: Application of differential evolution in system identification of a servo-hydraulic system with a flexible load. Mechatron. 18(9), 513–528 (2008)
Rocca, P., Oliveri, G., Massa, A.: Differential evolution as applied to electromagnetics. Antennas Propag. Mag. 53(1), 38–49 (2011)
Wang, Y., Li, H.X., Huang, T.: Differential evolution based on covariance matrix learning and bimodal distribution parameter setting. Appl. Softw. Comput. 18, 232–247 (2014)
Wang, Y., Cai, Z., Zhang, Q.: Enhancing the search ability of differential evolution through orthogonal crossover. Inf. Sci. 185(1), 153–177 (2012)
Wang, Y., Cai, Z., Zhang, Q.: Differential evolution with composite trial vector generation strategies and control parameters. IEEE Trans. Evol. Comput. 15(1), 55–66 (2011)
Alba, E., Tomassini, M.: Parallelism and evolutionary algorithms. IEEE Trans. Evol. Comput. 6(5), 443–462 (2002)
Zaharie, D., Petcu, D.: Parallel implementation of multi-population differential evolution. Concurrent Inf. Process. Comput. 48, 223–232 (2005)
Wang, H., Rahnamayan, S., Wu, Z.: Parallel differential evolution with self-adapting control parameters and generalized opposition-based learning for solving high-dimensional optimization problems. J. Parallel Distrib. Comput. 73(1), 62–73 (2013)
Fabris, F., Krohling, R.A.: A co-evolutionary differential evolution algorithm for solving minCmax optimization problems implemented on GPU using C-CUDA. Expert Syst. Appl. 39(12), 10324–10333 (2012)
Dean, J., Ghemawat, S.: MapReduce: Simplified data processing on large clusters. Commun. ACM. 51(1), 107–113 (2008)
Zhou, C.: Fast parallelization of differential evolution algorithm using MapReduce. In: Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation, pp. 1113–1114, Dubin, Ireland (2011)
Pavlech, M.: Framework for development of distributed evolutionary algorithms based on MapReduce. In: Proceedings of the 22nd International DAAAM Symposium on Intelligent Manufacturing and Automation: Power of Knowledge and Creativity, pp. 1475–1476, Vienna (2011)
McNabb, A.W., Monson, C.K., Seppi, K.D.: Parallel PSO using mapreduce. In: Proceedings of IEEE Congress on Evolutionary Computation, pp. 7–14. IEEE, Singapore (2007)
Verma, A., Llora, X., Goldberg, D.E.: Scaling genetic algorithms using mapreduce. In: Proceedings of the Ninth International Conference on Intelligent Systems Design and Applications, pp. 13–18. IEEE, Pisa (2009)
Zaharia, M., Chowdhury, M., Das, T.: Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. In: The 9th USENIX Conference on Networked Systems Design and Implementation, 2012, pp. 1–16. USENIX Association, Berkeley (2012)
Isard, M., Budiu, M., Yu, Y., et al.: Dryad: distributed data-parallel programs from sequential building blocks. ACM SIGOPS Operating Syst. Rev. 41(3), 59–72 (2007)
Kiyouharu, T., Takashi, I.: Concurrent differential evolution based on MapReduce. Int. J. Comput. 4(4), 161–168 (2010)
Tang, K., Li, X., Suganthan, K.: Benchmark Functions for the CEC’2010 Special Session and Competition on Large Scale Global Optimization. Technical report, IEEE (2009)
Acknowledgments
This work is partially supported by Natural Science Foundation of China under grant No. 61364025, State Key Laboratory of Software Engineering Foundation under grant No. SKLSE2012-09-39 and the Science and Technology Foundation of Jiangxi Province, China under grant No. GJJ13729 and No. GJJ14742.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Deng, C., Tan, X., Dong, X., Tan, Y. (2015). A Parallel Version of Differential Evolution Based on Resilient Distributed Datasets Model. In: Gong, M., Linqiang, P., Tao, S., Tang, K., Zhang, X. (eds) Bio-Inspired Computing -- Theories and Applications. BIC-TA 2015. Communications in Computer and Information Science, vol 562. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-49014-3_8
Download citation
DOI: https://doi.org/10.1007/978-3-662-49014-3_8
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-49013-6
Online ISBN: 978-3-662-49014-3
eBook Packages: Computer ScienceComputer Science (R0)