research-article

Using free cloud storage services for distributed evolutionary algorithms

Authors:

Maribel García-Arenas,

Juan Julián Merelo Guervós,

Pedro Castillo,

Juan Luis Jiménez Laredo,

Gustavo Romero,

Antonio M. MoraAuthors Info & Claims

GECCO '11: Proceedings of the 13th annual conference on Genetic and evolutionary computation

Pages 1603 - 1610

https://doi.org/10.1145/2001576.2001792

Published: 12 July 2011 Publication History

Abstract

Cloud computing, in general, is becoming part of the toolset that the scientist uses to perform compute-intensive tasks. In particular, cloud storage is an easy and convenient way of storing files that will be accessible over the Internet, but also a way of distributing those files and performing distributed computation using them. In this paper we describe how such a service commercialized by Dropbox is used for pool-based evolutionary algorithms. A prototype system is described and its peformance measured over a deceptive combinatorial optimization problem, finding that, for some type of problems and using commodity hardware, cloud storage systems can profitably be used as a platform for distributed evolutionary algorithms. Preliminary results show that Dropbox is indeed a viable alternative for execution of pool-based distributed evolutionary algorithms, showing a good scaling behavior with up to 4 computers.

References

[1]

David P. Anderson, Eric Korpela, and Rom Walton. High-performance task distribution for volunteer computing. In E-SCIENCE '05: Proceedings of the First International Conference on e-Science and Grid Computing, pages 196--203, Washington, DC, USA, 2005. IEEE Computer Society.

Digital Library

[2]

Daniel Lombrana Gonzalez, Francisco Fernandez de Vega, Leonardo Trujillo, Gustavo Olague, F. Chavez de la O, M. Cardenas, Lourdes Araujo, Pedro A. Castillo, and Ken Sharman. Increasing gp computing power via volunteer computing. CoRR, abs/0801.1210, 2008.

[3]

D.L. Gonzalez, F.F. de Vega, L. Trujillo, G. Olague, L. Araujo, P. Castillo, J.J. Merelo, and K. Sharman. Increasing GP computing power for free via desktop GRID computing and virtualization. In Parallel, Distributed and Network-based Processing, 2009 17th Euromicro International Conference on, pages 419--423, Feb. 2009.

Digital Library

[4]

J.J. Merelo, P.A. Castillo, J.L.J. Laredo, A. Mora, and A. Prieto. Asynchronous distributed genetic algorithms with Javascript and JSON. In WCCI 2008 Proceedings, pages 1372-1379. IEEE Press, 2008.

[5]

Wikipedia. Cloud storage -- Wikipedia, The Free Encyclopedia, 2011. {Online; accessed 5-February-2011}.

[6]

J. Varia. Cloud architectures. White Paper of Amazon, 2008.

[7]

Wikipedia. Dropbox (service) -- wikipedia, the free encyclopedia, 2011. {Online; accessed 5-February-2011}.

[8]

E. Cantu-Paz. Topologies, migration rates, and multi-population parallel genetic algorithms. In Genetic and Evolutionary Computation Conference, GECCO-99, pages 13--17, 1999.

[9]

Michael Armbrust, Armando Fox, Rean Griffith, Anthony D. Joseph, Randy Katz, Andy Konwinski, Gunho Lee, David Patterson, Ariel Rabkin, Ion Stoica, and Matei Zaharia. A view of cloud computing. Commun. ACM, 53:50--58, April 2010.

Digital Library

[10]

Hai Jin, Shadi Ibrahim, Tim Bell, Wei Gao, Dachuan Huang, and Song Wu. Cloud types and services. In Borko Furht and Armando Escalante, editors, Handbook of Cloud Computing, pages 335--355. Springer US, 2010.

[11]

James Broberg, Rajkumar Buyya, and Zahir Tari. Metacdn: Harnessing {'}storage clouds' for high performance content delivery. Journal of Network and Computer Applications, 32(5):1012--1022, 2009. Next Generation Content Networks.

Digital Library

[12]

Lluis Pamies-Juarez, Pedro Garcia-Lopez, Marc Sanchez-Artigas, and Blas Herrera. Towards the design of optimal data redundancy schemes for heterogeneous cloud storage infrastructures. Computer Networks, In Press, Corrected Proof:, 2010.

Digital Library

[13]

A. Chervenak, I. Foster, C. Kesselman, C. Salisbury, and S. Tuecke. The data grid: Towards an architecture for the distributed management and analysis of large scientific datasets. Journal of Network and Computer Applications, 23(3):187--200, 2000.

Digital Library

[14]

Albert-Laszlo Barabasi, Vincent W. Freeh, Hawoong Jeong, and Jay B. Brockman. Parasitic computing. Nature, 412(6850):894--897, August 2001.

[15]

P.S. de Souza and S.N. Talukdar. Genetic algorithms in asynchronous teams. In Proceedings of the Fourth International Conference on Genetic Algorithms, pages 392--399. Morgan Kaufmann Publishers, 1991.

[16]

S. Talukdar, L. Baerentzen, A. Gove, and P. De Souza. Asynchronous teams: Cooperation schemes for autonomous agents. Journal of Heuristics, 4(4):295--321, 1998.

Digital Library

[17]

S. Talukdar, S. Murthy, and R. Akkiraju. Asynchronous teams. INTERNATIONAL SERIES IN OPERATIONS RESEARCH AND MANAGEMENT SCIENCE, pages 537--556, 2003.

[18]

X. Llora, B. Acs, L.S. Auvil, B. Capitanu, M.E. Welge, and D.E. Goldberg. Meandre: Semantic-driven data-intensive flows in the clouds. Technical Report 2008103, Illinois Genetic Algorithms Laboratory, 2008.

Digital Library

[19]

G. Roy, Hyunyoung Lee, J.L. Welch, Yuan Zhao, V. Pandey, and D. Thurston. A distributed pool architecture for genetic algorithms. In Evolutionary Computation, 2009. CEC '09. IEEE Congress on, pages 1177--1184, May 2009.

Digital Library

[20]

A. Bollini and M. Piastra. Distributed and persistent evolutionary algorithms: a design pattern. In Genetic Programming, Proceedings EuroGP'99, number 1598 in Lecture notes in computer science, pages 173--183. Springer, 1999.

Digital Library

[21]

J.J. Merelo. Fluid evolutionary algorithms. In Evolutionary Computation (CEC), 2010 IEEE Congress on, pages 1--8. IEEE, 2010.

[22]

D. Whitley, S. Rana, and R. Heckendorn. Island model genetic algorithms and linearly separable problems. Evolutionary Computing, pages 109--125, 1997.

Digital Library

[23]

David H. Ackley. A connectionist machine for genetic hillclimbing. Kluwer Academic Publishers, Norwell, MA, USA, 1987.

Digital Library

[24]

Juan Luis Jimenez Laredo, A. E. Eiben, Maarten van Steen, and Juan Julian Merelo Guervos. EvAg: a scalable peer-to-peer evolutionary algorithm. Genetic Programming and Evolvable Machines, 11(2):227--246, 2010.

Digital Library

[25]

David E. Goldberg, Kalyanmoy Deb, and Jeffrey Horn. Massive multimodality, deception, and genetic algorithms. In R. Manner and B. Manderick, editors, Parallel Problem Solving from Nature, 2, pages 37--48, Amsterdam, 1992. Elsevier Science Publishers, B. V.

[26]

Juan J. Merelo, Antonio M. Mora, Pedro A. Castillo, Juan L. J. Laredo, Lourdes Araujo, Ken C. Sharman, Anna I. Esparcia-Alcazar, Eva Alfaro-Cid, and Carlos Cotta. Testing the intermediate disturbance hypothesis: Effect of asynchronous population incorporation on multi-deme evolutionary algorithms. In Gunter Rudolph, Thomas Jansen, Simon Lucas, Carlo Poloni, and Nicola Beume, editors, Parallel Problem Solving from Nature - PPSN X, volume 5199 of LNCS, pages 266--275, Dortmund, 13-17 September 2008. Springer.

Digital Library

[27]

B. Baran, E. Kaszkurewicz, and A. Bhaya. Parallel asynchronous team algorithms: Convergence and performance analysis. IEEE transactions on parallel and distributed systems, 7(7):677--688, 1996.

Digital Library

Cited By

Mancilla ACastillo OValdez M(2022)Mixing Population-Based Metaheuristics: An Approach Based on a Distributed-Queue for the Optimal Design of Fuzzy ControllersIntelligent and Fuzzy Systems10.1007/978-3-031-09173-5_96(839-846)Online publication date: 5-Jul-2022
https://doi.org/10.1007/978-3-031-09173-5_96
Garcia-Valdez MMarquez RTrujillo LMerelo J(2021)Random Selection of Parameters in Asynchronous Pool-Based Evolutionary Algorithms2021 IEEE Congress on Evolutionary Computation (CEC)10.1109/CEC45853.2021.9504880(2531-2538)Online publication date: 28-Jun-2021
https://doi.org/10.1109/CEC45853.2021.9504880
Wang YLi YChen ZXue Y(2017)Cooperative particle swarm optimization using MapReduceSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-016-2390-921:22(6593-6603)Online publication date: 1-Nov-2017
https://dl.acm.org/doi/10.1007/s00500-016-2390-9
Show More Cited By

Index Terms

Using free cloud storage services for distributed evolutionary algorithms
1. Information systems
  1. Information systems applications

Recommendations

Cloud Storage as the Infrastructure of Cloud Computing
ICICCI '10: Proceedings of the 2010 International Conference on Intelligent Computing and Cognitive Informatics

As an emerging technology and business paradigm, Cloud Computing has taken commercial computing by storm. Cloud computing platforms provide easy access to a company’s high-performance computing and storage infrastructure through web services. With cloud ...
Middleware enabled data sharing on cloud storage services
MW4SOC '10: Proceedings of the 5th International Workshop on Middleware for Service Oriented Computing

With the emergence of public cloud storage platforms like Amazon, Microsoft and Google etc, individual applications and some enterprise storage are being increasingly deployed on Clouds. However, dynamic data sharing in public clouds face problems of ...
Cloud Computing-based Data Storage and Disaster Recovery
ICFCSE '11: Proceedings of the 2011 International Conference on Future Computer Science and Education

To solve the problems of business database services such as storage capacity, performance, stability, security, load, and many other issues, cloud storage was used to provide the cloud-based data platform. Data services were put on the cloud, and the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '11: Proceedings of the 13th annual conference on Genetic and evolutionary computation

July 2011

2140 pages

ISBN:9781450305570

DOI:10.1145/2001576

Editor:
Natalio Krasnogor
University of Nottingham, UK
,
General Chair:
Pier Luca Lanzi
Politecnico di Milano, Italy

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGEVO: ACM Special Interest Group on Genetic and Evolutionary Computation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 July 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

GECCO '11

Sponsor:

SIGEVO

GECCO '11: Genetic and Evolutionary Computation Conference

July 12 - 16, 2011

Dublin, Ireland

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
398
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Mancilla ACastillo OValdez M(2022)Mixing Population-Based Metaheuristics: An Approach Based on a Distributed-Queue for the Optimal Design of Fuzzy ControllersIntelligent and Fuzzy Systems10.1007/978-3-031-09173-5_96(839-846)Online publication date: 5-Jul-2022
https://doi.org/10.1007/978-3-031-09173-5_96
Garcia-Valdez MMarquez RTrujillo LMerelo J(2021)Random Selection of Parameters in Asynchronous Pool-Based Evolutionary Algorithms2021 IEEE Congress on Evolutionary Computation (CEC)10.1109/CEC45853.2021.9504880(2531-2538)Online publication date: 28-Jun-2021
https://doi.org/10.1109/CEC45853.2021.9504880
Wang YLi YChen ZXue Y(2017)Cooperative particle swarm optimization using MapReduceSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-016-2390-921:22(6593-6603)Online publication date: 1-Nov-2017
https://dl.acm.org/doi/10.1007/s00500-016-2390-9
Salto CAlba E(2015)Adapting Distributed Evolutionary Algorithms to Heterogeneous HardwareTransactions on Computational Collective Intelligence XIX - Volume 938010.1007/978-3-662-49017-4_7(103-125)Online publication date: 1-Sep-2015
https://dl.acm.org/doi/10.1007/978-3-662-49017-4_7

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten