skip to main content
10.1145/2001576.2001792acmconferencesArticle/Chapter ViewAbstractPublication PagesgeccoConference Proceedingsconference-collections
research-article

Using free cloud storage services for distributed evolutionary algorithms

Published: 12 July 2011 Publication History

Abstract

Cloud computing, in general, is becoming part of the toolset that the scientist uses to perform compute-intensive tasks. In particular, cloud storage is an easy and convenient way of storing files that will be accessible over the Internet, but also a way of distributing those files and performing distributed computation using them. In this paper we describe how such a service commercialized by Dropbox is used for pool-based evolutionary algorithms. A prototype system is described and its peformance measured over a deceptive combinatorial optimization problem, finding that, for some type of problems and using commodity hardware, cloud storage systems can profitably be used as a platform for distributed evolutionary algorithms. Preliminary results show that Dropbox is indeed a viable alternative for execution of pool-based distributed evolutionary algorithms, showing a good scaling behavior with up to 4 computers.

References

[1]
David P. Anderson, Eric Korpela, and Rom Walton. High-performance task distribution for volunteer computing. In E-SCIENCE '05: Proceedings of the First International Conference on e-Science and Grid Computing, pages 196--203, Washington, DC, USA, 2005. IEEE Computer Society.
[2]
Daniel Lombrana Gonzalez, Francisco Fernandez de Vega, Leonardo Trujillo, Gustavo Olague, F. Chavez de la O, M. Cardenas, Lourdes Araujo, Pedro A. Castillo, and Ken Sharman. Increasing gp computing power via volunteer computing. CoRR, abs/0801.1210, 2008.
[3]
D.L. Gonzalez, F.F. de Vega, L. Trujillo, G. Olague, L. Araujo, P. Castillo, J.J. Merelo, and K. Sharman. Increasing GP computing power for free via desktop GRID computing and virtualization. In Parallel, Distributed and Network-based Processing, 2009 17th Euromicro International Conference on, pages 419--423, Feb. 2009.
[4]
J.J. Merelo, P.A. Castillo, J.L.J. Laredo, A. Mora, and A. Prieto. Asynchronous distributed genetic algorithms with Javascript and JSON. In WCCI 2008 Proceedings, pages 1372-1379. IEEE Press, 2008.
[5]
Wikipedia. Cloud storage -- Wikipedia, The Free Encyclopedia, 2011. {Online; accessed 5-February-2011}.
[6]
J. Varia. Cloud architectures. White Paper of Amazon, 2008.
[7]
Wikipedia. Dropbox (service) -- wikipedia, the free encyclopedia, 2011. {Online; accessed 5-February-2011}.
[8]
E. Cantu-Paz. Topologies, migration rates, and multi-population parallel genetic algorithms. In Genetic and Evolutionary Computation Conference, GECCO-99, pages 13--17, 1999.
[9]
Michael Armbrust, Armando Fox, Rean Griffith, Anthony D. Joseph, Randy Katz, Andy Konwinski, Gunho Lee, David Patterson, Ariel Rabkin, Ion Stoica, and Matei Zaharia. A view of cloud computing. Commun. ACM, 53:50--58, April 2010.
[10]
Hai Jin, Shadi Ibrahim, Tim Bell, Wei Gao, Dachuan Huang, and Song Wu. Cloud types and services. In Borko Furht and Armando Escalante, editors, Handbook of Cloud Computing, pages 335--355. Springer US, 2010.
[11]
James Broberg, Rajkumar Buyya, and Zahir Tari. Metacdn: Harnessing {'}storage clouds' for high performance content delivery. Journal of Network and Computer Applications, 32(5):1012--1022, 2009. Next Generation Content Networks.
[12]
Lluis Pamies-Juarez, Pedro Garcia-Lopez, Marc Sanchez-Artigas, and Blas Herrera. Towards the design of optimal data redundancy schemes for heterogeneous cloud storage infrastructures. Computer Networks, In Press, Corrected Proof:, 2010.
[13]
A. Chervenak, I. Foster, C. Kesselman, C. Salisbury, and S. Tuecke. The data grid: Towards an architecture for the distributed management and analysis of large scientific datasets. Journal of Network and Computer Applications, 23(3):187--200, 2000.
[14]
Albert-Laszlo Barabasi, Vincent W. Freeh, Hawoong Jeong, and Jay B. Brockman. Parasitic computing. Nature, 412(6850):894--897, August 2001.
[15]
P.S. de Souza and S.N. Talukdar. Genetic algorithms in asynchronous teams. In Proceedings of the Fourth International Conference on Genetic Algorithms, pages 392--399. Morgan Kaufmann Publishers, 1991.
[16]
S. Talukdar, L. Baerentzen, A. Gove, and P. De Souza. Asynchronous teams: Cooperation schemes for autonomous agents. Journal of Heuristics, 4(4):295--321, 1998.
[17]
S. Talukdar, S. Murthy, and R. Akkiraju. Asynchronous teams. INTERNATIONAL SERIES IN OPERATIONS RESEARCH AND MANAGEMENT SCIENCE, pages 537--556, 2003.
[18]
X. Llora, B. Acs, L.S. Auvil, B. Capitanu, M.E. Welge, and D.E. Goldberg. Meandre: Semantic-driven data-intensive flows in the clouds. Technical Report 2008103, Illinois Genetic Algorithms Laboratory, 2008.
[19]
G. Roy, Hyunyoung Lee, J.L. Welch, Yuan Zhao, V. Pandey, and D. Thurston. A distributed pool architecture for genetic algorithms. In Evolutionary Computation, 2009. CEC '09. IEEE Congress on, pages 1177--1184, May 2009.
[20]
A. Bollini and M. Piastra. Distributed and persistent evolutionary algorithms: a design pattern. In Genetic Programming, Proceedings EuroGP'99, number 1598 in Lecture notes in computer science, pages 173--183. Springer, 1999.
[21]
J.J. Merelo. Fluid evolutionary algorithms. In Evolutionary Computation (CEC), 2010 IEEE Congress on, pages 1--8. IEEE, 2010.
[22]
D. Whitley, S. Rana, and R. Heckendorn. Island model genetic algorithms and linearly separable problems. Evolutionary Computing, pages 109--125, 1997.
[23]
David H. Ackley. A connectionist machine for genetic hillclimbing. Kluwer Academic Publishers, Norwell, MA, USA, 1987.
[24]
Juan Luis Jimenez Laredo, A. E. Eiben, Maarten van Steen, and Juan Julian Merelo Guervos. EvAg: a scalable peer-to-peer evolutionary algorithm. Genetic Programming and Evolvable Machines, 11(2):227--246, 2010.
[25]
David E. Goldberg, Kalyanmoy Deb, and Jeffrey Horn. Massive multimodality, deception, and genetic algorithms. In R. Manner and B. Manderick, editors, Parallel Problem Solving from Nature, 2, pages 37--48, Amsterdam, 1992. Elsevier Science Publishers, B. V.
[26]
Juan J. Merelo, Antonio M. Mora, Pedro A. Castillo, Juan L. J. Laredo, Lourdes Araujo, Ken C. Sharman, Anna I. Esparcia-Alcazar, Eva Alfaro-Cid, and Carlos Cotta. Testing the intermediate disturbance hypothesis: Effect of asynchronous population incorporation on multi-deme evolutionary algorithms. In Gunter Rudolph, Thomas Jansen, Simon Lucas, Carlo Poloni, and Nicola Beume, editors, Parallel Problem Solving from Nature - PPSN X, volume 5199 of LNCS, pages 266--275, Dortmund, 13-17 September 2008. Springer.
[27]
B. Baran, E. Kaszkurewicz, and A. Bhaya. Parallel asynchronous team algorithms: Convergence and performance analysis. IEEE transactions on parallel and distributed systems, 7(7):677--688, 1996.

Cited By

View all
  • (2022)Mixing Population-Based Metaheuristics: An Approach Based on a Distributed-Queue for the Optimal Design of Fuzzy ControllersIntelligent and Fuzzy Systems10.1007/978-3-031-09173-5_96(839-846)Online publication date: 5-Jul-2022
  • (2021)Random Selection of Parameters in Asynchronous Pool-Based Evolutionary Algorithms2021 IEEE Congress on Evolutionary Computation (CEC)10.1109/CEC45853.2021.9504880(2531-2538)Online publication date: 28-Jun-2021
  • (2017)Cooperative particle swarm optimization using MapReduceSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-016-2390-921:22(6593-6603)Online publication date: 1-Nov-2017
  • Show More Cited By

Index Terms

  1. Using free cloud storage services for distributed evolutionary algorithms

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    GECCO '11: Proceedings of the 13th annual conference on Genetic and evolutionary computation
    July 2011
    2140 pages
    ISBN:9781450305570
    DOI:10.1145/2001576
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 12 July 2011

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. cloud computing
    2. cloud storage
    3. distributed algorithms
    4. evolutionary algorithms

    Qualifiers

    • Research-article

    Conference

    GECCO '11
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)3
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 01 Mar 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2022)Mixing Population-Based Metaheuristics: An Approach Based on a Distributed-Queue for the Optimal Design of Fuzzy ControllersIntelligent and Fuzzy Systems10.1007/978-3-031-09173-5_96(839-846)Online publication date: 5-Jul-2022
    • (2021)Random Selection of Parameters in Asynchronous Pool-Based Evolutionary Algorithms2021 IEEE Congress on Evolutionary Computation (CEC)10.1109/CEC45853.2021.9504880(2531-2538)Online publication date: 28-Jun-2021
    • (2017)Cooperative particle swarm optimization using MapReduceSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-016-2390-921:22(6593-6603)Online publication date: 1-Nov-2017
    • (2015)Adapting Distributed Evolutionary Algorithms to Heterogeneous HardwareTransactions on Computational Collective Intelligence XIX - Volume 938010.1007/978-3-662-49017-4_7(103-125)Online publication date: 1-Sep-2015

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media