Abstract
Interactive evolution, i.e. leveraging human input for selection in an evolutionary algorithm, is effective when an appropriate fitness function is hard to quantify yet solution quality is easily recognizable by humans. However, single-user applications of interactive evolution are limited by user fatigue: Humans become bored with monotonous evaluations. This paper explores the potential for bypassing such fatigue by directly purchasing human input from human computation markets. Experiments evolving aesthetic images show that purchased human input can be leveraged more economically when evolution is first seeded by optimizing a purely-computational aesthetic measure. Further experiments in the same domain validate a system feature, demonstrating how human computation can help guide interactive evolution system design. Finally, experiments in an image composition domain show the approach’s potential to make interactive evolution scalable even in tasks that are not inherently enjoyable. The conclusion is that human computation markets make it possible to apply a powerful form of selection pressure mechanically in evolutionary algorithms.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Chou, C., Kimbrough, S., Sullivan-Fedock, J., Woodard, C., Murphy, F.: Using interactive evolutionary computation (iec) with validated surrogate fitness functions for redistricting. In: Proceedings of the Fourteenth International Conference on Genetic and Evolutionary Computation Conference, pp. 1071–1078. ACM (2012)
Clune, J., Lipson, H.: Evolving three-dimensional objects with a generative encoding inspired by developmental biology. In: Proceedings of the European Conference on Artificial Life (2011)
den Heijer, E., Eiben, A.E.: Comparing aesthetic measures for evolutionary art. In: Di Chio, C., Brabazon, A., Di Caro, G.A., Ebner, M., Farooq, M., Fink, A., Grahl, J., Greenfield, G., Machado, P., O’Neill, M., Tarantino, E., Urquhart, N. (eds.) EvoApplications 2010, Part II. LNCS, vol. 6025, pp. 311–320. Springer, Heidelberg (2010)
Goldberg, D.E.: Simple genetic algorithms and the minimal deceptive problem. In: Davis, L.D. (ed.) Genetic Algorithms and Simulated Annealing, Research Notes in Artificial Intelligence. Morgan Kaufmann (1987)
Gruau, F., Quatramaran, K.: Cellular encoding for interactive evolutionary robotics. In: Fourth European Conference on Artificial Life, pp. 368–377. MIT Press (1997)
Hastings, E., Stanley, K.: Interactive genetic engineering of evolved video game content. In: Proceedings of the 2010 Workshop on Procedural Content Generation in Games, p. 8. ACM (2010)
Ipeirotis, P.: Analyzing the amazon mechanical turk marketplace. XRDS: Crossroads. The ACM Magazine for Students 17(2), 16–21 (2010)
Kittur, A., Chi, E.H., Suh, B.: Crowdsourcing user studies with mechanical turk. In: Proceedings of the Twenty-sixth Annual SIGCHI Conference on Human Factors in Computing Systems, pp. 453–456. ACM (2008)
Kosorukoff, A.: Human based genetic algorithm. In: 2001 IEEE International Conference on Systems, Man, and Cybernetics, pp. 3464–3469. IEEE (2001)
Lehman, J., Stanley, K.O.: Abandoning objectives: Evolution through the search for novelty alone. Evolutionary Computation 19(2), 189–223 (2011)
Lehman, J., Stanley, K.O.: Beyond open-endedness: Quantifying impressiveness. In: Proceedings of Artificial Life Thirteen, ALIFE XIII (2012)
MacCallum, R., Mauch, M., Burt, A., Leroi, A.: Evolution of music by public choice. Proceedings of the National Academy of Sciences 109(30), 12081–12086 (2012)
McCormack, J.: Open problems in evolutionary music and art. In: Rothlauf, F., Branke, J., Cagnoni, S., Corne, D.W., Drechsler, R., Jin, Y., Machado, P., Marchiori, E., Romero, J., Smith, G.D., Squillero, G. (eds.) EvoWorkshops 2005. LNCS, vol. 3449, pp. 428–436. Springer, Heidelberg (2005)
Miconi, T., Channon, A.: An improved system for artificial creatures evolution. In: Proceedings of the Tenth International Conference on Artificial Life (ALIFE X), pp. 255–261. MIT Press (2006)
Orkin, J., Roy, D.: The restaurant game: Learning social behavior and language from thousands of players online. Journal of Game Development 3(1), 39–60 (2007)
Risi, S., Lehman, J., D’Ambrosio, D.B., Hall, R., Stanley, K.O.: Combining search-based procedural content generation and social gaming in the petalz video game. In: Proceedings of the Artificial Intelligence and Interactive Digital Entertainment Conference (AIIDE 2012) (2012)
Secretan, J., Beato, N., D’Ambrosio, D., Rodriguez, A., Campbell, A., Folsom-Kovarik, J., Stanley, K.: Picbreeder: A case study in collaborative evolutionary exploration of design space. Evol. Comp. (2011) (to appear)
Stanley, K.: Compositional pattern producing networks: A novel abstraction of development. Genetic Programming and Evolvable Machines 8(2), 131–162 (2007)
Takagi, H.: Interactive evolutionary computation: Fusion of the capacities of EC optimization and human evaluation. Proceedings of the IEEE 89(9), 1275–1296 (2001)
Von Ahn, L.: Games with a purpose. Computer 39(6), 92–94 (2006)
Von Ahn, L., Maurer, B., McMillen, C., Abraham, D., Blum, M.: Recaptcha: Human-based character recognition via web security measures. Science 321(5895), 1465–1468 (2008)
Woolley, B.G., Stanley, K.O.: Exploring promising stepping stones by combining novelty search with interactive evolution. CoRR abs/1207.6682 (2012)
Yu, L., Nickerson, J.: An internet-scale idea generation system. ACM Transactions on Interactive Intelligent Systems 3(1), 2013 (2011)
Yu, L., Nickerson, J.V.: Cooks or cobblers?: crowd creativity through combination. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1393–1402. ACM (2011)
Zaera, N., Cliff, D., Bruten, J.: (Not) evolving collective behaviours in synthetic fish. In: From Animals to Animats 4: Proceedings of the Fourth International Conference on Simulation of Adaptive Behavior. MIT Press Bradford Books (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lehman, J., Miikkulainen, R. (2013). Boosting Interactive Evolution Using Human Computation Markets. In: Dediu, AH., Martín-Vide, C., Truthe, B., Vega-Rodríguez, M.A. (eds) Theory and Practice of Natural Computing. TPNC 2013. Lecture Notes in Computer Science, vol 8273. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-45008-2_1
Download citation
DOI: https://doi.org/10.1007/978-3-642-45008-2_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-45007-5
Online ISBN: 978-3-642-45008-2
eBook Packages: Computer ScienceComputer Science (R0)