skip to main content
10.1145/2908812.2908888acmconferencesArticle/Chapter ViewAbstractPublication PagesgeccoConference Proceedingsconference-collections
research-article

Non-negative Matrix Factorization for Unsupervised Derivation of Search Objectives in Genetic Programming

Published: 20 July 2016 Publication History

Abstract

In genetic programming (GP), the outcomes of the evaluation phase in an evolutionary loop can be represented as an interaction matrix, with rows corresponding to programs in a population, columns corresponding to tests that define a program synthesis task, and ones and zeroes signaling respectively passing a test and failing to do so. The conventional fitness, equivalent to a row sum in that matrix, only crudely reflects program's compliance with desired output, and recent contributions in semantic and behavioral GP point to alternative, multifaceted characterizations that facilitate navigation in the search space. In this paper, we propose DOF, a method that uses the popular machine learning technique of non-negative matrix factorization to heuristically derive a low number of underlying objectives from an interaction matrix. The resulting objectives redefine the original single-objective synthesis problem as a multiobjective optimization problem, and we posit that such characterization fosters diversification of search directions while maintaining useful search gradient. The comparative experiment conducted on 15 problems from discrete domains confirms this claim: DOF outperforms the conventional GP and GP equipped with an alternative method of derivation of search objectives on success rate and convergence speed.

References

[1]
A. Arcuri and X. Yao. Co-evolutionary automatic programming for software development. Information Sciences, 259:412--432, 2014.
[2]
M. W. Berry, M. Browne, A. N. Langville, V. P. Pauca, and R. J. Plemmons. Algorithms and applications for approximate nonnegative matrix factorization. Computational statistics & data analysis, 52(1):155--173, 2007.
[3]
A. Bucci, J. B. Pollack, and E. de Jong. Automated extraction of problem structure. In K. D. et al., editor, Genetic and Evolutionary Computation -- GECCO-2004, Part I, volume 3102 of Lecture Notes in Computer Science, pages 501--512, Seattle, WA, USA, 26--30 June 2004. Springer-Verlag.
[4]
S. Y. Chong, P. Tino, D. C. Ku, and Y. Xin. Improving Generalization Performance in Co-Evolutionary Learning. IEEE Transactions on Evolutionary Computation, 16(1):70--85, 2012.
[5]
E. D. de Jong and A. Bucci. DECA: dimension extracting coevolutionary algorithm. In M. C. et al., editor, GECCO 2006: Proceedings of the 8th annual conference on Genetic and evolutionary computation, pages 313--320, Seattle, Washington, USA, 2006. ACM Press.
[6]
E. D. de Jong and J. B. Pollack. Ideal Evaluation from Coevolution. Evolutionary Computation, 12(2):159--192, Summer 2004.
[7]
K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. A fast and elitist multiobjective genetic algorithm: NSGA-II. Evolutionary Computation, IEEE Transactions on, 6(2):182 --197, apr 2002.
[8]
R. Gaujoux and C. Seoighe. A flexible r package for nonnegative matrix factorization. BMC Bioinformatics, 11(1):367, 2010.
[9]
I. Goncalves, S. Silva, J. B. Melo, and J. M. B. Carreiras. Random sampling technique for overfitting control in genetic programming. In A. Moraglio, S. Silva, K. Krawiec, P. Machado, and C. Cotta, editors, Proceedings of the 15th European Conference on Genetic Programming, EuroGP 2012, volume 7244 of LNCS, pages 218--229, Malaga, Spain, 11--13 Apr. 2012. Springer Verlag.
[10]
T. Helmuth, L. Spector, and J. Matheson. Solving uncompromising problems with lexicase selection. IEEE Transactions on Evolutionary Computation, 19(5):630--643, Oct. 2015.
[11]
M. Hollander, D. A. Wolfe, and E. Chicken. Nonparametric statistical methods, volume 751. John Wiley & Sons, 2013.
[12]
W. Jaskowski. Algorithms for Test-Based Problems. PhD thesis, Institute of Computing Science, Poznan University of Technology, Poznan, Poland, 2011. Adviser: Krzysztof Krawiec.
[13]
W. Jaskowski and K. Krawiec. Coordinate system archive for coevolution. In Evolutionary Computation (CEC), 2010 IEEE Congress on, pages 1--10. IEEE, 2010.
[14]
W. Jaskowski and K. Krawiec. Formal analysis, hardness and algorithms for extracting internal structure of test-based problems. Evolutionary Computation, 19(4):639--671, 2011.
[15]
W. Jaskowski, P. Liskowski, M. Szubert, and K. Krawiec. Improving coevolution by random sampling. In Proceedings of the 15th annual conference on Genetic and evolutionary computation, pages 1141--1148. ACM, 2013.
[16]
G. K. Kanji. 100 statistical tests. Sage, 2006.
[17]
Y. Koren, R. Bell, and C. Volinsky. Matrix factorization techniques for recommender systems. Computer, 42(8):30--37, 2009.
[18]
J. R. Koza. Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge, MA, USA, 1992.
[19]
K. Krawiec and P. Liskowski. Automatic derivation of search objectives for test-based genetic programming. In Genetic Programming, pages 53--65. Springer, 2015.
[20]
H. Laurberg, M. G. Christensen, M. D. Plumbley, L. K. Hansen, and S. H. Jensen. Theorems on positive data: On the uniqueness of nmf. Computational intelligence and neuroscience, 2008, 2008.
[21]
D. D. Lee and H. S. Seung. Learning the parts of objects by non-negative matrix factorization. Nature, 401(6755):788--791, 1999.
[22]
P. Liskowski and K. Krawiec. Discovery of implicit objectives by compression of interaction matrix in test-based problems. In Parallel Problem Solving from Nature--PPSN XIII, pages 611--620. Springer, 2014.
[23]
P. Liskowski and K. Krawiec. Online Discovery of Search Objectives for Test-based Problems. Evolutionary Computation, mar 2016.
[24]
R. I. B. McKay. Fitness sharing in genetic programming. In D. Whitley, D. Goldberg, E. Cantu-Paz, L. Spector, I. Parmee, and H.-G. Beyer, editors, Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2000), pages 435--442, Las Vegas, Nevada, USA, 10--12 July 2000. Morgan Kaufmann.
[25]
A. Moraglio and K. Krawiec. Semantic genetic programming. In A. Simoes, editor, GECCO 2015 Advanced Tutorials, pages 603--627, Madrid, Spain, 11--15 July 2015. ACM.
[26]
P. Paatero and U. Tapper. Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values. Environmetrics, 5(2):111--126, 1994.
[27]
V. P. Pauca, F. Shahnaz, M. W. Berry, and R. J. Plemmons. Text mining using non-negative matrix factorizations. In SDM, volume 4, pages 452--456. SIAM, 2004.
[28]
R. E. Smith, S. Forrest, and A. S. Perelson. Searching for diverse, cooperative populations with genetic algorithms. Evolutionary computation, 1(2):127--149, 1993.
[29]
L. Spector, D. M. Clark, I. Lindsay, B. Barr, and J. Klein. Genetic programming for finite algebras. In M. Keijzer, editor, GECCO '08: Proceedings of the 10th annual conference on Genetic and evolutionary computation, pages 1291--1298, Atlanta, GA, USA, 12--16 July 2008. ACM.
[30]
M. Tomassini, L. Vanneschi, P. Collard, and M. Clergue. A study of fitness distance correlation as a difficulty measure in genetic programming. Evolutionary Computation, 13(2):213--239, Summer 2005.

Cited By

View all
  • (2020)Adversarial genetic programming for cyber security: a rising application domain where GP mattersGenetic Programming and Evolvable Machines10.1007/s10710-020-09389-y21:1-2(219-250)Online publication date: 1-Jun-2020
  • (2020)Neuromemetic Evolutionary OptimizationParallel Problem Solving from Nature – PPSN XVI10.1007/978-3-030-58112-1_43(623-636)Online publication date: 5-Sep-2020
  • (2020)Exploring Adversarial Artificial Intelligence for Autonomous Adaptive Cyber DefenseAdaptive Autonomous Secure Cyber Systems10.1007/978-3-030-33432-1_3(41-61)Online publication date: 5-Feb-2020
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
GECCO '16: Proceedings of the Genetic and Evolutionary Computation Conference 2016
July 2016
1196 pages
ISBN:9781450342063
DOI:10.1145/2908812
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 July 2016

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. genetic programming
  2. machine learning
  3. multiobjective optimization
  4. nonnegative matrix factorization

Qualifiers

  • Research-article

Funding Sources

Conference

GECCO '16
Sponsor:
GECCO '16: Genetic and Evolutionary Computation Conference
July 20 - 24, 2016
Colorado, Denver, USA

Acceptance Rates

GECCO '16 Paper Acceptance Rate 137 of 381 submissions, 36%;
Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)5
  • Downloads (Last 6 weeks)0
Reflects downloads up to 19 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2020)Adversarial genetic programming for cyber security: a rising application domain where GP mattersGenetic Programming and Evolvable Machines10.1007/s10710-020-09389-y21:1-2(219-250)Online publication date: 1-Jun-2020
  • (2020)Neuromemetic Evolutionary OptimizationParallel Problem Solving from Nature – PPSN XVI10.1007/978-3-030-58112-1_43(623-636)Online publication date: 5-Sep-2020
  • (2020)Exploring Adversarial Artificial Intelligence for Autonomous Adaptive Cyber DefenseAdaptive Autonomous Secure Cyber Systems10.1007/978-3-030-33432-1_3(41-61)Online publication date: 5-Feb-2020
  • (2019)Investigating algorithms for finding nash equilibria in cyber security problemsProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3319619.3326851(1659-1667)Online publication date: 13-Jul-2019
  • (2019)Solving complex problems with coevolutionary algorithmsProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3319619.3323384(975-1001)Online publication date: 13-Jul-2019
  • (2018)Solving complex problems with coevolutionary algorithmsProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3205651.3207888(880-906)Online publication date: 6-Jul-2018
  • (2018)Investigation of the latent space of stock market patterns with genetic programmingProceedings of the Genetic and Evolutionary Computation Conference10.1145/3205455.3205493(1254-1261)Online publication date: 2-Jul-2018
  • (2017)Adaptive Test Selection for Factorization-based Surrogate Fitness in Genetic ProgrammingFoundations of Computing and Decision Sciences10.1515/fcds-2017-001742:4(339-358)Online publication date: 9-Dec-2017
  • (2017)Discovery of search objectives in continuous domainsProceedings of the Genetic and Evolutionary Computation Conference10.1145/3071178.3071344(969-976)Online publication date: 1-Jul-2017
  • (2017)Accelerating coevolution with adaptive matrix factorizationProceedings of the Genetic and Evolutionary Computation Conference10.1145/3071178.3071320(457-464)Online publication date: 1-Jul-2017
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media