skip to main content
10.1145/1276958.1277289acmconferencesArticle/Chapter ViewAbstractPublication PagesgeccoConference Proceedingsconference-collections
Article

Learning noise

Published: 07 July 2007 Publication History

Abstract

In this paper we propose a genetic programming approach to learning stochastic models with unsymmetrical noise distributions. Most learning algorithms try to learn from noisy data by modeling the maximum likelihood output or least squared error, assuming that noise effects average out. While this process works well for data with symmetrical noise distributions (such as Gaussian observation noise), many real-life sources of noise are not symmetrically distributed, thus this approach does not hold. We suggest improved learning can be obtained by including noise sources explicitly in the model as a stochastic element. A stochastic element is a random sub-process or latent variable of a hidden system that can propagate nonlinear noise to the observable outputs. Stochastic elements can skew and distort output features making regression of analytical models particularly difficult and error minimizing approaches inhibiting. We introduce a new method to infer the analytical model of a system by decomposing non-uniform noise observed at the outputs into uniform stochastic elements appearing symbolically inside the system. Results demonstrate the ability to regress exact analytical models where stochastic elements are embedded inside nonlinear and polynomial hidden systems.

References

[1]
Schmidt, M., and Lipson, H. "Coevolution of Fitness Maximizers and Fitness Predictors", GECCO Late Breaking Paper, 2005.
[2]
Rasmussen C. 1996. Evaluation of Gaussian Processes and Other Methods for Non--Linear Regression. PhD thesis, Department of Computer Science, University of Toronto.
[3]
Llorà, X., Goldberg, D.E. "Bounding the effect of noise in Multiobjective Learning Classifier Systems." Evolutionary Computation (2003) In press.
[4]
V.G. Kulkarni, Modeling and Analysis of Stochastic Systems. Chapman Hall, 1995.
[5]
Schafer, WM, Ellner S, Kot M. "Effects of noise on some dynamical models in ecology. J Math Biology 1986;24:479--523.
[6]
Theiler, J., Galdrikian, B., Longtin, A., Eubank, S. and farmer, J.D. (1991) "Detecting non--linear structure in time series." First Experimental Chaos Conference, Arlington, VA
[7]
Koza, J.R. Genetic Programming: On the Programming of Computers by Means of Natural Selection. Cambridge, MA: The MIT Press, 1992.
[8]
Augusto D. A. and Barbosa H. J. C. "Symbolic Regression via Genetic Programming," VI Brazilian Symposium on Neural Networks (SBRN'00), 01: 22--01, 2000.
[9]
Kleijnen, Jack P.C., "White Noise Assumptions Revisited: Regression Models and Statistical Designs for Simulation Practice" CentER Discussion Paper No. 2006--50.
[10]
Rousseeuw, P.J. and Leroy, A.M. "Robust Regression and Outlier Detection." New York: Wiley, 1987.
[11]
Nix, D.A., and A.S. Weigend (1995). "Learning Local Error Bars for Nonlinear Regression." In Advances in Neural Information Processing Systems 7 (NIPS*94) p. 489--496. Cambridge, MA: MIT Press.
[12]
Vugrin, K.W., Swiler, L.P., Roberts, R.M., et al. "Confidence Region Estimation Techniques for Nonlinear Regression: Three Case Studies." SAND Report, October 2005.
[13]
Arnold D., "Evolution strategies in noisy environments - A survey of existing work," in Theoretical Aspects of Evolutionary Computing, L. Kallel, B. Naudts, and A. Rogers, Eds., Heidelberg, Germany: Springer Verlag, 2001, pp. 239--24.

Cited By

View all
  • (2024)Tailor-Made Reinforcement Learning Approach With Advanced Noise Optimization for Soft Continuum RobotsIEEE Transactions on Artificial Intelligence10.1109/TAI.2024.34402255:11(5509-5518)Online publication date: Nov-2024
  • (2023)Automated learning of interpretable models with quantified uncertaintyComputer Methods in Applied Mechanics and Engineering10.1016/j.cma.2022.115732403(115732)Online publication date: Jan-2023
  • (2023)Machine learning prediction model for the axial strength of longitudinal branch plate-to-CHS T-connectionsAin Shams Engineering Journal10.1016/j.asej.2023.10255714:12(102557)Online publication date: Dec-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
GECCO '07: Proceedings of the 9th annual conference on Genetic and evolutionary computation
July 2007
2313 pages
ISBN:9781595936974
DOI:10.1145/1276958
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2007

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. dynamical systems
  2. stochastic elements
  3. symbolic regression

Qualifiers

  • Article

Conference

GECCO07
Sponsor:

Acceptance Rates

GECCO '07 Paper Acceptance Rate 266 of 577 submissions, 46%;
Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)27
  • Downloads (Last 6 weeks)1
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Tailor-Made Reinforcement Learning Approach With Advanced Noise Optimization for Soft Continuum RobotsIEEE Transactions on Artificial Intelligence10.1109/TAI.2024.34402255:11(5509-5518)Online publication date: Nov-2024
  • (2023)Automated learning of interpretable models with quantified uncertaintyComputer Methods in Applied Mechanics and Engineering10.1016/j.cma.2022.115732403(115732)Online publication date: Jan-2023
  • (2023)Machine learning prediction model for the axial strength of longitudinal branch plate-to-CHS T-connectionsAin Shams Engineering Journal10.1016/j.asej.2023.10255714:12(102557)Online publication date: Dec-2023
  • (2022)Value-free random exploration is linked to impulsivityNature Communications10.1038/s41467-022-31918-913:1Online publication date: 4-Aug-2022
  • (2022)Artificial intelligence-based modeling of extruded aluminum beams subjected to patch loadingThin-Walled Structures10.1016/j.tws.2022.109673179(109673)Online publication date: Oct-2022
  • (2021)Toward Data‐Driven Generation and Evaluation of Model Structure for Integrated Representations of Human Behavior in Water Resources SystemsWater Resources Research10.1029/2020WR02814857:2Online publication date: 12-Feb-2021
  • (2021)Machine learning approach for predicting the patch load resistance of slender austenitic stainless steel girdersStructures10.1016/j.istruc.2021.01.01230(198-205)Online publication date: Apr-2021
  • (2021)Automated adaptation strategies for stream learningMachine Learning10.1007/s10994-021-05992-xOnline publication date: 2-Jun-2021
  • (2016)Augmenting adaptation with retrospective model correction for non-stationary regression problems2016 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN.2016.7727278(771-779)Online publication date: Jul-2016
  • (2009)Mutation and crossover with abstract expression grammarsProceedings of the 11th Annual Conference Companion on Genetic and Evolutionary Computation Conference: Late Breaking Papers10.1145/1570256.1570308(2219-2224)Online publication date: 8-Jul-2009
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media