Skip to main content

Advertisement

SpringerLink
Log in

A hybrid evolutionary algorithm for the symbolic modeling of multiple-time-scale dynamical systems

  • Research Paper
  • Published:
Evolutionary Intelligence Aims and scope Submit manuscript

Abstract

Natural and artificial dynamical systems in the real world often have dynamics at multiple time scales. Such dynamics can contribute substantially to the complexity of a dynamical system and increase the difficulty with which it can be analyzed. Although evolutionary algorithms have been proposed that are amenable to the automated modeling of dynamical systems, none have explicitly taken into account multiple time scales or leveraged the information about these dynamics that is inherent in experimental observations. We propose a hybrid approach to the design of models for multiple-time-scale dynamical systems that combines an evolutionary algorithm with other metaheuristics and conventional nonlinear regression. With only minimal human-supplied domain knowledge, the algorithm automates the process of analyzing raw experimental observations and creating an interpretable symbolic model of the system under study. We describe the algorithm in detail and demonstrate its applicability to a variety of both physical and simulated systems. In addition, we study the performance and scalability of the algorithm under different types of dynamics, varying levels of experimental noise, and other factors relevant to the practical application of the algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Akansu AN, Haddad PR (2000) Multiresolution signal decomposition: transforms, subbands, and wavelets. Academic Press, Waltham

    Google Scholar 

  2. Argemí A, Saurina J (2007) Study of the degradation of 5-azacytidine as a model of unstable drugs using a stopped-flow method and further data analysis with multivariate curve resolution. Talanta 74:176–182

    Article  Google Scholar 

  3. Beisler JA (1978) Isolation, characterization, and properties of a labile hydrolysis product of the antitumor nucleoside, 5-azacytidine. J Med Chem 21:204–208

    Article  Google Scholar 

  4. Bishop CM (2006) Pattern recognition and machine learning. Springer, London

    MATH  Google Scholar 

  5. Bongard JC, Lipson H (2007) Automated reverse engineering of nonlinear dynamical systems. Proc Natl Acad Sci USA 104:9943–9948

    Article  MATH  Google Scholar 

  6. Borcard D, Legendre P, Avois-Jacquet C, Tuomisto H (2004) Dissecting the spatial structure of ecological data at multiple scales. Ecology 85:1826–1832

    Article  Google Scholar 

  7. Brameier MF, Banzhaf W (2007) Linear genetic programming. Springer, New York

    MATH  Google Scholar 

  8. Chen W, Fish J (2001) A dispersive model for wave propagation in periodic heterogeneous media based on homogenization with multiple spatial and temporal scales. J Appl Mech 68:153–161

    Article  MATH  Google Scholar 

  9. Chen WW, Niepel M, Sorger PK (2010) Classic and contemporary approaches to modeling biochemical reactions. Genes Dev 24:1861–1875

    Article  Google Scholar 

  10. Chung J, Jung D, Yoo HH (2004) Stability analysis for the flapwise motion of a cantilever beam with rotary oscillation. J Sound Vib 273:1047–1062

    Article  Google Scholar 

  11. Clery D, Voss D (2005) All for one and one for all. Science 308:809

    Article  Google Scholar 

  12. Cornforth TW, Lipson H (2012) In: proceedings of the genetic and evolutionary computation conference. Symbolic regression of multiple-time-scale dynamical systems, Philadelphia, USA, 2012, pp 735–742

  13. DeJong KA (2006) Evolutionary computation: a unified approach. MIT Press, Cambridge

    Google Scholar 

  14. Deng L, Trepat X, Butler JP, Millet E, Morgan KG, Weitz DA, Fredberg JJ (2006) Fast and slow dynamics of the cytoskeleton. Nat Mater 5:636–640

    Article  Google Scholar 

  15. Draper NR, Smith H (1998) Applied regression analysis. Wiley, New York

    Book  MATH  Google Scholar 

  16. Evans J, Rzhetsky A (2010) Machine science. Science 329:399–400

    Article  Google Scholar 

  17. Guven A (2009) Linear genetic programming for time-series modeling of daily flow rate. J Earth Syst Sci 118:137–146

    Article  Google Scholar 

  18. Harris-Warrick RM, Flamm RE (1987) Multiple mechanisms of bursting in a conditional bursting neuron. J Neurosci 7:2113–2128

    Google Scholar 

  19. Henzler-Wildman KA, Lei M, Thai V, Kerns SJ, Karplus M, Kern D (2007) A hierarchy of timescales in protein dynamics is linked to enzyme catalysis. Nature 450:913–916

    Article  Google Scholar 

  20. Hornby GS (2006) In: proceedings of the genetic and evolutionary computation conference. ALPS: the age-layered population structure for reducing the problem of premature convergence, New York, 206, pp 815–822

  21. Iba H, de Garis H, Sato T (1994), In Davidor Y, Schwefel H-P, Männer R (eds) proceedings of the PPSN III. Genetic programming with local hill-climbing, Springer, London, 1994, pp 302–311

  22. Jara EC (2011) Long memory time series forecasting by using genetic programming. Genet Program Evol Mach 12:429–456

    Article  Google Scholar 

  23. Jones CKRT (2001) Multiple-time-scale dynamical systems. Springer, New York

    Book  MATH  Google Scholar 

  24. Julier S, Uhlmann J, Durrant-Whyte HF (2000) A new method for the nonlinear transformation of means and covariances in filters and estimators. IEEE Trans Automat Control 45:477–482

    Article  MATH  MathSciNet  Google Scholar 

  25. King RD, Whelan KE, Jones FM, Reiser PGK, Bryant CH, Muggleton SH, Kell DB, Oliver SG (2004) Functional genomic hypothesis generation and experimentation by a robot scientist. Nature 427:247–252

    Article  Google Scholar 

  26. King RD, Rowland J, Oliver SG, Young M, Aubrey W, Byrne E, Liakata M, Markham M, Pir P, Soldatova LN, Sparkes A, Whelan KE, Clare A (2009) The automation of science. Science 324:85–89

    Article  Google Scholar 

  27. Klein R (2001) Asymptotic adaptive methods for multi-scale problems in fluid mechanics. J Eng Math 39:261–343

    Article  MATH  Google Scholar 

  28. Koza JR (1992) Genetic programming: on the programming of computers by means of natural selection. MIT Press, Cambridge

    MATH  Google Scholar 

  29. Koza JR, Bennett FH, Andre D, Keane MA (1999) Genetic programming III: Darwinian invention and problem solving. Morgan Kaufmann, San Francisco

    MATH  Google Scholar 

  30. Langley P, Simon HA, Bradshaw G, Zytkow JM (1987) Scientific discovery: computational exploration of the creative processes. MIT Press, Cambridge

    Google Scholar 

  31. Lawlor FD (1973) MFIT: a multiple nonlinear curve fitting technique. IBM Poughkeepsie laboratory technical report TR 00.2482

  32. Le Pourhiet A, Corrège M, Caruana D (2003) Control of self-oscillating systems. IEEE Proc Control Theory Appl 150:599–610

    Article  Google Scholar 

  33. Lee CH, Othmer HG (2009) A multi-time-scale analysis of chemical reaction networks: I Deterministic systems. J Math Biol 60:387–450

    Article  MathSciNet  Google Scholar 

  34. Mahfoud SW (1992), In: Männer R, Manderick R (eds) parallel problem solving from nature—PPSN-II. Crowding and preselection revisited, Elsevier

  35. Mulloy BS, Riolo RL, Savit RS (1996) In: Genetic programming 1996: proceedings of the first annual conference. Dynamics of genetic programming and chaotic time series prediction, pp 166–174

  36. Nocedal J, Wright SJ (2006) Numerical optimization. Springer, New York

    MATH  Google Scholar 

  37. Palsson BO (1987) On the dynamics of the irreversible Michaelis-Menten reaction mechanism. Chem Eng Sci 42:447–458

    Article  Google Scholar 

  38. Pandolfi JM (2002) Coral community dynamics at multiple scales. Coral Reefs 21:13–23

    Article  Google Scholar 

  39. Russell S, Norvig P (2009) Artificial intelligence: a modern approach, 3rd edn. Prentice Hall, Upper Saddle River

    Google Scholar 

  40. Sakamoto E, Iba H (2001) In: proceedings of the 2001 congress on evolutionary computation. Inferring a system of differential equations for a gene regulatory network by using genetic programming. pp 720–726

  41. Schmidt MD, Lipson H (2007) In: proceedings of the genetic and evolutionary computation conference 2007. Comparison of tree and graph encodings as a function of problem complexity. pp 1674–1679

  42. Schmidt MD, Lipson H (2008) Coevolution of fitness predictors. IEEE Trans Evol Comp. 12:736–749

    Article  Google Scholar 

  43. Schmidt MD, Lipson H (2009) Distilling free-form natural laws from experimental data. Science 324:81–85

    Article  Google Scholar 

  44. Schmidt MD, Lipson H (2010) In: proceedings of the genetic and evolutionary computation conference 2010. Predicting solution rank to improve performance. pp 949–956

  45. Schmidt MD, Lipson H (2010) Age-fitness Pareto optimization. Genet Program Theory Pract 8:129–146

    Google Scholar 

  46. Schmidt MD, Vallabhajosyula RR, Jenkins JW, Hood JE, Soni AS, Wikswo JP, Lipson H (2011) Automated refinement and inference of analytical models for metabolic networks. Phys Biol 8:055011

    Article  Google Scholar 

  47. Strogatz SH (2001) Exploring complex networks. Nature 410:268–276

    Article  Google Scholar 

  48. Szalay A, Gray J (2006) 2020 Computing: science in an exponential world. Nature 440:413–414

    Article  Google Scholar 

  49. Verhulst F (2005) Methods and applications of singular perturbations: boundary layers and multiple timescale dynamics. Springer, New York

    Book  Google Scholar 

  50. Wagner N, Michalewicz Z, Khouja M, McGregor RR (2007) Time series forecasting for dynamic environments: the DyFor genetic program model. IEEE Trans Evolut Comput 11:433–452

    Article  Google Scholar 

  51. Waltz D, Buchanan BG (2009) Automating science. Science 324:43–44

    Article  Google Scholar 

Download references

Acknowledgments

Support was provided by the Tri-Institutional Training Program in Computational Biology and Medicine, US National Science Foundation grant ECCS 0941561 on Cyber-enabled Discovery and Innovation (CDI), US National Institutes of Health NIDA grant RC2 DA028981, and the US Defense Threat Reduction Agency (DTRA) grant HDTRA 1-09-1-0013.

Author information

Authors and Affiliations

  1. Biological Statistics and Computational Biology, Cornell University, 1198 Comstock Hall, Ithaca, NY, 14853-2601, USA

    Theodore W. Cornforth

  2. Mechanical and Aerospace Engineering, Cornell University, 239 Upson Hall, Ithaca, NY, 14853-7501, USA

    Hod Lipson

Authors
  1. Theodore W. Cornforth
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hod Lipson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Theodore W. Cornforth.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cornforth, T.W., Lipson, H. A hybrid evolutionary algorithm for the symbolic modeling of multiple-time-scale dynamical systems. Evol. Intel. 8, 149–164 (2015). https://doi.org/10.1007/s12065-015-0126-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12065-015-0126-x

Keywords

Search

Navigation