Skip to main content
Springer Nature Link
Log in

DiffFind: Discovering Differential Equations from Time Series

  • Conference paper
  • First Online:
Advances in Knowledge Discovery and Data Mining (PAKDD 2024)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14650))

Included in the following conference series:

  • 712 Accesses

Abstract

Given one or more time sequences, how can we extract their governing equations? Single and co-evolving time sequences appear in numerous settings, including medicine (neuroscience - EEG signals, cardiology - EKG), epidemiology (covid/flu spreading over time), physics (astrophysics, material science), marketing (sales and competition modeling; market penetration), and numerous more. Linear differential equations will fail, since the underlying equations are often non-linear (SIR model for virus/product spread; Lotka-Volterra for product/species competition, Van der Pol for heartbeat modeling).

We propose DiffFind and we use genetic algorithms to find suitable, parsimonious, differential equations. Thanks to our careful design decisions, DiffFind has the following properties - it is: (a) Effective, discovering the correct model when applied on real and synthetic nonlinear dynamical systems, (b) Explainable, gives succinct differential equations, and (c) Hands-off, requiring no manual hyperparameter specification.

DiffFind outperforms traditional methods (like auto-regression), includes as special case and thus outperforms a recent baseline (‘SINDy’), and wins first or second place for all 5 real and synthetic datasets we tried, often achieving excellent, zero or near-zero RMSE of 0.005.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Identical mutations are assigned the same identifier.

References

  1. Anisiu, M.C.: Lotka, Volterra and their model. Didáctica Mathematica 32, 9–17 (2014)

    Google Scholar 

  2. Bongard, J., Lipson, H.: Automated reverse engineering of nonlinear dynamical systems. PNAS 104(24), 9943–9948 (2007)

    Article  Google Scholar 

  3. Box, G.E., Jenkins, G.M., MacGregor, J.F.: Some recent advances in forecasting and control. J. Roy. Stat. Soc.: Ser. C (Appl. Stat.) 23(2), 158–179 (1974)

    MathSciNet  Google Scholar 

  4. Brauer, F., Castillo-Chavez, C., Castillo-Chavez, C.: Mathematical Models in Population Biology and Epidemiology, vol. 2. Springer, Cham (2012). https://doi.org/10.1007/978-1-4614-1686-9

  5. Brown, R.G.: Statistical forecasting for inventory control (1959)

    Google Scholar 

  6. Brunton, S.L., Proctor, J.L., Kutz, J.N.: Discovering governing equations from data by sparse identification of nonlinear dynamical systems. PNAS 113(15), 3932–3937 (2016)

    Article  MathSciNet  Google Scholar 

  7. Chai, T., Draxler, R.R.: Root Mean Square Error (RMSE) or Mean Absolute Error (MAE)?-arguments against avoiding RMSE in the literature. Geosci. Model Dev. 7(3), 1247–1250 (2014)

    Article  Google Scholar 

  8. Chatfield, C.: Time-Series Forecasting. Chapman and Hall/CRC, Boca Raton (2000)

    Google Scholar 

  9. Chen, R.T., Rubanova, Y., Bettencourt, J., Duvenaud, D.K.: Neural ordinary differential equations. In: NeurIPS, vol. 31 (2018)

    Google Scholar 

  10. Cranmer, M., et al.: Discovering symbolic models from deep learning with inductive biases. In: NeurIPS, vol. 33, pp. 17429–17442 (2020)

    Google Scholar 

  11. Ding, S., Li, H., Su, C., Yu, J., Jin, F.: Evolutionary artificial neural networks: a review. Artif. Intell. Rev. 39(3), 251–260 (2013)

    Article  Google Scholar 

  12. Elsken, T., Metzen, J.H., Hutter, F.: Neural architecture search: a survey. JMLR 20(1), 1997–2017 (2019)

    MathSciNet  Google Scholar 

  13. FitzHugh, R.: Impulses and physiological states in theoretical models of nerve membrane. Biophys. J . 1(6), 445–466 (1961)

    Article  Google Scholar 

  14. Kramer, O., Kramer, O.: Genetic Algorithms. Springer, Cham (2017)

    Google Scholar 

  15. Lim, B., Zohren, S.: Time-series forecasting with deep learning: a survey. Phil. Trans. R. Soc. A 379(2194), 20200209 (2021)

    Article  MathSciNet  Google Scholar 

  16. Lorenz, E.N.: Deterministic nonperiodic flow. J. Atmos. Sci. 20(2), 130–141 (1963)

    Article  MathSciNet  Google Scholar 

  17. Luo, Y., Xu, C., Liu, Y., Liu, W., Zheng, S., Bian, J.: Learning differential operators for interpretable time series modeling. In: ACM SIGKDD, pp. 1192–1201 (2022)

    Google Scholar 

  18. Makridakis, S., Hibon, M.: ARMA models and the Box-Jenkins methodology. J. Forecast. 16(3), 147–163 (1997)

    Article  Google Scholar 

  19. Park, N., Kim, M., Hoai, N.X., McKay, R.B., Kim, D.K.: Knowledge-based dynamic systems modeling: a case study on modeling river water quality. In: ICDE, pp. 2231–2236. IEEE (2021)

    Google Scholar 

  20. Van der Pol, B.: LXXXVIII. On “relaxation-oscillations”. The London, Edinburgh, and Dublin Philos. Mag. J. Sci. 2(11), 978–992 (1926)

    Google Scholar 

  21. Raissi, M., Perdikaris, P., Karniadakis, G.E.: Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686–707 (2019)

    Article  MathSciNet  Google Scholar 

  22. Sahoo, S., Lampert, C., Martius, G.: Learning equations for extrapolation and control. In: ICML, pp. 4442–4450. PMLR (2018)

    Google Scholar 

  23. Salinas, D., Flunkert, V., Gasthaus, J., Januschowski, T.: DeepAR: probabilistic forecasting with autoregressive recurrent networks. Int. J. Forecast. 36(3), 1181–1191 (2020)

    Article  Google Scholar 

  24. Schmidt, M., Lipson, H.: Distilling free-form natural laws from experimental data. Science 324(5923), 81–85 (2009)

    Google Scholar 

  25. Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evol. Comput. 10(2), 99–127 (2002)

    Article  Google Scholar 

  26. Tealab, A.: Time series forecasting using artificial neural networks methodologies: a systematic review. Future Comput. Inform. J. 3(2), 334–340 (2018)

    Article  Google Scholar 

  27. Torres, J.F., Hadjout, D., Sebaa, A., Martínez-Álvarez, F., Troncoso, A.: Deep learning for time series forecasting: a survey. Big Data 9(1), 3–21 (2021)

    Article  Google Scholar 

  28. Wang, R., Maddix, D., Faloutsos, C., Wang, Y., Yu, R.: Bridging physics-based and data-driven modeling for learning dynamical systems. In: Learning for Dynamics and Control, pp. 385–398. PMLR (2021)

    Google Scholar 

  29. Wang, R., Robinson, D., Faloutsos, C., Wang, Y.B., Yu, R.: AutoODE: bridging physics-based and data-driven modeling for COVID-19 forecasting. In: NeurIPS 2020 Workshop on Machine Learning in Public Health (2020)

    Google Scholar 

  30. Zhou, X., Qin, A.K., Gong, M., Tan, K.C.: A survey on evolutionary construction of deep neural networks. IEEE Trans. Evol. Comput. 25(5), 894–912 (2021)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of California, Berkeley, USA

    Lalithsai Posam

  2. Brandeis University, Waltham, USA

    Shubhranshu Shekhar

  3. Carnegie Mellon University, Pittsburgh, USA

    Meng-Chieh Lee & Christos Faloutsos

Authors
  1. Lalithsai Posam
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shubhranshu Shekhar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Meng-Chieh Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Christos Faloutsos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shubhranshu Shekhar .

Editor information

Editors and Affiliations

  1. Taipei, Taiwan

    De-Nian Yang

  2. Microsoft Research Asia, Beijing, China

    Xing Xie

  3. National Yang Ming Chiao Tung University, Hsinchu, Taiwan

    Vincent S. Tseng

  4. Duke University, Durham, NC, USA

    Jian Pei

  5. National Cheng Kung University, Tainan, Taiwan

    Jen-Wei Huang

  6. Silesian University of Technology, Gliwice, Poland

    Jerry Chun-Wei Lin

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Posam, L., Shekhar, S., Lee, MC., Faloutsos, C. (2024). DiffFind: Discovering Differential Equations from Time Series. In: Yang, DN., Xie, X., Tseng, V.S., Pei, J., Huang, JW., Lin, J.CW. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2024. Lecture Notes in Computer Science(), vol 14650. Springer, Singapore. https://doi.org/10.1007/978-981-97-2266-2_14

Download citation

  • DOI: https://doi.org/10.1007/978-981-97-2266-2_14

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-97-2265-5

  • Online ISBN: 978-981-97-2266-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation