Skip to main content
SpringerLink
Log in

Physics-guided machine learning from simulated data with different physical parameters

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

Physics-based models are widely used to study dynamical systems in a variety of scientific and engineering problems. However, these models are necessarily approximations of reality due to incomplete knowledge or excessive complexity in modeling underlying processes. As a result, they often produce biased simulations due to inaccurate parameterizations or approximations used to represent the true physics. In this paper, we aim to build a new physics-guided machine learning framework to monitor dynamical systems. The idea is to use advanced machine learning model to extract complex spatio-temporal data patterns while also incorporating general scientific knowledge embodied in simulated data generated by the physics-based model. To handle the bias in simulated data caused by imperfect parameterization, we propose to extract general physical relations jointly from multiple sets of simulations generated by a physics-based model under different physical parameters. In particular, we develop a spatio-temporal network architecture that uses its gating variables to capture the variation of physical parameters. We initialize this model using a pre-training strategy that helps discover common physical patterns shared by different sets of simulated data. Then, we fine-tune it combining limited observations and adequate simulations. By leveraging the complementary strength of machine learning and domain knowledge, our method has been shown to produce accurate predictions, use less training samples and generalize to out-of-sample scenarios. We further show that the method can provide insights about the variation of physical parameters over space and time in two domain applications: predicting temperature in streams and predicting temperature in lakes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. https://drive.google.com/open?id=12l9RhiaGZqwZEp3URFY8GrQ4VAMpRtvy.

References

  1. Hipsey MR et al (2019) A general lake model (GLM 3.0) for linking with high-frequency sensor data from the global lake ecological observatory network (GLEON). Geosci Model Dev 12:473–523

    Article  Google Scholar 

  2. Cox P et al (1999) The impact of new land surface physics on the GCM simulation of climate and climate sensitivity. Clim Dyn 15:183–203

    Article  Google Scholar 

  3. Tonks MR et al (2017) Mechanistic materials modeling for nuclear fuel performance. Ann Nuclear Energy 105:11–24

    Article  Google Scholar 

  4. Gupta HV, Nearing GS (2014) Debates-The future of hydrological sciences: a (common) path forward? Using models and data to learn: a systems theoretic perspective on the future of hydrological science. Water Resour Res 50:5351–5359

    Article  Google Scholar 

  5. Lall U (2014) Debates-The future of hydrological sciences: a (common) path forward? One water. One world. Many climes. Many souls Water Resour Res 50:5335–5341

    Article  Google Scholar 

  6. McDonnell JJ, Beven K (2014) McDonnell JJ, Beven K (2014) Debates-The future of hydrological sciences: a (common) path forward? A call to action aimed at understanding velocities, celerities, and residence time distributions of the headwater hydrograph. Water Resour Res 50:5342–5350

    Article  Google Scholar 

  7. Voulodimos A, Doulamis N, Doulamis A, Protopapadakis E (2018) Deep learning for computer vision: a brief review. Comput Intell Neurosci 2018:7068349. https://doi.org/10.1155/2018/7068349

  8. Zhang L, Han J, Wang H, Car R, Weinan E (2018) Deep potential molecular dynamics: a scalable model with the accuracy of quantum mechanics. Phys Rev Lett 120(14):143001

    Article  Google Scholar 

  9. Otter DW, Medina JR, Kalita JK (2020) A survey of the usages of deep learning for natural language processing. IEEE Trans Neural Netw Learn Syst 32:604–624

    Article  MathSciNet  Google Scholar 

  10. Karpatne A, Atluri G, Faghmous JH, Steinbach M, Banerjee A, Ganguly A, Shekhar S, Samatova N, Kumar V (2017) Theory-guided data science: a new paradigm for scientific discovery from data. IEEE Trans Knowl Data Eng 29(10):2318–2331

    Article  Google Scholar 

  11. Willard J, Jia X, Xu S, Steinbach M, Kumar V (2021) Integrating scientific knowledge with machine learning for engineering and environmental systems. arXiv preprint arXiv:2003.04919

  12. Xu T et al (2015) Data-driven methods to improve baseflow prediction of a regional groundwater model. Comput Geosci 85:124–136

    Article  Google Scholar 

  13. Reichstein M, Camps-Valls G, Stevens B, Jung M, Denzler J, Carvalhais N (2019) Deep learning and process understanding for data-driven earth system science. Nature 566(7743):195–204

    Article  Google Scholar 

  14. Krasnopolsky VM, Fox-Rabinovitz MS (2006) Complex hybrid models combining deterministic and machine learning components for numerical climate modeling and weather prediction. Neural Netw 19(2):122–134

    Article  Google Scholar 

  15. Faghmous JH, Kumar V (2014) A big data guide to understanding climate change: the case for theory-guided data science. Big Data 2(3):155–163

    Article  Google Scholar 

  16. O’Gorman PA, Dwyer JG (2018) Using machine learning to parameterize moist convection: potential for modeling of climate, climate change, and extreme events. J Adv Model Earth Syst 10(10):2548–2563

    Article  Google Scholar 

  17. Graham-Rowe D, Goldston D, Doctorow C, Waldrop M, Lynch C, Frankel F, Reid R, Nelson S, Howe D, Rhee S (2008) Big data: science in the petabyte era. Nature 455(7209):8–9

    Google Scholar 

  18. Goh GB, Hodas NO, Vishnu A (2017) Deep learning for computational chemistry. J Comput Chem 38(16):1291–1307

    Article  Google Scholar 

  19. Forssell U, Lindskog P (1997) Combining semi-physical and neural network modeling: An example of its usefulness. In: IFAC

  20. Wan ZY et al (2018) Data-assisted reduced-order modeling of extreme events in complex dynamical systems. PloS One 13:e0197704

    Article  Google Scholar 

  21. Jia X, Willard J, Karpatne A, Read JS, Zwart JA, Steinbach M, Kumar V (2021) Physics-guided machine learning for scientific discovery: an application in simulating lake temperature profiles. ACM/IMS Trans Data Sci 2:1–26

    Article  Google Scholar 

  22. Fioretto F (2020) Predicting ac optimal power flows: Combining deep learning and lagrangian dual methods. In: AAAI

  23. Karpatne A, Watkins W, Read J, Kumar V (2017) Physics-guided neural networks (pgnn): An application in lake temperature modeling. arXiv preprint arXiv:1710.11431

  24. Read JS, Jia X, Willard J, Appling AP, Zwart JA, Oliver SK, Karpatne A, Hansen GJ, Hanson PC, Watkins W et al (2019) Process-guided deep learning predictions of lake water temperature. Water Resour Res 55(2019):9173–9190

    Article  Google Scholar 

  25. Stewart R, Ermon S (2017) Label-free supervision of neural networks with physics and domain knowledge. In: AAAI

  26. Hanson PC, Stillman AB, Jia X, Karpatne A, Dugan HA, Carey CC, Stachelek J, Ward NK, Zhang Y, Read JS et al (2020) Predicting lake surface water phosphorus dynamics using process-guided machine learning. Ecol Model 430:109136

    Article  Google Scholar 

  27. Chen S, Sammak S, Givi P, Yurko JP, Jia X (2021) Reconstructing high-resolution turbulent flows using physics-guided neural networks. In: 2021 IEEE international conference on big data (Big Data), IEEE, pp 1369–1379

  28. Hurtado DM et al (2018) Deep transfer learning in the assessment of the quality of protein models. arXiv preprint arXiv:1804.06281

  29. Sultan MM, Wayment-Steele HK, Pande VS (2018) Transferable neural networks for enhanced sampling of protein dynamics. J Chem Theory Comput 14:1887–1894

    Article  Google Scholar 

  30. Daw A, Thomas R, Carey C, Read J, Appling A, Karpatne A (2019) Physics-guided architecture (pga) of neural networks for quantifying uncertainty in lake temperature modeling. arXiv:1911.02682

  31. Muralidhar N (2020) Phynet: Physics guided neural networks for particle drag force prediction in assembly. In: SDM

  32. Ling J, Kurzawski A, Templeton J (2016) Reynolds averaged turbulence modelling using deep neural networks with embedded invariance. J Fluid Mech 807:155–166

    Article  MathSciNet  MATH  Google Scholar 

  33. Zhang L et al (2018) End-to-end symmetry preserving inter-atomic potential energy model for finite and extended systems. In: NeurIPS

  34. Schütt KT (2017) et al.: Schnet: A continuous-filter convolutional neural network for modeling quantum interactions. In: NeurIPS

  35. Bao T, Jia X, Zwart J, Sadler J, Appling A, Oliver S, Johnson TT (2021) Partial differential equation driven dynamic graph networks for predicting stream water temperature. In: 2021 IEEE international conference on data mining (ICDM), IEEE, pp 11–20

  36. Bao T, Chen S, Johnson TT, Givi P, Sammak S, Jia X (2022) Physics guided neural networks for spatio-temporal super-resolution of turbulent flows. In: Uncertainty in artificial intelligence, PMLR, pp 118–128

  37. Jia X, Willard J, Karpatne A, Read J, Zwart J, Steinbach M, Kumar V (2019) Physics guided rnns for modeling dynamical systems: a case study in simulating lake temperature profiles. In: SDM

  38. Data release: Process-based predictions of lake water temperature in the Midwest US. https://www.usgs.gov/data/data-release-process-based-predictions-lake-water-temperature-midwest-us

  39. Predicting water temperature in the Delaware River Basin. https://www.usgs.gov/data/predicting-water-temperature-delaware-river-basin

  40. Data release: a large-scale database of modeled contemporary and future water temperature data for 10,774 Michigan, Minnesota and Wisconsin Lakes. https://www.usgs.gov/data/data-release-a-large-scale-database-modeled-contemporary-and-future-water-temperature-data

  41. Anderson B, Hy TS, Kondor R (2019) Cormorant: Covariant molecular neural networks. In: NeurIPS

  42. Muralidhar N (2018) Incorporating prior domain knowledge into deep neural networks. In: IEEE big data. IEEE

  43. Ham Y-G, Kim J-H, Luo J-J (2019) Deep learning for multi-year ENSO forecasts. Nature 573(2019):568–572

    Article  Google Scholar 

  44. Lu J, Yao K, Gao F (2009) Process similarity and developing new process models through migration. AIChE J 55(2009):2318–2328

    Article  Google Scholar 

  45. Jia X, Zwart J, Sadler J, Appling A, Oliver S, Markstrom S, Willard J, Xu S, Steinbach M, Read J (2021) Physics-guided recurrent graph model for predicting flow and temperature in river networks. In: SDM. SIAM

  46. Kratzert F, Klotz D, Shalev G, Klambauer G, Hochreiter S, Nearing G (2019) Towards learning universal, regional, and local hydrological behaviors via machine learning applied to large-sample datasets. Hydrol Earth Syst Sci 23(12):5089–5110

    Article  Google Scholar 

  47. Lu J et al (2008) Model migration with inclusive similarity for development of a new process model. Ind Eng Chem Res 47(2008):9508–9516

    Article  Google Scholar 

  48. Li Y, Gao J, Meng C, Li Q, Su L, Zhao B, Fan W, Han J (2016) A survey on truth discovery. ACM Sigkdd Explor Newsl 17:1–16

    Article  Google Scholar 

  49. Van Engelen JE, Hoos HH (2020) A survey on semi-supervised learning. Mach Learn 109(2):373–440

    Article  MathSciNet  MATH  Google Scholar 

  50. Ouali Y, Hudelot C, Tami M (2020) An overview of deep semi-supervised learning. arXiv preprint arXiv:2006.05278

  51. Weiss K, Khoshgoftaar TM, Wang D (2016) A survey of transfer learning. J Big Data 3(1):1–40

    Article  Google Scholar 

  52. Wilson G, Cook DJ (2020) A survey of unsupervised deep domain adaptation. ACM Trans Intell Syst Technol (TIST) 11(5):1–46

    Article  Google Scholar 

  53. Ngo BH, Park JH, Park SJ, Cho SI (2021) Semi-supervised domain adaptation using explicit class-wise matching for domain-invariant and class-discriminative feature learning. IEEE Access 9:128467–128480

    Article  Google Scholar 

  54. Xu L, Hu C, Mei K (2022) Semi-supervised regression with manifold: a Bayesian deep kernel learning approach. Neurocomputing 497:76–85

    Article  Google Scholar 

  55. Jaiswal A, Babu AR, Zadeh MZ, Banerjee D, Makedon F (2020) A survey on contrastive self-supervised learning. Technologies 9(1):2

    Article  Google Scholar 

  56. Larsson G, Maire M, Shakhnarovich G (2017) Colorization as a proxy task for visual understanding. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6874–6883

  57. Appice A, Loglisci C, Malerba D (2018) Active learning via collective inference in network regression problems. Inf Sci 460:293–317

    Article  MathSciNet  MATH  Google Scholar 

  58. Markstrom SL et al. (2015) PRMS-IV, the precipitation-runoff modeling system, version 4. USGS Techniques and Methods (6-B7)

  59. Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In: International conference on machine learning, PMLR, pp 1126–1135

  60. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems, vol 30

  61. Luong M-T, Pham H, Manning CD (2015) Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025

  62. Geological Survey US (2016) National water information system data available on the world wide web (USGS water data for the nation)

  63. Read EK, Carr L, De Cicco L, Dugan HA, Hanson PC, Hart JA, Kreft J, Read JS, Winslow LA (2017) Water quality data for national-scale aquatic research: the water quality portal. Water Resour Res 53(2):1735–1745

    Article  Google Scholar 

  64. Regan RS, Markstrom SL, Hay LE, Viger RJ, Norton PA, Driscoll JM, LaFontaine JH (2018) Description of the national hydrologic model for use with the precipitation-runoff modeling system (prms). Technical report, US Geological Survey

  65. gridMET - Climatology Lab. http://www.climatologylab.org/gridmet.html

  66. Tzeng E, Hoffman J, Saenko K, Darrell T (2017) Adversarial discriminative domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7167–7176

  67. Burns DA, Murdoch PS, Lawrence GB, Michel RL (1998) Effect of groundwater springs on \(No_3^-\) concentrations during summer in Catskill mountain streams. Water Resour Res 34(8):1987–1996

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the USGS awards G21AC10207, G21AC10564, and G22AC00266, the NSF awards 2147195, 2105133, and 2126474, the NASA award 80NSSC22K1164, Google’s AI for Social Good Impact Scholars program, the Pitt Momentum Award, and the DRI award at the University of Maryland. This research was supported in part by the University of Pittsburgh Center for Research Computing through the resources provided. Any use of trade, firm, or product names is for descriptive purposes only and does not imply endorsement by the US Government.

Author information

Authors and Affiliations

  1. University of Pittsburgh, Pittsburgh, USA

    Shengyu Chen, Nasrin Kalanat & Xiaowei Jia

  2. University of Maryland, College Park, USA

    Yiqun Xie

  3. University of Virginia, Charlottesville, USA

    Sheng Li

  4. U.S. Geological Survey, Reston, USA

    Jacob A. Zwart, Jeffrey M. Sadler, Alison P. Appling, Samantha K. Oliver & Jordan S. Read

  5. Oklahoma State University, Stillwater, USA

    Jeffrey M. Sadler

  6. Consortium of Universities for the Advancement for Hydrologic Science, Inc., Arlington, USA

    Jordan S. Read

Authors
  1. Shengyu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nasrin Kalanat
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yiqun Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sheng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jacob A. Zwart
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jeffrey M. Sadler
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Alison P. Appling
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Samantha K. Oliver
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Jordan S. Read
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Xiaowei Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaowei Jia.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, S., Kalanat, N., Xie, Y. et al. Physics-guided machine learning from simulated data with different physical parameters. Knowl Inf Syst 65, 3223–3250 (2023). https://doi.org/10.1007/s10115-023-01864-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-023-01864-z

Keywords

Search

Navigation