Abstract
The enrichment of machine learning models with domain knowledge has a growing impact on modern engineering and physics problems. This trend stems from the fact that the rise of deep learning algorithms is closely associated with an increasing demand for data that is not acceptable or available in many use cases. In this context, the incorporation of physical knowledge or a-priori constraints has been shown to be beneficial in many tasks. On the other hand, this collection of approaches is context-specific, and it is difficult to generalize them to new problems. In this paper, we experimentally compare some of the most widely used theory injection strategies to perform a systematic analysis of their advantages. Selected state-of-the-art algorithms have been reproduced for different use cases to evaluate their effectiveness with smaller training data and to discuss how the underlined strategies can fit into new application contexts.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Anderson, C.: The end of theory: the data deluge makes the scientific method obsolete. Wired Mag. 16(7), 7–16 (2008)
Baldi, P., Brunak, S.: Bioinformatics: the Machine Learning Approach. MIT Press, Cambridge (2001)
Battaglia, P.W., et al.: Relational inductive biases, deep learning, and graph networks. arXiv:1806.01261 [cs, stat], October 2018
Beucler, T., Pritchard, M., Rasp, S., Ott, J., Baldi, P., Gentine, P.: Enforcing analytic constraints in neural networks emulating physical systems. Phys. Rev. Lett. 126(9), 098302 (2021)
Butter, A., Kasieczka, G., Plehn, T., Russell, M.: Deep-learned top tagging with a Lorentz layer. SciPost Phys. 5(3), 028 (2018)
Castelvecchi, D., et al.: Artificial intelligence called in to tackle LHC data deluge. Nature 528(7580), 18–19 (2015)
Daw, A., Karpatne, A., Watkins, W., Read, J., Kumar, V.: Physics-guided neural networks (PGNN): an application in Lake Temperature Modeling. arXiv:1710.11431 [physics, stat] (2017)
Daw, A., Thomas, R.Q., Carey, C.C., Read, J.S., Appling, A.P., Karpatne, A.: Physics-guided architecture (PGA) of neural networks for quantifying uncertainty in lake temperature modeling. arXiv:1911.02682 [physics, stat], November 2019
Dhar, V.: Data science and prediction. Commun. ACM 56(12), 64–73 (2013)
Donti, P.L., Rolnick, D., Kolter, J.Z.: DC3: a learning method for optimization with hard constraints. ICLR 21 + arXiv:2104.12225 [cs, math, stat], April 2021
Faghmous, J.H., Kumar, V.: A big data guide to understanding climate change: the case for theory-guided data science. Big Data 2(3), 155–163 (2014)
Karpatne, A., et al.: Theory-guided data science: a new paradigm for scientific discovery from data. IEEE Trans. Knowl. Data Eng. 29(10), 2318–2331 (2017). https://doi.org/10.1109/TKDE.2017.2720168, arXiv:1612.08544
Kashinath, K., et al.: Physics-informed machine learning: case studies for weather and climate modelling. Phil. Trans. R. Soc. A 379(2194), 20200093 (2021)
Liang, X., Hu, Z., Zhang, H., Lin, L., Xing, E.P.: Symbolic graph reasoning meets convolutions. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
Mohan, A.T., Gaitonde, D.V.: A deep learning based approach to reduced order modeling for turbulent flow control using LSTM neural networks. arXiv preprint arXiv:1804.09269 (2018)
Muralidhar, N., et al.: PhyNet: physics guided neural networks for particle drag force prediction in assembly. In: Proceedings of the 2020 SIAM International Conference on Data Mining, pp. 559–567. SIAM (2020)
O’Gorman, P.A., Dwyer, J.G.: Using machine learning to parameterize moist convection: potential for modeling of climate, climate change, and extreme events. J. Adv. Model. Earth Syst. 10(10), 2548–2563 (2018)
Peng, G.C., et al.: Multiscale modeling meets machine learning: what can we learn? Arch. Comput. Methods Eng. 28(3), 1017–1037 (2021)
Peters, M.E., et al.: Knowledge enhanced contextual word representations. arXiv preprint arXiv:1909.04164 (2019)
Raissi, M., Perdikaris, P., Karniadakis, G.E.: Physics informed deep learning (Part I): data-driven solutions of nonlinear partial differential equations. arXiv:1711.10561 [cs, math, stat], November 2017. Version: 1
von Rueden, L., et al.: Informed machine learning - a taxonomy and survey of integrating knowledge into learning systems. IEEE Trans. Knowl. Data Eng. 1 (2021). https://doi.org/10.1109/TKDE.2021.3079836, arXiv:1903.12394
Seo, S., Liu, Y.: Differentiable physics-informed graph networks. Technical report, February 2019. https://ui.adsabs.harvard.edu/abs/2019arXiv190202950S, Publication Title: arXiv e-prints ADS Bibcode: 2019arXiv190202950S Type: Article
Wang, R., Yu, R.: Physics-guided deep learning for dynamical systems: a survey. arXiv:2107.01272 [cs], March 2022
Willard, J., Jia, X., Xu, S., Steinbach, M., Kumar, V.: Integrating scientific knowledge with machine learning for engineering and environmental systems. arXiv:2003.04919 [physics, stat], March 2022
Acknowledgment
The research leading to these results has been partially supported by the SmartData@PoliTO center for Big Data and Machine Learning technologies.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Monaco, S., Apiletti, D. (2022). Experimental Comparison of Theory-Guided Deep Learning Algorithms. In: Chiusano, S., et al. New Trends in Database and Information Systems. ADBIS 2022. Communications in Computer and Information Science, vol 1652. Springer, Cham. https://doi.org/10.1007/978-3-031-15743-1_24
Download citation
DOI: https://doi.org/10.1007/978-3-031-15743-1_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-15742-4
Online ISBN: 978-3-031-15743-1
eBook Packages: Computer ScienceComputer Science (R0)