Abstract
Scientific Machine Learning (SciML) is a new multidisciplinary methodology that combines the data-driven machine learning models and the principle-based computational models to improve the simulations of scientific phenomenon and uncover new scientific rules from existing measurements. This article reveals the experience of using the SciML method to discover the nonlinear dynamics that may be hard to model or be unknown in the real-world scenario. The SciML method solves the traditional principle-based differential equations by integrating a neural network to accurately model the nonlinear dynamics while respecting the scientific constraints and principles. The paper discusses the latest SciML models and apply them to the oscillator simulations and experiment. Besides better capacity to simulate, and match with the observation, the results also demonstrate a successful discovery of the hidden physics in the pendulum dynamics using SciML.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Baker, N., et al.: Workshop report on basic research needs for scientific machine learning: core technologies for artificial intelligence, February 2019
Baydin, A.G., Pearlmutter, B.A., Radul, A.A., Siskind, J.M.: Automatic differentiation in machine learning: a survey. J. Mach. Learn. Res. 18(1), 5595–5637 (2017)
Bezanson, J., Edelman, A., Karpinski, S., Shah, V.B.: Julia: a fresh approach to numerical computing. SIAM Rev. 59(1), 65–98 (2017)
Chang, B., Meng, L., Haber, E., Ruthotto, L., Begert, D., Holtham, E.: Reversible architectures for arbitrarily deep residual neural networks. In: AAAI (2018)
Chang, B., Meng, L., Haber, E., Tung, F., Begert, D.: Multi-level residual networks from dynamical systems view. In: Conference on ICLR (2018)
Chen, R.T.Q., Rubanova, Y., Bettencourt, J., Duvenaud, D.: Neural ordinary differential equations (2019)
Chen, R.T.Q., Rubanova, Y., Bettencourt, J., Duvenaud, D.K.: Neural ordinary differential equations. In: Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 31, pp. 6571–6583. Curran Associates Inc. (2018)
Weinan, E.: A proposal on machine learning via dynamical systems. Commun. Math. Stat. 5, 1–11 (2017). https://doi.org/10.1007/s40304-017-0103-z
Mayers, D.F., Süli, E.: An Introduction to Numerical Analysis. Cambridge University Press, Cambridge (2003)
Fries, T.-P., Matthies, H.G.: A review of Petrov-Galerkin stabilization approaches and an extension to meshfree methods (2004)
Greydanus, S., Dzamba, M., Yosinski, J.: Hamiltonian neural networks (2019)
Griewank, A., Walther, A.: Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation, 2nd edn. SIAM, Philadelphia (2008)
He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 630–645. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_38
Kharazmi, E., Zhang, Z., Karniadakis, G.E.: Variational physics-informed neural networks for solving partial differential equations (2019)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Kolter, Z., Duvenaud, D., Johnson, M.: Deep implicit layers - neural ODEs, deep equilibirum models, and beyond
Li, Z., Shi, Z.: Deep residual learning and PDEs on manifold. CoRR, abs/1708.05115 (2017)
Lu, L., Meng, X., Mao, Z., Karniadakis, G.E.: DeepXDE: a deep learning library for solving differential equations. CoRR, abs/1907.04502 (2019)
Lu, Y., Zhong, A., Li, Q., Dong, B.: Beyond finite layer neural networks: bridging deep architectures and numerical differential equations. In: Proceedings of the 35th International Conference on Machine Learning (2018)
Mattheakis, M., Protopapas, P., Sondak, D., Di Giovanni, M., Kaxiras, E.: Physical symmetries embedded in neural networks (2020)
Mattheakis, M., Sondak, D., Protopapas, P.: Hamiltonian neural networks for solving differential equations. Preparation
Mohazzabi, P., Shankar, S.P.: Damping of a simple pendulum due to drag on its string. J. Appl. Math. Phys. 05(01), 122–130 (2017)
Pang, G., Lu, L., Karniadakis, G.E.: fPINNs: fractional physics-informed neural networks. SIAM J. Sci. Comput. 41(4), A2603–A2626 (2019)
Paticchio, A., Scarlatti, T., Mattheakis, M., Protopapas, P., Brambilla, M.: Semi-supervised Neural Networks solve an inverse problem for modeling Covid-19 spread (2020)
Petzold, L., Li, S., Cao, Y., Serban, R.: Sensitivity analysis of differential-algebraic equations and partial differential equations. Comput. Chem. Eng. 30(10), 1553–1559 (2006)
Pontryagin, L.S., Mishchenko, E.F., Boltyanskii, V.G., Gamkrelidze, R.V.: The Mathematical Theory of Optimal Processes. Wiley, New York (1962)
Rackauckas, C., et al.: Universal differential equations for scientific machine learning. arXiv preprint arXiv:2001.04385 (2020)
Rackauckas, C., Nie, Q.: Differentialequations.jl-a performant and feature-rich ecosystem for solving differential equations in Julia. J. Open Res. Softw. 5(1) (2017)
Raissi, M., Perdikaris, P., Karniadakis, G.E.: Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686–707 (2019)
Revels, J., Lubin, M., Papamarkou, T.: Forward-mode automatic differentiation in Julia. arXiv:1607.07892 [cs.MS] (2016)
Shin, Y., Darbon, J., Karniadakis, G.E.: On the convergence and generalization of physics informed neural networks (2020)
Sonoda, S., Murata, N.: Double continuum limit of deep neural networks. In: ICML Workshop Principled Approaches to Deep Learning (2017)
Toth, P., Rezende, D.J., Jaegle, A., Racanière, S., Botev, A., Higgins, I.: Hamiltonian generative networks (2020)
Yang, L., Zhang, D., Karniadakis, G.E.: Physics-informed generative adversarial networks for stochastic differential equations. SIAM J. Sci. Comput. 42, A292–A317 (2020)
Zhang, D., Lu, L., Guo, L., Karniadakis, G.E.: Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems. J. Comput. Phys. 397, 108850 (2019)
Zhang, X., Li, Z., Change Loy, C., Lin, D.: PolyNet: a pursuit of structural diversity in very deep networks. CoRR, abs/1611.05725 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Huang, L., Vrinceanu, D., Wang, Y., Kulathunga, N., Ranasinghe, N. (2022). Discovering Nonlinear Dynamics Through Scientific Machine Learning. In: Arai, K. (eds) Intelligent Systems and Applications. IntelliSys 2021. Lecture Notes in Networks and Systems, vol 294. Springer, Cham. https://doi.org/10.1007/978-3-030-82193-7_17
Download citation
DOI: https://doi.org/10.1007/978-3-030-82193-7_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-82192-0
Online ISBN: 978-3-030-82193-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)