Abstract
Considering the key role played by derivatives in Partial Differential Equations (PDEs), using the tanh activation function in Physics-Informed Neural Networks (PINNs) yields useful smoothness properties to derive theoretical guarantees in Sobolev norm. In this paper, we conduct an extensive functional analysis, unveiling tighter approximation bounds compared to prior works, especially for higher order PDEs. These better guarantees translate into smaller PINN architectures and improved generalization error with arbitrarily small Sobolev norms of the PDE residuals.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bajaj, C., McLennan, L., Andeen, T., Roy, A.: Recipes for when physics fails: recovering robust learning of physics informed neural networks. Mach. Learn.: Sci. Technol. 4(1), 015013 (2023). https://doi.org/10.1088/2632-2153/acb416
Constantine, G.M., Savits, T.H.: A multivariate Faa di Bruno formula with applications. Trans. Am. Math. Soc. 348(2), 503–520 (1996). https://doi.org/10.1090/S0002-9947-96-01501-2
Corless, R.M., Gonnet, G.H., Hare, D.E., Jeffrey, D.J., Knuth, D.E.: On the Lambert W function. Adv. Comput. Math. 5, 329–359 (1996). https://doi.org/10.1007/BF02124750
De Ryck, T., Jagtap, A.D., Mishra, S.: Error estimates for physics-informed neural networks approximating the Navier-Stokes equations. IMA J. Num. Anal. drac085 (01 2023). https://doi.org/10.1093/imanum/drac085
De Ryck, T., Lanthaler, S., Mishra, S.: On the approximation of functions by tanh neural networks. Neural Netw. 143, 732–750 (2021)
Doumèche, N., Biau, G., Boyer, C.: Convergence and error analysis of pinns (2023)
Gühring, I., Kutyniok, G., Petersen, P.: Error bounds for approximations with deep relu neural networks in \(w^{s,p}\) norms (2019)
Hao, Z., et al.: Physics-informed machine learning: a survey on problems, methods and applications (2023)
Herrmann, L., Opschoor, J.A.A., Schwab, C.: Constructive deep RELU neural network approximation. J. Sci. Comput. 90(1), 75 (2022)
Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural Netw. 2(5), 359–366 (1989)
Karniadakis, G.E., Kevrekidis, I.G., Lu, L., Perdikaris, P., Wang, S., Yang, L.: Physics-informed machine learning. Nature Rev. Phys. 3(6) (5 2021). https://doi.org/10.1038/s42254-021-00314-5
Raissi, M., Perdikaris, P., Karniadakis, G.E.: Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686–707 (2019)
Shin, Y.: On the convergence of physics informed neural networks for linear second-order elliptic and parabolic type pdes. Commun. Comput. Phys. 28(5), 2042–2074 (2020)
Shin, Y., Zhang, Z., Karniadakis, G.E.: Error estimates of residual minimization using neural networks for linear pdes (2023)
Wu, S., Zhu, A., Tang, Y., Lu, B.: Convergence of physics-informed neural networks applied to linear second-order elliptic interface problems (2023)
Yarotsky, D.: Error bounds for approximations with deep RELU networks. Neural Netw. 94, 103–114 (2017). https://doi.org/10.1016/j.neunet.2017.07.002
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Disclosure of Interests
The authors have no competing interests to declare that are relevant to the content of this article.
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Girault, B., Emonet, R., Habrard, A., Patracone, J., Sebban, M. (2024). Approximation Error of Sobolev Regular Functions with Tanh Neural Networks: Theoretical Impact on PINNs. In: Bifet, A., Davis, J., Krilavičius, T., Kull, M., Ntoutsi, E., Žliobaitė, I. (eds) Machine Learning and Knowledge Discovery in Databases. Research Track. ECML PKDD 2024. Lecture Notes in Computer Science(), vol 14944. Springer, Cham. https://doi.org/10.1007/978-3-031-70359-1_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-70359-1_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-70358-4
Online ISBN: 978-3-031-70359-1
eBook Packages: Computer ScienceComputer Science (R0)
