Abstract
We develop a Bayesian approach to estimate the parameters of ordinary differential equations (ODE) from the observed noisy data. Our method does not need to solve ODE directly. We replace the ODE constraint with a probability expression and combine it with the nonparametric data fitting procedure into a joint likelihood framework. One advantage of the proposed method is that for some ODE systems, one can obtain closed form conditional posterior distributions for all variables which substantially reduce the computational cost and facilitate the convergence process. An efficient Riemann manifold based hybrid Monte Carlo scheme is implemented to generate samples for variables whose conditional posterior distribution cannot be written in terms of closed form. Our approach can be applied to situations where the state variables are only partially observed. The usefulness of the proposed method is demonstrated through applications to both simulated and real data.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bates D, Watts D (1988) Nonlinear regression analysis and its applications. Wiley, New York
Bhaumik P, Ghosal S (2015) Bayesian two-step estimation in differential equation models. Electron J Stat 9(2):3124–3154
Calderhead B, Girolami M, Lawrence ND (2009) Accelerating Bayesian inference over nonlinear differential equations with Gaussian processes. Adv Neural Inf Process Syst 21:217–224
Campbell D, Steele RJ (2012) Smooth functional tempering for nonlinear differential equation models. Stat Comput 22(2):429–443
Chkrebtii OA, Campbell DA, Calderhead B, Girolami MA (2016) Bayesian solution uncertainty quantification for differential equations. Bayesian Anal 11(4):1239–1267
Dattner I (2015) A model-based initial guess for estimating parameters in systems of ordinary differential equations. Biometrics 71(4):1176–1184
Dondelinger F, Husmeier D, Rogers S, Filippone M (2013) ODE parameter inference using adaptive gradient matching with Gaussian processes. In: Carvalho CM, Ravikumar P (eds) Proceedings of the sixteenth international conference on artificial intelligence and statistics, AISTATS 2013, Scottsdale, AZ, USA, April 29–May 1, 2013, pp 216–228. JMLR.org
Gelman A, Bois F, Jiang J (1996) Physiological pharmacokinetic analysis using population modeling and informative prior distributions. J Am Stat Assoc 91(436):1400–1412
Girolami M, Calderhead B (2011) Riemann manifold Langevin and Hamiltonian Monte Carlo methods. J R Stat Soc Ser B Stat Methodol 73(2):1-37
Hairer E, Nørsett SP, Wanner G (1993) Solving ordinary differential equations I (2nd revised). Nonstiff problems. Springer, Berlin
Hemker P (1972) Numerical methods for differential equations in system simulation and in parameter estimation. In: Hemker HC, Hess B (eds) Analysis and simulation of biochemical systems. Elsevier, North Holland, pp 59–80
Huang Y, Wu H (2006) A bayesian approach for estimating antiviral efficacy in HIV dynamic models. J Appl Stat 33(2):155–174
Huang Y, Liu D, Wu H (2006) Hierarchical Bayesian methods for estimation of parameters in a longitudinal HIV dynamic system. Biometrics 62(2):413–423
Li Z, Osborne MR, Pravan T (2005) Parameter estimation of ordinary differential equations. IMA J Numer Anal 25:264–285
Liang H, Wu H (2008) Parameter estimation for differential equation models using a framework of measurement error in regression models. J Am Stat Assoc 103(484):1570–1583
Lotka AJ (1909) Contribution to the theory of periodic reactions. J Phys Chem 14(3):271–274
Matteij R, Molenaar J (2002) Ordinary differential equations in theory and practice. SIAM, Philadelphia
Mazur J, Ritter D, Reinelt G, Kaderali L (2009) Reconstructing nonlinear dynamic models of gene regulation using stochastic sampling. BMC Bioinform 10(1):448
Neal RM (1994) An improved acceptance procedure for the hybrid monte carlo algorithm. J Comput Phys 111(1):194–203
Ramsay J, Hooker G (2017) Dynamic data analysis: modeling data with differential equations. Springer, New York
Ramsay J, Silverman B (2005) Functional data analysis. Springer, New York
Ramsay JO, Hooker G, Campbell D, Cao J (2007) Parameter estimation for differential equations: a generalized smoothing approach. J R Stat Soc Ser B Stat Methodol 69(5):741–796
Ranciati S, Viroli C, Wit E (2016) Bayesian smooth-and-match strategy for ordinary differential equations models that are linear in the parameters. Preprint: arXiv:1604.02318
Schittkowski K (2002) Numerical data fitting in dynamical systems: a practical introduction with applications and software. Kluwer, Norwell
Schober M, Duvenaud D, Hennig P (2014) Probabilistic ODE solvers with Runge–Kutta means. In: Ghahramani Z, Welling M, Cortes C, Lawrence N, Weinberger K (eds) Advances in neural information processing systems 27. Curran Associates Inc, Red Hook, pp 739–747
Seber GAF, Wild CJ (1989) Nonlinear regression. Wiley, New York
Varah JM (1982) A spline least squares method for numerical parameter estimation in differential equations. SIAM J Sci Stat Comput 3(1):28–46
Wang Y, Barber D (2014) Gaussian processes for Bayesian estimation in ordinary differential equations. In: Xing EP, Jebara T (eds) Proceedings of the 31st international conference on international conference on machine learning—volume 32, ICML’14, Beijing, China, pp II-1485–II-1493. JMLR.org
Xun X, Cao J, Mallick B, Maity A, Carroll RJ (2013) Parameter estimation of partial differential equation models. J Am Stat Assoc 108(503):1009–1020
Acknowledgements
The authors thank the editor, associate editor, and three referees for many helpful comments and suggestions which led to a much improved presentation. This research is supported in part by Division of Mathematical Sciences (National Science Foundation) Grant DMS-1916411 (Huang, Song).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Huang, H., Handel, A. & Song, X. A Bayesian approach to estimate parameters of ordinary differential equation. Comput Stat 35, 1481–1499 (2020). https://doi.org/10.1007/s00180-020-00962-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00180-020-00962-8