Abstract
We investigate the performance of the adjoint approach and the variational approach for computing the sensitivities of the least squares objective function commonly used when fitting models to observations. We note that the discrete nature of the objective function makes the cost of the adjoint approach for computing the sensitivities dependent on the number of observations. In the case of ordinary differential equations (ODEs), this dependence is due to having to interrupt the computation at each observation point during numerical solution of the adjoint equations. Each observation introduces a jump discontinuity in the solution of the adjoint differential equations. These discontinuities are propagated in the case of delay differential equations (DDEs), making the performance of the adjoint approach even more sensitive to the number of observations for DDEs. We quantify this cost and suggest ways to make the adjoint approach scale better with the number of observations. In numerical experiments, we compare the adjoint approach with the variational approach for computing the sensitivities.
Similar content being viewed by others
References
Alexe, M., Sandu, A.: Forward and adjoint sensitivity analysis with continuous explicit Runge-Kutta schemes. Appl. Math. Comput. 208(2), 328–346 (2009)
Baker, C.T., Paul, C.A.: Pitfalls in parameter estimation for delay differential equations. SIAM J. Sci. Comput. 18(1), 305–314 (1997)
Bischof, C.H., Hovland, P.D., Norris, B.: On the implementation of automatic differentiation tools. Higher Order Symbol. Comput. 3(21), 311–331 (2008)
Cao, Y., Li, S., Petzold, L., Serban, R.: Adjoint sensitivity analysis for differential algebraic equations: the adjoint DAE system and its numerical solution. SIAM J. Sci. Comput. 3(24), 1076–1089 (2003)
Hutchinson, G.E.: Circular causal systems in ecology. Ann. New York Acad. Sci. 50(4), 221–246 (1948)
Kermack, W., McKendrick, A.: A contribution to the mathematical theory of epidemics. In: Proceedings of the Royal Society of London A: Mathematical, Physical and Engineering Sciences, vol. 115, pp. 700–721. The Royal Society (1927)
Kuang, Y.: Delay Differential Equations: with Applications in Population Dynamics. Academic Press (1993)
Lenz, S., Schlder, J., Bock, H.: Numerical computation of derivatives in systems of delay differential equations. Math. Comput. Simul. 96, 124–156 (2014)
Petzold, L., Li, S., Cao, Y., Serban, R.: Sensitivity analysis of differential-algebraic equations and partial differential equations. Comput. Chem. Eng. 30(10), 1553–1559 (2006)
Sengupta, B., Friston, K., Penny, W.: Efficient gradient computation for dynamical models. NeuroImage 98, 521–527 (2014)
Shakeri, F., Dehghan, M.: Solution of delay differential equations via a homotopy perturbation method. Math. Comput. Modell. 48(3), 486–498 (2008)
Varah, J.: A spline least squares method for numerical parameter estimation in differential equations. SIAM J. Sci. Stat. Comput. 3(1), 28–46 (1982)
Wang, B.: Parameter Estimation for Odes using a Cross-Entropy Approach. University of Toronto, Master’s thesis (2012)
Zivari-Piran, H.: Efficient Simulation, Accurate Sensitivity Analysis and Reliable Parameter Estimation for Delay Differential Equations. Ph.D. thesis, Univeristy of Toronto (2009)
Zivari-Piran, H., Enright, W.: Accurate First-Order Sensitivity Analysis for Delay Differential Equations: Part II: the Adjoint Approach. preprint, Department of Computer Science, University of Toronto (2009)
Zivari-Piran, H., Enright, W.H.: Accurate first-order sensitivity analysis for delay differential equations. SIAM J. Sci. Comput. 34(5), A2704–A2717 (2012)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Calver, J., Enright, W. Numerical methods for computing sensitivities for ODEs and DDEs. Numer Algor 74, 1101–1117 (2017). https://doi.org/10.1007/s11075-016-0188-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11075-016-0188-6