Skip to main content
Springer Nature Link
Log in

On the Efficient Evaluation of Higher-Order Derivatives of Real-Valued Functions Composed of Matrix Operations

  • Conference paper
  • First Online:
Modeling, Simulation and Optimization of Complex Processes

  • 1163 Accesses

Abstract

Two different hierarchical levels of algorithmic differentiation are compared: the traditional approach and a higher-level approach where matrix operations are considered to be atomic. More explicitly: It is discussed how computer programs that consist of matrix operations (e.g. matrix inversion) can be evaluated in univariate Taylor polynomial arithmetic. Formulas suitable for the reverse mode are also shown. The advantages of the higher-level approach are discussed, followed by an experimental runtime comparison.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Bruce Christianson. Reverse accumulation and accurate rounding error estimates for Taylor series coefficients. Optimization Methods and Software, 1(1):81–94, 1991. Also appeared as Tech. Report No. NOC TR239, The Numerical Optimisation Centre, University of Hertfordshire, U.K., July 1991.

    Google Scholar 

  2. Mike B. Giles. Collected matrix derivative results for forward and reverse mode algorithmic differentiation. In Christian H. Bischof, H. Martin Bücker, Paul D. Hovland, Uwe Naumann, and J. Utke, editors, Advances in Automatic Differentiation, pages 35–44. Springer, 2008.

    Google Scholar 

  3. Andreas Griewank, David Juedes, H. Mitev, Jean Utke, Olaf Vogel, and Andrea Walther. ADOL-C: A package for the automatic differentiation of algorithms written in C/C + + . Technical report, Institute of Scientific Computing, Technical University Dresden, 1999. Updated version of the paper published in ACM Trans. Math. Software 22, 1996, 131–167.

    Google Scholar 

  4. Andreas Griewank. A mathematical view of automatic differentiation. In Acta Numerica, volume 12, pages 321–398. Cambridge University Press, 2003.

    Google Scholar 

  5. Andreas Griewank, Jean Utke, and Andrea Walther. Evaluating higher derivative tensors by forward propagation of univariate Taylor series. Mathematics of Computation, 69:1117–1130, 2000.

    Article  MathSciNet  MATH  Google Scholar 

  6. Gene H. Golub and Charles F. Van Loan. Matrix computations (3rd ed.). Johns Hopkins University Press, Baltimore, MD, USA, 1996.

    Google Scholar 

  7. Andreas Griewank and Andrea Walther. Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation. Number 105 in Other Titles in Applied Mathematics. SIAM, Philadelphia, PA, 2nd edition, 2008.

    Google Scholar 

  8. Michael J.R. Healy. Matrices for Statistics. Clarendon Press, Oxford, 2nd edition, 2000.

    Google Scholar 

  9. Laurent Hascoët and Valérie Pascual. Tapenade 2.1 user’s guide. Technical Report 0300, INRIA, 2004.

    Google Scholar 

  10. Jan R. Magnus and Heinz Neudecker. Matrix differential calculus with applications in statistics and econometrics. John Wiley & Sons, 2nd edition, 1999.

    Google Scholar 

  11. James R. Schott, editor. Matrix Analysis for Statistics. Wiley, New York, 1997.

    Google Scholar 

  12. Sebastian F. Walter. Taylorpoly. http://github.com/b45ch1/taylorpoly, 2009–2010.

  13. Sebastian F. Walter. Source code of the performance comparison. http://github.com/b45ch1/hpsc_hanoi_2009_walter, 2010.

  14. R. Clint Whaley, Antoine Petitet, and Jack J. Dongarra. Automated empirical optimization of software and the ATLAS project. Parallel Computing, 27(1–2):3–35, 2001. Also available as University of Tennessee LAPACK Working Note #147, UT-CS-00-448, 2000 (www.netlib.org/lapack/lawns/lawn147.ps).

Download references

Author information

Author notes

  1. Sebastian F. Walter

    Present address: , Unter den Linden 6, 10099, Berlin, Germany

Authors and Affiliations

  1. Department of Mathematics, Faculty of Mathematics and Natural Sciences II, Humboldt-Universität zu Berlin, Rudower Chaussee 25, Adlershof, 12489, Berlin, Germany

    Sebastian F. Walter

Authors
  1. Sebastian F. Walter
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sebastian F. Walter .

Editor information

Editors and Affiliations

  1. Interdisziplinäres Zentrum für, Wissenschaftliches Rechnen (IWR), Universität Heidelberg, Im Neuenheimer Feld 368, Heidelberg, 69120, Germany

    Hans Georg Bock

  2. and Technology (VAST), Institute of Mathematics, Vietnamese Academy of Science, Hoang Quoc Viet Road, Hanoi, 10307, Vietnam

    Xuan Phu Hoang

  3. , Interdisciplinary Center for, University of Heidelberg, Im Neuenheimer Feld 368, Heidelberg, 69120, Germany

    Rolf Rannacher

  4. , Interdisciplinary Center for, University of Heidelberg, Im Neuenheimer Feld 368, Heidelberg, 69120, Germany

    Johannes P. Schlöder

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Walter, S.F. (2012). On the Efficient Evaluation of Higher-Order Derivatives of Real-Valued Functions Composed of Matrix Operations. In: Bock, H., Hoang, X., Rannacher, R., Schlöder, J. (eds) Modeling, Simulation and Optimization of Complex Processes. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-25707-0_26

Download citation

Publish with us

Policies and ethics