Skip to main content
SpringerLink
Log in

Computational Divided Differencing and Divided-Difference Arithmetics

  • Published:
Higher-Order and Symbolic Computation

Abstract

Tools for computational differentiation transform a program that computes a numerical function F(x) into a related program that computes F′(x) (the derivative of F). This paper describes how techniques similar to those used in computational-differentiation tools can be used to implement other program transformations—in particular, a variety of transformations for computational divided differencing. The specific technical contributions of the paper are as follows:

– It presents a program transformation that, given a numerical function F(x) defined by a program, creates a program that computes F[x 0, x 1], the first divided difference of F(x), where\(F\left[ {x_0 ,x_1 } \right]\mathop = \limits^{def} \left\{ {\begin{array}{*{20}c} {\frac{{F\left( {x_0 } \right) - F\left( {x_1 } \right)}}{{x_0 - x_0 }}{\text{ }}} \\ {\frac{d}{{dz}}F\left( z \right),\operatorname{evaluatedatz} = x_0 } \\ \end{array} } \right.\begin{array}{*{20}c} {\operatorname{if} x_0 \ne x_1 } \\ {} \\ {\operatorname{if} x_0 = x_1 } \\ \end{array}\)

– It shows how computational first divided differencing generalizes computational differentiation.

– It presents a second program transformation that permits the creation of higher-order divided differences of a numerical function defined by a program.

– It shows how to extend these techniques to handle functions of several variables.

The paper also discusses how computational divided-differencing techniques could lead to faster and/or more robust programs in scientific and graphics applications.

Finally, the paper describes how computational divided differencing relates to the numerical-finite-differencing techniques that motivated Robert Paige's work on finite differencing of set-valued expressions in SETL programs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Andersen, L.O. Program analysis and specialization for the C programming language. Ph.D. thesis, DIKU, Univ. of Copenhagen. (DIKU report 94/19), 1994.

  2. Austin, T., Breach, S., and Sohi, G. Efficient detection of all pointer and array access errors. In SIGPLAN Conf. on Prog. Lang. Design and Impl., ACM Press, New York, NY, 1994, pp. 290–301.

    Google Scholar 

  3. Bachmann, O., Wang, P., and Zima, E. Chains of recurrences—A method to expedite the evaluation of closed form functions. In Proc. of ISSAC 1994: Int. Symp. on Symb. and Alg. Comp., New York, NY, ACM Press, 1994, pp. 242–249.

    Google Scholar 

  4. Bates, S. and Horwitz, S. Incremental program testing using program dependence graphs. In Symp. on Princ. of Prog. Lang. 1993, pp. 384–396.

  5. Beck, T. and Fischer, H. The if-problem in automatic differentiation. J. Comp. and Appl. Math., 50 (1994) 119–131.

    Google Scholar 

  6. Berz, M., Bischof, C., Corliss, G., and Griewank, A. (Eds.). Computational Differentiation: Techniques, Applications, and Tools. Soc. for Indust. and Appl. Math. Philadelphia, PA, 1996.

    Google Scholar 

  7. Binkley, D. Using semantic differencing to reduce the cost of regression testing. In Proceedings of the 1992 Conference on Software Maintenance (Orlando, FL, Nov. 9–12, 1992), 1992, pp. 41–50.

  8. Bischof, C., Carle, A., Corliss, G., Griewank, A., and Hovland, P. ADIFOR: Generating derivative codes from Fortran programs. Scientific Programming, 1(1) (1992) 11–29.

    Google Scholar 

  9. Bischof, C., Carle, A., Khademi, P., and Mauer, A. ADIFOR 2.0: Automatic differentiation of Fortran 77 programs. IEEE Comp. Sci. and Eng., 3 (1996) 18–32.

    Google Scholar 

  10. Bischof, C. and Haghighat, M. Hierarchical approaches to automatic differentiation. In [6], 1996, pp. 83–94.

  11. Bischof, C., Roh, L., and Mauer, A. ADIC:An extensible automatic differentiation tool for ANSI-C. Software—Practice and Experience, 27(12) (1997) 1427–1456.

    Google Scholar 

  12. Bjørner, D., Ershov, A., and Jones, N. (Eds.). Partial evaluation and mixed computation: Proc. of the IFIP TC2 workshop on partial evaluation and mixed computation. North-Holland, New York, NY, 1988.

    Google Scholar 

  13. Cardelli, L. A semantics of multiple inheritance. In Semantics of Data Types: Proc. of the Int. Symp., G. Kahn, D. MacQueen, and G. Plotkin (Eds.), vol. 173, Lec. Notes in Comp. Sci., Springer-Verlag, New York, NY, 1984, pp. 51–67

    Google Scholar 

  14. Cocke, J. and Schwartz, J. Programming Languages and Their Compilers: Preliminary Notes, 2nd Rev. Version, New York, NY: Courant Inst. of Math. Sci., New York University, 1970.

    Google Scholar 

  15. Conte, S. and de Boor, C. Elementary Numerical Analysis: An Algorithmic Approach, 2nd. Ed. McGraw-Hill, New York, NY, 1972.

    Google Scholar 

  16. de Boor, C. A Practical Guide to Splines, Appl. Math. Sciences, vol. 27, Springer-Verlag, New York, NY, 1978.

    Google Scholar 

  17. Earley, J. High-level operations in automatic programming. In Proc. of the ACM SIGPLAN Symp. on Very High Level Languages, ACM Press, New York, NY, 1974.

    Google Scholar 

  18. Earley, J. High-level iterators and a method for automatically designing data structure representation. J. Comp. Lang., 1(4) (1976) 321–342.

    Google Scholar 

  19. Fischer, H. Special problems in automatic differentiation. In [30], 1992, pp. 43–50.

  20. Fong, A. Elimination of common subexpressions in very high level languages. In Symp. on Princ. of Prog. Lang., ACM Press, New York, NY, 1977, pp. 48–57.

    Google Scholar 

  21. Fong, A. Inductively computable constructs in very high level languages. In Symp. on Princ. of Prog. Lang., ACM Press, New York, NY, 1979, pp. 21–28.

    Google Scholar 

  22. Fong, A. and Ullman, J. Induction variables in very high level languages. In Symp. on Princ. of Prog. Lang., ACM Press, New York, NY, 1976, pp. 104–112.

    Google Scholar 

  23. Futamura, Y. Partial evaluation of computation process—An approach to a compiler-compiler. Systems · Computers · Controls, 2(5) (1971).

  24. Futamura, Y. Partial evaluation of computation process—An approach to a compiler-compiler. Higher-Order and Symb. Comp., 12(4) 1999. Reprinted from Systems · Computers · Controls 2(5), 1971.

  25. Gallagher, K. and Lyle, J. Using program slicing in software maintenance. IEEE Transactions on Software Engineering, SE-17(8) (1991) 751–761.

    Google Scholar 

  26. Goldstine, H. A History of Numerical Analysis, Springer-Verlag, New York, NY, 1977.

    Google Scholar 

  27. Griewank, A. On automatic differentiation. In Mathematical Programming: Recent Developments and Applications, M. Iri and K. Tanabe (Eds.). Kluwer Academic Publishers, Boston, MA, 1989, pp. 83–108.

    Google Scholar 

  28. Griewank, A. The chain rule revisited in scientific computing. SIAM News, 24 (1991).

  29. Griewank, A. Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation, Frontiers in Applied Mathematics., vol. 19, Soc. for Indust. and Appl. Math., Philadelphia, PA, 2000

    Google Scholar 

  30. Griewank, A. and Corliss, G. (Eds.). Automatic Differentiation of Algorithms: Theory, Implementation, and Application, Soc. for Indust. and Appl. Math., Philadelphia, PA, 1992.

    Google Scholar 

  31. Hammer, R., Hocks, M., Kulisch, U., and Ratz, D. Numerical Toolbox for Verified Computing I: Basic Numerical Problems, Springer Ser. in Comp. Math., vol. 21, Springer-Verlag, New York, NY, 1993.

    Google Scholar 

  32. Hammer, R., Hocks, M., Kulisch, U., and Ratz, D. C++ Toolbox for Verified Computing I: Basic Numerical Problems, Springer-Verlag, New York, NY, 1995.

    Google Scholar 

  33. Horwitz, S. Identifying the semantic and textual differences between two versions of a program. In SIGPLAN Conf. on Prog. Lang. Design and Impl., 1990, pp. 234–245.

  34. Horwitz, S., Prins, J., and Reps, T. Integrating non-interfering versions of programs. Trans. on Prog. Lang. and Syst., 11(3) (1989) 345–387.

    Google Scholar 

  35. Horwitz, S., Reps, T., and Binkley, D. Interprocedural slicing using dependence graphs. Trans. on Prog. Lang. and Syst., 12(1) (1990) 26–60.

    Google Scholar 

  36. Iri, M. Simultaneous computation of functions, partial derivatives and estimates of rounding errors: Complexity and practicality. Japan J. Appl. Math., 1(2) (1984) 223–252.

    Google Scholar 

  37. Jones, N., Gomard, C., and Sestoft, P. Partial Evaluation and Automatic Program Generation, Prentice-Hall International, 1993.

  38. Kahan, W. Personal communication to Thomas Reps and Louis Rall, 2000.

  39. Kahan, W. and Fateman, R. Symbolic computation of divided differences. Unpublished report; available at http://www.cs.berkeley.edu/fateman/papers/divdiff.pdf, 1985.

  40. Karczmarczuk, J. Functional differentiation of computer programs. In Proc of the ACM SIGPLAN Int. Conf. on Func. Prog. (ICFP’ 98), ACM Press, New York, NY, 1999, pp. 195–203.

    Google Scholar 

  41. Karczmarczuk, J. Traitement paresseux et optimisation des suites numériques. In Proc. of JFLA, 2000, pp. 17–30.

  42. Karczmarczuk, J. Functional differentiation of computer programs. Higher-Order and Symb. Comp., 14(1) (2001) 35–57.

    Google Scholar 

  43. Kearfott, R. Automatic differentiation of conditional branches in an operator overloading context. In [6], 1996, pp. 75–81.

  44. Krawczyk, R. and Neumaier, A. Interval slopes for rational functions and associated centered forms. SIAM J. Numer. Anal., 22(5) (1985) 604–616.

    Google Scholar 

  45. Linnainmaa, S. Taylor expansion of the accumulated rounding error. BIT 16(1) (1976) 146–160.

    Google Scholar 

  46. Liu, Y. Incremental computation: A semantics-based systematic transformation approach. Ph.D. diss. and Tech. Rep. TR95-1551, Dept. of Comp. Sci., Cornell Univ., Ithaca, NY, 1995.

    Google Scholar 

  47. Liu, Y. and Stoller, S. Loop optimization for aggregate array computations. In Proc. of the IEEE 1998 Int. Conf. on Comp. Lang., ACM Press, New York, NY, 1998.

    Google Scholar 

  48. Liu, Y. and Stoller, S. Dynamic programming via static incrementalization. In Proc. of ESOP’ 99: European Symp. on Programming, S. Swierstra (Ed.), Lec. Notes in Comp. Sci., vol. 1576, Springer-Verlag, New York, NY, 1999, pp. 288–305.

  49. Liu, Y. and Stoller, S. From recursion to iteration: What are the optimizations?. In Proc. of the ACM SIGPLAN Workshop on Part. Eval. and Sem.-Based Prog. Manip., ACM Press, New York, NY, 2000, pp. 73–82.

    Google Scholar 

  50. Liu, Y., Stoller, S., and Teitelbaum, T. Discovering auxiliary information for incremental computation. In Symp. on Princ. of Prog. Lang., ACM Press, New York, NY, 1996, pp. 157–170.

    Google Scholar 

  51. Liu, Y. and Teitelbaum, T. Caching intermediate results for program improvement. In Proc. of the ACM SIGPLAN Symp. on Part. Eval. and Sem.-Based Prog. Manip., ACM Press, New York, NY, 1995.

    Google Scholar 

  52. Liu Y. and Teitelbaum, T. Systematic derivation of incremental programs. Science of Computer Programming, 24 (1995) 1–39.

    Google Scholar 

  53. Lyle, J. and Weiser, M. Experiments on slicing-based debugging tools. In Proceedings of the First Conference on Empirical Studies of Programming, 1986.

  54. Makino, K. and Berz, M. Remainder differential algebras and their applications. In [6], 1996, pp. 63–74.

  55. McCurdy, A. Accurate computation of divided differences. Ph.D. diss. and Tech. Rep. UCB/ERL M80/28, Univ. of Calif.–Berkeley, Berkeley, CA, 1980.

    Google Scholar 

  56. Milner, R. A theory of type polymorphism in programming. J. Comp. Syst. Sci., 17 (1978) 348–375.

    Google Scholar 

  57. Mogensen, T. The application of partial evaluation to ray-tracing. Master's thesis, Datalogisk Institut, Univ. of Copenhagen, Copenhagen, Denmark, 1986.

    Google Scholar 

  58. Ning, J., Engberts, A., and Kozaczynski, W. Automated support for legacy code understanding. Commun. ACM, 37(5) (1994) 50–57.

    Google Scholar 

  59. Opitz, G. Steigungsmatrizen. Zeitschrift für Angewandte Mathematik und Mechanik 44 T52–T54. In German. (English translation available at http://www.cs.wisc.edu/wpis/papers/opitz.zamm64.ps), 1964.

    Google Scholar 

  60. Ottenstein, K. and Ottenstein, L. The program dependence graph in a software development environment. In Proc. of the ACM SIGSOFT/SIGPLAN Softw. Eng. Symp. on Practical Software Development Environments, ACM Press, New York, NY, 1984, pp. 177–184.

    Google Scholar 

  61. Paige, R. Formal Differentiation—A Program Synthesis Technique, UMI Research Press Ann Arbor, MI. 1981.

    Google Scholar 

  62. Paige, R. Transformational programming–Applications to algorithms and systems. In Symp. on Princ. of Prog. Lang., ACM Press, New York, NY, 1983, pp. 73–87.

    Google Scholar 

  63. Paige, R. and Koenig, S. Finite differencing of computable expressions. Trans. on Prog. Lang. and Syst., 4(3) (1982) 402–454.

    Google Scholar 

  64. Rall, L. Automatic Differentiation: Techniques and Applications, Lecture Notes in Computer Science, vol. 120, Springer-Verlag, 1981.

  65. Rall, L. Differentiation and generation of Taylor coefficients in Pascal-SC. In A New Approach to Scientific Computation. U. Kulisch and W. Miranker (Eds.). Academic Press, New York, NY, 1983, pp. 291–309.

    Google Scholar 

  66. Rall, L. Differentiation in Pascal-SC: Type GRADIENT. ACM Trans. Math. Softw., 10 (1984) 161–184.

    Google Scholar 

  67. Rall, L., The arithmetic of differentiation. Mathematics Magazine, 59 (1986) 275–282.

    Google Scholar 

  68. Rall, L. Differentiation arithmetics. In Computer Arithmetic and Self-Validating Numerical Methods, C. Ullrich (Ed.). New York, NY: Academic Press, 1990, pp. 73–90.

    Google Scholar 

  69. Rall, L. Point and interval differentiation arithmetics. In [30], 1992, pp. 17–24.

  70. Rall, L. and Reps, T. Algorithmic differencing. In Perspectives in Enclosure Methods, A. Facius and U. Kulisch (Eds.). Springer-Verlag, Vienna, 2001.

    Google Scholar 

  71. Ratz, D. An optimized interval slope arithmetic and its application. Bericht 4/1996, Institut für Angewandte Mathematik, Universität Karlsruhe, Karlsruhe, Germany, 1996.

    Google Scholar 

  72. Reps, T. and Turnidge, T. Program specialization via program slicing. In O. Danvy, R. Glück, and P. Thiemann (Eds.), Proc. of the Dagstuhl Seminar on Partial Evaluation, vol. 1110 of Lec. Notes in Comp. Sci., Schloss Dagstuhl, Wadern, Germany, Springer-Verlag, 1996, pp. 409–429.

    Google Scholar 

  73. Shamseddine, K. and Berz, M. Exception handling in derivative computation with nonarchimedean calculus. In [6], 1996, pp. 37–51.

  74. Speelpenning, B. Compiling fast partial derivatives of functions given by algorithms, 1980.

  75. Strachey, C. Fundamental concepts in programming languages. Higher-Order and Symb. Comp., 13(1/2) (2000) 1–49.

    Google Scholar 

  76. Veldhuizen, T. Expression templates. C++ Report 7(5) (1995) 26–31.

    Google Scholar 

  77. Weiser, M. Program slicing. IEEE Transactions on Software Engineering, SE-10(4) (1984) 352–357.

    Google Scholar 

  78. Wengert, R. A simple automatic derivative evaluation program. Commun. ACM, 7(8) (1964) 463–464.

    Google Scholar 

  79. Zippel, R. Personal communication to Thomas Reps, 1996.

  80. Zuhe, S. and Wolfe, M. On interval enclosures using slope arithmetic. Applied Mathematics and Computation, 39(1) (1990) 89–105.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science Department, University of Wisconsin, 1210 W. Dayton St., Madison, WI, 53706, USA

    Thomas W. Reps

  2. Department of Mathematics, University of Wisconsin, 480 Lincoln Dr., Madison, WI, 53706, USA

    Louis B. Rall

Authors
  1. Thomas W. Reps
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Louis B. Rall
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Reps, T.W., Rall, L.B. Computational Divided Differencing and Divided-Difference Arithmetics. Higher-Order and Symbolic Computation 16, 93–149 (2003). https://doi.org/10.1023/A:1023024221391

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1023024221391

Search

Navigation