Skip to main content
SpringerLink
Log in

Dynamic Programming via Static Incrementalization

  • Published:
Higher-Order and Symbolic Computation

Abstract

Dynamic programming is an important algorithm design technique. It is used for problems whose solutions involve recursively solving subproblems that share subsubproblems. While a straightforward recursive program solves common subsubproblems repeatedly, a dynamic programming algorithm solves every subsubproblem just once, saves the result, and reuses it when the subsubproblem is encountered again. This can reduce the time complexity from exponential to polynomial. This paper describes a systematic method for transforming programs written as straightforward recursions into programs that use dynamic programming. The method extends the original program to cache all possibly computed values, incrementalizes the extended program with respect to an input increment to use and maintain all cached results, prunes out cached results that are not used in the incremental computation, and uses the resulting incremental program to form an optimized new program. Incrementalization statically exploits semantics of both control structures and data structures and maintains as invariants equalities characterizing cached results. It provides the basis of a general method for achieving drastic program speedups. Compared with previous methods that perform memoization or tabulation, the method based on incrementalization is more powerful and systematic. It has been implemented in a prototype system CACHET and applied to numerous problems and succeeded on all of them.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Abadi, M., Lampson, B., and Lévy, J.-J. Analysis and caching of dependencies. In Proceedings of the 1996 ACM SIGPLAN International Conference on Functional Programming, ACM, New York, 1996, pp. 83–91.

    Google Scholar 

  2. Aho, A.V., Hopcroft, J.E., and Ullman, J.D. The Design and Analysis of Computer Algorithms. Addison-Wesley, Reading, MA, 1974.

    Google Scholar 

  3. Bauer, F.L., Möller, B., Partsch, H., and Pepper, P. Formal program construction by transformations—Computer-aided, intuition-guided programming. IEEE Trans. Softw. Eng., 15(2) (1989) 165–180.

    Google Scholar 

  4. Bellman, R.E. Dynamic Programming. Princeton University Press, Princeton, New Jersey, 1957.

    Google Scholar 

  5. Bird, R.S. Tabulation techniques for recursive programs. ACM Comput. Surv., 12(4) (1980) 403–417.

    Google Scholar 

  6. Bird, R.S. The promotion and accumulation strategies in transformational programming. ACM Trans. Program. Lang. Syst., 6(4) (1984) 487–504.

    Google Scholar 

  7. Bird, R.S. and de Moor, O. From dynamic programming to greedy algorithms. In Formal Program Development, B. Möller, H. Partsch, and S. Schuman (Eds.). Vol. 755 of Lecture Notes in Computer Science, Springer-Verlag, Berlin, 1993, pp. 43–61.

    Google Scholar 

  8. Boiten, E.A. Improving recursive functions by inverting the order of evaluation. Sci. Comput. Program., 18(2) (1992) 139–179.

    Google Scholar 

  9. Burstall, R.M. and Darlington, J. A transformation system for developing recursive programs. J. ACM, 24(1) (1977) 44–67.

    Google Scholar 

  10. Cai, J. and Paige, R. Program derivation by fixed point computation. Sci. Comput. Program., 11 (1988/89) 197–261.

    Google Scholar 

  11. Chin, W.-N. Towards an automated tupling strategy. In Proceedings of the ACM SIGPLAN Symposium on Partial Evaluation and Semantics-Based Program Manipulation, ACM, New York, 1993, pp. 119–132.

    Google Scholar 

  12. Chin, W.-N. and Hagiya, M. A bounds inference method for vector-based memoization. In ICFP 1997 [26], pp. 176–187.

  13. Chin W.-N. and Khoo, S.-C. Tupling functions with multiple recursion parameters. In Proceedings of the 3rd International Workshop on Static Analysis, P. Cousot, M. Falaschi, G. Filè, and A. Rauzy (Eds.). Vol. 724 of Lecture Notes in Computer Science, Springer-Verlag, Berlin, Sept. 1993, pp. 124–140.

    Google Scholar 

  14. Cohen, N.H. Eliminating redundant recursive calls. ACM Trans. Program. Lang. Syst., 5(3) (1983) 265–299.

    Google Scholar 

  15. Cormen, T.H., Leiserson, C.E., and Rivest, R.L. Introduction to Algorithms. The MIT Press/McGraw-Hill, 1990.

  16. Curtis, S. Dynamic programming: A different perspective. In Algorithmic Languages and Calculi, R. Bird and L. Meertens (Eds.). Chapman & Hall, London, UK, 1997, pp. 1–23.

    Google Scholar 

  17. de Moor, O. A generic program for sequential decision processes. In Programming Languages: Implementations, Logics, and Programs, M. Hermenegildo and D.S. Swierstra (Eds.). Vol. 982 of Lecture Notes in Computer Science, Springer-Verlag, Berlin, 1995, pp. 1–23.

    Google Scholar 

  18. de Moor, O. and Gibbons, J. Bridging the algorithm gap: A linear-time functional program for paragraph formatting. Technical Report CMS-TR-97-03, School of Computing and Mathematical Sciences, Oxford Brookes University, July 1997.

    Google Scholar 

  19. Field, J. and Teitelbaum, T. Incremental reduction in the lambda calculus. In Proceedings of the 1990 ACM Conference on LISP and Functional Programming, ACM, New York, 1990, pp. 307–322.

    Google Scholar 

  20. Friedman, D.P., Wise, D.S., and Wand, M. Recursive programming through table look-up. In Proceedings of the 1976 ACM Symposium on Symbolic and Algebraic Computation, ACM, New York, 1976, pp. 85–89.

    Google Scholar 

  21. Futamura, Y. and Nogi, K. Generalized partial evaluation. In Partial Evaluation and Mixed Computation, B. Bjørner, A.P. Ershov, and N.D. Jones (Eds.). North-Holland, Amsterdam, 1988, pp. 133–151.

    Google Scholar 

  22. Hu, T.C. and Shing, M.T. Computation of matrix chain products. Part i. SIAM J. Comput., 11(2) (1982) 362–373.

    Google Scholar 

  23. Hu, T.C. and Shing, M.T. Computation of matrix chain products. Part ii. SIAM J. Comput., 13(2) (1984) 228–251.

    Google Scholar 

  24. Hu, Z., Iwasaki, H., Takeichi, M., and Takano, A. Tupling calculation eliminates multiple data traversals. In ICFP 1997 [26], pp. 164–175.

  25. Hughes, J. Lazy memo-functions. In Proceedings of the 2nd Conference on Functional Programming Languages and Computer Architecture, volume 201 of Lecture Notes in Computer Science, Springer-Verlag, Berlin, 1985, pp. 129–146.

    Google Scholar 

  26. Proceedings of the 1997 ACM SIGPLAN International Conference on Functional Programming, ACM, New York, 1997.

  27. Keller, R.M. and Sleep, M.R. Applicative caching. ACM Trans. Program. Lang. Syst., 8(1) (1986) 88–108.

    Google Scholar 

  28. Khoshnevisan, H. Efficient memo-table management strategies. Acta Informatica, 28(1) (1990) 43–81.

    Google Scholar 

  29. Liu Y.A.CACHET:An interactive, incremental-attribution-based program transformation system for deriving incremental programs. In Proceedings of the 10th IEEE Knowledge-Based Software Engineering Conference, IEEE CS Press, Los Alamitos, CA, 1995, pp. 19–26.

  30. Liu, Y.A. Principled strength reduction. In Algorithmic Languages and Calculi, R. Bird and L. Meertens (Eds.). Chapman & Hall, London, UK, 1997, pp. 357–381.

    Google Scholar 

  31. Liu, Y.A. Dependence analysis for recursive data. In Proceedings of the IEEE 1998 International Conference on Computer Languages, IEEE CS Press, Los Alamitos, CA, 1998, pp. 206–215.

    Google Scholar 

  32. Liu, Y.A. and Gómez, G. Automatic accurate cost-bound analysis for high-level languages. IEEE Transctions on Computers, 50(12) (2001) 1295–1309.

    Google Scholar 

  33. Liu, Y.A., Li, N., and Stoller, S.D. Solving regular tree grammar based constraints. In Proceedings of the 8th International Static Analysis Symposium, volume 2126 of Lecture Notes in Computer Science, Springer-Verlag, Berlin, 2001, pp. 213–233.

    Google Scholar 

  34. Liu, Y.A. and Stoller, S.D. Loop optimization for aggregate array computations. In Proceedings of the IEEE 1998 International Conference on Computer Languages, IEEE CS Press, Los Alamitos, CA, 1998, pp. 262–271.

    Google Scholar 

  35. Liu, Y.A. and Stoller, S.D. Eliminating dead code on recursive data. In Proceedings of the 6th International Static Analysis Symposium, volume 1694 of Lecture Notes in Computer Science, Springer-Verlag, Berlin, 1999, pp. 211–231.

    Google Scholar 

  36. Liu, Y.A. and Stoller, S.D. From recursion to iteration: What are the optimizations? In Proceedings of the ACM SIGPLAN 2000 Workshop on Partial Evaluation and Semantics-Based Program Manipulation, ACM, New York, 2000, pp. 73–82.

    Google Scholar 

  37. Liu, Y.A., Stoller, S.D., and Teitelbaum, T. Discovering auxiliary information for incremental computation. In Conference Record of the 23rd Annual ACM Symposium on Principles of Programming Languages, ACM, New York, 1996, pp. 157–170.

    Google Scholar 

  38. Liu, Y.A., Stoller, S.D., and Teitelbaum, T. Static caching for incremental computation. ACM Trans. Program. Lang. Syst., 20(3) (1998) 546–585.

    Google Scholar 

  39. Liu, Y.A. and Teitelbaum, T. Systematic derivation of incremental programs. Sci. Comput. Program., 24(1) (1995) 1–39.

    Google Scholar 

  40. Michie, D. “memo” functions and machine learning. Nature, 218 (1968) 19–22.

    Google Scholar 

  41. Mostow, D.J. and Cohen, D. Automating program speedup by deciding what to cache. In Proceedings of the 9th International Joint Conference on Artificial Intelligence, Morgan Kaufmann Publishers, San Francisco, CA, Aug. 1985, pp. 165–172.

    Google Scholar 

  42. Paige, R. Programming with invariants. IEEE Software, 3(1) (1986) 56–69.

    Google Scholar 

  43. Paige, R. Symbolic finite differencing—Part I. In Proceedings of the 3rd European Symposium on Programming, N.D. Jones (Ed.). Vol. 432 of Lecture Notes in Computer Science, Springer-Verlag, Berlin, 1990, pp. 36–56.

    Google Scholar 

  44. Paige, R. and Koenig, S. Finite differencing of computable expressions. ACM Trans. Program. Lang. Syst., 4(3) (1982) 402–454.

    Google Scholar 

  45. Partsch, H.A. Specification and Transformation of Programs—A Formal Approach to Software Development. Springer-Verlag, Berlin, 1990.

    Google Scholar 

  46. Pettorossi, A. A powerful strategy for deriving efficient programs by transformation. In Conference Record of the 1984 ACM Symposium on LISP and Functional Programming. ACM, New York, 1984.

    Google Scholar 

  47. Pettorossi, A. and Proietti, M. Rules and strategies for transforming functional and logic programs. ACM Comput. Surv., 28(2) (1996) 360–414.

    Google Scholar 

  48. A. Pettorossi and M. Proietti. Program derivation via list introduction. In Algorithmic Languages and Calculi, R. Bird and L. Meertens (Eds.). Chapman & Hall, London, UK, 1997.

    Google Scholar 

  49. Pugh, W. An improved cache replacement strategy for function caching. In Proceedings of the 1988 ACM Conference on LISP and Functional Programming, ACM, New York, 1988, pp. 269–276.

    Google Scholar 

  50. Pugh, W. The Omega Test: A fast and practical integer programming algorithm for dependence analysis. Commun. ACM, 31(8) (1992) 102–114.

    Google Scholar 

  51. Pugh W. and Teitelbaum, T. Incremental computation via function caching. In Conference Record of the 16th Annual ACM Symposium on Principles of Programming Languages, ACM, New York, 1989, pp. 315–328.

    Google Scholar 

  52. Purdom, P.W. and Brown, C.A. The Analysis of Algorithms. Holt, Rinehart and Winston, 1985.

  53. Reps, T. and Teitelbaum, T. The Synthesizer Generator: A System for Constructing Language-Based Editors. Springer-Verlag, New York, 1988.

    Google Scholar 

  54. Rosendahl, M. Automatic complexity analysis. In Proceedings of the 4th International Conference on Functional Programming Languages and Computer Architecture, ACM, New York, 1989, pp. 144–156.

    Google Scholar 

  55. Scherlis W.L. Program improvement by internal specialization. In Conference Record of the 8th Annual ACM Symposium on Principles of Programming Languages, ACM, New York, 1981, pp. 41–49.

    Google Scholar 

  56. Smith, D.R. KIDS: A semiautomatic program development system. IEEE Trans. Softw. Eng., 16(9) (1990) 1024–1043.

    Google Scholar 

  57. Smith, D.R. Structure and design of problem reduction generators. In Constructing Programs from Specifications, B. Möller (Ed.). North-Holland, Amsterdam, 1991, pp. 91–124.

    Google Scholar 

  58. Sniedovich, M. Dynamic Programming. Marcel Dekker, New York, 1992.

    Google Scholar 

  59. Unnikrishnan, L., Stoller, S.D., and Liu, Y.A. Automatic accurate stack space and heap space analysis for high-level languages. Technical Report TR 538, Computer Science Department, Indiana University, April 2000.

  60. Unnikrishnan, L., Stoller, S.D., and Liu, Y.A. Automatic accurate live memory analysis for garbage-collected languages. In Proceedings of the ACM SIGPLAN 2001 Workshop on Languages, Compilers, and Tools for Embedded Systems, ACM, New York, 2001, pp. 102–111.

    Google Scholar 

  61. Wegbreit, B. Mechanical program analysis. Commun. ACM, 18(9) (1975) 528–538.

    Google Scholar 

  62. Wegbreit, B. Goal-directed program transformation. IEEE Trans. Softw. Eng., SE-2(2) (1976) 69–80.

    Google Scholar 

  63. Zhang, Y. and Liu, Y.A. Automating derivation of incremental programs. In Proceedings of the 1998 ACM SIGPLAN International Conference on Functional Programming, ACM, New York, 1998, p. 350.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science Department, State University of New York at Stony Brook, Stony Brook, NY, 11794, USA

    Yanhong A. Liu & Scott D. Stoller

Authors
  1. Yanhong A. Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Scott D. Stoller
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Liu, Y.A., Stoller, S.D. Dynamic Programming via Static Incrementalization. Higher-Order and Symbolic Computation 16, 37–62 (2003). https://doi.org/10.1023/A:1023068020483

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1023068020483

Search

Navigation