Program reversals for evolutions with non-uniform step costs

Abstract.

The reversal of a program execution can be helpful for several purposes. The reverse mode of automatic differentiation, parameter estimation, and debugging represent some of the tasks that may require the intermediate values of a calculation in reverse order. The recording of a complete execution log forms the simplest approach to providing the data required for the reversal. However, this “store-everything” method causes an enormous demand for memory. The generation of the execution log piece by piece offers the possibility to reduce the storage requirement. To that end the forward calculation is restarted from suitably placed checkpoints leading to a more or less complicated reversal schedule.

This paper deals with the reversal of evaluation procedures that consist of a sequence of time steps with varying computational complexity. We present a new search algorithm for determining a reversal schedule that minimizes the runtime of the reversal process for a given number of checkpoints. By exploiting a certain monotonicity property, the search algorithm based on dynamic programming can be made to grow only quadratically with the number of time steps to be reverted. We report the runtime savings that can be achieved performing the reversal with an optimal, i.e. time-minimal, reversal schedule for a test problem based on Burgers’ equation.