Numerical computation algorithms for sequential checkpoint placement

Abstract

This paper concerns sequential checkpoint placement problems under two dependability measures: steady-state system availability and expected reward per unit time in the steady state. We develop numerical computation algorithms to determine the optimal checkpoint sequence, based on the classical Brender’s fixed point algorithm and further give three simple approximation methods. Numerical examples with the Weibull failure time distribution are devoted to illustrate quantitatively the overestimation and underestimation of the sub-optimal checkpoint sequences based on the approximation methods.

Introduction

System failures in large scaled computer systems can lead to a huge economic or critical social loss. Checkpointing and rollback recovery is a commonly used solution for improving the dependability of file systems, and is regarded as a low-cost environment diversity technique from the standpoint of fault-tolerant computing. Especially, when the file system to write and/or read data is designed in terms of preventive maintenance, checkpoint generations can back up occasionally or periodically the significant data on the primary medium to the safe secondary medium, and can play a significant role to limit the amount of data processing for the recovery actions after system failures occur. If checkpoints are frequently taken, a larger overhead by checkpointing itself will be incurred. Conversely, if checkpoints are seldom placed, a larger recovery overhead after a system failure will be required. Hence, it is important to determine the optimal checkpoint sequence taking account of the trade-off between two kinds of overhead factor above. Since the system failure phenomenon under uncertainty is described by a probability distribution, called the system failure time distribution, the optimal checkpoint sequence should be determined based on any stochastic model [1], [2], [3], [4], [5].

Young [6] obtained the optimal checkpoint interval approximately for the computation restart after system failures. Baccelli [7], Chandy et al. [2], Dohi et al. [8], Gelenbe and Derochette [9], Gelenbe [10], Gelenbe and Hernandez [11], Goes and Sumita [12], Grassi et al. [13], Kulkarni et al. [14], Nicola and Van Spanje [15], Sumita et al. [16] proposed performance evaluation models for database recovery, and calculated the optimal checkpoint intervals which maximize the system availability or minimize the mean overhead during the normal operation. L’Ecuyer and Malenfant [17] formulated a dynamic checkpoint placement problem by a Markov decision process. Ziv and Bruck [18] reconsidered a checkpoint placement problem under a random environment, by taking account of the change of operation circumstance. Vaidya [19] examined the impact of checkpoint latency on overhead ratio for a simple checkpoint model. Recently, Okamura et al. [20] reformulated the Vaidya’s model [19] with a semi-Markov decision process.

On the other hand, some authors discussed the sequential checkpoint placement problems where the checkpoint intervals were not always constant. For instance, in almost all checkpoint models for transaction-based systems [7], [8], [9], [10], [11], [12], [16], it could be proved theoretically that the constant checkpoint intervals maximizing the system availability were better than the independent and identically distributed random checkpoint intervals. For any case, however, the sequential policy with aperiodic checkpoint interval can provide the general framework on the checkpoint placement, because the sequential checkpoint involves the periodic one as a special case. Duda [21] derived a recursive formula satisfying the optimal aperiodic checkpoint sequence maximizing the mean program execution time. Toueg and Babaog˜lu [22] developed a discrete dynamic programming algorithm which mminimizes the expected execution time of tasks placing checkpoints between two consecutive tasks under very general assumptions. Kaio and Osaki [23] and Ling et al. [24] proposed approximate methods to calculate the optimal checkpoint sequence minimizing the expected cumulative operation cost until the system failure.

In the sequential checkpoint placement problem, it is assumed that the system failure time obeys the common probability distribution, i.e. does not always obey the negative exponential distribution. Actually, the non-exponential system failure time distribution with increasing failure rate can be assumed for some real workstation failure data [25]. Also, it is reported that some system failures are caused by software aging such as resource exhaustion and that the system failure time cannot be regarded as an exponentially distributed random variable any more (see e.g.[26]). The sequential checkpoint placement problem is formulated as a complex non-linear optimization problem with unknown number of decision variables. This leads to the computational difficulty to place optimally the aperiodic checkpoint sequence even for a simple centralized system under a criterion of optimality.

Although the checkpointing models mentioned above mainly focused on centralized systems, the analytical techniques for reliability and performance evaluation can be applied to distributed systems [27]. Wong and Franklin [28] considered simple Markov models to determine the frequency of checkpointing in parallel systems. Plank and Thomaso [29] modeled the performance of coordinated checkpointing systems [30], [31], [32] where the number of processors dedicated to the application and the checkpoint intervals are selected by the user before running the program. They employed a birth and death Markov chain to determine the system availability of the parallel system over the long term. Agbaria et al. [33] took account of the rollback propagation [34] and evaluated the coordinated checkpointing protocols based on both the overhead ratio and simple Markov chain models. On the other hand, uncoordinated checkpointing techniques [35] are used to reduce the checkpointing overhead in normal processing. Soliman and Elmaghraby [36] developed the so-called hybrid state saving technique to reduce the mean time to execute a finite length task for an uncoordinated checkpointing system. However, the aperiodic checkpoint scheme has not been developed yet in the literature on parallel and distributed systems.

In this paper we consider sequential checkpoint placement problems for centralized systems under two dependability measures: steady-state system availability and expected reward per unit time in the steady state [37], [38]. When the checkpoint strategy is restricted to constant intervals, the past literature [7], [2], [8], [9], [10], [11], [12], [16], [6] provided satisfactory answers on the optimal constant checkpoint interval with the negative exponential or the general system failure time distribution under the specific cost criteria. Surprisingly, it should be noted that the general sequential checkpoint placement problems have not been studied sufficiently during the last three decades except for a few examples [21], [23], [24], [22]. Recently, Ozaki et al. [39] dealt with the same problem under the expected cumulative operation cost over infinite/finite time horizon, and developed an effective computation algorithm to calculate the optimal checkpoint sequence. However, the algorithm proposed in their paper [39] was not all-round and could not be applied to the general problems. In this paper, we develop numerical computation algorithms for the optimal sequential checkpoint placement under the steady-state system availability and the expected reward per unit time in the steady state. The basic idea is due to the Brender’s classical fixed-point theorem [40], that is, the computation algorithms proposed here converge to the real optimal solutions eventually.

The rest part of this paper is planned as follows: in Section 2, we define the notation and describe two sequential checkpoint placement models with perfect and imperfect checkpointing, referred as Model A and Model B, respectively. The system availability and the expected reward rate are formulated in Section 3 and Section 4, respectively, where the numerical computation algorithms to maximize them are derived. In Section 5 we introduce three approximation methods to calculate the sub-optimal checkpoint sequence. Numerical examples with the Weibull failure time distribution are devoted in Section 6 to illustrate the overestimation and underestimation of the sub-optimal checkpoint sequences based on the approximation methods. We compare the real optimal checkpoint sequence and its associated dependability measures with three approximate solutions. Finally, the paper is concluded with some remarks in Section 7. The computation algorithms in this paper provide evidently exact solutions for unsolved problems during the last three decades and their impact to the actual fault-tolerant file management will be very significant, because the underlying techniques may be applied to the aperiodic and distributed checkpointing protocols.