Data centers have shown great opportunities to participate in extensive demand response programs in recently years. This paper specifically focuses on data centers as participants in regulation service reserves (RSR) power market. We propose a novel approach to model the dynamics of the job processing Quality of Service (QoS) in data centers that offer RSR, and use stochastic dynamic programming (DP) to solve for the optimal reserve deployment policies. We show that the job QoS degradation can be modeled as a time varying probability distribution function (PDF) whose mean and variance evolve as functions of recent control statistics. The mean and variance are in fact additional state variables or sufficient statistics of the stochastic DP whose solution provides the data center operator (DCO) decision supports to minimize the average operating costs associated with RSR signal tracking error and job processing QoS degradation. Simulation results show that the feedback control policy obtained from the stochastic DP solution can reduce the DCO's operating costs compared to heuristic operating protocols reported in the literature. In addition, the DP value function can assist the DCO to bid optimally into the hour-ahead joint energy and reserve market.