In this paper, we combine a spherical coordinate method and a gradient learning optimization algorithm to solve the dynamic power management (DPM) problem. In our approach, the DPM is modeled as a constrained SMDP problem with unknown model parameters. By utilizing an augmented Lagrange multiplier method, we provide an online optimization method for the DPM. This method may estimate the gradient of the augmented Lagrange function with respect to the policy parameters and optimize the performance in an online way. The simulation tests show that the method has better convergence property and higher efficiency.