This paper concerns the deployment of a tethered space robot with only tension control under the optimal policy, which is generated from Q-learning iteration with fuzzy approximation. The Q-learning iteration gives rise to a feasible sequence of control input, that does not have to well consider the constrained tension, and the optimal policy is generated offline and runs onboard with the low computational requirements. Underactuated dynamics is transformed into the specified reduced-order system, which is uniformly ultimately bounded based on the analysis of the motion on the nonlinear sliding surface. Continuous inputs are generated from the interpolation strategy of discrete Q-learning iteration, which owns a better dynamic and steady-state performance. The proposed method is high real-time, effective and efficient, which has been verified by numerical simulations.