In this paper, we consider setpoint tracking for the suspension system of medium-speed maglev trains for keeping constant distance above the track. Due to complicated environment, unknown disturbances and strong nonlinear coupling of model, the problem can not be effectively solved by most of the model-based controllers. To this purpose, we formulate the setpoint tracking problem as continuous-state, continuous-action Markov decision processes under unknown transition probabilities. Based on the deterministic policy gradient and neural network approximation, we propose a model-free reinforcement learning (RL) algorithm that learns a state-feedback controller from sampled data of the suspension system. We illustrate with simulations that our model-free method has high performance.