This paper proposes an improved deep deterministic policy gradient (DDPG) algorithm in the morphing policy designing for a kind of morphing unmanned aerial vehicles (UAVs) Considering that random selection in reinforcement learning structure is not always an efficient iterative update method, prioritized sweeping approach is introduced into the DDPG-based deep reinforcement learning framework, and the original DDPG algorithm is optimized to avoid random selection of state action pairs (SAPs). Consequently, the efficiency reduction problem in the traditional reinforcement learning structure is weakened. The proposed improved DDPG algorithm has better learning performance and can make reasonable decisions about environmental changes. A simulation experiment is carried out on the designed algorithm. By building a reinforcement learning model of the Markov decision process, the simulation results verify the effectiveness and superiority of the designed algorithm.