A quadratic programming based neural dynamic (QPND) controller design method is proposed for time-varying trajectory tracking of unmanned aerial vehicles (UAVs). Different from the traditional controller frameworks which only use pitch and roll angles to achieve the horizontal movement of the UAV, all the three attitude angles, i.e., roll, pitch and yaw angles, are applied for horizontal moving in this paper. There are at least three great superiorities of the proposed QPND controller. Firstly, three attitude angles can be coordinated control since the quadratic programming framework incorporates three attitudes. Secondly, the new framework enables the UAV to achieve the additional performance criteria or to perform subtasks. Third, the new controller can track time-varying trajectories due to neural dynamic approach. Specifically, the design process of the controller is divided into position controller design, attitude controller design and quadratic programming formulation. Finally, simulation experiments verify the feasibility and high efficiency of the proposed controller and its superiority compared with some state-of-the-art UAV controllers.