Numerous algorithms have been proposed for quadrotor flight control. Conventional methods require massive labor of parameter adjustment. Deep reinforcement learning (DRL) methods also need enormous computation and complicated hyperparameter tuning, since most of them regard the quadrotor dynamics as a black box. To overcome the drawbacks of both conventional and learning-based methods, this letter proposes a DRL-based cascade quadrotor flight controller. Under the small-angle restriction, the quadrotor dynamics are decomposed into six subsystems, each containing only one degree of freedom (DOF) for the agent to control. Six agents are sequentially trained to fully control the corresponding DOF without any prior knowledge of quadrotor dynamic parameters. Experiments show that, with a total training time of 17 min, the proposed controller could accomplish the fixed point tracking task with an error lower than 5 mm, rise time lower than 1.5 s, and peak value lower than 1%. The controller could also track time-variant trajectories.