Recently, the control problem of a quadrotor unmanned aerial vehicle (UAV) has been undergoing massive research. In this paper, a nonlinear disturbance observer-based (NDO) controller is proposed for attitude and altitude control of a quadrotor, in order to estimate and compensate disturbances that are imposed naturally on the quadrotor due to aerodynamics and parameter uncertainties. It is demonstrated herein that the proposed observer can estimate external disturbances asymptotically. Subsequently, it is employed with an input-output feedback linearization (FBL) controller - rendered as a baseline controller - to achieve a composite controller capable of rejecting external disturbances rigorously. The resulting controller is compared with a FBL controller that is equipped with an integral component. Simulation results demonstrate a superior performance using the former controller for disturbance rejection.