This paper investigates the trajectory tracking problem for quadrotor with finite-time convergence via geometric approach. Based on a global geometric dynamic description on SE(3), we decompose the finite-time trajectory tracking control into two cascaded tracking loops: the position finite-time tracking control loop and the attitude finite-time tracking control loop. A finite-time controller based on exponential coordinate is proposed via a sliding mode approach, which avoids artificial singularity and redundancy. Further, an adaptive law is utilized to estimate the uncertainties of the mass and inertia matrix. Numerical results are presented to demonstrate the effectiveness of this trajectory tracking strategy.