This letter proposes a robust secure and energy efficient beamforming (BF) scheme for a millimeter-wave unmanned aerial vehicle (UAV) communication system with imperfect angle-of-departure (AoD) estimation of air-to-ground channel caused by jittering. Specifically, an optimization problem is formulated to maximize the worst-case secrecy energy efficiency (SEE), defined as the ratio of the sum achievable secrecy rate (ASR) to the total power consumption, subject to the UAV transmit power constraint. Due to the difficulty in solving this problem arisen from the AoD uncertainties and the non-convex structures of SEE and ASR, we first adopt the discretization method to simplify AoD uncertainties to a deterministic form and then exploit the successive convex approximation approach with auxiliary variables to convert the original problem into a convex one. Finally, an iterative algorithm is designed to obtain the suboptimal solution. Numerical results are provided to confirm the effectiveness and superiority of the proposed robust BF scheme compared to some benchmark schemes.