The tracking and measurement methods of drogue in autonomous aerial refueling which based 2-D image, because of the loss of depth information, are susceptible to light changes, occlusion, etc., resulting in false tracking and the failure of position and orientation of drogue measurement, which may lead to collision of drones. In order to solve these problems, this article intends to introduce 3-D information to solve the position and orientation of drogue based on the original 2-D image tracking method. The method mainly includes two parts, tracking and measurement algorithms, using the decision function to effectively combine the two algorithms. The tracking algorithm provides a positioning area for the measurement algorithm, thereby reducing the amount of measurement operation data. The 3-D measurement result of the measurement algorithm based on stereo vision is used for measuring the drogue's position and orientation, and fed back to the tracking algorithm, which eliminates the candidate area and reduces the error rate to improve accuracy of the tracking algorithm. The measurement algorithm can still be realized stably high-precision under occlusion and no cooperation markers, which is not available in traditional 2-D image-based methods. We perform semiphysical simulation experiments to verify the effectiveness of the proposed method.