Aerial vehicles detection, especially quadrotor detection, is important for cooperative unmanned aerial vehicles (UAV). However, traditional object detection approaches require a well-annotated and large scale dataset which is labor-intensive and time-consuming. Since the appearance of UAV is various, it is difficult to establish a dataset covering all kinds of UAVs. An intuitive solution for it is to train a network on an off-the-shelf dataset of a class which is under the same parent category as quadrotors. However, domain gap between these two classes hinders the performance of the trained network. To address this issue, a two-stream network is proposed. The appearance information (from a spatial stream) and the motion information (from a temporal stream) are incorporated in this network. A fusion module, cross proposal module, is proposed to fuse these two streams. To verify the performance of this two-stream network, a fully annotated dataset of quadrotors is established. Extensive experiments are conducted on it and the results show that our two-stream network performs better than traditional detection approaches in this task.