Aerial image object detection has a wide range of application values in civilian or military fields. Due to its unique high-altitude imaging viewpoint and the multiangle shooting method, aerial images lead to problems, such as small objects being detected in the image, large variations in object scales, and dense distribution. To alleviate the above problems, we propose an improved aerial image object detection algorithm aerial images object detection based on YOLO (AIOD-YOLO) based on YOLOv8-s. First, we propose the multibranch contextual information aggregation module. It enhances the network’s perception of small objects by associating object information with the surrounding environment, thereby compensating for the lack of feature information for small objects. In addition, we propose the multilayer feature cascade efficient aggregation network, which leverages multigradient flow fusion of features at different scales. This approach aids the network in capturing a wide range of scale information and effectively mitigates the issue of missed detections caused by variations in object scales. Finally, we propose the adaptive task alignment label assignment strategy to address the issue of dense object distribution. The strategy incorporates the cosine similarity calculation to assess alignment globally and simultaneously adjusts the weights of positive and negative samples. We optimize the precision of label assignment for dense objects in aerial images, effectively resolving the challenges posed by their close proximity. The experiments on the VisDrone dataset demonstrate that AIOD-YOLO achieves a significant 7.2% improvement in mAP compared to the baseline model YOLOv8-s. The mAP0.5 of AIOD-YOLO is also improved by 14.1%, 7.9%, and 7.5% on SeaDronesSee v2, AI-TOD, and TinyPerson datasets, respectively, which validates the generalization of our proposed algorithm. AIOD-YOLO offers a superior information processing approach for tasks related to aerial image object detection in both civilian and military applications.