Unmanned aerial vehicles (UAVs) have been used extensively in traffic data collection owing to their flexibility, stability, and ease of operation. However, vehicle detection methods using horizontal detectors are less efficient and accurate for UAV vehicle detection, whereas rotated detectors suffer the discontinuous boundaries problem. Therefore, we proposed a traffic data extraction framework based on YOLOv5-OBB object detection and DeepSORT-OBB tracking algorithms to extract highly accurate traffic data from UAV videos. The framework was tested using aerial videos recorded by a UAV-mounted high-definition camera. Field experiments were conducted to collect reference data from an onboard high-precision sensor for use in evaluating the precision of the extracted traffic data. The traffic data, including trajectory, yaw angle, speed, and heading of vehicles, were extracted from UAV videos captured in different traffic scenes. The overall extraction accuracy reached 98.5%, indicating the reliability of the proposed framework in extracting highly accurate traffic data.