VisDrone-MOT2020: The Vision Meets Drone Multiple Object Tracking Challenge Results

Abstract

The Vision Meets Drone (VisDrone2020) Multiple Object Tracking (MOT) is the third annual UAV MOT tracking evaluation activity organized by the VisDrone team, in conjunction with European Conference on Computer Vision (ECCV 2020). The VisDrone-MOT2020 consists of 79 challenging video sequences, including 56 videos ($\sim$24K frames) for training, 7 videos ($\sim$3K frames) for validation and 17 videos ($\sim$6K frames) for evaluation. All frames in these sequences are manually annotated with high-quality bounding boxes. Results of 12 participating MOT algorithms are presented and analyzed in detail. The challenging results, video sequences as well as the evaluation toolkit are made available at http://aiskyeye.com/. By holding VisDrone-MOT2020 challenge, we hope to facilitate future research and applications of MOT algorithms on drone videos.

A Descriptions of Submitted Trackers

In the appendix, we summarize 12 trackers submitted in the VisDrone-MOT2020 Challenge, which are ordered according to the submissions of their final results.

1.1 A.1 Coarse-to-Fine Multi-Class Multi-Object Tracking (COFE)

Yuhang He, Wentao Yu, Jie Han, Xiaopeng Hong, Xing Wei and Yihong Gong

{hyh1379478,yu1034397129,hanjie1997}@stu.xjtu.edu.cn,

{hongxiaopeng,weixing,ygong}@mail.xjtu.edu.cn

COFE is proposed to track multiple targets in different categories under different scenarios. As shown in Fig. 1, the proposed method contains three major modules: 1) Multi-class object detection, 2) Coarse-category multi-object tracking, and 3) Fine-grained trajectory finetuning. Firstly, we use a Deep Convolutional Neural Network (DCNN) based object detector [6] to detect interested targets in the image plane, where each detection is denoted by a bounding box with a class label and a confidence score. Secondly, we track multiple targets in coarse categories, where fine-grained classes (such as van, bus, car) are summarized into coarse categories (e.g., vehicle). For each coarse category, we perform multi-object tracking by exploiting the appearance and motion information of targets, where the appearance feature is extracted using a DCNN feature extractor [54] and the motion pattern of each target is modeled by a Kalman Filter. Finally, for each obtained trajectory, we finetune its fine-grained class label by a simple voting and refine the tracking results by post processing (i.e., bounding box smoothing).

Fig. 1.

The framework of COFE.

1.2 A.2 Simple Online Multi-Object Tracker (SOMOT)

Zhipeng Luo, Yuehan Yao, Zhenyu Xu, Bin Dong and Wang Sai

{luozp,yaoyh,xuzy,dongb,wangs}@deepblueai.com

Following separate detection and embedding model, we build a strong detector based on Cascade R-CNN [6] and a embedding model based on Multiple Granularity Network (MGN). For association step, we build simple online multi-object tracker, which is inspired by DeepSORT [46] and FairMOT [51]. For detector, Cascade R-CNN [6] pretrained on COCO [29] is applied. For embedding model, bag of tricks are used to improve the performance of MGN [40]. For association step, we initialize a number of tracklets based on the estimated boxes in the first frame. In the subsequent frames, we associate the boxes to the existing tracklets (all activated tracklets) according to their distances measured by embedding features. We update the appearance features of the trackers in each time step to handle appearance variations. Then, unmatched activated tracklets and estimated boxes are associated by their distance of Intersection over Union (IoU). Also, inactivated tracklets and estimated boxes are associated by their distance of IoU.

1.3 A.3 Position-, Appearance- and Size-aware Tracker (PAS tracker)

Daniel Stadler, Lars Wilko Sommer and Arne Schumann

daniel.stadler@kit.edu,{lars.sommer,arne.schumann}@iosb.fraunhofer.de

The PAS algorithm follows the tracking-by-detection paradigm. As detectors, we train two Cascade R-CNN [6] with FPN [28] on the VisDrone2020 MOT train and val set applying as backbone ResNeXt-101 [49] and HRNetV2p-W32 [41], respectively. Training is performed on randomly sampled image crops ($608 \times 608$ pixels) and the SSD [30] data augmentation pipeline is used. To improve the quality of the detections, we utilize test-time strategies like horizontal flipping and multi-scale testing. Additionally, we generate category-specific expert models using weights from different epochs and from the two detectors with different backbones. For associating detections, we build a similarity measure that integrates position, appearance and size information of objects. A constant velocity model is assumed for the motion prediction of objects and a camera motion compensation model based on the Enhanced Correlation Coefficient Maximization [15] is also applied. The appearance of an object is represented by a feature vector computed with a re-identification model from [31] based on a ResNet-50 [19]. The association of tracks and new detections is solved by the Hungarian method [23]. Additionally, to remove false positive detections in crowded scenarios, a simple filtering approach considering the overlap of existing tracks and new detections is proposed. Finally, we remove short tracks with less than 10 frames and small tracks with a mean size of less than 100 pixels as most of them are false positives.

1.4 A.4 Simple Online and Realtime Tracking with a Deep Association (Deepsort)

Zhaoze Zhao

hanjie@smail.swufe.edu.cn

Simple Online and Realtime Tracking (SORT) [46] is a pragmatic approach to multiple object tracking with a focus on simple, effective algorithms. In this paper, we integrate appearance information to improve the performance of SORT. Due to this extension we are able to track objects through longer periods of occlusions, effectively reducing the number of identity switches. In spirit of the original framework we place much of the computational complexity into an offline pre-training stage where we learn a deep association metric on a large-scale person re-identification dataset. During online application, we establish measurement-to-track associations using nearest neighbour queries in visual appearance space.

1.5 A.5 YOLOv5 based V-IOU tracker (YOLO-TRAC)

Zhizhao Duan, Xi Wu, Duo Xu and Zhen Xie

{Duanai,21725018}@zju.edu.cn,wuxi9410@gmail.com,zjutxz@hotmail.com

Trac is a track by detection framework. We use YOLO-V5¹ as our detection network, and V-IOU Tracker [4] is used for tracking.

1.6 A.6 An improved multi-object tracking method for the VisDrone videos based on CenterTrack (VDCT)

Shengwen Li and Yanyun Zhao

{2019140337,zyy}@bupt.edu.cn

VDCT is improved from CenterTrack, which is a point-based framework that combines detection and tracking [56]. Its inputs include the current frame, the previous frame, and the tracked objects in the previous frame; and it outputs the displacements of tracked objects. Our improvements include: (1) The tracked objects which do not match within 20 frames are allowed to associate with objects detected in current frame by properly extending the survival time of the tracked objects. (2) The motion direction of adjacent frame objects usually does not change abruptly due to the continuity of object motion, so we calculate the dot product of the displacements of adjacent frame objects and decide whether to associate the objects. (3) We use the NIOU method [34] to perform non-maximum suppression on vehicle objects. (4) We adopt the hierarchical matching strategy in DeepSORT [46] to solve the long occlusion problem. (5) OSNet [54] is used to extract each trajectory’s appearance feature, measure their distance from others and we simply merge two trajectories if their distance is close enough. The experimental results show the effectiveness of our improved method.

1.7 A.7 Cascade RCNN based IOU tracker (Cascade RCNN+IOU)

Ting Sun and Xingjie Zhao

sunting9999@stu.xjtu.edu.cn,1243273854@qq.com

We use Cascade R-CNN [6] as the detector with three improvements: (1) We use Group normalization [48] instead of Batch normalization; (2) We use online hard example mining to select positive and negative samples; (3) We use multiple scales to test our data; (4) We use two stronger backbones to train models and integrate them. Then, we perform detection association using the IOU tracker [4].

1.8 A.8 Hybrid task cascade based IOU tracker (HTC+IOU)

Ting Sun, Xingjie Zhao and Guizhong Liu

sunting9999@stu.xjtu.edu.cn,1243273854@qq.com

We use hybrid task cascade for instance segmentation [9] as the detector with three improvements: (1) We use Group normalization [48] instead of Batch normalization; (2) We use online hard example mining to select positive and negative samples; (3) We use multiple scales to test our data; (4) We use two stronger backbones to train models and integrate them. Then, we perform detection association using the IOU tracker [4].

1.9 A.9 Multi-object Tracking based on HRNet (HR-GNN)

Zheng Yang and Kaojin Zhu

151776257@qq.com,1320531351@qq.com

HR-GNN is built based on the detector using HRNet [41] as backbone. Then the tracking results are generated by using GNN to analyze the detection results.

1.10 A.10 Multi-object tracking with TrackletNet (TNT)

Haritha V, Melvin Kuriakose, Hrishikesh PS and Linu Shine

vakkatharitha@gmail.com

TNT is based on the work of [39] by merging temporal and appearance information together as a unified framework. We learn appearance similarity among tracklets by a graph model, where we use CNN features and intersection-over-union (IOU) with epipolar constraints to compensate camera movement between adjacent frames. Finally, the tracklets can be clustered into groups, resulting in trajectories with individual object IDs.

1.11 A.11 A simple baseline for one-shot multi-object tracking (anchor-free_mot)

Min Yao and Libo Zhang

libo@iscas.ac.cn

The anchor-free_mot method is based on FairMOT [51]. Specifically, we use the encoder-decoder network to extract feature maps. Then, two simple parallel heads are used to predict the bounding box and re-ID features of the targets, respectively. Notably, the targets are represented by points from the anchor-free object detection method.

1.12 A.12 Semantic Color Correlation Tracker (SCTrack)

Noor M. Al-Shakarji, Filiz Bunyak, Guna Seetharaman and Kannappan Palaniappan

{nmahyd,bunyak,palaniappank}@mail.missouri.edu,

gunasekaran.seetharaman@rl.af.mil

SCTrack is a time-efficient detection-based multi-object tracking method. Specifically, we use a three-step cascaded data association scheme to combine a fast spatial distance only short-term data association, a robust tracklet linking step using discriminative object appearance models, and an explicit occlusion handling unit relying not only on tracked objects’ motion patterns but also on environmental constraints such as presence of potential occluders in the scene. The details can be referred to [1, 2].