Abstract
In thermal video security monitoring the reliability of deployed systems rely on having varied training data that can effectively generalize and have consistent performance in the deployed context. However, for security monitoring of an outdoor environment the amount of variation introduced to the imaging system would require extensive annotated data to fully cover for training and evaluation. To this end we designed and ran a challenge to stimulate research towards alleviating the impact of concept drift on object detection performance. We used an extension of the Long-Term Thermal Imaging Dataset, composed of thermal data acquired from 14th May 2020 to 30th of April 2021, with a total of 1689 2-min clips with bounding-box annotations for 4 different categories. The data covers a wide range of different weather conditions and object densities with the goal of measuring the thermal drift over time, from the coldest day/week/month of the dataset. The challenge attracted 184 registered participants, which was considered a success from the perspective of the organizers. While participants managed to achieve higher mAP when compared to a baseline, concept drift remains a strongly impactful factor. This work describes the challenge design, the adopted dataset and obtained results, as well as discuss top-winning solutions and future directions on the topic.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
Track 1 (on Codalab) - https://codalab.lisn.upsaclay.fr/competitions/4272.
- 3.
Track 2 (on Codalab) - https://codalab.lisn.upsaclay.fr/competitions/4273.
- 4.
Track 3 (on Codalab) - https://codalab.lisn.upsaclay.fr/competitions/4276.
- 5.
- 6.
- 7.
- 8.
Codalab - https://codalab.lisn.upsaclay.fr.
References
Bochkovskiy, A., Wang, C., Liao, H.M.: Yolov4: optimal speed and accuracy of object detection. CoRR abs/2004.10934 (2020)
Bodla, N., Singh, B., Chellappa, R., Davis, L.S.: Soft-NMS - improving object detection with one line of code. In: ICCV (2017)
Buslaev, A.V., Iglovikov, V.I., Khvedchenya, E., Parinov, A., Druzhinin, M., Kalinin, A.A.: Albumentations: fast and flexible image augmentations. Information 11(2), 125 (2020)
Cai, Z., Vasconcelos, N.: Cascade R-CNN: delving into high quality object detection. In: CVPR (2018)
Chen, K., et al.: MMDetection: open MMLab detection toolbox and benchmark. CoRR abs/1906.07155 (2019)
Chen, Y.Y., Jhong, S.Y., Li, G.Y., Chen, P.H.: Thermal-based pedestrian detection using faster r-cnn and region decomposition branch. In: ISPACS (2019)
Dai, D., Van Gool, L.: Dark model adaptation: semantic image segmentation from daytime to nighttime. In: ITSC (2018)
Dai, X., Yuan, X., Wei, X.: Tirnet: object detection in thermal infrared images for autonomous driving. Appl. Intell. 51(3), 1244–1261 (2021)
Davis, J.W., Keck, M.A.: A two-stage template approach to person detection in thermal imagery. In: WACV-W (2005)
Devaguptapu, C., Akolekar, N., M Sharma, M., N Balasubramanian, V.: Borrow from anywhere: pseudo multi-modal object detection in thermal imagery. In: CVPR-W (2019)
Ghiasi, G., et al.: Simple Copy-Paste is a strong data augmentation method for instance segmentation. In: CVPR (2021)
Han, W., et al.: Seq-NMS for video object detection. CoRR abs/1602.08465 (2016)
Hwang, S., Park, J., Kim, N., Choi, Y., So Kweon, I.: Multispectral pedestrian detection: benchmark dataset and baseline. In: CVPR (2015)
Jia, X., Zhu, C., Li, M., Tang, W., Zhou, W.: Llvip: a visible-infrared paired dataset for low-light vision. In: ICCV (2021)
Kieu, M., Bagdanov, A.D., Bertini, M., del Bimbo, A.: Task-conditioned domain adaptation for pedestrian detection in thermal imagery. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12367, pp. 546–562. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58542-6_33
Kim, J., Kim, H., Kim, T., Kim, N., Choi, Y.: Mlpd: multi-label pedestrian detector in multispectral domain. Rob. Autom. Lett. 6(4), 7846–7853 (2021)
Krišto, M., Ivasic-Kos, M., Pobar, M.: Thermal object detection in difficult weather conditions using yolo. IEEE Access 8, 125459–125476 (2020)
Liang, T., et al.: Cbnetv2: a composite backbone network architecture for object detection. CoRR abs/2107.00420 (2021)
Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48
Liu, Z., et al.: Swin transformer: hierarchical vision transformer using shifted windows. In: ICCV (2021)
Nikolov, I., et al.: Seasons in drift: a long-term thermal imaging dataset for studying concept drift. In: NeurIPS (2021)
Pavao, A., et al.: CodaLab Competitions: An open source platform to organize scientific challenges. Ph.D. thesis, Université Paris-Saclay, FRA (2022)
Siris, A., Jiao, J., Tam, G.K., Xie, X., Lau, R.W.: Scene context-aware salient object detection. In: ICCV (2021)
Telodyne: FLIR AADAS Dataset. https://www.flir.com/oem/adas/adas-dataset-form/
Vertens, J., Zürn, J., Burgard, W.: Heatnet: bridging the day-night domain gap in semantic segmentation with thermal images. In: IROS (2020)
Wang, C., Bochkovskiy, A., Liao, H.: Scaled-YOLOv4: scaling cross stage partial network. In: CVPR (2021)
Wortsman, M., et al.: Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. CoRR abs/2203.05482 (2022)
Wu, X., Wu, Z., Guo, H., Ju, L., Wang, S.: Dannet: a one-stage domain adaptation network for unsupervised nighttime semantic segmentation. In: CVPR (2021)
Zhang, H., Fromont, E., Lefevre, S., Avignon, B.: Multispectral fusion for object detection with cyclic fuse-and-refine blocks. In: ICIP (2020)
Zhang, H., Fromont, E., Lefèvre, S., Avignon, B.: Guided attentive feature fusion for multispectral pedestrian detection. In: WACV (2021)
Zhang, H., Cissé, M., Dauphin, Y.N., Lopez-Paz, D.: mixup: beyond empirical risk minimization. CoRR abs/1710.09412 (2017)
Acknowledgements
This work has been partially supported by Milestone Research Program at AAU, the Spanish project PID2019-105093GB-I00 and by ICREA under the ICREA Academia programme.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Johansen, A.S., Junior, J.C.S.J., Nasrollahi, K., Escalera, S., Moeslund, T.B. (2023). ChaLearn LAP Seasons in Drift Challenge: Dataset, Design and Results. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds) Computer Vision – ECCV 2022 Workshops. ECCV 2022. Lecture Notes in Computer Science, vol 13805. Springer, Cham. https://doi.org/10.1007/978-3-031-25072-9_50
Download citation
DOI: https://doi.org/10.1007/978-3-031-25072-9_50
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-25071-2
Online ISBN: 978-3-031-25072-9
eBook Packages: Computer ScienceComputer Science (R0)