Abstract
In the field of Intelligent Transportation Systems (ITS), the challenge of performance degradation in lightweight object detection models on edge devices is significant. This issue primarily arises from environmental changes and shifts in data distribution. The problem is twofold: the limited computational capacity of edge devices, which hinders timely model updates, and the inherent limitations in the generalization capabilities of lightweight models. While large-scale models may have superior generalization, their deployment at the edge is impractical due to computational constraints. To address this challenge, we propose a cloud-edge collaborative continual adaptation learning framework, specifically designed for the DETR model family, aimed at enhancing the generalization ability of lightweight edge models. This framework uses visual prompts to collect and upload data from the edge, which helps to fine-tune cloud-based models for improved target domain generalization. The refined knowledge is then distilled back into the edge models, enabling continuous adaptation to diverse and dynamic conditions. The effectiveness of this approach has been validated through extensive experiments on two datasets for traffic object detection in dynamic environments. The results indicate that our learning method outperforms existing techniques in continual adaptation and cloud-edge collaboration, highlighting its potential in addressing the challenges posed by dynamic environmental changes in ITS.
Z. Lian and M. Lv—have contributed equally to this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12346, pp. 213–229. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58452-8_13
Chinchali, S., et al.: Network offloading policies for cloud robotics: a learning-based approach. Autonomous Robots, Autonomous Robots (2019)
Crankshaw, D., Wang, X., Zhou, G., Franklin, M.J., Gonzalez, J.E., Stoica, I.: Clipper: a \(\{\)Low-Latency\(\}\) online prediction serving system. In: 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17), pp. 613–627 (2017)
Du, K., et al.: Server-driven video streaming for deep learning inference. In: Proceedings of the Annual Conference of the ACM Special Interest Group on Data Communication on the Applications, Technologies, Architectures, and Protocols for Computer Communication, pp. 557–570 (2020)
Gan, Y., et al.: Decorate the newcomers: visual domain prompt for continual test time adaptation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, pp. 7595–7603 (2023)
Gan, Y., et al.: Cloud-device collaborative adaptation to continual changing environments in the real-world. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12157–12166 (2023)
Jia, M., et al.: Visual prompt tuning. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) ECCV 2022. LNCS, vol. 13693, pp. 709–727. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-19827-4_41
Khani, M., Hamadanian, P., Nasr-Esfahany, A., Alizadeh, M.: Real-time video inference on edge devices via adaptive model streaming. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4572–4582 (2021)
Li, A., Sun, S., Zhang, Z., Feng, M., Wu, C., Li, W.: A multi-scale traffic object detection algorithm for road scenes based on improved yolov5. Electronics 12(4), 878 (2023)
Li, Y., Padmanabhan, A., Zhao, P., Wang, Y., Xu, G.H., Netravali, R.: Reducto: on-camera filtering for resource-efficient real-time video analytics. In: Proceedings of the Annual Conference of the ACM Special Interest Group on Data Communication on the Applications, Technologies, Architectures, and Protocols for Computer Communication, pp. 359–376 (2020)
Liu, X., et al.: P-tuning: prompt tuning can be comparable to fine-tuning across scales and tasks. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 61–68 (2022)
Madhuri, T., Sowjanya, P.: Microsoft azure v/s amazon AWS cloud services: a comparative study. Int. J. Innov. Res. Sci. Eng. Technol. 5(3), 3904–3907 (2016)
Maltoni, D., Lomonaco, V.: Continuous learning in single-incremental-task scenarios. Neural Netw. 116, 56–73 (2019)
McCloskey, M., Cohen, N.J.: Catastrophic interference in connectionist networks: the sequential learning problem. In: Psychology of Learning and Motivation, vol. 24, pp. 109–165. Elsevier (1989)
Pseudo-Label, D.H.L.: The simple and efficient semi-supervised learning method for deep neural networks. In: ICML 2013 Workshop: Challenges in Representation Learning, pp. 1–6 (2013)
Shao, M., Fang, Y., Guo, L., Xue, Q.: Research on yolov5 vehicle object detection algorithm based on attention mechanism. In: 2022 3rd International Conference on Big Data, Artificial Intelligence and Internet of Things Engineering (ICBAIE), pp. 609–612. IEEE (2022)
Smith, J.S., et al.: Coda-prompt: continual decomposed attention-based prompting for rehearsal-free continual learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11909–11919 (2023)
Wang, Q., Fink, O., Van Gool, L., Dai, D.: Continual test-time domain adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7201–7211 (2022)
Wang, W., et al.: Exploring sequence feature alignment for domain adaptive detection transformers. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 1730–1738 (2021)
Woo, S., Park, J., Lee, J.Y., Kweon, I.S.: CBAM: convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19 (2018)
Yao, J., Wang, F., Jia, K., Han, B., Zhou, J., Yang, H.: Device-cloud collaborative learning for recommendation. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 3865–3874 (2021)
Zhang, X., Gu, S.S., Matsuo, Y., Iwasawa, Y.: Domain prompt learning for efficiently adapting clip to unseen domains. Trans. Japanese Soc. Artif. Intell. 38(6), B–MC2_1 (2023)
Zhao, P., et al.: Neural pruning search for real-time object detection of autonomous vehicles. In: 2021 58th ACM/IEEE Design Automation Conference (DAC), pp. 835–840. IEEE (2021)
Zhu, X., Su, W., Lu, L., Li, B., Wang, X., Dai, J.: Deformable DETR: deformable transformers for end-to-end object detection. In: International Conference on Learning Representations (2020)
Zong, Z., Song, G., Liu, Y.: DETRs with collaborative hybrid assignments training. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6748–6758 (2023)
Acknowledgments
This work is supported by National Key R & D Program of China (No. 2022YFF0503900) and Key R & D Program of Shandong Province (No. 2021CXGC010104).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Lian, Z. et al. (2024). Cloud-Edge Collaborative Continual Adaptation for ITS Object Detection. In: Meng, X., Zhang, X., Guo, D., Hu, D., Zheng, B., Zhang, C. (eds) Spatial Data and Intelligence. SpatialDI 2024. Lecture Notes in Computer Science, vol 14619. Springer, Singapore. https://doi.org/10.1007/978-981-97-2966-1_2
Download citation
DOI: https://doi.org/10.1007/978-981-97-2966-1_2
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-2965-4
Online ISBN: 978-981-97-2966-1
eBook Packages: Computer ScienceComputer Science (R0)