Skip to main content
SpringerLink
Log in

Lightweight Model Inference on Resource-Constrained Computing Nodes in Intelligent Surveillance Systems

  • Conference paper
  • First Online:
Web and Big Data (APWeb-WAIM 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13421))

  • 892 Accesses

Abstract

Intelligent Surveillance System (ISS) is an important application combining deep learning with IoT technologies. Meanwhile, multiple targets multiple camera tracking (MTMCT) has been widely recognized as a promising solution for ISS. However, current terminal devices have limited memory, power and computing power. Deep learning models deployed on these resource-constrained devices need to maintain a low-level number of parameters while ensuring inference delay and accuracy as much as possible. In this paper, we propose a lightweight model inference approach for resource-constrained edge computing nodes, which leverages the limited computing capability of edge devices for collaborative processing of model inference tasks. We consider a system model that includes a combination of both horizontal and vertical methods to dynamically partition deep neural networks. Further, we propose an adaptive strategy that dynamically controls the execution part in the terminal devices. In addition, a learning-based algorithm is used to obtain the near-optimal solutions for dynamic resource allocation decisions. Finally, we evaluate the system in terms of two metrics, i.e., computational latency and network throughput, based on a real-world dataset of campus surveillance. The experimental results reveal the superior effectiveness of the proposed scheme.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Change history

  • 10 February 2023

    A correction has been published.

References

  1. Bertinetto, L., Valmadre, J., Henriques, J.F., Vedaldi, A., Torr, P.H.S.: Fully-convolutional Siamese networks for object tracking. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9914, pp. 850–865. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-48881-3_56

    Chapter  Google Scholar 

  2. Chang, H.C., Hsu, Y.L., Hsiao, C.Y., Chen, Y.F.: Design and implementation of an intelligent autonomous surveillance system for indoor environments. IEEE Sens. J. 21(15), 17335–17349 (2021)

    Article  Google Scholar 

  3. Chou, Y.S., Wang, C.Y., Chen, M.C., Lin, S.D., Liao, H.Y.M.: Dynamic gallery for real-time multi-target multi-camera tracking, pp. 1–8 (2019)

    Google Scholar 

  4. Das, S., Dereniowski, D., Uznanski, P.: Brief announcement: energy constrained depth first search, vol. 107, pp. 1–5 (2018)

    Google Scholar 

  5. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014)

    Google Scholar 

  6. Gündogan, A., Gürsu, H.M., Pauli, V., Kellerer, W.: Distributed resource allocation with multi-agent deep reinforcement learning for 5g–v2v communication. ACM (2020)

    Google Scholar 

  7. He, L., Liu, G., Tian, G., Zhang, J., Ji, Z.: Efficient multi-view multi-target tracking using a distributed camera network. IEEE Sens. J. 20(4), 2056–2063 (2020)

    Article  Google Scholar 

  8. He, W., Guo, S., Guo, S., Qiu, X., Qi, F.: Joint DNN partition deployment and resource allocation for delay-sensitive deep learning inference in IoT. IEEE Internet Things J. 7(10), 9241–9254 (2020)

    Article  Google Scholar 

  9. He, Y., Wei, X., Hong, X., Shi, W., Gong, Y.: Multi-target multi-camera tracking by tracklet-to-target assignment. IEEE Trans. Image Process. 29, 5191–5205 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  10. Hu, C., Bao, W., Wang, D., Liu, F.: Dynamic adaptive DNN surgery for inference acceleration on the edge. In: IEEE INFOCOM 2019-IEEE Conference on Computer Communications, pp. 1423–1431 (2019)

    Google Scholar 

  11. Islam, K.A., Hill, V., Schaeffer, B., Zimmerman, R., Li, J.: Semi-supervised adversarial domain adaptation for seagrass detection using multispectral images in coastal areas. Data Sci. Eng. 5(2), 111–125 (2020)

    Article  Google Scholar 

  12. Kang, Y., et al.: Neurosurgeon: Collaborative intelligence between the cloud and mobile edge. ACM SIG. Comput. Archit. News 45(1), 615–629 (2017)

    Article  Google Scholar 

  13. Ke, R., Zhuang, Y., Pu, Z., Wang, Y.: A smart, efficient, and reliable parking surveillance system with edge artificial intelligence on IoT devices. IEEE Trans. Intell. Transp. Syst. 22(8), 4962–4974 (2021)

    Article  Google Scholar 

  14. Kim, H., Cha, Y., Kim, T., Kim, P.: A study on the security threats and privacy policy of intelligent video surveillance system considering 5G network architecture. In: 2020 International Conference on Electronics, Information, and Communication (ICEIC), pp. 1–4 (2020)

    Google Scholar 

  15. Li, D., Zhang, Z., Yu, K., Huang, K., Tan, T.: ISEE: An intelligent scene exploration and evaluation platform for large-scale visual surveillance. IEEE Trans. Parallel Distrib. Syst. 30(12), 2743–2758 (2019)

    Article  Google Scholar 

  16. Li, E., Zhou, Z., Chen, X.: Edge intelligence: on-demand deep learning model co-inference with device-edge synergy. In: Workshop on Mobile Edge Communications, pp. 31–36. ACM (2018)

    Google Scholar 

  17. Lu, S., Yao, Y., Shi, W.: Clone: Collaborative learning on the edges. IEEE Internet Things J. 7, 10222–10236 (2020)

    Google Scholar 

  18. Mao, J., Chen, X., Nixon, K.W., Krieger, C., Chen, Y.: MoDNN: local distributed mobile computing system for deep neural network. In: Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 1396–1401 (2017)

    Google Scholar 

  19. Mohammed, T., Joe-Wong, C., Babbar, R., Francesco, M.D.: Distributed Inference Acceleration with Adaptive DNN Partitioning and Offloading. IEEE INFOCOM, pp. 854–863 (2020)

    Google Scholar 

  20. Muchtar, K., Afdhal, A., Nasaruddin, N.: Convolutional network and moving object analysis for vehicle detection in highway surveillance videos. In: 2020 3rd International Seminar on Research of Information Technology and Intelligent Systems (ISRITI), pp. 509–513 (2020)

    Google Scholar 

  21. Redmon, J., Farhadi, A.: Yolo9000: Better, faster, stronger. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7263–7271 (2017)

    Google Scholar 

  22. Tian, B., et al.: Hierarchical and networked vehicle surveillance in its: a survey. IEEE Trans. Intell. Transp. Syst. 16(2), 557–580 (2015)

    Article  Google Scholar 

  23. Winter, K., Wien, J., Molin, E., Cats, O., Morsink, P., van Arem, B.: Taking the self-driving bus: a passenger choice experiment. In: 2019 6th International Conference on Models and Technologies for Intelligent Transportation Systems (MT-ITS) (2019)

    Google Scholar 

  24. Xie, J., et al.: Deep learning-based computer vision for surveillance in its: evaluation of state-of-the-art methods. IEEE Trans. Veh. Technol. 70(4), 3027–3042 (2021)

    Article  Google Scholar 

  25. Yoo, H., Kim, K., Byeon, M., Jeon, Y., Choi, J.Y.: Online scheme for multiple camera multiple target tracking based on multiple hypothesis tracking. IEEE Trans. Circuits Syst. Video Technol. 27(3), 454–469 (2017)

    Article  Google Scholar 

  26. You, S., Yao, H., Xu, C.: Multi-target multi-camera tracking with optical-based pose association. IEEE Trans. Circuits Syst. Video Technol. 31(8), 3105–3117 (2021)

    Article  Google Scholar 

  27. Zeng, X., Fang, B., Shen, H., Zhang, M.: Distream: scaling live video analytics with workload-adaptive distributed edge intelligence. In: Proceedings of the 18th Conference on Embedded Networked Sensor Systems, pp. 409–421 (2020)

    Google Scholar 

  28. Zhao, Z., Barijough, K.M., Gerstlauer, A.: Deepthings: Distributed adaptive deep learning inference on resource-constrained IoT edge clusters. IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 37(11), 2348–2359 (2018)

    Article  Google Scholar 

  29. Zhou, X., Xu, X., Liang, W., Zeng, Z., Yan, Z.: Deep-learning-enhanced multitarget detection for end-edge-cloud surveillance in smart IoT. IEEE Internet Things J. 8(16), 12588–12596 (2021)

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by research on key technologies of electrical cloud-edge-end collaborative AI model sharing in Science and Technology Project of State Grid Headquarters (2021, Power base support technology-30, Shandong Electric Power Company, No. 520627210010).

Author information

Authors and Affiliations

  1. College of Intelligence and Computing, Tianjin University, Tianjin, China

    Zhuohang Wang, Yunfeng Zhao, Zhicheng Liu, Chao Qiu, Xiaofei Wang & Qinghua Hu

  2. Information & Telecommunications Company State grid Shandong electric power Company, Jinan Shandong, China

    Yong Wang & Li Yan

Authors
  1. Zhuohang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yunfeng Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Li Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhicheng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chao Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Xiaofei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Qinghua Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaofei Wang .

Editor information

Editors and Affiliations

  1. Nanjing University of Aeronautics and Astronautics, Nanjing, China

    Bohan Li

  2. Newcastle University, Callaghan, NSW, Australia

    Lin Yue

  3. Nanjing University of Aeronautics and Astronautics, Nanjing, China

    Chuanqi Tao

  4. Jinan University, Guangzhou, China

    Xuming Han

  5. Free University of Bozen-Bolzano, Bolzano, Italy

    Diego Calvanese

  6. University of Tsukuba, Tsukuba, Japan

    Toshiyuki Amagasa

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, Z. et al. (2023). Lightweight Model Inference on Resource-Constrained Computing Nodes in Intelligent Surveillance Systems. In: Li, B., Yue, L., Tao, C., Han, X., Calvanese, D., Amagasa, T. (eds) Web and Big Data. APWeb-WAIM 2022. Lecture Notes in Computer Science, vol 13421. Springer, Cham. https://doi.org/10.1007/978-3-031-25158-0_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-25158-0_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-25157-3

  • Online ISBN: 978-3-031-25158-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation