Skip to main content
Springer Nature Link
Log in

Dynamic node selection in camera networks based on approximate reinforcement learning

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In camera networks, dynamic node selection is an effective technique that enables video stream transmission with constrained network bandwidth, more economical node cooperation for nodes with constrained power supplies, and optimal use of a limited number of display terminals, particularly for applications that need to obtain high-quality video of specific targets. However, the nearest camera in a network cannot be identified by directional measurements alone. Furthermore, errors are introduced into computer vision algorithms by complex background, illumination, and other factors, causing unstable and jittery processing results. Consequently, in selecting camera network nodes, two issues must be addressed: First, a dynamic selection mechanism that can choose the most appropriate node is needed. Second, metrics to evaluate the visual information in a video stream must be modeled and adapted to various camera parameters, backgrounds, and scenes. This paper proposes a node selection method based on approximate reinforcement learning in which nodes are selected to obtain the maximum expected reward using approximate Q-learning. The Q-function is approximated by a Gaussian Mixture Model with parameters that are sequentially updated by a mini-batch stepwise Expectation–Maximization algorithm. To determine the most informative camera node dynamically, the immediate reward in Q-learning integrates the visibility, orientation, and image clarity of the object in view. Experimental results show that the proposed visual evaluation metrics can effectively capture the motion state of objects, and that the selection method reduces camera switching and related errors compared with state-of-the art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Agarwal A, Triggs B (2006) Recovering 3D human pose from monocular images. IEEE Trans Patt Anal Mach Intell 28(1):44–58. doi:10.1109/TPAMI.2006.21

    Article  Google Scholar 

  2. Agostini A, Celaya E (2010) Reinforcement learning with a Gaussian mixture model. In Neural Networks (IJCNN), The 2010 Int Joint Conf : 1–8. doi: 10.1109/IJCNN.2010.5596306

  3. Botvinick M, Niv Y, Barto A (2009) Hierarchically organized behavior and its neural foundations: a reinforcement learning perspective. Cognition 113(3):262–280. doi:10.1016/j.cognition.2008.08.011

    Article  Google Scholar 

  4. Busoniu L, Babuska R, DeSchutter B, Ernst D (2010) Reinforcement learning and dynamic programming using function approximators. CRC press. doi: 10.1201/9781439821091

  5. Charfi Y, Wakamiya N, Murata M (2009) Challenging issues in visual sensor networks. IEEE Wirel Commun 16(2):44. doi:10.1109/MWC.2009.4907559

    Article  Google Scholar 

  6. Cohn D, Ghahramani Z, Jordan M (1996) Active learning with statistical models. J Artif Intell Res 4(1):129–145

  7. Daniyal F, Taj M, Cavallaro A (2010) Content and task-based view selection from multiple video streams. Multimed Tools Applic 46:235–258. doi:10.1007/s11042-009-0355-z

    Article  Google Scholar 

  8. Fu Y, Guo Y, Zhu Y (2010) Multi-view video summarization. IEEE Trans Multimed 12(7):717–729. doi:10.1109/TMM.2010.2052025

    Article  Google Scholar 

  9. Gupta A, Mittal A, Davis L (2007) Cost: An approach for camera selection and multi-object inference ordering in dynamic scenes. Proceedings of IEEE International Conference on Computer Vision, Rio de Janeiro: 1–8. doi:10.1109/ICCV.2007.4408842

  10. Han B, Joo S, Davis L (2011) Multi-camera tracking with adaptive resource allocation. Int J Comput Vis 91(1):45–58. doi:10.1007/s11263-010-0373-3

  11. Huber M (2012) Optimal pruning for multi-step sensor scheduling. IEEE Trans Autom Control 57(5):1338–1343. doi:10.1109/TAC.2011.2175070

    Article  MathSciNet  Google Scholar 

  12. Javed O, Khan S, Rasheed Z (2000) Camera handoff: tracking in multiple uncalibrated stationary cameras. Proc Workshop Human Motion, IEEE: 113–118. doi: 10.1109/HUMO.2000.897380

  13. Jiang H, Fels S, Little J (2008) Optimizing multiple object tracking and best view video synthesis. IEEE Trans Multimed 10(6):997–1012. doi:10.1109/TMM.2008.2001379

    Article  Google Scholar 

  14. Kveton B, Theocharous G (2012) Kernel-based reinforcement learning on representative states. Association for the Advancement of Artificial Intelligence, pp 977–983

  15. Kwon J, Lee K (2010) Visual tracking decomposition. IEEE Conf Comput Vision Patt Recognit: 1269–1276. doi: 10.1109/CVPR.2010.5539821

  16. Li Y, Bhanu B (2011) Utility-based camera assignment in a video network: a game theoretic framework. IEEE Sensors J 11(3):676–687. doi:10.1109/JSEN.2010.2051148

    Article  Google Scholar 

  17. Li Q, Sun Z, Chen S, Liu Y (2013) A method of camera selection based on partially observable Markov decision process model in camera networks. Ame Contrl Conf: 3833–3839. doi:10.1109/ACC.2013.6580424

  18. Li W, Zhang W (2012) Sensor selection for improving accuracy of target localization in wireless visual sensor networks. IET Wireless Sens Syst 2(4):293–301. doi:10.1049/iet-wss.2012.0033

    Article  Google Scholar 

  19. Liang P, Klein D (2009) Online EM for unsupervised models. In Proceedings of human language technologies: The 2009 annual conference of the North American chapter of the association for computational linguistics. Assoc Comput Linguist: 611–619. doi: 10.3115/1620754.1620843

  20. McLachlan G, Peel D (2004) Finite mixture models. New York: J. Wiley. doi:10.1002/0471721182

  21. Mo Y, Ambrosino R, Sinopoli B (2011) Sensor selection strategies for state estimation in energy constrained wireless sensor networks. Automatica 47(7):1330–1338. doi:10.1016/j.automatica.2011.02.001

    Article  MathSciNet  MATH  Google Scholar 

  22. Monari E, Kroschel K (2009) A knowledge-based camera selection approach for object tracking in large sensor networks. Third ACM/IEEE Int Conf Distrib Smart Cam. doi:10.1109/ICDSC.2009.5289400

    Google Scholar 

  23. Park J, Bhat C, Kak A (2006) A look-up table based approach for solving the camera selection problem in large camera networks. In IEEE Workshop on Distributed Smart Cameras, Boulder, CO, USA, Oct. 31, 2006

  24. Rudoy D, Zelnik-Manor L (2012) Viewpoint selection for human actions. Int J Comput Vis 97(3):243–254. doi:10.1007/s11263-011-0484-5

    Article  Google Scholar 

  25. Rudoy D, Zelnik-Manor L (2012) Viewpoint selection for human actions. Int J Comput Vis 97(3):243–254. doi:10.1007/s11263-011-0484-5

    Article  Google Scholar 

  26. Sato M, Ishii S (2000) On-line EM algorithm for the normalized Gaussian network. Neural Comput 12(2):407–432. doi:10.1162/089976600300015853

    Article  Google Scholar 

  27. Shen C, Zhang C, Fels S (2007) A multi-camera surveillance system that estimates quality-of-view measurement. Proc IEEE Int Conf Image Process, San Antonio: III 193-III 196. doi:10.1109/ICIP.2007.4379279

  28. Singh S, Jaakkola T, Jordan M (1995) Reinforcement learning with soft state aggregation. Advances in neural information processing systems: 361–368. doi: 10.1162/089976600300015961

  29. Soro S, Heinzelman W (2007) Camera selection in visual sensor networks. Proc IEEE Conf Adv Video Signal Based Surveill: 81–86. doi: 10.1109/AVSS.2007.4425290

  30. Soro S, Heinzelman W (2009) A survey of visual sensor networks. Adv Multimed: 1–22. doi: 10.1155/2009/640386

  31. Spaan M, Lima P (2009) A decision-theoretic approach to dynamic sensor selection in camera networks. In ICAPS: 1–8. doi: ocs/index.php/ICAPS/ICAPS09/paper/viewFile/758/1125

  32. Sutton R, Barto A (1998) Reinforcement learning: an introduction. MIT Press. Cambridge, Massachusetts London, England. doi:10.1109/TNN.1998.712192

  33. Tessens L, Morbee M, Lee H, Philips W, Aghajan H (2008) Principal view determination for camera selection in distributed smart camera networks. Sec ACM/IEEE Int Conf Distrib Smart Cam: 1–10. doi:10.1109/ICDSC.2008.4635699

  34. Watkins C, Dayan P (1992) Q-learning. Mach Learn 8(3):279–292. doi:10.1007/BF00992698

    MATH  Google Scholar 

Download references

Acknowledgments

This work is supported by the National Natural Science Foundation of China (61272219, 61100110, 41305138, 61473310 and 61321491), Program for New Century Excellent Talents of Ministry of Education of China (NCET-04-0460), Science and Technology Plan of Jiangsu Province (BE2010072, BE2011058, BY2012190, and BY2013072-04), and Innovation Foundation of State Key Lab for Novel Software Technology of China (ZZKT2013A12).

Author information

Authors and Affiliations

  1. State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, 210093, China

    Qian Li, Zhengxing Sun & Songle Chen

  2. College of Meteorology and Oceanography, PLA University of Science and Technology, Nanjing, 211101, China

    Qian Li & Shiming Xia

Authors
  1. Qian Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhengxing Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Songle Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shiming Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhengxing Sun.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Q., Sun, Z., Chen, S. et al. Dynamic node selection in camera networks based on approximate reinforcement learning. Multimed Tools Appl 75, 17393–17419 (2016). https://doi.org/10.1007/s11042-015-3003-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-3003-9

Keywords