Skip to main content
Springer Nature Link
Log in

Real-time wireless multisensory smart surveillance with 3D-HEVC streams for internet-of-things (IoT)

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

This paper presents the design of a novel, real-time, wireless, multisensory, smart surveillance system with 3D-HEVC features. The proposed high-level system architecture of the surveillance system is analyzed. The advantages of HEVC encoding are presented. Methods for synchronization between multiple streams are presented. Available wireless standards are presented and compared. A network-adaptive transmission protocol for a reliable, real-time, multisensory surveillance system is proposed. Adaptive packet frame grouping (APFG) and adaptive quantization are deployed to maximize the quality-of-experience (QoE). Measurements of the proposed protocol have been shown to provide superior results compared to existing transport protocols.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. I. D. N. Enterprise, R. I. G. Micro, Nanosystems, W. G. R. of the ETP EPOSS (2008) Internet of things in 2020: roadmap for the future. Version 1.1. Tech. Rep. 27, May 2008

  2. Garcia R, Kalva H (2014) Subjective evaluation of HEVC and AVC/H.264 in mobile environments. IEEE Trans Consum Electron 60(1):116–123

    Article  Google Scholar 

  3. Valera M, Velastin SA (2005) Intelligent distributed surveillance systems: a review. In: IEE proceedings—vision, image and signal processing, vol 152. No. 2. IET, 2005

  4. Kim HI, Lee SH, Ro YM (2016) Low-power face detection for smart camera. In: Theory and applications of smart cameras, pp 139–155

  5. Collins RT, Lipton AJ, Kanade T, Fujiyoshi H, Duggins D, Tsin Y, Tolliver D, Enomoto N, Hasegawa O, Burt P, Wixson L (2000) A system for video surveillance and monitoring. Carnegie Mellon University Technical Report, CMU-RI-TR-00-12, 2000

  6. Siebel NT, Maybank S (2004) The advisor visual surveillance system. In: Proc. of the ECCV workshop on applications of computer vision, pp 103–111

  7. Shu CF, Hampapur A, Lu M, Brown L, Connell J, Senior A, Tian Y (2005) IBM smart surveillance system (S3): an open and extensible framework for event based surveillance. In: Proc. of IEEE Conf. advanced video and signal based surveillance, pp 318–323

  8. Regazzoni C, Ramesh V, Foresti GL (2001) Special issue on video communications, processing, and understanding for third generation surveillance systems. Proc IEEE 89(10):1355–1367

    Article  Google Scholar 

  9. Dong X, Wen J (2016) Low lighting image enhancement using local maximum color value prior. Front Comput Sci 10(1):147–156

    Article  Google Scholar 

  10. Sanjay S, Shekhar C, Vohra A (2016) FPGA-based real-time motion detection for automated video surveillance systems. Electronics 5(1):10

    Article  Google Scholar 

  11. Guo DJ, Zhe-Ming L, Hao L (2016) Multi-channel adaptive mixture background model for real-time tracking. J Inform Hiding Multimed Signal Process 7(1):216–221

    Google Scholar 

  12. Maadi AE, Djouadi MS (2016) Large-scale surveillance system: detection and tracking of suspicious motion patterns in crowded traffic scenes. Automatika J Control Meas Electron Comput Commun 60(1):1–15

    Google Scholar 

  13. Zhengguo S, Shusen Y, Yu Y, Vasilakos A, McCann J, Kin L (2013) A survey on the IETF protocol suite for the internet of things: standards, challenges, and opportunities. IEEE Wirel Commun 20(6):91–98

    Article  Google Scholar 

  14. Mongay Batalla J (2015) Advanced multimedia service provisioning based on efficient interoperability of adaptive streaming protocol and high efficient video coding. J Real Time Image Process. doi:10.1007/s11554-015-0496-4

  15. Kurniawati E, Lau C, Premkumar B, Absar J, George S (2004) New implementation techniques of an efficient MPEG advanced audio coder. IEEE Trans Consumer Electron 50(2):655–665

    Article  Google Scholar 

  16. Ohm J, Sullivan G, Schwarz H, Tan TK, Wiegand T (2012) Comparison of the coding efficiency of video coding standards—including high efficiency video coding (HEVC). IEEE Trans Circuits Syst Video Technol 22(12):1669–1684

    Article  Google Scholar 

  17. Fitzek F, Reisslein M, Video traces for network performance evaluation: Yuv 4:2:0 video sequences. http://trace.eas.asu.edu/-yuv/-yuv.html

  18. Kokkonis G, Psannis KE, Roumeliotis M, Ishibashi Y (2015) Efficient algorithm for transferring a real-time HEVC stream with haptic data through the internet. J Real Time Image Process. doi:10.1007/s11554-015-0505-7

  19. Zhang Z (2012) Microsoft kinect sensor and its effect. IEEE Multimed 19(2):4–10

    Article  Google Scholar 

  20. Silberman N, Hoiem D, Kohli P, Fergus R (2012) Indoor segmentation and support inference from RGBD images. In: Fitzgibbon A, Lazebnik S, Perona P, Sato Y, Schmid C (eds) Proceedings of 12th European Conference on Computer Vision, October 2012. Springer, Berlin, pp 746–760

  21. Ishibashi Y, Kanbara T, Tasaka S (2004) Inter-stream synchronization between haptic media and voice in collaborative virtual environments. In: Proc. of the 12th annual ACM Int. Conf. on multimedia, pp 604–611

  22. Lee J-S, Su Y-W, Shen C-C (2007) A comparative study of wireless protocols: bluetooth, UWB, ZigBee, and Wi-Fi. In: Industrial Electronics Society, 2007. IECON 2007. 33rd annual conference of the IEEE, Nov 2007, pp 46–51

  23. Ong EH, Kneckt J, Alanen O, Chang Z, Huovinen T, Nihtila T (2011) IEEE 802.11ac: enhancements for very high throughput WLANs. In: IEEE 22nd Int. Symp. on personal indoor and mobile radio communications (PIMRC), Sept 2011, pp 849–853

  24. Huang J, Qian F, Gerber A, Mao ZM, Sen S, Spatscheck O (2012) A close examination of performance and power characteristics of 4G LTE networks. In: Proc. 10th international conference on mobile systems, applications, and services, 2012, pp 225–238

  25. Andrews J, Buzzi S, Choi W, Hanly S, Lozano A, Soong A, Zhang J (2014) What will 5G be? IEEE J Sel Areas Commun 32(6):1065–1082

    Article  Google Scholar 

  26. Tourrilhes J (1998) Packet frame grouping: improving IP multimedia performance over CSMA/CA. In: IEEE Int. Conf. on universal personal communications, Oct 1998, vol 2, pp 1345–1349

  27. Iwata K, Ishibashi Y, Fukushima N, Sugawara S (2010) Qoe assessment in haptic media, sound, and video transmission: effect of playout buffering control. Comput Entertain (CIE) 8(2):12

    Google Scholar 

  28. Guiyun L, Yao J, Liu Y, Chen H, Tang D (2015) Channel-aware adaptive quantization method for source localization in wireless sensor networks. Int J Distrib Sensor Netw 214081:13

    Google Scholar 

  29. Baek J, Kim C (2014) An energy-efficient video transport protocol for personal cloud-based computing. J Real Time Image Process. doi:10.1007/s11554-014-0475-1

  30. Mulabegovic E, Schonfeld D, Ansari R (2002) Lightweight streaming protocol (LSP). In: Proc. 10th ACM Int. Conf. on multimedia, MULTIMEDIA ’02, pp 227–230

Download references

Author information

Authors and Affiliations

  1. Department of Applied Informatics, University of Macedonia, 156 Egnatia Street, 54006, Thessaloniki, Greece

    George Kokkonis, Kostas E. Psannis & Manos Roumeliotis

  2. Department of Electrical and Computer Engineering, University of Illinois at Chicago, Chicago, IL, 60607-7053, USA

    Dan Schonfeld

Authors
  1. George Kokkonis
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Kostas E. Psannis
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Manos Roumeliotis
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Dan Schonfeld
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to George Kokkonis.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kokkonis, G., Psannis, K.E., Roumeliotis, M. et al. Real-time wireless multisensory smart surveillance with 3D-HEVC streams for internet-of-things (IoT). J Supercomput 73, 1044–1062 (2017). https://doi.org/10.1007/s11227-016-1769-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-016-1769-9

Keywords