Skip to main content
Log in

A scalable system for the monitoring of video transmission components in delay-sensitive networked applications

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The paper addresses the issue of measuring the end-to-end video delay due to the grabbing, display, and transmission components in networked video communication. We discuss the design and implementation of a scalable tool for the analysis and design of applications involving video based networked interaction or teleoperation, such as remotely operated devices and collaborative environments. The tool is intended for providing an accurate measure of the video lag involved in the encoding and streaming of video over packet networks, and for assessing the impact that delays in the video communication has in terms of human performance. The system is demonstrated through two different tasks: first, the measurement of video end-to-end delay for different networking configurations. Then, a visuomotor experiment is reported in which a subject is requested to accomplish a remote control task by using delayed video feedback, under different delay conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Allen PK, Timcenko A, Yoshimi B, Michelman P (1993) Automated tracking and grasping of a moving object with a robotic hand-eye system. IEEE Trans Robot Autom 9(2):152–165

    Article  Google Scholar 

  2. Boyaci O, Forte A, Baset SA, Schulzrinne H (2009) vdelay: a tool to measure capture-to-display latency and frame rate. In: 11th IEEE International symposium on multimedia, 2009. ISM ’09, pp 194–200

  3. Buttazzo GC, Allotta B, Fanizza FP (1993) Mousebuster: a robot system for catching fast moving objects by vision. In: Proceedings of the 1993 IEEE international conference on robotics and automatio (ICRA). IEEE Computer Society Press, pp 932–937

  4. Chafe C, Gurevich M, Leslie G, Tyan S (2004) Effect of time delay on ensemble accuracy. In: Proceedings of the international symposium on musical acoustics

  5. Chen H, Wei C, Song M, Mm Sun, Lau K (2015) Capture-to-display delay measurement for visual communication applications. APSIPA Trans Signal Inform Process, 4

  6. Di Luca M (2010) New method to measure end-to-end delay of virtual reality. Presence: Teleoper Virtual Environ 19(6):569–584

    Article  Google Scholar 

  7. Drioli C, Foresti GL (2015) Time-varying delay measurement of video capture-to-display components with application to visual servoing. Signal Process Image Commun 39(Part A):84–97

    Article  Google Scholar 

  8. Ellis SR, Adelstein BD, Baumeler S, Jense GJ, Jacoby RH (1999) Sensor spatial distortion, visual latency, and update rate effects on 3d tracking in virtual environments. In: Virtual reality, 1999. Proceedings. IEEE, pp 218–221

  9. Gergle D, Kraut RE, Fussell SR (2006) The impact of delayed visual feedback on collaborative performance. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’06. ACM, New York, pp 1303–1312

  10. Goel A, Krasic C, Walpole J (2008) Low-latency adaptive streaming over tcp. ACM Trans Comput Multimed Commun Appl 4(3):20,1–20,20

    Article  Google Scholar 

  11. Gopalan K, Huang L, Peng G, Chiueh T, Lin Y (2006) Statistical admission control using delay distribution measurements. ACM Trans Comput Multimed Commun Appl 2(4):258–281

    Article  Google Scholar 

  12. Hager GD, Okamura AM, Kazanzides P, Whitcomb L, Fichtinger G, Taylor RH (2008) Surgical and interventional robotics: part iii [tutorial]. Robot Autom Mag IEEE 15(4):84–93

    Article  Google Scholar 

  13. He W, Nahrstedt K, Liu X (2008) End-to-end delay control of multimedia applications over multihop wireless links. ACM Trans Multimed Comput Commun Appl 5(2):16,1–16,20

    Article  Google Scholar 

  14. Hill R, Madden C, Hengel A, Detmold H, Dick A (2009) Measuring latency for video surveillance systems. In: Digital image computing: techniques and applications, 2009. DICTA ’09, pp 89–95

  15. Honda T, Hirashima M, Nozaki D (2012) Adaptation to visual feedback delay influences visuomotor learning. PLoS ONE, 7(5)

  16. Jansen J, Bulterman DCA (2013) User-centric video delay measurements. In: Proceeding of the 23rd ACM workshop on network and operating systems support for digital audio and video, NOSSDAV ’13. ACM, New York, pp 37–42

  17. Konstantas D (1998) Overview of a telepresence environment for distributed musical rehearsals. In: Proceedings of the 1998 ACM symposium on applied computing, SAC ’98. ACM, New York, pp 456–457

  18. Kryczka A, Arefin A, Nahrstedt K (2013) Avcloak: a tool for black box latency measurements in video conferencing applications. In: 2013 IEEE International symposium on multimedia (ISM), pp 271–278

  19. Lee YS, Kim S-H (2019) A proposal of novel design on the wave mac algorithm with low-latency for seamless video surveillance in v2x environment. Multimedia Tools and Applications

  20. Leung GMH, Francis BA, Apkarian J (1995) Bilateral controller for teleoperators with time delay via mu;-synthesis. IEEE Trans Robot Autom 11(1):105–116

    Article  Google Scholar 

  21. Li Y, Shen D, Zhou G (2016) Energy optimization for mobile video streaming via an aggregate model. Multimed Tools Appl 76:20781–20797

    Article  Google Scholar 

  22. Lippi V, Avizzano CA, Mottet D, Ruffaldi E (2010) Effect of delay on dynamic targets tracking performance and behavior in virtual environment. In: RO-MAN, 2010 IEEE, pp 446–451

  23. Liu Z, Wang Z (Dec 2017) Action recognition with low observational latency via part movement model. Multimed Tools Appl 76(24):26675–26693

  24. Marescaux J, Leroy J, Rubino F, Smith M, Vix M, Simone M, Mutter D (2002) Transcontinental robot-assisted remote telesurgery: feasibility and potential applications. Ann Surg 235(4):487

    Article  Google Scholar 

  25. Martinel N, Micheloni C, Piciarelli C, Foresti GL (2014) Camera selection for adaptive human-computer interface. IEEE Trans Syst Man Cybernet: Syst 44(5):653–664

    Article  Google Scholar 

  26. Miller K, Al-Tamimi A, Wolisz A (2016) Qoe-based low-delay live streaming using throughput predictions. ACM Trans Comput Multimed Commun Appl 13(1):4,1–4,24

    Google Scholar 

  27. Morice AHP, Siegler IA, Bardy BG (2008) Action-perception patterns in virtual ball bouncing Combating system latency and tracking functional validity. J Neurosci Methods 169(1):255–266

    Article  Google Scholar 

  28. Nahrstedt K, Arefin A, Rivas R, Agarwal P, Huang Z, Wu W, Yang Z (2011) Qos and resource management in distributed interactive multimedia environments. Multimed Tools Appl 51(1):99–132

    Article  Google Scholar 

  29. Niemeyer G, Slotine JE (2004) Telemanipulation with time delays. I J Robotic Res 23(9):873–890

    Article  Google Scholar 

  30. Oboe R, Fiorini P (1998) A design and control environment for internet-based telerobotics. Int J Robot Res 17(4):433–449

    Article  Google Scholar 

  31. Piciarelli C, Esterle L, Khan A, Rinner B, Foresti GL (2016) Dynamic reconfiguration in camera networks: a short survey. IEEE Trans Circ Syst Video Technol 26(5):965–977

    Article  Google Scholar 

  32. Qi X, Yang Q, Nguyen DT, Peng G, Zhou G, Dai B, Zhang D, Li Y (2016) A context-aware framework for reducing bandwidth usage of mobile video chats. IEEE Trans Multimed 18(8):1640–1649

    Article  Google Scholar 

  33. Rodriguez-Gil L, Orduña P, García-Zubia J, López-de Ipińa D (2018) Interactive live-streaming technologies and approaches for web-based applications. Multimed Tools Appl 77(6):6471–6502

    Article  Google Scholar 

  34. Sanders D (2009) Analysis of the effects of time delays on the teleoperation of a mobile robot in various modes of operation. Industr Robot: Int J 36(6):570–584

    Article  Google Scholar 

  35. Saunders JA, Knill DC (2004) Visual feedback control of hand movements. J Neurosci 24(13):3223–3234

    Article  Google Scholar 

  36. Spearing R, Regan M (2005) 4.0 nasa communication and navigation capability roadmap; executive summary. Technical report NASA

  37. Teather RJ, Pavlovych A, Stuerzlinger W, MacKenzie IS (2009) Effects of tracking technology, latency, and spatial jitter on object movement. In: IEEE Symposium on 3D user interfaces, 2009. 3DUI 2009, pp 43–50

  38. Ware C, Balakrishnan R (1994) Reaching for objects in vr displays Lag and frame rate. ACM Trans Comput-Hum Interact 1(4):331–356

    Article  Google Scholar 

  39. Watson B, Walker N, Ribarsky W, Spaulding V (1998) Effects of variation in system responsiveness on user performance in virtual environments. Hum Factors 40:403–414

    Article  Google Scholar 

  40. Winck RC, Sketch SM, Hawkes EW, Christensen DL, Jiang H, Cutkosky MR, Okamura AM (2014) Time-delayed teleoperation for interaction with moving objects in space. In: IEEE International conference on robotics and automation

  41. Wu W, Dong Y (2013) A Hoover measuring digital system latency from sensing to actuation at continuous 1-ms resolution. Presence Teleoper Virtual Environ 22(1):20–35

    Article  Google Scholar 

  42. Xu J, Wah BW (2013) Exploiting just-noticeable difference of delays for improving quality of experience in video conferencing. In: Proceedings of the 4th ACM multimedia systems conference, MMSys ’13. ACM, New York, pp 238–248

  43. Xu Y, Yu C, Li J, Liu Y (2014) Video telephony for end-consumers: measurement study of google+, ichat, and skype. IEEE/ACM Trans Network 22 (3):826–839

    Article  Google Scholar 

  44. Zhang L, Zheng L, Ngee KS (2002) Effect of delay and delay jitter on voice/video over ip. Comput Commun 25(9):863–873

    Article  Google Scholar 

  45. Zhang J, Zhu X, Zhou Z (2010) Design of time delayed control systems in uav using model based predictive algorithm. IEEE Press, Piscataway, pp 269–272

    Google Scholar 

  46. Zimmermann R, Chew E, Ay SA, Pawar M (2008) Distributed musical performances: architecture and stream management. ACM Trans Multimed Comput Commun Appl 4(2):14,1–14,23

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by Italian MoD Project a2018 − 045 “A proactive counter- UAV system to protect army tanks and patrols in urban areas” (Proactive_Counter_UAV).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Carlo Drioli.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bulfone, A., Drioli, C., Ferrin, G. et al. A scalable system for the monitoring of video transmission components in delay-sensitive networked applications. Multimed Tools Appl 79, 18727–18745 (2020). https://doi.org/10.1007/s11042-020-08743-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-08743-7

Keywords

Navigation