Abstract
The paper addresses the issue of measuring the end-to-end video delay due to the grabbing, display, and transmission components in networked video communication. We discuss the design and implementation of a scalable tool for the analysis and design of applications involving video based networked interaction or teleoperation, such as remotely operated devices and collaborative environments. The tool is intended for providing an accurate measure of the video lag involved in the encoding and streaming of video over packet networks, and for assessing the impact that delays in the video communication has in terms of human performance. The system is demonstrated through two different tasks: first, the measurement of video end-to-end delay for different networking configurations. Then, a visuomotor experiment is reported in which a subject is requested to accomplish a remote control task by using delayed video feedback, under different delay conditions.
Similar content being viewed by others
References
Allen PK, Timcenko A, Yoshimi B, Michelman P (1993) Automated tracking and grasping of a moving object with a robotic hand-eye system. IEEE Trans Robot Autom 9(2):152–165
Boyaci O, Forte A, Baset SA, Schulzrinne H (2009) vdelay: a tool to measure capture-to-display latency and frame rate. In: 11th IEEE International symposium on multimedia, 2009. ISM ’09, pp 194–200
Buttazzo GC, Allotta B, Fanizza FP (1993) Mousebuster: a robot system for catching fast moving objects by vision. In: Proceedings of the 1993 IEEE international conference on robotics and automatio (ICRA). IEEE Computer Society Press, pp 932–937
Chafe C, Gurevich M, Leslie G, Tyan S (2004) Effect of time delay on ensemble accuracy. In: Proceedings of the international symposium on musical acoustics
Chen H, Wei C, Song M, Mm Sun, Lau K (2015) Capture-to-display delay measurement for visual communication applications. APSIPA Trans Signal Inform Process, 4
Di Luca M (2010) New method to measure end-to-end delay of virtual reality. Presence: Teleoper Virtual Environ 19(6):569–584
Drioli C, Foresti GL (2015) Time-varying delay measurement of video capture-to-display components with application to visual servoing. Signal Process Image Commun 39(Part A):84–97
Ellis SR, Adelstein BD, Baumeler S, Jense GJ, Jacoby RH (1999) Sensor spatial distortion, visual latency, and update rate effects on 3d tracking in virtual environments. In: Virtual reality, 1999. Proceedings. IEEE, pp 218–221
Gergle D, Kraut RE, Fussell SR (2006) The impact of delayed visual feedback on collaborative performance. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’06. ACM, New York, pp 1303–1312
Goel A, Krasic C, Walpole J (2008) Low-latency adaptive streaming over tcp. ACM Trans Comput Multimed Commun Appl 4(3):20,1–20,20
Gopalan K, Huang L, Peng G, Chiueh T, Lin Y (2006) Statistical admission control using delay distribution measurements. ACM Trans Comput Multimed Commun Appl 2(4):258–281
Hager GD, Okamura AM, Kazanzides P, Whitcomb L, Fichtinger G, Taylor RH (2008) Surgical and interventional robotics: part iii [tutorial]. Robot Autom Mag IEEE 15(4):84–93
He W, Nahrstedt K, Liu X (2008) End-to-end delay control of multimedia applications over multihop wireless links. ACM Trans Multimed Comput Commun Appl 5(2):16,1–16,20
Hill R, Madden C, Hengel A, Detmold H, Dick A (2009) Measuring latency for video surveillance systems. In: Digital image computing: techniques and applications, 2009. DICTA ’09, pp 89–95
Honda T, Hirashima M, Nozaki D (2012) Adaptation to visual feedback delay influences visuomotor learning. PLoS ONE, 7(5)
Jansen J, Bulterman DCA (2013) User-centric video delay measurements. In: Proceeding of the 23rd ACM workshop on network and operating systems support for digital audio and video, NOSSDAV ’13. ACM, New York, pp 37–42
Konstantas D (1998) Overview of a telepresence environment for distributed musical rehearsals. In: Proceedings of the 1998 ACM symposium on applied computing, SAC ’98. ACM, New York, pp 456–457
Kryczka A, Arefin A, Nahrstedt K (2013) Avcloak: a tool for black box latency measurements in video conferencing applications. In: 2013 IEEE International symposium on multimedia (ISM), pp 271–278
Lee YS, Kim S-H (2019) A proposal of novel design on the wave mac algorithm with low-latency for seamless video surveillance in v2x environment. Multimedia Tools and Applications
Leung GMH, Francis BA, Apkarian J (1995) Bilateral controller for teleoperators with time delay via mu;-synthesis. IEEE Trans Robot Autom 11(1):105–116
Li Y, Shen D, Zhou G (2016) Energy optimization for mobile video streaming via an aggregate model. Multimed Tools Appl 76:20781–20797
Lippi V, Avizzano CA, Mottet D, Ruffaldi E (2010) Effect of delay on dynamic targets tracking performance and behavior in virtual environment. In: RO-MAN, 2010 IEEE, pp 446–451
Liu Z, Wang Z (Dec 2017) Action recognition with low observational latency via part movement model. Multimed Tools Appl 76(24):26675–26693
Marescaux J, Leroy J, Rubino F, Smith M, Vix M, Simone M, Mutter D (2002) Transcontinental robot-assisted remote telesurgery: feasibility and potential applications. Ann Surg 235(4):487
Martinel N, Micheloni C, Piciarelli C, Foresti GL (2014) Camera selection for adaptive human-computer interface. IEEE Trans Syst Man Cybernet: Syst 44(5):653–664
Miller K, Al-Tamimi A, Wolisz A (2016) Qoe-based low-delay live streaming using throughput predictions. ACM Trans Comput Multimed Commun Appl 13(1):4,1–4,24
Morice AHP, Siegler IA, Bardy BG (2008) Action-perception patterns in virtual ball bouncing Combating system latency and tracking functional validity. J Neurosci Methods 169(1):255–266
Nahrstedt K, Arefin A, Rivas R, Agarwal P, Huang Z, Wu W, Yang Z (2011) Qos and resource management in distributed interactive multimedia environments. Multimed Tools Appl 51(1):99–132
Niemeyer G, Slotine JE (2004) Telemanipulation with time delays. I J Robotic Res 23(9):873–890
Oboe R, Fiorini P (1998) A design and control environment for internet-based telerobotics. Int J Robot Res 17(4):433–449
Piciarelli C, Esterle L, Khan A, Rinner B, Foresti GL (2016) Dynamic reconfiguration in camera networks: a short survey. IEEE Trans Circ Syst Video Technol 26(5):965–977
Qi X, Yang Q, Nguyen DT, Peng G, Zhou G, Dai B, Zhang D, Li Y (2016) A context-aware framework for reducing bandwidth usage of mobile video chats. IEEE Trans Multimed 18(8):1640–1649
Rodriguez-Gil L, Orduña P, García-Zubia J, López-de Ipińa D (2018) Interactive live-streaming technologies and approaches for web-based applications. Multimed Tools Appl 77(6):6471–6502
Sanders D (2009) Analysis of the effects of time delays on the teleoperation of a mobile robot in various modes of operation. Industr Robot: Int J 36(6):570–584
Saunders JA, Knill DC (2004) Visual feedback control of hand movements. J Neurosci 24(13):3223–3234
Spearing R, Regan M (2005) 4.0 nasa communication and navigation capability roadmap; executive summary. Technical report NASA
Teather RJ, Pavlovych A, Stuerzlinger W, MacKenzie IS (2009) Effects of tracking technology, latency, and spatial jitter on object movement. In: IEEE Symposium on 3D user interfaces, 2009. 3DUI 2009, pp 43–50
Ware C, Balakrishnan R (1994) Reaching for objects in vr displays Lag and frame rate. ACM Trans Comput-Hum Interact 1(4):331–356
Watson B, Walker N, Ribarsky W, Spaulding V (1998) Effects of variation in system responsiveness on user performance in virtual environments. Hum Factors 40:403–414
Winck RC, Sketch SM, Hawkes EW, Christensen DL, Jiang H, Cutkosky MR, Okamura AM (2014) Time-delayed teleoperation for interaction with moving objects in space. In: IEEE International conference on robotics and automation
Wu W, Dong Y (2013) A Hoover measuring digital system latency from sensing to actuation at continuous 1-ms resolution. Presence Teleoper Virtual Environ 22(1):20–35
Xu J, Wah BW (2013) Exploiting just-noticeable difference of delays for improving quality of experience in video conferencing. In: Proceedings of the 4th ACM multimedia systems conference, MMSys ’13. ACM, New York, pp 238–248
Xu Y, Yu C, Li J, Liu Y (2014) Video telephony for end-consumers: measurement study of google+, ichat, and skype. IEEE/ACM Trans Network 22 (3):826–839
Zhang L, Zheng L, Ngee KS (2002) Effect of delay and delay jitter on voice/video over ip. Comput Commun 25(9):863–873
Zhang J, Zhu X, Zhou Z (2010) Design of time delayed control systems in uav using model based predictive algorithm. IEEE Press, Piscataway, pp 269–272
Zimmermann R, Chew E, Ay SA, Pawar M (2008) Distributed musical performances: architecture and stream management. ACM Trans Multimed Comput Commun Appl 4(2):14,1–14,23
Acknowledgements
This work was supported in part by Italian MoD Project a2018 − 045 “A proactive counter- UAV system to protect army tanks and patrols in urban areas” (Proactive_Counter_UAV).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Bulfone, A., Drioli, C., Ferrin, G. et al. A scalable system for the monitoring of video transmission components in delay-sensitive networked applications. Multimed Tools Appl 79, 18727–18745 (2020). https://doi.org/10.1007/s11042-020-08743-7
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-020-08743-7