Abstract
Quality of Service (QoS) is becoming an integral part of current ubiquitous Distributed Interactive Multimedia Environments (DIMEs) because of their high resource and real-time interactivity demands. It is highly influenced by the management techniques of available resources in these cyber-physical environments. We consider QoS and resource management influenced by two most important resources; the computing (CPU) and networking resources. In this paper, we survey existing DIME-relevant QoS and resource management techniques for these two resources, present their taxonomy, compare them, and show their impacts on DIMEs. Finally, we discuss appropriateness of those techniques in a sample DIME scenario.












Similar content being viewed by others
Notes
We will use streams and flows notations interchangeably throughout the paper.
Another resource would be storage, but since we are discussing distributed interactive systems, networking and CPU are of higher importance.
OS that is I/O intensive and spends most of its time processing I/O in the kernel, leaves very little time to applications that are scheduled under a Proportional-Share policy.
Marco-frame is a group of correlated 3D frames captured at the same time t and at the same site.
We assume that each frame in the macro-frame has the same size since the cameras producing the frames are the same, however, each frame might have a different contribution to the user’s view. If the frame contributes fully to the view (i.e., camera is placed in the front of the user), then CF= 1, and the desired frame size to be transmitted is fs × CF = fs. If the frame contributes to the view only half-way (e.g., camera is placed on the side of the user), then CF = 0.5 and the frame size to be transmitted will be fs × 0.5.
Bundle is defined as a group of correlated streams/flows originated at one site.
By delaying the response, we mean that during the resynchronization phase when bringing in a new dominant stream, DIME and its TSync service play the old video streams with the new audio until new video streams corresponding to the new view arrive. This action will cause a delayed synchronized response to the user’s view change.
References
Agarwal V, Rejaie R (2005) Adaptive multi-source streaming in heterogeneous peer-to-peer networks. In: Proceedings of the 12th Annual Multimedia Computing and Networking (MMCN ‘05)
Antoniades D, Athanatos M, Papagiannakis A, Markatos E, Dovrolis C (2006) Available bandwidth measurement as simple as running wget. In: Proceedings of Passive and Active Measurement (PAM’06)
Baccichet P, Bagni D, Chimienti A, Pezzoni L, Rovati F (2005) Frame concealment for H. 264/AVC decoders. IEEE Trans Consum Electron 51:227–233
Baker H, Bhatti N, Tanguay D, Sobel I, Gelb D, Goss M, Culbertson W, Malzbender T (2005) Understanding performance in Coliseum, an immersive videoconferencing system. ACM Trans Multimed Comput Commun Appl, vol 1
Banachowski SA, Brandt SA (2002) The BEST scheduler for integrated processing of best-effort and soft real-time processing. In: Proceedings of Multimedia Computing and Networking (MMCN’02)
Boutremans C, Boudec JYL (2003) Adaptive joint playout buffer and fec adjustment for internet telephony. In: Proceedings of 22nd Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM’03), pp 652–662
Braden R, Zhang L, Berson S, Herzog S, Jamin S (1997) Resource ReSerVation Protocol (RSVP)—Version 1 Functional Specification, RFC 2205
Brandt SA, Banachowski S, Lin C, Bisson T (2003) Dynamic integrated scheduling of hard real-time, soft real-time and non-real-time processes. In: Proceedings of the 24th IEEE Real-Time Systems Symposium (RTSS’03), pp 396–407
Castro M, Druschel P, Kermarrec A-M, Rowstron AIT (2002) Scribe: a large-scale and decentralized application-level multicast infrastructure. IEEE J on Selected Areas in Communications (SAC’02) 20(8):1489–1499
Castro DM, Kermarrec P, Nandi A-M, Rowstron A, Singh A (2003) SplitStream: high-bandwidth multicast in cooperative environments. J Operating Systems Review 37(5):298–313
Chang S, Zhong D, Kumar R (2001) Real-time content-based adaptive streaming of sports videos. Proceedings of IEEE Workshop on Content-based Access of Image and Video Libraries, In
Chu H-H, Nahrstedt K (1999) CPU service classes for multimedia applications. In: Proceedings of IEEE International Conference on Multimedia Computing and Systems (ICMCS'99), vol 1
Chu Y, Rao SG, Zhang H (2000) A case for end system multicast. In: Proceedings of ACM Annual Conference of the ACM Special Interest Group on Measurement and Modeling of Computer Systems (SIGMETRICS’00)
Cisco telepresence. http://www.Cisco.com/TelePresence
Daniilidis F, Mulligan J, Mckendal R, Majumder A, Kamberova G, Schid D, Bajcsy R, Fuchs H (1999) Towards the holodeck: an initial testbed for real-time 3D teleimmersion. In: Proceedings of Annual Conference of the ACM Special Interest Group on Computer Graphics and Interactive Techniques (SIGGRAPH’99)
Deering S (1988) Multicast routing in internetworks and extended LANs. In: Proceedings of Annual Conference of the ACM Special Interest Group on Data Communication (SIGCOMM’88), pp 55–64
Fernandez-Escribano G, Kalva H, Cuenca P, Orozco-Barbosa L (2006) Speeding-up the macroblock partition mode decision in MPEG-2/H.264 transcoding. In: Proceedings of IEEE Conference on Image Processing (ICIP’06), pp 869–872
Gautier L, Diot C (1998) Design and evaluation of mimaze, a multi-player game on the internet. In: Proceedings of IEEE Multimedia Systems Conference, pp 233–236
Goldenstein S (1999) Time warping of audio signals. In: Proceedings of IEEE Conference on Computer Graphics, pp 52–57
Hellerstein JL (1993) Achieving service rate objectives with decay usage scheduling. IEEE Trans Softw Eng
Hosseini M, Georganas ND (2003) Design of a multi-sender 3D videoconferencing application over an end system multicast protocol. In: Proceedings of the eleventh ACM international conference on Multimedia (MM’03)
HP Halo System http://h71028.www7.hp.com/enterprise/us/en/halo/index.html
Huang C-M, Lin C-W, Yang C-C, Chang C-H, Ku H-H (2009) An SVC-MDC video coding scheme using the multi-core parallel programming paradigm for P2P video streaming. In: Proceedings of IEEE International Conference on Computer Science and Application (ICCSA’09)
Huang Y-L, Shen Y-C, Wu J-L (2009) Scalable computation for spatially scalable video coding using NVIDIA CUDA and multi-core CPU. In: Proceedings of ACM Multimedia (MM’09)
Huang Z, Wu W, Nahrstedt K, Arefin A, Rivas R (2010) TSync: a new synchronization framework for multi-site 3D tele-immersion. In: Proceedings of International Workshop on Network and Operating System Support for Digital Audio and Video (NOSSDAV’10)
Hwang FK, Richards DS (1992) Steiner tree problems. Networks 22:55–89
IEEE 1588 standard (2008) Precise time synchronization as the basis for real time applications in automation.
ITU-G.114 (2003) One-way transmission time.
Jain M, Dovrolis C (2002) Pathload: a measurement tool for end-to-end available bandwidth. In: Proceedings of the 3rd Passive and Active Measurements (PAM ’02)
Jain M, Dovrolis C (2003) End-to-end available bandwidth: measurement methodology, dynamics, and relation with TCP throughput. IEEE/ACM Trans Netw
Jeffay K, Smith FD, Moorthy A, Anderson J (1998) Proportional share scheduling of operating system services for real-time applications. In: Proceedings of Real-Time Systems Symposium (RTSS’98), vol. 0
Jia X (1998) A distributed algorithm of delay-bounded multicast routing for multimedia applications in wide area networks. IEEE/ACM Transaction on Networking 6:828–837
Jones MB, Leach PJ, Draves RP, Barrera I (1995) Modular real-time resource management in the rialto operating system. In: Proceedings of the Fifth Workshop on Hot Topics in Operating Systems (HotOS’95)
Kompella VP, Pasquale JC, Polyzos GC (1993) Multicast routing for multimedia communication. IEEE/ACM Trans Netw
Kostic D, Rodriguez A, Albrecht J, Vahdat A (2003) Bullet: high bandwidth data dissemination using an overlay mesh. ACM SIGOPS Operating Systems Review 3(5)
Kurillo G, Vasudevan R, Lobaton E, Bajcsy E (2008) A framework for collaborative real-time 3D teleimmersion in a geographically distributed environment. In: Proceedings of IEEE International Symposium on Multimedia (ISM’08)
Lee H (1997) A proportional-share scheduler for multimedia applications. In: Proceedings on International Conference on Multimedia Computing and Systems (ICMCS’97)
Lee X, Zhang Y (1996) A fast hierarchical motion-compensation scheme for video coding using block feature matching. IEEE Transaction on Circuits and Systems for Video Technology 6:627–635
Liang YJ, Faber N, Girod B (2003) Adaptive playout scheduling and loss concealment for voice communication over IP networks. IEEE Transaction on Multimedia 5(4):532–543
Little T (1993) A framework for synchronous delivery of time-dependent multimedia data. Multimedia Systems 1(2):87–94
Liu T, Choudary C (2004) Real-time content analysis and adaptive transmission of lecture videos for mobile applications. In: Proceedings of the 12th annual ACM international conference on Multimedia (MM’04), pp 400–403
Liu L-K, Feig E (1996) A block-based gradient descent search algorithm and block motion estimation in video coding. IEEE Transaction on Circuits and Systems for Video Technology 6(4):419–421
Liu CL, Layland JW (1973) Scheduling algorithms for multiprogramming in a hard real-time environment. J ACM 20(1):46–61
Liu T, Nelakuditi S (2004) Disruption-tolerant content-aware video streaming. In: Proceedings of ACM Multimedia (MM’04)
Liu J, Niu Z (2004) An adaptive receiver buffer adjust algorithm for VoIP applications considering voice characters. In: Proceedings of 10th Asia-Pacific Conference on Communications and 5th International Symposium on Multi-Dimensional Mobile Communication, pp 597–601
Liu H, Zarki M (2006) An adaptive delay and synchronization control scheme for Wi-Fi based audio/video conferencing. Springer Wireless Networks 12(4):511–522
Microsoft NetMeeting, http://www.microsoft.com.
Nahrstedt K, Qiao L (1997) Stability and adaptation control for lip synchronization skews. Technical Report, University of Illinois
Nahrstedt K, Steinmetz R (1995) Resource management in multimedia systems. IEEE Computer 28(5):52–65
Nahrstedt K, Chu H, Narayan S (1998) QoS-aware resource management for distributed multimedia applications. J on High-Speed Networking 8(3):227–255
Narbutt M, Kelly A, Murphy L, Perry P (2005) Adaptive VoIP playout scheduling: assessing user satisfaction. IEEE Internet Computing 9(4):28–34
Nguyen H, Rivas R, Nahrstedt K (2009) iDSRT: Integrated dynamic soft real-time architecture for critical infrastructure data delivery over wlan. In: Proceedings of International ICST conference on Heterogeneous Networking for Quality, Reliability, Security, and Robustness (QShine’09)
Nichols K, Blake S, Baker F, Black D (1998) Definition of the differentiated services field (DS Field) in the IPv4 and IPv6 headers, RFC 2474
Nieh J, Lam MS (2003) A smart scheduler for multimedia applications. ACM Trans Computer Systems 21(2):117–163
POSIX (1992) Realtime extension for portable operating systems (posix 1003.4). Technical Report
Rajkumar R, Juvva K, Molano A, Oikawa S (1998) Resource kernels: A resource-centric approach to real-time and multimedia systems. In: Proceedings of the SPIE/ACM Conference on Multimedia Computing and Networking (MCN’98), pp 150–164
Ramjee R, Kurose J, Towsley D, Schulzrinne H (2004) Adaptive playout mechanisms for packetized audio applications in wide-area networks. Proceedings of 13th IEEE Annual Joint Conf on Networking for Global Communication (GLOBECOM’04) 2:680–688
Rivas R, Arefin A, Nahrstedt K (2010) Janus: a cross-layer soft real-time architecture for virtualization. In: Proceedings of the 4th International Workshop on Virtualization Technologies in Distributed Computing (VTDC’10)
Russinovich M (2007) Inside the windows vista kernel: Part 1. Technet Magazine
Saroiu S, Gummadi P, Gribble S (2002) SProbe: a fast technique for measuring bottleneck bandwidth in uncooperative environments. In: Proceedings of IEEE International Conference on Computer Communications (INFOCOM’02)
Sat B, Huang Z, Wah BW (2007) The design of a multi-party VoIP conferencing system over the internet. In: Proceedings of IEEE International Symposium on Multimedia (ISM’07)
Savage S (1999) Sting: a TCP-based network measurement tool. In: Proceedings of USENIX Symposium on Internet Technologies and Systems (SITS’99)
Sheppard R, Kamali M, Rivas R, Tamai M, Yang Z, Wu W, Nahrstedt K (2008) Advancing interactive collaborative mediums through tele-immersive dance (TED): a symbiotic creativity and design environment for art and computer science. In: Proceedings of ACM International Conference on Multimedia (MM’08)
Shin I, Lee I (2003) Periodic resource model for compositional real-time guarantees. In: Proceedings of the 24th IEEE International Real-Time Systems Symposium (RTSS’03)
Skype. http://www.skype.com
Steinmetz R (1996) Human perception of jitter and media synchronization. IEEE Journal on Selected Areas in Communications 14(1):61–72
Stoica I, Abdel-Wahab H (1995) Earliest eligible virtual deadline first: A flexible and accurate mechanism for proportional share resource allocation. Old Dominion University, Technical Report
Tham JY, Ranganath S, Ranganath M, Kassim AA (1998) A novel unrestricted center biased diamond search algorithm for block motion estimation. IEEE Transaction on Circuits and Systems for Video Technology 8(4):369–377
Tommasi F, Molendini S (2000) Some extensions to enhance the scalability of the RSVP protocol. Internet Draft
Vasudevan R, Lobaton E, Kurillo G, Bajcsy R et al (2010) A methodology for remote virtual interaction in teleimmersive environments. In: Proceedings of the first annual ACM SIGMM Conference on Multimedia Systems (MMSys’10), pp 281–292
Vickers BJ, Albuquerque C, Suda T (2000) Source-adaptive multilayered multicast algorithms for real-time video distribution. IEEE/ACM Transaction of Network 8(6):720–733
Wah BW, Lin D (1999) Transformation-based reconstruction for real-time voice transmissions over the Internet. IEEE Transaction on Multimedia 1(4):342–351
Waldspurger C (1995) Lottery and stride scheduling: Flexible proportional-share resource management. Dissertation, MIT
Wang L, Terzis A, Zhang L (1999) RSVP refresh overhead reduction by state compression, Internet Draft
Wu W, Yang Z, Nahrstedt K (2008) Implementing a distributed 3D tele-immersive system. In: Proceedings of IEEE International Symposium on Multimedia (ISM’08)
Wu W, Yang Z, Nahrstedt K (2008) A study of visual context representation and control for remote sport learning tasks. In: Proceedings of AACE World Conference on Educational Multimedia, Hypermedia and Telecommunications (ED-MEDIA’08)
Wu W, Arefin A, Rivas R, Yang Z, Sheppard R, Nahrstedt K (2009) Quality of experience in distributed interactive multimedia environments: Toward a theoretical framework. In: Proceedings of ACM Multimedia (MM’09)
Xin J, Vetro A, Sun H (2004) Efficient macroblock coding-mode decision for H.264/AVC video coding. In: Proceedings of Picture Coding Symposium
Yang Z, Cui Y, Yu B, Liang J, Nahrstedt K, Jung SH, Bajscy R (2005) TEEVE: The next generation architecture for tele-immersive environments. In: Proceedings of IEEE International Symposium on Multimedia (ISM’05)
Yang Z, Yu B, Nahrstedt K, Bajcsy R (2006) A Multi- stream adaptation framework for bandwidth management in 3D tele- immersion. In: Proceedings of International Workshop on Network and Operating System Support for Digital Audio and Video (NOSSDAV’06)
Yeung A, Liew SC (1997) Multiplexing video track using frame-skipping aggregation technique. Proceedings of International Conference on Image Processing (ICIP’97) 1:334–337
Zhu Q, Garcia-Luna-Aceves J (1995) A source-based algorithm for delay-constrained minimum-cost multicasting. In: Proceedings of IEEE International Conference on Computer Communications (INFOCOM’95)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Nahrstedt, K., Arefin, A., Rivas, R. et al. QoS and resource management in distributed interactive multimedia environments. Multimed Tools Appl 51, 99–132 (2011). https://doi.org/10.1007/s11042-010-0627-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-010-0627-7