Skip to main content
SpringerLink
Log in

Design and implementation of parallel video combiner architecture for multi-user video conferencing at ultra-high definition resolution

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In software driven multi-user video conferencing systems, consumer progression towards ultra-high definition (i.e., 8 k) resolution introduces new challenges to the video combination process in sustaining smooth combined video playback at high frame rates. As such, this paper analyzes the underlying architecture of a conventional video combiner to identify the performance impact of combined video frame rates at ultra-high definition resolutions. Then, two straightforward parallel video combination architectures using software application threads, namely PVC-1 and PVC-2, are studied as a benchmark. In PVC-1, the number of application threads tallies with the number of client videos to be combined, which improves playback performance at the expense of inconsistencies in inset client frame rates within a combined video. PVC-2 includes a synchronizer to address these inconsistencies, but exhibits marginal performance gains. To address the aforementioned problems, a balanced workload parallel video combiner architecture is proposed, namely PVC-3. In this architecture, a balanced workload management algorithm stitches client videos into an ultra-high definition combined frame, based on the number of available logical processors on a multi-core processor. This method improves frame rate performance and sustains consistent client frame rates within a combined video. Experimental results suggest that PVC-3 is superior over PVC-2 and achieves a frame rate performance gain of 27 % against a conventional video combiner for a combined video of 16 clients (each at a resolution of 720p) and zero standard deviation in combined frame rate consistency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22

Similar content being viewed by others

References

  1. Banerji AK, Panchapakesan K, Swaminathan K (2006) Stitching of H.264 video streams for continuous presence multipoint videoconferencing. J Vis Commun Image Represent 17(2):490–508

    Article  Google Scholar 

  2. Baskaran VM, Chang YC, Loo J, Wong K (2013) Software-based serverless endpoint video combiner architecture for high-definition multiparty videoconferencing. J Netw Comput Appl 36(1):336–352

    Article  Google Scholar 

  3. Baskaran VM, Low YS, Wong K (2010) Building a real-time multiple H.264 video streaming system based on intel IPP. IEEE Asia Pacific Conference on Circuits and Systems (APCCAS). Kuala Lumpur, Malaysia, pp 156–159

  4. Bhandarkar SM, Chandrasekaran SR (2004) Parallel parsing of MPEG video on a shared-memory symmetric multiprocessor. Parallel Comput 30(11):1233–1276

    Article  Google Scholar 

  5. Chen JYC, Thropp JE (2007) Review of low frame rate effects on human performance. IEEE Trans Syst Man Cybern Syst Hum 37(6):1063–1076

    Article  Google Scholar 

  6. Cucchiara R, Gualdi G (2010) Mobile video surveillance systems: an architectural overview, mobile multimedia processing. Lect Notes Comput Sci 5960:89–109

    Article  Google Scholar 

  7. Cyconx HL, Hege G, Marpey D, Palkow M, Schmidt TC, Wahlisch M (2009) Connecting the Worlds: Multipoint Videoconferencing Integrating H.323 and IPv4, SIP and IPv6 with Autonomous Sender Authentication. IEEE 13th International Symposium on Consumer Electronics (ISCE2009). Kyoto, Japan, pp 890–893

  8. Diaz J, Muñoz-Caro C, Niño A (2012) A survey of parallel programming models and tools in the multi and many-core era. IEEE Trans Parallel Distrib Syst 23(8):1369–1386

    Article  Google Scholar 

  9. Fung KT, Chan YL, Siu WC (2004) Low-complexity and high-quality frame-skipping transcoder for continuous presence multipoint video conferencing. IEEE Trans Multimed 6(1):31–46

    Article  Google Scholar 

  10. Haering N, Venetianer PL, Lipton A (2008) The evolution of video surveillance: an overview. Mach Vis Appl 19(5–6):279–290

    Article  MATH  Google Scholar 

  11. Halák J, Krsek M, Ubik S, Žejdl P, Nevrela F (2011) Real-time long-distance transfer of uncompressed 4K video for remote collaboration. Futur Gener Comput Syst 27(7):886–892

    Article  Google Scholar 

  12. Hyun W, Kang SG (2011) Interoperable telepresence services: B yond HD-videoconferences and Towards Telepresence. IEEE Asia-Pacific Services Computing Conference (APSCC). Jeju Island, South Korea, pp 327–329

  13. International Telecommunications Union Radiocommunications Broadcasting Service (Television) (2012) Parameter values for ultra-high definition television systems for production and international programme exchange. Recommendation ITU-R, BT, 2020 (08/12). Available online at http://www.itu.int/rec/R-REC-BT.2020-0-201208-I/en

  14. Jie Y, Sun S (2006) Implementation of multipoint video conference in software. IEEE Asia Pacific Conference on Circuits and Systems (APCCAS). Singapore, pp 1980–1983

  15. Kalva H, Colic A, Garcia A, Furht B (2011) Parallel programming for multimedia applications. Multimedia Tools Appl 51(2):801–818

    Article  Google Scholar 

  16. Kim CG, Kim JG, Lee DH (2014) Optimizing image processing on multi-core CPUs with Intel parallel programming technologies. Multimedia Tools Appl 68(2):237--251

    Google Scholar 

  17. Kitamura M, Shirai D, Kanekob K, Murookaa T, Sawabea T, Fujii T, Takahara A (2011) Beyond 4K: 8K 60p live video streaming to multiple sites. Futur Gener Comput Syst 27(7):952–959

    Article  Google Scholar 

  18. Le TA, Nguyen H (2010) Centralized and distributed architectures of scalable video conferencing services. 2nd International Conference on Ubiquitous and Future Networks (ICUFN). Paris, France, pp 394–399

  19. Li P, Veeravalli B, Kassim AA (2005) Design and implementation of parallel video encoding strategies using divisible load analysis. IEEE Trans Circ Syst Video Technol 15(9):1098–1112

    Article  Google Scholar 

  20. Lin CW, Chen YC, Sun MT (2003) Dynamic region of interest transcoding for multipoint video conferencing. IEEE Trans Circ Syst Video Technol 13(10):982–992

    Article  Google Scholar 

  21. Lin D, Huang V, Nguyen Q, Blackburn J, Rodrigues C, Huang T, Do MN, Patel SJ, Hwu W-MW (2009) The parallelization of video processing. IEEE Signal Process Mag 26(6):103–112

    Article  Google Scholar 

  22. Lo W-Y, Lun DP-K, Siu W-C, Wang W, Song J (2011) Improved SIMD architecture for high performance video processors. IEEE Trans Circ Syst Video Technol 21(12):1769–1783

    Article  Google Scholar 

  23. Lu Y, Zhao Y, Kuipers FA, Mieghem PV (2010) Measurement study of multi-party video conferencing. NetWORKing’ 2010. Chennai, India, pp 96–108

  24. Lucarz C, Roquier G, Mattavelli M, Raulet M, Nezan J-F, Deforges O (2009) Reconfigurable video coding on multicore. IEEE Signal Process Mag 26(6):113–123

    Article  Google Scholar 

  25. Meier R, Wattenhofer R (2012) Peer-to-peer streaming in heterogeneous environments. Signal Process Image Commun 27(5):457–469

    Article  Google Scholar 

  26. Rodriguez A, Gonzalez A, Malumbres MP (2006) Hierarchical parallelization of an H.264/AVC video encoder. International Symposium on Parallel Computing in Electrical Engineering (PARELEC’06). Bialystok, Poland, pp 363–368

  27. Rodríguez-Sánchez R, Martínez JL, Fernández-Escribano G, Sánchez JL, Claver JM (2013) H.264/AVC inter prediction on accelerator-based multi-core systems. Multimedia Tools Appl 66(3):361--381

    Google Scholar 

  28. Saddik AE, Rahman A, Abdala S, Solomon B (2008) PECOLE: P2P multimedia collaborative environment. Multimedia Tools Appl 39(3):353–377

    Article  Google Scholar 

  29. Shiraia D, Kitamuraa M, Fujiia T, Takaharaa A, Kanekob K, Ohtac N (2011) Multi-point 4K/2K layered video streaming for remote collaboration. Futur Gener Comput Syst 27(7):986–990

    Article  Google Scholar 

  30. Sodan AC, Machina J, Deshmeh A, Macnaughton K, Esbaugh B (2010) Parallelism via multithreaded and multicore CPUs. IEEE Signal Process Mag 43(3):24–32

    Google Scholar 

  31. Sullivan GJ, Ohm J-R, Han W-J, Wiegand T (2012) Overview of the High Efficiency Video Coding (HEVC) Standard. IEEE Trans Circ Syst Video Technol 22(12):1649–1668

    Article  Google Scholar 

  32. Sun MT, Loui AC, Chen TC (1997) A coded-domain video combiner for multipoint continuous presence video conferencing. IEEE Trans Circ Syst Video Technol 7(6):855–863

    Article  Google Scholar 

  33. Varoglu S, Jenks S (2011) Architectural support for thread communications in multi-core processors. Parallel Comput 37(1):26–41

    Article  Google Scholar 

  34. Vatavu R-D (2010) Creativity in interactive TV: Personalize, share, and invent interfaces, mobile TV: Customizing content and experience. Human-Computer Interaction Series, pp 121–139

  35. Zhu QF, Kerofsky L, Garrison MB (1999) Low-delay, low-complexity rate reduction and continuous presence for multipoint videoconferencing. IEEE Trans Circ Syst Video Technol 9(4):666–676

    Article  Google Scholar 

Download references

Acknowledgement

This work was supported by the University Malaya Research Collaboration Grant (Title: Realistic Video-based Assistive Living, Grant Number: CG009-2013) under the purview of University of Malaya Research, and the Multimedia University Residual Fund. The authors would like to thank the anonymous reviewers for their constructive and invaluable comments in improving the quality of this article.

Author information

Authors and Affiliations

  1. Faculty of Engineering, Multimedia University, Persiaran Multimedia, 63100, Cyberjaya, Selangor, Malaysia

    Vishnu Monn Baskaran & Yoong Choon Chang

  2. School of Science and Technology, Middlesex University, The Burroughs, London, NW4 4BT, UK

    Jonathan Loo

  3. Faculty of Computer Science and Information Technology, University of Malaya, 50603, Kuala Lumpur, Malaysia

    KokSheik Wong

Authors
  1. Vishnu Monn Baskaran
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yoong Choon Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jonathan Loo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. KokSheik Wong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vishnu Monn Baskaran.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Baskaran, V.M., Chang, Y.C., Loo, J. et al. Design and implementation of parallel video combiner architecture for multi-user video conferencing at ultra-high definition resolution. Multimed Tools Appl 74, 6589–6622 (2015). https://doi.org/10.1007/s11042-014-1907-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-014-1907-4

Keywords

Search

Navigation