Skip to main content
SpringerLink
Log in

Asymmetric and selective object rendering for optimized Cloud Mobile 3D Display Gaming user experience

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Cloud Mobile 3D Display Gaming has been recently proposed where 3D video rendering and encoding are performed on cloud servers, with the resulting 3D video streamed wirelessly to mobile devices with 3D displays. This approach has the advantage of relieving high computation, power and storage requirements of 3D display gaming from mobile devices, while enabling game developers to focus on a single rich version of the game which can be experienced from any mobile device and platform. However, it is challenging to stream 3D video over dynamically fluctuating and often constrained mobile networks. In this paper, we propose a novel technique called Asymmetric and Selective Object Rendering (ASOR) which proves to be more powerful than previous solutions for Cloud based Mobile 3D display gaming. Specifically, this technique will enable rendering engine to intelligently decide whether or not to render an individual object and how good the corresponding texture detail will be if rendered, and the settings can be asymmetric for two views. Thus, unimportant objects can trade quality for reduced bitrate while important objects can remain high quality so that the overall user experience is optimized given certain bandwidth constraints. To quantitatively measure the user experience and bitrate by applying different rendering settings, we develop a user experience model and a bitrate model. We further propose an optimization algorithm which uses the above two models to automatically decide the optimal rendering settings for left view and right view to ensure the best user experience given the network conditions. Experiments conducted using real 4G-LTE network profiles on commercial cloud service with different genres of games demonstrate significant improvement in user experience when the proposed optimization algorithm is applied.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

References

  1. x264, http://www.videolan.org/developers/x264.html/, Jun. 2015

  2. Agrafiotis D, Bull DR, Canagarajah N, Kamnoonwatana N (2006) “Multiple priority region of interest coding with H. 264.” in proceedings of IEEE International Conference on Image Processing (ICIP)

  3. Aksay S, Pehlivan E, Kurutepe C, Bilen T, Ozcelebi, Akar GB, Tekalp AM (2007) End-to-end stereoscopic video streaming with content-adaptive rate and format control. Signal Process Image Commun 22(2):157–168

    Article  Google Scholar 

  4. Broadsides, http://cse125.ucsd.edu/cse125/2012/cse125g1/, Jun. 2015

  5. Cai W, Leung V, Chen M (2013) “Next generation mobile cloud gaming.” in proceedings of IEEE 7th International Symposium on Service Oriented System Engineering (SOSE), pp. 551–560

  6. Chiang T, Lee H.-J., Sun H (2000) “An overview of the encoding tools in the MPEG-4 reference software” in proceedings of IEEE International Symposium on Circuit and Systems (ISCAS’00). 1:295–298

  7. Chu PC, Beasley JE (1998) A genetic algorithm for the multidimensional knapsack problem. J Heuristics 4(1):63–86

    Article  MATH  Google Scholar 

  8. Gaikai, https://www.gaikai.com/, Jun. 2015

  9. Gurler CG, Bagci KT, Tekalp AM (2010) “Adaptive stereoscopic 3D video streaming” in proceedings of IEEE International Conference on Image Processing (ICIP’10) pp. 2409–2412

  10. Hemmati M, Javadtalab A, Nazari Shirehjini AA, Shirmohammadi S, Arici T (2013) “Game as video: Bit rate reduction through adaptive object encoding” in proceedings of ACM Workshop on Network and Operating Systems Support for Digital Audio and Video, pp. 7–12

  11. Huang C, Chen K, Chen D, Hsu H, Hsu C (2014) GamingAnywhere: the first open source cloud gaming system. ACM Trans Multimed Comput Commun Appl 10, No 1s

  12. IBM ILOG CPLEX optimizer, http://www-01.ibm.com/software/integration/optimization/cplex-optimizer/, Jun. 2015

  13. ITU-T Rec. G.107 (2005) The E-model, a computational model for use in transmission planning

  14. Liu Y, Li ZG, Soh YC (2008) Region-of-interest based resource allocation for conversational video communication of H.264/ AVC. IEEE Trans Circuits Syst Video Technol 18(1):134–139

    Article  Google Scholar 

  15. Liu Y, Wang S, Dey S (2014) Content-aware modeling and enhancing user experience in Cloud Moble Rendering and Streaming. IEEE J Emerging Sel Top Circuits Syst 4(1):43–56

    Article  Google Scholar 

  16. Lu Y, Liu Y, Dey S (2014) “Enhancing Cloud Mobile 3D Display Gaming User Experience by Asymmetric Graphics Rendering”, in proceedings of IEEE International Conference on Computing, Networking and Communications(ICNC)

  17. Lu Y, Liu Y, Dey S (2015a) Modeling and optimizing Cloud Mobile 3D Display Gaming user experience by asymmetric graphics rendering. IEEE J Sel Top Sign Proces 9(3):1–16

    Article  Google Scholar 

  18. Lu Y, Liu Y, Dey S (2015b) “Optimizing Cloud Mobile 3D Display Gaming User Experience by Asymmetric Object of Interest Rendering.” in proceedings of IEEE International Conference on Communication (ICC)

  19. Ma Z, Xu M, Ou YF, Wang Y (2012) Modeling of rate and perceptual quality of compressed video as functions of frame rate and quantization stepsize and its applications. IEEE Trans Circuits Syst Video Technol 22(5):671–682

    Article  Google Scholar 

  20. NVIDIA GRID, http://shield.nvidia.com/grid-game-streaming

  21. OnLive, http://www.onlive.com/, Jun. 2015

  22. Ou YF, Ma Z, Liu T, Wang Y (2011) Perceptual quality assessment of video considering both frame rate and quantization artifacts. IEEE Trans Circuits Syst Video Technol 21(3):286–298

    Article  Google Scholar 

  23. Planeshift, http://www.planeshift.it/, Jun. 2015

  24. Ribas-Corbera J, Lei S (1999) Rate control in DCT video coding for low-delay communications. IEEE Trans Circuits Syst Video Technol 9(2):172–185

    Article  Google Scholar 

  25. Saygili G, Cihat CG, Tekalp AM (2012) Evaluation of asymmetric stereo video coding and rate scaling for adaptive 3D video streaming. IEEE Trans Broadcast 57(2):593–601

    Article  Google Scholar 

  26. Shea R, Liu J, Ngai E, Cui Y (2013) Cloud gaming: architecture and performance. IEEE Netw 27(4):16–21

    Article  Google Scholar 

  27. Sinha P, Zoltners AA (1979) The multiple-choice knapsack problem. Oper Res 27(3):503–515

    Article  MathSciNet  MATH  Google Scholar 

  28. Speedtest.net, https://itunes.apple.com/us/app/speedtest.net-mobile-speed/id300704847?mt=8/, Jun. 2015

  29. Vankeirsbilck T, Verbelen D, Verslype N Staelens, De Turck F, Demeester P, Dhoedt B (2013) “Quality of experience driven control of interactive media stream parameters” in proceedings of IFIP/IEEE International Symposium on Integrated Network Management (IM’13), pp. 1282–1287

  30. Wang S, Dey S (2013) Adaptive mobile cloud computing to enable rich mobile multimedia applications. IEEE Trans Multimedia 15(4):870–883

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Mobile Systems Design Laboratory, Department of Electrical and Computer Engineering, University of California, San Diego, CA, USA

    Yao Lu, Yao Liu & Sujit Dey

Authors
  1. Yao Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sujit Dey
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yao Lu.

Additional information

This material is based upon work supported by the National Science Foundation (US) under Grant No. CCF-1160832.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lu, Y., Liu, Y. & Dey, S. Asymmetric and selective object rendering for optimized Cloud Mobile 3D Display Gaming user experience. Multimed Tools Appl 76, 18291–18320 (2017). https://doi.org/10.1007/s11042-016-3798-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-3798-z

Keywords

Search

Navigation