Skip to main content
SpringerLink
Log in

Integrated approach of streaming 3d multimedia contents in real-time for mobile devices

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

It is popular to watch a 3D video through a 3D display nowadays. However, it is still difficult to enjoy the 3D multimedia contents with a mobile device even if a mobile device with a 3D display is currently introduced into the market. The main technological challenges for watching 3D contents via the mobile devices can be identified as the following: generating and streaming 3D contents. Generating 3D contents requires extra computational resources. Moreover, streaming 3D contents demands additional network bandwidth for receiving and transmitting the 3D data. To overcome these technological challenges, we propose ReMA, a novel 3D video streaming system in this paper. We devised a novel architecture for transmitter, receiver, and a distribution system to efficiently disseminate and generate 3D videos for the mobile devices. We implemented ReMA in a real test-bed and conducted a thorough empirical evaluation study to see the feasibility of streaming 3D contents for the mobile devices. Based on our empirical study, the resulting system presents a great promise in streaming 3D video in real-time to the mobile devices.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. We employed Kinect for Microsoft’s Xbox 360 [20].

  2. 3D data from a smartphone with a 3D camera was not accessible for computation.

  3. The performance of the H.264 is not solely relied on the CPU, but it is also assisted with an additional computational resources such as GPU.

References

  1. Akhshabi S, Begen AC, Dovrolis C (2011) An experimental evaluation of rate-adaptation algorithms in adaptive streaming over http. In: Proceedings of the second annual ACM conference on Multimedia systems. ACM, pp 157–168

  2. Algorri JF, del Pozo VU, Sánchez-Pena JM, Otón JM (2014) An autostereoscopic device for mobile applications based on a liquid crystal microlens array and an oled display. J Disp Technol 10(9):713–720

    Article  Google Scholar 

  3. Apple iOS SDK: https://developer.apple.com/devcenter/ios/index.action

  4. Calagari K, Templin K, Elgamal T, Diab K, Didyk P, Matusik W, Hefeeda M (2014) Anahita: A system for 3d video streaming with depth customization. In: Proceedings of the ACM International Conference on Multimedia. ACM, pp 337–346

  5. Chikkerur S, Sundaram V, Reisslein M, Karam LJ (2011) Objective video quality assessment methods: A classification, review, and performance comparison. IEEE Trans Broadcast 57(2):165–182

    Article  Google Scholar 

  6. Dodgson NA (2005) Autostereoscopic 3d displays. Computer 38(8):31–36

    Article  Google Scholar 

  7. Fattal D, Peng Z, Tran T, Vo S, Fiorentino M, Brug J, Beausoleil RG (2013) A multi-directional backlight for a wide-angle, glasses-free three-dimensional display. Nature 495(7441):348–351

    Article  Google Scholar 

  8. Flack J, Harrold J, Woodgate GJ, et al. (2007) A prototype 3d mobile phone equipped with a next generation autostereoscopic display. In: Proceedings SPIE, vol 6490, pp 502–523

  9. Google Android SDK: http://developer.android.com/sdk/index.html

  10. Google Inc.: Project Tango. https://www.google.com/atap/projecttango/

  11. Halle M (2005) Autostereoscopic displays and computer graphics. In: ACM SIGGRAPH 2005 Courses, p 104

  12. Kamolrat B, Fernando W, Mrak M, Kondoz A (2008) Joint source and channel coding for 3d video with depth image-based rendering. IEEE Trans Consum Electron 54(2):887–894

    Article  Google Scholar 

  13. Kunic S, Sego Z (2011) 3d television. In: 2011 Proceedings of ELMAR, pp 127–131

  14. Lee SK Comparison between rema-vc and h.264. [Video] Retrieved from (2015). http://www.youtube.com/watch?v=bm2atlgH4w4

  15. Lee SK, Kim H, Lee W, Kim H, Jung J, Kim H (2014) ReMA: Real-Time 3D video streaming system for mobile devices. In: 33rd IEEE International Performance Computing and Communications Conference (IPCCC 2014). USA, Austin, Texas

  16. Lee SK, Yoo S, Jung J, Kim H, Ryoo J (2015) Link-aware reconfigurable point-to-point video streaming for mobile devices. ACM Trans Multimed Comput Commun Appl (TOMM) 12(1):9

    Google Scholar 

  17. Lee SK, Yoo S, Kim H (2016) Devising a user collaboration scheme to automatically generate videos. Multimedia Tools and Applications 75(8):4615–4638

    Article  Google Scholar 

  18. LG Electronics: LG Electronics Optimus 3D. http://www.lgmobile.co.kr/event/optimus3d/page.jsp

  19. McVeigh JS, Siegel M, Jordan AG (1996) Intermediate view synthesis considering occluded and ambiguously referenced image regions. Signal Process Image Commun 9(1):21–28

    Article  Google Scholar 

  20. Microsoft Corporation: Kinect for Windows. http://www.microsoft.com/en-us/kinectforwindows/

  21. Müller K, Schwarz H, Marpe D, Bartnik C, Bosse S, Brust H, Hinz T, Lakshman H, Merkle P, Rhee FH et al (2013) 3d high-efficiency video coding for multi-view video and depth data. IEEE Trans Image Process 22(9):3366–3378

    Article  MathSciNet  MATH  Google Scholar 

  22. Muller K, Smolic A, Dix K, Kauff P, Wiegand T (2008) Reliability-based generation and view synthesis in layered depth video. In: IEEE 10th Workshop on Multimedia Signal Processing, pp 34–39

  23. Muller K, Smolic A, Dix K, Merkle P, Wiegand T (2009) Coding and intermediate view synthesis of multiview video plus depth. In: 16th IEEE International Conference on Image Processing (ICIP), pp 741–744

  24. Okamoto Y, Oishi T, Ikeuchi K (2011) Image-based network rendering of large meshes for cloud computing. Int J Comput Vis 94(1):12–22

    Article  Google Scholar 

  25. OpenCV Team: Opencv. http://opencv.org/

  26. OpenKinect Project: http://openkinect.org/

  27. Pinson MH, Wolf S (2004) A new standardized method for objectively measuring video quality. IEEE Trans Broadcast 50(3):312–322

    Article  Google Scholar 

  28. Puri R, Ramchandran K (2002) Prism: A new robust video coding architecture based on distributed compression principles. In: Proceedings of the Annual Allerton Conference on Communication Control and Computing, vol 40, pp 586–595

  29. Recommendation I (2004) 144: Objective perceptual video quality measurement techniques for digital cable television in the presence of a full reference International Telecommunication Union

  30. i Ribera RB, Choi S, Kim Y, Lee J, Noh J (2012) Video panorama for 2d to 3d conversion. In: Computer Graphics Forum, vol 31, pp 2213–2222

  31. Saygili G, Gurler CG, Tekalp AM (2011) Evaluation of asymmetric stereo video coding and rate scaling for adaptive 3d video streaming. IEEE Trans Broadcast 57(2):593–601

    Article  Google Scholar 

  32. Schwarz H, Marpe D, Wiegand T (2007) Overview of the scalable video coding extension of the h. 264/avc standard. IEEE Trans Circuits Syst Video Technol 17(9):1103–1120

    Article  Google Scholar 

  33. Sexton I, Surman P (1999) Stereoscopic and autostereoscopic display systems. IEEE Signal Process Mag 16(3):85–99

    Article  Google Scholar 

  34. Sgardoni V, Nix A (2014) Raptor code-aware link adaptation for spectrally efficient unicast video streaming over mobile broadband networks

  35. Shi W, Lu Y, Li Z, Engelsma J (2011) Sharc: A scalable 3d graphics virtual appliance delivery framework in cloud. J Netw Comput Appl 34(4):1078–1087

    Article  Google Scholar 

  36. Stockhammer T (2011) Dynamic adaptive streaming over http–: standards and design principles. In: Proceedings of the second annual ACM conference on Multimedia systems. ACM, pp 133–144

  37. Vatolin D, Moskvin A, Petrov O, Trunichkin N (2009) Msu video quality measurement tool

  38. Vetro A, Wiegand T, Sullivan GJ (2011) Overview of the stereo and multiview video coding extensions of the h. 264/mpeg-4 avc standard. Proc IEEE 99(4):626–642

    Article  Google Scholar 

  39. Vetro A, Yea S, Smolic A (2008) Towards a 3d video format for auto-stereoscopic displays. In: Proceedings of the SPIE, San Diego, CA USA

  40. Xu Y, Zhou Y, Chiu DM (2014) Analytical qoe models for bit-rate switching in dynamic adaptive streaming systems

  41. Yeo C, Wang J, Ramchandran K (2007) View synthesis for robust distributed video compression in wireless camera networks. In: IEEE International Conference on Image Processing, vol 3, pp III–21

  42. Zhu W, Luo C, Wang J, Li S (2011) Multimedia cloud computing. IEEE Processing Magazine 28(3):59–69

    Article  Google Scholar 

Download references

Acknowledgments

This research was supported in part by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2015R1D1A1A01059151), and in part by ”Human Resources program in Energy Technology” of the Korea Institute of Energy Technology Evaluation and Planning (KETEP) granted financial resource from the Ministry of Trade, Industry & Energy, Republic of Korea (No. 20154030200610).

Author information

Authors and Affiliations

  1. LG Electronics Inc., Seoul, Korea

    Suk Kyu Lee

  2. School of Electrical Engineering, Korea University, Seoul, Korea

    Hyunsoon Kim, Albert Yongjoon Chung & Hwangnam Kim

Authors
  1. Suk Kyu Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hyunsoon Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Albert Yongjoon Chung
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hwangnam Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hwangnam Kim.

Additional information

A preliminary version of this work appeared in the IEEE International Performance Computing and Communications Conference (IPCCC) 2014 [15].

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, S.K., Kim, H., Yongjoon Chung, A. et al. Integrated approach of streaming 3d multimedia contents in real-time for mobile devices. Multimed Tools Appl 77, 1811–1842 (2018). https://doi.org/10.1007/s11042-016-4339-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-4339-5

Keywords

Search

Navigation