Abstract
With the advancement of 3D display technology, the original 3D stereo-view systems requiring 3D glasses have evolved to multiple-view naked-eyes 3D displays. For 2D broadcasting systems, we can pack one view and its corresponding depth map into a frame in the transmission side and generate multiple views by the depth image-based rendering (DIBR) engine at the receivers. Up to now, the centralized texture depth packing (CTDP) frame-compatible format is the best solution for view and depth packing. In this paper, we propose an improved CTDP depacking method and realize it with DIBR process to achieve real-time 3D multiview exhibition. To maintain the quality of the depacked depth map, in the CTDP depacking process, we suggest a deep learning-based guided depth upsampling network. The experimental results demonstrate that the proposed 3D multiview system can successfully depack CTDP video and generate 9-view 3D movies in real-time. The proposed guided depth upsampling network achieves a better quality of the generated views than the traditional algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Yang, W.-J., Yang, J.-F., Chen, G.-C., Chung, P.-C., Chung, M.F.: An assigned color depth packing method with centralized texture depth packing formats for 3D VR broadcasting services. Proc. IEEE J. Emerg. Sel. Topics Circuits Syst. 9(1), 122–132 (2019)
Yang, J.-F., Lee, K.-T., Chen, G.-C., Yang, W.-J., Yu, L.: A YCbCr color depth packing method and TTS extension for 3D video broadcasting services. IEEE Trans. Circuits Syst. Video Technol. 30(9), 3043–3053 (2020)
Hsiao, T., Yang, J.-F.: Improved depth upsampling and multi-view generation for depacking centralized texture depth depacked 3D videos. M.S. thesis, National Cheng Kung University, Tainan, Taiwan (2019)
Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 184–199 (2014). Springer, Cham. https://doi.org/10.1007/978-3-319-10593-2_13
Kim, J., Lee, J.K., Lee K.M.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2016)
Kim, J., Lee, J.K., Lee, K.M.: Deeply-recursive convolutional network for image super-resolution. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2016)
Shi, W., et al.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2016)
Tong, T., Li, G., Liu, X., Gao, Q.: Image super-resolution using dense skip connections. In: Proceedings of IEEE International Conference on Computer Vision (2017)
Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image super-resolution. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2018)
Li, J., Fang, F., Mei, K., Zhang, G.: Multi-scale residual network for image super-resolution. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11212, pp. 527–542. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01237-3_32
Fu, J., Liu, J., Jiang, J., Li, Bao, Y.: Scene segmentation with dual relation-aware attention network. Proc. IEEE Trans. Neural Netw. Learn. Syst. 32, 2547–2560 (2020)
Lee, K.-T., Liu, E.-R., Yang, J.-F., Hung, L.: An image-guided network for depth edge enhancement. EURASIP J. Image Video Process. (2022). https://doi.org/10.1186/s13640-022-00583-9
Acknowledgments
This work was partially supported by the Ministry of Science and Technology, Taiwan, under Grant MOST 111-2221-E-080 and Qualcomm, USA under Grant SOW#NAT-435536.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Yang, WJ., Chen, BX., Yang, JF. (2023). CTDP Depacking with Guided Depth Upsampling Networks for Realization of Multiview 3D Video. In: Arai, K. (eds) Advances in Information and Communication. FICC 2023. Lecture Notes in Networks and Systems, vol 651. Springer, Cham. https://doi.org/10.1007/978-3-031-28076-4_13
Download citation
DOI: https://doi.org/10.1007/978-3-031-28076-4_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-28075-7
Online ISBN: 978-3-031-28076-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)