Applying Holo360 Video and Image Super-Resolution Generative Adversarial Networks to Virtual Reality Immersion

Abstract

Super-resolution deep learning methods focus on image processing solutions and discussions in two-dimensional super-resolution image processing NOT for 360 equirectangular images. Therefore, the motivation of this research is to establish the deep learning network model Holo360 SRGAN and data set of 360 equirectangular images, and observe whether the sharpness and noise of Holo360 SRGAN compared with the original image reach the optical verification standard. The results of this study point out two significant points: 1) For a convolution training core neuron with the best model architecture of Holo360 SRGAN with 360 images 8 K (8192 × 4096 px), FOV: 360°, the expanded the convolution core neuron size as 5 × 5 to contains more learning features. And 2) Holo360 SRGAN image experiment results, 6 ROI optical analysis clarity increased by 27%, and sharpness increased by 42%. The experimental original image noise SNR is 28.2 dB, and the Holo360 SRGAN (×2) noise SNR is 36.8 dB, so it is increased by +8.6 dB, and the amount of image detail is also increased. Contributions enhance the super-resolution visual experience of equirectangular video or image.