Abstract
Super-resolution is an area of Computer Vision comprising various techniques to recover a high-resolution image from a low-resolution counterpart. These techniques can also be used to enhance a low-resolution input image without a native high-resolution original. Single Image Super-Resolution (SISR) techniques aim to do this in a picture-by-picture fashion. In recent years, deep learning models have achieved the best performance, using neural networks to find a mapping between an input low-resolution image and its high-resolution analogous. This work will compare three approaches based on some of the most notable works in neural-network based super-resolution: SRCNN, FSRCNN, and ESRGAN. These methods will be used to enhance 3D computer-generated low-resolution pictures obtained from popular video games and will be evaluated with respect to the quality of the enhanced picture and the required computation time. From our study, we can attest to the superiority of neural network-based methods on the SISR problem and the benefits of taking a perceptual approach when the quality of the resulting images.
Supported by the Spanish National Project TED2021-129151B-I00/AEI/10.13039/ 501100011033/European Union NextGenerationEU/PRTR and project PID2019-103871GB-I00 of the Spanish Ministry of Economy, Industry and Competitiveness.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Code available in GitHub: https://github.com/rafabs97/superresolution.
References
AMD FSR product page. https://www.amd.com/es/technologies/fidelityfx-super-resolution/. Accessed 19 Apr 2023
Grand Theft Auto V Official Site. https://www.rockstargames.com/es/gta-v/. Accessed 09 Apr 2023
LPIPS GitHub repository. https://github.com/richzhang/PerceptualSimilarity. Accessed 10 Apr 2023
Nvidia DLSS product page. https://www.nvidia.com/es-es/geforce/technologies/dlss/. Accessed 09 Apr 2023
Phantasy Star Online 2 Official Site. https://pso2.com/. Accessed 09 Apr 2023
Shadow of the Tomb Raider at Epic Games. https://store.epicgames.com/es-ES/p/shadow-of-the-tomb-raider/. Accessed 09 Apr 2023
Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016). https://doi.org/10.1109/TPAMI.2015.2439281
Dong, C., Loy, C.C., Tang, X.: Accelerating the super-resolution convolutional neural network. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 391–407. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_25
Dong, T., Yan, H., Parasar, M., Krisch, R.: RenderSR: a lightweight super-resolution model for mobile gaming upscaling. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 3086–3094 (2022). https://doi.org/10.1109/CVPRW56347.2022.00348
He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: ICCV (2015). https://doi.org/10.1109/ICCV.2015.123
Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.: Densely connected convolutional networks (2017). https://doi.org/10.1109/CVPR.2017.243
Jolicoeur-Martineau, A.: The relativistic discriminator: a key element missing from standard GAN. CoRR abs/1807.00734 (2018)
Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network, pp. 105–114 (2017). https://doi.org/10.1109/CVPR.2017.19
Li, X., Wu, Y., Zhang, W., Wang, R., Hou, F.: Deep learning methods in real-time image super-resolution: a survey. J. Real-Time Image Proc. 17(6), 1885–1909 (2020). https://doi.org/10.1007/s11554-019-00925-3
Wang, X., et al.: ESRGAN: enhanced super-resolution generative adversarial networks. In: ECCV Workshops (2018). https://doi.org/10.1007/978-3-030-11021-5_5
Wang, Z., Chen, J., Hoi, S.C.H.: Deep learning for image super-resolution: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 43(10), 3365–3387 (2021). https://doi.org/10.1109/TPAMI.2020.2982166
Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004). https://doi.org/10.1109/TIP.2003.819861
Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: CVPR (2018). https://doi.org/10.1109/CVPR.2018.00068
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Berral-Soler, R., Madrid-Cuevas, F.J., Ventura, S., Muñoz-Salinas, R., Marín-Jiménez, M.J. (2023). A Comparison of Neural Network-Based Super-Resolution Models on 3D Rendered Images. In: Tsapatsoulis, N., et al. Computer Analysis of Images and Patterns. CAIP 2023. Lecture Notes in Computer Science, vol 14184. Springer, Cham. https://doi.org/10.1007/978-3-031-44237-7_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-44237-7_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44236-0
Online ISBN: 978-3-031-44237-7
eBook Packages: Computer ScienceComputer Science (R0)