skip to main content
research-article

Aliasing Detection in Rendered Images via a Multi-Task Learning

Published: 09 August 2024 Publication History

Abstract

As technology advances from simple 2D designs to intricate 3D environments, the demand for high-quality visuals in video games and interactive media necessitates robust image quality assessment (IQA) techniques. Traditional methods like PSNR and SSIM, reliant on reference images, struggle with the unique challenges of 3D rendered content, highlighting the need for specialized non-reference IQA approaches. This paper introduces a novel multi-task learning architecture that corrects and predicts aliasing artifacts simultaneously, enhancing predictive accuracy without reference images. It also incorporates temporal information to improve visual coherence and smoothness. An automated labeling pipeline developed using Unity ensures a stable and unbiased dataset for model training and evaluation. Our experiments demonstrate that this approach reliably detects aliasing across various complexities, achieving state-of-the-art performance. By addressing specific challenges in rendered image assessment and leveraging innovative learning techniques, our work advances IQA for video games and simulations, ensuring high visual quality.

Supplemental Material

MP4 File - supplemental video
supplemental video

References

[1]
AMD. 2023. FidelityFX Super Resolution 2. https://gpuopen.com/fidelityfx-superresolution-2/.
[2]
Pontus Andersson, Jim Nilsson, Tomas Akenine-Möller, Magnus Oskarsson, Karl Johan Åström, and Mark D. Fairchild. 2020. FLIP: A Difference Evaluator for Alternating Images. Proc. ACM Comput. Graph. Interact. Tech. 3 (2020), 15:1--15:23. https://api.semanticscholar.org/CorpusID:220643528
[3]
Saeed Anwar and Nick Barnes. 2019. Real Image Denoising With Feature Attention. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV). 3155--3164. https://doi.org/10.1109/ICCV.2019.00325
[4]
Steve Bako, Thijs Vogels, Brian McWilliams, Mark Meyer, Jan Novák, Alex Harvill, Pradeep Sen, Tony DeRose, and Fabrice Rousselle. 2017. Kernel-predicting convolutional networks for denoising Monte Carlo renderings. ACM Transactions on Graphics (TOG) 36 (2017), 1--14. https://api.semanticscholar.org/CorpusID:31004998
[5]
Bolun Cai, Xiangmin Xu, Kui Jia, Chunmei Qing, and Dacheng Tao. 2016. DehazeNet: An End-to-End System for Single Image Haze Removal. IEEE Transactions on Image Processing 25 (2016), 5187--5198. https://api.semanticscholar.org/CorpusID:14092238
[6]
Marcos V. Conde, Ui-Jin Choi, Maxime Burchi, and Radu Timofte. 2022. Swin2SR: SwinV2 Transformer for Compressed Image Super-Resolution and Restoration. In ECCV Workshops. https://api.semanticscholar.org/CorpusID:252519482
[7]
Killian Herveau, Max Piochowiak, and Carsten Dachsbacher. 2023. Minimal Convolutional Neural Networks for Temporal Anti Aliasing. https://api.semanticscholar.org/CorpusID:259305872
[8]
Jiaxi Jiang, Kai Zhang, and Radu Timofte. 2021. Towards Flexible Blind JPEG Artifacts Removal. arXiv:2109.14573 [eess.IV]
[9]
Arthur Juliani, Vincent-Pierre Berges, Ervin Teng, Andrew Cohen, Jonathan Harper, Chris Elion, Chris Goy, Yuan Gao, Hunter Henry, Marwan Mattar, and Danny Lange. 2020. Unity: A General Platform for Intelligent Agents. arXiv:1809.02627 [cs.LG]
[10]
Jingyun Liang, Jie Cao, Guolei Sun, K. Zhang, Luc Van Gool, and Radu Timofte. 2021. SwinIR: Image Restoration Using Swin Transformer. 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) (2021), 1833--1844. https://api.semanticscholar.org/CorpusID:237266491
[11]
Ding Liu, Bihan Wen, Yuchen Fan, Chen Change Loy, and Thomas S. Huang. 2018a. Non-Local Recurrent Network for Image Restoration. In Neural Information Processing Systems. https://api.semanticscholar.org/CorpusID:47007607
[12]
Pengju Liu, Hongzhi Zhang, K. Zhang, Liang Lin, and Wangmeng Zuo. 2018b. Multi-level Wavelet-CNN for Image Restoration. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (2018), 886--88609. https://api.semanticscholar.org/CorpusID:29151865
[13]
Rafał K. Mantiuk, Gyorgy Denes, Alexandre Chapiro, Anton Kaplanyan, Gizem Rufo, Romain Bachy, Trisha Lian, and Anjul Patney. 2021. FovVideoVDP: a visible difference predictor for wide field-of-view video. ACM Trans. Graph. 40, 4, Article 49 (jul 2021), 19 pages. https://doi.org/10.1145/3450626.3459831
[14]
Rafał K. Mantiuk, Kil Joong Kim, Allan G. Rempel, and Wolfgang Heidrich. 2011. HDR-VDP-2: a calibrated visual metric for visibility and quality predictions in all luminance conditions. ACM SIGGRAPH 2011 papers (2011). https://api.semanticscholar.org/CorpusID:756729
[15]
Xiaoxu Meng, Quan Zheng, Amitabh Varshney, Gurprit Singh, and Matthias Zwicker. 2020. Real-time Monte Carlo Denoising with the Neural Bilateral Grid. In Eurographics Symposium on Rendering. https://api.semanticscholar.org/CorpusID:220284605
[16]
Anjul Patney and Aaron Lefohn. 2018. Detecting Aliasing Artifacts in Image Sequences Using Deep Neural Networks. In Proceedings of the Conference on High-Performance Graphics (Vancouver, British Columbia, Canada) (HPG '18). Association for Computing Machinery, New York, NY, USA, Article 4, 4 pages. https://doi.org/10.1145/3231578.3231580
[17]
Karen Simonyan and Andrew Zisserman. 2014. Very Deep Convolutional Networks for Large-Scale Image Recognition. CoRR abs/1409.1556 (2014). https://api.semanticscholar.org/CorpusID:14124313
[18]
Manu Mathew Thomas, Karthikeyan Vaidyanathan, Gabor Liktor, and Angus Graeme Forbes. 2020. A reduced-precision network for image reconstruction. ACM Transactions on Graphics (TOG) 39 (2020), 1 - 12. https://api.semanticscholar.org/CorpusID:221492171
[19]
Etienne Vouga, Christopher Wojtan, Yu-Xiao Guo, Guojun Chen, Yue Dong, and Xin Tong. 2022. Classifier Guided Temporal Supersampling for Real-time Rendering. Computer Graphics Forum 41 (2022). https://api.semanticscholar.org/CorpusID:254249752
[20]
Zhou Wang, Alan Conrad Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13 (2004), 600--612. https://api.semanticscholar.org/CorpusID:207761262
[21]
Zhendong Wang, Xiaodong Cun, Jianmin Bao, and Jianzhuang Liu. 2021. Uformer: A General U-Shaped Transformer for Image Restoration. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021), 17662--17672. https://api.semanticscholar.org/CorpusID:235358213
[22]
Lei Xiao, Salah Nouri, Matthew Chapman, Alexander Fix, Douglas Lanman, and Anton Kaplanyan. 2020. Neural supersampling for real-time rendering. ACM Transactions on Graphics (TOG) 39 (2020), 142:1 - 142:12. https://api.semanticscholar.org/CorpusID:221105079
[23]
Zongsheng Yue, Hongwei Yong, Qian Zhao, Lei Zhang, and Deyu Meng. 2019. Variational Denoising Network: Toward Blind Noise Modeling and Removal. ArXiv abs/1908.11314 (2019). https://api.semanticscholar.org/CorpusID:201667906
[24]
Syed Waqas Zamir, Aditya Arora, Salman Hameed Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang. 2021. Restormer: Efficient Transformer for High-Resolution Image Restoration. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021), 5718--5729. https://api.semanticscholar.org/CorpusID:244346144
[25]
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. 2018. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In CVPR.

Index Terms

  1. Aliasing Detection in Rendered Images via a Multi-Task Learning

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image Proceedings of the ACM on Computer Graphics and Interactive Techniques
      Proceedings of the ACM on Computer Graphics and Interactive Techniques  Volume 7, Issue 3
      August 2024
      363 pages
      EISSN:2577-6193
      DOI:10.1145/3688389
      Issue’s Table of Contents
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 09 August 2024
      Published in PACMCGIT Volume 7, Issue 3

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. aliasing detection
      2. image quality assessment
      3. machine learning
      4. multi-task learning

      Qualifiers

      • Research-article
      • Research
      • Refereed

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • 0
        Total Citations
      • 101
        Total Downloads
      • Downloads (Last 12 months)101
      • Downloads (Last 6 weeks)6
      Reflects downloads up to 03 Mar 2025

      Other Metrics

      Citations

      View Options

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media