skip to main content
10.1145/1730804.1730819acmconferencesArticle/Chapter ViewAbstractPublication Pagesi3dConference Proceedingsconference-collections
research-article

Spatio-temporal upsampling on the GPU

Published: 19 February 2010 Publication History

Abstract

Pixel processing is becoming increasingly expensive for real-time applications due to the complexity of today's shaders and high-resolution framebuffers. However, most shading results are spatially or temporally coherent, which allows for sparse sampling and reuse of neighboring pixel values. This paper proposes a simple framework for spatio-temporal upsampling on modern GPUs. In contrast to previous work, which focuses either on temporal or spatial processing on the GPU, we exploit coherence in both. Our algorithm combines adaptive motion-compensated filtering over time and geometry-aware upsampling in image space. It is robust with respect to high-frequency temporal changes, and achieves substantial performance improvements by limiting the number of recomputed samples per frame. At the same time, we increase the quality of spatial upsampling by recovering missing information from previous frames. This temporal strategy also allows us to ensure that the image converges to a higher quality result.

References

[1]
Adelson, S. J., and Hughes, L. F. 1995. Generating Exact Ray-Traced Animation Frames by Reprojection. IEEE Computer Graphics & Applications 15, 3, 43--53.
[2]
Bala, K., Dorsey, J., and Teller, S. 1999. Ray-Traced Interactive Scene Editing Using Ray Segment Trees. In Proceedings of the 10th Eurographics Workshop on Rendering.
[3]
Bavoil, L., and Sainz, M. 2008. Image-space horizon-based ambient occlusion. In SIGGRAPH 2008, Talk Program.
[4]
Bennett, E. P., and McMillan, L. 2005. Video enhancement using per-pixel virtual exposures. ACM Transactions on Graphics 24, 3, 845--852.
[5]
Bishop, G., Fuchs, H., McMillan, L., and Zagier, E. J. S. 1994. Frameless rendering: double buffering considered harmful. In Proceedings of SIGGRAPH '94, 175--176.
[6]
Calle, D., and Montanvert, A. 1998. Super-resolution inducing of an image. Image Processing, International Conference on 3, 232.
[7]
Dachsbacher, C., and Stamminger, M. 2005. Reflective shadow maps. In I3D '05: Proceedings of the symposium on Interactive 3D graphics and games, 203--231.
[8]
Damez, C., Dmitriev, K., and Myszkowski, K. 2003. State of the art in global illumination for interactive applications and high-quality animations. Computer Graphics Forum 22, 1 (Mar.), 55--77.
[9]
Dayal, A., Woolley, C., Watson, B., and Luebke, D. P. 2005. Adaptive frameless rendering. In Rendering Techniques, 265--275.
[10]
Eisemann, E., and Durand, F. 2004. Flash photography enhancement via intrinsic relighting. In ACM Transactions on Graphics (Proceedings of Siggraph Conference), ACM Press, vol. 23.
[11]
Hasselgren, J., and Akenine-Möller, T. 2007. PCU: the programmable culling unit. In Proc. of SIGGRAPH '07, ACM.
[12]
Koltun, V., Chrysanthou, Y., and Cohen-or, D. 2000. Virtual occluders: An efficient intermediate pvs representation. In 11th Eurographics Workshop on Rendering, 59--70.
[13]
Kopf, J., Cohen, M. F., Lischinski, D., and Uyttendaele, M. 2007. Joint bilateral upsampling. ACM Transactions on Graphics 26, 3 (July).
[14]
Mark, W., McMillan, L., and Bishop, G. 1997. Post-rendering 3D warping. In 1997 Symposium on Interactive 3D Graphics, ACM SIGGRAPH, 7--16.
[15]
Nehab, D., Sander, P. V., Lawrence, J., Tatarchuk, N., and Isidoro, J. R. 2007. Accelerating real-time shading with reverse reprojection caching. In Graphics Hardware.
[16]
Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., and Toyama, K. 2004. Digital photography with flash and no-flash image pairs. ACM Transactions on Graphics (Proceedings of Siggraph Conference) 23, 3, 664--672.
[17]
Reeves, W. T., Salesin, D. H., and Cook, R. L. 1987. Rendering antialiased shadows with depth maps. In Proc. of Siggraph'87.
[18]
Ritschel, T., Grosch, T., and Seidel, H.-P. 2009. Approximating Dynamic Global Illumination in Screen Space. In Proc. of ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games.
[19]
Scherzer, D., Jeschke, S., and Wimmer, M. 2007. Pixel-correct shadow maps with temporal reprojection and shadow test confidence. In Rendering Techniques, 45--50.
[20]
Segovia, B., Iehl, J. C., Mitanchey, R., and Proche, B. 2006. Bidirectional Instant Radiosity. In Rendering Techniques, 389--397.
[21]
Shinya, M. 1993. Spatial Anti-aliasing for Animation Sequences with Spatio-temporal Filtering. In Proceedings of SIGGRAPH '93, 289--296.
[22]
Sitthi-amorn, P., Lawrence, J., Yang, L., Sander, P. V., and Nehab, D. 2008. An improved shading cache for modern gpus. In Graphics Hardware.
[23]
Sitthi-amorn, P., Lawrence, J., Yang, L., Sander, P. V., Nehab, D., and Xi, J. 2008. Automated reprojection-based pixel shader optimization. In SIGGRAPH Asia '08, ACM, 1--11.
[24]
Tekalp, A. M. 1995. Digital video Processing. Prentice Hall.
[25]
Wald, I., Kollig, T., Benthin, C., Keller, A., and Slusallek, P. 2002. Interactive global illumination using fast ray tracing. In Rendering Techniques, 15--24.
[26]
Walter, B., Drettakis, G., and Parker, S. 1999. Interactive Rendering using the Render Cache. In Proceedings of the 10th Eurographics Workshop on Rendering, 235--246.
[27]
Yang, L., Sander, P. V., and Lawrence, J. 2008. Geometry-aware framebuffer level of detail. In Rendering Techniques.
[28]
Yang, L., Nehab, D., Sander, P. V., Sitthi-amorn, P., Lawrence, J., and Hoppe, H. 2009. Amortized supersampling. Proc. of SIGGRAPH Asia '09.

Cited By

View all
  • (2024)FASSET: Frame Supersampling and Extrapolation Using Implicit Neural Representations of Rendering ContentsComputational Visual Media10.1007/978-981-97-2095-8_10(177-196)Online publication date: 30-Mar-2024
  • (2024)Neural foveated super‐resolution for real‐time VR renderingComputer Animation and Virtual Worlds10.1002/cav.228735:4Online publication date: 11-Jul-2024
  • (2023)Efficient neural supersampling on a novel gaming dataset2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.00034(296-306)Online publication date: 1-Oct-2023
  • Show More Cited By
  1. Spatio-temporal upsampling on the GPU

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    I3D '10: Proceedings of the 2010 ACM SIGGRAPH symposium on Interactive 3D Graphics and Games
    February 2010
    201 pages
    ISBN:9781605589398
    DOI:10.1145/1730804
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 19 February 2010

    Permissions

    Request permissions for this article.

    Check for updates

    Qualifiers

    • Research-article

    Conference

    I3D '10
    Sponsor:
    I3D '10: Symposium on Interactive 3D Graphics and Games
    February 19 - 21, 2010
    D.C., Washington

    Acceptance Rates

    Overall Acceptance Rate 148 of 485 submissions, 31%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)21
    • Downloads (Last 6 weeks)6
    Reflects downloads up to 13 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)FASSET: Frame Supersampling and Extrapolation Using Implicit Neural Representations of Rendering ContentsComputational Visual Media10.1007/978-981-97-2095-8_10(177-196)Online publication date: 30-Mar-2024
    • (2024)Neural foveated super‐resolution for real‐time VR renderingComputer Animation and Virtual Worlds10.1002/cav.228735:4Online publication date: 11-Jul-2024
    • (2023)Efficient neural supersampling on a novel gaming dataset2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.00034(296-306)Online publication date: 1-Oct-2023
    • (2021)Temporally Adaptive Shading Reuse for Real-Time Rendering and Virtual RealityACM Transactions on Graphics10.1145/344679040:2(1-14)Online publication date: 27-Apr-2021
    • (2021)Interactive Black-Hole VisualizationIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2020.303045227:2(796-805)Online publication date: Feb-2021
    • (2020)A Survey of Temporal Antialiasing TechniquesComputer Graphics Forum10.1111/cgf.1401839:2(607-621)Online publication date: 13-Jul-2020
    • (2020)Cast Shadow Generation Using Generative Adversarial NetworksComputational Science – ICCS 202010.1007/978-3-030-50426-7_36(481-495)Online publication date: 15-Jun-2020
    • (2019)Temporally dense ray tracingProceedings of the Conference on High-Performance Graphics10.2312/hpg.20191193(33-38)Online publication date: 8-Jul-2019
    • (2019)Handheld multi-frame super-resolutionACM Transactions on Graphics10.1145/3306346.332302438:4(1-18)Online publication date: 12-Jul-2019
    • (2019)Single Image Dehazing via NIN-DehazeNetIEEE Access10.1109/ACCESS.2019.29586077(181348-181356)Online publication date: 2019
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media