skip to main content
research-article

Apparent display resolution enhancement for moving images

Published: 26 July 2010 Publication History

Abstract

Limited spatial resolution of current displays makes the depiction of very fine spatial details difficult. This work proposes a novel method applied to moving images that takes into account the human visual system and leads to an improved perception of such details. To this end, we display images rapidly varying over time along a given trajectory on a high refresh rate display. Due to the retinal integration time the information is fused and yields apparent super-resolution pixels on a conventional-resolution display. We discuss how to find optimal temporal pixel variations based on linear eye-movement and image content and extend our solution to arbitrary trajectories. This step involves an efficient method to predict and successfully treat potentially visible flickering. Finally, we evaluate the resolution enhancement in a perceptual study that shows that significant improvements can be achieved both for computer generated images and photographs.

Supplementary Material

JPG File (tp088-10.jpg)
Supplemental material. (113.zip)
MP4 File (tp088-10.mp4)

References

[1]
Allen, W., and Ulichney, R. 2005. Wobulation: Doubling the addressed resolution of projection displays. In Proceedings of the Symposium Digest of Technical Papers (SID), vol. 47.4 of The Society for Information Display, 1514--1517.
[2]
Bijl, P., Schutte, K., and Hogervorst, M. A. 2006. Applicability of TOD, MTDP, MRT and DMRT for dynamic image enhancement techniques. In Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, vol. 6207.
[3]
Burr, D. C. 1979. Acuity for apparent vernier offset. Vision Research 19, 7, 835--837.
[4]
Coleman, T. F., and Li, Y. 1996. A reflective newton method for minimizing a quadratic function subject to bounds on some of the variables. SIAM J. on Optimization 6, 4, 1040--1058.
[5]
Curcio, C. A., Sloan, K. R., Kalina, R. E., and Hendrickson, A. E. 1990. Human photoreceptor topography. The Journal of Comparative Neurology 292, 4, 497--523.
[6]
Damera-Venkata, N., and Chang, N. L. 2009. Display supersampling. ACM Trans. Graph. 28, 1, 9:1--9:19.
[7]
de Lange, H. 1958. Research into the dynamic nature of the human fovea - Cortex systems with intermittent and modulated light. I. Attenuation characteristics with white and colored light. J. Opt. Soc. Am. 48, 11, 777--783.
[8]
Deering, M. F. 2005. A photon accurate model of the human eye. ACM Trans. Graph. (Proc. SIGRAPH 2005) 24, 3, 649--658.
[9]
Fahle, M., and Poggio, T. 1981. Visual hyperacuity: Spatiotemporal interpolation in human vision. Proceedings of the Royal Society of London. Series B, Biological Sciences 213, 1193, 451--477.
[10]
Hara, Z., and Shiramatsu, N. 2000. Improvement in the picture quality of moving pictures for matrix displays. J. SID 8, 2, 129--137.
[11]
Kalloniatis, M., and Luu, C. 2009. Temporal resolution. http://webvision.med.utah.edu/temporal.html.
[12]
Klompenhouwer, M. A., and de Haan, G. 2003. Subpixel image scaling for color-matrix displays. J. SID 11, 1, 99--108.
[13]
Kopf, J., Uyttendaele, M., Deussen, O., and Cohen, M. 2007. Capturing and viewing gigapixel images. ACM Trans. Graph. (Proc. SIGGRAPH 2007) 26, 3.
[14]
Krapels, K., Driggers, R. G., and Teaney, B. 2005. Target-acquisition performance in undersampled infrared imagers: static imagery to motion video. Applied Optics 44, 33, 7055--7061.
[15]
Krauzlis, R., and Lisberger, S. 1994. Temporal properties of visual motion signals for the initiation of smooth pursuit eye movements in monkeys. J Neurophysiol. 72, 1, 150--162.
[16]
Laird, J., Rosen, M., Pelz, J., Montag, E., and Daly, S. 2006. Spatio-velocity CSF as a function of retinal velocity using unstabilized stimuli. In Human Vision and Electronic Imaging XI, vol. 6057 of SPIE Proceedings Series, 32--43.
[17]
Mäkelä, P., Rovamo, J., and Whitaker, D. 1994. Effects of luminance and external temporal noise on flicker sensitivity as a function of stimulus size at various eccentricities. Vision Research 34, 15, 1981--91.
[18]
Martinez-Conde, S., Macknik, S. L., and Hubel, D. H. 2004. The role of fixational eye movements in visual perception. Nature Reviews Neuroscience 5, 3, 229--239.
[19]
McKee, S. P., and Taylor, D. G. 1984. Discrimination of time: comparison of foveal and peripheral sensitivity. J. Opt. Soc. Am. A 1, 6, 620--628.
[20]
Messing, D. S., and Kerofsky, L. J. 2006. Using optimal rendering to visually mask defective subpixels. In Human Vision and Electr. Imaging XI, vol. 6057 of SPIE Proc. Series, 236--247.
[21]
Mitchell, D. P., and Netravali, A. N. 1988. Reconstruction filters in computer-graphics. Proc. SIGGRAPH 22, 4, 221--228.
[22]
Park, S., Park, M., and Kang, M. 2003. Super-resolution image reconstruction: A technical overview. IEEE Signal Processing Magazine 20, 3, 21--36.
[23]
Platt, J. 2000. Optimal filtering for patterned displays. Signal Processing Letters, IEEE 7, 7, 179--181.
[24]
Purves, D., Shimpi, A., and Lotto, B. R. 1999. An empirical explanation of the Cornsweet effect. J. Neuroscience 19, 19, 8542--8551.
[25]
Schütz, A. C., Braun, D. I., Kerzel, D., and Gegenfurtner, K. R. 2008. Improved visual sensitivity during smooth pursuit eye movements. Nat. Neuroscience 11, 10, 1211--1216.
[26]
Tekalp, A. 1995. Digital Video Processing. Prentice Hall.
[27]
Van Hateren, J. H. 2005. A cellular and molecular model of response kinetics and adaptation in primate cones and horizontal cells. J. Vision 5, 4, 331--347.
[28]
Wandell, B. 1995. Foundations of Vision. Sinauer Associates.
[29]
Zavagno, D., and Caputo, G. 2001. The glare effect and the perception of luminosity. Perception 30, 2, 209--222.

Cited By

View all
  • (2022)Perceptual Visibility Model for Temporal Contrast Changes in PeripheryACM Transactions on Graphics10.1145/356424142:2(1-16)Online publication date: 22-Nov-2022
  • (2022)Preemptive text warping to prevent appearance of motion blurThe Visual Computer10.1007/s00371-022-02545-838:9-10(3391-3403)Online publication date: 28-Jun-2022
  • (2021)Learning to Perceive: Perceptual Resolution Enhancement for VR Display with Efficient Neural Network Processing2021 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct54149.2021.00036(133-138)Online publication date: Oct-2021
  • Show More Cited By

Index Terms

  1. Apparent display resolution enhancement for moving images

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Transactions on Graphics
      ACM Transactions on Graphics  Volume 29, Issue 4
      July 2010
      942 pages
      ISSN:0730-0301
      EISSN:1557-7368
      DOI:10.1145/1778765
      Issue’s Table of Contents
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 26 July 2010
      Published in TOG Volume 29, Issue 4

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. eye tracking
      2. image reconstruction
      3. perception

      Qualifiers

      • Research-article

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)9
      • Downloads (Last 6 weeks)0
      Reflects downloads up to 13 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2022)Perceptual Visibility Model for Temporal Contrast Changes in PeripheryACM Transactions on Graphics10.1145/356424142:2(1-16)Online publication date: 22-Nov-2022
      • (2022)Preemptive text warping to prevent appearance of motion blurThe Visual Computer10.1007/s00371-022-02545-838:9-10(3391-3403)Online publication date: 28-Jun-2022
      • (2021)Learning to Perceive: Perceptual Resolution Enhancement for VR Display with Efficient Neural Network Processing2021 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct54149.2021.00036(133-138)Online publication date: Oct-2021
      • (2021)A Resolution-Enhanced Digital Micromirror Device (DMD) Projection SystemIEEE Access10.1109/ACCESS.2021.30825649(78153-78164)Online publication date: 2021
      • (2020)Patch scanning displays: spatiotemporal enhancement for displaysOptics Express10.1364/OE.38085828:2(2107)Online publication date: 15-Jan-2020
      • (2019)Temporal Resolution Multiplexing: Exploiting the limitations of spatio-temporal vision for more efficient VR renderingIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2019.2898741(1-1)Online publication date: 2019
      • (2018)Long working range light field microscope with fast scanning multifocal liquid crystal microlens arrayOptics Express10.1364/OE.26.01098126:8(10981)Online publication date: 13-Apr-2018
      • (2018)Resolution Enhancement for AR/VR Displays Based on LC Sub-pixel Frame ShiftFrontiers in Optics / Laser Science10.1364/FIO.2018.JTu2A.26(JTu2A.26)Online publication date: 2018
      • (2018)Real-time Apparent Resolution Enhancement for Head-mounted DisplaysProceedings of the ACM on Computer Graphics and Interactive Techniques10.1145/32032021:1(1-15)Online publication date: 25-Jul-2018
      • (2018)63‐3: Real‐time Spatial‐based Projector Resolution EnhancementSID Symposium Digest of Technical Papers10.1002/sdtp.1224349:1(831-834)Online publication date: 30-May-2018
      • Show More Cited By

      View Options

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media