research-article

Cascaded displays: spatiotemporal superresolution using offset pixel layers

Authors:

Douglas Lanman,

David LuebkeAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 33, Issue 4

Article No.: 60, Pages 1 - 11

https://doi.org/10.1145/2601097.2601120

Published: 27 July 2014 Publication History

Abstract

We demonstrate that layered spatial light modulators (SLMs), subject to fixed lateral displacements and refreshed at staggered intervals, can synthesize images with greater spatiotemporal resolution than that afforded by any single SLM used in their construction. Dubbed cascaded displays, such architectures enable superresolution flat panel displays (e.g., using thin stacks of liquid crystal displays (LCDs)) and digital projectors (e.g., relaying the image of one SLM onto another). We introduce a comprehensive optimization framework, leveraging non-negative matrix and tensor factorization, that decomposes target images and videos into multi-layered, time-multiplexed attenuation patterns---offering a flexible trade-off between apparent image brightness, spatial resolution, and refresh rate. Through this analysis, we develop a real-time dual-layer factorization method that quadruples spatial resolution and doubles refresh rate. Compared to prior superresolution displays, cascaded displays place fewer restrictions on the hardware, offering thin designs without moving parts or the necessity of temporal multiplexing. Furthermore, cascaded displays are the first use of multi-layer displays to increase apparent temporal resolution. We validate these concepts using two custom-built prototypes: a dual-layer LCD and a dual-modulation liquid crystal on silicon (LCoS) projector, with the former emphasizing head-mounted display (HMD) applications.

Supplementary Material

ZIP File (a60-heide.zip)

Supplemental material.

Download
1212.31 MB

MP4 File (a60-sidebyside.mp4)

Download
33.45 MB

References

[1]

Aliaga, D. G., Yeung, Y. H., Law, A., Sajadi, B., and Majumder, A. 2012. Fast high-resolution appearance editing using superimposed projections. ACM Trans. Graph. 31, 2.

Digital Library

[2]

Allen, W., and Ulichney, R. 2005. Wobulation: Doubling the addressed resolution of projection displays. SID Symposium Digest of Technical Papers 36, 1, 1514--1517.

[3]

Baker, S., and Kanade, T. 2002. Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence 24, 9, 1167--1183.

Digital Library

[4]

Bell, G. P., Craig, R., Paxton, R., Wong, G., and Galbraith, D. 2008. Beyond flat panels: Multi layer displays with real depth. SID Digest 39, 1, 352--355.

[5]

Ben-Ezra, M., Zomet, A., and Nayar, S. K. 2004. Jitter camera: High resolution video from a low resolution detector. In IEEE Computer Vision and Pattern Recognition (CVPR), 135--142.

Digital Library

[6]

Berthouzoz, F., and Fattal, R. 2012. Apparent resolution enhancement for motion videos. In ACM Applied Perception.

Digital Library

[7]

Berthouzoz, F., and Fattal, R. 2012. Resolution enhancement by vibrating displays. ACM Trans. Graph. 31, 2, 15:1--15:14.

Digital Library

[8]

Blondel, V., Ho, N.-D., and van Dooren, P. 2008. Weighted nonnegative matrix factorization and face feature extraction. In Image and Vision Computing, 1--17.

[9]

Damera-Venkata, N., and Chang, N. L. 2009. Display supersampling. ACM Trans. Graph. 28, 1, 9:1--9:19.

Digital Library

[10]

Didyk, P., Eisemann, E., Ritschel, T., Myszkowski, K., and Seidel, H.-P. 2010. Apparent display resolution enhancement for moving images. ACM Trans. Graph. 29, 4, 113:1--113:8.

Digital Library

[11]

Gotoda, H. 2010. A multilayer liquid crystal display for autostereoscopic 3D viewing. In SPIE Stereoscopic Displays and Applications XXI, vol. 7524, 1--8.

[12]

Gotoda, H. 2011. Reduction of image blurring in an autostereoscopic multilayer liquid crystal display. In SPIE Stereoscopic Displays and Applications XXII, vol. 7863, 1--7.

[13]

Hart, W. M. 1987. The temporal responsiveness of vision. In Adler's Physiology of the Eye, R. A. Moses and W. M. Hart, Eds. C. V. Moseby Company.

[14]

Heide, F., Wetzstein, G., Raskar, R., and Heidrich, W. 2013. Adaptive image synthesis for compressive displays. ACM Trans. Graph. 32, 4, 132:1--132:12.

Digital Library

[15]

Heide, F., Gregson, J., Wetzstein, G., Raskar, R., and Heidrich, W. 2014. A compressive multi-mode superresolution display. Technical Report arXiv:1404.5916.

[16]

Ho, N.-D. 2008. Nonnegative Matrix Factorization Algorithms and Applications. PhD thesis, Université catholique de Louvain.

[17]

Ives, F. E., 1903. Parallax stereogram and process of making same. U.S. Patent 725,567.

[18]

Jaynes, C., and Ramakrishnan, D. 2003. Super-resolution composition in multi-projector displays. In IEEE Projector-Camera Systems (PROCAMS).

[19]

Kusakabe, Y., Kanazawa, M., Nojiri, Y., Furuya, M., and Yoshimura, M. 2008. A YC-separation-type projector: High dynamic range with double modulation. Journal of the Society for Information Display 16, 2, 383--391.

[20]

Kusakabe, Y., Kanazawa, M., Nojiri, Y., Furuya, M., and Yoshimura, M. 2009. A high-dynamic-range and high-resolution projector with dual modulation. Proc. SPIE 7241.

[21]

Lanman, D., Hirsch, M., Kim, Y., and Raskar, R. 2010. Content-adaptive parallax barriers: Optimizing dual-layer 3D displays using low-rank light field factorization. ACM Trans. Graph. 29, 6, 163:1--163:10.

Digital Library

[22]

Lanman, D., Wetzstein, G., Hirsch, M., Heidrich, W., and Raskar, R. 2011. Polarization fields: Dynamic light field display using multi-layer LCDs. ACM Trans. Graph. 30, 6, 186:1--186:10.

Digital Library

[23]

Lippmann, G. 1908. Épreuves réversibles donnant la sensation du relief. Journal of Physics 7, 4, 821--825.

[24]

Majumder, A. 2005. Is spatial super-resolution feasible using overlapping projectors? In IEEE Acoustics, Speech, and Signal Processing (ICASSP), vol. 4.

[25]

Olson, J., Krum, D., Suma, E., and Bolas, M. 2011. A design for a smartphone-based head mounted display. In IEEE Virtual Reality, 233--234.

Digital Library

[26]

Pavlovych, A., and Stuerzlinger, W. 2005. A high-dynamic range projection system. Proc. SPIE 5969.

[27]

Sajadi, B., Gopi, M., and Majumder, A. 2012. Edge-guided resolution enhancement in projectors via optical pixel sharing. ACM Trans. Graph. 31, 4, 79:1--79:122.

Digital Library

[28]

Sajadi, B., Qoc-Lai, D., Ihler, A., Gopi, M., and Majumder, A. 2013. Image enhancement in projectors via optical pixel shift and overlay. In IEEE International Conference on Computational Photography (ICCP), 1--10.

[29]

Seetzen, H., Heidrich, W., Stuerzlinger, W., Ward, G., Whitehead, L., Trentacoste, M., Ghosh, A., and Vorozcovs, A. 2004. High dynamic range display systems. ACM Trans. Graph. 23, 3, 760--768.

Digital Library

[30]

Wang, Z., Bovik, A., Sheikh, H., and Simoncelli, E. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13, 4, 600--612.

Digital Library

[31]

Watson, B. A., and Hodges, L. F. 1995. Using texture maps to correct for optical distortion in head-mounted displays. In IEEE Virtual Reality Annual International Symposium, 172--178.

Digital Library

[32]

Wetzstein, G., Lanman, D., Heidrich, W., and Raskar, R. 2011. Layered 3D: Tomographic image synthesis for attenuation-based light field and high dynamic range displays. ACM Trans. Graph. 30, 4, 95:1--95:12.

Digital Library

[33]

Wetzstein, G., Lanman, D., Hirsch, M., and Raskar, R. 2012. Tensor displays: Compressive light field synthesis using multilayer displays with directional backlighting. ACM Trans. Graph. 31, 4, 80:1--80:11.

Digital Library

[34]

Woo, S., Kim, D.-H., Han, Y. S., and Choi, B.-D. 2011. Full-color LCD microdisplay system based on OLED backlight unit and field-sequential color driving method. Journal of Photoenergy.

Cited By

Chen LWeng LCheng HCheng AHuang C(2025)Temporal Fusion: Continuous-Time Light Field Video FactorizationIEEE Transactions on Image Processing10.1109/TIP.2025.353320334(955-968)Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1109/TIP.2025.3533203
Carmona-Ballester DBonaque-González STrujillo-Sevilla JCeruso SRodríguez-Ramos J(2025)Enhancing display resolution through virtual pixel creation: a software-based approach for existing displaysVirtual Reality10.1007/s10055-024-01091-z29:1Online publication date: 8-Feb-2025
https://dl.acm.org/doi/10.1007/s10055-024-01091-z
Gao Chen 高Tan Xiaodi 谭Li Haifeng 李Liu Xu 刘(2024)基于U-Net的压缩光场显示图案生成方法Acta Optica Sinica10.3788/AOS23168344:10(1026027)Online publication date: 2024
https://doi.org/10.3788/AOS231683
Show More Cited By

Index Terms

Cascaded displays: spatiotemporal superresolution using offset pixel layers
1. Computing methodologies
  1. Computer graphics
    1. Image manipulation
    2. Rendering

Recommendations

Cascaded displays: spatiotemporal superresolution using offset pixel layers
SIGGRAPH '14: ACM SIGGRAPH 2014 Emerging Technologies

We demonstrate that layered spatial light modulators (SLMs), subject to fixed lateral displacements and refreshed at staggered intervals, can synthesize images with greater spatiotemporal resolution than that afforded by any single SLM used in their ...
Touchless gestural interaction with small displays: a case study
CHItaly '13: Proceedings of the Biannual Conference of the Italian Chapter of SIGCHI

Touchless gestural interaction enables users to interact with digital devices using body movements and gestures, and without the burden of a physical contact with technology (e.g., data gloves, body markers, or remote controllers). Most gesture-based ...
Peephole displays: pen interaction on spatially aware handheld computers
CHI '03: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

The small size of handheld computers provides theconvenience of mobility at the expense of reduced screen space for display and interaction. Prior research has identified the value of spatially aware displays, in which a position-tracked display ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 33, Issue 4

July 2014

1366 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2601097

Issue’s Table of Contents

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 July 2014

Published in TOG Volume 33, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

56
Total Citations
View Citations
953
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)1

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chen LWeng LCheng HCheng AHuang C(2025)Temporal Fusion: Continuous-Time Light Field Video FactorizationIEEE Transactions on Image Processing10.1109/TIP.2025.353320334(955-968)Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1109/TIP.2025.3533203
Carmona-Ballester DBonaque-González STrujillo-Sevilla JCeruso SRodríguez-Ramos J(2025)Enhancing display resolution through virtual pixel creation: a software-based approach for existing displaysVirtual Reality10.1007/s10055-024-01091-z29:1Online publication date: 8-Feb-2025
https://dl.acm.org/doi/10.1007/s10055-024-01091-z
Gao Chen 高Tan Xiaodi 谭Li Haifeng 李Liu Xu 刘(2024)基于U-Net的压缩光场显示图案生成方法Acta Optica Sinica10.3788/AOS23168344:10(1026027)Online publication date: 2024
https://doi.org/10.3788/AOS231683
Chen LWeng LCheng HCheng ALin KHuang C(2024)VLSI Design of Light-Field Factorization for Dual-Layer Factored DisplayIEEE Transactions on Very Large Scale Integration (VLSI) Systems10.1109/TVLSI.2024.341426232:11(2093-2106)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1109/TVLSI.2024.3414262
Ramirez JRautek PBohak CStrnad OZhang ZLi SViola IHeidrich W(2024)GPU Accelerated 3D Tomographic Reconstruction and Visualization From Noisy Electron Microscopy Tilt-SeriesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.323044530:7(3331-3345)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1109/TVCG.2022.3230445
Fu JXie QMeng DXu Z(2024)Rotation Equivariant Proximal Operator for Deep Unfolding Methods in Image RestorationIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.338353246:10(6577-6593)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1109/TPAMI.2024.3383532
Alpay KAkyüz ABrandonisio NMeehan JChalmers A(2024)DeepDuoHDR: A Low Complexity Two Exposure Algorithm for HDR Deghosting on Mobile DevicesIEEE Transactions on Image Processing10.1109/TIP.2024.349783833(6592-6606)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIP.2024.3497838
Pan SLi Y(2024)EBDNet: Integrating Optical Flow With Kernel Prediction for Burst DenoisingIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.338386234:9(8456-8468)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1109/TCSVT.2024.3383862
Ma TBoloor AYang XCao WWilliams PSun NChakrabarti AZhang XSolihin YHeinrich M(2023)LeCA: In-Sensor Learned Compressive Acquisition for Efficient Machine Vision on the EdgeProceedings of the 50th Annual International Symposium on Computer Architecture10.1145/3579371.3589089(1-14)Online publication date: 17-Jun-2023
https://dl.acm.org/doi/10.1145/3579371.3589089
Yamamoto KIwai DTani ISato K(2023)A Monocular Projector-Camera System Using Modular ArchitectureIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.321726629:12(5586-5592)Online publication date: Dec-2023
https://doi.org/10.1109/TVCG.2022.3217266
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents