Jonghyun Kim
ABSTRACT
We present Holographic Glasses, a holographic near-eye display system with an eyeglasses-like form factor for virtual reality. Holographic Glasses are composed of a pupil-replicating waveguide, a spatial light modulator, and a geometric phase lens to create holographic images in a lightweight and thin form factor. The proposed design can deliver full-color 3D holographic images using an optical stack of 2.5 mm thickness. A novel pupil-high-order gradient descent algorithm is presented for the correct phase calculation with the user’s varying pupil size. We implement benchtop and wearable prototypes for testing. Our binocular wearable prototype supports 3D focus cues and provides a diagonal field of view of 22.8° with a 2.3 mm static eye box and additional capabilities of dynamic eye box with beam steering, while weighing only 60 g excluding the driving board.
Supplemental Material
Available for Download
- Kurt Akeley, Simon J Watt, Ahna Reza Girshick, and Martin S Banks. 2004. A stereo display prototype with multiple focal distances. ACM transactions on graphics (TOG) 23, 3 (2004), 804–813.Google Scholar
- Kaan Akşit, Jan Kautz, and David Luebke. 2015. Slim near-eye display using pinhole aperture arrays. Applied optics 54, 11 (2015), 3422–3427.Google Scholar
- Kiseung Bang, Youngjin Jo, Minseok Chae, and Byoungho Lee. 2021. Lenslet VR: Thin, Flat and Wide-FOV Virtual Reality Display Using Fresnel Lens and Lenslet Array. IEEE TVCG 27, 5 (2021), 2545–2554.Google Scholar
- Ozan Cakmakci, Yi Qin, Peter Bosel, and Gordon Wetzstein. 2021. Holographic pancake optics for thin and lightweight optical see-through augmented reality. Optics Express 29, 22 (2021), 35206–35215.Google ScholarCross Ref
- Ozan Cakmakci and Jannick Rolland. 2006. Head-worn displays: a review. Journal of Display Technology 2, 3 (2006), 199–216.Google ScholarCross Ref
- Praneeth Chakravarthula, Yifan Peng, Joel Kollin, Henry Fuchs, and Felix Heide. 2019. Wirtinger holography for near-eye displays. ACM Trans. Graph 38, 6 (2019), 1–13.Google ScholarDigital Library
- Praneeth Chakravarthula, Ethan Tseng, Tarun Srivastava, Henry Fuchs, and Felix Heide. 2020. Learned hardware-in-the-loop phase retrieval for holographic near-eye displays. ACM Transactions on Graphics (TOG) 39, 6 (2020), 1–18.Google ScholarDigital Library
- Chenliang Chang, Kiseung Bang, Gordon Wetzstein, Byoungho Lee, and Liang Gao. 2020. Toward the next-generation VR/AR optics: a review of holographic near-eye displays from a human-centric perspective. Optica 7, 11 (2020), 1563–1578.Google ScholarCross Ref
- Jen-Hao Rick Chang, BVK Vijaya Kumar, and Aswin C Sankaranarayanan. 2018. Towards multifocal displays with dense focal stacks. ACM Transactions on Graphics (TOG) 37, 6 (2018), 1–13.Google ScholarDigital Library
- Suyeon Choi, Manu Gopakumar, Yifan Peng, Jonghyun Kim, Matthew O’Toole, and Gordon Wetzstein. 2022. Time-multiplexed Neural Holography: A Flexible Framework for Holographic Near-eye Displays with Fast Heavily-quantized Spatial Light Modulators. In Proceedings of the ACM SIGGRAPH.Google ScholarDigital Library
- Suyeon Choi, Manu Gopakumar, Yifan Peng, Jonghyun Kim, and Gordon Wetzstein. 2021a. Neural 3D holography: Learning accurate wave propagation models for 3D holographic virtual and augmented reality displays. ACM Transactions on Graphics (TOG) 40, 6 (2021), 1–12.Google ScholarDigital Library
- Suyeon Choi, Jonghyun Kim, Yifan Peng, and Gordon Wetzstein. 2021b. Optimizing image quality for holographic near-eye displays with michelson holography. Optica 8, 2 (2021), 143–146.Google ScholarCross Ref
- David Dunn, Cary Tippets, Kent Torell, Petr Kellnhofer, Kaan Akşit, Piotr Didyk, Karol Myszkowski, David Luebke, and Henry Fuchs. 2017. Wide Field Of View Varifocal Near-Eye Display Using See-Through Deformable Membrane Mirrors. IEEE Trans. on Visualization and Computer Graphics 23, 4 (2017), 1322–1331.Google ScholarDigital Library
- Daniel Glasner, Todd Zickler, and Anat Levin. 2014. A reflectance display. ACM Transactions on Graphics (TOG) 33, 4 (2014), 1–12.Google ScholarDigital Library
- Manu Gopakumar, Jonghyun Kim, Suyeon Choi, Yifan Peng, and Gordon Wetzstein. 2021. Unfiltered holography: optimizing high diffraction orders without optical filtering for compact holographic displays. Optics Letters 46, 23 (2021), 5822–5825.Google ScholarCross Ref
- Hong Hua. 2017. Enabling focus cues in head-mounted displays. Proc. IEEE 105, 5 (2017), 805–824.Google ScholarCross Ref
- Hong Hua and Bahram Javidi. 2014. A 3D integral imaging optical see-through head-mounted display. Opt. Express 22, 11 (2014), 13484–13491.Google ScholarCross Ref
- Fu-Chung Huang, Kevin Chen, and Gordon Wetzstein. 2015. The light field stereoscope: immersive computer graphics via factored near-eye light field displays with focus cues. ACM Transactions on Graphics (TOG) 34, 4 (2015), 1–12.Google ScholarDigital Library
- M. Ibbotson and S. Cloherty. 2009. Visual perception: saccadic omission, suppression or temporal masking?Current Biology 19, 12 (2009).Google Scholar
- Changwon Jang, Kiseung Bang, Gang Li, and Byoungho Lee. 2018. Holographic near-eye display with expanded eye-box. ACM Transactions on Graphics (TOG) 37, 6 (2018), 1–14.Google ScholarDigital Library
- Changwon Jang, Kiseung Bang, Seokil Moon, Jonghyun Kim, Seungjae Lee, and Byoungho Lee. 2017. Retinal 3D: augmented reality near-eye display via pupil-tracked light field projection on retina. ACM Transactions on Graphics (TOG) 36, 6 (2017), 1–13.Google ScholarDigital Library
- Jonghyun Kim, Youngmo Jeong, Michael Stengel, Kaan Akşit, Rachel Albert, Ben Boudaoud, Trey Greer, Joohwan Kim, Ward Lopes, Zander Majercik, Peter Shirley, Josef Spjut, Morgan McGuire, and David Luebke. 2019. Foveated AR: Dynamically-Foveated Augmented Reality Display. ACM Trans. Graph. 38, 4, Article 99 (2019).Google ScholarDigital Library
- Robert Konrad, Emily A Cooper, and Gordon Wetzstein. 2016. Novel optical configurations for virtual reality: Evaluating user preference and performance with focus-tunable and monovision near-eye displays. In Proceedings of the 2016 CHI conference on human factors in computing systems. 1211–1220.Google ScholarDigital Library
- George Alex Koulieris, Kaan Akşit, Michael Stengel, Rafał K Mantiuk, Katerina Mania, and Christian Richardt. 2019. Near-eye display and tracking technologies for virtual and augmented reality. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 493–519.Google Scholar
- Gregory Kramida. 2015. Resolving the vergence-accommodation conflict in head-mounted displays. IEEE transactions on visualization and computer graphics 22, 7(2015), 1912–1931.Google Scholar
- Bernard C Kress. 2020. Optical architectures for augmented-, virtual-, and mixed-reality headsets. (2020).Google Scholar
- Marc Lambooij, Marten Fortuin, Ingrid Heynderickx, and Wijnand IJsselsteijn. 2009. Visual discomfort and visual fatigue of stereoscopic displays: A review. Journal of imaging science and technology 53, 3 (2009), 30201–1.Google ScholarCross Ref
- Douglas Lanman and David Luebke. 2013. Near-eye light field displays. ACM Transactions on Graphics (TOG) 32, 6 (2013), 1–10.Google ScholarDigital Library
- Yun-Han Lee, Guanjun Tan, Tao Zhan, Yishi Weng, Guigeng Liu, Fangwang Gou, Fenglin Peng, Nelson V Tabiryan, Sebastian Gauza, and Shin-Tson Wu. 2017. Recent progress in Pancharatnam–Berry phase optical elements and the applications for virtual/augmented realities. Optical Data Processing and Storage 3, 1 (2017), 79–88.Google ScholarCross Ref
- Sheng Liu, Dewen Cheng, and Hong Hua. 2008. An optical see-through head mounted display with addressable focal planes. In 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality. IEEE, 33–42.Google Scholar
- Gordon D Love, David M Hoffman, Philip JW Hands, James Gao, Andrew K Kirby, and Martin S Banks. 2009. High-speed switchable lens enables the development of a volumetric stereoscopic display. Optics express 17, 18 (2009), 15716–15725.Google Scholar
- Andrew Maimone, Andreas Georgiou, and Joel S Kollin. 2017. Holographic near-eye displays for virtual and augmented reality. ACM Trans. Grap. 36, 4 (2017), 1–16.Google ScholarDigital Library
- Andrew Maimone, Douglas Lanman, Kishore Rathinavel, Kurtis Keller, David Luebke, and Henry Fuchs. 2014. Pinlight Displays: Wide Field of View Augmented Reality Eyeglasses Using Defocused Point Light Sources. ACM Trans. Graph. (SIGGRAPH) 33, 4, Article 89 (jul 2014), 11 pages.Google ScholarDigital Library
- Andrew Maimone and Junren Wang. 2020. Holographic optics for thin and lightweight virtual reality. ACM Transactions on Graphics (TOG) 39, 4 (2020), 67–1.Google ScholarDigital Library
- E. Matin. 1974. Saccadic suppression: a review and an analysis.Psychological bulletin 81, 12 (1974).Google Scholar
- Seokil Moon, Seung-Woo Nam, Youngmo Jeong, Chang-Kun Lee, Hong-Seok Lee, and Byoungho Lee. 2020. Compact augmented reality combiner using Pancharatnam-Berry phase lens. IEEE Photonics Technology Letters 32, 5 (2020), 235–238.Google ScholarCross Ref
- Seung-Woo Nam, Seokil Moon, Byounghyo Lee, Dongyeon Kim, Seungjae Lee, Chang-Kun Lee, and Byoungho Lee. 2020. Aberration-corrected full-color holographic augmented reality near-eye display using a Pancharatnam-Berry phase lens. Optics Express 28, 21 (2020), 30836–30850.Google ScholarCross Ref
- Bharathwaj Appan Narasimhan. 2018. Ultra-Compact pancake optics based on Thin Eyes super-resolution technology for virtual reality headsets. In Digital Optics for Immersive Displays, Vol. 10676. International Society for Optics and Photonics, 106761G.Google Scholar
- Nitish Padmanaban, Robert Konrad, Tal Stramer, Emily A. Cooper, and Gordon Wetzstein. 2017. Optimizing virtual reality for all users through gaze-contingent and adaptive focus displays. Proceedings of the National Academy of Sciences 114, 9 (2017), 2183–2188. https://doi.org/10.1073/pnas.1617251114 arXiv:https://www.pnas.org/content/114/9/2183.full.pdfGoogle ScholarCross Ref
- Nitish Padmanaban, Yifan Peng, and Gordon Wetzstein. 2019. Holographic near-eye displays based on overlap-add stereograms. ACM Trans. Graph. 38, 6 (2019), 1–13.Google ScholarDigital Library
- Vitor F Pamplona, Manuel M Oliveira, and Gladimir VG Baranoski. 2009. Photorealistic models for pupil light reflex and iridal pattern deformation. ACM Transactions on Graphics (TOG) 28, 4 (2009), 1–12.Google ScholarDigital Library
- Yifan Peng, Suyeon Choi, Jonghyun Kim, and Gordon Wetzstein. 2021. Speckle-free holography with partially coherent light sources and camera-in-the-loop calibration. Science advances 7, 46 (2021), eabg5040.Google Scholar
- Yifan Peng, Suyeon Choi, Nitish Padmanaban, and Gordon Wetzstein. 2020. Neural holography with camera-in-the-loop training. ACM Transactions on Graphics (TOG) 39, 6 (2020), 1–14.Google ScholarDigital Library
- PerkinsCoie. 2019. 2019 Augmented and Virtual Reality Survey Report. https://www.perkinscoie.com/images/content/2/1/v4/218679/2019-VR-AR-Survey-Digital-v1.pdfGoogle Scholar
- PerkinsCoie. 2021. XR Industry Insider 2021 XR Survey: Industry Insights into the Future of Immersive Technology. https://www.perkinscoie.com/content/designinteractive/xr2021/Google Scholar
- Joshua Ratcliff, Alexey Supikov, Santiago Alfaro, and Ronald Azuma. 2020. ThinVR: Heterogeneous microlens arrays for compact, 180 degree FOV VR near-eye displays. IEEE transactions on visualization and computer graphics 26, 5(2020), 1981–1990.Google Scholar
- Kishore Rathinavel, Hanpeng Wang, Alex Blate, and Henry Fuchs. 2018. An extended depth-at-field volumetric near-eye augmented reality display. IEEE transactions on visualization and computer graphics 24, 11(2018), 2857–2866.Google ScholarCross Ref
- Jannick P Rolland, Myron W Krueger, and Alexei Goon. 2000. Multifocal planes head-mounted displays. Applied Optics 39, 19 (2000), 3209–3215.Google ScholarCross Ref
- Liang Shi, Beichen Li, Changil Kim, Petr Kellnhofer, and Wojciech Matusik. 2021. Towards real-time photorealistic 3D holography with deep neural networks. Nature 591, 7849 (2021), 234–239.Google Scholar
- Takashi Shibata, Joohwan Kim, David M Hoffman, and Martin S Banks. 2011. The zone of comfort: Predicting visual discomfort with stereo displays. Journal of vision 11, 8 (2011), 11–11.Google ScholarCross Ref
- Andrew B Watson and John I Yellott. 2012. A unified formula for light-adapted pupil size. Journal of vision 12, 10 (2012), 12–12.Google ScholarCross Ref
- Jianghao Xiong, En-Lin Hsiang, Ziqian He, Tao Zhan, and Shin-Tson Wu. 2021. Augmented reality and virtual reality displays: emerging technologies and future perspectives. Light: Science & Applications 10, 1 (2021), 1–30.Google ScholarCross Ref
- Chanhyung Yoo, Jianghao Xiong, Seokil Moon, Dongheon Yoo, Chang-Kun Lee, Shin-Tson Wu, and Byoungho Lee. 2020. Foveated display system based on a doublet geometric phase lens. Optics Express 28, 16 (2020), 23690–23702.Google ScholarCross Ref
- Tao Zhan, Kun Yin, Jianghao Xiong, Ziqian He, and Shin-Tson Wu. 2020. Augmented reality and virtual reality displays: Perspectives and challenges. Iscience (2020), 101397.Google Scholar
Recommendations
Holographic near-eye display with expanded eye-box
Holographic displays have great potential to realize mixed reality by modulating the wavefront of light in a fundamental manner. As a computational display, holographic displays offer a large degree of freedom, such as focus cue generation and vision ...
Holographic optics for thin and lightweight virtual reality
We present a class of display designs combining holographic optics, directional backlighting, laser illumination, and polarization-based optical folding to achieve thin, lightweight, and high performance near-eye displays for virtual reality. Several ...
Holographic near-eye displays for virtual and augmented reality
We present novel designs for virtual and augmented reality near-eye displays based on phase-only holographic projection. Our approach is built on the principles of Fresnel holography and double phase amplitude encoding with additional hardware, phase ...
Comments