Real-time simulation of accommodation and low-order aberrations of the human eye using light-gathering trees

Abstract

We present a real-time technique for simulating accommodation and low-order aberrations (e.g., myopia, hyperopia, and astigmatism) of the human eye. Our approach models the corresponding point spread function, producing realistic depth-dependent simulations. Real-time performance is achieved with the use of a novel light-gathering tree data structure, which allows us to approximate the contributions of over 300 samples per pixel under 6 ms per frame. For comparison, with the same time budget, an optimized ray tracer exploring specialized hardware acceleration traces two samples per pixel. We demonstrate the effectiveness of our approach through a series of qualitative and quantitative experiments on images with depth from real environments. Our results achieved SSIM values ranging from 0.94 to 0.99 and PSNR ranging from 32.4 to 43.0 in objective evaluations, indicating good agreement with the ground truth.

A optical power and image adjustments

Given the distance v from the camera lens to the external lens, known as vertex distance, the resulting anisotropic magnification due to astigmatism is given by

$$\begin{aligned} M_{\varphi } = \frac{1}{1 - v\,S}\quad \text {and}\quad M_{\varphi ^{\bot }} = \frac{1}{1 - v\,S_{pC}} , \end{aligned}$$

(5)

where $M_{\varphi }$ and $M_{\varphi ^{\bot }}$ are, respectively, the magnification factors along the directions that make angles $\varphi$ and $\varphi + 90^{\circ }$ with the horizontal axis. In the absence of astigmatism, the magnification is isotropic, with ${M_{\varphi } = M_{\varphi ^{\bot }}}$. The effective optical power is then obtained as $S'_{\varphi } = SM_{\varphi }$, which is the same as $S'_{\varphi ^{\bot }} = SM_{\varphi ^{\bot }}$.

Image magnification may introduce incorrect values due to interpolation. Thus, when comparing our results to ground-truth, rather than magnifying a smaller dimension to match a larger one, we downscale the larger to match the smaller. One should note, however, that magnification is a function of vertex distance and vanishes when $v=0$. Thus, magnification and its compensation have only been used for the sake of the validation experiment that uses an external lens. This is not required by the LGT technique itself.

Brightness adjustment is required to compensate for some amount of light that is reflected/absorbed by the extra lens, effectively not reaching the sensor. The images captured with the extra lens tend to be darker than ones captured without it. To perform brightness adjustment, for each different external lens, a small white patch is taken from the same area in images captured with and without the additional lens. The ratio between the average intensities from the darker and brighter patches was used to modulate the brightness of the images simulated with our technique, making them exhibit brightness similar to the ground-truth images. This is important when performing quantitative comparisons using metrics such as SSIM and PSNR (Table 2).

Chromatic aberration due to the external lens is given by

$$\begin{aligned} S''_{\varphi c} = \frac{S'_{\varphi }(\mu _c - 1)}{\mu _y - 1} \quad \text {and}\quad S''_{\varphi ^{\bot } c} = \frac{S'_{\varphi ^{\bot }}(\mu _c - 1)}{\mu _y - 1}, \end{aligned}$$

where $S''_{\varphi {c}}$ and $S''_{\varphi ^{\bot } c}$ are the resulting aberrated powers (in diopters) for wavelength $\lambda _c$. $S'_{\varphi }$ and $S'_{\varphi ^{\bot }}$ are the effective optical powers due to the vertex distance v, $\mu _c$ is the lens refractive index for wavelength $\lambda _c$, and $\mu _y=1.5085$ is the reference refractive index, which is usually on the yellow region of the spectrum. For our experiments, we used the following indices of refraction for red, green, and blue, respectively: $\mu _r = 1.4998$, $\mu _g = 1.5085$, and $\mu _b = 1.5152$, which were obtained from an online refractive index database [22], and correspond to the wavelengths $\lambda _r=700$ nm, $\lambda _g=510$ nm, and $\lambda _b=440$ nm [14].