Measuring Refractive Properties of Human Vision by Showing 4D Light Fields

Abstract

In this paper, we propose a novel method for measuring refractive properties of human vision. Our method generates a special 4D light field and present it to human observers, so that the observers will see different images according to their dioptric properties, i.e. the amount of near sightedness and far sightedness. Thus, we can measure the visual properties of observers just by showing the light field images. The proposed method does not require expensive measurement systems nor well trained medical doctors unlike the existing eyesight measurement methods. The real image experiments and the synthetic image experiments show the efficiency of the proposed method.

1 Introduction

In recent years, many people are suffering from eyesight problems. Thus, it is important to measure the refractive properties of human vision easily and accurately. Traditionally, the refractive properties of human vision have been measured by using autorefractors [3, 12]. Although they can measure precise dioptric properties of human vision, they are very expensive and must be operated by well trained medical doctors. Thus, we in this paper propose a novel method for measuring dioptric properties of human vision easily and quickly.

Our estimation method is very different from the standard computer vision techniques. In computer vision, we usually obtain camera images of objects and estimate their properties, such as shape, reflectance and refractivity, from images. On the contrary, our method measures the refractive properties of eyesight by showing images to observers. In this method we generate a special 4D light field and present it to human observers. Then, the human observers will see specific images according to the refractive properties of their eyesight. As a result, we can measure the refractive properties of their eyesight from the observed images. For example, a person who has nearsightedness with the dioptric power of $-0.50 D$ will observe a flower, while a person whose dioptric power is $+0.10 D$ will observe a dog.

Since the proposed method does not need to capture images nor compute them, it does not require computation time and is very efficient.

We first present related work in Sect. 2, and show the advance of the proposed method. We next explain the refractive properties of human vision in Sect. 3. Then, we propose a novel method for measuring the refractive properties of human vision by presenting 4D light field in Sect. 4. The experimental results in Sect. 5 show the efficiency of the proposed method.

2 Related Work

Since the pioneering work of a light field display by Levoy and Hanrahan [11], the light field displays and their applications have been studied extensively [6,7,8,9,10, 18, 19]. Recently, Huang et al. [7] proposed a novel displaying method, which enables weak sighted people to see visual information correctly without wearing eyeglasses. Their method generates 4D light fields, so that the distorted visual system of a weak sighted person observes in-focus images. The visual acuity can also be improved by displaying images convolved with the inverse PSF of human vision [1, 2]. These vision correction methods require the measurement of the refractive properties of human vision before using them. However, the precise measurement of the refractive properties of human vision requires very expensive measurement systems, such as autorefractors [3, 12], and only well trained medical doctors can operate them. Thus, we need more simple and low cost methods for measuring the precise refractive properties of human vision.

For measuring the refractive properties of transparent objects, the ray tracing has often been used in the existing methods [5, 14,15,16,17]. These methods use the relationship between the input light rays and the output light rays of transparent objects for analyzing their refractive properties. However, it is very difficult to obtain light rays which passed the human visual systems, and thus we cannot use these existing methods for measuring the refractive properties of human vision. For extracting the dioptric properties of human vision, Pamplona et al. [13] used a light field display. They showed that it is possible to measure refractive properties of human vision by controlling light rays of the light field display so that two light rays meet at the retina of observers. Although their method is efficient and can measure the refractive properties interactively, it requires the observers to control the light rays iteratively, so that their spots on the retina coincide. Thus, people who do not have skills to control the light field display, such as infants, cannot use their method.

Therefore, we in this paper propose a novel method for measuring the refractive properties of human vision just by showing a 4D light field to human observers. Our method generate a special 4D light field, so that the human observers see different images according to the refractive properties of their vision. Thus, we can measure their refractive properties just by checking what are observed by them. Hence, our method does not need any skills to control the light field display.

Fig. 1.

Eyesight

3 Refractive Properties of Human Vision

If we have a normal vision, the light rays which come into the eye intersect on the retina as shown in Fig. 1(a), and sharp images are obtained on the retina. If the eye lens is not controlled properly and is thicker than usual, the input light rays intersect in front of the retina and are spread on the retina as shown in Fig. 1(b). As a result, the observed images are blurred. This is called nearsightedness. Similarly, if the eye lens is thinner than usual, the input light rays also spread on the retina, and observed images are blurred as shown in Fig. 1(c). This is called farsightedness. In general, the refractive properties of human vision are described by using a dioptric power D, which is the reciprocal of the maximum or minimum focal length f controllable by the eye lens of the human observer as follows:

$$\begin{aligned} D = \frac{1}{f} \end{aligned}$$

(1)

For example, the dioptric power of a near sight observer with the maximum visible distance of 2 m is $\frac{1}{-2}= -0.5 D$, and that of a far sight observer with the minimum visible distance of 0.5 m is $\frac{1}{0.5}= +2.0 D$.

The visual acuity can be measured by several methods. The most popular method is the Landolt ring method [4], in which a ring with a gap is shown to the human observer to check the visibility of the gap. The Landolt ring method is very simple and easy to check the grade of eyesight. However, since it is too simple, we cannot obtain the detail characteristics of the eyesight. For example, we cannot distinguish nearsightedness and farsightedness just from the Landolt ring method. This is because the Landolt ring method only measures the amount of blur on the human retina.

For measuring the refractive properties of human vision more precisely, autorefractors [3, 12] are often used by medical doctors and eyeglass designers. Although the autorefractor can measure precise refractive properties of eyesight, it is very expensive and must be operated by well trained medical doctors or eyeglass designers, and cannot be used by ordinary people. Thus, for using flexible eyesight correction systems such as [7], we need more simple and low cost methods for measuring precise refractive properties of human vision. In the following sections, we show that by controlling the 4D light field, we can measure the detail characteristics of human eyesight very easily.

4 Estimating Refractive Properties from 4D Light Field

4.1 Observation of Light Field in Human Vision

For measuring the refractive properties of human vision by showing images, we use a light field display. The light field display consists of a 2D display and a micro lens array, and can generate arbitrary intensity of light toward arbitrary orientations as shown in Fig. 2.

Fig. 2.

Light field display

Fig. 3.

Observation of light field (Color figure online)

Suppose a farsighted person observes a standard 2D display which is close to the person. Then, because of the farsightedness, light rays emitted from different points on the display converge to an identical point on the retina as shown in Fig. 3(a). Since the different points on the display emit different colors in general, the point on the retina observes the sum of these colors as shown in Fig. 3(a). As a result, the farsighted person observes blurred images.

On the contrary, if we use a 4D light field display, we can emit different light in each orientation at each position on the display. Thus, we can emit light fields, so that the light rays, which converge to an identical point on the retina, have the same color as shown in Fig. 3(b). As a result, the farsighted person can observe clear sharp images, even if the display is closer than the minimal local length of the person.

We next consider the observation model of light emitted from a 4D light field display. Let us consider a light L(s, t, u, v) emitted from a point (s, t) on the display into the orientation (u, v) by using the light field display. Then, since the observed intensity I(x, y) is the sum of light which comes into a point (x, y) on the retina, it can be described as follows:

$$\begin{aligned} I(x,y) = \sum _{s=1}^S\sum _{t=1}^T\sum _{u=1}^U\sum _{v=1}^V\delta (x,y,s,t,u,v) L(s,t,u,v) \end{aligned}$$

(2)

where, S and T are the number of horizontal and vertical points on the light field display, and U and V are the number of horizontal and vertical orientations of the light field display. $\delta$ denotes a 6th order tensor, whose component $\delta (x,y,s,t,u,v)$ is 1 if a light ray L(s, t, u, v) passes through a point (x, y) on the retina, and is 0 if it does not pass through the point (x, y) as follows:

$$\begin{aligned} \delta (x,y,s,t,u,v)= & {} \left\{ \begin{array}{rc} 1 &{} if \ L(s,t,u,v) \rightarrow I(x,y) \\ 0 &{} otherwise. \end{array}\right. \end{aligned}$$

(3)

Let us consider a vector $\mathbf{L}$ which consists of light rays L(s, t, u, v), and a vector $\mathbf{I}$ which consists of the observed intensities I(x, y) as follows:

$$\begin{aligned} \mathbf{L}= & {} \left[ \begin{array}{c} L(1,1,1,1) \\ \vdots \\ L(S,T,U,V) \end{array}\right] \end{aligned}$$

(4)

$$\begin{aligned} \mathbf{I}= & {} \left[ \begin{array}{c} I(1,1) \\ \vdots \\ I(X, Y) \end{array}\right] \end{aligned}$$

(5)

where, X and Y are the number of horizontal and vertical points on the retina. Then, the observation model shown in Eq. (2) can be rewritten as follows:

$$\begin{aligned} \mathbf{I} = \mathbf{P}{} \mathbf{L} \end{aligned}$$

(6)

where, $\mathbf{P}$ is a light field projection matrix, and is described by using $\delta (x,y,s,t,u,v)$ as follows:

$$\begin{aligned} \mathbf{P} = \left[ \begin{array}{ccc} \delta (1,1,1,1,1,1) &{} \cdots &{} \delta (1,1,S,T,U,V) \\ \vdots &{} \ddots &{} \vdots \\ \delta (X,Y,1,1,1,1) &{} \cdots &{} \delta (X,Y,S,T,U,V) \end{array}\right] \end{aligned}$$

(7)

In the following sections, we use Eq. (7) for measuring the refractive properties of human vision.

4.2 Measuring Eyesight Characteristics from 4D Light Field

We next describe a method for measuring the refractive properties of human vision by using the light field display. In our method, we present a specially designed light field from a light field display, so that human observers see different images according to their refractive properties.

Suppose we have N different eyesight characteristics. Then, these N eyesight characteristics are represented by N different light field projection matrices $\mathbf{P}_i$ $(i=1,\cdots ,N)$. Now, let us consider N observers $O_i$ $(i=1,\cdots ,N)$, whose eyesight characteristics are described by $\mathbf{P}_i$ $(i=1,\cdots ,N)$. We show N different images $\mathbf{I}_i$ $(i=1,\cdots ,N)$ to these N observers $O_i$ $(i=1,\cdots ,N)$. Then, the observation of these N observers can be described as follows:

$$\begin{aligned} \left[ \begin{array}{c} \mathbf{I}_1 \\ \vdots \\ \mathbf{I}_N\end{array}\right] = \left[ \begin{array}{c} \mathbf{P}_1 \\ \vdots \\ \mathbf{P}_N\end{array}\right] \mathbf{L} \end{aligned}$$

(8)

Thus, the light field $\mathbf{L}$, which provides $N$ different images $\mathbf{I}_i$ $(i=1\cdots ,N)$ toward $N$ observers $O_i$ $(i=1\cdots ,N)$, can be derived as follows:

$$\begin{aligned} \mathbf{L} = \left[ \begin{array}{c} \mathbf{P}_1 \\ \vdots \\ \mathbf{P}_N\end{array}\right] ^{-} \left[ \begin{array}{c} \mathbf{I}_1 \\ \vdots \\ \mathbf{I}_N\end{array}\right] \end{aligned}$$

(9)

where, $(^-)$ denotes the pseudo inverse of a matrix. This means $N$ different visual acuities of human observer can be identified just by showing the light filed $\mathbf{L}$ to the observer and checking the image observed by the observer.

However, unfortunately the light field $\mathbf{L}$ derived from Eq. (9) cannot be displayed by the real light field display, since the light field $\mathbf{L}$ derived from Eq. (9) includes negative intensity values in general. To cope with this problem, we derive a light field $\mathbf{L}$ by solving the following conditional minimization problem:

$$\begin{aligned} \min _\mathbf{L} \sum _{i=1}^{N} ||\mathbf{I}_i-\mathbf{P}_i\mathbf{L}||^2 \quad \mathrm{subject\,\, to } \quad 0 \le L(s,t,u,v) \le I_{max} \end{aligned}$$

(10)

where, $I_{max}$ denotes a maximum intensity value of the light field display. Then, the refractive properties of a human observer can be measured by showing the light filed $\mathbf{L}$ derived from Eq. (10) to the observer.

5 Experiments

5.1 Real Image Experiments

We next show the results from real image experiments. In our experiments, we put a micro lens array on the display of a Nexus 7 tablet, and used them as a light field display. The resolution of the display of the tablet was $1920 \times 1200$ pixels, but we used only $80\times 80$ pixels for a light field display. The resolution of position of the light field display was $20\times 20$, and the resolution of orientation of the light field display was $4 \times 4$. We used a CCD camera with the resolution of $1292\times 964$ as an observer, and it observed the light field display as shown in Fig. 4. The distance from the display to the camera was 10 cm.

Fig. 4.

Experimental setup.

Fig. 5.

Target images for a normal sight observer and a near sight observer.

Fig. 6.

4D light field derived from Fig. 5.

Fig. 7.

Images observed by a normal sight observer and a near sight observer.

For simulating two different eyesight characteristics, we put eyeglasses of nearsightedness in front of the camera, and took images by the camera with and without eyeglasses. The focus of the cameras was set up, so that it focuses on the display when we put eyeglasses in front of the camera. Thus, the camera with eyeglasses is a normal sight observer, and the camera without eyeglasses is a near sight observer.

We next derived a 4D light field from Eq. (10), so that the normal sight observer observes the image shown in Fig. 5(a) and the near sight observer observes the image shown in Fig. 5(b). The derived 4D light field is shown in Fig. 6, in which the 4D light field is represented as a 2D image.

The light field shown in Fig. 6 was presented by using the light field display, and was observed by the normal sight observer and the near sight observer. The images observed by these two observers are shown in Fig. 7(a) and (b) respectively. As shown in Fig. 7(a) and (b), the images observed by the normal sight observer and the near sight observer are different from each other, and are close to the target images shown in Fig. 5(a) and (b). These results show that observers with different eyesight characteristics see different images, and their refractive properties can be measured just from observed images by using the proposed method.

5.2 Accuracy Evaluation

We next evaluated the accuracy of identification of refractive properties in the proposed method by using synthetic light fields. In this experiment, the distance from the lens center to the retina was 24 mm, and the distance from the display to the lens center was 250 mm. Thus, the focal length of the normal sight observer was $f=21.8978$ mm in this case. Then, the focal length of the near sight observer is $f-\alpha$, and the focal length of the far sight observer is $f+\alpha$ respectively, where $\alpha$ is the magnitude of nearsightedness or farsightedness. The light field display had the resolution of $30\times 30$ in position and $5\times 5$ in orientation, and thus its total resolution was $30\times 30\times 5\times 5$. The resolution of the image on the retina was $30\times 30$ pixels.

The accuracy of identification of refractive properties was evaluated by using the similarity between a target image I and an observed image $\hat{I}$. The similarity was measured by the zero-mean normalized cross correlation (ZNCC) as follows:

$$\begin{aligned} ZNCC=\frac{ \sum ^{X}_{x=1} \sum ^{Y}_{y=1} (I(x,y)-\bar{I}(x,y))(\hat{I}(x,y)-\bar{\hat{I}}(x,y))}{\sqrt{ \sum ^{X}_{x=1} \sum ^{Y}_{y=1} (I(x,y)-\bar{I}(x,y))^2}\sqrt{\sum ^{X}_{x=1} \sum ^{Y}_{y=1}(\hat{I}(x,y)-\bar{\hat{I}}(x,y))^2}} \nonumber \end{aligned}$$

where, $\bar{I}$ and $\bar{\hat{I}}$ are typical values of I and $\hat{I}$ respectively. By using ZNCC, we evaluated the accuracy of identification in the case of three eyesight characteristics and five eyesight characteristics respectively.

Fig. 8.

The target images for the near sight, normal sight and the far sight observers.

Fig. 9.

The observed images of the near sight, normal sight and the far sight observers.

Fig. 10.

The similarity between observed images and target images. (Color figure online)

Fig. 11.

The target images of $f-0.10$, $f-0.05$, f, $f+0.05$ and $f+0.10$.

We first evaluated the accuracy of identification in the case of three eyesight characteristics. The focal length of these three eyesight characteristics was $f-0.10$, f and $f+0.10$, which correspond to near sight, normal sight and far sight observes respectively. The target images for these three eyesight characteristics are shown in Fig. 8(a)–(c). The 4D light field for identifying these three eyesight characteristics was derived from the proposed method and presented to the observer. The focal length of the observer was changed from $f-0.15$ to $f+0.15$, and the light field was observed at each focal length. Figure 9 shows observed images when the focal length of the observer was $f-0.10$, $f-0.05$, f, $f+0.05$ and $f+0.10$. As shown in these images, the observed images at $f-0.10$, f and $f+0.10$ are almost identical with target images at $f-0.10$, f and $f+0.10$ respectively. The similarity ZNCC between observed images and target images was computed by changing the focal length of the observer from $f-0.15$ to $f+0.15$. The result is shown in Fig. 10. The horizontal axis is the focal length of the observer, and the vertical axis is the similarity between the observed image and the target image. The blue solid line in Fig. 10 shows the similarity between the target image at $f-0.10$ and observed images at various focal length. The green and red solid lines show those of the target image at f and $f+0.10$ respectively. As shown in Fig. 10, the similarity takes the largest value when the focal length of the observer was identical with that of the target image. For the reference of readers, we also show the ground truth of the similarity by the dash-dot lines in Fig. 10. From these results, we find that we can identify the refractive properties of observers properly by using the proposed method.

Fig. 12.

The images observed by an observer with the focal length of $f-0.10$, $f-0.05$, f, $f+0.05$ and $f+0.10$.

Fig. 13.

The similarity between observed images and target images. (Color figure online)

We next evaluated the accuracy of identification in the case of five eyesight characteristics. The focal length of these five eyesight characteristics was $f-0.10$, $f-0.05$, f, $f+0.05$ and $f+0.10$. The target images for these five eyesight characteristics are shown in Fig. 11. The 4D light field for identifying these five eyesight characteristics was computed from the proposed method and presented to the observer. Figure 12 shows observed images when the focal length of the observer was $f-0.10$, $f-0.05$, f, $f+0.05$ and $f+0.10$. Again, the observed images are almost identical with target images, when the focal length of the observer is identical with that of target images. The similarity ZNCC between observed images and target images was computed by changing the focal length of the observer from $f-0.15$ to $f+0.15$. The result is shown in Fig. 13. The horizontal axis is the focal length of the observer, and the vertical axis is the similarity between the observed image and the target image. The blue solid line in Fig. 13 shows the similarity between the target image at $f-0.10$ and observed images at various focal length. The green, red, light blue and purple solid lines show those of the target image at $f-0.05$, f, $f+0.05$ and $f+0.10$ respectively. As shown in Fig. 13, the similarity takes the largest value when the focal length of the observer was identical with that of the target image. From these results, we find that the refractive properties of observers can be identified properly by using the proposed method. The proposed method requires just a tablet or smart phone and a lens array sheet, and thus it is quite simple and efficient. The result of the measurement can be used for controlling the display, so that near sight and far sight observers can see clear images on the display.

6 Conclusion

In this paper, we proposed a novel method for measuring the refractive properties of human vision just by showing 4D light fields to observers. For this objective, we generated a 4D light field, so that the human observers see different images according to their eyesight characteristics. By using our method, we can identify the refractive properties of observers just by showing the light field to the observers.

The proposed method does not require expensive eyesight measurement systems nor well trained medical doctors unlike the existing eyesight measurement methods. The efficiency of the proposed method was shown by the real image experiments as well as the synthetic image experiments.