A non-contact device for tracking gaze in a human computer interface

doi:10.1016/j.cviu.2004.07.005

Computer Vision and Image Understanding

Volume 98, Issue 1, April 2005, Pages 52-82

https://doi.org/10.1016/j.cviu.2004.07.005 Get rights and content

Abstract

This paper presents a novel design for a non-contact eye detection and gaze tracking device. It uses two cameras to maintain real-time tracking of a person’s eye in the presence of head motion. Image analysis techniques are used to obtain accurate locations of the pupil and corneal reflections. All the computations are performed in software and the device only requires simple, compact optics and electronics attached to the user’s computer. Three methods of estimating the user’s point of gaze on a computer monitor are evaluated. The camera motion system is capable of tracking the user’s eye in real-time (9 fps) in the presence of natural head movements as fast as 100°/s horizontally and 77°/s vertically. Experiments using synthetic images have shown its ability to track the location of the eye in an image to within 0.758 pixels horizontally and 0.492 pixels vertically. The system has also been tested with users with different eye colors and shapes, different ambient lighting conditions and the use of eyeglasses. A gaze accuracy of 2.9° was observed.

Introduction

Tracking and recording a person’s eye movements has been shown to be useful in diverse applications [1], [2], [3], [4], [5]. However, speed, accuracy, cost, and ease of use have so far limited its widespread use in a human computer interface beyond narrow research markets. The research reported in this paper attempts to overcome these limitations.

Background physiology and neurology of eye movements used to define design specifications are covered in [1], [6], [7]. Useful characteristics of the eye include corneal reflections (glint) [6] and the maximum angular velocity of head movement (100°/s). While physiologically, fixations during reading are about 200 ms long, a fixation as defined here extends that period to 350 ms to include tremors, microsaccades, and slow drifts. The 3D location of a point on the object that falls within the foveal center during this period is considered to be the point of gaze (POG). The angle of gaze (AOG) has been defined (see Fig. 1) as the angle between the line of sight and either the user’s frontal plane (θ₁) or, as used here, the normal to the viewing plane (θ₂).

Section snippets

Previous work

Video-oculography (VOG) [6], [8], [9] is a minimally intrusive or completely non-contact technique commonly used for tracking and recording eye movements. In this paper, VOG is used to track the eyes and determine a user’s gaze. In a complete VOG system that determines where an unconstrained user is looking, there are three basic tasks [10]. The first task is to compensate for natural head movements to ensure that the user’s eye is always in the field of view of the camera(s) tracking the eye.

Tracking approach

To permit natural head movements, a two camera system is used, as in Fig. 2 (see also Fig. 6). A fixed wide angle (WA) camera facing the user (i.e., in the positive z direction) is used to locate the eyes. A second narrow angle (NA) camera is mounted on a frame that rotates about the x-axis (tilt) and holds a mirror that rotates about the y-axis (pan) and reflects light rays into the NA camera. Rotating the lightweight panning mirror requires less torque from the stepper motors than tilting the

Detection and parameterization

This section reports the accuracy, speed, and reliability of the algorithms used for detecting and parameterizing the pupil in both cameras and the glint in the NA camera. Both synthetic and real data were used. The tests used to obtain the measurements were performed on a a 1.2 GHz AMD Athlon processor, using an image size of 640 × 480 pixels for each camera.

Discussion

The results of experiments with the GTD system indicate that it is capable of tracking the pupil and glint of the eye of a user in real-time (9 fps) in the presence of natural head movements. Specifically, the tracking algorithm positions the two motors correctly so that the left eye of the user falls within the field of view of the NA camera. It is also able to track the pupil with sub-pixel accuracy, and is tolerant to changes in eye color and shape, and lighting conditions (indicated by a

Conclusions

A novel design for a VOG-based eye tracking device for use in a human computer interface has been presented and the performance of a particular implementation of that design reported. The main contribution of this paper is the design of a complete video-based eye tracker. The device is able to accurately (with less than one pixel error in synthetic images) track the eyes in the presence of normal head motion. The system also tracks gaze but with more limited accuracy. An indirect approach to

References (36)

J.R. Leigh, D.S. Zee, The Neurology of Eye Movements, Davis, Philadelphia,...
C. Morimoto, D. Koons, A. Amir, M. Flickner, S. Zhai, Keeping an eye for HCI, in: Proc 12th Brazilian Symp. on Computer...
R. Yang, Z. Zhang, Eye gaze correction with stereovision for video tele-conferencing, in: Proc. 7th Eur. Conf. on...
K. Talmi, J. Liu, Eye and gaze tracking for visually controlled interactive stereoscopic displays, Signal Processing:...
L.E. Sibert, J.N. Templeman, R.J.K. Jacob, Evaluation and Analysis of Eye Gaze Interaction, NRL Report...
R.H.S. Carpenter
Movements of the Eyes
(1988)
D.A. Goss et al.
Introduction to the Optics of the Eye
(2002)
A.H. Clarke et al.
Video-oculography—an alternative method for measurement of three-dimensional eye movements
L.R. Young et al.
Survey of eye movement recording methods
Behav. Res. Methods Instrument.
(1975)
B. Noureddin, A Non-Contact Video-oculograph For Tracking Gaze in a Human Computer Interface, Thesis (M.A.Sc.),...

A.L. Yuille et al.

Feature extraction from faces using deformable templates

Int. J. Comput. Vis.

(1992)

R. Wagner et al.

Evaluation of three template matching algorithms for registering images of the eye

IEEE Trans. Biomed. Eng.

(1992)

M. Betke, J. Kawai, Gaze detection via self-organizing grey-scale units, in: Proc. Int. Workshop on Recognition,...

R. Stiefelhagen, J. Yang, A. Waibel, Tracking eyes and monitoring eye gaze, in: Proc. Workshop on Perceptual User...

Q. Ji et al.

Non-intrusive eye and gaze tracking for natural human computer interaction

MMI-interaktiv J.

(2003)

J. Zhu, J. Yang, Subpixel eye gaze tracking, in: Proc. 5th Int. Conf. on Automatic Face and Gesture Recognition,...

S.-W. Shih, Y.-T. Wu, J. Liu, A calibration-free gaze tracking technique, in: Proc. 15th Int. Conf. on Pattern...

D.H. Yoo, J.H. Kim, B.R. Lee, M.J. Chung, Non-contact eye gaze tracking system by mapping of corneal reflections, in:...

Cited by (83)

Calibration free, user-independent gaze estimation with tensor analysis
2018, Image and Vision Computing
Citation Excerpt :
Since it is invisible to human eyes, infrared camera can robustly capture the glint (reflections of light off the cornea) against various illumination conditions [6]. In fact, several studies have used the glint for eyes detection [10-12]. The glint provides a reference point of the human eye to the IR light sources.
Human gaze directly signals visual attention, therefore, estimation of gaze has been an important research topic in fields such as human attention modeling and human-computer interaction. Accurate gaze estimation requires user, system and even session dependent parameters, which can be obtained by calibration process. However, this process has to be repeated whenever the parameter changes (head movement, camera movement, monitor movement). This study aims to eliminate the calibration process of gaze estimation by building a user-independent, appearance-based gaze estimation model. The system is ideal for multimodal interfaces, where the gaze is tracked without the cooperation from the users. The main goal is to capture the essential representation of the gaze appearance of the target user. We investigate the tensor analysis framework that decomposes the high dimension gaze data into different factors including individual differences, gaze differences, user-screen distances and session differences. The axis that is representative for a particular subject is automatically chosen in the tensor analysis framework using LASSO regression. The proposed approaches show promising results on capturing the test subject gaze changes. To address the estimation shift caused by the variations in individual heights, or relative position to the monitor, we apply domain adaptation to adjust the gaze estimation, observing further improvements. These promising results suggest that the proposed gaze estimation approach is a feasible and flexible scheme to facilitate gaze-based multimodal interfaces.
Gazing point dependent eye gaze estimation
2017, Pattern Recognition
Citation Excerpt :
This kind of approaches extract local features such as contours, eye corners and reflections from eye images. These approaches can be classified into two types: the model-based or geometric approach [23–33] and the interpolation-based approach [20,34–40]. The interpolation-based approaches use particular parametric to map the image features to gazing coordinates.
Cross-ratio invariant is used widely in projective transformations for eye gaze estimation. Establishing a virtual plane projection is an important step to use this property. Most of traditional cross-ratio approaches only used fixed parameters to calculate the gazing point. This paper proposes gazing point dependent eye gazing estimation approach. Our contributions are three-folded. First, we model a dynamic virtual plane projection, which is tangent to the cornea of pupil, to estimate the position of the gazing point. Second, we introduce a two stage approach consisting of rough-to-precise framework for gazing point estimation based on the gazing point dependent virtual plane projection. Third, a heuristic strategy which contains off-line and on-line parameter learning for gazing point estimation is proposed. The experiment results show that our approach can significantly improve the gazing estimation performance with an average accuracy of 0.70°.
Object-aware Gaze Target Detection
2023, Proceedings of the IEEE International Conference on Computer Vision
Research on Virtual Reality Head-Mounted Displays based on MEMS Technology
2023, Journal of Physics: Conference Series
Replicueauth: Validating the use of a lab-based virtual reality setup for evaluating authentication systems
2021, Conference on Human Factors in Computing Systems - Proceedings
Effectiveness of an eye-gaze controlled typing system for the japanese syllabary
2021, International Journal of Innovative Computing, Information and Control

View all citing articles on Scopus

^☆: This work was supported by the Precarn IRIS NCE and the Natural Sciences and Engineering Research Council of Canada.

View full text

A non-contact device for tracking gaze in a human computer interface☆