Using human observer eye movements in automatic image classifiers

Alejandro Jaimes; Jeff B. Pelz; Tim Grabowski; Jason S. Babcock; Shih-Fu Chang

doi:10.1117/12.429507

8 June 2001 Using human observer eye movements in automatic image classifiers

Alejandro Jaimes, Jeff B. Pelz, Tim Grabowski, Jason S. Babcock, Shih-Fu Chang

Proceedings Volume 4299, Human Vision and Electronic Imaging VI; (2001) https://doi.org/10.1117/12.429507
Event: Photonics West 2001 - Electronic Imaging, 2001, San Jose, CA, United States

Abstract

We explore the way in which people look at images of different semantic categories and directly relate those results to computational approaches for automatic image classification. Our hypothesis is that the eye movements of human observers differ for images of different semantic categories, and that this information can be effectively used in automatic content-based classifiers. First, we present eye tracking experiments that show the variation in eye movements across different individuals for image of 5 different categories: handshakes, crowd, landscapes, main object in uncluttered background, and miscellaneous. The eye tracking results suggest that similar viewing patterns occur when different subjects view different images in the same semantic category. Using these results, we examine how empirical data obtained from eye tracking experiments across different semantic categories can be integrated with existing computational frameworks, or used to construct new ones. In particular, we examine the Visual Apprentice, a system in which images classifiers are learned form user input as the user defines a multiple level object definition hierarchy based on an object and its parts and labels examples for specific classes. The resulting classifiers are applied to automatically classify new images. Although many eye tracking experiments have been performed, to our knowledge, this is the first study that specifically compares eye movements across categories, and that links category-specific eye tracking results to automatic image classification techniques.

Citation Download Citation

Alejandro Jaimes, Jeff B. Pelz, Tim Grabowski, Jason S. Babcock, and Shih-Fu Chang "Using human observer eye movements in automatic image classifiers", Proc. SPIE 4299, Human Vision and Electronic Imaging VI, (8 June 2001); https://doi.org/10.1117/12.429507

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available