InfoScope: Link from Real World to Digital Information Space

Abstract

We describe an information augmentation system (infoScope) and applications integrating handheld device with a color camera to provide enhanced information perception services to users. InfoScope uses a color camera as an input device to capture scene images from the real world and utilize computer vision techniques to extract information from real world, convert them into digital world as text information and augment them back to the original scene location. The user can see both the real world and information together on display of the handheld device. We have implemented two applications: First one is an automatic sign/text translation for foreign travelers where a user may use infoScope whenever they want to see texts or signs in their own language where they are originally written in foreign language in the scene and extracted from scene images automatically by using computer vision techniques. The second application is “Information Augmentation in the City” where a user can see information associated with building, or a place, overlaid onto real scene images on their PDA's display.