On Making Projector Both a Display Device and a 3D Sensor

Abstract

We describe a system of embedding codes into projection display for structured light based sensing, with the purpose of letting projector serve as both a display device and a 3D sensor. The challenge is to make the codes imperceptible to human eyes so as not to disrupt the content of the original projection. There is the temporal resolution limit of human vision that one can exploit, by having a higher than necessary frame rate in the projection and stealing some of the frames for code projection. Yet there is still the conflict between imperceptibility of the embedded codes and the robustness of code retrieval that has to be addressed. We introduce noise-tolerant schemes to both the coding and decoding stages. At the coding end, specifically designed primitive shapes and large Hamming distance are employed to enhance tolerance toward noise. At the decoding end, pre-trained primitive shape detectors are used to detect and identify the embedded codes – a task difficult to achieve by segmentation that is used in regular structured light methods, for the weakly embedded information is generally interfered by substantial noise. Extensive experiments including evaluations of both code imperceptibility and decoding accuracy show that the proposed system is effective, even with the prerequisite of incurring minimum disturbance to the original projection.