Skip to main content
SpringerLink
Log in

Efficient Range Sensing Using Imperceptible Structured Light

  • Conference paper
  • First Online:
Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2020)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1474))

  • 823 Accesses

Abstract

A novel projector-camera method is presented that interleaves a sequence of pattern images in the dithering sequence of a DLP projector, in a way that the patterns are imperceptible, and can be acquired cleanly with a synchronized high speed camera. This capability enables the procam system to perform as a real-time range sensor, without affecting the appearance of the projected data. The system encodes and decodes a stream of Gray code patterns imperceptibly, and is deployed on a calibrated and stereo rectified procam system to perform depth triangulation from the extracted patterns. The bandwidth achieved imperceptibly is close to 8 million points per second using a general purpose CPU, which is comparable to perceptible commercial hardware accelerated structured light depth cameras.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Epson answers automatic keystone correction (2002). https://www.projectorpeople.com/slis/downloads/whtpapers/epson/auto-keystone.pdf

  2. Pokemon go (2016). http://itunes.apple.com/us/app/pokemon-go/id1094591345?mt=8

  3. Agin, B.: Computer description of curved objects. IEEE Trans. Comput. C–25(4), 439–449 (1976). https://doi.org/10.1109/TC.1976.1674626

    Article  MATH  Google Scholar 

  4. Bandyopadhyay, D., Raskar, R., Fuchs, H.: Dynamic shader lamps: painting on movable objects. In: Proceedings IEEE and ACM International Symposium on Augmented Reality, pp. 207–216. IEEE (2001)

    Google Scholar 

  5. Bermano, A.H., Billeter, M., Iwai, D., Grundhöfer, A.: Makeup lamps: live augmentation of human faces via projection. In: Computer Graphics Forum, vol. 36, pp. 311–323. Wiley Online Library (2017)

    Google Scholar 

  6. Chen, J., Yamamoto, T., Aoyama, T., Takaki, T., Ishii, I.: Real-time projection mapping using high-frame-rate structured light 3D vision. SICE J. Control Measur. Syst. Integr. 8(4), 265–272 (2015)

    Article  Google Scholar 

  7. Cole, A., Ziauddin., S., Greenspan., M.: High-speed imperceptible structured light depth mapping. In: Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, VISAPP, vol. 4, pp. 676–684. INSTICC, SciTePress (2020). https://doi.org/10.5220/0008955906760684

  8. Corporation, M.: Microsoft hololens | mixed reality technology for business (2016). https://www.microsoft.com/en-us/hololens

  9. Corporation, S.E.: Moverio - smart glasses - epson (2011). https://moverio.epson.com/

  10. Cotting, D., Naef, M., Gross, M., Fuchs, H.: Embedding imperceptible patterns into projected images for simultaneous acquisition and display. In: Proceedings of the 3rd IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 100–109. IEEE Computer Society (2004)

    Google Scholar 

  11. Dai, J., Chung, R.: Head pose estimation by imperceptible structured light sensing. In: 2011 IEEE International Conference on Robotics and Automation, pp. 1646–1651. IEEE (2011)

    Google Scholar 

  12. Dai, J., Chung, R.: Making any planar surface into a touch-sensitive display by a mere projector and camera. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 35–42. IEEE (2012)

    Google Scholar 

  13. FLIR Systems, I.: Machine vision cameras (2019). https://www.flir.ca/browse/industrial/machine-vision-cameras/

  14. Fusiello, A., Trucco, E., Verri, A.: A compact algorithm for rectification of stereo pairs. Mach. Vis. Appl. 12(1), 16–22 (2000). https://doi.org/10.1007/s001380050120

    Article  Google Scholar 

  15. Gao, Z., Zhai, G., Wu, X., Min, X., Zhi, C.: DLP based anti-piracy display system. In: 2014 IEEE Visual Communications and Image Processing Conference, pp. 145–148. IEEE (2014)

    Google Scholar 

  16. Gerald, J., Agin, P.T.H.: Movable light-stripe sensor for obtaining three-dimensional coordinate measurements, vol. 0360 (1983). https://doi.org/10.1117/12.934118

  17. Greenspan, M.: Method and apparatus for positional error correction in a robotic pool system using a cue-aligned local camera (2011). uS Patent App. 12/896,045

    Google Scholar 

  18. Grundhöfer, A., Seeger, M., Hantsch, F., Bimber, O.: Dynamic adaptation of projected imperceptible codes. In: Proceedings of the 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 1–10. IEEE Computer Society (2007)

    Google Scholar 

  19. Hieda, N., Cooperstock, J.R.: Digital facial augmentation for interactive entertainment. In: 2015 7th International Conference on Intelligent Technologies for Interactive Entertainment (INTETAIN), pp. 9–16. IEEE (2015)

    Google Scholar 

  20. Intel Corporation: Intel realsense depth camera d400-series. https://software.intel.com/en-us/realsense/d400. Accessed 07 June 2019

  21. Jo, K., Gupta, M., Nayar, S.K.: Disco: display-camera communication using rolling shutter sensors. ACM Trans. Graph. (TOG) 35(5), 150 (2016)

    Article  Google Scholar 

  22. Kagami, S., Hashimoto, K.: Sticky projection mapping: 450-fps tracking projection onto a moving planar surface. In: SIGGRAPH Asia 2015 Emerging Technologies, p. 23. ACM (2015)

    Google Scholar 

  23. Kuroki, Y., Nishi, T., Kobayashi, S., Oyaizu, H., Yoshimura, S.: A psychophysical study of improvements in motion-image quality by using high frame rates. J. Soc. Inform. Display 15(1), 61–68 (2007)

    Article  Google Scholar 

  24. Langerman, D., Sabogal, S., Ramesh, B., George, A.: Accelerating real-time, high-resolution depth Upsampling on FPGAs. In: 2018 IEEE International Conference on Image Processing, Applications and Systems (IPAS), pp. 37–42. IEEE (2018)

    Google Scholar 

  25. Lee, J.C., Hudson, S.E., Tse, E.: Foldable interactive displays. In: Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology, pp. 287–290. ACM (2008)

    Google Scholar 

  26. Li, B., Sezan, I.: Automatic keystone correction for smart projectors with embedded camera. In: 2004 International Conference on Image Processing, ICIP’04, vol. 4, pp. 2829–2832. IEEE (2004)

    Google Scholar 

  27. Lohry, W., Zhang, S.: High-speed absolute three-dimensional shape measurement using three binary dithered patterns. Opt. Exp. 22(22), 26752–26762 (2014)

    Article  Google Scholar 

  28. Marin, G., Dominio, F., Zanuttigh, P.: Hand gesture recognition with leap motion and kinect devices. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 1565–1569 (2014). https://doi.org/10.1109/ICIP.2014.7025313

  29. McDowall, I., Bolas, M.: Fast light for display, sensing and control applications. In: Proceedings of IEEE VR 2005 Workshop on Emerging Display Technologies (EDT), pp. 35–36 (2005)

    Google Scholar 

  30. Morishima, S., Yotsukura, T., Binsted, K., Nielsen, F., Pinhanez, C.: Hypermask talking head projected onto real object. In: Multimedia Modeling: Modeling Multimedia Information and Systems, pp. 403–412. World Scientific (2000)

    Google Scholar 

  31. Mouclier, J.: Glasses (2007). uS Patent App. 29/228,279

    Google Scholar 

  32. Narasimhan, S.G., Koppal, S.J., Yamazaki, S.: Temporal dithering of illumination for fast active vision. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5305, pp. 830–844. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-88693-8_61

    Chapter  Google Scholar 

  33. Narita, G., Watanabe, Y., Ishikawa, M.: Dynamic projection mapping onto a deformable object with occlusion based on high-speed tracking of dot marker array. In: Proceedings of the 21st ACM Symposium on Virtual Reality Software and Technology, pp. 149–152. ACM (2015)

    Google Scholar 

  34. Nelson, S., Ivashin, V.: Method for securely distributing meeting data from interactive whiteboard projector (2014). uS Patent 8,874,657

    Google Scholar 

  35. Park, H., Lee, M.-H., Seo, B.-K., Jin, Y., Park, J.-I.: Content adaptive embedding of complementary patterns for nonintrusive direct-projected augmented reality. In: Shumaker, R. (ed.) ICVR 2007. LNCS, vol. 4563, pp. 132–141. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-73335-5_15

    Chapter  Google Scholar 

  36. Projector, P.: Real time tracking & projection mapping (2017). https://www.youtube.com/watch?v=XkXrLZmnQ_M&feature=youtu.be&fbclid=IwAR0pN2j95uxwMvc1fISs_SL5fmWM4Al3zTTnyEiKwmcp2lg0ReEpZfu2c8c

  37. Pvt.Ltd., T.M.: Motionmagix\(^{\rm TM}\) interactive wall and floor. http://www.touchmagix.com/interactive-floor-interactive-wall

  38. Raskar, R., Welch, G., Cutts, M., Lake, A., Stesin, L., Fuchs, H.: The office of the future: a unified approach to image-based modeling and spatially immersive displays. In: Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques, pp. 179–188. ACM (1998)

    Google Scholar 

  39. Raskar, R., Welch, G., Low, K.-L., Bandyopadhyay, D.: Shader lamps: animating real objects with image-based illumination. In: Gortler, S.J., Myszkowski, K. (eds.) EGSR 2001. E, pp. 89–102. Springer, Vienna (2001). https://doi.org/10.1007/978-3-7091-6242-2_9

    Chapter  Google Scholar 

  40. Resch, C., Keitler, P., Klinker, G.: Sticky projections-a model-based approach to interactive shader lamps tracking. IEEE Trans. Visual Comput. Graph. 22(3), 1291–1301 (2015)

    Article  Google Scholar 

  41. Sagawa, R., Furukawa, R., Kawasaki, H.: Dense 3D reconstruction from high frame-rate video using a static grid pattern. IEEE Trans. Pattern Anal. Mach. Intell. 36(9), 1733–1747 (2014)

    Article  Google Scholar 

  42. Salmi, T., Ahola, J.M., Heikkilä, T., Kilpeläinen, P., Malm, T.: Human-robot collaboration and sensor-based robots in industrial applications and construction. In: Bier, H. (ed.) Robotic Building. SSAE, pp. 25–52. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-70866-9_2

    Chapter  Google Scholar 

  43. Scharstein, D., Szeliski, R.: High-accuracy stereo depth maps using structured light. In: 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003, Proceedings, vol. 1, pp. I-I. IEEE (2003)

    Google Scholar 

  44. Sehn, T.: Apparatus and method for supplying content aware photo filters (2015). uS Patent 9,225,897

    Google Scholar 

  45. Silapasuphakornwong, P., Unno, H., Uehira, K.: Information embedding in real object images using temporally brightness-modulated light. In: Applications of Digital Image Processing XXXVIII, vol. 9599, p. 95992W. International Society for Optics and Photonics (2015)

    Google Scholar 

  46. Texas Instruments: Lightcrafter 6500 Evaluation Module (2016)

    Google Scholar 

  47. den Uyl, T.M., Tasli, H.E., Ivan, P., Snijdewind, M.: Who do you want to be? Real-time face swap. In: 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), vol. 1, p. 1. IEEE (2015)

    Google Scholar 

  48. Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22, 1330–1334 (2000)

    Article  Google Scholar 

  49. Zhang, Z.: Microsoft kinect sensor and its effect. IEEE Multimed. 19(2), 4–10 (2012)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer, Engineering, Queen’s University, Kingston, ON, Canada

    Avery Cole, Sheikh Ziauddin, Jonathon Malcolm & Michael Greenspan

  2. Ingenuity Labs, Queen’s University, Kingston, ON, Canada

    Avery Cole, Sheikh Ziauddin, Jonathon Malcolm & Michael Greenspan

  3. School of Computing, Queen’s University, Kingston, ON, Canada

    Michael Greenspan

Authors
  1. Avery Cole
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sheikh Ziauddin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jonathon Malcolm
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael Greenspan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Greenspan .

Editor information

Editors and Affiliations

  1. IRISA, University of Rennes 1, Rennes, France

    Kadi Bouatouch

  2. Universidade do Porto, Porto, Portugal

    A. Augusto de Sousa

  3. University of Genova, Genova, Italy

    Manuela Chessa

  4. Mines ParisTech, Paris, France

    Alexis Paljic

  5. Linnaeus University, Växjö, Sweden

    Andreas Kerren

  6. French Civil Aviation University (ENAC), Toulouse, France

    Christophe Hurter

  7. Università di Catania, Catania, Italy

    Giovanni Maria Farinella

  8. Universitat de Barcelona, Barcelona, Spain

    Petia Radeva

  9. Escola Superior de Tecnologia de Setúbal, Setúbal, Portugal

    Jose Braz

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Cole, A., Ziauddin, S., Malcolm, J., Greenspan, M. (2022). Efficient Range Sensing Using Imperceptible Structured Light. In: Bouatouch, K., et al. Computer Vision, Imaging and Computer Graphics Theory and Applications. VISIGRAPP 2020. Communications in Computer and Information Science, vol 1474. Springer, Cham. https://doi.org/10.1007/978-3-030-94893-1_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-94893-1_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-94892-4

  • Online ISBN: 978-3-030-94893-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation