Skip to main content
SpringerLink
Log in

High-accuracy, high-speed 3D structured light imaging techniques and potential applications to intelligent robotics

  • Regular Paper
  • Published:
International Journal of Intelligent Robotics and Applications Aims and scope Submit manuscript

Abstract

This paper presents some of the high-accuracy and high-speed structured light 3D imaging methods developed in the optical metrology community. These advanced 3D optical imaging technologies could substantially benefit the intelligent robotics community as another sensing tool. This paper mainly focuses on one special 3D imaging technique: the digital fringe projection (DFP) method because of its numerous advantageous features compared to other 3D optical imaging methods in terms of accuracy, resolution, speed, and flexibility. We will discuss technologies that enabled 3D data acquisition, reconstruction, and display at 30 Hz or higher with over 300,000 measurement points per frame. This paper intends to introduce the DFP technologies to the intelligent robotics community, and casts our perspectives on potential applications for which such sensing methods could be of value.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

References

  • Abbott, J., Nagy, Z., Beyeler, F., Nelson, B.: Robotics in the small. IEEE Robot. Autom. Mag. 14, 92–103 (2007)

    Article  Google Scholar 

  • Bell, T., Zhang, S.: Towards superfast three-dimensional optical metrology with digital micromirrror device (dmd) platforms. Opt. Eng. 53(11), 112206 (2014)

  • Cappelleri, D., Efthymiou, D., Goswami, A., Vitoroulis, N., Zavlanos, M.: Towards mobile microrobot swarms for additive micromanufacturing. Int. J. Adv. Robot. Syst. 11(9), 150 (2014).

    Article  Google Scholar 

  • Cheang, U.K., Lee, K., Julius, A.A., Kim, M.J.: Multiple-robot drug delivery strategy through coordinated teams of microswimmers. Appl. Phys. Lett. 105(8), 083705 (2014)

  • Cheng, Y.Y., Wyant, J.C.: Two-wavelength phase shifting interferometry. Appl. Opt. 23, 4539–4543 (1984)

    Article  Google Scholar 

  • Cheng, Y.Y., Wyant, J.C.: Multiple-wavelength phase shifting interferometry. Appl. Opt. 24, 804–807 (1985)

    Article  Google Scholar 

  • Chowdhury, S., Jing, W., Cappelleri, D.J.: Controlling multiple microrobots: recent progress and future challenges. J. Micro Bio Robot. 10(1), 1–11 (2015a)

  • Chowdhury, S., Jing, W., Cappelleri, D.J.: Towards independent control of multiple magnetic mobile microrobots. Micromachines 7(1), 3 (2015b)

  • Chowdhury, S., Jing, W., Jaron, P., Cappelleri, D.: Path planning and control for autonomous navigation of single and multiple magnetic mobile microrobots. In: Proceedings of the ASME 2015 International Design Engineering Technical Conferences & Computers and Informatio in Engineering Conference. Boston, Massachusetts (2015c)

  • DeVon, D., Bretl, T.: Control of many robots moving in the same direction with different speeds: a decoupling approach. In: American Control Conference. ACC’09., pp. 1794–1799. IEEE (2009)

  • Dhond, U.R., Aggarwal, J.K.: Structure from stereo-a review. IEEE Trans. Syst. Man. Cybern. 19(6), 1489–1510 (1989)

  • Diller, E., Floyd, S., Pawashe, C., Sitti, M.: Control of multiple heterogeneous magnetic microrobots in two dimensions on nonspecialized surfaces. IEEE Trans. Robot. 28(1), 172–182 (2012)

    Article  Google Scholar 

  • Diller, E., Sitti, M., et al.: Micro-scale mobile robotics. Found. Trends Robot. 2(3), 143–259 (2013)

    Article  Google Scholar 

  • Floyd, S., Pawashe, C., Sitti, M.: An untethered magnetically actuated micro-robot capable of motion on arbitrary surfaces. In: Robotics and Automation, 2008. ICRA 2008. IEEE International Conference on, pp. 419–424. IEEE (2008)

  • Frutiger, D.R., Vollmers, K., Kratochvil, B.E., Nelson, B.J.: Small, fast, and under control: wireless resonant magnetic micro-agents. Int. J. Robot. Res. 29(5), 613–636 (2010)

    Article  Google Scholar 

  • Fusco, S., Huang, H.W., Peyer, K.E., Peters, C., Hberli, M., Ulbers, A., Spyrogianni, A., Pellicer, E., Sort, J., Pratsinis, S.E., Nelson, B.J., Sakar, M.S., Pan, S.: Shape-switching microrobots for medical applications: The influence of shape in drug delivery and locomotion. ACS Appl. Mater. Interfaces 7(12), 6803–6811 (2015)

    Article  Google Scholar 

  • Ghiglia, D.C., Pritt, M.D. (eds.): Two-Dimensional Phase Unwrapping: Theory, Algorithms, and Software. Wiley, New York (1998)

    MATH  Google Scholar 

  • Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2000). ISBN:0521623049

  • Huang, H.W., Sakar, M.S., Riederer, K., Shamsudhin, N., Petruska, A., Pan, S., Nelson, B.J.: Magnetic microrobots with addressable shape control. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 1719–1724 (2016)

  • Huang, P.S., Zhang, S.: Fast three-step phase-shifting algorithm. Appl. Opt. 45(21), 5086–5091 (2006)

    Article  Google Scholar 

  • Huang, P.S., Zhang, S., Chiang, F.P.: Trapezoidal phase-shifting method for three-dimensional shape measurement. Opt. Eng. 44(12), 123601 (2005)

  • Huang, Y., Shang, Y., Liu, Y., Bao, H.: Handbook of 3D Machine Vision: Optical Metrology and Imaging, 1st edn., chap. 3D shapes from Speckle, pp. 33–56. CRC, USA (2013)

  • Jia, P., Kofman, J., English, C.: Two-step triangular-pattern phase-shifting method for three-dimensional object-shape measurement. Opt. Eng. 46(8), 083201 (2007)

  • Jing, W., Cappelleri, D.: Incorporating in-situ force sensing capabilities in a magnetic microrobot. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2014), pp. 4704–4709 (2014a)

  • Jing, W., Cappelleri, D.: A magnetic microrobot with in situ force sensing capabilities. Robotics 3(2), 106–119 (2014b)

  • Jing, W., Cappelleri, D.J.: Towards functional mobile magnetic microrobots. In: Small-Scale Robotics. From Nano-to-Millimeter-Sized Robotic Systems and Applications, pp. 81–100. Springer, New York (2014c)

  • Jing, W., Chen, X., Lyttle, S., Fu, Z., Shi, Y., Cappelleri, D.: A magnetic thin film microrobot with two operating modes. In: 2011 IEEE International Conference on Robotics and Automation (ICRA), pp. 96–101 (2011)

  • Jing, W., Pagano, N., Cappelleri, D.J.: A micro-scale magnetic tumbling microrobot. In: ASME 2012 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, pp. 187–196 (2012)

  • Jing, W., Pagano, N., Cappelleri, D.J.: A novel micro-scale magnetic tumbling microrobot. J. Micro Bio Robot. 8(1), 1–12 (2013a)

  • Jing, W., Pagano, N., Cappelleri, D.J.: A tumbling magnetic microrobot with flexible operating modes. In: 2013 IEEE International Conference on Robotics and Automation (ICRA), pp. 5514–5519 (2013b)

  • Karpinsky, N., Hoke, M., Chen, V., Zhang, S.: High-resolution, real-time three-dimensional shape measurement on graphics processing unit. Opt. Eng. 53(2), 024105 (2014)

  • Keferstein, C.P., Marxer, M.: Testing bench for laser triangulation sensors. Sensor Rev. 18(3), 183–187 (1998)

    Article  Google Scholar 

  • Kummer, M.P., Abbott, J.J., Kratochvil, B.E., Borer, R., Sengul, A., Nelson, B.J.: Octomag: An electromagnetic system for 5-dof wireless micromanipulation. Robot. IEEE Trans. 26(6), 1006–1017 (2010)

    Article  Google Scholar 

  • Lei, S., Zhang, S.: Flexible 3-d shape measurement using projector defocusing. Opt. Lett. 34(20), 3080–3082 (2009)

    Article  Google Scholar 

  • Li, B., Wang, Y., Dai, J., Lohry, W., Zhang, S.: Some recent advances on superfast 3d shape measurement with digital binary defocusing techniques. Opt. Laser Eng. 54, 236–246 (2014)

    Article  Google Scholar 

  • Li, B., Zhang, S.: Microscopic structured light 3d profilometry: Binary defocusing technique vs. sinusoidal fringe projection. Opt. Laser Eng. (2016) (in press)

  • Liu, K., Wang, Y., Lau, D.L., Hao, Q., Hassebrook, L.G.: Dual-frequency pattern scheme for high-speed 3-d shape measurement. Opt. Express 18, 5229–5244 (2010)

    Article  Google Scholar 

  • Malacara, D. (ed.): Optical Shop Testing, 3rd edn. Wiley, New York (2007)

    Google Scholar 

  • Manneberg, G., Hertegard, S., Liljencrantz, J.: Measurement of human vocal fold vibrations with laser triangulation. Opt. Eng. 40(9), 2041–2044 (2001)

    Article  Google Scholar 

  • Morano, R.A., Ozturk, C., Conn, R., Dubin, S., Zietz, S., Nissanov, J.: Structured light using pseudorandom codes. IEEE Trans. Pattern Anal. Mach. Intell. 20, 322–327 (1998)

  • Palagi, S., Mark, A.G., Reigh, S.Y., Melde, K., Qiu, T., Zeng, H., Parmeggiani, C., Martella, D., Sanchez-Castillo, A., Kapernaum, N., Giesselmann, F., Wiersma, D.S., Lauga, E., Fischer, P.: Structured light enables biomimetic swimming and versatile locomotion of photoresponsive soft microrobots. Nat. Mater. 15(6), 647–653 (2016)

  • Pan, J., Huang, P., Zhang, S., Chiang, F.P.: Color n-ary gray code for 3-d shape measurement. In: 12th International Conference on Experimental Mechanics. Politecnico di Bari, Italy (2004)

  • Pawashe, C., Floyd, S., Sitti, M.: Modeling and experimental characterization of an untethered magnetic micro-robot. Int. J. Robot. Res. 28(8), 1077–1094 (2009a)

  • Pawashe, C., Floyd, S., Sitti, M.: Multiple magnetic microrobot control using electrostatic anchoring. Appl. Phys. Lett. 94(16), 164108 (2009b)

  • Salvi, J., Fernandez, S., Pribanic, T., Llado, X.: A state of the art in structured light patterns for surface profilometry. Patt. Recogn. 43(8), 2666–2680 (2010)

    Article  MATH  Google Scholar 

  • Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comp. Vis. 47(1–3), 7–42 (2002)

    Article  MATH  Google Scholar 

  • Sirikasemlert, A., Tao, X.: Objective evaluation of textural changes in knitted fabrics by laser triangulation. Text. Res. J. 70(12), 1076–1087 (2000)

    Article  Google Scholar 

  • Steager, E.B., Sakar, M.S., Magee, C., Kennedy, M., Cowley, A., Kumar, V.: Automated biomanipulation of single cells using magnetic microrobots. Int. J. Robot. Res. 32(3), 346–359 (2013)

    Article  Google Scholar 

  • Towers, D.P., Jones, J.D.C., Towers, C.E.: Optimum frequency selection in multi-frequency interferometry. Opt. Lett. 28, 1–3 (2003)

    Article  Google Scholar 

  • Wang, Y., Laughner, J.I., Efimov, I.R., Zhang, S.: 3d absolute shape measurement of live rabbit hearts with a superfast two-frequency phase-shifting technique. Opt. Express 21(5), 5822–5632 (2013)

    Article  Google Scholar 

  • Wang, Y., Zhang, S.: Superfast multifrequency phase-shifting technique with optimal pulse width modulation. Opt. Express 19(6), 5143–5148 (2011)

    Article  Google Scholar 

  • Xu, J., Zhao, J., Luo, R.: A 3d imaging sensor for mobile manipulation. IEEE International Conference on Cyber Technology in Automation. Control and Intelligent Systems (CYBER), pp. 315–320. Bangkok, Thailand (2012)

  • Zhang, C., Xu, J., Xi, N., Jia, Y., Li, W.: Development of an omni-directional 3d camera for robot navigation. In: IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), pp. 262–267. Kaohsiung, Taiwan (2012)

  • Zhang, S.: Recent progresses on real-time 3-d shape measurement using digital fringe projection techniques. Opt. Laser Eng. 48(2), 149–158 (2010)

    Article  Google Scholar 

  • Zhang, S.: High-speed 3D imaging with digital fringe projection technique, 1st edn. CRC, New York (2016)

    Google Scholar 

  • Zhang, S., Huang, P.S.: High-resolution real-time three-dimensional shape measurement. Opt. Eng. 45(12), 123601 (2006)

  • Zhang, S., Huang, P.S.: Novel method for structured light system calibration. Opt. Eng. 45(8), 083601 (2006)

  • Zhang, S., Li, X., Yau, S.T.: Multilevel quality-guided phase unwrapping algorithm for real-time three-dimensional shape reconstruction. Appl. Opt. 46(1), 50–57 (2007)

    Article  Google Scholar 

  • Zhang, S., Royer, D., Yau, S.T.: Gpu-assisted high-resolution, real-time 3-d shape measurement. Opt. Express 14(20), 9120–9129 (2006)

    Article  Google Scholar 

  • Zhang, S., van der Weide, D., Oliver, J.: Superfast phase-shifting method for 3-d shape measurement. Opt. Express 18(9), 9684–9689 (2010)

    Article  Google Scholar 

  • Zhang, S., Yau, S.T.: High-resolution, real-time 3-d absolute coordinate measurement based on a phase-shifting method. Opt. Express 14(7), 2644–2649 (2006)

    Article  Google Scholar 

  • Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)

    Article  Google Scholar 

  • Zhang, Z.: Microsoft kinect sensor and its effect. IEEE Multimed. 19(2), 4–10 (2012)

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank many current and former students working in our laboratory for their tremendous work in advancing 3D imaging technologies to their current state. In particular, we would like to thank Dr. Yajun Wang for his development of binary dithering and optimal pulse width modulation methods; Dr. Nik Karpinsky for implementation of graphics processing unit (GPU) programming; Jae-Sang Hyun and Zexuan Zhu for their hardware system design and developments; and Maggie Hao, Chufan Jiang and Ziping Liu for serving as models to test our systems.

Author information

Authors and Affiliations

  1. School of Mechanical Engineering, Purdue University, West Lafayette, IN, 47907, USA

    Beiwen Li, Yatong An, David Cappelleri & Song Zhang

  2. Department of Mechanical Engineering, Tsinghua University, Beijing, 100084, China

    Jing Xu

Authors
  1. Beiwen Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yatong An
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David Cappelleri
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jing Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Song Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Song Zhang.

Additional information

This research was supported by National Science Foundation research Grant Numbers CMMI-1523048 and CMMI-1531048.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, B., An, Y., Cappelleri, D. et al. High-accuracy, high-speed 3D structured light imaging techniques and potential applications to intelligent robotics. Int J Intell Robot Appl 1, 86–103 (2017). https://doi.org/10.1007/s41315-016-0001-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41315-016-0001-7

Keywords

Search

Navigation