Skip to main content
SpringerLink
Log in

Development of a monocular vision system for robotic drilling

  • Published:
Journal of Zhejiang University SCIENCE C Aims and scope Submit manuscript

Abstract

Robotic drilling for aerospace structures demands a high positioning accuracy of the robot, which is usually achieved through error measurement and compensation. In this paper, we report the development of a practical monocular vision system for measurement of the relative error between the drill tool center point (TCP) and the reference hole. First, the principle of relative error measurement with the vision system is explained, followed by a detailed discussion on the hardware components, software components, and system integration. The elliptical contour extraction algorithm is presented for accurate and robust reference hole detection. System calibration is of key importance to the measurement accuracy of a vision system. A new method is proposed for the simultaneous calibration of camera internal parameters and hand-eye relationship with a dedicated calibration board. Extensive measurement experiments have been performed on a robotic drilling system. Experimental results show that the measurement accuracy of the developed vision system is higher than 0.15 mm, which meets the requirement of robotic drilling for aircraft structures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Armingol, J.M., Otamendi, J., de la Escalera, A., et al., 2003. Statistical pattern modeling in vision-based quality control systems. J. Intell. Robot. Syst., 37(3):321–336. [doi:10.1023/A:1025489610281]

    Article  MATH  Google Scholar 

  • Bilen, H., Hocaoglu, M., Unel, M., et al., 2012. Developing robust vision modules for microsystems applications. Mach. Vis. Appl., 23(1):25–42. [doi:10.1007/s00138-010-0267-y]

    Article  Google Scholar 

  • Bone, G.M., Capson, D., 2003. Vision-guided fixtureless assembly of automotive components. Robot. Comput.-Integr. Manuf., 19(1–2):79–87. [doi:10.1016/S0736-5845 (02)00064-9]

    Article  Google Scholar 

  • Bradski, G., Kaehler, A., 2008. Learning OpenCV: Computer Vision with the OpenCV Library. O’reilly, Sebastopol, CA.

    Google Scholar 

  • Chen, S., Zhang, Y., Qiu, T., et al., 2003. Robotic welding systems with vision-sensing and self-learning neuron control of arc welding dynamic process. J. Intell. Robot. Syst., 36(2):191–208. [doi:10.1023/A:1022652706683]

    Article  Google Scholar 

  • DeVlieg, R., Sitton, K., Feikert, E., et al., 2002. ONCE (ONe-sided Cell End effector) Robotic Drilling System. SAE Technical Paper 2002-01-2626. [doi:10.4271/2002-01-2626]

    Book  Google Scholar 

  • Dornaika, F., Horaud, R., 1998. Simultaneous robot-world and hand-eye calibration. IEEE Trans. Robot. Autom., 14(4): 617–622. [doi:10.1109/70.704233]

    Article  Google Scholar 

  • Eberli, D., Scaramuzza, D., Weiss, S., et al., 2010. Vision based position control for MAVs using one single circular landmark. J. Intell. Robot. Syst., 61(1–4):495–512. [doi:10.1007/s10846-010-9494-8]

    Google Scholar 

  • Fitzgibbon, A., Pilu, M., Fisher, R.B., 1999. Direct least square fitting of ellipses. IEEE Trans. Pattern Anal. Mach. Intell., 21(5):476–480. [doi:10.1109/34.765658]

    Article  Google Scholar 

  • Forsyth, D.A., Ponce, J., 2011. Computer Vision: a Modern Approach (2nd Ed.). Prentice-Hall, Englewood Cliffs, New Jersey.

    Google Scholar 

  • Ibarguren, A., Martínez-Otzeta, J.M., Maurtua, I., 2014. Particle filtering for industrial 6DOF visual servoing. J. Intell. Robot. Syst., 74(3–4):689–696. [doi:10.1007/s10846-013-9854-2]

    Article  Google Scholar 

  • Lee, B., Tarng, Y., 2001. Surface roughness inspection by computer vision in turning operations. Int. J. Mach. Tool. Manuf., 41(9):1251–1263. [doi:10.1016/S0890-6955(01)00023-2]

    Article  Google Scholar 

  • Malassiotis, S., Strintzis, M.G., 2003. Stereo vision system for precision dimensional inspection of 3D holes. Mach. Vis. Appl., 15(2):101–113. [doi:10.1007/s00138-003-0132-3]

    Article  Google Scholar 

  • Malti, A., 2012. Hand-eye calibration with epipolar constraints: application to endoscopy. Robot. Auton. Syst., 61(2):161–169. [doi:10.1016/j.robot.2012.09.029]

    MathSciNet  Google Scholar 

  • Motta, J.M.C.S., de Carvalho, G.C., McMaster, R., 2001. Robot calibration using a 3D vision-based measurement system with a single camera. Robot. Comput.-Integr. Manuf., 17(6):487–497. [doi:10.1016/S0736-5845(01)00024-2]

    Article  Google Scholar 

  • Neto, H.V., Nehmzow, U., 2007. Real-time automated visual inspection using mobile robots. J. Intell. Robot. Syst., 49(3):293–307. [doi:10.1007/s10846-007-9146-9]

    Article  Google Scholar 

  • Olsson, T., Haage, M., Kihlman, H., et al., 2010. Cost-efficient drilling using industrial robots with high-bandwidth force feedback. Robot. Comput.-Integr. Manuf., 26(1):24–38. [doi:10.1016/j.rcim.2009.01.002]

    Article  Google Scholar 

  • Otsu, N., 1979. A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern., 9(1):62–66. [doi:10.1109/TSMC.1979.4310076]

    Article  MathSciNet  Google Scholar 

  • Pachidis, T.P., Lygouras, J.N., 2007. Vision-based path generation method for a robot-based arc welding system. J. Intell. Robot. Syst., 48(3):307–331. [doi:10.1007/s10846-006-9076-y]

    Article  Google Scholar 

  • Simonvsky, M., 2011. Ellipse Detection Using 1D Hough Transform. Available from http://www.mathworks.com/matlabcentral/fileexchange/33970-ellipse-detection-using-1d-hough-transform.

    Google Scholar 

  • Strobl, K.H., Hirzinger, G., 2008. More accurate camera and hand-eye calibrations with unknown grid pattern dimensions. IEEE Int. Conf. on Robotics and Automation, p.1398–1405. [doi:10.1109/ROBOT.2008.4543398]

    Google Scholar 

  • Strobl, K.H., Sepp, W., Hirzinger, G., 2009. On the issue of camera calibration with narrow angular field of view. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, p.309–315. [doi:10.1109/IROS.2009.5354776]

  • Suzuki, S., 1985. Topological structural analysis of digitized binary images by border following. Comput. Vis. Graph. Image Process., 30(1):32–46. [doi:10.1016/0734-189X(85)90016-7]

    Article  MATH  Google Scholar 

  • Thompson, P., Hartmann, J., Feikert, E., et al., 2005. Flex Track for Use in Production. SAE Technical Paper 2005-01-3318. [doi:10.4271/2005-01-3318]

    Book  Google Scholar 

  • Tsai, R., 1987. A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE J. Robot. Autom., 3(4):323–344. [doi:10.1109/JRA.1987.1087109]

    Article  Google Scholar 

  • Webb, P., Eastwood, S., Jayaweera, N., et al., 2005. Automated aerostructure assembly. Ind. Robot., 32(5):383–387. [doi:10.1108/01439910510614646]

    Article  Google Scholar 

  • Williams, D.J., Shah, M., 1992. A fast algorithm for active contours and curvature estimation. CVGIP Image Understand., 55(1):14–26. [doi:10.1016/1049-9660(92)90003-L]

    Article  MATH  Google Scholar 

  • Zhan, Q., Wang, X., 2012. Hand-eye calibration and positioning for a robot drilling system. Int. J. Adv. Manuf. Technol., 61(5–8):691–701. [doi:10.1007/s00170-011-3741-4]

    Article  Google Scholar 

  • Zhang, Z., 2000. A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell., 22(11): 1330–1334. [doi:10.1109/34.888718]

    Article  Google Scholar 

  • Zhao, Z., Weng, Y., 2013. A flexible method combining camera calibration and hand-eye calibration. Robotica, 31(5):747–756. [doi:10.1017/S0263574713000040]

    Article  Google Scholar 

  • Zhou, Y., Nelson, B.J., Vikramaditya, B., 2000. Integrating optical force sensing with visual servoing for microassembly. J. Intell. Robot. Syst., 28(3):259–276. [doi:10. 1023/A:1008136711577]

    Article  MATH  Google Scholar 

  • Zhu, W., Qu, W., Cao, L., et al., 2013. An off-line programming system for robotic drilling in aerospace manufacturing. Int. J. Adv. Manuf. Technol., 68(9–12):2535–2545. [doi:10.1007/s00170-013-4873-5]

    Article  Google Scholar 

  • Zou, X., Zou, H., Lu, J., 2012. Virtual manipulator-based binocular stereo vision positioning system and errors modelling. Mach. Vis. Appl., 23(1):43–63. [doi:10.1007/s00138-010-0291-y]

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, Zhejiang University, Hangzhou, 310027, China

    Wei-dong Zhu, Biao Mei, Guo-rui Yan & Ying-lin Ke

  2. The State Key Lab of Fluid Power Transmission and Control, Zhejiang University, Hangzhou, 310027, China

    Wei-dong Zhu, Biao Mei, Guo-rui Yan & Ying-lin Ke

Authors
  1. Wei-dong Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Biao Mei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guo-rui Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ying-lin Ke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei-dong Zhu.

Additional information

Project supported by the National Natural Science Foundation of China (Nos. 51205352 and 51221004)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, Wd., Mei, B., Yan, Gr. et al. Development of a monocular vision system for robotic drilling. J. Zhejiang Univ. - Sci. C 15, 593–606 (2014). https://doi.org/10.1631/jzus.C1300379

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1631/jzus.C1300379

Key words

CLC number

Search

Navigation