Skip to main content
SpringerLink
Log in

A semi-supervised deep learning approach for circular hole detection on composite parts

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

This paper introduces the usage of semi-supervised learning to obtain competitive detection accuracy of measuring drilled holes on composite parts with very limited noisy training data. An improved texture segmentation algorithm based on local binary patterns algorithm is proposed, named local exponential patterns. The algorithm divides the image texture into nine levels, of which the highest level of texture is selected for contour extraction. An ellipse fitting method is used to fit the target contours and vote for the candidate ellipses. The regions inside the candidate ellipses are taken as the semi-supervised semantic label for images. A new loss named round loss is proposed, and a superior circle segmentation model was trained by learning from incompletely annotated data. To verify the effectiveness of the method, experiments were conducted with the drilled holes on the composite parts. The results show that the proposed semi-supervised deep learning approach is exceedingly suitable for circle detection of holes with different texture information commonly found in robotic drilling. Massive data labeling can be completely avoided with proposed method. The measurement accuracy can reach 0.03 mm, which can meet the visual measurement requirements of the circular holes on composite parts in the robotic drilling system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Pham, S.: Digital circles with non-lattice point centers. Vis. Comput. 9(1), 1–24 (1992). https://doi.org/10.1007/BF01901025

    Article  Google Scholar 

  2. Hu, P., Peng, D., Guo, J., Zhen, L.: Local feature based multi-view discriminant analysis. Knowl.-Based Syst. 149, 34–46 (2018). https://doi.org/10.1016/j.knosys.2018.02.008

    Article  Google Scholar 

  3. Hu, P., Peng, D., Sang, Y., Xiang, Y.: Multi-view linear discriminant analysis network. IEEE Trans. Image Process. (2019). https://doi.org/10.1109/tip.2019.2913511

    Article  MathSciNet  MATH  Google Scholar 

  4. Martínez, J., García, A.S., Molina, J.P., Martínez, D., González, P.: Erratum to: an empirical evaluation of different haptic feedback for shape and texture recognition. Vis. Comput. 29(5), 467 (2013). https://doi.org/10.1007/s00371-013-0781-9

    Article  Google Scholar 

  5. Lingkun, L., Wang, X., Hu, S., Hu, X., Zhang, H., Liu, Y., Zhang, J.: A unified framework for interactive image segmentation via Fisher rules. Vis. Comput. (2018). https://doi.org/10.1007/s00371-018-1580-0

    Article  Google Scholar 

  6. Peng, X., Feng, J., Xiao, S., Yau, W., Zhou, J.T., Yang, S.: Structured autoencoders for subspace clustering. IEEE Trans. Image Process. 27(10), 5076–5086 (2018). https://doi.org/10.1109/TIP.2018.2848470

    Article  MathSciNet  Google Scholar 

  7. Amirolad, A., Arashloo, S.R., Amirani, M.C.: Multi-layer local energy patterns for texture representation and classification. Vis. Comput. 32(12), 1633–1644 (2016). https://doi.org/10.1007/s00371-016-1220-5

    Article  Google Scholar 

  8. Duda, R.O., Hart, P.E.: Use of the Hough transform to detect lines and curves in pictures. Commun. ACM-CACM 15(1), 11 (1975)

    Article  Google Scholar 

  9. Nakanishi, M., Ogura, T.: Real-time CAM-based Hough transform algorithm and its performance evaluation. Mach. Vis. Appl. 12(2), 59–68 (2000). https://doi.org/10.1007/s001380050125

    Article  MATH  Google Scholar 

  10. Shaked, D., Yaron, O., Kiryati, N.: Deriving stopping rules for the probabilistic Hough transform by sequential analysis. Comput. Vis. Image Underst. 63(3), 512–526 (1996). https://doi.org/10.1006/cviu.1996.0038

    Article  Google Scholar 

  11. Xu, L., Oja, E., Kultanen, P.: A new curve detection method: randomized hough transform (RHT). Pattern Recognit. Lett. 11(5), 331–338 (1990). https://doi.org/10.1016/0167-8655(90)90042-Z

    Article  MATH  Google Scholar 

  12. Han, J.H., Kóczy, L.T., Poston, T.: Fuzzy Hough transform. Pattern Recognit. Lett. 15(7), 649–658 (1994). https://doi.org/10.1016/0167-8655(94)90068-x

    Article  Google Scholar 

  13. Chen, T.-C., Chung, K.-L.: An efficient randomized algorithm for detecting circles. Comput. Vis. Image Underst. 83(2), 172–191 (2001). https://doi.org/10.1006/cviu.2001.0923

    Article  MATH  Google Scholar 

  14. Chung, K.-L., Huang, Y.-H.: Speed up the computation of randomized algorithms for detecting lines, circles, and ellipses using novel tuning-and LUT-based voting platform. Appl. Math. Comput. 190(1), 132–149 (2007). https://doi.org/10.1016/j.amc.2007.01.012

    Article  MathSciNet  MATH  Google Scholar 

  15. Chung, K.-L., Huang, Y.-H., Shen, S.-M., Krylov, A.S., Yurin, D.V., Semeikina, E.V.: Efficient sampling strategy and refinement strategy for randomized circle detection. Pattern Recognit. 45(1), 252–263 (2012). https://doi.org/10.1016/j.patcog.2011.07.004

    Article  Google Scholar 

  16. Ayala-Ramirez, V., Garcia-Capulin, C.H., Perez-Garcia, A., Sanchez-Yanez, R.E.: Circle detection on images using genetic algorithms. Pattern Recognit. Lett. 27(6), 652–657 (2006). https://doi.org/10.1016/j.patrec.2005.10.003

    Article  Google Scholar 

  17. Cuevas, E., Oliva, D., Zaldivar, D., Pérez-Cisneros, M., Sossa, H.: Circle detection using electro-magnetism optimization. Inf. Sci. 182(1), 40–55 (2012). https://doi.org/10.1016/j.ins.2010.12.024

    Article  MathSciNet  Google Scholar 

  18. Akinlar, C., Topal, C.: EDCircles: a real-time circle detector with a false detection control. Pattern Recognit. 46(3), 725–740 (2013). https://doi.org/10.1016/j.patcog.2012.09.020

    Article  Google Scholar 

  19. Chen, S., Xia, R., Zhao, J., Chen, Y., Hu, M.: A hybrid method for ellipse detection in industrial images. Pattern Recognit. 68, 82–98 (2017). https://doi.org/10.1016/j.patcog.2017.03.007

    Article  Google Scholar 

  20. Smith, J.R., Shih-Fu, C.: Automated binary texture feature sets for image retrieval. In: 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings, 9–9 May 1996, vol. 2234, pp. 2239–2242

  21. Haralick, R., Shanmugam, K., Dinstein, I.H.: Texture features for image classification. IEEE Trans. Syst. Man Cybern. 3, 610–621 (1973)

    Article  Google Scholar 

  22. Mallat, S.G.: A theory for multiresolution signal decomposition: the wavelet representation. IEEE Trans. Pattern Anal. Mach. Intell. 11(7), 674–693 (1989). https://doi.org/10.1109/34.192463

    Article  MATH  Google Scholar 

  23. Galloway, M.M.: Texture analysis using gray level run lengths. Comput. Gr. Image Process. 4(2), 172–179 (1975). https://doi.org/10.1016/S0146-664X(75)80008-6

    Article  Google Scholar 

  24. Hassner, M., Sklansky, J.: The use of Markov random fields as models of texture. Comput. Gr. Image Process. 12(4), 357–370 (1980). https://doi.org/10.1016/0146-664X(80)90019-2

    Article  Google Scholar 

  25. Ojala, T., Pietikäinen, M., Harwood, D.: A comparative study of texture measures with classification based on featured distributions. Pattern Recogn. 29(1), 51–59 (1996). https://doi.org/10.1016/0031-3203(95)00067-4

    Article  Google Scholar 

  26. Pietikäinen, M., Ojala, T., Xu, Z.: Rotation-invariant texture classification using feature distributions. Pattern Recognit. 33(1), 43–52 (2000). https://doi.org/10.1016/S0031-3203(99)00032-1

    Article  Google Scholar 

  27. Ojala, T., Pietikäinen, M., Mäenpää, T.: A generalized local binary pattern operator for multiresolution gray scale and rotation invariant texture classification. In: Singh, S., Murshed, N., Kropatsch, W. (eds.) Advances in pattern recognition—ICAPR 2001, pp. 399–408. Springer, Berlin (2001)

  28. Gou, J., Wang, L., Hou, B., Lv, J., Yuan, Y., Mao, Q.: Two-phase probabilistic collaborative representation-based classification. Expert Syst. Appl. 133, 9–20 (2019). https://doi.org/10.1016/j.eswa.2019.05.009

    Article  Google Scholar 

  29. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. Arxiv 79 (2014)

  30. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) Medical image Computing and Computer-Assisted Intervention–MICCAI 2015, pp. 234–241. Springer International Publishing, Cham (2015)

    Chapter  Google Scholar 

  31. Sánchez López, L.: Local Binary Patterns applied to Face Detection and Recognition. Universitat Politècnica De Catalunya, Spain (2010)

    Google Scholar 

  32. Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. PAMI-8(6), 679–698 (1986). https://doi.org/10.1109/tpami.1986.4767851

    Article  Google Scholar 

  33. Mei, B., Zhu, W., Yan, G., Ke, Y.: A new elliptic contour extraction method for reference hole detection in robotic drilling. Pattern Anal. Appl. 18(3), 695–712 (2015). https://doi.org/10.1007/s10044-014-0394-6

    Article  MathSciNet  Google Scholar 

  34. Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017). https://doi.org/10.1109/TPAMI.2016.2644615

    Article  Google Scholar 

  35. Everingham, M., Eslami, S.M.A., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The pascal visual object classes challenge: a retrospective. Int. J. Comput. Vis. 111(1), 98–136 (2015). https://doi.org/10.1007/s11263-014-0733-5

    Article  Google Scholar 

  36. Maletic, J.I., Marcus, A., Grzymala-Busse, J.W., Grzymala-Busse, W.J., Burges, C.J.C., Chizi, B., Maimon, O., Ying, Y., Webb, G.I., Wu, X.: Data mining and knowledge discovery handbook. Kybernetes 33(7), 809–835 (2005)

    Google Scholar 

  37. Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybernet. 9(1), 62–66 (1979). https://doi.org/10.1109/TSMC.1979.4310076

    Article  Google Scholar 

  38. Bieniek, A., Moga, A.: An efficient watershed algorithm based on connected components. Pattern Recognit. 33(6), 907–916 (2000). https://doi.org/10.1016/S0031-3203(99)00154-5

    Article  Google Scholar 

  39. Achantay, R., Hemamiz, S., Estraday, F., Süsstrunky, S.: Frequency-tuned salient region detection. In: 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2009, pp. 1597–1604 (2009). https://doi.org/10.1109/cvprw.2009.5206596

Download references

Acknowledgements

This research is supported by National Natural Science Foundation of China (No.51675479, 51205352, 51575479, 51521064) and Special scientific research for civil aircraft (NO.MJZ-G-2011-07).

Author information

Authors and Affiliations

  1. State Key Laboratory of Fluid Power and Mechatronic Systems, School of Mechanical Engineering, Zhejiang University, Hangzhou, 310027, China

    Guanhua Li, Shuang Yang, Siming Cao, Weidong Zhu & Yinglin Ke

  2. Key Laboratory of Advanced Manufacturing Technology of Zhejiang Province, School of Mechanical Engineering, Zhejiang University, Hangzhou, 310027, China

    Guanhua Li, Shuang Yang & Siming Cao

Authors
  1. Guanhua Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Siming Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Weidong Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yinglin Ke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Weidong Zhu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, G., Yang, S., Cao, S. et al. A semi-supervised deep learning approach for circular hole detection on composite parts. Vis Comput 37, 433–445 (2021). https://doi.org/10.1007/s00371-020-01812-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-020-01812-w

Keywords

Search

Navigation