Skip to main content
SpringerLink
Log in

Palletizing Robot Positioning Bolt Detection Based on Improved YOLO-V3

  • Regular paper
  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

To improve the detection accuracy and speed of palletizing robot positioning bolts in complex scenes, we proposed a positioning bolt (PB) detection method based on improved YOLO-V3. First, due to the actual detection requirement, we constructed the PB data set by using a series of data enhancement operations such as horizontal flip, ± 30degree rotation, and random luminance enhancement or decrease. Then, an improved anchor box mechanism based on the k-means++ algorithm was designed to obtain a more accurate anchor box for the PB data. According to the feature of the PB data in the palletizing robot, such as the existence of dust and dirt on the surface, the feature extraction network was further enhanced by adding a Densenet-4 module. In this way, the low-level semantics and high-level abstract features can be extracted effectively to improve detection performance. Finally, a new bounding box regression loss function was elaborated to accelerate the neural network training. The experimental results demonstrated the effectiveness of the proposed improvement mechanisms. The comparable results also show that our method is superior to the original YOLO-V3, SSD, and Faster R-CNN for PB data, and has a detection AP of 86.7%, a recall rate of 97%, and a detection speed of 25.47 FPS, which can achieve high-efficiency and high-precision detection in complex industrial scenarios.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Code Availability

Code generated or used during the study is available from the corresponding author by request.

References

  1. Moura, F.M., Silva, M.F.: Application for automatic programming of palletizing robots. In: 2018 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), pp 48–53. IEEE (2018)

  2. de Souza, J.P.C., Castro, A.L., Rocha, L.F., et al.: Adaptpack studio translator: translating offline programming to real palletizing robots Industrial Robot: The International Journal of Robotics Research and Application (2020)

  3. Li, C., Ma, Y., Wang, S., et al.: Novel industrial robot sorting technology based on machine vision. In: 2017 9th International Conference on Modelling, Identification and Control (ICMIC), pp 902–907. IEEE (2017)

  4. Wang, J., Zhang, X., Dou, H., et al.: Study on the target recognition and location technology of industrial sorting robot based on machine vision. J. Robot. Netw. Artif. Life 1(2), 108–110 (2014)

    Article  Google Scholar 

  5. Chen, Z.N., Zhang, X., Peng, Z.R., et al.: Workpiece location and recognition based on machine vision. Electron. Sci. Technol. 29(4), 99–103 (2016)

    Google Scholar 

  6. Huang, C., Chen, D., Tang, X.: Implementation of workpiece recognition and location based on opencv. In: 2015 8th International Symposium on Computational Intelligence and Design (ISCID), vol. 2, pp 228–232. IEEE (2015)

  7. Jinqiu, M., Tongshuai, Z., Zhiyu, Z.: An Approach for Picking T-shape workpiece based on monocular vision. In: 2018 3rd International Conference on Information Systems Engineering (ICISE), pp 1–5. IEEE (2018)

  8. Choi, C., Taguchi, Y., Tuzel, O., et al.: Voting-based pose estimation for robotic assembly using a 3D sensor. In: 2012 IEEE International Conference on Robotics and Automation, pp 1724–1731. IEEE (2012)

  9. Yang, L, Chong, M, Bai, C, et al.: A multi-workpieces recognition algorithm based on shape-SVM learning model. J. Phys. Conf. Series. IOP Publishing 1087(2), 022025 (2018)

    Article  Google Scholar 

  10. Fu, T., Li, F., Zheng, Y., et al.: Dynamically grasping with incomplete information workpiece based on machine vision. In: 2019 IEEE International Conference on Unmanned Systems (ICUS), pp 502–507. IEEE (2019)

  11. Zhao, Z.Q., Zheng, P., Xu, S., et al.: Object detection with deep learning: a review. IEEE Trans. Neural Netw. Learn. Syst. 30(11), 3212–3232 (2019)

    Article  Google Scholar 

  12. Wang, X., Liu, M., Raychaudhuri, D.S., et al.: Learning person Re-Identification models from videos with weak supervision. IEEE Trans. Image Process. 30, 3017–3028 (2021)

    Article  Google Scholar 

  13. Hu, H., Zhang, Z., Xie, Z., et al.: Local relation networks for image recognition. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 3464–3473 (2019)

  14. Jiang, W., Liu, M., Peng, Y., et al.: HDCB-Net: A neural network with the hybrid dilated convolution for pixel-level crack detection on concrete bridges. IEEE Trans. Indust. Inform. 17(8), 5485–5494 (2020)

    Article  Google Scholar 

  15. Li, C.H.G., Chang, Y.M.: Automated visual positioning and precision placement of a workpiece using deep learning. Int. J. Adv. Manufact. Technol. 104(9), 4527–4538 (2019)

    Article  Google Scholar 

  16. Lin, X., Wang, X., Li, L.: Intelligent detection of edge inconsistency for mechanical workpiece by machine vision with deep learning and variable geometry model. Appl. Intell. 50(7), 2105–2119 (2020)

    Article  Google Scholar 

  17. Redmon, J., Farhadi A.: Yolov3: An incremental improvement. arXiv:1804.02767 (2018)

  18. Kapoor, A., Singhal, A.: A comparative study of K-Means, K-Means++ and Fuzzy C-Means clustering algorithms. In: 2017 3rd International Conference on Computational Intelligence & Communication Technology (CICT), pp 1–6. IEEE (2017)

  19. Huang, G., Liu, Z., Van Der Maaten, L., et al.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4700–4708 (2017)

  20. Zhou, P., Ni, B., Geng, C., et al.: Scale-transferrable object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 528–537 (2018)

  21. He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 770–778 (2016)

  22. Agarap, A.F.: Deep learning using rectified linear units (relu). arXiv:1803.08375 (2018)

  23. Rezatofighi, H., Tsoi, N., Gwak, J.Y., et al.: Generalized intersection over union: a metric and a loss for bounding box regression. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 658–666 (2019)

  24. Ren, S., He, K., Girshick, R., et al.: Faster r-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Machine Intell. 39(6), 1137–1149 (2016)

    Article  Google Scholar 

  25. Liu, W., Anguelov, D., Erhan, D., et al.: Ssd: Single shot multibox detector. European conference on computer vision, pp 21–37. Springer, Cham (2016)

    Google Scholar 

  26. Tang, Y., Li, B., Liu, M., et al.: Autopedestrian: an automatic data augmentation and loss function search scheme for pedestrian detection. IEEE Transactions on Image Processing (2021)

  27. Girshick, R., Donahue, J., Darrell, T., et al.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 580–587 (2014)

  28. Girshick, R.: Fast r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision, pp 1440–1448 (2015)

  29. Redmon, J., Divvala, S., Girshick, R., et al.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 779–788 (2016)

  30. Redmon, J., Farhadi, A.: YOLO9000: Better, faster, stronger. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 7263–7271 (2017)

  31. Redmon, J., Farhadi A.: Yolov3: An incremental improvement. arXiv:1804.02767 (2018)

  32. Lin, T.Y., Goyal, P., Girshick, R., et al.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp 2980–2988 (2017)

  33. Wang, K, Ma, S, Chen, J, et al.: Approaches challenges and applications for deep visual odometry toward to complicated and emerging areas. IEEE Transactions on Cognitive and Developmental Systems (2020)

  34. Wang, K, Ma, S, Ren, F, et al.: SBAS: Salient bundle adjustment for visual SLAM. IEEE Trans. Instrum. Meas. 70, 1–9 (2021)

    Google Scholar 

Download references

Acknowledgements

The authors would like to acknowledge the support of the National Natural Science Foundation of China - Key Project 61733004, 62027810, 62076091 and 62133005.

Author information

Authors and Affiliations

  1. National Engineering laboratory for robot visual perception & control technology, Hunan University, Changsha, 410082, China

    Ke Zhao, Yaonan Wang & Chujin Zhang

  2. Hunan University of Finance and Economics, Changsha, 410082, China

    Yi Zuo

Authors
  1. Ke Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yaonan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yi Zuo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chujin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The overall study supervised by Yaonan Wang; Methodology, hardware, software, and preparing the original draft by Ke Zhao; Review and editing by Qing Zhu and Yi Zuo; The results were analyzed and validated by Chujin Zhang. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Ke Zhao.

Ethics declarations

Conflict of Interests

All the authors of this paper have no conflicts of interest, financial or otherwise.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, K., Wang, Y., Zuo, Y. et al. Palletizing Robot Positioning Bolt Detection Based on Improved YOLO-V3. J Intell Robot Syst 104, 41 (2022). https://doi.org/10.1007/s10846-022-01580-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10846-022-01580-w

Keywords

Search

Navigation