Skip to main content
SpringerLink
Log in

IPPE-PCR: a novel 6D pose estimation method based on point cloud repair for texture-less and occluded industrial parts

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

Abstract

Fast and accurate 6D pose estimation can help a robot arm grab industrial parts efficiently. The previous 6D pose estimation algorithms mostly target common items in daily life. Few algorithms are aimed at texture-less and occluded industrial parts and there are few industrial parts datasets. A novel method called the Industrial Parts 6D Pose Estimation framework based on point cloud repair (IPPE-PCR) is proposed in this paper. A synthetic dataset of industrial parts (SD-IP) is established as the training set for IPPE-PCR and an annotated real-world, low-texture and occluded dataset of industrial parts (LTO-IP) is constructed as the test set for IPPE. To improve the estimation accuracy, a new loss function is used for the point cloud repair network and an improved ICP method is proposed to optimize template matching. The experiment result shows that IPPE-PCR performs better than the state-of-the-art algorithms on LTO-IP.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  • Achlioptas, P., Diamanti, O., Mitliagkas, I., & Guibas, L. (2018, July). Learning representations and generative models for 3d point clouds. In International conference on machine learning (pp. 40–49). PMLR.

  • Balntas, V., Doumanoglou, A., Sahin, C., Sock, J., Kouskouridas, R., & Kim, T. K. (2017). Pose guided rgbd feature learning for 3d object pose estimation. In Proceedings of the IEEE international conference on computer vision (pp. 3856–3864).

  • Brachmann, E., Michel, F., Krull, A., Ying Yang, M., & Gumhold, S. (2016). Uncertainty-driven 6d pose estimation of objects and scenes from a single rgb image. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3364–3372).

  • Cao, Z., Sheikh, Y., & Banerjee, N. K. (2016, May). Real-time scalable 6DOF pose estimation for textureless objects. In 2016 IEEE International conference on Robotics and Automation (ICRA) (pp. 2441–2448). IEEE.

  • Chen, L. C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018). Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision (ECCV) (pp. 801–818).

  • Collet, A., Martinez, M., & Srinivasa, S. S. (2011). The MOPED framework: Object recognition and pose estimation for manipulation. The International Journal of Robotics Research, 30(10), 1284–1306.

    Article  Google Scholar 

  • Drost, B., & Ilic, S. (2012, October). 3d object detection and localization using multimodal point pair features. In 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission (pp. 9–16). IEEE.

  • Hagelskjær, F., & Buch, A. G. (2020, October). Pointvotenet: Accurate Object Detection And 6 DOF Pose Estimation In Point Clouds. In 2020 IEEE International Conference on Image Processing (ICIP) (pp. 2641–2645). IEEE.

  • He, X., Zhou, Y., Zhou, Z., Bai, S., & Bai, X. (2018). Triplet-center loss for multi-view 3d object retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1945–1954).

  • Hinterstoisser, S., Cagniart, C., Ilic, S., Sturm, P., Navab, N., Fua, P., & Lepetit, V. (2011). Gradient response maps for real-time detection of textureless objects. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(5), 876–888.

    Article  Google Scholar 

  • Hodaň, T., Zabulis, X., Lourakis, M., Obdržálek, Š., & Matas, J. (2015, September). Detection and fine 3D pose estimation of texture-less objects in RGB-D images. In 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 4421–4428). IEEE.

  • Huang, Z., Yu, Y., Xu, J., Ni, F., & Le, X. (2020). PF-Net: Point Fractal Network for 3D Point Cloud Completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 7662–7670).

  • Kakish, J., Zhang, P. L., & Zeid, I. (2000). Towards the design and development of a knowledge-based universal modular jigs and fixtures system. Journal of Intelligent Manufacturing, 11(4), 381–401.

    Article  Google Scholar 

  • Kong, X. T., Luo, H., Huang, G. Q., & Yang, X. (2019). Industrial wearable system: the human-centric empowering technology in Industry 4.0. Journal of Intelligent Manufacturing, 30(8), 2853–2869.

    Article  Google Scholar 

  • Krull, A., Brachmann, E., Michel, F., Ying Yang, M., Gumhold, S., & Rother, C. (2015). Learning analysis-by-synthesis for 6D pose estimation in RGB-D images. In Proceedings of the IEEE international conference on computer vision (pp. 954–962).

  • Long, J., Shelhamer, E., & Darrell, T. (2015). Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3431–3440).

  • Marion, P., Florence, P. R., Manuelli, L., & Tedrake, R. (2018, May). Label fusion: A pipeline for generating ground truth labels for real rgbd data of cluttered scenes. In 2018 IEEE International Conference on Robotics and Automation (ICRA) (pp. 1–8). IEEE.

  • Mur-Artal, R., Montiel, J. M. M., & Tardos, J. D. (2015). ORB-SLAM: A versatile and accurate monocular SLAM system. IEEE Transactions on Robotics, 31(5), 1147–1163.

    Article  Google Scholar 

  • Oztemel, E., & Gursev, S. (2020). Literature review of Industry 4.0 and related technologies. Journal of Intelligent Manufacturing, 31(1), 127–182.

    Article  Google Scholar 

  • Park, K., Patten, T., Prankl, J., & Vincze, M. (2019, May). Multi-task template matching for object detection, segmentation and pose estimation using depth images. In 2019 International Conference on Robotics and Automation (ICRA) (pp. 7207–7213). IEEE.

  • Pavlakos, G., Zhou, X., Chan, A., Derpanis, K. G., & Daniilidis, K. (2017, May). 6-dof object pose from semantic keypoints. In 2017 IEEE international conference on robotics and automation (ICRA) (pp. 2011–2018). IEEE.

  • Peng, S., Liu, Y., Huang, Q., Zhou, X., & Bao, H. (2019). Pvnet: Pixel-wise voting network for 6dof pose estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 4561–4570).

  • Qi, C. R., Su, H., Mo, K., & Guibas, L. J. (2017). Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 652–660).

  • Ramachandram, D., & Rajeswari, M. (2004). Neural network-based robot visual positioning for intelligent assembly. Journal of Intelligent Manufacturing, 15(2), 219–231.

    Article  Google Scholar 

  • Sarmad, M., Lee, H. J., & Kim, Y. M. (2019). Rl-gan-net: A reinforcement learning agent controlled gan network for real-time point cloud shape completion. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 5898–5907).

  • Sundermeyer, M., Marton, Z. C., Durner, M., Brucker, M., & Triebel, R. (2018). Implicit 3d orientation learning for 6d object detection from rgb images. In Proceedings of the European Conference on Computer Vision (ECCV) (pp. 699–715).

  • Tekin, B., Sinha, S. N., & Fua, P. (2018). Real-time seamless single shot 6d object pose prediction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 292–301).

  • Tulsiani, S., & Malik, J. (2015). Viewpoints and keypoints. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1510–1519).

  • Wang, C., Xu, D., Zhu, Y., Martín-Martín, R., Lu, C., Fei-Fei, L., & Savarese, S. (2019b). Densefusion: 6d object pose estimation by iterative dense fusion. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 3343–3352).

  • Wang, H., Sridhar, S., Huang, J., Valentin, J., Song, S., & Guibas, L. J. (2019a). Normalized object coordinate space for category-level 6d object pose and size estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 2642–2651).

  • Wang, Z., Xu, Y., He, Q., Fang, Z., Xu, G., & Fu, J. (2020). Grasping pose estimation for SCARA robot based on deep learning of point cloud. The International Journal of Advanced Manufacturing Technology, 108(4), 1217–1231.

    Article  Google Scholar 

  • Xiang, Y., Schmidt, T., Narayanan, V., & Fox, D. (2017). Posecnn: A convolutional neural network for 6d object pose estimation in cluttered scenes. arXiv preprint arXiv:1711.00199.

Download references

Acknowledgements

This research is supported by the Ministry of Education-China Mobile Research Fund Construction Project (No. MCM20180703) and the Shanghai Science and Technology Innovation Action Plan (No. 20511106200).

Author information

Authors and Affiliations

  1. School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China

    Wei Qin, Qing Hu, Zilong Zhuang & Haozhe Huang

  2. China Mobile (Shanghai) ICT Co. Ltd, Shanghai, China

    Xiaodan Zhu & Lin Han

Authors
  1. Wei Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qing Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zilong Zhuang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Haozhe Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaodan Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lin Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Wei Qin or Zilong Zhuang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qin, W., Hu, Q., Zhuang, Z. et al. IPPE-PCR: a novel 6D pose estimation method based on point cloud repair for texture-less and occluded industrial parts. J Intell Manuf 34, 2797–2807 (2023). https://doi.org/10.1007/s10845-022-01965-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10845-022-01965-6

Keywords

Search

Navigation