Abstract
The demand for 3D information in intelligent manufacturing makes complete point cloud of large workpiece increasingly important in the industrial field. However, due to the limited measurement range, the existing 3D reconstruction methods are diffcult to measure the large workpiece. The self-similar structure of workpiece also results in the low performance of existing 3D registration methods. To address the above problems, the point cloud registration system based on super-point space guidance is proposed by combining fringe projection profilometry (FPP) and point cloud registration technology to register multi-view point clouds of large workpiece. Specifically, to reduce the impact of self-similar structure on registration, we utilize spatial compatibility to partition the point clouds into local super-point space pairs, based on which to guide the multi-scale feature extraction network (MFENet) to mine effective super-point features, then the super-point features with high confidence is selected to estimate the optimal pose matrix. Experimental results show our registration error measured by standard ball is 0.024 mm, and the point cloud of large workpiece we measured reach the accuracy level of laser tracker. In addition, the registration recall of our system at higher accuracy thresholds is 95%, which demonstrates the high reliability of the method for accuracy-critical applications.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aoki, Y., Goforth, H., Srivatsan, R. A., & Lucey, S. (2019). Pointnetlk: Robust & efficient point cloud registration using pointnet. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 7163–7172). https://doi.org/10.1109/CVPR.2019.00733
Bai, X., Luo, Z., Zhou, L., Fu, H., Quan, L., & Tai, C. L. (2020). D3feat: Joint learning of dense detection and description of 3d local features. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 6359–6367). https://doi.org/10.1109/CVPR42600.2020.00639
Besl, P. J., & McKay, N. D. (1992). Method for registration of 3-D shapes. In Sensor fusion IV: Control paradigms and data structures (Vol. 1611, pp. 586–606). SPIE. https://doi.org/10.1117/12.57955
Bi, Z. M., & Wang, L. (2010). Advances in 3D data acquisition and processing for industrial applications. Robotics and Computer-Integrated Manufacturing, 26(5), 403–413. https://doi.org/10.1016/j.rcim.2010.03.003
Bo, L. I., Wei, T. I. A. N., Zhang, C., Fangfang, H. U. A., Guangyu, C. U. I., & Yufei, L. I. (2022). Positioning error compensation of an industrial robot using neural networks and experimental study. Chinese Journal of Aeronautics, 35(2), 346–360. https://doi.org/10.1016/j.cja.2021.03.027
Cao, X., Gong, X., Xie, Q., Huang, J., Xu, Y., Liu, Y., & Wang, J. (2022). Raw scanned point cloud registration with repetition for aircraft fuel tank inspection. Computer-Aided Design, 144, 103164. https://doi.org/10.1016/j.cad.2021.103164
Choy, C., Park, J., & Koltun, V. (2019). Fully convolutional geometric features. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 8958–8966). https://doi.org/10.1109/ICCV.2019.00905
Deng, H., Birdal, T., & Ilic, S. (2018). Ppfnet: Global context aware local features for robust 3d point matching. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 195–205). https://doi.org/10.1109/CVPR.2018.00028
Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6), 381–395. https://doi.org/10.1145/358669.358692
Geng, J. (2011). Structured-light 3D surface imaging: a tutorial. Advances in Optics and Photonics, 3(2), 128–160. https://doi.org/10.1364/AOP.3.000128
Gojcic, Z., Zhou, C., & Wieser, A. (2018). Learned compact local feature descriptor for TLS-based geodetic monitoring of natural outdoor scenes. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 4, 113–120. https://doi.org/10.5194/isprs-annals-IV-2-113-2018
Huang, S., Gojcic, Z., Usvyatsov, M., Wieser, A., & Schindler, K. (2021). Predator: Registration of 3d point clouds with low overlap. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 4267–4276). https://doi.org/10.1109/CVPR46437.2021.00425
Huang, X., Mei, G., Zhang, J., & Abbas, R. (2021). A comprehensive survey on point cloud registration. https://doi.org/10.48550/arXiv.2103.02690
Kriegel, S., Rink, C., Bodenmüller, T., & Suppa, M. (2015). Efficient next-best-scan planning for autonomous 3D surface reconstruction of unknown objects. Journal of Real-Time Image Processing, 10(4), 611–631. https://doi.org/10.1007/s11554-013-0386-6
Lee, J., Kim, S., Cho, M., & Park, J. (2021). Deep hough voting for robust global registration. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 15994–16003). https://doi.org/10.1109/ICCV48922.2021.01569
Li, F., Hitchens, C., & Stoddart, D. (2018). A performance evaluation method to compare the multi-view point cloud data registration based on ICP algorithm and reference marker. Journal of Modern Optics, 65(1), 30–37. https://doi.org/10.1080/09500340.2017.1375566
Li, G., Gan, Y., Liu, G., & Chen, F. (2022). High-accuracy point cloud registration for 3D shape measurement based on double constrained intersurface mutual projections. Measurement, 194, 111050. https://doi.org/10.1016/j.measurement.2022.111050
Liu, H., Liu, T., Li, Y., Xi, M., Li, T., & Wang, Y. (2019). Point cloud registration based on MCMC-SA ICP algorithm. IEEE Access, 7, 73637–73648. https://doi.org/10.1109/ACCESS.2019.2919989
Lowe, D. G. (1999, September). Object recognition from local scale-invariant features. In Proceedings of the seventh IEEE international conference on computer vision (Vol. 2, pp. 1150–1157). IEEE. https://doi.org/10.1109/ICCV.1999.790410
Lu, Z., Fan, J., Hou, Z., Deng, S., Zhou, C., & Jing, F. (2021). Automatic 3D seam extraction method for welding robot based on monocular structured light. IEEE Sensors Journal, 21(14), 16359–16370. https://doi.org/10.1109/JSEN.2021.3076341
Nguyen, C. H. P., & Choi, Y. (2018). Comparison of point cloud data and 3D CAD data for on-site dimensional inspection of industrial plant piping systems. Automation in Construction, 91, 44–52. https://doi.org/10.1016/j.autcon.2018.03.008
Ouyang, J., Liang, Z., Zhang, H., & Yan, Y. (2006). Research of measuring accuracy of laser tracker system. In Third international symposium on precision mechanical measurements (Vol. 6280, pp. 186-191). SPIE. https://doi.org/10.1117/12.716187
Pomerleau, F., Colas, F., & Siegwart, R. (2015). A review of point cloud registration algorithms for mobile robotics. Foundations and trends®in robotics, 4(1), 1–104. https://doi.org/10.1561/2300000035
Qi, C. R., Su, H., Mo, K., & Guibas, L. J. (2017). Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 652–660). https://doi.org/10.1109/CVPR.2017.16
Qian, X., Tu, J., & Lou, P. (2019). A general architecture of a 3D visualization system for shop floor management. Journal of Intelligent Manufacturing, 30(4), 1531–1545. https://doi.org/10.1007/s10845-017-1335-1
Quan, S., & Yang, J. (2020). Compatibility-guided sampling consensus for 3-d point cloud registration. IEEE Transactions on Geoscience and Remote Sensing, 58(10), 7380–7392. https://doi.org/10.1109/TGRS.2020.2982221
Radhakrishna, D., & Usha, S. (2018). Development of a robot-mounted 3D scanner and multi-view registration techniques for industrial applications. Procedia Computer Science, 133, 256–267. https://doi.org/10.1016/j.procs.2018.07.032
Rusu, R. B., Blodow, N., & Beetz, M. (2009). Fast point feature histograms (FPFH) for 3D registration. In 2009 IEEE international conference on robotics and automation (pp. 3212–3217). IEEE. https://doi.org/10.1109/ROBOT.2009.5152473
Rusu, R. B., Blodow, N., Marton, Z. C., & Beetz, M. (2008). Aligning point cloud views using persistent feature histograms. In 2008 IEEE/RSJ international conference on intelligent robots and systems (pp. 3384–3391). IEEE. https://doi.org/10.1109/IROS.2008.4650967
Servin, M., Garnica, G., Estrada, J. C., & Padilla, J. M. (2014). High-resolution low-noise 360-degree digital solid reconstruction using phase-stepping profilometry. Optics Express, 22(9), 10914–10922. https://doi.org/10.1364/OE.22.010914
Servin, M., Garnica, G., & Padilla, J. M. (2014). Co-phased 360-degree profilometry of discontinuous solids with 2-projectors and 1-camera. In Latin America optics and photonics conference (pp. LTh2B-2). Optica Publishing Group. https://doi.org/10.1364/LAOP.2014.LTh2B.2
Shao, W. J., Huang, Y., & Zhang, Y. (2018). A novel weld seam detection method for space weld seam of narrow butt joint in laser welding. Optics & Laser Technology, 99, 39–51. https://doi.org/10.1016/j.optlastec.2017.09.037
Sun, Y., Cheng, C., Zhang, Y., Zhang, C., Zheng, L., Wang, Z., & Wei, Y. (2020). Circle loss: A unified perspective of pair similarity optimization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 6398–6407). https://doi.org/10.1109/CVPR42600.2020.00643
Wang, X., Chen, T., Wang, Y., Zheng, D., Chen, X., & Zhao, Z. (2022). The 3D narrow butt weld seam detection system based on the binocular consistency correction. Journal of Intelligent Manufacturing,1–12. https://doi.org/10.1007/s10845-022-01927-y
Yang, J., Li, H., Campbell, D., & Jia, Y. (2015). Go-ICP: A globally optimal solution to 3D ICP point-set registration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(11), 2241–2254. https://doi.org/10.1109/TPAMI.2015.2513405
Yang, L., Li, E., Long, T., Fan, J., & Liang, Z. (2018). A novel 3-D path extraction method for arc welding robot based on stereo structured light sensor. IEEE Sensors Journal, 19(2), 763–773. https://doi.org/10.1109/JSEN.2018.2877976
Yang, L., Liu, Y., Peng, J., & Liang, Z. (2020). A novel system for off-line 3D seam extraction and path planning based on point cloud segmentation for arc welding robot. Robotics and Computer-Integrated Manufacturing, 64, 101929. https://doi.org/10.1016/j.rcim.2019.101929
Yang, L., Wang, B., Zhang, R., Zhou, H., & Wang, R. (2017). Analysis on location accuracy for the binocular stereo vision system. IEEE Photonics Journal, 10(1), 1–16. https://doi.org/10.1109/JPHOT.2017.2784958
Žagar, B. L., Yurtsever, E., Peters, A., & Knoll, A. C. (2022). Point cloud registration with object-centric alignment. IEEE Access, 10, 76586-76595. https://doi.org/10.1109/ACCESS.2022.3191352
Zeng, A., Song, S., Nießner, M., Fisher, M., Xiao, J., & Funkhouser, T. (2017). 3dmatch: Learning local geometric descriptors from RGB-D reconstructions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1802–1811). https://doi.org/10.1109/CVPR.2017.29
Zhang, B., Shi, Y., & Gu, S. (2019). Narrow-seam identification and deviation detection in keyhole deep-penetration TIG welding. The International Journal of Advanced Manufacturing Technology, 101(5), 2051–2064. https://doi.org/10.1007/s00170-018-3089-0
Zhang, K., Yan, M., Huang, T., Zheng, J., & Li, Z. (2019). 3D reconstruction of complex spatial weld seam for autonomous welding by laser structured light scanning. Journal of Manufacturing Processes, 39, 200–207. https://doi.org/10.1016/j.jmapro.2019.02.010
Zhang, L., Guo, J., Cheng, Z., Xiao, J., & Zhang, X. (2021). Efficient pairwise 3-D registration of urban scenes via hybrid structural descriptors. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–17. https://doi.org/10.1109/TGRS.2021.3091380
Zhang, L., Xu, Y., Du, S., Zhao, W., Hou, Z., & Chen, S. (2018). Point cloud based three-dimensional reconstruction and identification of initial welding position. In Transactions on intelligent welding manufacturing (pp. 61–77). Springer, Singapore. https://doi.org/10.1007/978-981-10-8330-3_4
Zhong, K., Li, Z., Zhou, X., Li, Y., Shi, Y., & Wang, C. (2015). Enhanced phase measurement profilometry for industrial 3D inspection automation. The International Journal of Advanced Manufacturing Technology, 76(9), 1563–1574. https://doi.org/10.1007/s00170-014-6360-z
Zhou, Q. Y., Park, J., & Koltun, V. (2016, October). Fast global registration. In European conference on computer vision (pp. 766–782). Springer, Cham. https://doi.org/10.1007/978-3-319-46475-6_47
Acknowledgements
This work was supported by the National Natural Science Foundation of China (Grant Nos. 61727802, 62101256) and the China Postdoctoral Science Foundation (2021M691591).
Author information
Authors and Affiliations
Corresponding authors
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Xingguo Wang and Xiaoyu Chen equally contributed to this work.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Wang, X., Chen, X., Zhao, Z. et al. High-precision point cloud registration system of multi-view industrial self-similar workpiece based on super-point space guidance. J Intell Manuf 35, 1765–1779 (2024). https://doi.org/10.1007/s10845-023-02136-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10845-023-02136-x