Skip to main content
SpringerLink
Log in

Two-view point cloud registration network: feature and geometry

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Rigid point cloud registration is a crucial upstream task in computer vision, whose goal is to align two misaligned point clouds using a rigid transformation. Existing methods, which directly utilize extracted point features for computing point relationships, are likely to result in a wrong-matching relationship since two or more similar feature points in the source point cloud easily correspond to the same point in the target point cloud. To this end, this paper proposes a two-view point cloud registration network that better alleviates the problem of similar feature points from both the feature and geometry levels. Specifically, at the feature level, a residual correction unit is proposed to learn feature-aware coefficients from raw 3D point clouds to adaptively increase or decrease the difference between features. An attention mechanism is established in two-point clouds to capture the implicit feature relationship between the two-point clouds, gathering the information of another point cloud to enrich the feature information of its own point cloud. Second, at the geometry level, a dual-view graph topology fusion module is described. All points in the graph structure are no longer independent but connected by their geometry structure. Therefore, each point can aggregate neighbor information in the same point cloud and in different point clouds through the constructed single graph and interactive graph, so that each point can enhance the difference between points through its geometry structure. In order to fuse the information of a single graph and an interactive graph, a cross-attention module is proposed to supplement contextual information to obtain point features that are more suitable for matching. Experimental results demonstrate that our method achieves excellent results on complete and partial noisy point clouds.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data Availability

The data generated and analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Cattaneo D, Vaghi M, Valada A (2022) LCDNet: deep loop closure detection and point cloud registration for LiDAR SLAM. IEEE Trans Robot 38(4):2074–2093

    Article  Google Scholar 

  2. Yang T, Ye J, Zhou S, Xu A, Yin J (2022) 3D reconstruction method for tree seedlings based on point cloud self-registration. Comput Electron Agric 200:107210

    Article  Google Scholar 

  3. Islam KT, Wijewickrema S, O’Leary S (2021) A deep learning based framework for the registration of three dimensional multi-modal medical images of the head. Sci Rep 11(1):1860

  4. Li Y, Ma L, Zhong Z, Liu F, Chapman MA, Cao D, Li J (2020) Deep learning for lidar point clouds in autonomous driving: a review. IEEE Trans Neural Netw Learn Syst 32(8):3412–3432

    Article  Google Scholar 

  5. Cui Y, Chen R, Chu W, Chen L, Tian D, Li Y, Cao D (2021) Deep learning for image and point cloud fusion in autonomous driving: a review. IEEE Trans Intell Transp Syst 23(2):722–739

    Article  Google Scholar 

  6. Qi CR, Su H, Mo K, Guibas LJ (2017) PointNet: Deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, hawaii, USA, pp 652–660

  7. Qi CR, Yi L, Su H, Guibas LJ (2017) PointNet++: deep hierarchical feature learning on point sets in a metric space. In: Advances in neural information processing systems, California, USA, vol 30

  8. Wang Y, Sun Y, Liu Z, Sarma SE, Bronstein MM, Solomon JM (2019) Dynamic graph CNN for learning on point clouds. ACM Trans Graph 38(5):1–12

    Article  Google Scholar 

  9. Yew ZJ, Lee GH (2020) RPM-Net: robust point matching using learned features. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Washington, USA, pp 11824–11833

  10. Fu K, Luo J, Luo X, Liu S, Zhang C, Wang M (2023) Robust point cloud registration framework based on deep graph matching. IEEE Trans Pattern Anal Mach Intell 45(05):6183–6195

    Google Scholar 

  11. Zhang Z, Sun J, Dai Y, Zhou D, Song X, He M (2022) End-to-end learning the partial permutation matrix for robust 3D point cloud registration. In: Proceedings of the AAAI conference on artificial intelligence. Ottawa, Canada vol 36, pp 3399–3407

  12. Veličković P (2023) Everything is connected: graph neural networks. Curr Opin Struct Biol 79:102538

    Article  Google Scholar 

  13. Wu Z, Pan S, Chen F, Long G, Zhang C, Philip SY (2020) A comprehensive survey on graph neural networks. IEEE Trans Neural Netw Learn Syst 32(1):4–24

    Article  MathSciNet  Google Scholar 

  14. Wu L, Chen Y, Ji H, Liu B (2021) Deep learning on graphs for natural language processing. In: Proceedings of the international ACM SIGIR conference on research and development in information retrieval, online, pp 2651–2653

  15. Wu S, Sun F, Zhang W, Xie X, Cui B (2020) Graph neural networks in recommender systems: a survey. ACM Computing Surveys

  16. Fischer K, Simon M, Milz S, Mäder P (2022) StickyLocalization: robust end-to-end relocalization on point clouds using graph neural networks. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision. Hawaii, USA, pp 2962–2971

  17. Shi C, Chen X, Huang K, Xiao J, Lu H, Stachniss C (2021) Keypoint matching for point cloud registration using multiplex dynamic graph attention networks. IEEE Robot Autom Lett 6(4):8221–8228

    Article  Google Scholar 

  18. Wang R, Yan J, Yang X (2019) Learning combinatorial embedding networks for deep graph matching. In: Proceedings of the IEEE/CVF international conference on computer vision. Seoul, Korea, pp 3056–3065

  19. Besl PJ, McKay ND (1992) A method for registration of 3D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence pp 239–256

  20. Bouaziz S, Tagliasacchi A, Pauly M (2013) Sparse iterative closest point. Comput Graph Forum 32(5):113–123

    Article  Google Scholar 

  21. Segal A, Haehnel D, Thrun S (2009) Generalized-ICP. In: Robotics: science and systems, seattle. WA, Washington, USA, vol 2, pp 435

  22. Aiger D, Mitra NJ, Cohen-Or D (2008) 4-points congruent sets for robust pairwise surface registration. ACM Trans Graph 27(3):10

    Article  Google Scholar 

  23. Mellado N, Aiger D, Mitra NJ (2014) Super 4PCS fast global pointcloud registration via smart indexing. Comput Graph Forum 33(5):205–215

    Article  Google Scholar 

  24. Magnusson M, Lilienthal A, Duckett T (2007) Scan registration for autonomous mining vehicles using 3D-NDT. J Field Robot 24(10):803–827

    Article  Google Scholar 

  25. Das A, Waslander SL (2012) Scan registration with multi-scale k-means normal distributions transform. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems. Algarve, Portugal, pp 2705–2710

  26. Aoki Y, Goforth H, Srivatsan RA, Lucey S (2019) PointNetLK: robust & efficient point cloud registration using PointNet. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. California, USA, pp 7163–7172

  27. Yuan Y, Borrmann D, Hou J, Ma Y, Nüchter A, Schwertfeger S (2021) Self-supervised point set local descriptors for point cloud registration. Sensors 21(2):486

    Article  Google Scholar 

  28. Wang Y, Solomon JM (2019a) Deep closest point: learning representations for point cloud registration. In: Proceedings of the IEEE/CVF international conference on computer vision. Seoul, Korea, pp 3523–3532

  29. Wang Y, Solomon JM (2019b) PRNet: self-supervised learning for partial-to-partial registration. In: Advances in neural information processing systems. Vancouver, Canada, vol 32

  30. Xu H, Ye N, Liu G, Zeng B, Liu S (2022) FINet: dual branches feature interaction for partial-to-partial point cloud registration. In: Proceedings of the AAAI conference on artificial intelligence, online vol 36, pp 2848–2856

  31. Yew ZJ, Lee GH (2022) RegTR: end-to-end point cloud correspondences with transformers. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. Louisiana, USA, pp 6677–6686

  32. Fischer K, Simon M, Olsner F, Milz S, Gross HM, Mader P (2021) StickyPillars: robust and efficient feature matching on point clouds using graph neural networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, online, pp 313–323

  33. Yuan M, Li Z, Jin Q, Chen X, Wang M (2022) PointCLM: a contrastive learning-based framework for multi-instance point cloud registration. In: Proceedings of the european conference on computer vision. Springer, pp 595–611

  34. Kenton JDMWC, Toutanova LK (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the North American chapter of the association for computational linguistics. Minnesota, USA, pp 4171–4186

  35. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems. California, USA, vol 30

  36. Lin ZH, Huang SY, Wang YCF (2020) Convolution in the cloud: learning deformable kernels in 3D graph convolution networks for point cloud analysis. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. Washington, USA, pp 1800–1809

  37. Qian G, Abualshour A, Li G, Thabet A, Ghanem B (2021) PU-GCN: point cloud upsampling using graph convolutional networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, online, pp 11683–11692

  38. Du Y, Pei B, Zhao X, Ji J (2020) Deep scaled dot-product attention based domain adaptation model for biomedical question answering. Methods 173:69–74

    Article  Google Scholar 

  39. Liu W, Liu Z, Rehg JM, Song L (2019) Neural similarity learning. In: Advances in neural information processing systems. Vancouver, Canada, vol 32

  40. Welling M, Kipf TN (2017) Semi-supervised classification with graph convolutional networks. In: Proceedings of the international conference on learning representations. Var, France

  41. Sinkhorn R (1964) A relationship between arbitrary positive matrices and doubly stochastic matrices. Ann Math Stat 35(2):876–879

    Article  MathSciNet  Google Scholar 

  42. Mena G, Belanger D, Linderman S, Snoek J (2018) Learning latent permutations with Gumbel-Sinkhorn networks. In: Proceedings of the international conference on learning representations. Vancouver, Canada

  43. Wu Z, Song S, Khosla A, Yu F, Zhang L, Tang X, Xiao J (2015) 3D ShapeNets: a deep representation for volumetric shapes. In: Proceedings of the IEEE conference on computer vision and pattern recognition. Massachusetts, USA, pp 1912–1920

  44. Li J, Zhang C, Xu Z, Zhou H, Zhang C (2020) Iterative distance-aware similarity matrix convolution with mutual-supervised point elimination for efficient point cloud registration. In: Proceedings of the european conference on computer vision. Springer, Glasgow, UK, pp 378–394

  45. Hezroni I, Drory A, Giryes R, Avidan S (2021) DeepBBS: deep best buddies for point cloud registration. In: Proceedings of the international conference on 3D vision. IEEE, Fujian, China, pp 342–351

  46. Huang S, Gojcic Z, Usvyatsov M, Wieser A, Schindler K (2021) PREDATOR: registration of 3D point clouds with low overlap. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. online, pp 4267–4276

  47. Wang H, Liu X, Kang W, Yan Z, Wang B, Ning Q (2022) Multi-features guidance network for partial-to-partial point cloud registration. Neural Comput Appl 34(2):1623–1634

    Article  Google Scholar 

  48. Mei G, Poiesi F, Saltori C, Zhang J, Ricci E, Sebe N (2023) Overlap-guided gaussian mixture models for point cloud registration. In: Proceedings of the IEEE/CVF Winter conference on applications of computer vision. Waikoloa, USA, pp 4511–4520

  49. Qin Z, Yu H, Wang C, Guo Y, Peng Y, Ilic S, Hu D, Xu K (2023) GeoTransformer: fast and robust point cloud registration with geometric transformer. IEEE Trans Pattern Anal Mach Intell 45(8):9806–9821

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grant 62176244, Grant 62032022, and Grant 62006215; in part by the Zhejiang Provincial Natural Science Foundation under Grant LQ23F020012; and in part by the Fundamental Research Funds for the Provincial Universities of Zhejiang under Grant 2022YW65.

Author information

Authors and Affiliations

  1. College of Sciences, China Jiliang University, Hangzhou, 310018, China

    Lingpeng Wang, Bing Yang, Hailiang Ye & Feilong Cao

Authors
  1. Lingpeng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bing Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hailiang Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Feilong Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Lingpeng Wang: Writing original draft, Conceptualization, Methodology, Formal analysis and investigation. Bing Yang: Methodology, Formal analysis and investigation, Funding acquisition. Hailiang Ye: Writing - review and editing, Funding acquisition. Feilong Cao: Supervision, Resources, Conceptualization, Writing - review and editing, Funding acquisition.

Corresponding author

Correspondence to Feilong Cao.

Ethics declarations

Competing of interest

The authors have no competing interests to declare that are relevant to the content of this article.

Compliance with Ethical Standards

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, L., Yang, B., Ye, H. et al. Two-view point cloud registration network: feature and geometry. Appl Intell 54, 3135–3151 (2024). https://doi.org/10.1007/s10489-023-05263-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-05263-3

Keywords

Search

Navigation