Skip to main content

Correspondence-Free SE(3) Point Cloud Registration in RKHS via Unsupervised Equivariant Learning

  • Conference paper
  • First Online:
Computer Vision – ECCV 2024 (ECCV 2024)

Abstract

This paper introduces a robust unsupervised SE(3) point cloud registration method that operates without requiring point correspondences. The method frames point clouds as functions in a reproducing kernel Hilbert space (RKHS), leveraging SE(3)-equivariant features for direct feature space registration. A novel RKHS distance metric is proposed, offering reliable performance amidst noise, outliers, and asymmetrical data. An unsupervised training approach is introduced to effectively handle limited ground truth data, facilitating adaptation to real datasets. The proposed method outperforms classical and supervised methods in terms of registration accuracy on both synthetic (ModelNet40) and real-world (ETH3D) noisy, outlier-rich datasets. To our best knowledge, this marks the first instance of successful real RGB-D odometry data registration using an equivariant method. The code is available at https://sites.google.com/view/eccv24-equivalign.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bai, X., Luo, Z., Zhou, L., Fu, H., Quan, L., Tai, C.L.: D3Feat: joint learning of dense detection and description of 3D local features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6359–6367 (2020)

    Google Scholar 

  2. Berlinet, A., Thomas-Agnan, C.: Reproducing Kernel Hilbert Spaces in Probability and Statistics. Springer US, Boston, MA (2004). https://doi.org/10.1007/978-1-4419-9096-9

    Book  Google Scholar 

  3. Besl, P.J., McKay, N.D.: A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992). https://doi.org/10.1109/34.121791

    Article  Google Scholar 

  4. Biber, P., Fleck, S., Strasser, W.: A probabilistic framework for robust and accurate matching of point clouds. In: Rasmussen, C.E., Bülthoff, H.H., Schölkopf, B., Giese, M.A. (eds.) Pattern Recognition, pp. 480–487. Springer Berlin Heidelberg, Berlin, Heidelberg (2004). https://doi.org/10.1007/978-3-540-28649-3_59

    Chapter  Google Scholar 

  5. Bishop, C.M.: Pattern Recognition and Machine Learning. Springer (2006)

    Google Scholar 

  6. Campbell, D., Petersson, L.: An adaptive data representation for robust point-set registration and merging. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4292–4300 (2015)

    Google Scholar 

  7. Chen, H., Liu, S., Chen, W., Li, H., Hill, R.: Equivariant point network for 3D point cloud analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14514–14523 (2021)

    Google Scholar 

  8. Chen, X., Milioto, A., Palazzolo, E., Giguere, P., Behley, J., Stachniss, C.: Suma++: efficient lidar-based semantic slam. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4530–4537. IEEE (2019)

    Google Scholar 

  9. Chen, Y., Medioni, G.G.: Object modeling by registration of multiple range images. Image Vis. Comput. 10(3), 145–155 (1992)

    Article  Google Scholar 

  10. Choy, C., Park, J., Koltun, V.: Fully convolutional geometric features. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 8958–8966 (2019)

    Google Scholar 

  11. Choy, C., Park, J., Koltun, V.: Fully convolutional geometric features. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 8958–8966 (2019)

    Google Scholar 

  12. Chui, H., Rangarajan, A.: A feature registration framework using mixture models. In: Proceedings IEEE Workshop on Mathematical Methods in Biomedical Image Analysis, pp. 190–197. IEEE (2000)

    Google Scholar 

  13. Clark, W., Ghaffari, M., Bloch, A.: Nonparametric continuous sensor registration. J. Mach. Learn. Res. 22(271), 1–50 (2021)

    MathSciNet  Google Scholar 

  14. Cohen, T., Weiler, M., Kicanaoglu, B., Welling, M.: Gauge equivariant convolutional networks and the icosahedral CNN. In: Proceedings of the International Conference on Machine Learning, pp. 1321–1330. PMLR (2019)

    Google Scholar 

  15. Cohen, T., Welling, M.: Group equivariant convolutional networks. In: Proceedings of the International Conference on Machine Learning, pp. 2990–2999. PMLR (2016)

    Google Scholar 

  16. Cohen, T.S., Geiger, M., Köhler, J., Welling, M.: Spherical CNNs. arXiv preprint arXiv:1801.10130 (2018)

  17. Cohen, T.S., Welling, M.: Steerable CNNs. arXiv preprint arXiv:1612.08498 (2016)

  18. Deng, C., Litany, O., Duan, Y., Poulenard, A., Tagliasacchi, A., Guibas, L.J.: Vector Neurons: a general framework for SO(3)-equivariant networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 12200–12209 (2021)

    Google Scholar 

  19. Deng, H., Birdal, T., Ilic, S.: PPFNet: global context aware local features for robust 3D point matching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 195–205 (2018)

    Google Scholar 

  20. Eckart, B., Kim, K., Kautz, J.: HGMR: hierarchical Gaussian mixtures for adaptive 3D registration. In: Proceedings of the European Conference on Computer Vision, pp. 705–721 (2018)

    Google Scholar 

  21. Evangelidis, G.D., Horaud, R.: Joint alignment of multiple point sets with batch and incremental expectation-maximization. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1397–1410 (2017)

    Article  Google Scholar 

  22. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  23. Fuchs, F., Worrall, D., Fischer, V., Welling, M.: SE(3)-transformers: 3D roto-translation equivariant attention networks. In: Proceedings of Advances Neural Information Processing Systems Conference, vol. 33, pp. 1970–1981 (2020)

    Google Scholar 

  24. Ghaffari, M., Clark, W., Bloch, A., Eustice, R.M., Grizzle, J.W.: Continuous direct sparse visual odometry from RGB-D images. In: Proceedings of Robotics: Science and System Conference, Freiburg, Germany (2019)

    Google Scholar 

  25. Ghaffari, M., Clark, W., Bloch, A., Eustice, R.M., Grizzle, J.W.: Continuous direct sparse visual odometry from RGB-D images. arXiv preprint arXiv:1904.02266 (2019)

  26. Horaud, R., Forbes, F., Yguel, M., Dewaele, G., Zhang, J.: Rigid and articulated point registration with expectation conditional maximization. IEEE Trans. Pattern Anal. Mach. Intell. 33(3), 587–602 (2010)

    Article  Google Scholar 

  27. Huang, S., Gojcic, Z., Usvyatsov, M., Wieser, A., Schindler, K.: Predator: registration of 3D point clouds with low overlap. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4267–4276 (2021)

    Google Scholar 

  28. Jian, B., Vemuri, B.C.: Robust point set registration using Gaussian mixture models. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1633–1645 (2011)

    Article  Google Scholar 

  29. Kerl, C.: Dense Visual Odometry (DVO). https://github.com/tum-vision/dvo (2013)

  30. Li, F., Fujiwara, K., Okura, F., Matsushita, Y.: Generalized shuffled linear regression. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6474–6483 (2021)

    Google Scholar 

  31. Li, Y., Bu, R., Sun, M., Wu, W., Di, X., Chen, B.: PointCNN: convolution on X-transformed points. In: Proceedings of Advances in Neural Information Processing Systems Conference, pp. 820–830 (2018)

    Google Scholar 

  32. Lin, T.Y., Clark, W., Eustice, R.M., Grizzle, J.W., Bloch, A., Ghaffari, M.: Adaptive continuous visual odometry from RGB-D images. arXiv preprint arXiv:1910.00713 (2019)

  33. MacDonald, L.E., Ramasinghe, S., Lucey, S.: Enabling equivariance for arbitrary lie groups. In: Proceedings of the IEEE Conference on Computer Vision Pattern Recognition, pp. 8183–8192 (2022)

    Google Scholar 

  34. Magnusson, M., Lilienthal, A., Duckett, T.: Scan registration for autonomous mining vehicles using 3D-NDT. J. Field Robot. 24(10), 803–827 (2007)

    Article  Google Scholar 

  35. Mitra, N.J., Gelfand, N., Pottmann, H., Guibas, L.: Registration of point cloud data from a geometric optimization perspective. In: Proceedings of the 2004 Eurographics/ACM SIGGRAPH Symposium on Geometry Processing, pp. 22–31. SGP ’04, Association for Computing Machinery, New York, NY, USA (2004). https://doi.org/10.1145/1057432.1057435

  36. Park, J., Zhou, Q.Y., Koltun, V.: Colored point cloud registration revisited. In: Proceedings of the IEEE Conference on Computer Vision, pp. 143–152 (2017)

    Google Scholar 

  37. Parkison, S.A., Gan, L., Jadidi, M.G., Eustice, R.M.: Semantic iterative closest point through expectation-maximization. In: Proceedings of British Machine Vision Conference, p. 280 (2018)

    Google Scholar 

  38. Parkison, S.A., Ghaffari, M., Gan, L., Zhang, R., Ushani, A.K., Eustice, R.M.: Boosting shape registration algorithms via reproducing kernel Hilbert space regularizers. IEEE Robot. Autom. Lett. 4(4), 4563–4570 (2019)

    Article  Google Scholar 

  39. Paszke, A., et al.: Automatic differentiation in PyTorch (2017)

    Google Scholar 

  40. Pineda, L., et al.: Theseus: a library for differentiable nonlinear optimization. In: Proceedings of Advances Neural Information Processing Systems Conference, vol. 35, pp. 3801–3818 (2022)

    Google Scholar 

  41. Qi, C.R., Su, H., Mo, K., Guibas, L.J.: PointNet: deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 652–660 (2017)

    Google Scholar 

  42. Qin, Z., et al.: GeoTransformer: fast and robust point cloud registration with geometric transformer. IEEE Trans. Pattern Anal. Mach. Intell. (2023)

    Google Scholar 

  43. Rasmussen, C., Williams, C.: Gaussian processes for machine learning, vol. 1. MIT Press (2006)

    Google Scholar 

  44. Rusu, R.B., Blodow, N., Beetz, M.: Fast point feature histograms (FPFH) for 3D registration. In: 2009 IEEE International Conference on Robotics and Automation, pp. 3212–3217. IEEE (2009)

    Google Scholar 

  45. Satorras, V.G., Hoogeboom, E., Welling, M.: E(n) equivariant graph neural networks. In: Proceedings of the International Conference on Machine Learning, pp. 9323–9332. PMLR (2021)

    Google Scholar 

  46. Schops, T., Sattler, T., Pollefeys, M.: Bad SLAM: bundle adjusted direct RGB-D SLAM. In: Proceedings of the IEEE Conference on Computer Vision Pattern Recognition, pp. 134–144 (2019)

    Google Scholar 

  47. Segal, A., Haehnel, D., Thrun, S.: Generalized-ICP. In: Robotics: Science and Systems, vol. 2, issue 4, p. 435. Seattle, WA (2009)

    Google Scholar 

  48. Servos, J., Waslander, S.L.: Multi channel generalized-ICP. In: Proceedings of International Conference on Robotics and Automation, pp. 3644–3649. IEEE (2014)

    Google Scholar 

  49. Thomas, H., Qi, C.R., Deschaud, J.E., Marcotegui, B., Goulette, F., Guibas, L.J.: Kpconv: Flexible and deformable convolution for point clouds. In: Proceedings of the IEEE International Conference Computer Vision, pp. 6411–6420 (2019)

    Google Scholar 

  50. Thomas, N., et al.: Tensor field networks: rotation-and translation-equivariant neural networks for 3D point clouds. arXiv preprint arXiv:1802.08219 (2018)

  51. Tsin, Y., Kanade, T.: A correlation-based approach to robust point set registration. In: Pajdla, T., Matas, J. (eds.) Computer Vision - ECCV 2004: 8th European Conference on Computer Vision, Prague, Czech Republic, May 11-14, 2004. Proceedings, Part III, pp. 558–569. Springer, Berlin, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24672-5_44

    Chapter  Google Scholar 

  52. Wang, C., et al.: PyPose: a library for robot learning with physics-based optimization. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2023)

    Google Scholar 

  53. Wang, Y., Solomon, J.M.: Deep closest point: Learning representations for point cloud registration. In: Proceedings of the IEEE International Conference Computer Vision, pp. 3523–3532 (2019)

    Google Scholar 

  54. Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M., Solomon, J.M.: Dynamic graph CNN for learning on point clouds. ACM Trans. Graph. (TOG) (2019)

    Google Scholar 

  55. Weiler, M., Geiger, M., Welling, M., Boomsma, W., Cohen, T.S.: 3D steerable CNNs: learning rotationally equivariant features in volumetric data. In: Proceedings of the Advances in Neural Information Processing Systems Conference, vol. 31 (2018)

    Google Scholar 

  56. Whelan, T., Salas-Moreno, R.F., Glocker, B., Davison, A.J., Leutenegger, S.: ElasticFusion: real-time dense slam and light source estimation. Int. J. Robot. Res. 35(14), 1697–1716 (2016)

    Article  Google Scholar 

  57. Wu, Z., et al.: 3D ShapeNets: a deep representation for volumetric shapes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1912–1920 (2015)

    Google Scholar 

  58. Yang, H., Shi, J., Carlone, L.: TEASER: fast and certifiable point cloud registration. IEEE Trans. Rob. 37(2), 314–333 (2020)

    Article  Google Scholar 

  59. Yu, H., Li, F., Saleh, M., Busam, B., Ilic, S.: CoFiNet: reliable coarse-to-fine correspondences for robust pointcloud registration. In: Proceedings of the Advances in Neural Information Processing Systems Conference, vol. 34, 23872–23884 (2021)

    Google Scholar 

  60. Zeng, A., Song, S., Nießner, M., Fisher, M., Xiao, J., Funkhouser, T.: 3DMatch: learning local geometric descriptors from RGB-D reconstructions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1802–1811 (2017)

    Google Scholar 

  61. Zhang, R., et al.: A new framework for registration of semantic point clouds from stereo and RGB-D cameras. In: Proceedings of the IEEE International Conference Robotics and Automation, pp. 12214–12221 (2020)

    Google Scholar 

  62. Zhou, Q.-Y., Park, J., Koltun, V.: Fast global registration. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) Computer Vision – ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II, pp. 766–782. Springer International Publishing, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_47

    Chapter  Google Scholar 

  63. Zhou, Q.Y., Park, J., Koltun, V.: Open3D: a modern library for 3D data processing. arXiv preprint arXiv:1801.09847 (2018)

  64. Zhu, M., Ghaffari, M., Clark, W.A., Peng, H.: E2PN: efficient SE(3)-equivariant point network. In: Proceedings of the IEEE conference on Computer Vision Pattern Recognition, pp. 1223–1232 (2023)

    Google Scholar 

  65. Zhu, M., Ghaffari, M., Peng, H.: Correspondence-free point cloud registration with SO(3)-equivariant implicit shape representations. In: Conference on Robot Learning, pp. 1412–1422. PMLR (2022)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ray Zhang .

Editor information

Editors and Affiliations

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 2679 KB)

Supplementary material 2 (mp4 27631 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, R. et al. (2025). Correspondence-Free SE(3) Point Cloud Registration in RKHS via Unsupervised Equivariant Learning. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15146. Springer, Cham. https://doi.org/10.1007/978-3-031-73223-2_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-73223-2_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-73222-5

  • Online ISBN: 978-3-031-73223-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics