Abstract
This paper introduces a robust unsupervised SE(3) point cloud registration method that operates without requiring point correspondences. The method frames point clouds as functions in a reproducing kernel Hilbert space (RKHS), leveraging SE(3)-equivariant features for direct feature space registration. A novel RKHS distance metric is proposed, offering reliable performance amidst noise, outliers, and asymmetrical data. An unsupervised training approach is introduced to effectively handle limited ground truth data, facilitating adaptation to real datasets. The proposed method outperforms classical and supervised methods in terms of registration accuracy on both synthetic (ModelNet40) and real-world (ETH3D) noisy, outlier-rich datasets. To our best knowledge, this marks the first instance of successful real RGB-D odometry data registration using an equivariant method. The code is available at https://sites.google.com/view/eccv24-equivalign.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bai, X., Luo, Z., Zhou, L., Fu, H., Quan, L., Tai, C.L.: D3Feat: joint learning of dense detection and description of 3D local features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6359–6367 (2020)
Berlinet, A., Thomas-Agnan, C.: Reproducing Kernel Hilbert Spaces in Probability and Statistics. Springer US, Boston, MA (2004). https://doi.org/10.1007/978-1-4419-9096-9
Besl, P.J., McKay, N.D.: A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992). https://doi.org/10.1109/34.121791
Biber, P., Fleck, S., Strasser, W.: A probabilistic framework for robust and accurate matching of point clouds. In: Rasmussen, C.E., Bülthoff, H.H., Schölkopf, B., Giese, M.A. (eds.) Pattern Recognition, pp. 480–487. Springer Berlin Heidelberg, Berlin, Heidelberg (2004). https://doi.org/10.1007/978-3-540-28649-3_59
Bishop, C.M.: Pattern Recognition and Machine Learning. Springer (2006)
Campbell, D., Petersson, L.: An adaptive data representation for robust point-set registration and merging. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4292–4300 (2015)
Chen, H., Liu, S., Chen, W., Li, H., Hill, R.: Equivariant point network for 3D point cloud analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14514–14523 (2021)
Chen, X., Milioto, A., Palazzolo, E., Giguere, P., Behley, J., Stachniss, C.: Suma++: efficient lidar-based semantic slam. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4530–4537. IEEE (2019)
Chen, Y., Medioni, G.G.: Object modeling by registration of multiple range images. Image Vis. Comput. 10(3), 145–155 (1992)
Choy, C., Park, J., Koltun, V.: Fully convolutional geometric features. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 8958–8966 (2019)
Choy, C., Park, J., Koltun, V.: Fully convolutional geometric features. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 8958–8966 (2019)
Chui, H., Rangarajan, A.: A feature registration framework using mixture models. In: Proceedings IEEE Workshop on Mathematical Methods in Biomedical Image Analysis, pp. 190–197. IEEE (2000)
Clark, W., Ghaffari, M., Bloch, A.: Nonparametric continuous sensor registration. J. Mach. Learn. Res. 22(271), 1–50 (2021)
Cohen, T., Weiler, M., Kicanaoglu, B., Welling, M.: Gauge equivariant convolutional networks and the icosahedral CNN. In: Proceedings of the International Conference on Machine Learning, pp. 1321–1330. PMLR (2019)
Cohen, T., Welling, M.: Group equivariant convolutional networks. In: Proceedings of the International Conference on Machine Learning, pp. 2990–2999. PMLR (2016)
Cohen, T.S., Geiger, M., Köhler, J., Welling, M.: Spherical CNNs. arXiv preprint arXiv:1801.10130 (2018)
Cohen, T.S., Welling, M.: Steerable CNNs. arXiv preprint arXiv:1612.08498 (2016)
Deng, C., Litany, O., Duan, Y., Poulenard, A., Tagliasacchi, A., Guibas, L.J.: Vector Neurons: a general framework for SO(3)-equivariant networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 12200–12209 (2021)
Deng, H., Birdal, T., Ilic, S.: PPFNet: global context aware local features for robust 3D point matching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 195–205 (2018)
Eckart, B., Kim, K., Kautz, J.: HGMR: hierarchical Gaussian mixtures for adaptive 3D registration. In: Proceedings of the European Conference on Computer Vision, pp. 705–721 (2018)
Evangelidis, G.D., Horaud, R.: Joint alignment of multiple point sets with batch and incremental expectation-maximization. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1397–1410 (2017)
Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)
Fuchs, F., Worrall, D., Fischer, V., Welling, M.: SE(3)-transformers: 3D roto-translation equivariant attention networks. In: Proceedings of Advances Neural Information Processing Systems Conference, vol. 33, pp. 1970–1981 (2020)
Ghaffari, M., Clark, W., Bloch, A., Eustice, R.M., Grizzle, J.W.: Continuous direct sparse visual odometry from RGB-D images. In: Proceedings of Robotics: Science and System Conference, Freiburg, Germany (2019)
Ghaffari, M., Clark, W., Bloch, A., Eustice, R.M., Grizzle, J.W.: Continuous direct sparse visual odometry from RGB-D images. arXiv preprint arXiv:1904.02266 (2019)
Horaud, R., Forbes, F., Yguel, M., Dewaele, G., Zhang, J.: Rigid and articulated point registration with expectation conditional maximization. IEEE Trans. Pattern Anal. Mach. Intell. 33(3), 587–602 (2010)
Huang, S., Gojcic, Z., Usvyatsov, M., Wieser, A., Schindler, K.: Predator: registration of 3D point clouds with low overlap. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4267–4276 (2021)
Jian, B., Vemuri, B.C.: Robust point set registration using Gaussian mixture models. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1633–1645 (2011)
Kerl, C.: Dense Visual Odometry (DVO). https://github.com/tum-vision/dvo (2013)
Li, F., Fujiwara, K., Okura, F., Matsushita, Y.: Generalized shuffled linear regression. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6474–6483 (2021)
Li, Y., Bu, R., Sun, M., Wu, W., Di, X., Chen, B.: PointCNN: convolution on X-transformed points. In: Proceedings of Advances in Neural Information Processing Systems Conference, pp. 820–830 (2018)
Lin, T.Y., Clark, W., Eustice, R.M., Grizzle, J.W., Bloch, A., Ghaffari, M.: Adaptive continuous visual odometry from RGB-D images. arXiv preprint arXiv:1910.00713 (2019)
MacDonald, L.E., Ramasinghe, S., Lucey, S.: Enabling equivariance for arbitrary lie groups. In: Proceedings of the IEEE Conference on Computer Vision Pattern Recognition, pp. 8183–8192 (2022)
Magnusson, M., Lilienthal, A., Duckett, T.: Scan registration for autonomous mining vehicles using 3D-NDT. J. Field Robot. 24(10), 803–827 (2007)
Mitra, N.J., Gelfand, N., Pottmann, H., Guibas, L.: Registration of point cloud data from a geometric optimization perspective. In: Proceedings of the 2004 Eurographics/ACM SIGGRAPH Symposium on Geometry Processing, pp. 22–31. SGP ’04, Association for Computing Machinery, New York, NY, USA (2004). https://doi.org/10.1145/1057432.1057435
Park, J., Zhou, Q.Y., Koltun, V.: Colored point cloud registration revisited. In: Proceedings of the IEEE Conference on Computer Vision, pp. 143–152 (2017)
Parkison, S.A., Gan, L., Jadidi, M.G., Eustice, R.M.: Semantic iterative closest point through expectation-maximization. In: Proceedings of British Machine Vision Conference, p. 280 (2018)
Parkison, S.A., Ghaffari, M., Gan, L., Zhang, R., Ushani, A.K., Eustice, R.M.: Boosting shape registration algorithms via reproducing kernel Hilbert space regularizers. IEEE Robot. Autom. Lett. 4(4), 4563–4570 (2019)
Paszke, A., et al.: Automatic differentiation in PyTorch (2017)
Pineda, L., et al.: Theseus: a library for differentiable nonlinear optimization. In: Proceedings of Advances Neural Information Processing Systems Conference, vol. 35, pp. 3801–3818 (2022)
Qi, C.R., Su, H., Mo, K., Guibas, L.J.: PointNet: deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 652–660 (2017)
Qin, Z., et al.: GeoTransformer: fast and robust point cloud registration with geometric transformer. IEEE Trans. Pattern Anal. Mach. Intell. (2023)
Rasmussen, C., Williams, C.: Gaussian processes for machine learning, vol. 1. MIT Press (2006)
Rusu, R.B., Blodow, N., Beetz, M.: Fast point feature histograms (FPFH) for 3D registration. In: 2009 IEEE International Conference on Robotics and Automation, pp. 3212–3217. IEEE (2009)
Satorras, V.G., Hoogeboom, E., Welling, M.: E(n) equivariant graph neural networks. In: Proceedings of the International Conference on Machine Learning, pp. 9323–9332. PMLR (2021)
Schops, T., Sattler, T., Pollefeys, M.: Bad SLAM: bundle adjusted direct RGB-D SLAM. In: Proceedings of the IEEE Conference on Computer Vision Pattern Recognition, pp. 134–144 (2019)
Segal, A., Haehnel, D., Thrun, S.: Generalized-ICP. In: Robotics: Science and Systems, vol. 2, issue 4, p. 435. Seattle, WA (2009)
Servos, J., Waslander, S.L.: Multi channel generalized-ICP. In: Proceedings of International Conference on Robotics and Automation, pp. 3644–3649. IEEE (2014)
Thomas, H., Qi, C.R., Deschaud, J.E., Marcotegui, B., Goulette, F., Guibas, L.J.: Kpconv: Flexible and deformable convolution for point clouds. In: Proceedings of the IEEE International Conference Computer Vision, pp. 6411–6420 (2019)
Thomas, N., et al.: Tensor field networks: rotation-and translation-equivariant neural networks for 3D point clouds. arXiv preprint arXiv:1802.08219 (2018)
Tsin, Y., Kanade, T.: A correlation-based approach to robust point set registration. In: Pajdla, T., Matas, J. (eds.) Computer Vision - ECCV 2004: 8th European Conference on Computer Vision, Prague, Czech Republic, May 11-14, 2004. Proceedings, Part III, pp. 558–569. Springer, Berlin, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24672-5_44
Wang, C., et al.: PyPose: a library for robot learning with physics-based optimization. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2023)
Wang, Y., Solomon, J.M.: Deep closest point: Learning representations for point cloud registration. In: Proceedings of the IEEE International Conference Computer Vision, pp. 3523–3532 (2019)
Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M., Solomon, J.M.: Dynamic graph CNN for learning on point clouds. ACM Trans. Graph. (TOG) (2019)
Weiler, M., Geiger, M., Welling, M., Boomsma, W., Cohen, T.S.: 3D steerable CNNs: learning rotationally equivariant features in volumetric data. In: Proceedings of the Advances in Neural Information Processing Systems Conference, vol. 31 (2018)
Whelan, T., Salas-Moreno, R.F., Glocker, B., Davison, A.J., Leutenegger, S.: ElasticFusion: real-time dense slam and light source estimation. Int. J. Robot. Res. 35(14), 1697–1716 (2016)
Wu, Z., et al.: 3D ShapeNets: a deep representation for volumetric shapes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1912–1920 (2015)
Yang, H., Shi, J., Carlone, L.: TEASER: fast and certifiable point cloud registration. IEEE Trans. Rob. 37(2), 314–333 (2020)
Yu, H., Li, F., Saleh, M., Busam, B., Ilic, S.: CoFiNet: reliable coarse-to-fine correspondences for robust pointcloud registration. In: Proceedings of the Advances in Neural Information Processing Systems Conference, vol. 34, 23872–23884 (2021)
Zeng, A., Song, S., Nießner, M., Fisher, M., Xiao, J., Funkhouser, T.: 3DMatch: learning local geometric descriptors from RGB-D reconstructions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1802–1811 (2017)
Zhang, R., et al.: A new framework for registration of semantic point clouds from stereo and RGB-D cameras. In: Proceedings of the IEEE International Conference Robotics and Automation, pp. 12214–12221 (2020)
Zhou, Q.-Y., Park, J., Koltun, V.: Fast global registration. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) Computer Vision – ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II, pp. 766–782. Springer International Publishing, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_47
Zhou, Q.Y., Park, J., Koltun, V.: Open3D: a modern library for 3D data processing. arXiv preprint arXiv:1801.09847 (2018)
Zhu, M., Ghaffari, M., Clark, W.A., Peng, H.: E2PN: efficient SE(3)-equivariant point network. In: Proceedings of the IEEE conference on Computer Vision Pattern Recognition, pp. 1223–1232 (2023)
Zhu, M., Ghaffari, M., Peng, H.: Correspondence-free point cloud registration with SO(3)-equivariant implicit shape representations. In: Conference on Robot Learning, pp. 1412–1422. PMLR (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, R. et al. (2025). Correspondence-Free SE(3) Point Cloud Registration in RKHS via Unsupervised Equivariant Learning. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15146. Springer, Cham. https://doi.org/10.1007/978-3-031-73223-2_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-73223-2_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-73222-5
Online ISBN: 978-3-031-73223-2
eBook Packages: Computer ScienceComputer Science (R0)