Abstract
3D Registration is a fundamental part of several robotics and automation tasks. While classical methods predominantly exploit constraints from points or plane correspondences, we have a different take using line intersections. In other words, we focus on exploiting geometric constraints arising from the intersection of two (different) 3D line segments in two scans. In particular, we derive nine minimal solvers from various geometric constraints arising from line intersections along with other constraints: plane correspondences, point correspondences, and line matches. We follow a two-step method for 3D registration: a coarse estimation with outlier rejection followed by refinement. In the first step, we use a hybrid RANSAC loop that utilizes all the minimal solvers. This RANSAC outputs a rough estimate for the 3D registration and the outlier/inlier classification for the 3D features. As for the refinement, we offer a non-linear technique using all the inliers obtained from the RANSAC and the coarse estimate. This method is of alternate minimization type, in which we alternate between estimating the rotation and the translation at each step. Thorough experiments with simulated data and two real-world datasets show that using these features and the combined solvers improves accuracy and is faster than the baselines.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Notes
There are pros and cons to both approaches. Both need to apply transformations to the data to get a single-unknown polynomial equation. However, we understand the problem better and its degeneracies by deriving the polynomial analytically.
We are using Cayley’s parameterization for \(\mathcal{S}\mathcal{O}(3)\) matrices because they allow a more compact representation and are more suited for deriving the polynomials.
Due to space limitations, we omit the coefficients and monomials.
Notice we have simplified the constraint by pre-multiplying \(1 + s_1^2 + s_2^2 + s_3^2\)
We do not show all these derivations for space purposes.
We solved the polynomial using the Eigen’s library PolynomialSolver class.
We use the version trained on 3DMatch.
We note that comparisons in this paper are focusing only on the alignment, not in the feature extraction and matching.
Although we never lose much when compared to the baselines, we know that not every sequence have the perfect conditions for the line intersection constraints.
References
Aiger, D., Mitra, N. J., & Cohen-Or, D. (2008) 4-points congruent sets for robust surface registration, ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2008), 27(3), pp. #85, 1–10
Akiba, T., Sano, S., Yanase, T., Ohta, T., & Koyama, M. (2019)“Optuna: A next-generation hyperparameter optimization framework,” In Proceedings of the 25rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.
Aoki, Y., Goforth, H., Srivatsan, R. A., & Lucey, S. (2019) Pointnetlk: Robust & efficient point cloud registration using pointnet, In IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 7163–7172.
Bai, X., Luo, Z., Zhou, L., Fu, H., Quan, L., & Tai, C.-L. (2020) D3feat: Joint learning of dense detection and description of 3d local features, In IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 6358–6366.
Bamji, C. (2010). Method and system for lossless dealiasing in time-of-flight (tof) systems, U.S. Patent 7 791 715B1, 9 7.
Barath, D., & Matas, J. (2018). Graph-cut ransac, In IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 6733–6741.
Barath, D., Matas, J., & Noskova, J. (2019) Magsac: Marginalizing sample consensus, In IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 10 189–10 197.
Bauschke, H. H., & Borwein, J. M. (1996). On projection algorithms for solving convex feasibility problems. SIAM Review, 38(3), 367–426.
Benko, H., Jota, R., & Wilson, A. (2012). Miragetable: Freehand interaction on a projected augmented reality tabletop, In ACM SIGCHI Conference on Human Factors in Computing Systems (CHI), pp. 199–208.
Besl, P. J., & McKay, N. D. (1992). A method for registration of 3-d shapes. IEEE Transactions Pattern Analysis and Machine Intelligence (T-PAMI), 14(2), 239–256.
Bhattacharya, U., & Govindu, V. M. (2019) Efficient and robust registration on the 3d special euclidean group, In IEEE Int’l Conf. Computer Vision (ICCV), vol. 2.
Bhayani, S., Kukelova, Z., & Heikkilä, J. (2021)“Computing stable resultant-based minimal solvers by hiding a variable,” In IEEE Int’l Conf. Pattern Recognition (ICPR), pp. 6104–6111.
Campos, J., Cardoso, J. R., & Miraldo, P. (2019) Poseamm: A unified framework for solving pose problems using an alternating minimization method, In IEEE Int’l Conf. Robotics and Automation (ICRA), pp. 3493–3499.
Camposeco, F., Cohen, A., Pollefeys, M., & Sattler, T. (2018) Hybrid camera pose estimation, In IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 136–144.
Chen, Y., & Medioni, G. (1991). Object modeling by registration of multiple range images. IEEE International Conference Robotics and Automation (ICRA), 3, 2724–2729.
Chen, J., Izadi, S., & Fitzgibbon, A. (2012). Kinetre: Animating the world with the human body, In ACM Symposium on User Interface Software and Technology (UIST), pp. 435–444.
Chetverikov, D., Svirko, D., Stepanov, D., & Krsek, P. (2002). The trimmed iterative closest point algorithm. In Object recognition supported by user interaction for service robots, 3, 545–548.
Choi, S., Zhou, Q.-Y., & Koltun, V. (2015) Robust reconstruction of indoor scenes, In IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 5556–5565.
Choy, C., Dong, W., & Koltun, V. (2020) Deep global registration, In IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2511–2520.
Choy, C., Park, J., & Koltun, V. (2019) Fully convolutional geometric features, In IEEE International Conference Computer Vision (ICCV), 8957–8965.
Chum, O., & Matas, J. (2005). Matching with prosac - progressive sample consensus. In IEEE Conference Computer Vision and Pattern Recognition (CVPR), 1, 220–226.
Chum, O., Matas, J., & Kittler, J. (2003) Locally optimized ransac, In Pattern Recognition, 236–243.
Csiszár, I., & Tusnády, G. (1984). Information geometry and alternating minimization procedures. Statistics and Decisions, Supplement Issue, 1, 205–237.
Deng, H., Birdal, T., & Ilic, S. (2018) Ppf-foldnet: Unsupervised learning of rotation invariant 3d local descriptors, In European Conference Computer Vision (ECCV), 02–618.
Deng, H., Birdal, T., & Ilic, S. (2018). Ppfnet: Global context aware local features for robust 3d point matching, In IEEE Conference Computer Vision and Pattern Recognition (CVPR), 195–205.
Dou, M., Fuchs, H., & Frahm, J.-M. (2013). Scanning and tracking dynamic objects with commodity depth cameras, In IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 99–106.
Elbaz, G., Avraham, T., & Fischer, A. (2017) 3d point cloud registration for localization using a deep neural network auto-encoder, In IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 4631–4640.
Endres, F., Hess, J., Sturm, J., Cremers, D., & Burgard, W. (2014). 3-d mapping with an rgb-d camera. IEEE Transactions Robotics (T-RO), 30(1), 177–187.
Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications ACM, 24(6), 381–395.
Fitzgibbon, A. W. (2003). Robust registration of 2d and 3d point sets. Image and Vision Computing (IVC), 21(13), 1145–1153.
Forstner, W., & Khoshelham, K. (2017) Efficient and accurate registration of point clouds with plane to plane correspondences, In IEEE International Conference Computer Vision Workshops (ICCVW), 2165–2173.
Freedman, B., Shpunt, A., Machline, M., & Arieli, Y. (2008). “Depth mapping using projected patterns,” U.S. Patent 200 802 405O2A1, 10 2.
Geiger, A., Lenz, P., & Urtasun, R. (2012) Are we ready for autonomous driving? the kitti vision benchmark suite, In IEEE Conference Computer Vision and Pattern Recognition (CVPR).
Gold, S., Lu, C.-P., Rangarajan, A., Pappu, S., & Mjolsness, E. (1994) New algorithms for 2d and 3d point matching: Pose estimation and correspondence, In Advances in Neural Information Processing Systems (NIPS), 957–964.
Govindu, V. M. (2001) Combining two-view constraints for motion estimation, In IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2.
Hartley, R., Trumpf, J., Dai, Y., & Li, H. (2013). Rotation averaging. International Journal of Computer Vision (IJCV), 103(3), 267–305.
Henry, P., Krainin, M., Herbst, E., Ren, X., & Fox, D. (2012). Rgb-d mapping: Using kinect-style depth cameras for dense 3d modeling of indoor environments. The International Journal of Robotics Research (IJRR), 31(5), 647–663.
Huang, A. S., Bachrach, A., Henry, P., Krainin, M., Maturana, D., Fox, D., & Roy, N. (2017). Visual odometry and mapping for autonomous flight using an rgb-d camera, In International Symposium Robotics Research (ISRR), 235–252.
Huang, S., Gojcic, Z., Usvyatsov, M., Wieser, A., & Schindler, K. (2021) Predator: Registration of 3d point clouds with low overlap, In IEEE Conference Computer Vision and Pattern Recognition (CVPR), 4265–4274.
Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., & Fitzgibbon, A. (2011). Kinectfusion: Real-time 3d reconstruction and interaction using a moving depth camera, In ACM Symposium on User Interface Software and Technology (UIST), pp. 559–568.
Karpathy, A., & Fei-Fei, L. (2015). Deep visual-semantic alignments for generating image descriptions, In IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 3128–3137.
Kerl, C., Sturm, J., & Cremers, D. (2013). Dense visual slam for rgb-d cameras, In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2100–2106.
Kerl, C., Sturm, J., & Cremers, D. (2013). Robust odometry estimation for rgb-d cameras, In IEEE International Conference on Robotics and Automation (ICRA), 3748–3754.
Kluger, F., Brachmann, E., Ackermann, H., Rother, C., Yang, M. Y., & Rosenhahn, B. (2020). Consac: Robust multi-model fitting by conditional sample consensus, In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 4633–4642.
Kneip, L., Scaramuzza, D., & Siegwart, R. (2011) A novel parametrization of the perspective-three-point problem for a direct computation of absolute camera position and orientation, In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2969–2976.
Kukelova, Z., Bujnak, M., & Pajdla, T. (2008) “Automatic generator of minimal problem solvers,” In European Conference on Computer Vision (ECCV), 302–315.
Kukelova, Z., Heller, J., & Fitzgibbon, A. (2016) Efficient intersection of three quadrics and applications in computer vision, In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1799–1808.
Kummerle, R., Grisetti, G., Strasdat, H., Konolige, K., & Burgard, W. (2011) \(\text{g}^{2}\text{ o }\): A general framework for graph optimization, In IEEE International Conference on Robotics and Automation (ICRA), pp. 3607–3613.
Larsson, V., Kukelova, Z., & Zheng, Y. (2017). Making minimal solvers for absolute pose estimation compact and robust, In 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2335–2343.
Lee, G. H., Li, B., Pollefeys, M., & Fraundorfer, F. (2015). Minimal solutions for the multi-camera pose estimation problem. The International Journal of Robotics Research, 34(7), 837–848.
Li, H., & Hartley, R. (2006). Five-point motion estimation made easy. In IEEE International Conference on Pattern Recognition (ICPR), 1, 630–633.
Li, H., & Hartley, R. (2007) The 3d-3d registration problem revisited, In IEEE International Conference on Computer Vision (ICCV), 1–8.
Li, X., Pontes, J. K., & Lucey, S. (2021) Pointnetlk revisited, In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12 758–12 767.
Li, K., Yao, J., & Lu, X. (2014). “Robust line matching based on ray-point-ray structure descriptor,” In Asian Conference on Computer Vision (ACCV), 554–569.
Li, K., Yao, J., Lu, X., Li, L., & Zhang, Z. (2016). Hierarchical line matching based on line-junction-line structure descriptor and local homography estimation. Neurocomputing, 184, 207–220.
Lu, Y., & Song, D. (2015). Robust rgb-d odometry using point and line features, In IEEE International Conference on Computer Vision (ICCV), 3934–3942.
Mateus, A., Ramalingam, S., & Miraldo, P. (June 2020). Minimal solvers for 3d scan alignment with pairs of intersecting lines, In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
Mellado, N. et al., (2017) “Opengr: A c++ library for 3d global registration,” https://storm-irit.github.io/OpenGR/.
Mellado, N., Aiger, D., & Mitra, N. J. (2014). Super 4pcs fast global pointcloud registration via smart indexing. Computer Graphics Forum, 33(5), 205–215.
Minguez, J., Montesano, L., & Lamiraux, F. (2006). Metric-based iterative closest point scan matching for sensor displacement estimation. IEEE Transactions of Robotics (T-RO), 22(5), 1047–1054.
Miraldo, P., Dias, T., & Ramalingam, S. (2018) A minimal closed-form solution for multi-perspective pose estimation using points and lines, In European Conference Computer Vision (ECCV), 490–507.
Miraldo, P., Saha, S., & Ramalingam, S. (2019). Minimal solvers for mini-loop closures in 3d multi-scan alignment, In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
Mur-Artal, R., & Tardos, J. D. (2017). Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras. IEEE Transactions of Robotics (T-RO), 33(5), 1255–1262.
Myronenko, A., Song, X., & Carreira-Perpian, M. A. (2007) Non-rigid point set registration: Coherent point drift, In Advances in Neural Information Processing Systems (NIPS).
Niclass, C. L., Shpunt, A., Agranov, G. A., Waldon, M. C., Rezk, M. A., & Oggier, T. (2019) Light detection and ranging sensor, U.S. Patent 10 324 171B2, 6 18.
Niesen, U., Shah, D., & Wornell, G. W. (2009). Adaptive alternating minimization algorithms. IEEE Transactions of Information Theory, 55(3), 1423–1429.
Nister, D. (2004). An efficient solution to the five-point relative pose problem. IEEE Transactions of Pattern Analysis and Machine Intelligence (T-PAMI), 26(6), 756–770.
Olsson, C., Kahl, F., & Oskarsson, M. (2009). Branch-and-bound methods for euclidean registration problems. IEEE Transactions of Pattern Analysis and Machine Intelligence (T-PAMI), 31(5), 783–794.
Pais, G. D., Miraldo, P., Ramalingam, S., Govindu, V. M., Nascimento, J. C., & Chellappa, R. (2020). 3DRegNet: A deep neural network for 3d point registration, In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
Park, J., Zhou, Q.-Y., & Koltun, V. (2017). Colored point cloud registration revisited, In IEEE International Conference on Computer Vision (ICCV), 143–152.
Pless, R. (2003). Using many cameras as one. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2, 587.
Pottmann, H., & Wallner, J. (2001). Computational Line Geometry (1st ed.). Berlin Heidelberg: Springer-Verlag.
Ramalingam, S., & Taguchi, Y. (2013). A theory of minimal 3d point to 3d plane registration and its generalization. International Journal of Computer Vision (IJCV), 102(1), 73–90.
Ranade, S., Yu, X., Kakkar, S., Miraldo, P., & Ramalingam, S. (2020) “Mapping of sparse 3d data using alternating projection,” In Asian Conference on Computer Vision (ACCV), pp. 295–313.
Raposo, C., & Barreto, J. (2018). 3d registration of curves and surfaces using local differential information, In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 9300–9308.
Raposo, C., Lourenco, M., Barreto, M., & Antunes, M. (2013). Plane-based odometry using an rgb-d camera, In British Machine Vision Conference (BMVC)
Rusinkiewicz, S., & Levoy, M. (2001) Efficient variants of the icp algorithm. In 3-D Digital Imaging and Modeling (3DIM), 1, 145–152.
Rusu, R. B., Blodow, N., & Beetz, M. (2009) Fast point feature histograms (fpfh) for 3d registration, In IEEE International Conference on Robotics and Automation (ICRA), 3212–3217.
Schonemann, P. H. (1966). A generalized solution of the orthogonal procrustes problem. Psychometrika, 31(1), 1–10.
Schops, T., Sattler, T., & Pollefeys, M. (2019). “Bad slam: Bundle adjusted direct rgb-d slam,” In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
Steinbrucker, F., Kerl, C., Cremers, D., & Sturm, J. (2013) Large-scale multi-resolution surface reconstruction from rgb-d sequences, In IEEE International Conference on Computer Vision (ICCV), 3264–3271.
Stewenius, H., Oskarsson, M., Astrom, K., & Nister, D. (2005). Solutions to minimal generalized relative pose problems, In Workshop on Omnidirectional Vision (OMNIVIS).
Sturm, J., Engelhard, N., Endres, F., Burgard, W., & Cremers, D. (2012). A benchmark for the evaluation of rgb-d slam systems, In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 573–580.
Taguchi, Y., Jian, Y.-D., Ramalingam, S., & Feng, C. (2013). Point-plane slam for hand-held 3d sensors, In IEEE International Conference on Robotics and Automation (ICRA), 5182–5189.
Torr, P., & Zisserman, A. (2000). Mlesac: A new robust estimator with application to estimating image geometry. Computer Vision and Image Understanding (CVIU), 78(1), 138–156.
Tsin, Y., & Kanade, T. (2004). A correlation-based approach to robust point set registration, In European Conference on Computer Vision (ECCV), 558–569.
Umeyama, S. (1991). Least-squares estimation of transformation parameters between two point patterns. IEEE Transactions of Pattern Analysis and Machine Intelligence (T-PAMI), 13(4), 376–380.
Ventura, J., Arth, C., Reitmayr, G., & Schmalstieg, D. (2014). A minimal solution to the generalized pose-and-scale problem, In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 422–429.
Wang, Y., & Solomon, J. M. (2019).“Deep closest point: Learning representations for point cloud registration,” In IEEE International Conference on Computer Vision (ICCV).
Whelan, T., Kaess, M., Leonard, J. J., & McDonald, J. (2013) Deformation-based loop closure for large scale dense rgb-d slam, In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 548–555.
Xiao, J., Owens, A., & Torralba, A. (2013). Sun3d: A database of big spaces reconstructed using sfm and object labels, In IEEE International Conference on Computer Vision (ICCV), 1625–1632.
Yan, J., Wang, J., Zha, H., Yang, X., & Chu, S. M. (2015). “Multi-view point registration via alternating optimization,” In AAAI Conference on Artificial Intelligence.
Yang, J., Li, H., & Jia, Y. (2013) Go-icp: Solving 3d registration efficiently and globally optimally, In IEEE International Conference on Computer Vision (ICCV), 1457–1464.
Yang, H., Shi, J., & Carlone, L. (2020). Teaser: Fast and certifiable point cloud registration. IEEE Transactions of Robotics (T-RO), 37(2), 314–333.
Yew, Z. J., & Lee, G. H. (2020) Rpm-net: Robust point matching using learned features, In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11 821–11 830.
Yew, Z. Jian, & Lee, G. Hee (2018)“3dfeat-net: Weakly supervised local 3d features for point cloud registration,” In European Conference on Computer Vision (ECCV), 607–623.
Zhang, Z. (1994). Iterative point matching for registration of free-form curves and surfaces. International Journal of Computer Vision (IJCV), 13(2), 119–152.
Zhou, X., Leonardos, S., Hu, X., & Daniilidis, K. (2015). 3d shape reconstruction from 2d landmarks: A convex formulation, In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
Zhou, Q.-Y., Park, J., & Koltun, V. (2016) Fast global registration, In European Conference on Computer Vision (ECCV), 766–782.
Zhou, Q.-Y., Park, J., & Koltun, V. (2018)“Open3d: A modern library for 3d data processing,” arXiv:1801.09847.
Zhou, X., Zhu, M., & Daniilidis, K. (2015). Multi-image matching via fast alternating minimization, In IEEE International Conference on Computer Vision (ICCV).
Acknowledgements
A. Mateus was partially supported by the Portuguese National Funding Agency for Science, Research and Technology (FCT) grant PD/BD/135015/2017 and project LARSyS - FCT Plurianual funding 2020-2023.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Federica Arrigoni.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Mateus, A., Ranade, S., Ramalingam, S. et al. Fast and Accurate 3D Registration from Line Intersection Constraints. Int J Comput Vis 131, 2044–2069 (2023). https://doi.org/10.1007/s11263-023-01774-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-023-01774-1