Abstract
Robust feature matching is a fundamental capability for visual SLAM. It remains, however, a challenging task, particularly for changed environments. Some researchers use semantic segmentation to remove potentially dynamic objects which cause changes in the indoor environment. However, removing these objects may reduce the quantity and quality of feature matching. We observed that objects are moved but the room layout does not change. Inspired by this, we proposed to leverage the room layout information for feature matching. Considering current image matching datasets do not have obvious changes caused by dynamic objects and lack layout information, we created a dataset named Changed Indoor 10k (CR10k) to evaluate if we can utilize layout information for image matching in the changed environment. Our dataset contains apparent movements of large objects, and room layout can be extracted from it. We evaluate the performance of existing image matching methods on our dataset and ScanNet dataset. In addition, we propose Layout Constraint Matching (LCM) which is robust to changed environments and the LCM outperforms conventional approaches on the task of pose estimation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bai, X., Luo, Z., Zhou, L., Fu, H., Quan, L., Tai, C.L.: D3feat: joint learning of dense detection and description of 3D local features. In: 2020 IEEE Conference on Computer Vision and Pattern Recognition (2020)
Coughlan, J.M., Yuille, A.L.: Manhattan world: compass direction from a single image by Bayesian inference. In: Proceedings of the Seventh IEEE International Conference on Computer Vision, vol. 2 (1999)
Dai, A., Chang, A.X., Savva, M., Halber, M., Funkhouser, T.A., Nießner, M.: Scannet: richly-annotated 3D reconstructions of indoor scenes. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (2017)
DeTone, D., Malisiewicz, T., Rabinovich, A.: Superpoint: self-supervised interest point detection and description. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition Workshops (2018)
Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24, 381–395 (1981)
Li, Z., Snavely, N.: Megadepth: learning single-view depth prediction from internet photos. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition (2018)
Lin, H.J., Huang, S.W., Lai, S.H., Chiang, C.K.: Indoor scene layout estimation from a single image. In: 2018 24th International Conference on Pattern Recognition (2018)
Mallya, A., Lazebnik, S.: Learning informative edge maps for indoor scene layout prediction. In: 2015 IEEE International Conference on Computer Vision (2015)
Ono, Y., Trulls, E., Fua, P.V., Yi, K.M.: Lf-net: Learning local features from images. In: Neural Information Processing Systems (2018)
Pero, L.D., Bowdish, J., Fried, D., Kermgard, B., Hartley, E., Barnard, K.: Bayesian geometric modeling of indoor scenes. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (2012)
Ren, Y., Li, S., Chen, C., Kuo, C.C.J.: A coarse-to-fine indoor layout estimation (cfile) method. In: 2016 Asian Conference on Computer Vision (2016)
Revaud, J., Leroy, V., Weinzaepfel, P., Chidlovskii, B.: Pump: pyramidal and uniqueness matching priors for unsupervised learning of local descriptors. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3916–3926 (2022)
Sarlin, P.E., DeTone, D., Malisiewicz, T., Rabinovich, A.: Superglue: learning feature matching with graph neural networks. In: 2020 IEEE Conference on Computer Vision and Pattern Recognition (2020)
Savinov, N., Seki, A., Ladicky, L., Sattler, T., Pollefeys, M.: Quad-networks: unsupervised learning to rank for interest point detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (2017)
Schönberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (2016)
Schönberger, J.L., Zheng, E., Frahm, J.-M., Pollefeys, M.: Pixelwise view selection for unstructured multi-view stereo. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 501–518. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_31
Shen, Z., Sun, J., Wang, Y., He, X.H., Bao, H., Zhou, X.: Semi-dense feature matching with transformers and its applications in multiple-view geometry. IEEE Trans. Pattern Anal. Mach. Intell. 45, 7726–7738 (2022)
Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D.: A benchmark for the evaluation of RGB-D slam systems. In: 2012 IEEE International Conference on Intelligent Robots and Systems (2012)
Wald, J., Sattler, T., Golodetz, S., Cavallari, T., Tombari, F.: Beyond controlled environments: 3D camera re-localization in changing indoor scenes. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12352, pp. 467–487. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58571-6_28
Yi, K.M., Trulls, E., Ono, Y., Lepetit, V., Salzmann, M., Fua, P.V.: Learning to find good correspondences. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition (2018)
Zhang, J., et al.: Learning two-view correspondences and geometry using order-aware network. In: 2019 IEEE International Conference on Computer Vision (2019)
Zhang, L., Rusinkiewicz, S.M.: Learning to detect features in texture images. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition (2018)
Acknowledgment
This work is supported by the China Postdoctoral Science Foundation (No.2021M690094) and the FuXiaQuan National Independent Innovation Demonstration Zone Collaborative Innovation Platform (No. 3502ZCQXT2021003).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Hu, Q., Shen, X., Li, Z., Liu, W., Wang, C. (2024). Feature Matching in the Changed Environments for Visual Localization. In: Liu, Q., et al. Pattern Recognition and Computer Vision. PRCV 2023. Lecture Notes in Computer Science, vol 14435. Springer, Singapore. https://doi.org/10.1007/978-981-99-8552-4_14
Download citation
DOI: https://doi.org/10.1007/978-981-99-8552-4_14
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8551-7
Online ISBN: 978-981-99-8552-4
eBook Packages: Computer ScienceComputer Science (R0)