Skip to main content
SpringerLink
Log in

Segmentation by Continuous Latent Semantic Analysis for Multi-structure Model Fitting

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

In this paper, we propose a novel continuous latent semantic analysis fitting method, to efficiently and effectively estimate the parameters of model instances in data, based on latent semantic analysis and continuous preference analysis. Specifically, we construct a new latent semantic space (LSS): where inliers of different model instances are mapped into several independent directions, while gross outliers are distributed close to the origin of LSS. After that, we analyze the data distribution to effectively remove gross outliers in LSS, and propose an improved clustering algorithm to segment the remaining data points. On the one hand, the proposed fitting method is able to achieve excellent fitting results; due to the effective continuous preference analysis in LSS. On the other hand, the proposed method can efficiently obtain final fitting results due to the dimensionality reduction in LSS. Experimental results on both synthetic data and real images demonstrate that the proposed method achieves significant superiority over several state-of-the-art model fitting methods on both fitting accuracy and computational speed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. That is, among the generated model hypotheses, there is at least one model hypothesis (generated by a minimal subset) corresponding to a true model instance in data.

  2. CPA is a soft generalization of the binary preference analysis.

  3. The elements of the preference set \(\varvec{PS}_i\) are computed according to the preference function \(f(\cdot )\) in Eq. (1), based on the corresponding data point \(x_i\) and all the generated model hypotheses. Similarly, the elements of \(\varvec{PS}_j\) can be computed.

  4. For MSHF, we use the code provided by the authors. For RansaCov, T-linkage and RPA, we download the codes from the following website: http://www.diegm.uniud.it/fusiello/demo.

  5. For the CPU time, we exclude the time used for sampling minimal subsets and generating model hypotheses for all the competing fitting methods in all the following experiments.

  6. http://cs.adelaide.edu.au/~hwong/doku.php?id=data

References

  • Amayo, P., Pinis, P., Paz, L. M., & Newman, P. (2018). Geometric multi-model fitting with a convex relaxation algorithm. In Proceedings of the IEEE conference on computer vision and pattern recognition, 8138–8146.

  • Arthur, D., & Vassilvitskii, S. (2007). K-means++: The advantages of careful seeding. In Proceedings of the eighteenth annual ACM-SIAM symposium on discrete algorithms, 1027–1035.

  • Barath, D., Matas, J., & Noskova, J. (2019). Magsac: Marginalizing sample consensus. In Proceedings of the IEEE conference on computer vision and pattern recognition, 10197–10205.

  • Barath, D., Noskova, J., Ivashechkin, M., & Matas, J. (2020). Magsac++, a fast, reliable and accurate robust estimator. In Proceedings of the IEEE conference on computer vision and pattern recognition, 1304–1312.

  • Chin, T. J., Wang, H., & Suter, D. (2009). Robust fitting of multiple structures: The statistical learning approach. In Proceedings of the IEEE international conference on computer vision, 413–420.

  • Chin, T. J., Yu, J., & Suter, D. (2012). Accelerated hypothesis generation for multistructure data via preference analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(4), 625–638.

    Article  Google Scholar 

  • Chum, O., & Matas, J. (2005). Matching with prosac-progressive sample consensus. In Proceedings of the IEEE conference on computer vision and pattern recognition, 220–226.

  • Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990). Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41(6), 391–407.

    Article  Google Scholar 

  • Ester, M., Kriegel, H. P., Sander, J., & Xu, X. (1996). A density-based algorithm for discovering clusters in large spatial databases with noise. In IEEE Transactions on Pattern Analysis and Machine Intelligence pp. 226–231.

  • Ferraz, L., Felip, R., Martínez, B., & Binefa, X. (2007). A density-based data reduction algorithm for robust estimators. In Proceedings of Iberian conference on pattern recognition and image analysis, 355–362.

  • Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6), 381–395.

    Article  MathSciNet  Google Scholar 

  • Isack, H., & Boykov, Y. (2012). Energy-based geometric multi-model fitting. International Journal of Computer Vision, 97(2), 123–147.

    Article  Google Scholar 

  • Jiang, X., Ma, J., Jiang, J., & Guo, X. (2020). Robust feature matching using spatial clustering with heavy outliers. IEEE Transactions on Image Processing, 29, 736–746.

    Article  MathSciNet  Google Scholar 

  • Kanazawa, Y., & Kawakami, H. (2004). Detection of planar regions with uncalibrated stereo using distributions of feature points. In Proceedings of British machine vision conference, 2701–2710.

  • Le, H. M., Chin, T. J., Eriksson, A., Do, T. T., & Suter, D. (2019). Deterministic approximate methods for maximum consensus robust fitting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1, 1–14.

    Google Scholar 

  • Lee, J. K., & Yoon, K. J. (2019). Joint estimation of camera orientation and vanishing points from an image sequence in a non-manhattan world. International Journal of Computer Vision, 127(10), 1426–1442.

    Article  Google Scholar 

  • Ma, J., Jiang, X., Fan, A., Jiang, J., & Yan, J. (2021). Image matching from handcrafted to deep features: A survey. International Journal of Computer Vision, 129(1), 23–79.

    Article  MathSciNet  Google Scholar 

  • Ma, J., Zhao, J., Jiang, J., Zhou, H., & Guo, X. (2019). Locality preserving matching. International Journal of Computer Vision, 127(5), 512–531.

    Article  MathSciNet  Google Scholar 

  • Maaten, L. V. D., & Hinton, G. (2008). Visualizing data using t-sne. Journal of Machine Learning Research, 9(11), 2579–2605.

    MATH  Google Scholar 

  • MacQueen, J. (1967). Some methods for classification and analysis of multivariate observations. In Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, 281–296.

  • Magri, L., & Fusiello, A. (2014). T-linkage: A continuous relaxation of j-linkage for multi-model fitting. In Proceedings of the IEEE conference on computer vision and pattern recognition, 3954–3961.

  • Magri, L., & Fusiello, A. (2016). Multiple model fitting as a set coverage problem. In Proceedings of the IEEE conference on computer vision and pattern recognition, 3318–3326.

  • Magri, L., & Fusiello, A. (2017). Multiple structure recovery via robust preference analysis. Image and Vision Computing, 67, 1–15.

    Article  Google Scholar 

  • Magri, L., & Fusiello, A. (2019). Fitting multiple heterogeneous models by multi-class cascaded t-linkage. In Proceedings of the IEEE conference on computer vision and pattern recognition, 7460–7468.

  • Meer, P. (2020). Robust estimation techniques. Computer Vision: A Reference Guide, pp. 1–8.

  • Mittal, S., Anand, S., & Meer, P. (2012). Generalized projection-based m-estimator. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(12), 2351–2364.

    Article  Google Scholar 

  • Pham, T. T., Chin, T. J., Yu, J., & Suter, D. (2014). The random cluster model for robust geometric fitting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(8), 1658–1671.

    Article  Google Scholar 

  • Purkait, P., Chin, T. J., Ackermann, H., & Suter, D. (2017). Clustering with hypergraphs: The case for large hyperedges. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(9), 1697–1711.

    Article  Google Scholar 

  • Rodriguez, A., & Laio, A. (2014). Clustering by fast search and find of density peaks. Science, 344(6191), 1492–1496.

    Article  Google Scholar 

  • Tanimoto, T. (1957). An elementary mathematical theory of classification and prediction. IBM: Internal technical report.

  • Tennakoon, R., Bab-Hadiashar, A., Cao, Z., Hoseinnezhad, R., & Suter, D. (2016). Robust model fitting using higher than minimal subset sampling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(2), 350–362.

    Article  Google Scholar 

  • Toldo, R., & Fusiello, A. (2008). Robust multiple structures estimation with j-linkage. In Proceedings of European conference on computer vision, 537–547.

  • Torr, P. H., & Murray, D. W. (1997). The development and comparison of robust methods for estimating the fundamental matrix. International Journal of Computer Vision, 24(3), 271–300.

    Article  Google Scholar 

  • Tran, Q. H., Chin, T. J., Chojnacki, W., & Suter, D. (2014). Sampling minimal subsets with large spans for robust estimation. International Journal of Computer Vision, 106(1), 93–112.

    Article  MathSciNet  Google Scholar 

  • Wang, H., Chin, T. J., & Suter, D. (2012). Simultaneously fitting and segmenting multiple-structure data with outliers. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(6), 1177–1192.

    Article  Google Scholar 

  • Wang, H., Xiao, G., Yan, Y., & Suter, D. (2019). Searching for representative modes on hypergraphs for robust geometric model fitting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(3), 697–711.

    Article  Google Scholar 

  • Wong, H. S., Chin, T. J., Yu, J., & Suter, D. (2011). Dynamic and hierarchical multi-structure geometric model fitting. In Proceedings of the IEEE international conference on computer vision, 1044–1051.

  • Wu, Y., Li, Y., & Hu, Z. (2008). Detecting and handling unreliable points for camera parameter estimation. International Journal of Computer Vision, 79(2), 209–223.

    Article  Google Scholar 

  • Xiao, G., Wang, H., Lai, T., & Suter, D. (2016). Hypergraph modelling for geometric model fitting. Pattern Recognition, 60(1), 748–760.

    Article  Google Scholar 

  • Xiao, G., Wang, H., Yan, Y., & Suter, D. (2019). Superpixel-guided two-view deterministic geometric model fitting. International Journal of Computer Vision, 127(4), 323–329.

    Article  Google Scholar 

  • Xu, L., Oja, E., & Kultanen, P. (1990). A new curve detection method: randomized hough transform (rht). Pattern Recognition Letter, 11(5), 331–338.

    Article  Google Scholar 

  • Xu, X., Cheong, L. F., & Li, Z. (2019). 3d rigid motion segmentation with mixed and unknown number of models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1, 1–14.

    Google Scholar 

  • Yang, X., & Meer, P. M. J. (2020). A new approach to robust estimation of parametric structures. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1, 1–14.

    Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grants 62072223, 61872307, and by the Natural Science Foundation of Fujian Province under Grants 2020J01829.

Author information

Authors and Affiliations

  1. Fujian Key Laboratory of Sensing and Computing for Smart City, School of Informatics, Xiamen University, Xiamen, China

    Guobao Xiao & Hanzi Wang

  2. Fujian Provincial Key Laboratory of Information Processing and Intelligent Control, College of Computer and Control Engineering, Minjiang University, Fuzhou, China

    Guobao Xiao

  3. Electronic Information School, Wuhan University, Wuhan, China

    Jiayi Ma

  4. School of Science, Edith Cowan University, Joondalup, Australia

    David Suter

Authors
  1. Guobao Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hanzi Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiayi Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. David Suter
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hanzi Wang.

Additional information

Communicated by Zhouchen Lin.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xiao, G., Wang, H., Ma, J. et al. Segmentation by Continuous Latent Semantic Analysis for Multi-structure Model Fitting. Int J Comput Vis 129, 2034–2056 (2021). https://doi.org/10.1007/s11263-021-01468-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-021-01468-6

Keywords

Search

Navigation