Skip to main content
SpringerLink
Log in

Adaptive weighted least squares regression for subspace clustering

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

In this research paper, we consider the subspace clustering problem which aims at finding a low-dimensional representation of a high-dimensional data set. In particular, our central focus is upon the least squares regression based on which we elaborate an adaptive weighted least squares regression for subspace clustering. Compared to the least squares regression, we consider the data locality to adaptively select relevant and close samples and discard irrelevant and faraway ones. Additionally, we impose a weight matrix on the representation errors to adaptively highlight the meaningful features and minimize the effect of redundant/noisy ones. Finally, we also add a non-negativity constraint on the representation coefficients to enhance the graph interpretability. These interesting properties allow to build up a more informative and quality graph, thereby yielding very promising clustering results. Extensive experiments on synthetic and real databases demonstrated that our clustering method achieves consistently optimal results, compared to multiple clustering methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. http://www.cad.zju.edu.cn/home/dengcai/Data/FaceData.html.

  2. http://www.cs.columbia.edu/CAVE/software/softlib.

  3. http://www.gaussianprocess.org/gpml/data/.

  4. http://yann.lecun.com/exdb/mnist/.

References

  1. Ben Abdallah F, Feki G, Ben Ammar A, Ben Amar C (2018) Multilevel deep learning-based processing for lifelog image retrieval enhancement. In: 2018 IEEE international conference on systems, man, and cybernetics (SMC), pp 1348–1354

  2. Bouchrika T, Zaied M, Jemai O, Ben Amar C (2012) Ordering computers by hand gestures recognition based on wavelet networks. In: CCCA12, pp 1–6

  3. Bouhlel N, Feki G, Ben Ammar A, Ben Amar C (2017) A hypergraph-based reranking model for retrieving diverse social images. In: Lecture Notes in Computer Science, vol 10424 LNCS, Springer International Publishing, Cham, pp 279–291

  4. Bouhlel N, Feki G, Ben Ammar A, Ben Amar C (2020) Hypergraph learning with collaborative representation for image search reranking. Int J Multimedia Inform Retrieval

  5. Bouhlel N, Ksibi A, Ben Ammar A, Ben Amar C (2016) Semantic-aware framework for Mobile Image Search. In: International conference on intelligent systems design and applications, ISDA, vol 2016, IEEE, pp 479–484

  6. Chen Y, Yi Z (2019) Locality-constrained least squares regression for subspace clustering. Knowledge-Based Systems 163:51–56

    Article  Google Scholar 

  7. Elhamifar E, Vidal R (2009) Sparse subspace clustering. In: 2009 IEEE conference on computer vision and pattern recognition, pp 2790–2797

  8. Esser E (2010) Applications of lagrangian-based alternating direction methods and connections to split bregman ernie esser

  9. Fakhfakh R, Feki G, Ben Ammar A, Ben Amar C (2016) Personalizing information retrieval: A new model for user preferences elicitation. In: 2016 IEEE international conference on systems, man, and cybernetics (SMC), pp 002091–002096

  10. Feki G, Ben Ammar A, Ben Amar C (2014) Adaptive semantic construction for diversity-based image retrieval. In: Proceedings of the international joint conference on knowledge discovery, knowledge engineering and knowledge management, Vol 1, p 444–449. SCITEPRESS - Science and Technology Publications, Lda, Setubal, PRT

  11. Feki G, Ksibi A, Ben Ammar A, Ben Amar C (2013) Improving image search effectiveness by integrating contextual information

  12. Georghiades AS, Belhumeur PN, Kriegman DJ (2001) From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Mach Intell 23(6):643–660

    Article  Google Scholar 

  13. Hoerl AE, Kennard RW (1970) Ridge regression: applications to nonorthogonal problems. Technometrics 12(1):69–82

    Article  Google Scholar 

  14. Hu H, Lin Z, Feng J, Zhou J (2014) Smooth representation clustering. 2014 IEEE conference on computer vision and pattern recognition, pp 3834–3841

  15. Huang J, Nie F, Huang H (2015) A new simplex sparse learning model to measure data similarity for clustering. IJCAI International Joint Conference on Artificial Intelligence 2015-Janua(Ijcai), pp 3569–3575

  16. Jing L, Ng MK, Huang JZ (2007) An entropy weighting k-means algorithm for subspace clustering of high-dimensional sparse data. IEEE Trans Knowl Data Eng 19(8):1026–1041

    Article  Google Scholar 

  17. Ksibi A, Feki G, Ammar AB, Amar C (2013) Effective diversification for ambiguous queries in social image retrieval. In: CAIP

  18. Lai Z, Xu Y, Chen Q, Yang J, Zhang D (2014) Multilinear sparse principal component analysis. IEEE Trans Neural Netw Learn Syst 25(10):1942–1950

    Article  Google Scholar 

  19. Liu G, Lin Z, Yu Y (2010) Robust Subspace Segmentation by Low-Rank Representation. In: Proceedings of the 27th international conference on international conference on machine learning. Omnipress, Madison, WI, USA

  20. Liu J, Chen Y, Zhang J, Xu Z (2014) Enhancing Low-Rank Subspace Clustering by Manifold Regularization 23(9): 4022–4030

  21. Lu C, Min H, Zhao Z.Q, Zhu L, shuang Huang D, Yan S (2012) Robust and efficient subspace segmentation via least squares regression. In: ECCV (2012)

  22. Luxburg UV (2006) A tutorial on spectral clustering. Tech Rep pp 1–32

  23. Nie F, Wang X, Jordan MI, Huang H (2016) The constrained laplacian rank algorithm for graph-based clustering. 30th AAAI Conference on Artificial Intelligence, AAAI 2016 (1), pp 1969–1976

  24. Peng C, Kang Z, Cheng Q (2017) Subspace clustering via variance regularized ridge regression. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 682–691

  25. Peng X, Yu Z, Yi Z, Tang H (2017) Constructing the L2-graph for robust subspace learning and subspace clustering. IEEE Trans Cybern 47(4):1053–1066

    Article  Google Scholar 

  26. Peng X, Zhang L, Yi Z (2013) Scalable sparse subspace clustering. In: 2013 IEEE conference on computer vision and pattern recognition, pp 430–437

  27. Samaria FS, *t, F.S.S., Harter A, Site OA (1994) Parameterisation of a stochastic model for human face identification

  28. Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22:888–905

    Article  Google Scholar 

  29. Tang M, Nie F, Jain R (2016) Capped lp-norm graph embedding for photo clustering. In: Proceedings of the 24th ACM International Conference on Multimedia, MM ’16, pp 431–435. Association for Computing Machinery, New York, NY, USA

  30. Tang M, Nie F, Jain R (2017) A graph regularized dimension reduction method for out-of-sample data. Neurocomputing 225:58–63

    Article  Google Scholar 

  31. Vidal R (2011) Subspace Clustering. IEEE Signal Process Magaz 28(2):52–68. https://doi.org/10.1109/MSP.2010.939739

    Article  Google Scholar 

  32. Wen J, Fang X, Xu Y, Tian C, Fei L (2018) Low-rank representation with adaptive graph regularization. Neural Netw 108:83–96

    Article  Google Scholar 

  33. Zheng J, Yang P, Chen S, Shen G, Wang W (2017) Iterative re-constrained group sparse face recognition with adaptive weights learning. IEEE Trans Image Process 26:2408–2423

    Article  MathSciNet  Google Scholar 

  34. Zhuang L, Gao H, Lin Z, Ma Y, Zhang X, Yu N (2012) Non-negative low rank and sparse graph for semi-supervised learning. 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2328–2335

  35. Zou H, Hastie TJ (2005) Regularization and variable selection via the elastic net

Download references

Acknowledgements

The research leading to these results has received funding from the Ministry of Higher Education and Scientific Research of Tunisia under the grant agreement number LR11ES48.

Author information

Authors and Affiliations

  1. REGIM: Research Groups in Intelligent Machines, University of Sfax, National Engineering School of Sfax (ENIS), BP 1173, 3038, Sfax, Tunisia

    Noura Bouhlel, Ghada Feki & Chokri Ben Amar

Authors
  1. Noura Bouhlel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ghada Feki
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chokri Ben Amar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Noura Bouhlel.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bouhlel, N., Feki, G. & Ben Amar, C. Adaptive weighted least squares regression for subspace clustering. Knowl Inf Syst 63, 2883–2900 (2021). https://doi.org/10.1007/s10115-021-01612-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-021-01612-1

Keywords

Search

Navigation