Chunyu Yang
ABSTRACT
Due to mathematical simplicity and computational efficiency, the least square regression (LSR) has become a popular method for multicategory classification. However, the discriminative and generalization capability of existing methods are limited. In this work, we pursue a structure constraint of the predicted label indicators of data from the same class because such structure ensures good discriminative capability and stability of the classification. To guarantee this, we propose a new regularization on the label transformation matrix. Specifically, we regularize the label transformation matrix by minimizing the difference l2,1 ofnorm and l2,2 norm of the predicted label matrix of each class. We call it group discriminative least square regression (GDLSR). In addition, we also adopt the -dragging technique to relax the binary label for better generalization capability. Experiments on several commonly used datasets show that our method outperforms other state-of-the-art LSR-based methods.
- D. Ruppert, S. J. Sheather, and M. P. Wand, "An effective band width selector for local least squares regression," J. Amer. Statist. Assoc., 90(432), 1257--1270, 1995.Google Scholar
Cross Ref
- T. Strutz, "Data fitting and uncertainty: a practical introduction to weighted least squares and beyond," Wiesbaden, Germany: Vieweg, 2010.Google Scholar
- H. Abdi, "Partial least squares regression and projection on latent structure regression (pls regression)". Wiley Interdiscip. Rev. Comput. Stat., 2(1), 97--106, 2010.Google ScholarDigital Library
- C. C. Chang, and C. Lin, "Libsvm: a library for support vector machines," ACM Transactions on Intelligent Systems and Technology (TIST), 2(3), 27, 2011.Google ScholarDigital Library
- J. Wright, A. Y. Yang, A. Ganesh, S. S. Sastry, and Y. Ma, "Robust face recognition via sparse representation," IEEE Transactions on Pattern Analysis and MachineIntelligence, 31(2), 210--227, 2009.Google ScholarDigital Library
- I. Naseem, R. Togneri, and M. Bennamoun, "Linear regression for face recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(11), 2106--2112, 2010.Google ScholarDigital Library
- Z. Lai, Y. Xu, Z. Jin, and D. Zhang, "Human gait recognition via sparse discriminant projection learning," IEEE Transactions on Circuits & Systems for Video Technology, 24(10), 1651--1662, 2014.Google ScholarCross Ref
- H. Xue, S. Chen, and Q. Yang, "Discriminatively regularized least squares classification," Pattern Recognition, 42(1), 93--104, 2009.Google ScholarDigital Library
- Cai, C. Ding, F. Nie, and H. Huang, "On the equivalent of low-rank linear regressions and linear discriminant analysis based regressions," In ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1124--1132, 2013.Google Scholar
- S. M. Xiang, F. P. Nie, G. F. Meng, C. H. Pan, and C. S. Zhang, "Discriminative least squares regressions for multiclass classification and feature selection," IEEE Trans. Neural Netw. Learn. Syst., 23(11), 1738--1754, Nov. 2012.Google ScholarCross Ref
- X. Y. Zhang, L. Wang, S. Xiang, and C. Liu, "Retargeted least squares regression algorithm," IEEE Transactions on Neural Networks and Learning Systems, 26(9), 2206--2213, 2015.Google ScholarCross Ref
- X. Cai, F. Nie, and H. Huang, "Exact top-k feature selection via l2,0-norm constraint," In International joint conference on artificial intelligence, 1240--1246, 2013.Google Scholar
- F. Bunea, Y. She, and M. H. Wegkamp, "Optimal selection of reduced rank estimators of high-dimensional matrices," Ann. Statist., 1282--1309, 2011.Google Scholar
- S. Xiang, Y. Zhu, X. Shen, and J. Ye, "Optimal exact least squares rank minimization" In Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining, 480--488, 2012.Google Scholar
- X. Fang, Y. Xu, X. Li, Z. Lai, and B. Fang, "Regularized label relaxation linear regression," IEEE Transactions on Neural Networks and Learning Systems, 29(4), pp. 1006--1018, 2017.Google ScholarCross Ref
- J. Wen, Y. Xu, Z. Li, Z. Ma, and Y. Xu, "Inter-class sparsity based discriminative least square regression," Neural Networks, 102, 36--47, 2018.Google ScholarDigital Library
- S. Zhan, J. Wu, N. Han, J. Wen, and X. Fang, "Group low-rank representation based discriminant linear regression," IEEE Transactions on Circuits and Systems for Video Technology, 2019.Google ScholarCross Ref
- Y. Lou, P. Yin, and Q. He, "Computing sparse representation in a highly coherent dictionary based on difference of and," Scientific Computing, 64(1), 178--196, 2015.Google ScholarDigital Library
- Z. Lin, M. Chen, and Y. Ma, "The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices," UIUC, Champaign, IL, USA, Tech. Rep. UILU-ENG-09-2215, Oct. 2009.Google Scholar
- Y. Zhang, "Recent advances in alternating direction methods: Practice and theory," in Proc. IPAM Workshop Numer. Methods Continuous Optim. UCLA, Los Angeles, CA, USA, pp. 1--3, 2010Google Scholar
- J. Yang and Y. Zhang, "Alternating direction algorithms for -problems in compressive sensing," SIAM J. Sci. Comput., 33(1), 250--278, 2011.Google ScholarDigital Library
- R. A. Horn and C. R. Johnson, Matrix Analysis. New York, NY, USA: Cambridge Univ. Press, 2012.Google Scholar
- S. M. Xiang, F. P. Nie, G. F. Meng, C. H. Pan, and C. S. Zhang, "Discriminative least squares regressions for multiclass classification and feature selection," IEEE Trans. Neural Netw. Learn. Syst., 23(11), 1738--1754, 2012.Google ScholarCross Ref
- A. S. Georghiades, P. N. Belhumeur, and D. J. Kriegman, "From few to many: Illumination cone models for face recognition under variable lighting and pose," IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6), 643--660, 2001.Google ScholarDigital Library
- C. Lu, H. Min, Z. Zhao, L. Zhu, D. Huang, and S. Yan Yan, "Robust and efficient subspace segmentation via least squares regression," in Computer Vision--ECCV 2012. Florence, Italy: Springer, pp. 347--360, Oct. 2012.Google Scholar
- S. Lazebnik, C. Schmid, and J. Ponce, "Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories," In 2006 IEEE computer society conference on computer vision and pattern recognition, 2, 2169--2178, 2006.Google Scholar
Index Terms
- Group Discriminative Least Square Regression
Recommendations
Discriminative sparse least square regression for semi-supervised learning
AbstractThe various variants of the classical least square regression (LSR) have been extensively utilized in numerous applications. However, most previous linear regression methods only consider the fitting between the original features and ...
Inter-class sparsity based discriminative least square regression
AbstractLeast square regression is a very popular supervised classification method. However, two main issues greatly limit its performance. The first one is that it only focuses on fitting the input features to the corresponding output labels ...
Semi-supervised graph-based retargeted least squares regression
A semi-supervised graph-based retargeted least squares regression model is proposed for multicategory classification.Our aim is to utilize a graph regularization to restrict the regression labels of ReLSR, such that similar samples should have similar ...
Comments