Abstract
Current multi-output regression method usually ignores the relationship among response variables, and thus it is challenging to obtain an effective coefficient matrix for predicting the response variables with the features. We address these problems by proposing a novel multi-output regression method, which combines sparse feature selection and low-rank linear regression in a unified framework. Specifically, we first utilize a hypergraph Laplacian regularization term to preserve the high-order structure among all the samples, and then use a low-rank constraint to respectively discover the hidden structure among the response variables and explore the relationship among different features in a least square regression framework. As a result, we integrate subspace learning with sparse feature selection to select useful features for multi-output regression. We tested our proposed method using several public data sets, and the experimental results showed that our method outperformed other comparison methods.


Similar content being viewed by others
References
Cai, X., Ding, C., Nie, F., Huang, H.: On the equivalent of low-rank linear regressions and linear discriminant analysis based regressions. In: ACM SIGKDD International conference on knowledge discovery and data mining, pp. 1124–1132 (2013)
Cai, X., Nie, F., Cai, W., Huang, H.: New graph structured sparsity model for multi-label image annotations. In: ICCV, pp. 801–808 (2014)
Chang, X., Nie, F., Yi, Y., Huang, H.: A convex formulation for semi-supervised multi-label feature selection. In: AAAI, pp. 1171–1177 (2014)
Chen, H.T., Chang, H.W., Liu, T.L.: Local discriminant embedding and its variants. In: CVPR, pp. 846–853 (2005)
Gao, L., Guo, Z., Zhang, H., Xing, X., Shen, H.T.: Video captioning with attention-based lstm and semantic consistency. IEEE Trans. Multimed. 19(9), 2045–2055 (2017)
Gao, L., Liu, X., Liu, X., Shao, J., Liu, J., Shao, J.: Learning in high-dimensional multimedia data: the state of the art. Multimed. Syst. 23(3), 303–313 (2017)
Gao, L., Wang, Y., Li, D., Shao, J., Song, J.: Real-time social media retrieval with spatial, temporal and social constraints. Neurocomputing 253, 77–88 (2017)
Gheyas, I.A., Smith, L.S.: Feature subset selection in large dimensionality domains. Pattern Recogn. 43(1), 5–13 (2010)
Guo, Y., Chen, T.: Semantic segmentation of rgbd images based on deep depth regression. Pattern Recogn. Lett. (2017) https://doi.org/https://doi.org/10.1016/j.patrec.2017.08.026
Guo, Y., Guorong, W., Jiang, J., Shen, D.: Robust anatomical correspondence detection by hierarchical sparse graph matching. IEEE Trans. Med. Imaging 32(2), 268–277 (2013)
Guo, Y., Gao, Y., Shao, Y., Price, T., Oto, A., Shen, D.: Deformable segmentation of 3d mr prostate images via distributed discriminative dictionary and ensemble learning. Med. Phys. 41, 7 (2014)
Guo, Y., Gao, Y., Shen, D.: Deformable mr prostate segmentation via deep feature learning and sparse patch matching. IEEE Trans. Med. Imaging 35(4), 1077–1089 (2016)
He, X., Cai, D., Niyogi, P.: Laplacian score for feature selection. NIPS 18, 507–514 (2005)
Ling, C.X., Yang, Q., Wang, J., Zhang, S.: Decision trees with minimal costs. In: ICML, p. 69 (2004)
Luo, D., Ding, C.H.Q., Huang, H.: Linear discriminant analysis: New formulations and overfit analysis. In: AAAI (2011)
Qian, B., Wang, X., Cao, N., Gang Jiang, Y., Davidson, I.: Learning multiple relative attributes with humans in the loop. IEEE Trans. Image Process. 23 (12), 5573–5585 (2014)
Qian, B., Wang, X., Cao, N., Li, Hi., Gang Jiang, Y.: A relative similarity based method for interactive patient risk prediction. Data Mining Knowl. Discov. 29 (4), 1070–1093 (2015)
Qian, B., Wang, X., Ye, J., Davidson, I.: A reconstruction error based framework for multi-label and multi-view learning. IEEE Trans. Knowl. Data Eng. 27 (3), 594–607 (2015)
Qin, Y., Zhang, S., Zhu, X., Zhang, J., Zhang, C.: Semi-parametric optimization for missing data imputation. Appl. Intell. 27(1), 79–88 (2007)
Rongyao, H., Zhu, X., Cheng, D., He, W., Yan, Y., Song, J., Zhang, S.: Graph self-representation method for unsupervised feature selection. Neurocomputing 220, 130–137 (2017)
Sammon, J.W.: A nonlinear mapping for data structure analysis. IEEE Trans. Comput. C-18(5), 401–409 (2006)
Schölkopf, B., Platt, J., Hofmann, T.: Learning with hypergraphs: Clustering, classification, and embedding. In: NIPS, pp. 1601–1608 (2007)
Smola, A.: Nonlinear Component Analysis as a Kernel Eigenvalue Problem. MIT Press (1998)
Song, J., Yi, Y., Zi, H., Shen, H.T., Luo, J.: Effective multiple feature hashing for large-scale near-duplicate video retrieval. IEEE Trans. Multimed. 15(8), 1997–2008 (2013)
Song, J., Gao, L., Nie, F., Shen, H.T., Yan, Y., Sebe, N.: Optimized graph learning using partial tags and multiple features for image and video annotation. IEEE Trans. Image Process. 25(11), 4999–5011 (2016)
Song, J., Gao, L., Zou, F., Yan, Y., Sebe, N.: Deep and fast: Deep learning hashing with semi-supervised graph construction. Image Vis. Comput. 55, 101–108 (2016)
Song, J., Shen, H.T., Wang, J., Zi, H., Sebe, N., Wang, J.: A distance-computation-free search scheme for binary code databases. IEEE Trans. Multimed. 18(3), 484–495 (2016)
Song, J., Gao, L., Liu, L., Zhu, X., Sebe, N.: Quantization-based hashing: A general framework for scalable image and video retrieval. Pattern Recogn. (2017)
Spyromitros-Xioufis, E., Tsoumakas, G., Groves, W., Vlahavasm I.: Multi-label classification methods for multi-target regression. arXiv preprint, 1211 (2012)
Unler, A., Murat, A., Chinnam, R.B.: Mr2pso: A maximum relevance minimum redundancy feature selection method based on swarm intelligence for support vector machine classification. Inform. Sci. 181(20), 4625–4641 (2011)
Wang, X., Qian, B., Davidson, I.: On constrained spectral clustering and its applications. Data Min. Knowl. Disc. 28(1), 1–30 (2012)
Wang, X., Gao, L., Wang, P., Sun, X., Liu, Xi.: Two-stream 3d convnet fusion for action recognition in videos with arbitrary size and length. IEEE Trans. Multimed. PP(99), 1–1 (2017)
Xie, Q., Pang, C., Zhou, X., Zhang, X., Ke, D.: Maximum error-bounded piecewise linear representation for online stream approximation. Vldb J. 23(6), 915–937 (2014)
Xie, Q., Wang, S., Zhu, J., Zhang, X.: Modeling and predicting ad progression by regression analysis of sequential clinical data. Neurocomputing 195(C), 50–55 (2016)
Xie, Q., Zhang, X., Li, Z., Zhou, X.: Optimizing cost of continuous overlapping queries over data streams by filter adaption. IEEE Trans. Knowl. Data Eng. 28(5), 1258–1271 (2016)
Xindong, W., Zhang, S.: Synthesizing high-frequency rules from different data sources. IEEE Trans. Knowl. Data Eng. 15(2), 353–367 (2003)
Xindong, W., Zhang, C., Zhang, S.: Efficient mining of both positive and negative association rules. ACM Trans. Inf. Syst. 22(3), 381–405 (2004)
Xindong, W., Zhang, C., Zhang, S.: Database classification for multi-database mining. Inf. Syst. 30(1), 71–88 (2005)
Zeng, Z., Wang, X., Zhang, J., Qun, W.: Semi-supervised feature selection based on local discriminative information. Neurocomputing 173(P1), 102–109 (2015)
Zhang, S., Wu, X., Zhang, C.: Multi-database Mining. 2, 5–13 (2003)
Zhang, S., Zhang, C., Yang, Q.: Data preparation for data mining. Appl. Artif. Intell. 17(5–6), 375–381 (2003)
Zhang, S., Qin, Z., Ling, C.X., Sheng, S.: “missing is useful”: Missing values in cost-sensitive decision trees. IEEE Trans. Knowl. Data Eng. 17(12), 1689–1693 (2005)
Zhang, S., Jin, Z., Zhu, X.: Missing data imputation by utilizing information within incomplete instances. J. Syst. Softw. 84(3), 452–459 (2011)
Zhang, S.: Decision tree classifiers sensitive to heterogeneous costs. J. Syst. Softw. 85(4), 771–779 (2012)
Zhang, S., Li, X., Zong, M., Zhu, X., Cheng, D.: Learning k for knn classification. ACM Trans. Intell. Syst. Technol. (TIST) 8(3), 43 (2017)
Zhang, S., Li, X., Zong, M., Zhu, X., Wang, R.: Efficient knn classification with different numbers of nearest neighbors. IEEE Trans Neural Netw Learn Syst. (2017) https://doi.org/10.1109/TNNLS.2017.2673241
Zhao, Y., Zhang, S.: Generalized dimension-reduction framework for recent-biased time series analysis. IEEE Trans. Knowl. Data Eng. 18(2), 231–244 (2006)
Zhu, X., Zhang, S., Jin, Z., Zhang, Z., Zhuoming, X.: Missing value estimation for mixed-attribute data sets. IEEE Trans. Knowl. Data Eng. 23(1), 110–121 (2010)
Zhu, X., Zi, H., Shen, H.T., Cheng, J., Changsheng, X.: Dimensionality reduction by mixed kernel canonical correlation analysis. Pattern Recogn. 45(8), 3003–3016 (2012)
Zhu, X., Zi, H., Cheng, H., Cui, J., Shen, H.T.: Sparse hashing for fast multimedia search. ACM Trans. Inf. Syst. 31(2), 9 (2013)
Zhu, X., Zhang, L., Zi, H.: A sparse embedding and least variance encoding approach to hashing. IEEE Trans. Image Process. 23(9), 3737–3750 (2014)
Zhu, P., Zuo, W., Zhang, L., Qinghua, H., Shiu, S.C.K.: Unsupervised feature selection by regularized self-representation. Pattern Recogn. 48(2), 438–446 (2015)
Zhu, X., Li, X., Zhang, S.: Block-row sparse multiview multilabel learning for image classification. IEEE Trans. Cybern. 46(2), 450 (2016)
Zhu, X., Suk, H.I., Lee, S.W., Shen, D.: Subspace regularized sparse multi-task learning for multi-class neurodegenerative disease identification. IEEE Trans. Biomed. Eng. 63(3), 607–618 (2016)
Zhu, X., Li, X., Zhang, S., Chunhua, J., Xindong, W.: Robust joint graph sparse coding for unsupervised spectral feature selection. IEEE Trans. Neural Netw. Learn. Syst. 28(6), 1263–1275 (2017)
Zhu, X., Li, X., Zhang, S., Zongben, X., Litao, Y., Wang, C.: Graph pca hashing for similarity search. IEEE Trans. Multimed. 19(9), 2033–2044 (2017)
Zhu, X., Suk, H.I.I., Huang, H., Shen, D.: Low-rank graph-regularized structured sparse regression for identifying genetic biomarkers. IEEE Trans. Big Data (2017). https://doi.org/10.1109/TBDATA.2017.2735991
Zhu, X., Zhang, S., Hu, R., Zhu, Y., Song, J.: Local and global structure preservation for robust unsupervised spectral feature selection. IEEE Trans. Knowl. Data Eng. PP(99), 1–1 (2017). https://doi.org/10.1109/TKDE.2017.2763618
Zhu, X., Suk, H.-I., Wang, L., Lee, S.-W., Shen, D.: A novel relational regularization feature selection method for joint regression and classification in AD diagnosis. Med. Image Anal. 38, 205–214 (2017)
Acknowledgments
This work was supported in part by the China Key Research Program (Grant No: 2016YFB1000905), the Nation Natural Science Foundation of China (Grants No: 61573270 and 61672177), the Guangxi Natural Science Foundation (Grant No: 2015GXNSFCB139011), the Guangxi High Institutions Program of Introducing 100 High-Level Overseas Talents, the Guangxi Collaborative Innovation Center of Multi-Source Information Integration and Intelligent Processing, and the Innovation Project of Guangxi Graduate Education (Grants No: YCSW2017065, XYCSZ2017064, and XYCSZ2017067).
Author information
Authors and Affiliations
Corresponding author
Additional information
This article belongs to the Topical Collection: Special Issue on Deep vs. Shallow: Learning for Emerging Web-scale Data Computing and Applications
Guest Editors: Jingkuan Song, Shuqiang Jiang, Elisa Ricci, and Zi Huang
Rights and permissions
About this article
Cite this article
Zhu, X., Hu, R., Lei, C. et al. Low-rank hypergraph feature selection for multi-output regression. World Wide Web 22, 517–531 (2019). https://doi.org/10.1007/s11280-017-0514-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11280-017-0514-5