Skip to main content
SpringerLink
Log in

Low-rank hypergraph feature selection for multi-output regression

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

Current multi-output regression method usually ignores the relationship among response variables, and thus it is challenging to obtain an effective coefficient matrix for predicting the response variables with the features. We address these problems by proposing a novel multi-output regression method, which combines sparse feature selection and low-rank linear regression in a unified framework. Specifically, we first utilize a hypergraph Laplacian regularization term to preserve the high-order structure among all the samples, and then use a low-rank constraint to respectively discover the hidden structure among the response variables and explore the relationship among different features in a least square regression framework. As a result, we integrate subspace learning with sparse feature selection to select useful features for multi-output regression. We tested our proposed method using several public data sets, and the experimental results showed that our method outperformed other comparison methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2

Similar content being viewed by others

Notes

  1. http://archive.ics.uci.edu/ml/.

References

  1. Cai, X., Ding, C., Nie, F., Huang, H.: On the equivalent of low-rank linear regressions and linear discriminant analysis based regressions. In: ACM SIGKDD International conference on knowledge discovery and data mining, pp. 1124–1132 (2013)

  2. Cai, X., Nie, F., Cai, W., Huang, H.: New graph structured sparsity model for multi-label image annotations. In: ICCV, pp. 801–808 (2014)

  3. Chang, X., Nie, F., Yi, Y., Huang, H.: A convex formulation for semi-supervised multi-label feature selection. In: AAAI, pp. 1171–1177 (2014)

  4. Chen, H.T., Chang, H.W., Liu, T.L.: Local discriminant embedding and its variants. In: CVPR, pp. 846–853 (2005)

  5. Gao, L., Guo, Z., Zhang, H., Xing, X., Shen, H.T.: Video captioning with attention-based lstm and semantic consistency. IEEE Trans. Multimed. 19(9), 2045–2055 (2017)

    Article  Google Scholar 

  6. Gao, L., Liu, X., Liu, X., Shao, J., Liu, J., Shao, J.: Learning in high-dimensional multimedia data: the state of the art. Multimed. Syst. 23(3), 303–313 (2017)

    Article  Google Scholar 

  7. Gao, L., Wang, Y., Li, D., Shao, J., Song, J.: Real-time social media retrieval with spatial, temporal and social constraints. Neurocomputing 253, 77–88 (2017)

    Article  Google Scholar 

  8. Gheyas, I.A., Smith, L.S.: Feature subset selection in large dimensionality domains. Pattern Recogn. 43(1), 5–13 (2010)

    Article  MATH  Google Scholar 

  9. Guo, Y., Chen, T.: Semantic segmentation of rgbd images based on deep depth regression. Pattern Recogn. Lett. (2017) https://doi.org/https://doi.org/10.1016/j.patrec.2017.08.026

  10. Guo, Y., Guorong, W., Jiang, J., Shen, D.: Robust anatomical correspondence detection by hierarchical sparse graph matching. IEEE Trans. Med. Imaging 32(2), 268–277 (2013)

    Article  Google Scholar 

  11. Guo, Y., Gao, Y., Shao, Y., Price, T., Oto, A., Shen, D.: Deformable segmentation of 3d mr prostate images via distributed discriminative dictionary and ensemble learning. Med. Phys. 41, 7 (2014)

    Google Scholar 

  12. Guo, Y., Gao, Y., Shen, D.: Deformable mr prostate segmentation via deep feature learning and sparse patch matching. IEEE Trans. Med. Imaging 35(4), 1077–1089 (2016)

    Article  Google Scholar 

  13. He, X., Cai, D., Niyogi, P.: Laplacian score for feature selection. NIPS 18, 507–514 (2005)

    Google Scholar 

  14. Ling, C.X., Yang, Q., Wang, J., Zhang, S.: Decision trees with minimal costs. In: ICML, p. 69 (2004)

  15. Luo, D., Ding, C.H.Q., Huang, H.: Linear discriminant analysis: New formulations and overfit analysis. In: AAAI (2011)

  16. Qian, B., Wang, X., Cao, N., Gang Jiang, Y., Davidson, I.: Learning multiple relative attributes with humans in the loop. IEEE Trans. Image Process. 23 (12), 5573–5585 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  17. Qian, B., Wang, X., Cao, N., Li, Hi., Gang Jiang, Y.: A relative similarity based method for interactive patient risk prediction. Data Mining Knowl. Discov. 29 (4), 1070–1093 (2015)

    Article  MathSciNet  Google Scholar 

  18. Qian, B., Wang, X., Ye, J., Davidson, I.: A reconstruction error based framework for multi-label and multi-view learning. IEEE Trans. Knowl. Data Eng. 27 (3), 594–607 (2015)

    Article  Google Scholar 

  19. Qin, Y., Zhang, S., Zhu, X., Zhang, J., Zhang, C.: Semi-parametric optimization for missing data imputation. Appl. Intell. 27(1), 79–88 (2007)

    Article  MATH  Google Scholar 

  20. Rongyao, H., Zhu, X., Cheng, D., He, W., Yan, Y., Song, J., Zhang, S.: Graph self-representation method for unsupervised feature selection. Neurocomputing 220, 130–137 (2017)

    Article  Google Scholar 

  21. Sammon, J.W.: A nonlinear mapping for data structure analysis. IEEE Trans. Comput. C-18(5), 401–409 (2006)

    Article  Google Scholar 

  22. Schölkopf, B., Platt, J., Hofmann, T.: Learning with hypergraphs: Clustering, classification, and embedding. In: NIPS, pp. 1601–1608 (2007)

  23. Smola, A.: Nonlinear Component Analysis as a Kernel Eigenvalue Problem. MIT Press (1998)

  24. Song, J., Yi, Y., Zi, H., Shen, H.T., Luo, J.: Effective multiple feature hashing for large-scale near-duplicate video retrieval. IEEE Trans. Multimed. 15(8), 1997–2008 (2013)

    Article  Google Scholar 

  25. Song, J., Gao, L., Nie, F., Shen, H.T., Yan, Y., Sebe, N.: Optimized graph learning using partial tags and multiple features for image and video annotation. IEEE Trans. Image Process. 25(11), 4999–5011 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  26. Song, J., Gao, L., Zou, F., Yan, Y., Sebe, N.: Deep and fast: Deep learning hashing with semi-supervised graph construction. Image Vis. Comput. 55, 101–108 (2016)

    Article  Google Scholar 

  27. Song, J., Shen, H.T., Wang, J., Zi, H., Sebe, N., Wang, J.: A distance-computation-free search scheme for binary code databases. IEEE Trans. Multimed. 18(3), 484–495 (2016)

    Article  Google Scholar 

  28. Song, J., Gao, L., Liu, L., Zhu, X., Sebe, N.: Quantization-based hashing: A general framework for scalable image and video retrieval. Pattern Recogn. (2017)

  29. Spyromitros-Xioufis, E., Tsoumakas, G., Groves, W., Vlahavasm I.: Multi-label classification methods for multi-target regression. arXiv preprint, 1211 (2012)

  30. Unler, A., Murat, A., Chinnam, R.B.: Mr2pso: A maximum relevance minimum redundancy feature selection method based on swarm intelligence for support vector machine classification. Inform. Sci. 181(20), 4625–4641 (2011)

    Article  Google Scholar 

  31. Wang, X., Qian, B., Davidson, I.: On constrained spectral clustering and its applications. Data Min. Knowl. Disc. 28(1), 1–30 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  32. Wang, X., Gao, L., Wang, P., Sun, X., Liu, Xi.: Two-stream 3d convnet fusion for action recognition in videos with arbitrary size and length. IEEE Trans. Multimed. PP(99), 1–1 (2017)

    Google Scholar 

  33. Xie, Q., Pang, C., Zhou, X., Zhang, X., Ke, D.: Maximum error-bounded piecewise linear representation for online stream approximation. Vldb J. 23(6), 915–937 (2014)

    Article  Google Scholar 

  34. Xie, Q., Wang, S., Zhu, J., Zhang, X.: Modeling and predicting ad progression by regression analysis of sequential clinical data. Neurocomputing 195(C), 50–55 (2016)

    Article  Google Scholar 

  35. Xie, Q., Zhang, X., Li, Z., Zhou, X.: Optimizing cost of continuous overlapping queries over data streams by filter adaption. IEEE Trans. Knowl. Data Eng. 28(5), 1258–1271 (2016)

    Article  Google Scholar 

  36. Xindong, W., Zhang, S.: Synthesizing high-frequency rules from different data sources. IEEE Trans. Knowl. Data Eng. 15(2), 353–367 (2003)

    Article  Google Scholar 

  37. Xindong, W., Zhang, C., Zhang, S.: Efficient mining of both positive and negative association rules. ACM Trans. Inf. Syst. 22(3), 381–405 (2004)

    Article  Google Scholar 

  38. Xindong, W., Zhang, C., Zhang, S.: Database classification for multi-database mining. Inf. Syst. 30(1), 71–88 (2005)

    Article  MATH  Google Scholar 

  39. Zeng, Z., Wang, X., Zhang, J., Qun, W.: Semi-supervised feature selection based on local discriminative information. Neurocomputing 173(P1), 102–109 (2015)

    Google Scholar 

  40. Zhang, S., Wu, X., Zhang, C.: Multi-database Mining. 2, 5–13 (2003)

    Google Scholar 

  41. Zhang, S., Zhang, C., Yang, Q.: Data preparation for data mining. Appl. Artif. Intell. 17(5–6), 375–381 (2003)

    Article  Google Scholar 

  42. Zhang, S., Qin, Z., Ling, C.X., Sheng, S.: “missing is useful”: Missing values in cost-sensitive decision trees. IEEE Trans. Knowl. Data Eng. 17(12), 1689–1693 (2005)

    Article  Google Scholar 

  43. Zhang, S., Jin, Z., Zhu, X.: Missing data imputation by utilizing information within incomplete instances. J. Syst. Softw. 84(3), 452–459 (2011)

    Article  Google Scholar 

  44. Zhang, S.: Decision tree classifiers sensitive to heterogeneous costs. J. Syst. Softw. 85(4), 771–779 (2012)

    Article  Google Scholar 

  45. Zhang, S., Li, X., Zong, M., Zhu, X., Cheng, D.: Learning k for knn classification. ACM Trans. Intell. Syst. Technol. (TIST) 8(3), 43 (2017)

    Google Scholar 

  46. Zhang, S., Li, X., Zong, M., Zhu, X., Wang, R.: Efficient knn classification with different numbers of nearest neighbors. IEEE Trans Neural Netw Learn Syst. (2017) https://doi.org/10.1109/TNNLS.2017.2673241

  47. Zhao, Y., Zhang, S.: Generalized dimension-reduction framework for recent-biased time series analysis. IEEE Trans. Knowl. Data Eng. 18(2), 231–244 (2006)

    Article  Google Scholar 

  48. Zhu, X., Zhang, S., Jin, Z., Zhang, Z., Zhuoming, X.: Missing value estimation for mixed-attribute data sets. IEEE Trans. Knowl. Data Eng. 23(1), 110–121 (2010)

    Article  Google Scholar 

  49. Zhu, X., Zi, H., Shen, H.T., Cheng, J., Changsheng, X.: Dimensionality reduction by mixed kernel canonical correlation analysis. Pattern Recogn. 45(8), 3003–3016 (2012)

    Article  MATH  Google Scholar 

  50. Zhu, X., Zi, H., Cheng, H., Cui, J., Shen, H.T.: Sparse hashing for fast multimedia search. ACM Trans. Inf. Syst. 31(2), 9 (2013)

    Article  Google Scholar 

  51. Zhu, X., Zhang, L., Zi, H.: A sparse embedding and least variance encoding approach to hashing. IEEE Trans. Image Process. 23(9), 3737–3750 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  52. Zhu, P., Zuo, W., Zhang, L., Qinghua, H., Shiu, S.C.K.: Unsupervised feature selection by regularized self-representation. Pattern Recogn. 48(2), 438–446 (2015)

    Article  MATH  Google Scholar 

  53. Zhu, X., Li, X., Zhang, S.: Block-row sparse multiview multilabel learning for image classification. IEEE Trans. Cybern. 46(2), 450 (2016)

    Article  Google Scholar 

  54. Zhu, X., Suk, H.I., Lee, S.W., Shen, D.: Subspace regularized sparse multi-task learning for multi-class neurodegenerative disease identification. IEEE Trans. Biomed. Eng. 63(3), 607–618 (2016)

    Article  Google Scholar 

  55. Zhu, X., Li, X., Zhang, S., Chunhua, J., Xindong, W.: Robust joint graph sparse coding for unsupervised spectral feature selection. IEEE Trans. Neural Netw. Learn. Syst. 28(6), 1263–1275 (2017)

    Article  MathSciNet  Google Scholar 

  56. Zhu, X., Li, X., Zhang, S., Zongben, X., Litao, Y., Wang, C.: Graph pca hashing for similarity search. IEEE Trans. Multimed. 19(9), 2033–2044 (2017)

    Article  Google Scholar 

  57. Zhu, X., Suk, H.I.I., Huang, H., Shen, D.: Low-rank graph-regularized structured sparse regression for identifying genetic biomarkers. IEEE Trans. Big Data (2017). https://doi.org/10.1109/TBDATA.2017.2735991

  58. Zhu, X., Zhang, S., Hu, R., Zhu, Y., Song, J.: Local and global structure preservation for robust unsupervised spectral feature selection. IEEE Trans. Knowl. Data Eng. PP(99), 1–1 (2017). https://doi.org/10.1109/TKDE.2017.2763618

    Article  Google Scholar 

  59. Zhu, X., Suk, H.-I., Wang, L., Lee, S.-W., Shen, D.: A novel relational regularization feature selection method for joint regression and classification in AD diagnosis. Med. Image Anal. 38, 205–214 (2017)

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the China Key Research Program (Grant No: 2016YFB1000905), the Nation Natural Science Foundation of China (Grants No: 61573270 and 61672177), the Guangxi Natural Science Foundation (Grant No: 2015GXNSFCB139011), the Guangxi High Institutions Program of Introducing 100 High-Level Overseas Talents, the Guangxi Collaborative Innovation Center of Multi-Source Information Integration and Intelligent Processing, and the Innovation Project of Guangxi Graduate Education (Grants No: YCSW2017065, XYCSZ2017064, and XYCSZ2017067).

Author information

Authors and Affiliations

  1. Guangxi Key Lab of Multi-source Information Mining and Security, Guangxi Normal University, Guilin, 541004, Guangxi, People’s Republic of China

    Xiaofeng Zhu, Rongyao Hu, Cong Lei & Wei Zheng

  2. BRIC Center of the University of North Carolina at Chapel Hill, Chapel Hill, NC, 27599, USA

    Kim Han Thung

  3. School of Information and Communication Technology, Griffith University, Gold Coast Campus, Queensland, QLD 4222, Australia

    Can Wang

Authors
  1. Xiaofeng Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rongyao Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cong Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kim Han Thung
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wei Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Can Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kim Han Thung.

Additional information

This article belongs to the Topical Collection: Special Issue on Deep vs. Shallow: Learning for Emerging Web-scale Data Computing and Applications

Guest Editors: Jingkuan Song, Shuqiang Jiang, Elisa Ricci, and Zi Huang

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, X., Hu, R., Lei, C. et al. Low-rank hypergraph feature selection for multi-output regression. World Wide Web 22, 517–531 (2019). https://doi.org/10.1007/s11280-017-0514-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-017-0514-5

Keywords

Search

Navigation