Liang Sun
ABSTRACT
A hypergraph is a generalization of the traditional graph in which the edges are arbitrary non-empty subsets of the vertex set. It has been applied successfully to capture high-order relations in various domains. In this paper, we propose a hypergraph spectral learning formulation for multi-label classification, where a hypergraph is constructed to exploit the correlation information among different labels. We show that the proposed formulation leads to an eigenvalue problem, which may be computationally expensive especially for large-scale problems. To reduce the computational cost, we propose an approximate formulation, which is shown to be equivalent to a least squares problem under a mild condition. Based on the approximate formulation, efficient algorithms for solving least squares problems can be applied to scale the formulation to very large data sets. In addition, existing regularization techniques for least squares can be incorporated into the model for improved generalization performance. We have conducted experiments using large-scale benchmark data sets, and experimental results show that the proposed hypergraph spectral learning formulation is effective in capturing the high-order relations in multi-label problems. Results also indicate that the approximate formulation is much more efficient than the original one, while keeping competitive classification performance.
- S. Agarwal, K. Branson, and S. Belongie. Higher order learning with graphs. In ICML, pages 17--24, 2006. Google Scholar
Digital Library
- M. Belkin and P. Niyogi. Laplacian eigenmaps and spectral techniques for embedding and clustering. In Advances in Neural Information Processing Systems 14, pages 585--591, 2001.Google Scholar
- C. M. Bishop. Pattern Recognition and Machine Learning. Springer, New York, 2006. Google ScholarDigital Library
- M. R. Boutell, J. Luo, X. Shen, and C. M. Brown. Learning multi-label scene classification. Pattern Recognition, 37(9):1757--1771, 2004.Google ScholarCross Ref
- D. Cai, X. He, and J. Han. SRDA: An efficient algorithm for large-scale discriminant analysis. IEEE Transactions on Knowledge and Data Engineering, 20(1):1--12, 2008. Google ScholarDigital Library
- O. Chapelle, B. Scholkopf, and A. Zien. Semi-Supervised Learning. MIT Press, Cambridge, MA, 2006. Google ScholarDigital Library
- F. R. K. Chung. Spectral Graph Theory. American Mathematical Society, 1997.Google Scholar
- A. Edelman, T. A. Arias, and S. T. Smith. The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl., 20(2):303--353, 1999. Google ScholarDigital Library
- A. Elisseeff and J. Weston. A kernel method for multi-labelled classification. In Advances in Neural Information Processing Systems 14, pages 681--687, 2001.Google Scholar
- G. H. Golub and C. F. V. Loan. Matrix Computations. Johns Hopkins Press, Baltimore, MD, 1996.Google Scholar
- D. R. Hardoon, S. R. Szedmak, and J. R. Shawe-taylor. Canonical correlation analysis: An overview with application to learning methods. Neural Comput., 16(12):2639--2664, 2004. Google ScholarDigital Library
- T. Hastie, R. Tibshirani, and J. H. Friedman. The Elements of Statistical Learning. Springer, New York, NY, 2001.Google ScholarCross Ref
- A. E. Hoerl and R. W. Kennard. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1):55--67, 1970.Google ScholarCross Ref
- H. Hotelling. Relations between two sets of variables. Biometrika, 28:312--377, 1936.Google ScholarCross Ref
- F. Kang, R. Jin, and R. Sukthankar. Correlated label propagation with application to multi-label learning. In Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 1719--1726, 2006. Google ScholarDigital Library
- H. Kazawa, T. Izumitani, H. Taira, and E. Maeda. Maximal margin labeling for multi-topic text categorization. In Advances in Neural Information Processing Systems 17, pages 649--656. 2005.Google Scholar
- W. Noble. Support vector machine applications in computational biology. Kernel Methods in Computational Biology. B. Schoelkopf, K. Tsuda and J.-P. Vert, ed. MIT Press, pages 71--92, 2004.Google Scholar
- C. C. Paige and M. A. Saunders. LSQR: An algorithm for sparse linear equations and sparse least squares. ACM Transactions on Mathematical Software, 8(1):43--71, 1982. Google ScholarDigital Library
- J. A. Rodriguez. On the laplacian spectrum and walk-regular hypergraphs. Linear and Multilinear Algebra,:285--297, 2003.Google Scholar
- A. Torralba, K. P. Murphy, and W. T. Freeman. Sharing visual features for multiclass and multiview object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(5):854--869, 2007. Google ScholarDigital Library
- N. Ueda and K. Saito. Parametric mixture models for multi-labeled text. In Advances in Neural Information Processing Systems 16, pages 721--728. 2003.Google Scholar
- R. Yan, J. Tesic, and J. R. Smith. Model-shared subspace boosting for multi-label classification. In Proceedings of the thirteenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 834--843, 2007. Google ScholarDigital Library
- Y. Yang and J. O. Pedersen. A comparative study on feature selection in text categorization. In ICML, pages 412--420, 1997. Google ScholarDigital Library
- J. Ye. Least squares linear discriminant analysis. In ICML, pages 1087--1094, 2007. Google ScholarDigital Library
- S. Yu, K. Yu, V. Tresp, and H.-P. Kriegel. Multi-output regularized feature projection. IEEE Transactions on Knowledge and Data Engineering, 18(12):1600--1613, 2006. Google ScholarDigital Library
- M.-L. Zhang and Z.-H. Zhou. Multilabel neural networks with applications to functional genomics and text categorization. IEEE Transactions on Knowledge and Data Engineering, 18(10):1338--1351, 2006. Google ScholarDigital Library
- M.-L. Zhang and Z.-H. Zhou. Ml-knn: A lazy learning approach to multi-label learning. Pattern Recognition, 40(7):2038--2048, 2007. Google ScholarDigital Library
- D. Zhou, J. Huang, and B. Scholkopf. Learning with hypergraphs: Clustering, classification, and embedding. In Advances in Neural Information Processing Systems 19, pages 1601--1608, 2007.Google Scholar
- J. Zien, M. Schlag, and P. Chan. Multilevel spectral hypergraph partitioning with arbitrary vertex sizes. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 18(9):1389--1399, 1999. Google ScholarDigital Library
Index Terms
- Hypergraph spectral learning for multi-label classification
Recommendations
Learning label-specific features with global and local label correlation for multi-label classification
AbstractMulti-label algorithms often use an identical feature space to build classification models for all labels. However, labels generally express different semantic information and should have their own characteristics. A few algorithms have been ...
Multi-label classification by exploiting label correlations
Nowadays, multi-label classification methods are of increasing interest in the areas such as text categorization, image annotation and protein function classification. Due to the correlation among the labels, traditional single-label classification ...
Improving multi-label classification using semi-supervised learning and dimensionality reduction
PRICAI'12: Proceedings of the 12th Pacific Rim international conference on Trends in Artificial IntelligenceMulti-label classification has been increasingly recognized since it can assign multiple class labels to an object. This paper proposes a new method to solve simultaneously two major problems in multi-label classification; (1) requirement of sufficient ...
Comments