Abstract
It has recently been suggested that assuming independence between labels is not suitable for real-world multi-label classification. To account for label dependencies, this paper proposes a supervised topic modeling algorithm, namely labelset topic model (LsTM). Our algorithm uses two labelset layers to capture label dependencies. LsTM offers two major advantages over existing supervised topic modeling algorithms: it is straightforward to interpret and it allows words to be assigned to combinations of labels, rather than a single label. We have performed extensive experiments on several well-known multi-label datasets. Experimental results indicate that the proposed model achieves performance on par with and often exceeding that of state-of-the-art methods both qualitatively and quantitatively.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Blei, D.M., Ng, A.Y., Jordan, M.I. (2003). Latent Dirichlet allocation. Journal of Machine Learning Research, 993–1022.
Blei, D.M., & Lafferty, J.D. (2007). A correlated topic model fo science. The Annals of Applied Statistics, 17–35.
Boutell, M.R., Luo, J., Shen, X., Browna, C.M. (2004). Learning multi-label scene classification. Pattern Recognition, 1757–1771.
Brinker, K., & Hullermeier, E. (2007). Case-based multilabel ranking. In International joint conference on artificial intelligence (pp. 702–707).
Clare, A., & King, R.D. (2001). Knowledge discovery in multi-label phenotype data. Principles of Data Mining and Dnowledge Discovery, 42–53.
Elisseeff, A. (2002). JasonWeston: a kernel method for multi-labelled classification. In Neural information processing systems.
Fan, J., Gao, Y., Luo, H. (2007). Hierarchical classifcation for automatic image annotation. In International ACM SIGIR conference on research and development in information retrieval (pp. 111–118).
Griffiths, T.L., & Steyvers, M. (2004). Finding scientific topics. In National academy of sciences of the United States of America (Vol. 101–101, pp. 5228–5235).
Guo, Y., & Gu, S. (2011). Multi-label classification using conditional dependency networks. In International joint conference on artificial intelligence (pp. 1300–1305).
Hofmann, T (1999). Probabilistic latent semantic indexing. In ACM SIGIR international conference on research and development in information retrieval (pp. 50–57).
Ji, S., Tang, L., Yu, S., Ye, J (2008). Extracting shared subspace for multi-label classification. In ACM SIGKDD international conference on knowledge discovery and data mining (pp. 381–389).
Jiang, W., & Ras, Z.W. (2013). Multi-label automatic indexing of music by cascade classifiers. Web Intelligence and Agent Systems International Journal, 149–170.
Kazawa, H., Izumitani, T., Taira, H., Maeda, E. (2004). Maximal margin labeling for multi-topic text categorization. In Neural information processing systems (pp. 649–656).
Kim, D., Kim, S., Oh, A. (2012). Dirichlet process with mixed random measures: a nonparametric topic model for labeled data. In International conference on machine learning (pp. 727–734).
Lewis, D.D., Yang, Y., Rose, T.G., Li, F. (2004). Rcv1: a new benchmark collection for text categorization research. Journal of Machine Learning Research, 361–397.
Li, T., & Ogihara, M. (2006). Towards intelligent music information retrieval. IEEE Transactions on Multimedia, 564–574.
Li, W., & McCallum, A. (2006). Pachinko allocation: dag-structured mixture models of topic correlations. In International conference on machine learning (pp. 577–584).
Nguyen, V.A., Boyd-Graber, J., Chang, J., Resnik, P. (2013). Tree-based label dependency topic models. In Neural information processins systems workshop on topic models.
Qi, G.J., Hua, X.S., Rui, Y., Tang, J., Mei, T., Zhang, H.J (2007). Correlative multi-label video annotation. In International conference on music information retrieval (pp. 17–26).
Ramage, D., Hall, D., Nallapati, R., Manning, C.D. (2009). Labeled LDA: a supervised topic model for credit attribution in multi-labeled corpora. In Conference on empirical methods in natural language processing (pp. 248–256).
Ramage, D., Manning, C.D., Dumais, S. (2011). Partially labeled topic models for interpretable text mining. In ACM SIGKDD international conference on knowledge discovery and data mining (pp. 457–465).
Read, J., Pfahringer, B., Holmes, G., Frank, E. (2011). Classifier chains for multi-label classification. Machine Learning, 1–27.
Rubin, T.N., Chambers, A., Smyth, P., Steyvers, M. (2012). Statistical topic models for multi-label document classification. Machine learning, 157–208.
Teh, Y.W., Jordan, M.I., Beal, M.J., Blei, D.M. (2006). Hierarchical Dirichlet processes. Journal of the American Statistical Association, 1566–1581.
Trohidis, K., Tsoumakas, G., Kalliris, G., Vlahavas, I. (2008). Multilabel classification of music into emotions. In International conference on music information retrieval.
Tsoumakas, G., & Katakis, I. (2007). Multi label classification: an overview. International Journal of Data Warehousing and Mining, 1–13.
Tsoumakas, G., Katakis, I., Vlahavas, I. (2011). Random k-labelsets for multi-label classification. IEEE Transactions on Knowledge and Data Engineering, 1079–1089.
Tsoumakas, G., Spyromitros-Xioufis, E., Vilcek, J., Vlahavas, I. (2011). Mulan: a java library for multi-label learning. Journal of Machine Learning Research, 2411–2414.
Ueda, N., & Saito, K. (2002). Single-shot detection of multiple categories of text using parametric mixture models. In ACM SIGKDD international conference on knowledge discovery and data mining (pp. 626–631).
Wang, C., Yan, S., Zhang, L., Zhang, H.J. (2009). Multi-label sparse coding for automatic image annotation. In IEEE conference on computer vision and pattern recognition (pp. 1643–1650).
Yuret, D., Yatbaz, M.A., Ural, A.E. (2008). Discriminative vs. generative approaches in semantic role labeling. In Conference on computational natural language learning (pp. 223–227).
Zhang, M.L., & Zhou, Z.H. (2006). Multi-label neural networks with applications to functional genomics and text categorization. IEEE Transactions on Knowledge and Data Engineering, 1338–1351.
Zhang, Y., Burer, S., Street, W.N. (2006). Ensemble pruning via semi-definite programming. Journal of Machine Learning Research, 1315–1338.
Zhang, M.L., & Zhou, Z.H. (2007). Ml-knn: a lazy learning approach to multi-label learning. Pattern Recognition, 2038–2048.
Zhang, M.L. (2009). Ml-rbf: Rbf neural networks for multi-label learning. Neural Processing Letters, 61–74.
Acknowledgments
This work was supported by National Nature Science Foundation of China (NSFC) under the Grant No. 61170092, 61133011, and 61103091.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Li, X., Ouyang, J. & Zhou, X. Labelset topic model for multi-label document classification. J Intell Inf Syst 46, 83–97 (2016). https://doi.org/10.1007/s10844-014-0352-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10844-014-0352-1