Article

Multi-labelled classification using maximum entropy method

Authors:

Yihong GongAuthors Info & Claims

SIGIR '05: Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval

Pages 274 - 281

https://doi.org/10.1145/1076034.1076082

Published: 15 August 2005 Publication History

Abstract

Many classification problems require classifiers to assign each single document into more than one category, which is called multi-labelled classification. The categories in such problems usually are neither conditionally independent from each other nor mutually exclusive, therefore it is not trivial to directly employ state-of-the-art classification algorithms without losing information of relation among categories. In this paper, we explore correlations among categories with maximum entropy method and derive a classification algorithm for multi-labelled documents. Our experiments show that this method significantly outperforms the combination of single label approach.

References

[1]

Benson, S. J., McInnes, L. C., Moré, J., & Sarich, J. (2004). TAO user manual (revision 1.7) (Technical Report ANL/MCS-TM-242). Mathematics and Computer Science Division, Argonne National Laboratory. http://www.mcs.anl.gov/tao.]]

[2]

Cai, L., & Hofmann, T. (2004). Hierarchical document categorization with support vector machines CIKM '04: Proceedings of the Thirteenth ACM conference on Information and knowledge management (pp. 78--87). Washington, D.C., USA: ACM Press.]]

Digital Library

[3]

Chen, S. F., & Rosenfeld, R. (1999). A Gaussian prior for smoothing maximum entropy models (Technical Report CMU-CS-99-108). School of Computer Science Carnegie Mellon University.]]

[4]

Clare, A., & King, R. D. (2001). Knowledge discovery in multi-label phenotype data. PKDD '01: Proceedings of the 5th European Conference on Principles of Data Mining and Knowledge Discovery (pp. 42--53). Springer-Verlag.]]

Digital Library

[5]

Comite, F. D., Gilleron, R., & Tommasi, M. (2001). Learning multi-label alternating decision trees and applications. Proceedings of CAP'01 (pp. 195--210).]]

[6]

Crammer, K., & Singer, Y. (2002). A new family of online algorithms for category ranking. Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval (pp. 151--158). Tampere, Finland: ACM Press.]]

Digital Library

[7]

Della Pietra, S., Della Pietra, V. J., & Lafferty, J. D. (1997). Inducing features of random fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19, 380--393.]]

Digital Library

[8]

Elisseeff, A., & Weston, J. (2002). A kernel method for multi-labelled classification. Advances in Neural Information Processing Systems 14 (pp. 681--687). Cambridge, MA: MIT Press.]]

[9]

Gao, S., Wu, W., Lee, C.-H., & Chua, T.-S. (2004). A mfom learning approach to robust multiclass multi-label text categorization. ICML '04: Twenty-first international conference on Machine learning. Banff, Alberta, Canada: ACM Press.]]

Digital Library

[10]

Godbole, S., & Sarawagi, S. (2004). Discriminative methods for multi-labeled classification. PAKDD.]]

[11]

Har-Peled, S., Roth, D., & Zimak, D. Constraint classification for multiclass classification and ranking. In S. T. S. Becker and K. Obermayer (Eds.), Advances in neural information processing systems 15. MIT Press.]]

[12]

Jaynes, E. T. (1957). Information theory and statistical mechanics. Physical Review, 106, 620--630.]]

[13]

Malouf, R. (2002). A comparison of algorithms for maximum entropy parameter estimation. Proc. of the sixth CoNLL.]]

Digital Library

[14]

McCallum, A. (1999). Multi-label text classification with a mixture model trained by EM. AAAI'99 Workshop on Text Learning.]]

[15]

Nigam, K., Lafferty, J., & McCallum, A. (1999). Using maximum entropy for text classification. IJCAI-99 Workshop on Machine Learning for Information Filtering (pp. 61--67).]]

[16]

Schapire, R. E., & Singer, Y. (2000). Boostexter: A boosting-based system for text categorization. Machine Learning, 39, 135--168.]]

Digital Library

[17]

Ueda, N., & Saito, K. Parametric mixture models for multi-labeled text. Advances in Neural Information Processing Systems 15. MIT Press.]]

[18]

Wilcoxon, F. (1945). Individual comparisons by ranking methods. Biometrics, 1, 80--93.]]

[19]

Yang, Y., & Liu, X. (1999). A re-examination of text categorization methods. Proceedings of the 22nd Annual International Conference on Research and Development in Information Retrieval (SIGIR'99) (pp. 42--49). Berkley: ACM Press.]]

Digital Library

[20]

Zhang, T., & Oles, F. J. (2001). Text categorization based on regularized linear classification methods. Inf. Retr., 4, 5--31.]]

Digital Library

[21]

Zhu, J., & Hastie, T. (2003). Classification of gene microarrays by penalized logistic regression. Biostatistics.]]

Cited By

Rose AKabban CGraham SHenry WRondeau C(2025)Malware classification through Abstract Syntax Trees and L-momentsComputers and Security10.1016/j.cose.2024.104082148:COnline publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1016/j.cose.2024.104082
Lin YYu ZYang KFan ZChen C(2024)Boosting Adaptive Weighted Broad Learning System for Multi-Label LearningIEEE/CAA Journal of Automatica Sinica10.1109/JAS.2024.12455711:11(2204-2219)Online publication date: Nov-2024
https://doi.org/10.1109/JAS.2024.124557
Awal Kassim MViktor HMichalowski W(2024)Multi-Label Lifelong Machine Learning: A Scoping Review of Algorithms, Techniques, and ApplicationsIEEE Access10.1109/ACCESS.2024.340356912(74539-74557)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3403569
Show More Cited By

Index Terms

Multi-labelled classification using maximum entropy method
1. Information systems
  1. Information retrieval

Recommendations

Text Classification from Labeled and Unlabeled Documents using EM
Special issue on information retrieval

This paper shows that the accuracy of learned text classifiers can be improved by augmenting a small number of labeled training documents with a large pool of unlabeled documents. This is important because in many text classification problems obtaining ...
Weak Labeled Multi-Label Active Learning for Image Classification
MM '15: Proceedings of the 23rd ACM international conference on Multimedia

In order to achieve better classification performance with even fewer labeled images, active learning is suitable for these situations. Several active learning methods have been proposed for multi-label image classification, but all of them assume that ...
Entropy-Based Estimation in Classification Problems

The problem of binary classification is considered, an algorithm for its solution is proposed, based on the method of entropy-based estimation of the decision rule parameters. A detailed description of the entropy-based estimation method and the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '05: Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval

August 2005

708 pages

ISBN:1595930345

DOI:10.1145/1076034

General Chairs:
Ricardo Baeza-Yates
University of Chile, Chile
,
Nivio Ziviani
Federal University of Minas Gerais, Brazil
,
Program Chairs:
Gary Marchionini
University of North Carolina, USA
,
Alistair Moffat
University of Melbourne, Australia
,
John Tait
University of Sunderland, UK

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 August 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

SIGIR05

Sponsor:

SIGIR

SIGIR05: The 28th ACM/SIGIR International Symposium on Information Retrieval 2005

August 15 - 19, 2005

Salvador, Brazil

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

148
Total Citations
View Citations
2,115
Total Downloads

Downloads (Last 12 months)26
Downloads (Last 6 weeks)8

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Rose AKabban CGraham SHenry WRondeau C(2025)Malware classification through Abstract Syntax Trees and L-momentsComputers and Security10.1016/j.cose.2024.104082148:COnline publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1016/j.cose.2024.104082
Lin YYu ZYang KFan ZChen C(2024)Boosting Adaptive Weighted Broad Learning System for Multi-Label LearningIEEE/CAA Journal of Automatica Sinica10.1109/JAS.2024.12455711:11(2204-2219)Online publication date: Nov-2024
https://doi.org/10.1109/JAS.2024.124557
Awal Kassim MViktor HMichalowski W(2024)Multi-Label Lifelong Machine Learning: A Scoping Review of Algorithms, Techniques, and ApplicationsIEEE Access10.1109/ACCESS.2024.340356912(74539-74557)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3403569
Rastogi RChowdhury S(2024)Binary-Tree Based Mean-Averaging Estimation for Multi-label ClassificationPattern Recognition10.1007/978-3-031-78192-6_18(271-285)Online publication date: 4-Dec-2024
https://doi.org/10.1007/978-3-031-78192-6_18
Vancompernolle Vromman FCourtain SLeleux Pde Schaetzen CBeghein EKneip ASaerens M(2024)Maximum Entropy Logistic Regression for Demographic Parity in Supervised ClassificationArtificial Intelligence and Machine Learning10.1007/978-3-031-74650-5_11(189-208)Online publication date: 2-Nov-2024
https://doi.org/10.1007/978-3-031-74650-5_11
Mahani A(2023)Classification in Multi-Label DatasetsInformation Systems Management10.5772/intechopen.109352Online publication date: 18-Oct-2023
https://doi.org/10.5772/intechopen.109352
Malik SIdrees MDanish HAhmad AKhalid SShahzad S(2023)Classification of Call TranscriptionsVAWKUM Transactions on Computer Sciences10.21015/vtcs.v11i2.159111:2(18-34)Online publication date: 7-Oct-2023
https://doi.org/10.21015/vtcs.v11i2.1591
Zhang YLian HYang GZhao SNi PChen HLi C(2023)Inaccurate-Supervised Learning With Generative Adversarial NetsIEEE Transactions on Cybernetics10.1109/TCYB.2021.310484853:3(1522-1536)Online publication date: Mar-2023
https://doi.org/10.1109/TCYB.2021.3104848
Jiang QLi PZhang YHu X(2023)Global and Adaptive Local Label Correlation for Multi-label Learning with Missing Labels2023 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN54540.2023.10191231(1-8)Online publication date: 18-Jun-2023
https://doi.org/10.1109/IJCNN54540.2023.10191231
Villa-Blanco CBielza CLarrañaga P(2023)Feature subset selection for data and feature streams: a reviewArtificial Intelligence Review10.1007/s10462-023-10546-956:S1(1011-1062)Online publication date: 13-Jul-2023
https://doi.org/10.1007/s10462-023-10546-9
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten