Skip to main content
SpringerLink
Log in

Multilabel Text Classification of Unbalanced Datasets: Two-Pass NNMF

  • Conference paper
  • First Online:
Computational Linguistics and Intelligent Text Processing (CICLing 2018)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13397))

  • 198 Accesses

Abstract

The natural distribution of textual data used in text classification is often imbalanced. Categories with fewer examples are under-represented and their classifiers trained on the datasets transformed to bag-of-words representations or basic topic modeling transformations often perform far below a satisfactory level. We tackle this problem using a two-pass non-negative matrix factorization algorithm. This approach finds topics for each category independently allowing to better define topics for underrepresented categories. The results are analyzed from multiple goal perspectives - H-loss, accuracy, F-measure, precision, and recall, from the micro, macro and example-based aspect since each is appropriate in different situations. Through experimental validation, it is shown that the two-pass matrix factorization improves classification results achieved using bag-of-words representations.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The dataset can be downloaded at: http://mulan.sourceforge.net/datasets.html.

  2. 2.

    https://radimrehurek.com/gensim/.

  3. 3.

    http://scikit-learn.org/.

References

  1. Schapire, R.E., Singer, Y.: BoosTexter: a boosting-based system for text categorization. Mach. Learn. 39, 135–168 (2000)

    Article  Google Scholar 

  2. Godbole, S., Sarawagi, S.: Discriminative methods for multi-labeled classification. In: Dai, H., Srikant, R., Zhang, C. (eds.) PAKDD 2004. LNCS (LNAI), vol. 3056, pp. 22–30. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24775-3_5

    Chapter  Google Scholar 

  3. Boutell, M.R., Luo, J., Shen, X., Brown, C.M.: Learning multi-label scene classification. Pattern Recogn. 37, 1757–1771 (2004)

    Article  Google Scholar 

  4. Fürnkranz, J., Hüllermeier, E., Loza Mencía, E., Brinker, K.: Multilabel classification via calibrated label ranking. Mach. Learn. 73, 133–153 (2008)

    Article  Google Scholar 

  5. Dimou, A., Tsoumakas, G., Mezaris, V., Kompatsiaris, I., Vlahavas, I.: An empirical study of multi-label learning methods for video annotation. In: 7th International Workshop on Content-Based Multimedia Indexing, CBMI 2009, pp. 19–24 (2009)

    Google Scholar 

  6. Madjarov, G., Kocev, D., Gjorgjevikj, D., Džeroski, S.: An extensive experimental comparison of methods for multi-label learning. Pattern Recogn. 45, 3084–3104 (2012)

    Article  Google Scholar 

  7. Moyano, J.M., Gibaja, E.L., Cios, K.J., Ventura, S.: Review of ensembles of multi-label classifiers: models, experimental study and prospects. Inf. Fusion 44, 33–45 (2018)

    Article  Google Scholar 

  8. Ramage, D., Hall, D., Nallapati, R., Manning, C.D.: Labeled LDA: a supervised topic model for credit attribution in multi-labeled corpora. In: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pp. 248–256 (2009)

    Google Scholar 

  9. Rubin, T.N., Chambers, A., Smyth, P., Steyvers, M.: Statistical topic models for multi-label document classification. Mach. Learn. 88, 157–208 (2012)

    Article  Google Scholar 

  10. Ma, H., Chen, E., Xu, L., Xiong, H.: Capturing correlations of multiple labels: a generative probabilistic model for multi-label learning. Neurocomputing 92, 116–123 (2012)

    Article  Google Scholar 

  11. Skitalinskaya, G., Alexandrov, M., Cardiff, J.: Comparison of two-pass algorithms for dynamic topic modeling based on matrix decompositions. In: Castro, F., Miranda-Jiménez, S., González-Mendoza, M. (eds.) MICAI 2017. LNCS (LNAI), vol. 10633, pp. 27–43. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-02840-4_3

    Chapter  Google Scholar 

  12. Tsoumakas, G., Katakis, I., Overview, A.: Multi-label classification: an overview. Int. J. Data Warehousing Mining 3, 1–13 (2007)

    Article  Google Scholar 

  13. Read, J., Pfahringer, B., Holmes, G., Frank, E.: Classifier chains for multi-label classification. Mach. Learn. 85, 333–359 (2011)

    Article  Google Scholar 

  14. Tsoumakas, G., Katakis, I., Vlahavas, I.: Random k-labelsets for multilabel classification. IEEE Trans. Knowl. Data Eng. 23, 1079–1089 (2011)

    Article  Google Scholar 

  15. Guo, Y., Gu, S.: Multi-label classification using conditional dependency networks. In: Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence, vol. 2, pp. 1300–1305 (2011)

    Google Scholar 

  16. Elisseeff, A.: Kernel methods for multi-labelled classification and categorical regression problems. In: Advances in Neural Information Processing, pp. 1–18 (2002)

    Google Scholar 

  17. Clare, A., King, R.D.: Knowledge discovery in multi-label phenotype data. In: De Raedt, L., Siebes, A. (eds.) PKDD 2001. LNCS (LNAI), vol. 2168, pp. 42–53. Springer, Heidelberg (2001). https://doi.org/10.1007/3-540-44794-6_4

    Chapter  Google Scholar 

  18. Zhang, M.L., Zhou, Z.H.: ML-KNN: a lazy learning approach to multi-label learning. Pattern Recogn. 40, 2038–2048 (2007)

    Article  Google Scholar 

  19. Zhang, M.L.: Ml-rbf: RBF neural networks for multi-label learning. Neural Process. Lett. 29, 61–74 (2009)

    Article  Google Scholar 

  20. Tsoumakas, G., Katakis, I., Vlahavas, I.: Effective and efficient multilabel classification in domains with large number of labels. In: Proceedings of ECML/PKDD 2008 Workshop on Mining Multidimensional Data (MMD 2008), pp. 30–44 (2008)

    Google Scholar 

  21. Read, J., Pfahringer, B., Holmes, G.: Multi-label classification using ensembles of pruned sets. In: Proceedings - IEEE International Conference on Data Mining, ICDM, pp. 995–1000 (2008)

    Google Scholar 

  22. He, J., Gu, H., Liu, W.: Imbalanced multi-modal multi-label learning for subcellular localization prediction of human proteins with both single and multiple sites. PLoS ONE 7, e37155 (2012)

    Google Scholar 

  23. Li, C., Shi, G.: Improvement of learning algorithm for the multi-instance multi-label RBF neural networks trained with imbalanced samples. J. Inf. Sci. Eng. 29, 765–776 (2013)

    Google Scholar 

  24. Tahir, M.A., Kittler, J., Bouridane, A.: Multilabel classification using heterogeneous ensemble of multi-label classifiers. Pattern Recogn. Lett. 33, 513–523 (2012)

    Article  Google Scholar 

  25. Charte, F., Rivera, A.J., del Jesus, M.J., Herrera, F.: Addressing imbalance in multilabel classification: measures and random resampling algorithms. Neurocomputing 163, 3–16 (2015)

    Article  Google Scholar 

  26. Giraldo-Forero, A.F., Jaramillo-Garzón, J.A., Ruiz-Muñoz, J.F., Castellanos-Domínguez, C.G.: Managing imbalanced data sets in multi-label problems: a case study with the SMOTE algorithm. In: Ruiz-Shulcloper, J., Sanniti di Baja, G. (eds.) CIARP 2013. LNCS, vol. 8258, pp. 334–342. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-41822-8_42

    Chapter  Google Scholar 

  27. Zhang, M.L., Li, Y.K., Liu, X.Y.: Towards class-imbalance aware multi-label learning. In: IJCAI International Joint Conference on Artificial Intelligence, vol. 2015-January, pp. 4041–4047 (2015)

    Google Scholar 

  28. García, V., Sánchez, J.S., Mollineda, R.A.: On the effectiveness of preprocessing methods when dealing with different levels of class imbalance. Knowl.-Based Syst. 25, 13–21 (2012)

    Article  Google Scholar 

  29. Charte, F., Rivera, A.J., del Jesus, M.J., Herrera, F.: Mlsmote: approaching imbalanced multilabel learning through synthetic instance generation. Knowl.-Based Syst. 89, 385–397 (2015)

    Article  Google Scholar 

  30. Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999)

    Article  Google Scholar 

  31. Newman, D., Lau, J.H., Grieser, K., Baldwin, T.: Automatic evaluation of topic coherence. In: Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pp. 100–108. Association for Computational Linguistics (2010)

    Google Scholar 

  32. Bouma, G.: Normalized (Pointwise) mutual information in collocation extraction. In: Proceedings of German Society for Computational Linguistics (GSCL 2009), pp. 31–40 (2009)

    Google Scholar 

  33. Aletras, N., Stevenson, M.: Evaluating topic coherence using distributional semantics. In: Proceedings of the 10th International Conference on Computational Semantics (IWCS 2013)-Long Papers, pp. 13–22 (2013)

    Google Scholar 

  34. O’Callaghan, D., Greene, D., Carthy, J., Cunningham, P.: An analysis of the coherence of descriptors in topic modeling. Expert Syst. Appl. 13, 5645–5657 (2015)

    Article  Google Scholar 

  35. Mikolov, T., Corrado, G., Chen, K., Dean, J.: Efficient estimation of word representations in vector space. In: Proceedings of the International Conference on Learning Representations (ICLR 2013), pp. 1–12 (2013)

    Google Scholar 

  36. Sebastiani, F.: Machine learning in automated text categorization. ACM Comput. Surv. 34, 1–47 (2002)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Technology Tallaght, Dublin, Ireland

    Gabriella Skitalinskaya & John Cardiff

Authors
  1. Gabriella Skitalinskaya
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. John Cardiff
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gabriella Skitalinskaya .

Editor information

Editors and Affiliations

  1. Instituto Politécnico Nacional, Mexico City, Mexico

    Alexander Gelbukh

Rights and permissions

Reprints and permissions

Copyright information

© 2023 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Skitalinskaya, G., Cardiff, J. (2023). Multilabel Text Classification of Unbalanced Datasets: Two-Pass NNMF. In: Gelbukh, A. (eds) Computational Linguistics and Intelligent Text Processing. CICLing 2018. Lecture Notes in Computer Science, vol 13397. Springer, Cham. https://doi.org/10.1007/978-3-031-23804-8_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-23804-8_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-23803-1

  • Online ISBN: 978-3-031-23804-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation