Skip to main content
SpringerLink
Log in

Rule-Based Multi-label Classification: Challenges and Opportunities

  • Conference paper
  • First Online:
Rules and Reasoning (RuleML+RR 2020)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 12173))

Included in the following conference series:

Abstract

In the context of multi-label classification (MLC), rule-based learning algorithms have a number of appealing properties that are not, at least not as a whole, shared by other approaches. This includes the potential interpretability of rules, their ability to model (local) label dependencies in a flexible way, and the facile customization of a predictor to different loss functions. In this paper, we present a modular framework for rule-based MLC and discuss related challenges and opportunities for multi-label rule learning.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    \(\llbracket \cdot \rrbracket \!\) is the indicator function, i.e., \(\llbracket A \rrbracket = 1\) if the predicate A is true and \(=0\) otherwise.

References

  • Allahyari, H., Lavesson, N.: User-oriented assessment of classification model understandability. In: Proceedings of 11th Scandinavian Conference on Artificial Intelligence (SCAI), pp. 11–19. IOS Press (2011)

    Google Scholar 

  • Allamanis, M., Tzima, F.A., Mitkas, P.A.: Effective rule-based multi-label classification with learning classifier systems. In: Tomassini, M., Antonioni, A., Daolio, F., Buesser, P. (eds.) ICANNGA 2013. LNCS, vol. 7824, pp. 466–476. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-37213-1_48

    Chapter  Google Scholar 

  • Arunadevi, J., Rajamani, V.: An evolutionary multi label classification using associative rule mining. Int. J. Soft Comput. 6, 20–25 (2011)

    Article  Google Scholar 

  • Ávila-Jiménez, J.L., Gibaja, E., Ventura, S.: Evolving multi-label classification rules with gene expression programming: a preliminary study. In: Corchado, E., Graña Romay, M., Manhaes Savio, A. (eds.) HAIS 2010. LNCS (LNAI), vol. 6077, pp. 9–16. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-13803-4_2

    Chapter  Google Scholar 

  • Bosc, G., et al.: Local subgroup discovery for eliciting and understanding new structure-odor relationships. In: Calders, T., Ceci, M., Malerba, D. (eds.) DS 2016. LNCS (LNAI), vol. 9956, pp. 19–34. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46307-0_2

    Chapter  Google Scholar 

  • Boutell, M.R., Luo, J., Shen, X., Brown, C.M.C.M.: Learning multi-label scene classification. Pattern Recogn. 37(9), 1757–1771 (2004)

    Article  Google Scholar 

  • Charte, F., Rivera, A.J., del Jesús, M.J., Herrera, F.: LI-MLC: a label inference methodology for addressing high dimensionality in the label space for multilabel classification. IEEE Trans. Neural Netw. Learn. Syst. 25(10), 1842–1854 (2014)

    Article  Google Scholar 

  • Cortes, C., DeSalvo, G., Mohri, M.: Learning with rejection. In: Ortner, R., Simon, H.U., Zilles, S. (eds.) ALT 2016. LNCS (LNAI), vol. 9925, pp. 67–82. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46379-7_5

    Chapter  MATH  Google Scholar 

  • Dembczynski, K., Kotlowski, W., Slowinski, R.: ENDER: a statistical framework for boosting decision rules. Data Min. Knowl. Disc. 21(1), 52–90 (2010)

    Article  MathSciNet  Google Scholar 

  • Dembczyński, K., Waegeman, W., Cheng, W., Hüllermeier, E.: On label dependence and loss minimization in multi-label classification. Mach. Learn. 88(1–2), 5–45 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  • Elisseeff, A., Weston, J.: A kernel method for multi-labelled classification. In: Dietterich, T.G., Becker, S., Ghahramani, Z. (eds.) Advances in Neural Information Processing Systems (NeurIPS), vol. 14, pp. 681–687. MIT Press (2001)

    Google Scholar 

  • Franc, V., Prusa, D.: On discriminative learning of prediction uncertainty. In: Proceedings of 36th International Conference on Machine Learning (ICML), pp. 1963–1971 (2019)

    Google Scholar 

  • Freitas, A.A.: Comprehensible classification models: a position paper. SIGKDD Explor. 15(1), 1–10 (2014)

    Article  Google Scholar 

  • Fürnkranz, J.: Separate-and-conquer rule learning. Artif. Intell. Rev. 13(1), 3–54 (1999)

    Article  MATH  Google Scholar 

  • Fürnkranz, J., Flach, P.A.: ROC ‘n’ rule learning - towards a better understanding of covering algorithms. Mach. Learn. 58(1), 39–77 (2005)

    Article  MATH  Google Scholar 

  • Fürnkranz, J., Gamberger, D., Lavrač, N.: Foundations of Rule Learning. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-540-75197-7

    Book  MATH  Google Scholar 

  • Fürnkranz, J., Kliegr, T.: A brief overview of rule learning. In: Bassiliades, N., Gottlob, G., Sadri, F., Paschke, A., Roman, D. (eds.) RuleML 2015. LNCS, vol. 9202, pp. 54–69. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-21542-6_4

    Chapter  Google Scholar 

  • Fürnkranz, J., Kliegr, T., Paulheim, H.: On cognitive preferences and the plausibility of rule-based models. Mach. Learn. 109(4), 853–898 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  • Gharroudi, O.: Ensemble multi-label learning in supervised and semi-supervised settings. Ph.D. thesis, Université de Lyon (2017)

    Google Scholar 

  • Guidotti, R., Monreale, A., Ruggieri, S., Pedreschi, D., Turini, F., Giannotti, F.: Local rule-based explanations of black box decision systems. arXiv:1805.10820 (2018)

  • Hayes, P.J., Weinstein, S.P.: CONSTRUE/TIS: a system for content-based indexing of a database of news stories. In: Rappaport, A.T., Smith, R.G. (eds.) Proceedings of 2nd Conference on Innovative Applications of Artificial Intelligence (IAAI 1990), Washington, DC, USA, pp. 49–64. AAAI Press (1991)

    Google Scholar 

  • Klein, Y., Rapp, M., Loza Mencía, E.: Efficient discovery of expressive multi-label rules using relaxed pruning. In: Kralj Novak, P., Šmuc, T., Džeroski, S. (eds.) DS 2019. LNCS (LNAI), vol. 11828, pp. 367–382. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33778-0_28

    Chapter  Google Scholar 

  • Lewis, D.D.: An evaluation of phrasal and clustered representations on a text categorization task. In: Proceedings of 15th Annual International Conference on Research and Development in Information Retrieval (SIGIR), pp. 37–50 (1992)

    Google Scholar 

  • Lewis, D.D.: Reuters-21578 text categorization test collection distribution 1.0. README file (V 1.3), May 2004

    Google Scholar 

  • Li, B., Li, H., Wu, M., Li, P.: Multi-label classification based on association rules with application to scene classification. In: Proceedings of 9th International Conference for Young Computer Scientists (ICYCS), pp. 36–41. IEEE (2008)

    Google Scholar 

  • Mencía, E.L., Fürnkranz, J., Hüllermeier, E., Rapp, M.: Learning interpretable rules for multi-label classification. In: Escalante, H.J., et al. (eds.) Explainable and Interpretable Models in Computer Vision and Machine Learning. TSSCML, pp. 81–113. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-98131-4_4

    Chapter  Google Scholar 

  • Loza Mencía, E., Janssen, F.: Learning rules for multi-label classification: a stacking and a separate-and-conquer approach. Mach. Learn. 105(1), 77–126 (2016)

    Article  MathSciNet  Google Scholar 

  • Lundberg, S.M., Lee, S.: A unified approach to interpreting model predictions. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems, Long Beach, CA, USA, vol. 30, pp. 4765–4774 (2017)

    Google Scholar 

  • Madjarov, G., Kocev, D., Gjorgjevikj, D., Džeroski, S.: An extensive experimental comparison of methods for multi-label learning. Pattern Recogn. 45(9), 3084–3104 (2012)

    Article  Google Scholar 

  • Montañés, E., Senge, R., Barranquero, J., Quevedo, J.R., del Coz, J.J., Hüllermeier, E.: Dependent binary relevance models for multi-label classification. Pattern Recogn. 47(3), 1494–1508 (2014)

    Article  Google Scholar 

  • Nguyen, V., Hüllermeier, E., Rapp, M., Mencía, E.L., Fürnkranz, J.: On aggregation in ensembles of multilabel classifiers. CoRR abs/2006.11916 (2020). http://arxiv.org/abs/2006.11916

  • Nguyen, V.L., Hüllermeier, E.: Reliable multi-label classification: prediction with partial abstention. In: Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI), pp. 5264–5271 (2020)

    Google Scholar 

  • Park, S.H., Fürnkranz, J.: Multi-label classification with label constraints. In: Hüllermeier, E., Fürnkranz, J. (eds.) Proceedings of ECML-PKDD-08 Workshop on Preference Learning (PL 2008), Antwerp, Belgium, pp. 157–171 (2008)

    Google Scholar 

  • Pillai, I., Fumera, G., Roli, F.: Multi-label classification with a reject option. Pattern Recogn. 46(8), 2256–2266 (2013)

    Article  Google Scholar 

  • Rapp, M., Loza Mencía, E., Fürnkranz, J.: Exploiting anti-monotonicity of multi-label evaluation measures for inducing multi-label rules. In: Phung, D., Tseng, V.S., Webb, G.I., Ho, B., Ganji, M., Rashidi, L. (eds.) PAKDD 2018. LNCS (LNAI), vol. 10937, pp. 29–42. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93034-3_3

    Chapter  Google Scholar 

  • Rapp, M., Loza Mencía, E., Fürnkranz, J.: On the trade-off between consistency and coverage in multi-label rule learning heuristics. In: Kralj Novak, P., Šmuc, T., Džeroski, S. (eds.) DS 2019. LNCS (LNAI), vol. 11828, pp. 96–111. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33778-0_9

    Chapter  Google Scholar 

  • Rapp, M., Loza Mencía, E., Fürnkranz, J., Nguyen, V.L., Hüllermeier, E.: Learning gradient boosted multi-label classification rules. In: Proceedings of European Conference on Machine Learning and Knowledge Discovery in Databases (ECML/PKDD). Springer (2020)

    Google Scholar 

  • Read, J., Pfahringer, B., Holmes, G., Frank, E.: Classifier chains for multi-label classification. Mach. Learn. 85(3), 333–359 (2011)

    Article  MathSciNet  Google Scholar 

  • Ribeiro, M.T., Singh, S., Guestrin, C.: “Why should I trust you?": explaining the predictions of any classifier. In: Krishnapuram, B., Shah, M., Smola, A.J., Aggarwal, C.C., Shen, D., Rastogi, R. (eds.) Proceedings of 22nd International Conference on Knowledge Discovery and Data Mining (KDD), pp. 1135–1144. ACM (2016)

    Google Scholar 

  • Senge, R., del Coz, J.J., Hüllermeier, E.: Rectifying classifier chains for multi-label classification. In: Henrich, A., Sperker, H. (eds.) Proc. Lernen, Wissen & Adaptivität (LWA), pp. 151–158. Bamberg, Germany (2013)

    Google Scholar 

  • Shi, C., Kong, X., Yu, P.S., Wang, B.: Multi-label ensemble learning. In: Gunopulos, D., Hofmann, T., Malerba, D., Vazirgiannis, M. (eds.) ECML PKDD 2011. LNCS (LNAI), vol. 6913, pp. 223–239. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-23808-6_15

    Chapter  Google Scholar 

  • Thabtah, F.A., Cowling, P.I., Peng, Y.: MMAC: a new multi-class, multi-label associative classification approach. In: Proceedings of 4th International Conference on Data Mining (ICDM), pp. 217–224. IEEE (2004)

    Google Scholar 

  • Thabtah, F.A., Cowling, P.I., Peng, Y.: Multiple labels associative classification. Knowl. Inf. Syst. 9(1), 109–129 (2006)

    Article  Google Scholar 

  • Trohidis, K., Tsoumakas, G., Kalliris, G., Vlahavas, I.P.: Multilabel classification of music into emotions. In: Proceedings of 9th International Conference on Music Information Retrieval (ISMIR), pp. 325–330 (2008)

    Google Scholar 

  • Tsoumakas, G., Vlahavas, I.: Random k-labelsets: an ensemble method for multilabel classification. In: Kok, J.N., Koronacki, J., Mantaras, R.L., Matwin, S., Mladenič, D., Skowron, A. (eds.) ECML 2007. LNCS (LNAI), vol. 4701, pp. 406–417. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74958-5_38

    Chapter  Google Scholar 

  • Tsoumakas, G., Katakis, I., Vlahavas, I.P.: Mining multi-label data. In: Maimon, O., Rokach, L. (eds.) Data Mining and Knowledge Discovery Handbook, pp. 667–685. Springer, Boston (2010). https://doi.org/10.1007/978-0-387-09823-4_34

    Chapter  Google Scholar 

  • Veloso, A., Meira, W., Gonçalves, M., Zaki, M.: Multi-label lazy associative classification. In: Kok, J.N., Koronacki, J., Lopez de Mantaras, R., Matwin, S., Mladenič, D., Skowron, A. (eds.) PKDD 2007. LNCS (LNAI), vol. 4702, pp. 605–612. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74976-9_64

    Chapter  Google Scholar 

  • Zhang, M., Zhou, Z.: A review on multi-label learning algorithms. IEEE Trans. Knowl. Data Eng. 26(8), 1819–1837 (2014)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the German Research Foundation (DFG) under grant no. 400845550.

Author information

Authors and Affiliations

  1. Paderborn University, Paderborn, Germany

    Eyke Hüllermeier & Vu-Linh Nguyen

  2. Johannes Kepler University, Linz, Austria

    Johannes Fürnkranz

  3. Technical University Darmstadt, Darmstadt, Germany

    Eneldo Loza Mencia & Michael Rapp

Authors
  1. Eyke Hüllermeier
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Johannes Fürnkranz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eneldo Loza Mencia
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Vu-Linh Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Michael Rapp
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eyke Hüllermeier .

Editor information

Editors and Affiliations

  1. School of Computer Science and Informatics, Cardiff University, Cardiff, UK

    Víctor Gutiérrez-Basulto

  2. University of Economics, Prague, Prague, Czech Republic

    Tomáš Kliegr

  3. Norwegian University of Science and Technology, Gjøvik, Norway

    Ahmet Soylu

  4. Department Of Informatics, University of Oslo, Oslo, Norway

    Martin Giese

  5. SINTEF AS, Oslo, Norway

    Dumitru Roman

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hüllermeier, E., Fürnkranz, J., Loza Mencia, E., Nguyen, VL., Rapp, M. (2020). Rule-Based Multi-label Classification: Challenges and Opportunities. In: Gutiérrez-Basulto, V., Kliegr, T., Soylu, A., Giese, M., Roman, D. (eds) Rules and Reasoning. RuleML+RR 2020. Lecture Notes in Computer Science(), vol 12173. Springer, Cham. https://doi.org/10.1007/978-3-030-57977-7_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-57977-7_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-57976-0

  • Online ISBN: 978-3-030-57977-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation