Skip to main content
SpringerLink
Log in

Mining Interpretable Rules for Sentiment and Semantic Relation Analysis Using Tsetlin Machines

  • Conference paper
  • First Online:
Artificial Intelligence XXXVII (SGAI 2020)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12498))

Abstract

Tsetlin Machines (TMs) are an interpretable pattern recognition approach that captures patterns with high discriminative power from data. Patterns are represented as conjunctive clauses in propositional logic, produced using bandit-learning in the form of Tsetlin Automata. In this work, we propose a TM-based approach to two common Natural Language Processing (NLP) tasks, viz. Sentiment Analysis and Semantic Relation Categorization. By performing frequent itemset mining on the patterns produced, we show that they follow existing expert-verified rule-sets or lexicons. Further, our comparison with other widely used machine learning techniques indicates that the TM approach helps maintain interpretability without compromising accuracy – a result we believe has far-reaching implications not only for interpretable NLP but also for interpretable AI in general.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Note that the probability \(\frac{s-1}{s}\) is replaced by 1 when boosting true positives.

  2. 2.

    Python implementation of Tsetlin Machine based classifier: code retrieved from https://github.com/cair/pyTsetlinMachine.

References

  1. Abeyrathna, K.D., Granmo, O.C., Zhang, X., Jiao, L., Goodwin, M.: The regression Tsetlin machine - a novel approach to interpretable non-linear regression. Philos. Trans. R. Soc. A 378(2164), 20190165 (2019)

    Article  Google Scholar 

  2. Aggarwal, C.C., Zhai, C.: Mining Text Data. Springer, Boston (2012)

    Book  Google Scholar 

  3. Agrawal, R., Srikant, R., et al.: Fast algorithms for mining association rules. In: Proceeding of the 20th International Conference Very Large Data Bases, VLDB, vol. 1215, pp. 487–499 (1994)

    Google Scholar 

  4. Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C.: A neural probabilistic language model. J. Mach. Learn. Res. 3(2), 1137–1155 (2003)

    MATH  Google Scholar 

  5. Berge, G.T., Granmo, O.C., Tveit, T.O., Goodwin, M., Jiao, L., Matheussen, B.V.: Using the Tsetlin machine to learn human-interpretable rules for high-accuracy text categorization with medical applications. IEEE Access 7, 115134–115146 (2019)

    Article  Google Scholar 

  6. Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12(8), 2493–2537 (2011)

    MATH  Google Scholar 

  7. Girju, R.: Automatic detection of causal relations for question answering. In: Proceedings of the ACL 2003 Workshop on Multilingual Summarization and Question Answering, vol. 12, pp. 76–83. Association for Computational Linguistics (2003)

    Google Scholar 

  8. Go, A., Bhayani, R., Huang, L.: Twitter sentiment classification using distant supervision. CS224N Proj. Rep. Stanford 1(12), 2009 (2009)

    Google Scholar 

  9. Granmo, O.C.: The Tsetlin Machine - A Game Theoretic Bandit Driven Approach to Optimal Pattern Recognition with Propositional Logic. arXiv preprint arXiv:1804.01508 (2018)

  10. Granmo, O.C., Glimsdal, S., Jiao, L., Goodwin, M., Omlin, C.W., Berge, G.T.: The convolutional Tsetlin machine. arXiv preprint arXiv:1905.09688 (2019)

  11. Hancock, B., Bringmann, M., Varma, P., Liang, P., Wang, S., Ré, C.: Training classifiers with natural language explanations. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics Conference, vol. 2018, p. 1884. NIH Public Access (2018)

    Google Scholar 

  12. Hendrickx, I., et al.: Semeval-2010 task 8: multi-way classification of semantic relations between pairs of nominals. In: Proceedings of the Workshop on Semantic Evaluations: Recent Achievements and Future Directions, pp. 94–99. Association for Computational Linguistics (2009)

    Google Scholar 

  13. Hu, M., Liu, B.: Mining and summarizing customer reviews. In: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 168–177. ACM (2004)

    Google Scholar 

  14. Jones, K.S.: A statistical interpretation of term specificity and its application in retrieval. J. Doc. 28(1), 11–21 (2004)

    Article  Google Scholar 

  15. Kiritchenko, S., Mohammad, S.M.: The effect of negators, modals, and degree adverbs on sentiment composition. arXiv preprint arXiv:1712.01794 (2017)

  16. Kiritchenko, S., Zhu, X., Mohammad, S.M.: Sentiment analysis of short informal texts. J. Artif. Intell. Res. 50, 723–762 (2014)

    Article  Google Scholar 

  17. Lipton, Z.C.: The mythos of model interpretability. arXiv preprint arXiv:1606.03490 (2016)

  18. Liu, H., Yin, Q., Wang, W.Y.: Towards explainable NLP: a generative explanation framework for text classification. arXiv preprint arXiv:1811.00196 (2018)

  19. Mohammad, S.M.: Word affect intensities. In: Proceedings of the 11th Edition of the Language Resources and Evaluation Conference (LREC-2018), Miyazaki, Japan (2018)

    Google Scholar 

  20. Mohammad, S.M., Turney, P.D.: Crowdsourcing a word-emotion association lexicon. Comput. Intell. 29(3), 436–465 (2013)

    Article  MathSciNet  Google Scholar 

  21. Murdoch, W.J., Singh, C., Kumbier, K., Abbasi-Asl, R., Yu, B.: Interpretable machine learning: definitions, methods, and applications. arXiv preprint arXiv:1901.04592 (2019)

  22. Nielsen, F.Å.: A new anew: evaluation of a word list for sentiment analysis in microblogs. arXiv preprint arXiv:1103.2903 (2011)

  23. Phoulady, A., Granmo, O.C., Gorji, S.R., Phoulady, H.A.: The weighted tsetlin machine: compressed representations with weighted clauses. arXiv preprint arXiv:1911.12607 (2019)

  24. Ribeiro, M.T., Singh, S., Guestrin, C.: Why should i trust you? explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1135–1144. ACM (2016)

    Google Scholar 

  25. Rudin, C.: Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat. Mach. Intell. 1(5), 206 (2019)

    Article  Google Scholar 

  26. Samek, W., Wiegand, T., Müller, K.R.: Explainable artificial intelligence: understanding, visualizing and interpreting deep learning models. arXiv preprint arXiv:1708.08296 (2017)

  27. Shafik, R., Wheeldon, A., Yakovlev, A.: Explainability and dependability analysis of learning automata based AI hardware. In: IEEE 26th International Symposium on On-Line Testing and Robust System Design (IOLTS). IEEE (2020)

    Google Scholar 

  28. Socher, R., et al.: Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1631–1642 (2013)

    Google Scholar 

  29. Tausczik, Y.R., Pennebaker, J.W.: The psychological meaning of words: LIWC and computerized text analysis methods. J. Lang. Soc. Psychol. 29(1), 24–54 (2010)

    Article  Google Scholar 

  30. Tsetlin, M.L.: On behaviour of finite automata in random medium. Avtom I Telemekhanika 22(10), 1345–1354 (1961)

    Google Scholar 

  31. Wang, T., Rudin, C., Velez-Doshi, F., Liu, Y., Klampfl, E., MacNeille, P.: Bayesian rule sets for interpretable classification. In: 2016 IEEE 16th International Conference on Data Mining (ICDM), pp. 1269–1274. IEEE (2016)

    Google Scholar 

  32. Wheeldon, A., Shafik, R., Yakovlev, A., Edwards, J., Haddadi, I., Granmo, O.C.: Tsetlin machine: a new paradigm for pervasive AI. In: SCONA Workshop at Design, Automation and Test in Europe (DATE 2020) (2020)

    Google Scholar 

  33. Wilson, T., Wiebe, J., Hoffmann, P.: Recognizing contextual polarity in phrase-level sentiment analysis. In: Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (2005)

    Google Scholar 

  34. Xuelan, F., Kennedy, G.: Expressing causation in written english. RELC J. 23(1), 62–80 (1992)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Centre for AI Research, University of Agder, Grimstad, Norway

    Rupsa Saha, Ole-Christoffer Granmo & Morten Goodwin

Authors
  1. Rupsa Saha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ole-Christoffer Granmo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Morten Goodwin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rupsa Saha .

Editor information

Editors and Affiliations

  1. School of Computing, University of Portsmouth, Portsmouth, UK

    Max Bramer

  2. RKE Consulting, Micheldever, UK

    Richard Ellis

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Saha, R., Granmo, OC., Goodwin, M. (2020). Mining Interpretable Rules for Sentiment and Semantic Relation Analysis Using Tsetlin Machines. In: Bramer, M., Ellis, R. (eds) Artificial Intelligence XXXVII. SGAI 2020. Lecture Notes in Computer Science(), vol 12498. Springer, Cham. https://doi.org/10.1007/978-3-030-63799-6_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-63799-6_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-63798-9

  • Online ISBN: 978-3-030-63799-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation