Skip to main content
SpringerLink
Log in

KERMITviz: Visualizing Neural Network Activations on Syntactic Trees

  • Conference paper
  • First Online:
Metadata and Semantic Research (MTSR 2021)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1537))

Included in the following conference series:

Abstract

The study of symbolic syntactic interpretations has been the cornerstone of natural language understanding for many years.

Today, modern artificial neural networks are widely searched to assess their syntactic ability, through several probing tasks.

In this paper, we propose a neural network system that explicitly includes syntactic interpretations: Kernel-inspired Encoder with Recursive Mechanism for Interpretable Trees Visualizer (KERMITviz). The most important result is that KERMITviz allows to visualize how syntax is used in inference. This system can be used in combination with transformer architectures like BERT, XLNet and clarifies the use of symbolic syntactic interpretations in specific neural networks making the black-box neural network neural networks explainable, interpretable and clear.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The code is available at https://github.com/ART-Group-it/KERMIT.

  2. 2.

    https://colab.research.google.com.

References

  1. Ancona, M., Ceolini, E., Öztireli, C., Gross, M.: Towards better understanding of gradient-based attribution methods for deep neural networks (2018)

    Google Scholar 

  2. Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.R., Samek, W.: On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE 10(7), 1–46 (2015). https://doi.org/10.1371/journal.pone.0130140

    Article  Google Scholar 

  3. Bien, J., Tibshirani, R.: Prototype selection for interpretable classification. Ann. Appl. Stat. 5(4) (2011). https://doi.org/10.1214/11-aoas495. http://dx.doi.org/10.1214/11-AOAS495

  4. Collins, M., Duffy, N.: New ranking algorithms for parsing and tagging: kernels over discrete structures, and the voted perceptron. In: Proceedings of ACL 2002 (2002)

    Google Scholar 

  5. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. CoRR abs/1810.0 (2018). http://arxiv.org/abs/1810.04805

  6. Foster, Z.S.L., Sharpton, T.J., Grünwald, N.J.: Metacoder: an R package for visualization and manipulation of community taxonomic diversity data. PLoS Comput. Biol. 13(2) (2017). https://doi.org/10.1371/journal.pcbi.1005404

  7. Hoover, B., Strobelt, H., Gehrmann, S.: exBERT: a visual analysis tool to explore learned representations in transformers models. arXiv e-prints arXiv:1910.05276, October 2019

  8. Johnson, W., Lindenstrauss, J.: Extensions of Lipschitz mappings into a Hilbert space. Contemp. Math. 26, 189–206 (1984)

    Article  MathSciNet  Google Scholar 

  9. Kahng, M., Andrews, P.Y., Kalro, A., Chau, D.H.: ActiVis: visual exploration of industry-scale deep neural network models. CoRR abs/1704.01942 (2017). http://arxiv.org/abs/1704.01942

  10. Liang, B., Yin, R., Gui, L., Du, J., He, Y., Xu, R.: Aspect-invariant sentiment features learning: Adversarial multi-task learning for aspect-based sentiment analysis. In: Proceedings of the 29th ACM International Conference on Information amp; Knowledge Management, CIKM 2020, New York, NY, USA, pp. 825–834. Association for Computing Machinery (2020). https://doi.org/10.1145/3340531.3411868

  11. Ming, Y., et al.: Understanding hidden memories of recurrent neural networks. CoRR abs/1710.10777 (2017). http://arxiv.org/abs/1710.10777

  12. Onorati, D., Tommasino, P., Ranaldi, L., Fallucchi, F., Zanzotto, F.M.: Pat-in-the-loop: declarative knowledge for controlling neural networks. Future Internet 12(12) (2020). https://doi.org/10.3390/fi12120218. https://www.mdpi.com/1999-5903/12/12/218

  13. Plate, T.A.: Holographic reduced representations. IEEE Trans. Neural Networks 6(3), 623–641 (1995). https://doi.org/10.1109/72.377968

    Article  Google Scholar 

  14. Ribeiro, M.T., Singh, S., Guestrin, C.: “Why should i trust you?”: explaining the predictions of any classifier (2016)

    Google Scholar 

  15. Rizos, G., Hemker, K., Schuller, B.: Augment to prevent: short-text data augmentation in deep learning for hate-speech classification. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM 2019, New York, NY, USA, pp. 991–1000. Association for Computing Machinery (2019). https://doi.org/10.1145/3357384.3358040

  16. Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: visual explanations from deep networks via gradient-based localization. Int. J. Comput. Vision 128(2), 336–359 (2019). https://doi.org/10.1007/s11263-019-01228-7. http://dx.doi.org/10.1007/s11263-019-01228-7

  17. Shen, Y., Lin, Z., Huang, C.W., Courville, A.: Neural language modeling by jointly learning syntax and lexicon (2018)

    Google Scholar 

  18. Smilkov, D., Thorat, N., Nicholson, C., Reif, E., Viégas, F.B., Wattenberg, M.: Embedding projector: interactive visualization and interpretation of embeddings. arXiv preprint arXiv:1611.05469 (2016)

  19. Strobelt, H., Gehrmann, S., Huber, B., Pfister, H., Rush, A.M.: Visual analysis of hidden state dynamics in recurrent neural networks. CoRR abs/1606.07461 (2016). http://arxiv.org/abs/1606.07461

  20. Vaswani, A., et al.: Attention is all you need. In: NIPS (2017)

    Google Scholar 

  21. Vig, J.: A multiscale visualization of attention in the transformer model. In: ACL 2019–57th Annual Meeting of the Association for Computational Linguistics, Proceedings of System Demonstrations, pp. 37–42 (2019)

    Google Scholar 

  22. Wallace, E., Tuyls, J., Wang, J., Subramanian, S., Gardner, M., Singh, S.: AllenNLP Interpret: a framework for explaining predictions of NLP models. In: Empirical Methods in Natural Language Processing (2019)

    Google Scholar 

  23. Wang, Y., Lee, H.Y., Chen, Y.N.: Tree transformer: integrating tree structures into self-attention. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, pp. 1061–1070. Association for Computational Linguistics, November 2019. https://doi.org/10.18653/v1/D19-1098. https://www.aclweb.org/anthology/D19-1098

  24. Zanzotto, F.M., Dell’Arciprete, L.: Distributed tree kernels. In: Proceedings of the 29th International Conference on Machine Learning, ICML 2012, vol. 1, pp. 193–200 (2012). http://www.scopus.com/inward/record.url?eid=2-s2.0-84867126965&partnerID=MN8TOARS

  25. Zanzotto, F.M., Santilli, A., Ranaldi, L., Onorati, D., Tommasino, P., Fallucchi, F.: KERMIT: complementing transformer architectures with encoders of explicit syntactic interpretations. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 256–267. Association for Computational Linguistics, November 2020. https://doi.org/10.18653/v1/2020.emnlp-main.18. https://www.aclweb.org/anthology/2020.emnlp-main.18

Download references

Author information

Authors and Affiliations

  1. Department of Innovation and Information Engineering, Guglielmo Marconi University, Rome, Italy

    Leonardo Ranaldi & Francesca Fallucchi

  2. Department of Computer Science, Sapienza University Rome, Rome, Italy

    Andrea Santilli

  3. Department of Enterprise Engineering, University of Rome Tor Vergata, Rome, Italy

    Fabio Massimo Zanzotto

Authors
  1. Leonardo Ranaldi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Francesca Fallucchi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andrea Santilli
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fabio Massimo Zanzotto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Leonardo Ranaldi .

Editor information

Editors and Affiliations

  1. International Hellenic University, Thessaloniki, Greece

    Emmanouel Garoufallou

  2. Complutense University of Madrid, Madrid, Spain

    María-Antonia Ovalle-Perandones

  3. University College London, London, UK

    Andreas Vlachidis

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ranaldi, L., Fallucchi, F., Santilli, A., Zanzotto, F.M. (2022). KERMITviz: Visualizing Neural Network Activations on Syntactic Trees. In: Garoufallou, E., Ovalle-Perandones, MA., Vlachidis, A. (eds) Metadata and Semantic Research. MTSR 2021. Communications in Computer and Information Science, vol 1537. Springer, Cham. https://doi.org/10.1007/978-3-030-98876-0_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-98876-0_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-98875-3

  • Online ISBN: 978-3-030-98876-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation