Skip to main content
SpringerLink
Log in
Book cover

Language and Technology Conference

LTC 2017: Human Language Technology. Challenges for Computer Science and Linguistics pp 329–340Cite as

Title Categorization Based on Category Granularity

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12598))

Abstract

We focus on a problem of short text categorization, i.e. categorization of newspaper titles, and present a method that maximizes the impact of informative words due to the sparseness of titles. We used the hierarchical structure of categories and a transfer learning technique based on pre-training and fine-tuning to incorporate the granularity of categories into categorization. According to the hierarchical structure of categories, we transferred trained parameters of Convolutional Neural Networks (CNNs) on upper layers to the related lower ones, and finely tuned parameters of CNNs. The method was tested on titles collected from the Reuters corpus, and the results showed the effectiveness of the method.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://github.com/facebookresearch/fastText.

  2. 2.

    www.jst.go.jp/EN/index.html.

References

  1. Bojanowski, P., Grave, E., Joulin, A., Mikolov, T.: Enriching Word Vectors with Subword Information. arXiv preprint arXiv:1607.04606 (2016)

  2. Chen, D., Manning, C.D.: A Fast and accurate dependency parser using neural networks. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 740–750 (2014)

    Google Scholar 

  3. Chen, M., Jin, X., Shen, D.: Short text classification improved by learning multi-granularity topics. In: Proceedings of the 22nd International Joint Conference on Artificial Intelligence, pp. 1776–1781 (2011)

    Google Scholar 

  4. Devlin, J., Zbib, R., Huang, Z., Lamar, T., Schwartz, R., Makhoul, J.: Fast and robust neural network joint models for statistical machine translation. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pp. 1370–1380 (2014)

    Google Scholar 

  5. Goodman, J.: Classes for fast maximum entropy training. In: IEEE International Conference on Acoustics, Speech, and Signal Processing, pp. 561–564 (2001)

    Google Scholar 

  6. Johnson, R., Zhang, T.: Effective use of word order for text categorization with convolutional neural networks. In: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 103–112 (2015)

    Google Scholar 

  7. Johnson, R., Zhang, T.: Semi-supervised convolutional neural networks for text categorization vis region embedding. In: Proceedings of the Advances in Neural Information Processing Systems, vol. 28, pp. 919–927 (2015)

    Google Scholar 

  8. Joulin, A., Grave, E., Bojanowski, P., Mikolov, T.: Bag of Tricks for Efficient Text Classification. arXiv preprint arXiv:1607.01759 (2016)

  9. Joulin, A., Grave, E., Bojanowski, P., Mikolov, T.: Bag of tricks for efficient text classification. In: Proceedings of the 15th Conference of the European Chapter of the Association for Conputational Linguistics, pp. 427–431 (2017)

    Google Scholar 

  10. Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 1746–1751 (2014)

    Google Scholar 

  11. Kusner, M.J., Sun, Y., Kolkin, N.L., Weinberger, K.Q.: From word embeddings to document distances. In: Proceedings of the 32nd International Conference on Machine Learning, pp. 957–966 (2015)

    Google Scholar 

  12. Lewis, D.D., Yang, Y., Rose, T.G., Li, F.: RCV1: a new benchmark collection for text categorization research. J. Mach. Learn. Res. 5, 361–397 (2004)

    Google Scholar 

  13. Liu, J., Chang, W.C., Wu, Y., Yang, Y.: Deep learning for extreme multi-label text classification. In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 115–124 (2017)

    Google Scholar 

  14. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: Proceedings of the International Conference on Learning Representations Workshop (2013)

    Google Scholar 

  15. Pennington, J., Socher, R., Manning, C.D.: Glove: gloval vectors for word representation. In: Proceedings of the Empirical Methods in Natural Language Processing (EMNLP2014), pp. 1532–1543 (2014)

    Google Scholar 

  16. Phan, X.H., Nguyen, L.M., Horiguchi, S.: Learning to classify short and sparse text & web with hidden topics from large-scale data collections. In: Proceedings of the 17th International World Wide Web Conference, pp. 91–100 (2008)

    Google Scholar 

  17. Schmid, H.: Improvements in part-of-speech tagging with an application to German. In: Proceedings of the EACL SIGDAT Workshop, pp. 47–50 (1995)

    Google Scholar 

  18. Song, G., Ye, Y., Du, X., Huang, X., Bie, S.: Short text classification: a survey. Multimedia 9(5), 635–643 (2014)

    Google Scholar 

  19. Tajbakhsh, N., et al.: Convolutional neural networks for medical image analysis: full training or fine tuning? IEEE Trans. Med. Imaging 35(5), 1299–1312 (2016)

    Article  Google Scholar 

  20. Tan, B., Zhang, Y., Pan, S.J., Yang, Q.: Distant domain transfer learning. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence, pp. 2604–2610 (2017)

    Google Scholar 

  21. Wang, F., Wang, Z., Li, Z., Wen, J.R.: Concept-based short text classification and ranking. In: Proceedings of the 23rd ACM International Conference on Information and Knowledge Management, pp. 1069–1078 (2008)

    Google Scholar 

  22. Wang, J., Wang, Z., Zhang, D., Yan, J.: Combining knowledge with deep convolutional neural networks for short text classification. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence, pp. 2915–2921 (2017)

    Google Scholar 

  23. Wang, P., et al.: Semantic clustering and convolutional neural network for short text categorization. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pp. 352–357 (2015)

    Google Scholar 

  24. Wang, Y., et al.: Dual transfer learning for neural machine translation with marginal distribution regularization. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence (2018)

    Google Scholar 

  25. Wu, W., Li, H., Wang, H., Zhu, K.Q.: A probabilistic taxonomy for text understanding. In: Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, pp. 481–492 (2012)

    Google Scholar 

  26. Xiao, L., Huang, X., Chen, B., Jing, L.: Label-specific document representation for multi-label text classification. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pp. 466–475 (2019)

    Google Scholar 

  27. Yang, Y., Lin, X.: A re-examination of text categorization. In: Proceedings of the 22nd International Conference on Research and Development in Information Retrieval, pp. 42–49 (1999)

    Google Scholar 

  28. Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., Hovy, E.: Hierarchical attention networks for document classification. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics Human Language Technologies, pp. 1480–1489 (2016)

    Google Scholar 

  29. Zhang, R., Lee, H., Radev, D.: Dependency sensitive convolutional neural networks for modeling sentences and documents. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics Human Language Technologies, pp. 1512–1521 (2016)

    Google Scholar 

  30. Zhang, S., Jin, X., Shen, D., Cao, B., Ding, X., Zhang, X.: Short text classification by detecting information path. In: Proceedings of the 22rd ACM International Conference on Information and Knowledge Management, pp. 727–732 (2013)

    Google Scholar 

  31. Zhang, W., Yan, J., Wang, X., Zha, H.: Deep extreme multi-label learning. In: Proceedings of the ACM International Conference on Multimedia Retrieval, pp. 100–107 (2018)

    Google Scholar 

  32. Zhang, X., Zhao, J., LeCun, Y.: Character-level convolutional networks for text classification. In: Advances in Neural Information Processing systems, pp. 649–657 (2015)

    Google Scholar 

  33. Zhang, Y., Lease, M., Wallace, B.C.: Exploiting domain knowledge via grouped weight sharing with application to text categorization. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp. 155–160 (2017)

    Google Scholar 

  34. Zhang, Y., Wallace, B.C.: A sensitivity analysis of (and practitioners’ guide to) convolutional neural networks for sentence classification. In: Computing Research Repository (2015)

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank anonymous reviewers for their helpful comments. This work was supported by the Telecommunications Advancement Foundation, and Support Center for Advanced Telecommunications Technology Research, Foundation.

Author information

Authors and Affiliations

  1. Graduate School of Engineering, Kofu, Yamanashi, Japan

    Kazuya Shimura

  2. Graduate Faculty of Interdisciplinary Research, University of Yamanashi, Takeda, Kofu, 4-3-11, Yamanashi, Japan

    Fumiyo Fukumoto

Authors
  1. Kazuya Shimura
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fumiyo Fukumoto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fumiyo Fukumoto .

Editor information

Editors and Affiliations

  1. Adam Mickiewicz University, Poznań, Poland

    Zygmunt Vetulani

  2. Laboratoire d’Informatique pour la Méca, Orsay, France

    Patrick Paroubek

  3. Adam Mickiewicz University, Poznań, Poland

    Marek Kubis

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Shimura, K., Fukumoto, F. (2020). Title Categorization Based on Category Granularity. In: Vetulani, Z., Paroubek, P., Kubis, M. (eds) Human Language Technology. Challenges for Computer Science and Linguistics. LTC 2017. Lecture Notes in Computer Science(), vol 12598. Springer, Cham. https://doi.org/10.1007/978-3-030-66527-2_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-66527-2_25

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-66526-5

  • Online ISBN: 978-3-030-66527-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation