Skip to main content
SpringerLink
Log in

Analysis of Changing Trends in Textual Data Representation

  • Conference paper
  • First Online:
Recent Trends in Image Processing and Pattern Recognition (RTIP2R 2020)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1380))

Abstract

The most efficient way to convey human knowledge is through natural language text. In order to manage the exponential growth of digital data, it is high time to build a robust text information system. The system performs various techniques such as classification, clustering, summarization, etc. to organize the huge unstructured text data into well-defined forms. Text representation is the fundamental task in text mining and text retrieval. Just like a backbone gives structure to human body, text representation lays the foundation step for various other downstream tasks of Natural Language Processing (NLP), text mining and text retrieval. This paper concentrates on the analysis of various text representation models and also discusses major changing trends in textual data representation. This paper could be beneficial to those who wish to study and work on text data in NLP domain. The challenges and drawbacks of the existing text representation models are also discussed in this paper.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Harris, Z.: Distributional structure. Word 10, 146–162 (1954)

    Google Scholar 

  2. Manning, C.D., Schutze, H.: Foundations of Statistical Natural Language Processing. MIT press, Cambridge (2002)

    MATH  Google Scholar 

  3. Manning, C.D., Raghavan, P., Schutze, H.: Introduction to Information Retrieval. Cambridge University Press, Cambridge, England (2008)

    Book  Google Scholar 

  4. Jurafsky, D., Martin, J.H.: Speech and language processing: an introduction to natural language processing. In: Computational Linguistics and Speech Recognition. 1st edn. Prentice Hall PTR, Upper Saddle River (2000)

    Google Scholar 

  5. Hotho, A., Maedche, A., Staab, S.: Ontology-based text document clustering. KI J. 16, 48–54 (2002)

    Google Scholar 

  6. Jing, L., Zhou, L., Ng, M.K., Huang, J.Z.: Ontology-based distance measure for text clustering (2006)

    Google Scholar 

  7. Wang, Y., Ni, X., Sun, J.T., Tong, Y., Chen, Z.: Representing document as dependency graph for document clustering. In: CIKM 2011 Proceedings of the 20th ACM International Conference on Information and Knowledge Management, Glasgow, Scotland, pp. 2177–2180 (2011)

    Google Scholar 

  8. Chow, T.W., Zhang, H., Rahman, M.K.M.: A new document representation using term frequency and vectorized graph connectionists with application to document retrieval. Exp. Syst. Appl. 36(10), 12023–12035 (2009)

    Article  Google Scholar 

  9. Rafi, M., Amin, F., Shaikh, M.S.: Document clustering using graph based document representation with constraints. Pak. J. Eng. Appl. Sci. 18, 56–68 (2016)

    Google Scholar 

  10. Abdulsahib, A.K., Kamaruddin, S.S.: Graph based text representation for document clustering. J. Theor. Appl. Inf. Technol. 76(1) (2015)

    Google Scholar 

  11. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. CoRR J. (2013). Volume: abs-1301.3781

    Google Scholar 

  12. Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: distributed representations of words and phrased and their compositionality. In: The proceeding of Neural Information Processing Systems (NIPS), pp. 3111–3119, Curran Associates Inc., Nevada (2013)

    Google Scholar 

  13. Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543. Association for Computational Linguistics, Doha (2014)

    Google Scholar 

  14. Joulin, A., Grave, E., Bojanowski, P., Mikolov, T.: Bag of tricks for efficient text classification. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pp. 427–431, Association for Computational Linguistics, Valencia (2017)

    Google Scholar 

  15. Peters, M.E., et al.: Deep contextualized word representations. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 2227–2237. New Association for Computational Linguistics, Orleans (2018)

    Google Scholar 

  16. Le, Q., Mikolov, T.: Distributed representations of sentences and documents. In: Proceedings of the 31st International Conference on Machine Learning, JMLR: W&CP, Beijing, China, vol. 32, pp. 1188–1196 (2014)

    Google Scholar 

  17. Lau, J.H., Baldwin, T.: An empirical evaluation of doc2vec with practical insights into document embedding generation. In: Proceedings of the 1st Workshop on Representation Learning for NLP, pp. 78–86. Association for Computational Linguistics, Berlin (2016)

    Google Scholar 

  18. Kiros, R., et al.: Skip-thought vectors. In: the proceeding of Neural Information Processing Systems (NIPS), pp. 3294–3302. Curran Associates Inc., Montreal (2015)

    Google Scholar 

  19. Logeswaran, L., Lee, H.: An efficient framework for learning sentence representations. In: International Conference on Learning Representations (2018)

    Google Scholar 

  20. Zhai, C., Massung, S.: Text Data Management and Analysis – A Practical Introduction to Information Retrieval and Text Mining. Association for Computing Machinery and Morgan & Claypool Publishers (2016)

    Google Scholar 

  21. Huang, E.H., Socher, R., Manning, C.D., Ng, A.Y.: Improving word representations via global context and multiple word prototypes. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, Jeju, Republic of Korea, pp. 873–882 (2012)

    Google Scholar 

  22. Levy, O., Goldberg, Y.: Dependency-based word embeddings. In: Proceedings of the 52th Annual Meeting of the Association for Computational Linguistics, pp. 302–308. Association for Computational Linguistics, Baltimore (2014)

    Google Scholar 

  23. Gantz, J., Reinsel, D.: The Digital Universe in 2020: Big Data, Bigger Digital Shadows, and Biggest Growth in the Fast East, IDC Report (2012)

    Google Scholar 

  24. Ling, W., Dyer, C., Black, A.W., Trancoso, I.: Two/too simple adaptations of Word2Vec for syntax problems. In: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1299–1304. Association for Computational Linguistics, Denver, (2015)

    Google Scholar 

  25. Grave, E., Bojanowski, P., Gupta, P., Joulin, A., Mikolov, T.: Learning word vectors for 157 languages. In: Proceedings of the International Conference on Language Resources and Evaluation (2018)

    Google Scholar 

  26. Bojanowski, P., Grave, E., Joulin, A., Mikolov, T.: Enriching Word Vectors with Subword Information. arXiv:1607.04606v2 [cs.CL] 19 June (2017)

  27. Mitchell, J., Lapata, M.: Composition in distributional models of semantics. Cognit. Sci. 34(8), 1388–1429 (2010)

    Article  Google Scholar 

  28. Zanzotto, F.M., Korkontzelos, I., Fallucchi, F., Manandhar, S.: Estimating linear models for compositional distributional semantics. In: COLING (2010)

    Google Scholar 

  29. Yoav, G.: Neural Network Methods in Natural Language Processing (Synthesis Lectures on Human Language Technologies). Morgan & Claypool Publishers (2017)

    Google Scholar 

  30. Dai, A.M., Olah, C., Le, Q.V.: Document Embedding with Paragraph Vectors. arXiv:1507.07998v1 [cs.CL] 29 July (2015)

  31. Gan, Z., Pu, Y., Henao, R., Li, C., He, X., Carin, L.: Unsupervised learning of sentence representations using convolutional neural networks. arXiv preprint arXiv:1611.07897 (2016)

  32. Perone, C.S., Silveira, R., Paula, T.S.: Evaluation of sentence embeddings in downstream and linguistic probing tasks. arXiv:1806.06259v1 [cs.CL] 16 June (2018)

  33. Rong, X.: word2vec Parameter Learning Explained. arXiv:1411.2738v4 [cs.CL] 5 June (2016)

  34. Giannakopoulos, G., Mavridi, P., Paliouras, G., Papadakis, G., Tserpes, K.: Representation models for text classification: a comparative analysis over three Web document types. In: Proceedings of the 2nd International Conference on Web Intelligence, Mining and Semantics, Craiova, Romania, vol. 13 (2012)

    Google Scholar 

  35. Lyman, P., et al.: How much information? https://www2.sims.berkeley.edu/research/projects/how-much-info-2003.

  36. Roe, C.: The growth of unstructured data: what to do with all those zettabytes? https://www.dataversity.net/the-growth-of-unstructured-data-what-are-we-going-to-do-with-all-those-zettabytes/

  37. Applied Text Mining in Python, University of Michigan. https://www.coursera.org/lecture/python-text-mining/handling-text-in-python-MeheH

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Manipur University, Imphal, India

    Ksh. Nareshkumar Singh, A. Dorendro & H. Mamata Devi

  2. Department of Computer Science, Gauhati University, Guwahati, India

    Anjana Kakoti Mahanta

Authors
  1. Ksh. Nareshkumar Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Dorendro
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. H. Mamata Devi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anjana Kakoti Mahanta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to A. Dorendro , H. Mamata Devi or Anjana Kakoti Mahanta .

Editor information

Editors and Affiliations

  1. University of South Dakota, Vermillion, SD, USA

    K. C. Santosh

  2. Dr. Babasaheb Ambedkar Marathwada University, Aurangabad, India

    Bharti Gawali

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Singh, K.N., Dorendro, A., Devi, H.M., Mahanta, A.K. (2021). Analysis of Changing Trends in Textual Data Representation. In: Santosh, K.C., Gawali, B. (eds) Recent Trends in Image Processing and Pattern Recognition. RTIP2R 2020. Communications in Computer and Information Science, vol 1380. Springer, Singapore. https://doi.org/10.1007/978-981-16-0507-9_21

Download citation

  • DOI: https://doi.org/10.1007/978-981-16-0507-9_21

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-16-0506-2

  • Online ISBN: 978-981-16-0507-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation