Skip to main content
SpringerLink
Log in

Deep Graph-Long Short-Term Memory: A Deep Learning Based Approach for Text Classification

  • Published:
Wireless Personal Communications Aims and scope Submit manuscript

Abstract

Multi-label text classification is a challenging task in many real applications. Mostly, in all the traditional techniques, word2vec is used to show the sequential information among text. However, use of word2vec ignores logic and context relationship among text, and we treat each label as an individual unit. Therefore, the existing techniques failed to reflect the real scenarios and to gain the semantic information regarding the relationship among texts. In this paper, we propose a model Deep Graph-Long Short-Term Memory (DG-LSTM) for multi-label text classification. In the proposed model, we store the documents using the graph database. Initially, the documents are pre-processed using standard dictionaries, and afterwards it generates the classified dictionaries. These classified dictionaries are used to generate the subgraphs. The model maintains a lookup table to reduce the search space for the new documents. For classification, the model uses the deep learning technique DG-LSTM. DG-LSTM is using Deep Graph_Rectified Linear Unit activation function to avoid blow-up and dying neuron problem of Rectified Linear Unit activation function. We verify the proposed model on the legal case of Indian judiciary. The results show that the proposed model has achieved 99% accuracy to classify the fresh case into its corresponding category.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Kowsari, K., Meimandi, K. J., Heidarysafa, M., Mendu, S., Barnes, L. E., & Brown, D. E. (2019). Text classification algorithms: A survey. Information, 10(4), 150.

    Article  Google Scholar 

  2. Zhu, W., Liu, Y., Hu, G., Ni, J., & Lu, Z. (2018). A sample extension method based on Wikipedia and its application in text classification. Wireless Personal Communication, 102, 3851–3867.

    Article  Google Scholar 

  3. Lee, D. D., & Seung, H. S. (1999). Learning the parts of objects by non-negative matrix factorization. Nature, 401, 788–791.

    Article  Google Scholar 

  4. Sadeghi, M., Zadeh, M., & Jutten, C. (2013). Dictionary learning for sparse representation: A novel approach. IEEE Signal Processing Letters, 20(12), 1195–1198.

    Article  Google Scholar 

  5. Aharon, M., Elad, M., & Bruckstein, A. (2006). K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on Signal Processing, 54(11), 4311–4322.

    Article  Google Scholar 

  6. Kim, Y. (2014). Convolutional neural networks for sentence classification. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), Doha, Qatar (pp. 1746–1751).

  7. Cai, L., & Hoffman, T. (2003). Text categorization by boosting automatically extracted concepts. In SIGIR '03: Proceedings of the 26th annual international ACM SIGIR conference on research and development in information retrieval, Toronto, Canada (pp. 182–189).

  8. Xie, M., Yin, H., Wang, H., Xu, F., Chen, W., & Wang, S. (2016) Learning graph-based POI embedding for location-based recommendation. In Proceedings of the 25th ACM international on conference on information and knowledge management, Indianapolis, Indiana, USA (pp. 15–24).

  9. Li, J., Peng, H., Liu, L., Xiong, G., Du, B., Ma, H., Wang, L., & Bhuiyan, Md. Z. (2013). Graph CNNs for urban traffic passenger flows prediction. In 2018 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI), Guangzhou, China.

  10. Peng, H., Li, J., Wang, S., Wang, L., Gong, Q., Yang, R., Li, B., Yu, P., & He, L. (2019). Hierarchical taxonomy-aware and attentional graph capsule RCNNs for large-scale multi-label text classification. IEEE Transactions on Knowledge and Data Engineering, 17, 1–1.

    Google Scholar 

  11. Williams, R., & Zipser, D. (2000). A learning algorithm for continually running fully recurrent neural networks. Neural Computation, 1(2), 270–280.

    Article  Google Scholar 

  12. Zhou, P., Qi, Z., Zheng, S., Xu, J., Bao, H., & Xu, B. (2016). Text classification improved by integrating bidirectional LSTM with two-dimensional max pooling. In Proceedings of COLING 2016, the 26th international conference on computational linguistics: Technical papers, Osaka, Japan (pp. 3485–3495).

  13. Nickel, M., & Kiela, D. (2017). Poincar\’e embeddings for learning hierarchical representations. In Proceedings of advances in neural information processing systems 30 (NIPS 2017), Long Beach, USA (pp 1–10).

  14. Ye, J., Ni, J., & Yi, Y. (2017). Deep learning hierarchical representations for image steganalysis. IEEE Transactions on Information Forensics and Security, 12(11), 2545–2557.

    Article  Google Scholar 

  15. Engan, K., Aase, S. O., & Hakon Husoy, J. (1999). Method of optimal directions for frame design. In Proceedings of IEEE international conference on acoustics, speech, and signal processing, Phoenix, USA, 15–19 March.

  16. Wright, J., Yang, A. Y., Ganesh, A., Sastry, S., & Ma, Y. (2009). Robust face recognition via sparse representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(2), 210–237.

    Article  Google Scholar 

  17. Yang, M., Zhang, L., Yang, J., & Zhang, D. (2010). Metaface learning for sparse representation based face recognition. In Proceedings of 2010 IEEE international conference on image processing, Hong Kong, China.

  18. Liu, Y., Peng, H., Li, J., Song, Y., & Li, X. (2020). Event detection and evolution in multi-lingual social streams. Frontiers of Computer Science, 14(5), 145612.

    Article  Google Scholar 

  19. Bota, P. J., Wang, C., Fred, A. L. N., & Plácido Da Silva, H. (2019). A review, current challenges, and future possibilities on emotion recognition using machine learning and physiological signals. IEEE Access, 7, 140990–141020.

    Article  Google Scholar 

  20. Filliat, D. (2007). A visual bag of words method for interactive qualitative localization and mapping. In Proceedings of IEEE international conference on robotics and automation, Roma, Italy (pp. 3921–3926).

  21. Zhang, Y., Jin, R., & Zhou, Z.-H. (2010). Understanding bag-of-words model: A statistical framework. International Journal of Machine Learning and Cybernetics, 1(1), 43–52.

    Article  Google Scholar 

  22. Aizawa, A. (2003). An information-theoretic perspective of tf–idf measures. Information Processing and Management: an International Journal, 39(1), 45–65.

    Article  Google Scholar 

  23. Wu, B., Li, C., & Wang, B. (2011). Event detection and evolution based on entity separation. In Proceedings of 2011 eighth international conference on fuzzy systems and knowledge discovery (FSKD) (Vol. 3, pp. 1803–1806).

  24. Rousseau, F., Kiagias, E., & Vazirgiannis, M. (2015). Text categorization as a graph classification problem. In Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing, Beijing, China (pp. 1702–1712).

  25. Peng, H., Li, J., Gong, Q., Wang, S., Ning, Y., & Yu, P. S. (2019). Graph convolutional neural networks via motif-based attention. In CIKM '19: proceedings of the 28th ACM international conference on information and knowledge management (pp. 499–508).

  26. Liu, P., Qiu, X., & Huang, X. (2016). Recurrent neural network for text classification with multi-task learning. In JCAI'16: Proceedings of the twenty-fifth international joint conference on artificial intelligence (pp. 2873–2879).

  27. Hochreiter, S., Schmidhuber, J. (1996). LSTM can solve hard long time lag problems. In Proceedings of the 9th international conference on neural information processing systems, Denver, Colorado (pp. 473–479).

  28. Lundervold, A. S., & Lundervold, A. (2019). An overview of deep learning in medical imaging focusing on MRI. Zeitschrift fur Medizinische Physik, 29(2), 102–127.

    Article  Google Scholar 

  29. Nwankpa, C., Gachagan, A., Ijomah, W., Marshall, S. Activation functions: Comparison of trends in practice and research for deep learning. arXiv preprint arXiv:1811.03378.

  30. Supreme Court of India. https://indiankanoon.org/browse/supremecourt/ Retrieved May 18, 2020.

  31. Verma, A., & Ranga, V. (2020). Machine learning based intrusion detection systems for IoT applications. Wireless Personal Communications, 111, 2287–2310.

    Article  Google Scholar 

  32. Zulqarnain, M., Ghazali, R., Ghouse, M., & Mushtaq, M. (2019). Efficient processing of GRU based on word embedding for text classification. International Journal of Informatica Visualization, 3(4), 377–383.

    Google Scholar 

  33. Lecunn, Y., Hinton, G., & Bengio, Y. (2015). Deep learning. Nature, 521(7553), 436–444.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Graphic Era Deemed to be University, Dehradun, India

    Varsha Mittal, Duraprasad Gangodkar & Bhaskar Pant

Authors
  1. Varsha Mittal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Duraprasad Gangodkar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bhaskar Pant
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Varsha Mittal.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mittal, V., Gangodkar, D. & Pant, B. Deep Graph-Long Short-Term Memory: A Deep Learning Based Approach for Text Classification. Wireless Pers Commun 119, 2287–2301 (2021). https://doi.org/10.1007/s11277-021-08331-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11277-021-08331-4

Keywords

Search

Navigation