Skip to main content
SpringerLink
Log in

Deep learning based sequence to sequence model for abstractive telugu text summarization

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

With the emergence of deep learning, the attention of researchers has increased significantly towards abstractive text summarization approaches. Though extractive text summarization (ETS) is an important approach, the generated summaries are not always coherent. This paper mainly focuses on the abstractive text summarization (ATS) approach for Telugu language to generate coherent summary. The majority research on ATS approach is conducted in English, while no significant research in Telugu has been documented. An abstractive Telugu text summarization model based on sequence-to-sequence (seq2seq) encoder-decoder architecture is proposed in this paper. The seq2seq model is implemented with bidirectional long short-term memory (Bi-LSTM) based encoder and long short-term memory (LSTM) based decoder. The existing ATS approaches have some drawbacks such as they cannot handle out vocabulary words, attention deficiency issue arising while handling long text sequence and repetition problem. To overcome these issues, some operating mechanisms like pointer generator network, temporal attention mechanism and coverage mechanism are also integrated in the proposed model. Besides, diverse beam search decoding algorithm is also employed to increase the diversity of generated summary. Thus, the proposed seq2seq model is the combination of Bi-LSTM and LSTM based encoder-decoder, pointer generator network, temporal attention mechanism, coverage mechanism and diverse beam search decoding algorithm. The performance of the proposed work is evaluated using the ROUGE toolkit in terms of F-measure, recall and precision. The experimental results of the proposed scheme are evaluated with other existing methods to show that the proposed ATS model outperforms existing Telugu text summarization models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 1
Fig. 6

Similar content being viewed by others

Data availability

Data will be shared on the reasonable request.

References

  1. Allahyari M, Pouriyeh S, Assefi M, Safaei S, Trippe ED, Gutierrez JB, Kochut K (2017) Text summarization techniques: a brief survey. arXiv. https://doi.org/10.1177/1010428317692226

  2. Alquliti WH, Abdul Ghani NB (2019) Convolutional neural network based for automatic text summarization. Int J Advan Comput Sci Applic (IJACSA) 10(4). https://doi.org/10.14569/IJACSA.2019.0100424

  3. Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. ArXiv:1409–0473

  4. Bengio Y, Simard P, Frasconi P (1994) Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Netw 5(2):157–166. https://doi.org/10.1109/72.279181

    Article  Google Scholar 

  5. Chung J, Gulcehre C, Cho K and Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling In NIPS 2014 Workshop on Deep Learning: https://doi.org/10.1109/TVCG.2013.272

  6. Cibils A, Musat C, Hossman A, Baeriswyl M (2018) Diverse beam search for increased novelty in abstractive summarization. arXiv preprint arXiv:1802.01457

  7. Gimpel K, Batra D, Dyer C, Shakhnarovich G (2013) A systematic exploration of diversity in machine translation. In proceedings of the 2013 conference on empirical methods in natural language processing. 1100–1111: https://doi.org/10.1016/j.cbpa.2013.03.038

  8. Google colab: https://colab.research.google.com/notebooks/intro.ipynb#recent=true.

  9. Gu J, Lu Z, Li H, Li VOK (2016) Incorporating copying mechanism in sequence-to-sequence learning. https://arxiv.org/abs/1603.06393,  https://doi.org/10.13703/j.0255-2930.2016.11.022

  10. Gulati AN, Sawarkar SD (2017) A novel technique for multidocument Hindi text summarization. 2017 international conference on nascent Technologies in Engineering (ICNTE). https://doi.org/10.1109/icnte.2017.7947890

  11. Hernandez-Castaneda A, Garcia-Hernandez RA, Ledeneva Y, Millan-Hernandez CE (2020) Extractive automatic text summarization based on lexical-semantic keywords. IEEE Access, 1–1. https://doi.org/10.1109/access.2020.2980226

  12. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

  13. Joudar NE, Ettaouil M (2022) KRR-CNN: kernels redundancy reduction in convolutional neural networks. Neural Comput Applic 34(3):2443–2454

    Article  Google Scholar 

  14. Kallimani J, Srinivasa K, Eswara B (2011) Information extraction by an abstractive text summarization for an Indian regional language. 319–322. https://doi.org/10.1109/NLPKE.2011.6138217

  15. Kanitha DK, Mubarak DMN, Shanavas SA (2018) Malayalam Text Summarization Using Graph Based Method. Int J Comput Sci Info Technol 9(2):40–44

    Google Scholar 

  16. Krause J, Johnson J, Krishna R, Fei-Fei L (2017) A hierarchical approach for generating descriptive image paragraphs. In proceedings of the IEEE conference on computer vision and pattern recognition. 317–325.

  17. Latha YM, Sudha DN (2020) Multi-document abstractive text summarization through semantic similarity matrix for Telugu language. Int J Advanc Sci Technol 29(1):513–521 http://sersc.org/journals/index.php/IJAST/article/view/3105

    Google Scholar 

  18. Lin CY (2004) 2004. ROUGE, A package for automatic evaluation of summaries. Text Summarization Branches Out: https://doi.org/10.1179/cim.2004.5.Supplement-1.132

  19. Loper E, Bird S (2002). NLTK: The Natural Language Toolkit. CoRR. cs.CL/0205028. https://doi.org/10.3115/1118108.1118117

  20. Mamidala KK, Sanampudi S (2021) Text summarization on Telugu e-news based on long-short term memory with rectified Adam optimizer. Int J Comput Digital Syst 11(1)

  21. Manjari KU (2020) Extractive summarization of Telugu documents using TextRank algorithm. 2020 fourth international conference on I-SMAC (IoT in social, Mobile, analytics and cloud) (I-SMAC). https://doi.org/10.1109/i-smac49090.2020.9243568

  22. Mohammad Masum AK, Abujar S, Islam Talukder MA, Azad Rabby AS, Hossain SA (2019) Abstractive method of text summarization with sequence to sequence RNNs. 2019 10th international conference on computing, Commun Networking Technol (ICCCNT) https://doi.org/10.1109/icccnt45670.2019.8944620.

  23. Mohan Bharath B, Aravindh Gowtham B, Akhil M (2022) Neural abstractive text summarizer for Telugu language. In soft computing and signal processing (pp. 61–70). Springer, Singapore

  24. Naidu R, Bharti D, Babu K, Mohapatra R (2017) Text Summarization with Automatic Keyword Extraction in Telugu e-Newspapers: https://doi.org/10.21037/cdt.2017.08.14

  25. Nallapati R, Zhou B, Santos CD, Gulçehre CG, Xiang B (2016) Abstractive text summarization using sequence-to-sequence RNNs and beyond. CoNLL 2016:280

    Google Scholar 

  26. Norouzi R, Baziyad H, Aknondzadeh Noghabi E, Albadvi A (2022) Developing tourism users’ profiles with data-driven explicit information. Math Probl Eng 2022:1–14

    Google Scholar 

  27. Pan HX, Liu H, Tang Y (2019) A sequence-to-sequence text summarization model with topic based attention mechanism. In: Ni W, Wang X, Song W, Li Y (eds) Web Information Systems and Applications. WISA 2019. Lecture notes in computer science, vol. 11817. Springer, Cham. https://doi.org/10.1007/978-3-030-30952-7_29.

  28. Paulus R, Xiong C, Socher R (2017) A deep reinforced model for abstractive summarization. arXiv preprint arXiv:1705.04304: https://doi.org/10.3389/fpsyg.2017.01779

  29. Priyadharshan T, Sumathipala S (2018) Text summarization for Tamil online sports news using NLP. 2018 3rd international conference on information technology research (ICITR). https://doi.org/10.1109/icitr.2018.8736154

  30. Rodrigues S, Fernandes S, Pai A (2019) "Konkani Text Summarization By Sentence Extraction," 2019 10th international conference on computing, communication and networking technologies (ICCCNT), Kanpur, India, pp. 1–6, https://doi.org/10.1109/ICCCNT45670.2019.8944575

  31. Rush AM, Chopra S, Weston J (2015) A neural attention model for abstractive sentence summarization. In proceedings of the 2015 conference on empirical methods in natural language processing. 379–389.

  32. Rush AM, Chopra S, Weston J (2015) A neural attention model for abstractive sentence summarization. ArXiv:1509–00685

  33. Sarwadnya VV, Sonawane SS (2018) Marathi extractive text summarizer using graph based model. 2018 fourth international conference on computing communication control and automation (ICCUBEA). https://doi.org/10.1109/iccubea.2018.8697741

  34. See A, Liu PJ, Manning CD (2017) Get to the point: summarization with pointer-generator networks. In proceedings of the 55th annual meeting of the Association for Computational Linguistics (volume 1: long papers). Assoc Comput Linguistics. 1073–1083

  35. Shi T, Keneshloo Y, Ramakrishnan N, Reddy CK (2020) Neural abstractive text summarization with sequence-to-sequence models. ACM Trans, Data Sci

  36. Song S, Huang H, Ruan T (2018) Abstractive text summarization using LSTM-CNN based deep learning. Multimed Tools Appl 78:857–875. https://doi.org/10.1007/s11042-018-5749-3

    Article  Google Scholar 

  37. Sutskever I, Vinyals O, Le QV (2014) Sequence to sequence learning with neural networks. In: Proc. Adv Neural Inf Process Syst, 3104–3112

  38. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Advance Neural Inform Process Syst:6000–6010

  39. Vinyals O, Fortunato M, Jaitly N (2015) Pointer networks. In: Proc. Adv. Neural Inf. Process. Syst., pp. 2692–2700

  40. Wang S, Zhao X, Li B, Ge B, Tang D (2017) Integrating extractive and abstractive models for long text summarization. 2017 IEEE international congress on big data (BigData congress). https://doi.org/10.1109/bigdatacongress.2017.46

  41. Zhang Y, Xiao W (2018) Keyphrase generation based on deep Seq2seq model. IEEE access, 1–1. https://doi.org/10.1109/access.2018.2865589

  42. Zhang Y, Kampffmeyer M, Liang X, Zhang D, Tan M, Xing EP (2019) Dilated temporal relational adversarial network for generic video summarization. Multimed Tools Appl 78(24):35237–35261. https://doi.org/10.1007/s11042-019-08175-y

    Article  Google Scholar 

  43. Zhang Y, Liang X, Zhang D, Tan M, Xing EP (2020) Unsupervised object-level video summarization with online motion auto-encoder. Pattern Recogn Lett 130:376–385. https://doi.org/10.1016/j.patrec.2018.07.030

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University College of Engineering, Osmania University, Hyderabad, India

    G. L. Anand Babu

  2. Department of CSE, Stanley College of Engineering and Technology for Women, Abids, Hyderabad, India

    Srinivasu Badugu

Authors
  1. G. L. Anand Babu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Srinivasu Badugu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to G. L. Anand Babu.

Ethics declarations

I confirm that I have read, understand and agreed to the submission guidelines, policies and submission `declaration of the journal.

I confirm that the manuscript is the authors’ original work and the manuscript has not received prior publication and is not under consideration for publication elsewhere.

I confirm that all authors listed on the title page have contributed significantly to the work, have read the manuscript, attest to the validity and legitimacy of the data and its interpretation, and agree to its submission.

I confirm that the paper now submitted is not copied or plagiarized version of some other published work.

I declare that I shall not submit the paper for publication in any other Journal or Magazine till the decision is made by journal editors.

I understand that submission of false or incorrect information/undertaking would invite appropriate penal actions as per norms/rules of the journal.

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Babu, G.L.A., Badugu, S. Deep learning based sequence to sequence model for abstractive telugu text summarization. Multimed Tools Appl 82, 17075–17096 (2023). https://doi.org/10.1007/s11042-022-14099-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-14099-x

Keywords

Search

Navigation