Skip to main content

Adopting Pre-trained Large Language Models for Regional Language Tasks: A Case Study

  • Conference paper
  • First Online:
Intelligent Human Computer Interaction (IHCI 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14531))

Included in the following conference series:

Abstract

Large language models have revolutionized the field of Natural Language Processing. While researchers have assessed their effectiveness for various English language applications, a research gap exists for their application in low-resource regional languages like Marathi. The research presented in this paper intends to fill that void by investigating the feasibility and usefulness of employing large language models for sentiment analysis in Marathi as a case study. The study gathers a diversified and labeled dataset from Twitter that includes Marathi text with opinions classified as positive, negative, or neutral. We test the appropriateness of pre-existing language models such as Multilingual BERT (M-BERT), indicBERT, and GPT-3 ADA on the obtained dataset and evaluate how they performed on the sentiment analysis task. Typical assessment metrics such as accuracy, F1 score, and loss are used to assess the effectiveness of sentiment analysis models. This research paper presents additions to the growing area of sentiment analysis in languages that have not received attention. They open up possibilities for creating sentiment analysis tools and applications specifically tailored for Marathi-speaking communities.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Agüero-Torales, M.M., Salas, J.I.A., López-Herrera, A.G.: Deep learning and multilingual sentiment analysis on social media data: an overview. Appl. Soft Comput. 107, 107373 (2021)

    Article  Google Scholar 

  2. Ansari, M.A., Govilkar, S.: Sentiment analysis of mixed code for the transliterated Hindi and Marathi texts. Int. J. Nat. Lang. Comput. (IJNLC) 7 (2018)

    Google Scholar 

  3. Bender, E.M., Gebru, T., McMillan-Major, A., Shmitchell, S.: On the dangers of stochastic parrots: can language models be too big?. In: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pp. 610–623 (2021)

    Google Scholar 

  4. Chathuranga, P., Lorensuhewa, S., Kalyani, M.: Sinhala sentiment analysis using corpus based sentiment lexicon. In: 2019 19th International Conference on Advances in ICT for Emerging Regions (ICTer), vol. 250, pp. 1–7. IEEE (2019)

    Google Scholar 

  5. Deshmukh, R., Kiwelekar, A.W.: Deep convolutional neural network approach for classification of poems. In: Kim, J.-H., Singh, M., Khan, J., Tiwary, U.S., Sur, M., Singh, D. (eds.) IHCI 2021. LNCS, vol. 13184, pp. 74–88. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-98404-5_7

    Chapter  Google Scholar 

  6. Deshmukh, S., Patil, N., Rotiwar, S., Nunes, J.: Sentiment analysis of Marathi language. Int. J. Res. Publ. Eng. Technol. [IJRPET] 3, 93–97 (2017)

    Google Scholar 

  7. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

  8. Dhumal Deshmukh, R., Kiwelekar, A.: Deep learning techniques for part of speech tagging by natural language processing. In: 2020 2nd International Conference on Innovative Mechanisms for Industry Applications (ICIMIA), pp. 76–81 (2020)

    Google Scholar 

  9. Gillioz, A., Casas, J., Mugellini, E., Abou Khaled, O.: Overview of the transformer-based models for NLP tasks. In: 2020 15th Conference on Computer Science and Information Systems (FedCSIS), pp. 179–183. IEEE (2020)

    Google Scholar 

  10. Han, X., et al.: Pre-trained models: past, present and future. AI Open 2, 225–250 (2021)

    Article  Google Scholar 

  11. Jain, K., Deshpande, A., Shridhar, K., Laumann, F., Dash, A.: Indic-transformers: an analysis of transformer language models for Indian languages. arXiv preprint arXiv:2011.02323 (2020)

  12. Khan, R., Shrivastava, P., Kapoor, A., Tiwari, A., Mittal, A.: Social media analysis with AI: sentiment analysis techniques for the analysis of twitter COVID-19 data. J. Crit. Rev 7(9), 2761–2774 (2020)

    Google Scholar 

  13. Kublik, S., Saboo, S.: GPT-3. O’Reilly Media, Inc. (2022)

    Google Scholar 

  14. Kulkarni, A., Mandhane, M., Likhitkar, M., Kshirsagar, G., Joshi, R.: L3CubeMahaSent: a Marathi tweet-based sentiment analysis dataset. arXiv preprint arXiv:2103.11408 (2021)

  15. Lahoti, P., Mittal, N., Singh, G.: A survey on NLP resources, tools, and techniques for Marathi language processing. ACM Trans. Asian Low-Resour. Lang. Inf. Process. 22(2), 1–34 (2022)

    Article  Google Scholar 

  16. Min, B., et al.: Recent advances in natural language processing via large pre-trained language models: a survey. ACM Comput. Surv. (2021)

    Google Scholar 

  17. Naseem, U., Razzak, I., Khan, S.K., Prasad, M.: A comprehensive survey on word representation models: from classical to state-of-the-art word representation language models. Trans. Asian Low-Resour. Lang. Inf. Process. 20(5), 1–35 (2021)

    Article  Google Scholar 

  18. Nozza, D., Bianchi, F., Hovy, D.: What the [mask]? Making sense of language-specific BERT models. arXiv preprint arXiv:2003.02912 (2020)

  19. Patil, R.S., Kolhe, S.R.: Supervised classifiers with TF-IDF features for sentiment analysis of Marathi tweets. Soc. Netw. Anal. Min. 12(1), 51 (2022)

    Article  Google Scholar 

  20. Sawicki, P., et al.: On the power of special-purpose GPT models to create and evaluate new poetry in old styles (2023)

    Google Scholar 

  21. Smith, S., et al.: Using deepspeed and megatron to train megatron-turing NLG 530B, a large-scale generative language model. arXiv preprint arXiv:2201.11990 (2022)

  22. Soong, H.C., Jalil, N.B.A., Ayyasamy, R.K., Akbar, R.: The essential of sentiment analysis and opinion mining in social media: introduction and survey of the recent approaches and techniques. In: 2019 IEEE 9th Symposium on Computer Applications & Industrial Electronics (ISCAIE), pp. 272–277. IEEE (2019)

    Google Scholar 

  23. Torfi, A., Shirvani, R.A., Keneshloo, Y., Tavaf, N., Fox, E.A.: Natural language processing advancements by deep learning: a survey. arXiv preprint arXiv:2003.01200 (2020)

  24. Vidyavihar, M.: Sentiment analysis in Marathi language. Int. J. Recent Innov. Trends Comput. Commun. 5(8), 21–25 (2017)

    Google Scholar 

  25. Zhou, C., et al.: A comprehensive survey on pretrained foundation models: a history from BERT to ChatGPT. arXiv preprint arXiv:2302.09419 (2023)

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Harsha Gaikwad .

Editor information

Editors and Affiliations

Ethics declarations

Data and Model Availability

The dataset and working models for the proposed article are available on the GitHub repository. The link to the GitHub repository is https://github.com/CompDbatu/MarathiSentimentAnalysis.

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Gaikwad, H., Kiwelekar, A., Laddha, M., Shahare, S. (2024). Adopting Pre-trained Large Language Models for Regional Language Tasks: A Case Study. In: Choi, B.J., Singh, D., Tiwary, U.S., Chung, WY. (eds) Intelligent Human Computer Interaction. IHCI 2023. Lecture Notes in Computer Science, vol 14531. Springer, Cham. https://doi.org/10.1007/978-3-031-53827-8_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-53827-8_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-53826-1

  • Online ISBN: 978-3-031-53827-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics