research-article

Arabic Sentiment Analysis for ChatGPT Using Machine Learning Classification Algorithms: A Hyperparameter Optimization Technique

Authors:

Ahmad Nasayreh,

Rabia Emhamed Al Mamlook,

Ghassan Samara,

Hasan Gharaibeh,

Mohammad Aljaidi,

Essam Al-Daoud,

Laith AbualigahAuthors Info & Claims

ACM Transactions on Asian and Low-Resource Language Information Processing, Volume 23, Issue 3

Article No.: 36, Pages 1 - 19

https://doi.org/10.1145/3638285

Published: 09 March 2024 Publication History

Abstract

In the realm of ChatGPT's language capabilities, exploring Arabic Sentiment Analysis emerges as a crucial research focus. This study centers on ChatGPT, a popular machine learning model engaging in dialogues with users, garnering attention for its exceptional performance and widespread impact, particularly in the Arab world. The objective is to assess people's opinions about ChatGPT, categorizing them as positive or negative. Despite abundant research in English, there is a notable gap in Arabic studies. We assembled a dataset from X (formerly known as Twitter), comprising 2,247 tweets, classified by Arabic language specialists. Employing various machine learning algorithms, including Support Vector Machine (SVM), Logistic Regression (LR), Random Forest (RF), and Naïve Bayes (NB), we implemented hyperparameter optimization techniques such as Bayesian optimization, Grid Search, and random search to select the best hyperparameters that contribute to achieving the best performance. Through training and testing, performance enhancements were observed with optimization algorithms. SVM exhibited superior performance, achieving 90% accuracy, 88% precision, 95% recall, and 91% F1 score with Grid Search. These findings contribute valuable insights into ChatGPT's impact in the Arab world, offering a comprehensive understanding of sentiment analysis through machine learning methodologies.

References

[1]

M. A. Hassonah, R. Al-Sayyed, A. Rodan, A. M. Al-Zoubi, I. Aljarah, and H. Faris. 2020. An efficient hybrid filter and evolutionary wrapper approach for sentiment analysis of various topics on Twitter. Knowledge-Based Syst. 192 (2020), 105353. DOI:

[2]

M. Tubishat, M. A. M. Abushariah, N. Idris, and I. Aljarah. 2019. Improved whale optimization algorithm for feature selection in Arabic sentiment analysis. Appl. Intell. 49, 5 (2019), 1688–1707. DOI:

Digital Library

[3]

I. Guellil, F. Azouaou, and M. Mendoza. 2019. Arabic sentiment analysis: Studies, resources, and tools. Soc. Netw. Anal. Min. 9 (2019), 1–17.

[4]

X. Zhai. 2023. Chatgpt and ai: The game changer for education. Available at SSRN.

[5]

P. P. Ray. 2023. ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet Things Cyber-Phys. Syst. 3 (April 2023), 121–154. DOI:

[6]

M. M. Mr and S. R. Mr. 2020. A comparative study of lexicon based and machine learning based classifications in sentiment analysis. Int. J. Data Min. Tech. Appl. 9, 2 (2020), 83–87. DOI:

[7]

M. E. M. Abo, R. G. Raj, and A. Qazi. 2019. A review on Arabic sentiment analysis: State-of-the-art, taxonomy and open research challenges. IEEE Access 7 (2019), 162008–162024. DOI:

[8]

I. Gupta and N. Joshi. 2020. Enhanced Twitter sentiment analysis using hybrid approach and by accounting local contextual semantic. J. Intell. Syst. 29, 1 (2020), 1611–1625. DOI:

[9]

M. Hijjawi and Y. Elsheikh. 2015. Arabic language challenges in text based conversational agents compared to the English language. Int. J. Comput. Sci. Inf. Technol. 7, 3 (2015), 1–13. DOI:

[10]

N. E. Aoumeur, Z. Li, and E. M. Alshari. 2023. Improving the polarity of text through word2vec embedding for primary classical Arabic sentiment analysis. Neural Process. Lett. 55 (2023), 2249–2264. DOI:

Digital Library

[11]

A. Alsanad. 2022. An improved Arabic sentiment analysis approach using optimized multinomial Naïve Bayes classifier. Int. J. Adv. Comput. Sci. Appl. 13, 8 (2022), 90–98. DOI:

[12]

M. Heikal, M. Torki, and N. El-Makky. 2018. Sentiment analysis of Arabic tweets using deep learning. Procedia Comput. Sci. 142 (2018), 114–122. DOI:

Digital Library

[13]

S. Bahassine, A. Madani, M. Al-Sarem, and M. Kissi. 2020. Feature selection using an improved Chi-square for Arabic text classification. J. King Saud Univ. Comput. Inf. Sci. 32, 2 (2020), 225–231. DOI:

Digital Library

[14]

M. M. Ali. 2021. Arabic sentiment analysis about online learning to mitigate Covid-19. J. Intell. Syst. 30, 1 (2021), 524–540. DOI:

[15]

M. Maghfour and A. Elouardighi. 2018. Standard and Dialectal Arabic Text Classification for Sentiment Analysis, Lecture Notes in Computer Science, Vol. 11163. Springer International Publishing. DOI:

[16]

S. N. Alyami and S. O. Olatunji. 2020. Application of support vector machine for Arabic sentiment classification using Twitter-based dataset. J. Inf. Knowl. Manag. 19, 1 (2020), 1–13. DOI:

[17]

K. M. Alomari, H. M. Elsherif, and K. Shaalan. 2017. Arabic tweets sentimental analysis using machine learning. Lecture Notes in Computer Science. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), Vol. 10350. Springer. 602–610. DOI:

[18]

Abdulmajeed Aljuhani and Abdulaziz Alhubaishy. 2020. Incorporating a decision support approach within the agile mobile application development process. In Proceedings of the 3rd International Conference on Computer Applications & Information Security (ICCAIS ’20). 23–26.

[19]

M. Hadwan, M. A. Al-Hagery, M. Al-Sarem, and F. Saeed. 2022. Arabic sentiment analysis of users’ opinions of governmental mobile applications. Comput. Mater. Contin. 72, 3 (2022), 4675–4689. DOI:

[20]

A. Erfina. 2023. Implementation of Naive Bayes classification algorithm for Twitter user sentiment analysis on ChatGPT using Python programming. Data Metadata 2 (2023), 2–11. DOI:

[21]

J. P. Munggaran, A. A. Alhafidz, M. Taqy, D. Aprianti, R. Agustini, and M. Munawir. 2023. Sentiment analysis of Twitter users’ opinion data regarding the use of ChatGPT in education 2, 2 (2023), 75–88. https://ejournal.upi.edu/index.php/COELITE/article/view/59645

[22]

W. P. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer. 2002. SMOTE: Synthetic minority over-sampling technique. J. Artif. Intell. Res. 16 (2002), 321–357.

[23]

S. Malviya, A. K. Tiwari, R. Srivastava, and V. K. Tiwari. 2020. Machine learning techniques for sentiment analysis: A review. SAMRIDDHI: J. Phys. Sci. Eng. Technol. 12, 2 (2020), 72–78. DOI:

[24]

Y. Wang, Z. Zhou, S. Jin, D. Liu, and M. Lu. 2017. Comparisons and selections of features and classifiers for short text classification. IOP Conf. Ser. Mater. Sci. Eng. 261, 1 (2017), 261012018. DOI:

[25]

J. H. Jurafsky and D. Martin. 2017. Naive Bayes and sentiment classification. Speech and Language Processing, 74–91.

[26]

S. Huang, C. A. I. Nianguang, P. Penzuti Pacheco, S. Narandes, Y. Wang, and X. U. Wayne. 2018. Applications of support vector machine (SVM) learning in cancer genomics. Cancer Genomics Proteomics 15, 1 (2018), 41–51. DOI:

[27]

G. Biau and E. Scornet. 2016. A random forest guided tour. Test 25, 2 (2016), 197–227. DOI:

[28]

L. Yang and A. Shami. 2020. On hyperparameter optimization of machine learning algorithms: Theory and practice. Neurocomputing 415 (2020), 295–316, DOI:

[29]

B. H. Shekar and G. Dagnew. 2019. Grid search-based hyperparameter tuning and classification of microarray cancer data. In Proceedings of the 2019 2nd International Conference on Advanced Computational and Communication Paradigms (ICACCP ’19), 1–8. DOI:

[30]

J. Bergstra and Y. Bengio. 2012. Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13 (2012), 281–305.

[31]

J. Wu, X. Y. Chen, H. Zhang, L. D. Xiong, H. Lei, and S. H. Deng. 2019. Hyperparameter optimization for machine learning models based on Bayesian optimization. J. Electron. Sci. Technol. 17, 1 (2019), 26–40. DOI:

[32]

E. Brochu, V. M. Cora, and N. De Freitas. 2010. A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv:1012.2599.

[33]

E. Elgeldawi, A. Sayed, A. R. Galal, and A. M. Zaki. 2021. Hyperparameter tuning for machine learning algorithms used for Arabic sentiment analysis. Informatics 8, 4 (2021), 1–21. DOI:

Cited By

Al Daoud EAl Daoud LAsassfeh MAl-Shaikh AAl-Sherideh AAfaneh S(2024)Enhancing Cybersecurity with Transformers: Preventing Phishing Emails and Social Media Scams2024 IEEE Conference on Dependable and Secure Computing (DSC)10.1109/DSC63325.2024.00017(31-36)Online publication date: 6-Nov-2024
https://doi.org/10.1109/DSC63325.2024.00017
Ullah Sourav MSultan Mahmud MZheng HAljaidi MHasan Talukder MBin Sulaiman RNur AAl-Qerem A(2024)Transformer-based Text Classification on Unified Bangla Multi-class Emotion Corpus2024 25th International Arab Conference on Information Technology (ACIT)10.1109/ACIT62805.2024.10877210(1-7)Online publication date: 10-Dec-2024
https://doi.org/10.1109/ACIT62805.2024.10877210
Villegas-Ch WGovea JGutierrez R(2024)Optimizing Language Model-Based Educational Assistants Using Knowledge Graphs: Integration with Moodle LMSIEEE Access10.1109/ACCESS.2024.3518952(1-1)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3518952

Index Terms

Arabic Sentiment Analysis for ChatGPT Using Machine Learning Classification Algorithms: A Hyperparameter Optimization Technique
1. Computing methodologies
  1. Machine learning
    1. Machine learning algorithms
      1. Feature selection

Recommendations

Negation Handling in Machine Learning-Based Sentiment Classification for Colloquial Arabic

One crucial aspect of sentiment analysis is negation handling, where the occurrence of negation can flip the sentiment of a review and negatively affects the machine learning-based sentiment classification. The role of negation in Arabic sentiment ...
Sampling techniques for Arabic Sentiment Classification: A Comparative Study
NISS '20: Proceedings of the 3rd International Conference on Networking, Information Systems & Security

Over the last decade, the web 2.0 has been shifting the web to turn it into an opinion platform. This results in a large amount of raw data that overwhelms human capacity to extract valuable knowledge without assistance of machines. In real world ...
Sentiment Polarity Classification using Minimal Feature Vectors and Machine Learning Algorithms
IAIT '21: Proceedings of the 12th International Conference on Advances in Information Technology

Recently, social media users can comment as texts to describe their opinions. These texts can be analyzed to classify them into either positive or negative attitude. Feature vectors for representing the texts must be designed and prepared before ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Asian and Low-Resource Language Information Processing

ACM Transactions on Asian and Low-Resource Language Information Processing Volume 23, Issue 3

March 2024

277 pages

EISSN:2375-4702

DOI:10.1145/3613569

Editor:
Imed Zitouni
Google, USA

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 March 2024

Online AM: 15 January 2024

Accepted: 12 December 2023

Revised: 11 November 2023

Received: 04 October 2023

Published in TALLIP Volume 23, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
423
Total Downloads

Downloads (Last 12 months)255
Downloads (Last 6 weeks)21

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Al Daoud EAl Daoud LAsassfeh MAl-Shaikh AAl-Sherideh AAfaneh S(2024)Enhancing Cybersecurity with Transformers: Preventing Phishing Emails and Social Media Scams2024 IEEE Conference on Dependable and Secure Computing (DSC)10.1109/DSC63325.2024.00017(31-36)Online publication date: 6-Nov-2024
https://doi.org/10.1109/DSC63325.2024.00017
Ullah Sourav MSultan Mahmud MZheng HAljaidi MHasan Talukder MBin Sulaiman RNur AAl-Qerem A(2024)Transformer-based Text Classification on Unified Bangla Multi-class Emotion Corpus2024 25th International Arab Conference on Information Technology (ACIT)10.1109/ACIT62805.2024.10877210(1-7)Online publication date: 10-Dec-2024
https://doi.org/10.1109/ACIT62805.2024.10877210
Villegas-Ch WGovea JGutierrez R(2024)Optimizing Language Model-Based Educational Assistants Using Knowledge Graphs: Integration with Moodle LMSIEEE Access10.1109/ACCESS.2024.3518952(1-1)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3518952

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

Figures

Tables

Media

View full text|Download PDF

View Issue’s Table of Contents