skip to main content
10.1145/3585059.3611422acmconferencesArticle/Chapter ViewAbstractPublication PagesiteConference Proceedingsconference-collections
extended-abstract

The art of deep learning and natural language processing for emotional sentiment analysis on the academic scholars' peer review process.

Published:11 October 2023Publication History

ABSTRACT

Deep learning and natural language processing has emerged as modern machine with state-of-the-art modern learning and applicable techniques that help academicians, students, and teachers to identify, evaluate and validate documents and text features. Most educational systems have incorporated deep learning and natural language processing in fulfilling tasks except publishers and editorial boards, whom they handle important aspects of academic achievement in the decision-making for both journals and conferences. The aim is to introduce a standardized systematic peer review process based on factual justification, amendment, and recommendation approach on scholarly works free from emotional sentiments, politicization, bias, criticism, and condemnation. The study uses peer review feedback from journal XXXX to identify, evaluate and validate emotional sentimental aspects. A Confusion Metrix model was examined in the study to justify and validate the emotional sentiment pros and cons of the reviewer. Also, behavior-oriented drive and influential functions were used to convert text to value and rank the sentiment score. Each reviewer's text was converted into value using a behavior-oriented drive and influential function were used to examine the emotional sentiment level of the reviewers. Text mining and extraction of data uses a natural language process approach to identify sentiments from the reviewer's comments. Based on the confusion matrix and behavior-oriented drive, and influential function, the findings revealed limited significant emotional sentiment involved during the peer review process. The study concluded that the reviewers didn't perfectly validate the manuscript content due to minor emotional sentiments. The reviewer's evaluation of paper #Journal XXXX based on the findings exercise, few emotional sentiments were attracted during the review process. The study recommended editorial boards, journals, conferences, and publishing houses implement this approach to avoid emotional sentiments, politicization, and bias impact on academic evaluation, and validation.

References

  1. Adler, E., Hobbs, A., Dhaliwal, G., & Babik, J. M. (2020). Gender differences in authorship of clinical problem-solving articles. Journal of Hospital Medicine, 15(8), 475–478.Google ScholarGoogle Scholar
  2. Ain, Q. T., Ali, M., Riaz, A., Noureen, A., Kamran, M., Hayat, B., & Rehman, A. (2017). Sentiment analysis using deep learning techniques: a review. International Journal of Advanced Computer Science and Applications, 8(6).Google ScholarGoogle Scholar
  3. Alantari, H. J., Currim, I. S., Deng, Y., & Singh, S. (2022). An empirical comparison of machine learning methods for text-based sentiment analysis of online consumer reviews. International Journal of Research in Marketing, 39(1), 1-19.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. An, J., Mendenhall, A., & Kaeberlein, M. (2023). The collaborative peer review framework as a model for training biomedical graduate students to perform rigorous, ethical peer review. Translational Medicine of Aging.Google ScholarGoogle Scholar
  5. Asuquo, G., & UBANI, S. (2022). NEGRITUDE AND AFRICAN IDENTITY IN CONTEMPORARY AFRICAN THOUGHT. AMAMIHE Journal of Applied Philosophy, 20(1).Google ScholarGoogle Scholar
  6. Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.Google ScholarGoogle Scholar
  7. Beasley, Z. J., Friedman, A., & Rosen, P. (2021). Through the Looking Glass: Insights Into Visualization Pedagogy Through Sentiment Analysis of Peer Review Text. IEEE Computer Graphics and Applications, 41(6), 59-70.Google ScholarGoogle ScholarCross RefCross Ref
  8. Canbek, G., Taskaya Temizel, T., & Sagiroglu, S. (2021). BenchMetrics: A systematic benchmarking method for binary classification performance metrics. Neural Computing and Applications, 33(21), 14623-14650.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Chakraborty, S., Goyal, P., & Mukherjee, A. (2020, August). Aspect-based sentiment analysis of scientific reviews. In Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020 (pp. 207-216).Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Choi, G., Oh, S., & Kim, H. (2020). Improving document-level sentiment classification using importance of sentences. Entropy, 22(12), 1336.Google ScholarGoogle ScholarCross RefCross Ref
  11. Dou, Z. Y. (2017, September). Capturing user and product information for document level sentiment analysis with deep memory network. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (pp. 521-526).Google ScholarGoogle Scholar
  12. Edalati, M., Imran, A. S., Kastrati, Z., & Daudpota, S. M. (2022). The potential of machine learning algorithms for sentiment classification of students’ feedback on MOOC. In Intelligent Systems and Applications: Proceedings of the 2021 Intelligent Systems Conference (IntelliSys) Volume 3 (pp. 11-22). Springer International Publishing.Google ScholarGoogle Scholar
  13. Feldman, R. (2013). Techniques and applications for sentiment analysis. Communications of the ACM, 56(4), 82-89.Google ScholarGoogle Scholar
  14. Ghosal, T., Kumar, S., Bharti, P. K., & Ekbal, A. (2022). Peer review analyze: A novel benchmark resource for computational analysis of peer reviews. Plos one, 17(1), e0259238.Google ScholarGoogle Scholar
  15. Hoge, R. D., & Coladarci, T. (1989). Teacher-based judgments of academic achievement: A review of literature. Review of educational research, 59(3), 297-313.Google ScholarGoogle Scholar
  16. Huber, J., Inoua, S., Kerschbamer, R., König-Kersting, C., Palan, S., & Smith, V. L. (2022). Nobel and novice: Author prominence affects peer review. Proceedings of the National Academy of Sciences, 119(41), e2205779119.Google ScholarGoogle Scholar
  17. International Journal on Digital Libraries, 1-19.Google ScholarGoogle Scholar
  18. Iyoha, E. S., Aluko, J. O., & Ohwojevwe, E. G. (2018). Modern Drama: Signpost to Evolution Development of Science and Technology in Nigeria Polytechnics. S/N Contents, 1.Google ScholarGoogle Scholar
  19. Joseph-Edwards, A. (2023). The Perceptions and Use of Exemplars and Peer Feedback in Student Revisions. Caribbean Journal of Multidisciplinary Studies, 2(1), 45-64.Google ScholarGoogle Scholar
  20. Koczanowicz, D. (2009). COMMUNITY, INDIVIDUALITY, AND EXPERIENCE. In Self and Society (pp. 181-189). Brill.Google ScholarGoogle Scholar
  21. Kumar, A., & Garg, G. (2019). Sentiment analysis of multimodal twitter data. Multimedia Tools and Applications, 78, 24103-24119.Google ScholarGoogle Scholar
  22. Kumar, S., Ghosal, T., & Ekbal, A. (2023). DeepMetaGen: an unsupervised deep neural approach to generate template-based meta-reviews leveraging on aspect category and sentiment analysis from peer reviews. Google ScholarGoogle Scholar
  23. Le, Q., & Mikolov, T. (2014, June). Distributed representations of sentences and documents. In International conference on machine learning (pp. 1188-1196). PMLR.Google ScholarGoogle Scholar
  24. Levy, J. J., & O'Malley, A. J. (2020). Don't dismiss logistic regression: the case for sensible extraction of interactions in the era of machine learning. BMC medical research methodology, 20(1), 1-15.Google ScholarGoogle Scholar
  25. Li, D., Yin, H., Wang, C., Song, S., Li, K., & Li, C. (2022). Visual Recommendation for Peer-To-Peer Accommodation with Online Reviews based on Sentiment Analysis and Topic Models. Journal of Visualization, 25(6), 1309-1327.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Liu, B. (2012). Sentiment analysis and opinion mining. Synthesis lectures on human language technologies, 5(1), 1-167.Google ScholarGoogle Scholar
  27. Lou, Y., Zhang, Y., Li, F., Qian, T., & Ji, D. (2020). Emoji-based sentiment analysis using attention networks. ACM Transactions on asian and low-resource language information processing (TALLIP), 19(5), 1-13.Google ScholarGoogle Scholar
  28. Luo, J., Feliciani, T., Reinhart, M., Hartstein, J., Das, V., Alabi, O., & Shankar, K. (2021). Analyzing sentiments in peer review reports: Evidence from two science funding agencies. Quantitative Science Studies, 2(4), 1271-1295.Google ScholarGoogle ScholarCross RefCross Ref
  29. Malachowski, A. (2018). Rorty and the meaning of life. In The Meaning of Life and the Great Philosophers (pp. 266-273). Routledge.Google ScholarGoogle ScholarCross RefCross Ref
  30. Malviya, S., Tiwari, A. K., Srivastava, R., & Tiwari, V. (2020). Machine learning techniques for sentiment analysis: A review. SAMRIDDHI: A Journal of Physical Sciences, Engineering and Technology, 12(02), 72-78.Google ScholarGoogle Scholar
  31. Moraes, R., Valiati, J. F., & Neto, W. P. G. (2013). Document-level sentiment classification: An empirical comparison between SVM and ANN. Expert Systems with Applications, 40(2), 621-633.Google ScholarGoogle Scholar
  32. Morales-Trujillo, M. E., Galster, M., Gilson, F., & Mathews, M. (2021). A Three-Year Study on Peer Evaluation in a Software Engineering Project Course. IEEE Transactions on Education, 65(3), 409-418.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Moriguchi, Y. (2022). Beyond bias to Western participants, authors, and editors in developmental science. Infant and Child Development, 31(1), e2256.Google ScholarGoogle Scholar
  34. Mukerjee, J., & Metiu, A. (2022). Play and psychological safety: An ethnography of innovative work. Journal of Product Innovation Management, 39(3), 394-418.Google ScholarGoogle ScholarCross RefCross Ref
  35. Nigam, N., & Yadav, D. (2018). Lexicon-based approach to sentiment analysis of tweets using R language. In Advances in Computing and Data Sciences: Second International Conference, ICACDS 2018, Dehradun, India, April 20-21, 2018, Revised Selected Papers, Part I 2 (pp. 154-164). Springer Singapore.Google ScholarGoogle ScholarCross RefCross Ref
  36. Obi, J. C. (2023). A comparative study of several classification metrics and their performances on data. World Journal of Advanced Engineering Technology and Sciences, 8(1), 308-314.Google ScholarGoogle ScholarCross RefCross Ref
  37. Parekh, R., Patel, N. P., Thakkar, N., Gupta, R., Tanwar, S., Sharma, G., ... & Sharma, R. (2022). DL-GuesS: Deep learning and sentiment analysis-based cryptocurrency price prediction. IEEE Access, 10, 35398-35409.Google ScholarGoogle ScholarCross RefCross Ref
  38. Poria, S., Cambria, E., & Gelbukh, A. (2016). Aspect extraction for opinion mining with a deep convolutional neural network. Knowledge-Based Systems, 108, 42-49.Google ScholarGoogle Scholar
  39. Prasetyo, A. R., Sayono, J., & Rahmawati, N. (2022). Design of a Visual Identity to Build Branding of a Rattan Weaving Tourism Village in Malang City. KnE Social Sciences, 62-69.Google ScholarGoogle ScholarCross RefCross Ref
  40. Preston, T. (2006). The Private and Public Appeal of Self-Fashioning. Journal of Nietzsche Studies, 10-19.Google ScholarGoogle ScholarCross RefCross Ref
  41. Revathy, G., Alghamdi, S. A., Alahmari, S. M., Yonbawi, S. R., Kumar, A., & Haq, M. A. (2022). Sentiment analysis using machine learning: Progress in the machine intelligence for data science. Sustainable Energy Technologies and Assessments, 53, 102557.Google ScholarGoogle Scholar
  42. Ribeiro, A. C., Sizo, A., Lopes Cardoso, H., & Reis, L. P. (2021). Acceptance decision prediction in peer-review through sentiment analysis. In Progress in Artificial Intelligence: 20th EPIA Conference on Artificial Intelligence, EPIA 2021, Virtual Event, September 7–9, 2021, Proceedings 20 (pp. 766-777). Springer International Publishing.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Rowley, J., Sbaffi, L., Sugden, M., & Gilbert, A. (2022). Factors influencing researchers’ journal selection decisions. Journal of Information Science, 48(3), 321-335.Google ScholarGoogle Scholar
  44. Santamaría, L., & Mihaljević, H. (2018). Comparison and benchmark of name-to-gender inference services. PeerJ Computer Science, 4, e156.Google ScholarGoogle Scholar
  45. Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A. Y., & Potts, C. (2013, October). Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing (pp. 1631-1642).Google ScholarGoogle Scholar
  46. Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A. Y., & Potts, C. (2013, October). Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing (pp. 1631-1642).Google ScholarGoogle Scholar
  47. Suttles, J., & Ide, N. (2013). Distant supervision for emotion classification with discrete binary values. In Computational Linguistics and Intelligent Text Processing: 14th International Conference, CICLing 2013, Samos, Greece, March 24-30, 2013, Proceedings, Part II 14 (pp. 121-136). Springer Berlin Heidelberg.Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Tang, D. (2015, February). Sentiment-specific representation learning for document-level sentiment analysis. In Proceedings of the eighth ACM international conference on web search and data mining (pp. 447-452).Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Thelwall, M., Buckley, K., Paltoglou, G., Cai, D., & Kappas, A. (2010). Sentiment strength detection in short informal text. Journal of the American society for information science and technology, 61(12), 2544-2558.Google ScholarGoogle Scholar
  50. Tricco, A. C., Khalil, H., Holly, C., Feyissa, G., Godfrey, C., Evans, C., ... & Munn, Z. (2022). Rapid reviews and the methodological rigor of evidence synthesis: a JBI position statement. JBI evidence synthesis, 20(4), 944-949.Google ScholarGoogle Scholar
  51. Wang, K., & Wan, X. (2018, June). Sentiment analysis of peer review texts for scholarly papers. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval (pp. 175-184).Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Xu, J., Chen, D., Qiu, X., & Huang, X. (2016). Cached long short-term memory neural networks for document-level sentiment classification. arXiv preprint arXiv:1610.04989.Google ScholarGoogle Scholar
  53. Xu, J., Chen, D., Qiu, X., & Huang, X. (2016). Cached long short-term memory neural networks for document-level sentiment classification. arXiv preprint arXiv:1610.04989.Google ScholarGoogle Scholar
  54. Yadav, A., & Vishwakarma, D. K. (2020). Sentiment analysis using deep learning architectures: a review. Artificial Intelligence Review, 53(6), 4335-4385.Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Zhang, L., Shang, Y., Huang, Y., & Sivertsen, G. (2022). Gender differences among active reviewers: an investigation based on Publons. Scientometrics, 127(1), 145-179.Google ScholarGoogle Scholar
  56. Zhang, Z., Li, J., Zhu, P., Zhao, H., & Liu, G. (2018). Modeling multi-turn conversation with deep utterance aggregation. arXiv preprint arXiv:1806.09102.Google ScholarGoogle Scholar

Index Terms

  1. The art of deep learning and natural language processing for emotional sentiment analysis on the academic scholars' peer review process.
          Index terms have been assigned to the content through auto-classification.

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in
          • Published in

            cover image ACM Conferences
            SIGITE '23: Proceedings of the 24th Annual Conference on Information Technology Education
            October 2023
            230 pages
            ISBN:9798400701306
            DOI:10.1145/3585059

            Copyright © 2023 Owner/Author

            Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 11 October 2023

            Check for updates

            Qualifiers

            • extended-abstract
            • Research
            • Refereed limited

            Acceptance Rates

            Overall Acceptance Rate176of429submissions,41%
          • Article Metrics

            • Downloads (Last 12 months)56
            • Downloads (Last 6 weeks)1

            Other Metrics

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          HTML Format

          View this article in HTML Format .

          View HTML Format