Abstract
Modern Large Language Models (LLMs), such as ChatGPT, demonstrate exceptional capabilities in text classification and reasoning. The categorization of severity levels for descriptions of Power Defects and the inference of Defect Causes present an innovative and challenging task aimed at providing comprehensive and accurate reasoning pathways to power grid workers. In this study, a comparison is made among three Chain-of-Thought (CoT) prompting methods and the Role-Play prompting method using a Power Grid dataset. It is observed that the manually designed Manual-CoT method achieves the best results, with other methods showing significant improvements in classification accuracy and the coherence of reasoning pathways. This further highlights the potential for substantial enhancement of Large Language Models’ reasoning abilities in specialized domains through expert-guided template pathways.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Wei, J., et al.: Chain-of-thought prompting elicits reasoning in large language models. In: Advances in Neural Information Processing Systems, vol. 35, pp. 24824–24837 (2022)
Brown, T., et al.: Language models are few-shot learners. In: Advances in Neural Information Processing Systems, vol. 33, pp. 1877–1901 (2020)
Kojima, T., Gu, S.S., Reid, M., Matsuo, Y., Iwasawa, Y.: Large language models are zero-shot reasoners. In: Advances in Neural Information Processing Systems, vol. 35, pp. 22199–22213 (2022)
Zhang, Z., Zhang, A., Li, M., Smola, A.: Automatic chain of thought prompting in large language models. arXiv preprint arXiv:2210.03493 (2022)
Blair-Stanek, A., Holzenberger, N., Van Durme, B.: Can GPT-3 perform statutory reasoning? arXiv preprint arXiv:2302.06100 (2023)
Chowdhery, A., et al.: PaLM: scaling language modeling with pathways. arXiv preprint arXiv:2204.02311 (2022)
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al.: Language models are unsupervised multitask learners. Open AI Blog 1(8), 9 (2019)
Rubin, O., Herzig, J., Berant, J.: Learning to retrieve prompts for in-context learning. arXiv preprint arXiv:2112.08633 (2021)
Mishra, S., Khashabi, D., Baral, C., Hajishirzi, H.: Cross-task generalization via natural language crowdsourcing instructions. arXiv preprint arXiv:2104.08773 (2021)
Holtzman, A., West, P., Shwartz, V., Choi, Y., Zettlemoyer, L.: Surface form competition: why the highest probability answer isn’t always right. arXiv preprint arXiv:2104.08315 (2021)
Liu, J., Shen, D., Zhang, Y., Dolan, B., Carin, L., Chen, W.: What makes good in-context examples for GPT-3? arXiv preprint arXiv:2101.06804 (2021)
Webson, A., Pavlick, E.: Do prompt-based models really understand the meaning of their prompts? arXiv preprint arXiv:2109.01247 (2021)
Kong, A.: Better zero-shot reasoning with role-play prompting. arXiv preprint arXiv:2308.07702 (2023)
Kaddour, J., Harris, J., Mozes, M., Bradley, H., Raileanu, R., McHardy, R.: Challenges and applications of large language models. arXiv preprint arXiv:2307.10169 (2023)
Ling, C., et al.: Beyond one-model-fits-all: a survey of domain specialization for large language models. arXiv preprint arXiv:2305.18703 (2023)
Mirowski, P., Mathewson, K.W., Pittman, J., Evans, R.: Co-writing screen plays and theatre scripts with language models: evaluation by industry professionals. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pp. 1–34 (2023)
Park, P.S., Schoenegger, P., Zhu, C.: Artificial intelligence in psychology research. arXiv preprint arXiv:2302.07267 (2023)
Shuster, K., et al.: BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage. arXiv preprint arXiv:2208.03188 (2022)
Singhal, K., et al.: Large language models encode clinical knowledge. arXiv preprint arXiv:2212.13138 (2022)
Yu, F., Quartey, L., Schilder, F.: Legal prompting: teaching a language model to think like a lawyer. arXiv preprint arXiv:2212.01326 (2022)
Liu, S.: Towards emotional support dialog systems. arXiv preprint arXiv:2106.01144 (2021)
McHugh, M.L.: Interrater reliability: the kappa statistic. Biochemia medica 22(3), 276–282 (2012)
Acknowledgement
This work is supported by the Research Funds from State Grid Shaanxi (SGSNBJ 00BYJS2311111).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Xu, J. et al. (2024). Enhancing Reasoning Pathways for Power Defect Analysis Using CoT and Role-Play Prompt. In: Huang, DS., Premaratne, P., Yuan, C. (eds) Applied Intelligence. ICAI 2023. Communications in Computer and Information Science, vol 2015. Springer, Singapore. https://doi.org/10.1007/978-981-97-0827-7_30
Download citation
DOI: https://doi.org/10.1007/978-981-97-0827-7_30
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-0826-0
Online ISBN: 978-981-97-0827-7
eBook Packages: Computer ScienceComputer Science (R0)