Abstract
The recommendation of scientific workflows from public repositories that meet users’ natural language requirements is becoming increasingly essential in the scientific community. Nevertheless, existing methods that rely on direct text matching encounter difficulties when it comes to handling complex queries, which ultimately results in poor performance. Large language models (LLMs) have recently exhibited exceptional ability in planning and reasoning. We propose “Plan, Generate and Match” (PGM), a scientific workflow recommendation method leveraging LLMs. PGM consists of three stages: utilizing LLMs to conduct planning upon receiving a user query, generating a structured workflow specification guided by the solution steps, and using these plans and specifications to match with candidate workflows. By incorporating the planning mechanism, PGM leverages few-shot prompting to automatically generate well-considered steps for instructing the recommendation of reliable workflows. This method represents the first exploration of incorporating LLMs into the scientific workflow domain. Experimental results on real-world benchmarks demonstrate that PGM outperforms state-of-the-art methods with statistical significance, highlighting its immense potential in addressing complex requirements.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Djaffardjy, M., et al.: Developing and reusing bioinformatics data analysis pipelines using scientific workflow systems. Comput. Struct. Biotechnol. J. (2023)
Gu, Y., Cao, J., Qian, S., Zhu, N., Guan, W.: MANSOR: a module alignment method based on neighbor information for scientific workflow. Concurrency Comput. Pract. Exp., e7736 (2023)
De Roure, D., Goble, C., Stevens, R.: The design and realisation of the myExperiment Virtual Research Environment for social sharing of workflows. Futur. Gener. Comput. Syst. 25(5), 561–567 (2009)
Blanchi, C., Gebre, B., Wittenburg, P.: Canonical workflow for machine learning tasks. Data Intell. 4(2), 173–185 (2022)
da Silva, R.F., Pottier, L., Coleman, T., Deelman, E., Casanova, H.: WorkflowHub: community framework for enabling scientific workflow research and development. In: 2020 IEEE/ACM Workflows in Support of Large-Scale Science (WORKS), pp. 49–56. IEEE (2020)
Starlinger, J.: Similarity measures for scientific workflows. Ph.D. thesis, Humboldt-Universität zu Berlin, Mathematisch-Naturwissenschaftliche Fakultät (2016). https://doi.org/10.18452/17406
Zhou, Z., Wen, J., Wang, Y., Xue, X., Hung, P.C., Nguyen, L.D.: Topic-based crossing-workflow fragment discovery. Futur. Gener. Comput. Syst. 112, 1141–1155 (2020)
Starlinger, J., Brancotte, B., Cohen-Boulakia, S., Leser, U.: Similarity search for scientific workflows. Proc. VLDB Endowment (PVLDB) 7(12), 1143–1154 (2014)
Yu, X., Wu, W., Liao, X.: Workflow recommendation based on graph embedding. In: 2020 IEEE World Congress on Services (SERVICES), pp. 89–94. IEEE (2020)
Gu, Y., Cao, J., Qian, S., Guan, W.: SWORTS: a scientific workflow retrieval approach by learning textual and structural semantics. IEEE Trans. Serv. Comput. (2023)
Wang, L., et al.: Plan-and-solve prompting: improving zero-shot chain-of-thought reasoning by large language models. arXiv preprint arXiv:2305.04091 (2023)
Jiang, X., Dong, Y., Wang, L., Shang, Q., Li, G.: Self-planning code generation with large language model. arXiv preprint arXiv:2303.06689 (2023)
Yao, Y., Li, Z., Zhao, H.: Beyond chain-of-thought, effective graph-of-thought reasoning in large language models. arXiv preprint arXiv:2305.16582 (2023)
Kojima, T., Gu, S.S., Reid, M., Matsuo, Y., Iwasawa, Y.: Large language models are zero-shot reasoners. arXiv preprint arXiv:2205.11916 (2022)
Wei, J., et al.: Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903 (2022)
Wang, Z., Cai, S., Liu, A., Ma, X., Liang, Y.: Describe, explain, plan and select: interactive planning with large language models enables open-world multi-task agents. arXiv preprint arXiv:2302.01560 (2023)
Pallagani, V., et al.: Understanding the capabilities of large language models for automated planning. arXiv preprint arXiv:2305.16151 (2023)
Wen, Y., Hou, J., Yuan, Z., Zhou, D.: Heterogeneous information network-based scientific workflow recommendation for complex applications. Complexity 2020 (2020)
Zhou, Z., Cheng, Z., Zhang, L.J., Gaaloul, W., Ning, K.: Scientific workflow clustering and recommendation leveraging layer hierarchical analysis. IEEE Trans. Serv. Comput. 11(1), 169–183 (2016)
Lu, P., et al.: Chameleon: plug-and-play compositional reasoning with large language models. arXiv preprint arXiv:2304.09842 (2023)
OpenAI: GPT-4 Technical report (2023)
Yasunaga, M., Leskovec, J., Liang, P.: LinkBERT: pretraining language models with document links. arXiv preprint arXiv:2203.15827 (2022)
Kuhn, H.W.: The Hungarian method for the assignment problem. Nav. Res. Logist. Q. 2(1–2), 83–97 (1955)
Li, H.: Research progress on evaluation methods and factors influencing shale brittleness: a review. Energy Rep. 8, 4344–4358 (2022)
Woolson, R.F.: Wilcoxon signed-rank test. Wiley Encycl. Clin. Trials, 1–3 (2007)
Acknowledgements
This work is supported by China National Science Foundation (No. 62072301) and the Program of Technology Innovation of the Science and Technology Commission of Shanghai Municipality (No. 21511104700).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Gu, Y., Cao, J., Guo, Y., Qian, S., Guan, W. (2023). Plan, Generate and Match: Scientific Workflow Recommendation with Large Language Models. In: Monti, F., Rinderle-Ma, S., Ruiz Cortés, A., Zheng, Z., Mecella, M. (eds) Service-Oriented Computing. ICSOC 2023. Lecture Notes in Computer Science, vol 14419. Springer, Cham. https://doi.org/10.1007/978-3-031-48421-6_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-48421-6_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-48420-9
Online ISBN: 978-3-031-48421-6
eBook Packages: Computer ScienceComputer Science (R0)