Abstract
Generative AI experienced a boom in 2022 with highlights such as the releases of Stable Diffusion for image generation and ChatGPT for conversational text generation. Although millions of images have been generated using products such as DALL-E 2, DreamStudio, and Midjourney, the learning curve for developing good text prompts that can lead to high-quality images remains steep, especially for inexperienced and less-technical users. Although various prompt engineering guides and tutorials have been developed to provide tips and guidance on prompt writing, there has been scant research on automatic prompt improvement algorithms and methods. In this paper, we present an automatic prompt optimization framework based on NLP analysis of a large prompt database and various machine learning models. A product based on our framework was developed and deployed for two months and real data were collected to evaluate our framework.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Stable Diffusion prompt Generator-promptoMANIA. https://promptomania.com/stable-diffusion-prompt-builder/. Accessed 03 Mar 2023
Lexica. https://lexica.art/. Accessed 03 Mar 2023
MagicPrompt-Stable-Diffusion. https://huggingface.co/Gustavosta/MagicPrompt-Stable-Diffusion. Accessed 03 Mar 2023
Pushp, P.K., Srivastava, M.M.: Train Once, Test Anywhere: Zero-Shot Learning for Text Classification (2017). http://arxiv.org/abs/1712.05972
Priyam, A., Gupta, R., Rathee, A., Srivastava, S.: Comparative analysis of decision tree classification algorithms. Int. J. Curr. Eng. Technol. 2, 334–337 (2013)
spaCy. https://spacy.io/. Accessed 03 Mar 2023
Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21(1), 5485–5551 (2020)
Wang, Z.J., Montoya, E., Munechika, D., Yang, H., Hoover, B., Chau, D.H.: DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Models (2022). http://arxiv.org/abs/2210.14896
Liu, X.Y., Wu, J., Zhou, Z.H.: Exploratory undersampling for class-imbalance learning. IEEE Trans. Syst. Man Cybern. B. 39, 539–550 (2009)
t5-base. https://huggingface.co/t5-base. Accessed 03 Mar 2023
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Fan, L., Wang, H.J., Zhang, K., Pei, Z., Li, A. (2023). Towards an Automatic Prompt Optimization Framework for AI Image Generation. In: Stephanidis, C., Antona, M., Ntoa, S., Salvendy, G. (eds) HCI International 2023 Posters. HCII 2023. Communications in Computer and Information Science, vol 1836. Springer, Cham. https://doi.org/10.1007/978-3-031-36004-6_55
Download citation
DOI: https://doi.org/10.1007/978-3-031-36004-6_55
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-36003-9
Online ISBN: 978-3-031-36004-6
eBook Packages: Computer ScienceComputer Science (R0)