Prompting AI Art: An Investigation into the Creative Skill of Prompt Engineering

by   Jonas Oppenlaender, et al.

Humankind is entering a novel era of creativity - an era in which anybody can synthesize digital content. The paradigm under which this revolution takes place is prompt-based learning (or in-context learning). This paradigm has found fruitful application in text-to-image generation where it is being used to synthesize digital images from zero-shot text prompts in natural language for the purpose of creating AI art. This activity is referred to as prompt engineering - the practice of iteratively crafting prompts to generate and improve images. In this paper, we investigate prompt engineering as a novel creative skill for creating prompt-based art. In three studies with participants recruited from a crowdsourcing platform, we explore whether untrained participants could 1) recognize the quality of prompts, 2) write prompts, and 3) improve their prompts. Our results indicate that participants could assess the quality of prompts and respective images. This ability increased with the participants' experience and interest in art. Participants further were able to write prompts in rich descriptive language. However, even though participants were specifically instructed to generate artworks, participants' prompts were missing the specific vocabulary needed to apply a certain style to the generated images. Our results suggest that prompt engineering is a learned skill that requires expertise and practice. Based on our findings and experience with running our studies with participants recruited from a crowdsourcing platform, we provide ten recommendations for conducting experimental research on text-to-image generation and prompt engineering with a paid crowd. Our studies offer a deeper understanding of prompt engineering thereby opening up avenues for research on the future of prompt engineering. We conclude by speculating on four possible futures of prompt engineering.


A Taxonomy of Prompt Modifiers for Text-To-Image Generation

Text-to-image generation has seen an explosion of interest since 2021. T...

The Cultivated Practices of Text-to-Image Generation

Humankind is entering a novel creative era in which anybody can synthesi...

The Creativity of Text-to-Image Generation

Text-to-image synthesis has made a giant leap towards becoming a mainstr...

Grimm in Wonderland: Prompt Engineering with Midjourney to Illustrate Fairytales

The quality of text-to-image generation is continuously improving, yet t...

WorldSmith: Iterative and Expressive Prompting for World Building with a Generative AI

Crafting a rich and unique environment is crucial for fictional world-bu...

When happy accidents spark creativity: Bringing collaborative speculation to life with generative AI

Generative AI techniques like those that synthesize images from text (te...

AIwriting: Relations Between Image Generation and Digital Writing

During 2022, both transformer-based AI text generation sys-tems such as ...

Please sign up or login with your details

Forgot password? Click here to reset