  In an online experiment with N = 1891 participants, we collected and analyzed
over 18,000 prompts to explore how the importance of prompting will change as
the capabilities of generative AI models continue to improve. Each participant
in our experiment was randomly and blindly assigned to use one of three
text-to-image diffusion models: DALL-E 2, its more advanced successor DALL-E 3,
or a version of DALL-E 3 with automatic prompt revision. Participants were then
asked to write prompts to reproduce a target image as closely as possible in 10
consecutive tries. We find that task performance was higher for participants
using DALL-E 3 than for those using DALL-E 2. This performance gap corresponds
to a noticeable difference in the similarity of participants' images to their
target images, and was caused in equal measure by: (1) the increased technical
capabilities of DALL-E 3, and (2) endogenous changes in participants' prompting
in response to these increased capabilities. More specifically, despite being
blind to the model they were assigned, participants assigned to DALL-E 3 wrote
longer prompts that were more semantically similar to each other and contained
a greater number of descriptive words. Furthermore, while participants assigned
to DALL-E 3 with prompt revision still outperformed those assigned to DALL-E 2,
automatic prompt revision reduced the benefits of using DALL-E 3 by 58%. Taken
together, our results suggest that as models continue to progress, people will
continue to adapt their prompts to take advantage of new models' capabilities.
