IlluSonnet: Using Generative AI to Create Illustrations for Sonnets

Abstract

Poetry evokes imagery, and writers and readers alike desire to translate the artful wordplay to a beautiful image. To facilitate this process, we built IlluSonnet, a system that creates illustrations for poetry using text-to-image generative AI models. IlluSonnet works by labelling keywords, emotional qualities, and most related artistic style for the given sonnet before prompting DALL-E for an image. To evaluate IlluSonnet, we both ran a user study to assess the quality of the output images as well as the overall interface. Our study indicates that IlluSonnet helped users generate images that illustrated the sonnets well and that the process of creating and seeing imagery alongside the poem helped users understand the sonnets in a new light. We conclude by discussing how IlluSonnet can be used to further facilitate a deeper connection between both art and poetry.