DeepAI AI Chat
Log In Sign Up

3DALL-E: Integrating Text-to-Image AI in 3D Design Workflows

by   Vivian Liu, et al.
Autodesk Inc.
Columbia University

Text-to-image AI systems are capable of generating novel images for inspiration, but their applications for 3D design workflows and how designers can build 3D models using AI-provided inspiration is less understood. To investigate this, we integrated DALL-E, GPT-3, and CLIP within a CAD software in 3DALL-E, a plugin that allows users to construct text and image prompts based on what they are modelling. In a study with 13 designers, we found that designers saw great potential to incorporate 3DALL-E into their workflows and to use text-to-image AI for reference images, renders, materials, and design considerations. Additionally, we elaborate on prompting patterns and provide measures of prompt complexity observed across participants. We conclude on a discussion of how 3DALL-E can merge with existing generative design workflows and propose prompt bibliographies as a form of human-AI design history.


page 1

page 4

page 6

page 8

page 9

page 12

page 15

page 16


The AI Ghostwriter Effect: Users Do Not Perceive Ownership of AI-Generated Text But Self-Declare as Authors

Human-AI interaction in text production increases complexity in authorsh...

A Word is Worth a Thousand Pictures: Prompts as AI Design Material

Recent advances in Machine-Learning have led to the development of model...

AI Art in Architecture

Recent diffusion-based AI art platforms are able to create impressive im...

When happy accidents spark creativity: Bringing collaborative speculation to life with generative AI

Generative AI techniques like those that synthesize images from text (te...

Trash to Treasure: Using text-to-image models to inform the design of physical artefacts

Text-to-image generative models have recently exploded in popularity and...

Opal: Multimodal Image Generation for News Illustration

Multimodal AI advancements have presented people with powerful ways to c...

Augmenting Pathologists with NaviPath: Design and Evaluation of a Human-AI Collaborative Navigation System

Artificial Intelligence (AI) brings advancements to support pathologists...