Text to Mesh Without 3D Supervision Using Limit Subdivision

03/24/2022
by   Nasir Khalid, et al.
0

We present a technique for zero-shot generation of a 3D model using only a target text prompt. Without a generative model or any 3D supervision our method deforms a control shape of a limit subdivided surface along with a texture map and normal map to obtain a 3D model asset that matches the input text prompt and can be deployed into games or modeling applications. We rely only on a pre-trained CLIP model that compares the input text prompt with differentiably rendered images of our 3D model. While previous works have focused on stylization or required training of generative models we perform optimization on mesh parameters directly to generate shape and texture. To improve the quality of results we also introduce a set of techniques such as render augmentations, primitive selection, prompt augmentation that guide the mesh towards a suitable result.

READ FULL TEXT

page 1

page 2

page 4

research
06/13/2020

Convolutional Generation of Textured 3D Meshes

Recent generative models for 2D images achieve impressive visual results...
research
06/23/2023

Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale

Large-scale generative models such as GPT and DALL-E have revolutionized...
research
12/28/2022

Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models

Recent CLIP-guided 3D optimization methods, e.g., DreamFields and PureCL...
research
06/14/2023

ZeroForge: Feedforward Text-to-Shape Without 3D Supervision

Current state-of-the-art methods for text-to-shape generation either req...
research
04/11/2022

No Token Left Behind: Explainability-Aided Image Classification and Generation

The application of zero-shot learning in computer vision has been revolu...
research
04/06/2019

Dense 3D Face Decoding over 2500FPS: Joint Texture & Shape Convolutional Mesh Decoders

3D Morphable Models (3DMMs) are statistical models that represent facial...
research
03/28/2023

X-Mesh: Towards Fast and Accurate Text-driven 3D Stylization via Dynamic Textual Guidance

Text-driven 3D stylization is a complex and crucial task in the fields o...

Please sign up or login with your details

Forgot password? Click here to reset