Modeling 3D Shapes by Reinforcement Learning

03/27/2020 ∙ by Cheng Lin, et al. ∙ 23

We explore how to enable machines to model 3D shapes like human modelers using reinforcement learning (RL). In 3D modeling software like Maya, a modeler usually creates a mesh model in two steps: (1) approximating the shape using a set of primitives; (2) editing the meshes of the primitives to create detailed geometry. Inspired by such artist-based modeling, we propose a two-step neural framework based on RL to learn 3D modeling policies. By taking actions and collecting rewards in an interactive environment, the agents first learn to parse a target shape into primitives and then to edit the geometry. To effectively train the modeling agents, we introduce a novel training algorithm that combines heuristic policy, imitation learning and reinforcement learning. Our experiments show that the agents can learn good policies to produce regular and structure-aware mesh models, which demonstrates the feasibility and effectiveness of the proposed RL framework.



There are no comments yet.


page 11

page 13

Code Repositories


Modeling 3D Shapes by Reinforcement Learning (ECCV2020)

view repo


Deform meshes by reinforcement learning

view repo


Generate a Rhino file that matches a target input mesh

view repo


A Deep Reinforcement Learning application for 3D object modeling policies.

view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.