How Will It Drape Like? Capturing Fabric Mechanics from Depth Images

We propose a method to estimate the mechanical parameters of fabrics using a casual capture setup with a depth camera. Our approach enables to create mechanically-correct digital representations of real-world textile materials, which is a fundamental step for many interactive design and engineering applications. As opposed to existing capture methods, which typically require expensive setups, video sequences, or manual intervention, our solution can capture at scale, is agnostic to the optical appearance of the textile, and facilitates fabric arrangement by non-expert operators. To this end, we propose a sim-to-real strategy to train a learning-based framework that can take as input one or multiple images and outputs a full set of mechanical parameters. Thanks to carefully designed data augmentation and transfer learning protocols, our solution generalizes to real images despite being trained only on synthetic data, hence successfully closing the sim-to-real loop.Key in our work is to demonstrate that evaluating the regression accuracy based on the similarity at parameter space leads to an inaccurate distances that do not match the human perception. To overcome this, we propose a novel metric for fabric drape similarity that operates on the image domain instead on the parameter space, allowing us to evaluate our estimation within the context of a similarity rank. We show that out metric correlates with human judgments about the perception of drape similarity, and that our model predictions produce perceptually accurate results compared to the ground truth parameters.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 6

page 8

page 9

research
12/06/2019

Generating Patient-like Phantoms Using Fully Unsupervised Deformable Image Registration with Convolutional Neural Networks

The use of Convolutional neural networks (ConvNets) in medical imaging r...
research
05/04/2019

A Similarity Measure for Material Appearance

We present a model to measure the similarity in appearance between diffe...
research
08/23/2022

PIFu for the Real World: A Self-supervised Framework to Reconstruct Dressed Human from Single-view Images

It is very challenging to accurately reconstruct sophisticated human geo...
research
06/19/2020

Deep Learning-based Single Image Face Depth Data Enhancement

Face recognition can benefit from the utilization of depth data captured...
research
06/06/2021

Occlusion-aware Unsupervised Learning of Depth from 4-D Light Fields

Depth estimation is a fundamental issue in 4-D light field processing an...
research
04/01/2020

Two-shot Spatially-varying BRDF and Shape Estimation

Capturing the shape and spatially-varying appearance (SVBRDF) of an obje...
research
10/18/2017

Photo-Guided Exploration of Volume Data Features

In this work, we pose the question of whether, by considering qualitativ...

Please sign up or login with your details

Forgot password? Click here to reset