Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scans

10/11/2021
by   Ainaz Eftekhar, et al.
21

This paper introduces a pipeline to parametrically sample and render multi-task vision datasets from comprehensive 3D scans from the real world. Changing the sampling parameters allows one to "steer" the generated datasets to emphasize specific information. In addition to enabling interesting lines of research, we show the tooling and generated data suffice to train robust vision models. Common architectures trained on a generated starter dataset reached state-of-the-art performance on multiple common vision tasks and benchmarks, despite having seen no benchmark or non-pipeline data. The depth estimation network outperforms MiDaS and the surface normal estimation network is the first to achieve human-level performance for in-the-wild surface normal estimation – at least according to one metric on the OASIS benchmark. The Dockerized pipeline with CLI, the (mostly python) code, PyTorch dataloaders for the generated data, the generated starter dataset, download scripts and other utilities are available through our project website, https://omnidata.vision.

READ FULL TEXT

page 1

page 3

page 4

page 5

page 6

page 7

page 8

research
04/10/2017

Surface Normals in the Wild

We study the problem of single-image depth estimation for images in the ...
research
07/16/2018

Zap: Making Predictions Based on Online User Behavior

This paper introduces Zap, a generic machine learning pipeline for makin...
research
08/07/2020

HoliCity: A City-Scale Data Platform for Learning Holistic 3D Structures

We present HoliCity, a city-scale 3D dataset with rich structural inform...
research
08/25/2021

Multi-Task Self-Training for Learning General Representations

Despite the fast progress in training specialized models for various tas...
research
11/18/2014

Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture

In this paper we address three different computer vision tasks using a s...
research
08/23/2021

Vogtareuth Rehab Depth Datasets: Benchmark for Marker-less Posture Estimation in Rehabilitation

Posture estimation using a single depth camera has become a useful tool ...
research
07/30/2020

SimPose: Effectively Learning DensePose and Surface Normals of People from Simulated Data

With a proliferation of generic domain-adaptation approaches, we report ...

Please sign up or login with your details

Forgot password? Click here to reset