NViSII: A Scriptable Tool for Photorealistic Image Generation

05/28/2021
by   Nathan Morrical, et al.
9

We present a Python-based renderer built on NVIDIA's OptiX ray tracing engine and the OptiX AI denoiser, designed to generate high-quality synthetic images for research in computer vision and deep learning. Our tool enables the description and manipulation of complex dynamic 3D scenes containing object meshes, materials, textures, lighting, volumetric data (e.g., smoke), and backgrounds. Metadata, such as 2D/3D bounding boxes, segmentation masks, depth maps, normal maps, material properties, and optical flow vectors, can also be generated. In this work, we discuss design goals, architecture, and performance. We demonstrate the use of data generated by path tracing for training an object detector and pose estimator, showing improved performance in sim-to-real transfer in situations that are difficult for traditional raster-based renderers. We offer this tool as an easy-to-use, performant, high-quality renderer for advancing research in synthetic data generation and deep learning.

READ FULL TEXT

page 1

page 3

page 4

page 7

page 9

research
05/14/2022

RTMV: A Ray-Traced Multi-View Synthetic Dataset for Novel View Synthesis

We present a large-scale synthetic dataset for novel view synthesis cons...
research
04/23/2021

UnrealROX+: An Improved Tool for Acquiring Synthetic Data from Virtual 3D Environments

Synthetic data generation has become essential in last years for feeding...
research
08/26/2019

A Flexible Neural Renderer for Material Visualization

Photo realism in computer generated imagery is crucially dependent on ho...
research
09/21/2023

NeuralLabeling: A versatile toolset for labeling vision datasets using Neural Radiance Fields

We present NeuralLabeling, a labeling approach and toolset for annotatin...
research
06/07/2023

Integrating Geometric Control into Text-to-Image Diffusion Models for High-Quality Detection Data Generation via Text Prompt

Diffusion models have attracted significant attention due to their remar...
research
08/19/2020

MineNav: An Expandable Synthetic Dataset Based on Minecraft for Aircraft Visual Navigation

We propose a simply method to generate high quality synthetic dataset ba...
research
10/03/2022

WorldGen: A Large Scale Generative Simulator

In the era of deep learning, data is the critical determining factor in ...

Please sign up or login with your details

Forgot password? Click here to reset