UnrealCV: Connecting Computer Vision to Unreal Engine

09/05/2016
by   Weichao Qiu, et al.
0

Computer graphics can not only generate synthetic images and ground truth but it also offers the possibility of constructing virtual worlds in which: (i) an agent can perceive, navigate, and take actions guided by AI algorithms, (ii) properties of the worlds can be modified (e.g., material and reflectance), (iii) physical simulations can be performed, and (iv) algorithms can be learnt and evaluated. But creating realistic virtual worlds is not easy. The game industry, however, has spent a lot of effort creating 3D worlds, which a player can interact with. So researchers can build on these resources to create virtual worlds, provided we can access and modify the internal data structures of the games. To enable this we created an open-source plugin UnrealCV (http://unrealcv.github.io) for a popular game engine Unreal Engine 4 (UE4). We show two applications: (i) a proof of concept image dataset, and (ii) linking Caffe with the virtual world to test deep network algorithms.

READ FULL TEXT

page 2

page 3

page 5

research
01/05/2023

Character Simulation Using Imitation Learning With Game Engine Physics

Creating visual 3D sensing characters that interact with AI peers and vi...
research
12/14/2016

UnrealStereo: A Synthetic Dataset for Analyzing Stereo Vision

Stereo algorithm is important for robotics applications, such as quadcop...
research
05/20/2016

Virtual Worlds as Proxy for Multi-Object Tracking Analysis

Modern computer vision algorithms typically require expensive data acqui...
research
10/16/2018

UnrealROX: An eXtremely Photorealistic Virtual Reality Environment for Robotics Simulations and Synthetic Data Generation

Data-driven algorithms have surpassed traditional techniques in almost e...
research
08/07/2016

Playing for Data: Ground Truth from Computer Games

Recent progress in computer vision has been driven by high-capacity mode...
research
08/08/2023

PUG: Photorealistic and Semantically Controllable Synthetic Data for Representation Learning

Synthetic image datasets offer unmatched advantages for designing and ev...
research
06/16/2018

G2D: from GTA to Data

This document describes G2D, a software that enables capturing videos fr...

Please sign up or login with your details

Forgot password? Click here to reset