SAILenv: Learning in Virtual Visual Environments Made Simple

by   Enrico Meloni, et al.

Recently, researchers in Machine Learning algorithms, Computer Vision scientists, engineers and others, showed a growing interest in 3D simulators as a mean to artificially create experimental settings that are very close to those in the real world. However, most of the existing platforms to interface algorithms with 3D environments are often designed to setup navigation-related experiments, to study physical interactions, or to handle ad-hoc cases that are not thought to be customized, sometimes lacking a strong photorealistic appearance and an easy-to-use software interface. In this paper, we present a novel platform, SAILenv, that is specifically designed to be simple and customizable, and that allows researchers to experiment visual recognition in virtual 3D scenes. A few lines of code are needed to interface every algorithm with the virtual world, and non-3D-graphics experts can easily customize the 3D environment itself, exploiting a collection of photorealistic objects. Our framework yields pixel-level semantic and instance labeling, depth, and, to the best of our knowledge, it is the only one that provides motion-related information directly inherited from the 3D engine. The client-server communication operates at a low level, avoiding the overhead of HTTP-based data exchanges. We perform experiments using a state-of-the-art object detector trained on real-world images, showing that it is able to recognize the photorealistic 3D objects of our environment. The computational burden of the optical flow compares favourably with the estimation performed using modern GPU-based convolutional networks or more classic implementations. We believe that the scientific community will benefit from the easiness and high-quality of our framework to evaluate newly proposed algorithms in their own customized realistic conditions.


page 1

page 3

page 4

page 6


iGibson, a Simulation Environment for Interactive Tasks in Large Realistic Scenes

We present iGibson, a novel simulation environment to develop robotic so...

Evaluating Continual Learning Algorithms by Generating 3D Virtual Environments

Continual learning refers to the ability of humans and animals to increm...

Virtual Worlds as Proxy for Multi-Object Tracking Analysis

Modern computer vision algorithms typically require expensive data acqui...

Messing Up 3D Virtual Environments: Transferable Adversarial 3D Objects

In the last few years, the scientific community showed a remarkable and ...

VRKitchen: an Interactive 3D Virtual Environment for Task-oriented Learning

One of the main challenges of advancing task-oriented learning such as v...

Optical flow-based branch segmentation for complex orchard environments

Machine vision is a critical subsystem for enabling robots to be able to...

VPS: Excavating High-Level C++ Constructs from Low-Level Binaries to Protect Dynamic Dispatching

Polymorphism and inheritance make C++ suitable for writing complex softw...

Please sign up or login with your details

Forgot password? Click here to reset