DeepAI AI Chat
Log In Sign Up

VRKitchen: an Interactive 3D Virtual Environment for Task-oriented Learning

by   Xiaofeng Gao, et al.

One of the main challenges of advancing task-oriented learning such as visual task planning and reinforcement learning is the lack of realistic and standardized environments for training and testing AI agents. Previously, researchers often relied on ad-hoc lab environments. There have been recent advances in virtual systems built with 3D physics engines and photo-realistic rendering for indoor and outdoor environments, but the embodied agents in those systems can only conduct simple interactions with the world (e.g., walking around, moving objects, etc.). Most of the existing systems also do not allow human participation in their simulated environments. In this work, we design and implement a virtual reality (VR) system, VRKitchen, with integrated functions which i) enable embodied agents powered by modern AI methods (e.g., planning, reinforcement learning, etc.) to perform complex tasks involving a wide range of fine-grained object manipulations in a realistic environment, and ii) allow human teachers to perform demonstrations to train agents (i.e., learning from demonstration). We also provide standardized evaluation benchmarks and data collection tools to facilitate a broad use in research on task-oriented learning and beyond.


page 2

page 4

page 5

page 6

page 7

page 8


AI2-THOR: An Interactive 3D Environment for Visual AI

We introduce The House Of inteRactions (THOR), a framework for visual AI...

VECA : A Toolkit for Building Virtual Environments to Train and Test Human-like Agents

Building human-like agent, which aims to learn and think like human inte...

VRGym: A Virtual Testbed for Physical and Interactive AI

We propose VRGym, a virtual reality testbed for realistic human-robot in...

BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments

We introduce BEHAVIOR, a benchmark for embodied AI with 100 activities i...

Investigating the Sense of Presence Between Handcrafted and Panorama Based Virtual Environments

Virtual Reality applications are becoming increasingly mature. The requi...

Flightmare: A Flexible Quadrotor Simulator

Currently available quadrotor simulators have a rigid and highly-special...

SAILenv: Learning in Virtual Visual Environments Made Simple

Recently, researchers in Machine Learning algorithms, Computer Vision sc...