Embodied Question Answering

11/30/2017
by   Abhishek Das, et al.
0

We present a new AI task -- Embodied Question Answering (EmbodiedQA) -- where an agent is spawned at a random location in a 3D environment and asked a question ("What color is the car?"). In order to answer, the agent must first intelligently navigate to explore the environment, gather information through first-person (egocentric) vision, and then answer the question ("orange"). This challenging task requires a range of AI skills -- active perception, language understanding, goal-driven navigation, commonsense reasoning, and grounding of language into actions. In this work, we develop the environments, end-to-end-trained reinforcement learning agents, and evaluation protocols for EmbodiedQA.

READ FULL TEXT

page 6

page 13

page 14

page 15

page 16

page 17

page 18

page 19

research
05/22/2017

Ask the Right Questions: Active Question Reformulation with Reinforcement Learning

We frame Question Answering as a Reinforcement Learning task, an approac...
research
02/06/2016

End-to-End Goal-Driven Web Navigation

We propose a goal-driven web navigation as a benchmark task for evaluati...
research
12/19/2022

Don't Generate, Discriminate: A Proposal for Grounding Language Models to Real-World Environments

A key missing ability of current language models (LMs) is grounding to r...
research
08/14/2019

VideoNavQA: Bridging the Gap between Visual and Embodied Question Answering

Embodied Question Answering (EQA) is a recently proposed task, where an ...
research
04/06/2019

Embodied Question Answering in Photorealistic Environments with Point Cloud Perception

To help bridge the gap between internet vision-style problems and the go...
research
11/12/2018

Blindfold Baselines for Embodied QA

We explore blindfold (question-only) baselines for Embodied Question Ans...
research
04/09/2019

Multi-Target Embodied Question Answering

Embodied Question Answering (EQA) is a relatively new task where an agen...

Please sign up or login with your details

Forgot password? Click here to reset