CRAFT: A Benchmark for Causal Reasoning About Forces and inTeractions

12/08/2020
by   Tayfun Ates, et al.
5

Recent advances in Artificial Intelligence and deep learning have revived the interest in studying the gap between the reasoning capabilities of humans and machines. In this ongoing work, we introduce CRAFT, a new visual question answering dataset that requires causal reasoning about physical forces and object interactions. It contains 38K video and question pairs that are generated from 3K videos from 10 different virtual environments, containing different number of objects in motion that interact with each other. Two question categories from CRAFT include previously studied descriptive and counterfactual questions. Besides, inspired by the theory of force dynamics from the field of human cognitive psychology, we introduce new question categories that involve understanding the intentions of objects through the notions of cause, enable, and prevent. Our preliminary results demonstrate that even though these tasks are very intuitive for humans, the implemented baselines could not cope with the underlying challenges.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/07/2022

CRIPP-VQA: Counterfactual Reasoning about Implicit Physical Properties via Video Question Answering

Videos often capture objects, their visible properties, their motion, an...
research
01/17/2021

HySTER: A Hybrid Spatio-Temporal Event Reasoner

The task of Video Question Answering (VideoQA) consists in answering nat...
research
10/03/2019

CLEVRER: CoLlision Events for Video REpresentation and Reasoning

The ability to reason about temporal and causal events from videos lies ...
research
10/25/2021

IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning

Current visual question answering (VQA) tasks mainly consider answering ...
research
03/16/2018

A dataset and architecture for visual reasoning with a working memory

A vexing problem in artificial intelligence is reasoning about events th...
research
10/08/2022

EgoTaskQA: Understanding Human Tasks in Egocentric Videos

Understanding human tasks through video observations is an essential cap...
research
03/20/2018

IntPhys: A Framework and Benchmark for Visual Intuitive Physics Reasoning

In order to reach human performance on complex visual tasks, artificial ...

Please sign up or login with your details

Forgot password? Click here to reset