RecipeQA: A Challenge Dataset for Multimodal Comprehension of Cooking Recipes

09/04/2018
by   Semih Yagcioglu, et al.
0

Understanding and reasoning about cooking recipes is a fruitful research direction towards enabling machines to interpret procedural text. In this work, we introduce RecipeQA, a dataset for multimodal comprehension of cooking recipes. It comprises of approximately 20K instructional recipes with multiple modalities such as titles, descriptions and aligned set of images. With over 36K automatically generated question-answer pairs, we design a set of comprehension and reasoning tasks that require joint understanding of images and text, capturing the temporal flow of events and making sense of procedural knowledge. Our preliminary results indicate that RecipeQA will serve as a challenging test bed and an ideal benchmark for evaluating machine comprehension systems. The data and leaderboard are available at http://hucvl.github.io/recipeqa.

READ FULL TEXT
research
09/30/2020

A Vietnamese Dataset for Evaluating Machine Reading Comprehension

Over 97 million inhabitants speak Vietnamese as the native language in t...
research
12/11/2020

Comprehension and Knowledge

The ability of an agent to comprehend a sentence is tightly connected to...
research
11/29/2016

NewsQA: A Machine Comprehension Dataset

We present NewsQA, a challenging machine comprehension dataset of over 1...
research
09/20/2022

FACT: Learning Governing Abstractions Behind Integer Sequences

Integer sequences are of central importance to the modeling of concepts ...
research
07/04/2021

Audio-Oriented Multimodal Machine Comprehension: Task, Dataset and Model

While Machine Comprehension (MC) has attracted extensive research intere...
research
05/12/2021

Designing Multimodal Datasets for NLP Challenges

In this paper, we argue that the design and development of multimodal da...
research
09/10/2019

Everything Happens for a Reason: Discovering the Purpose of Actions in Procedural Text

Our goal is to better comprehend procedural text, e.g., a paragraph abou...

Please sign up or login with your details

Forgot password? Click here to reset