DeepAI AI Chat
Log In Sign Up

Interscript: A dataset for interactive learning of scripts through error feedback

by   Niket Tandon, et al.
Carnegie Mellon University
Allen Institute for Artificial Intelligence

How can an end-user provide feedback if a deployed structured prediction model generates inconsistent output, ignoring the structural complexity of human language? This is an emerging topic with recent progress in synthetic or constrained settings, and the next big leap would require testing and tuning models in real-world settings. We present a new dataset, Interscript, containing user feedback on a deployed model that generates complex everyday tasks. Interscript contains 8,466 data points – the input is a possibly erroneous script and a user feedback, and the output is a modified script. We posit two use-cases of that might significantly advance the state-of-the-art in interactive learning. The dataset is available at:


page 1

page 2

page 3

page 4


Improving scripts with a memory of natural feedback

How can an end-user provide feedback if a deployed structured prediction...

Reliability Check: An Analysis of GPT-3's Response to Sensitive Topics and Prompt Wording

Large language models (LLMs) have become mainstream technology with thei...

Break and Make: Interactive Structural Understanding Using LEGO Bricks

Visual understanding of geometric structures with complex spatial relati...

SURF: Improving classifiers in production by learning from busy and noisy end users

Supervised learning classifiers inevitably make mistakes in production, ...

Memory-assisted prompt editing to improve GPT-3 after deployment

Large LMs such as GPT-3, while powerful, are not immune to mistakes, but...

UIGR: Unified Interactive Garment Retrieval

Interactive garment retrieval (IGR) aims to retrieve a target garment im...