Nemo: Guiding and Contextualizing Weak Supervision for Interactive Data Programming

03/02/2022
by   Cheng-Yu Hsieh, et al.
0

Weak Supervision (WS) techniques allow users to efficiently create large training datasets by programmatically labeling data with heuristic sources of supervision. While the success of WS relies heavily on the provided labeling heuristics, the process of how these heuristics are created in practice has remained under-explored. In this work, we formalize the development process of labeling heuristics as an interactive procedure, built around the existing workflow where users draw ideas from a selected set of development data for designing the heuristic sources. With the formalism, we study two core problems of how to strategically select the development data to guide users in efficiently creating informative heuristics, and how to exploit the information within the development process to contextualize and better learn from the resultant heuristics. Building upon two novel methodologies that effectively tackle the respective problems considered, we present Nemo, an end-to-end interactive system that improves the overall productivity of WS learning pipeline by an average 20 prevailing WS approach.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/13/2022

Losses over Labels: Weakly Supervised Learning via Direct Loss Construction

Owing to the prohibitive costs of generating large amounts of labeled da...
research
09/03/2020

Data Programming by Demonstration: A Framework for Interactively Learning Labeling Functions

Data programming is a programmatic weak supervision approach to efficien...
research
11/28/2017

Snorkel: Rapid Training Data Creation with Weak Supervision

Labeling training data is increasingly the largest bottleneck in deployi...
research
05/01/2022

StreamingHub: Interactive Stream Analysis Workflows

Reusable data/code and reproducible analyses are foundational to quality...
research
11/03/2015

Visualising interactive inferences with IDPD3

A large part of the use of knowledge base systems is the interpretation ...
research
12/14/2018

Bootstrapping Conversational Agents With Weak Supervision

Many conversational agents in the market today follow a standard bot dev...
research
10/17/2019

Exploring Semi-Automatic Map Labeling

Label placement in maps is a very challenging task that is critical for ...

Please sign up or login with your details

Forgot password? Click here to reset