DeepAI
Log In Sign Up

Dependency Structure Misspecification in Multi-Source Weak Supervision Models

06/18/2021
by   Salva Rühling Cachay, et al.
0

Data programming (DP) has proven to be an attractive alternative to costly hand-labeling of data. In DP, users encode domain knowledge into labeling functions (LF), heuristics that label a subset of the data noisily and may have complex dependencies. A label model is then fit to the LFs to produce an estimate of the unknown class label. The effects of label model misspecification on test set performance of a downstream classifier are understudied. This presents a serious awareness gap to practitioners, in particular since the dependency structure among LFs is frequently ignored in field applications of DP. We analyse modeling errors due to structure over-specification. We derive novel theoretical bounds on the modeling error and empirically show that this error can be substantial, even when modeling a seemingly sensible structure.

READ FULL TEXT

page 1

page 2

page 3

page 4

08/24/2021

The Word is Mightier than the Label: Learning without Pointillistic Labels using Data Programming

Most advanced supervised Machine Learning (ML) models rely on vast amoun...
04/13/2022

Label Augmentation with Reinforced Labeling for Weak Supervision

Weak supervision (WS) is an alternative to the traditional supervised le...
06/20/2019

Few-Shot Sequence Labeling with Label Dependency Transfer

Few-shot sequence labeling faces a unique challenge compared with the ot...
03/06/2020

Parsing Thai Social Data: A New Challenge for Thai NLP

Dependency parsing (DP) is a task that analyzes text for syntactic struc...
07/09/2016

Classifier Risk Estimation under Limited Labeling Resources

In this paper we propose strategies for estimating performance of a clas...
03/21/2022

Effective Token Graph Modeling using a Novel Labeling Strategy for Structured Sentiment Analysis

The state-of-the-art model for structured sentiment analysis casts the t...