Self-Supervised Learning from Semantically Imprecise Data

04/22/2021
by   Clemens-Alexander Brust, et al.
0

Learning from imprecise labels such as "animal" or "bird", but making precise predictions like "snow bunting" at test time is an important capability when expertly labeled training data is scarce. Contributions by volunteers or results of web crawling lack precision in this manner, but are still valuable. And crucially, these weakly labeled examples are available in larger quantities for lower cost than high-quality bespoke training data. CHILLAX, a recently proposed method to tackle this task, leverages a hierarchical classifier to learn from imprecise labels. However, it has two major limitations. First, it is not capable of learning from effectively unlabeled examples at the root of the hierarchy, e.g. "object". Second, an extrapolation of annotations to precise labels is only performed at test time, where confident extrapolations could be already used as training data. In this work, we extend CHILLAX with a self-supervised scheme using constrained extrapolation to generate pseudo-labels. This addresses the second concern, which in turn solves the first problem, enabling an even weaker supervision requirement than CHILLAX. We evaluate our approach empirically and show that our method allows for a consistent accuracy improvement of 0.84 to 1.19 percent points over CHILLAX and is suitable as a drop-in replacement without any negative consequences such as longer training times.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/18/2021

Boosting Supervised Learning Performance with Co-training

Deep learning perception models require a massive amount of labeled trai...
research
01/23/2021

Online Adversarial Purification based on Self-Supervision

Deep neural networks are known to be vulnerable to adversarial examples,...
research
06/01/2023

Conformal Prediction with Partially Labeled Data

While the predictions produced by conformal prediction are set-valued, t...
research
03/17/2023

Data-Centric Learning from Unlabeled Graphs with Diffusion Model

Graph property prediction tasks are important and numerous. While each t...
research
02/18/2023

Data-Efficient Contrastive Self-supervised Learning: Easy Examples Contribute the Most

Self-supervised learning (SSL) learns high-quality representations from ...
research
03/01/2023

Self-Supervised Convolutional Visual Prompts

Machine learning models often fail on out-of-distribution (OOD) samples....
research
12/17/2021

A data-centric weak supervised learning for highway traffic incident detection

Using the data from loop detector sensors for near-real-time detection o...

Please sign up or login with your details

Forgot password? Click here to reset