The Missing Link: Finding label relations across datasets

06/09/2022
by   Jasper Uijlings, et al.
0

Computer Vision is driven by the many datasets which can be used for training or evaluating novel methods. However, each dataset has different set of class labels, visual definition of classes, images following a specific distribution, annotation protocols, etc. In this paper we explore the automatic discovery of visual-semantic relations between labels across datasets. We want to understand how the instances of a certain class in a dataset relate to the instances of another class in another dataset. Are they in an identity, parent/child, overlap relation? Or is there no link between them at all? To find relations between labels across datasets, we propose methods based on language, on vision, and on a combination of both. Our methods can effectively discover label relations across datasets and the type of the relations. We use these results for a deeper inspection on why instances relate, find missing aspects of a class, and use our relations to create finer-grained annotations. We conclude that label relations cannot be established by looking at the names of classes alone, as they depend strongly on how each of the datasets was constructed.

READ FULL TEXT

page 2

page 4

page 10

page 11

page 12

research
07/18/2022

Automatic universal taxonomies for multi-domain semantic segmentation

Training semantic segmentation models on multiple datasets has sparked a...
research
12/09/2020

Label Confusion Learning to Enhance Text Classification Models

Representing a true label as a one-hot vector is a common practice in tr...
research
08/15/2023

Action Class Relation Detection and Classification Across Multiple Video Datasets

The Meta Video Dataset (MetaVD) provides annotated relations between act...
research
03/31/2018

Multi-label Learning with Missing Labels using Mixed Dependency Graphs

This work focuses on the problem of multi-label learning with missing la...
research
04/08/2019

Pushing the right boundaries matters! Wasserstein Adversarial Training for Label Noise

Noisy labels often occur in vision datasets, especially when they are is...
research
12/12/2016

COCO-Stuff: Thing and Stuff Classes in Context

Semantic classes can be either things (objects with a well-defined shape...
research
06/27/2017

A Pig, an Angel and a Cactus Walk Into a Blender: A Descriptive Approach to Visual Blending

A descriptive approach for automatic generation of visual blends is pres...

Please sign up or login with your details

Forgot password? Click here to reset