DeepAI
Log In Sign Up

Preserving Fine-Grain Feature Information in Classification via Entropic Regularization

08/07/2022
by   Raphael Baena, et al.
8

Labeling a classification dataset implies to define classes and associated coarse labels, that may approximate a smoother and more complicated ground truth. For example, natural images may contain multiple objects, only one of which is labeled in many vision datasets, or classes may result from the discretization of a regression problem. Using cross-entropy to train classification models on such coarse labels is likely to roughly cut through the feature space, potentially disregarding the most meaningful such features, in particular losing information on the underlying fine-grain task. In this paper we are interested in the problem of solving fine-grain classification or regression, using a model trained on coarse-grain labels only. We show that standard cross-entropy can lead to overfitting to coarse-related features. We introduce an entropy-based regularization to promote more diversity in the feature space of trained models, and empirically demonstrate the efficacy of this methodology to reach better performance on the fine-grain problems. Our results are supported through theoretical developments and empirical validation.

READ FULL TEXT

page 14

page 15

08/16/2019

Symmetric Cross Entropy for Robust Learning with Noisy Labels

Training accurate deep neural networks (DNNs) in the presence of noisy l...
01/10/2022

Label Relation Graphs Enhanced Hierarchical Residual Network for Hierarchical Multi-Granularity Classification

Hierarchical multi-granularity classification (HMC) assigns hierarchical...
01/21/2019

Understanding the Impact of Label Granularity on CNN-based Image Classification

In recent years, supervised learning using Convolutional Neural Networks...
11/27/2022

Cross-domain Few-shot Segmentation with Transductive Fine-tuning

Few-shot segmentation (FSS) expects models trained on base classes to wo...
11/24/2021

Coarse-To-Fine Incremental Few-Shot Learning

Different from fine-tuning models pre-trained on a large-scale dataset o...
11/10/2022

Regression as Classification: Influence of Task Formulation on Neural Network Features

Neural networks can be trained to solve regression problems by using gra...
04/12/2019

Unifying Heterogeneous Classifiers with Distillation

In this paper, we study the problem of unifying knowledge from a set of ...