DeepAI AI Chat
Log In Sign Up

Label Encoding for Regression Networks

by   Deval Shah, et al.
The University of British Columbia

Deep neural networks are used for a wide range of regression problems. However, there exists a significant gap in accuracy between specialized approaches and generic direct regression in which a network is trained by minimizing the squared or absolute error of output labels. Prior work has shown that solving a regression problem with a set of binary classifiers can improve accuracy by utilizing well-studied binary classification algorithms. We introduce binary-encoded labels (BEL), which generalizes the application of binary classification to regression by providing a framework for considering arbitrary multi-bit values when encoding target values. We identify desirable properties of suitable encoding and decoding functions used for the conversion between real-valued and binary-encoded labels based on theoretical and empirical study. These properties highlight a tradeoff between classification error probability and error-correction capabilities of label encodings. BEL can be combined with off-the-shelf task-specific feature extractors and trained end-to-end. We propose a series of sample encoding, decoding, and training loss functions for BEL and demonstrate they result in lower error than direct regression and specialized approaches while being suitable for a diverse set of regression problems, network architectures, and evaluation metrics. BEL achieves state-of-the-art accuracies for several regression benchmarks. Code is available at


page 1

page 2

page 3

page 4


Learning Label Encodings for Deep Regression

Deep regression networks are widely used to tackle the problem of predic...

Training Binary Neural Networks with Real-to-Binary Convolutions

This paper shows how to train binary networks to within a few percent po...

Deep multi-class learning from label proportions

We propose a learning algorithm capable of learning from label proportio...

Regression Metric Loss: Learning a Semantic Representation Space for Medical Images

Regression plays an essential role in many medical imaging applications ...

Deep Ordinal Regression with Label Diversity

Regression via classification (RvC) is a common method used for regressi...

A Generalized Weighted Loss for SVC and MLP

Usually standard algorithms employ a loss where each error is the mere a...

Lifting Weak Supervision To Structured Prediction

Weak supervision (WS) is a rich set of techniques that produce pseudolab...