Learning Discriminative Features via Label Consistent Neural Network

02/03/2016
by   Zhuolin Jiang, et al.
0

Deep Convolutional Neural Networks (CNN) enforces supervised information only at the output layer, and hidden layers are trained by back propagating the prediction error from the output layer without explicit supervision. We propose a supervised feature learning approach, Label Consistent Neural Network, which enforces direct supervision in late hidden layers. We associate each neuron in a hidden layer with a particular class label and encourage it to be activated for input signals from the same class. More specifically, we introduce a label consistency regularization called "discriminative representation error" loss for late hidden layers and combine it with classification error loss to build our overall objective function. This label consistency constraint alleviates the common problem of gradient vanishing and tends to faster convergence; it also makes the features derived from late hidden layers discriminative enough for classification even using a simple k-NN classifier, since input signals from the same class will have very similar representations. Experimental results demonstrate that our approach achieves state-of-the-art performances on several public benchmarks for action and object category recognition.

READ FULL TEXT

page 3

page 4

page 5

research
04/17/2020

Adaptive Neuron-wise Discriminant Criterion and Adaptive Center Loss at Hidden Layer for Deep Convolutional Neural Network

A deep convolutional neural network (CNN) has been widely used in image ...
research
09/18/2014

Deeply-Supervised Nets

Our proposed deeply-supervised nets (DSN) method simultaneously minimize...
research
01/14/2021

Label Contrastive Coding based Graph Neural Network for Graph Classification

Graph classification is a critical research problem in many applications...
research
08/05/2023

Neural Collapse in the Intermediate Hidden Layers of Classification Neural Networks

Neural Collapse (NC) gives a precise description of the representations ...
research
09/30/2019

Towards Robust Direct Perception Networks for Automated Driving

We consider the problem of engineering robust direct perception neural n...
research
02/08/2018

Learning Latent Representations in Neural Networks for Clustering through Pseudo Supervision and Graph-based Activity Regularization

In this paper, we propose a novel unsupervised clustering approach explo...
research
07/24/2018

A Hybrid of Deep Audio Feature and i-vector for Artist Recognition

Artist recognition is a task of modeling the artist's musical style. Thi...

Please sign up or login with your details

Forgot password? Click here to reset