Sparse Code Formation with Linear Inhibition

03/13/2015
by   Nam Do-Hoang Le, et al.
0

Sparse code formation in the primary visual cortex (V1) has been inspiration for many state-of-the-art visual recognition systems. To stimulate this behavior, networks are trained networks under mathematical constraint of sparsity or selectivity. In this paper, the authors exploit another approach which uses lateral interconnections in feature learning networks. However, instead of adding direct lateral interconnections among neurons, we introduce an inhibitory layer placed right after normal encoding layer. This idea overcomes the challenge of computational cost and complexity on lateral networks while preserving crucial objective of sparse code formation. To demonstrate this idea, we use sparse autoencoder as normal encoding layer and apply inhibitory layer. Early experiments in visual recognition show relative improvements over traditional approach on CIFAR-10 dataset. Moreover, simple installment and training process using Hebbian rule allow inhibitory layer to be integrated into existing networks, which enables further analysis in the future.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/22/2022

Layer Freezing Data Sieving: Missing Pieces of a Generic Framework for Sparse Training

Recently, sparse training has emerged as a promising paradigm for effici...
research
04/24/2019

Deep Sparse Representation-based Classification

We present a transductive deep learning-based formulation for the sparse...
research
08/18/2023

End-to-end topographic networks as models of cortical map formation and human visual behaviour: moving beyond convolutions

Computational models are an essential tool for understanding the origin ...
research
09/30/2021

Forming a sparse representation for visual place recognition using a neurorobotic approach

This paper introduces a novel unsupervised neural network model for visu...
research
12/04/2018

From biological vision to unsupervised hierarchical sparse coding

The formation of connections between neural cells is emerging essentiall...
research
02/19/2021

A theory of capacity and sparse neural encoding

Motivated by biological considerations, we study sparse neural maps from...
research
06/05/2016

What is the Best Feature Learning Procedure in Hierarchical Recognition Architectures?

(This paper was written in November 2011 and never published. It is post...

Please sign up or login with your details

Forgot password? Click here to reset