An Adaptive Locally Connected Neuron Model: Focusing Neuron

08/31/2018
by   F. Boray Tek, et al.
0

We present a new artificial neuron model capable of learning its receptive field in the spatial domain of inputs. The name for the new model is focusing neuron because it can adapt both its receptive field location and size (aperture) during training. A network or a layer formed of such neurons can learn and generate unique connection structures for particular inputs/problems. The new model requires neither heuristics nor additional optimizations. Hence, all parameters, including those controlling the focus could be trained using the stochastic gradient descent optimization. We have empirically shown the capacity and viability of the new model with tests on synthetic and real datasets. We have constructed simple networks with one or two hidden layers; also employed fully connected networks with the same configurations as controls. In noise-added synthetic Gaussian blob datasets, we observed that focusing neurons can steer their receptive fields away from the redundant inputs and focused into more informative ones. Tests on common datasets such as MNIST have shown that a network of two hidden focusing layers can perform better (99.21 same configuration.

READ FULL TEXT
research
08/05/2017

Depth Adaptive Deep Neural Network for Semantic Segmentation

In this work, we present the depth-adaptive deep neural network using a ...
research
02/10/2022

Deep Learning in Random Neural Fields: Numerical Experiments via Neural Tangent Kernel

A biological neural network in the cortex forms a neural field. Neurons ...
research
06/23/2020

Density-embedding layers: a general framework for adaptive receptive fields

The effectiveness and performance of artificial neural networks, particu...
research
09/23/2020

Schizophrenia-mimicking layers outperform conventional neural network layers

We have reported nanometer-scale three-dimensional studies of brain netw...
research
11/29/2021

Image denoising by Super Neurons: Why go deep?

Classical image denoising methods utilize the non-local self-similarity ...
research
11/22/2017

Locally Smoothed Neural Networks

Convolutional Neural Networks (CNN) and the locally connected layer are ...
research
01/15/2013

Pushing Stochastic Gradient towards Second-Order Methods -- Backpropagation Learning with Transformations in Nonlinearities

Recently, we proposed to transform the outputs of each hidden neuron in ...

Please sign up or login with your details

Forgot password? Click here to reset