Improving Neural Network Learning Through Dual Variable Learning Rates

02/09/2020
by   Elizabeth Liner, et al.
0

This paper introduces and evaluates a novel training method for neural networks: Dual Variable Learning Rates (DVLR). Building on techniques and insights from behavioral psychology, the dual learning rates are used to emphasize correct and incorrect responses differently, thereby making the feedback to the network more specific. Further, the learning rates are varied as a function of the network's performance, thereby making it more efficient. DVLR was implemented on both a simple feedforward neural network and a convolutional neural network. Both networks are trained faster and achieve an increased accuracy on the MNIST and CIFAR-10 domains demonstrating that DVLR is a promising, psychologically motivated technique for training neural network models.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/20/2021

Multirate Training of Neural Networks

We propose multirate training of neural networks: partitioning neural ne...
research
08/15/2017

Learning with Rethinking: Recurrently Improving Convolutional Neural Networks through Feedback

Recent years have witnessed the great success of convolutional neural ne...
research
06/11/2018

Dual Pattern Learning Networks by Empirical Dual Prediction Risk Minimization

Motivated by the observation that humans can learn patterns from two giv...
research
09/16/2023

Learning a Stable Dynamic System with a Lyapunov Energy Function for Demonstratives Using Neural Networks

Autonomous Dynamic System (DS)-based algorithms hold a pivotal and found...
research
07/01/2021

Secure Quantized Training for Deep Learning

We have implemented training of neural networks in secure multi-party co...
research
03/25/2011

Using Variable Threshold to Increase Capacity in a Feedback Neural Network

The article presents new results on the use of variable thresholds to in...
research
02/01/2011

High-Performance Neural Networks for Visual Object Classification

We present a fast, fully parameterizable GPU implementation of Convoluti...

Please sign up or login with your details

Forgot password? Click here to reset