Training Recurrent Neural Networks via Dynamical Trajectory-Based Optimization

05/10/2018
by   Hamid Khodabandehlou, et al.
0

This paper introduces a new method to train recurrent neural networks using dynamical trajectory-based optimization. The optimization method utilizes a projected gradient system (PGS) and a quotient gradient system (QGS) to determine the feasible regions of an optimization problem and search the feasible regions for local minima. By exploring the feasible regions, local minima are identified and the local minimum with the lowest cost is chosen as the global minimum of the optimization problem. Lyapunov theory is used to prove the stability of the local minima and their stability in the presence of measurement errors. Numerical examples show that the new approach provides better results than genetic algorithm and error backpropagation (EBP) trained networks.

READ FULL TEXT

Authors

page 1

page 2

page 3

page 4

03/20/2018

Training Recurrent Neural Networks as a Constraint Satisfaction Problem

This paper presents a new approach for training artificial neural networ...
11/13/2019

Existence of local minima of a minimal 2D pose-graph SLAM problem

In this paper, we show that for a minimal pose-graph problem, even in th...
11/13/2019

Analysis of minima for geodesic and chordal cost for a minimal 2D pose-graph SLAM problem

In this paper, we show that for a minimal pose-graph problem, even in th...
07/27/2018

On the overfly algorithm in deep learning of neural networks

In this paper we investigate the supervised backpropagation training of ...
12/29/2019

Robust Real-time UAV Replanning Using Guided Gradient-based Optimization and Topological Paths

Gradient-based trajectory optimization (GTO) has gained wide popularity ...
04/24/2020

Computing multiple solutions of topology optimization problems

Topology optimization problems often support multiple local minima due t...
08/21/2018

Search for Common Minima in Joint Optimization of Multiple Cost Functions

We present a novel optimization method, named the Combined Optimization ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.