Generalization in quasi-periodic environments

07/14/2018
by   Giovanni Bellettini, et al.
0

By and large the behavior of stochastic gradient is regarded as a challenging problem, and it is often presented in the framework of statistical machine learning. This paper offers a novel view on the analysis of on-line models of learning that arises when dealing with a generalized version of stochastic gradient that is based on dissipative dynamics. In order to face the complex evolution of these models, a systematic treatment is proposed which is based on energy balance equations that are derived by means of the Caldirola-Kanai (CK) Hamiltonian. According to these equations, learning can be regarded as an ordering process which corresponds with the decrement of the loss function. Finally, the main results established in this paper is that in the case of quasi-periodic environments, where the pattern novelty is progressively limited as time goes by, the system dynamics yields an asymptotically consistent solution in the weight space, that is the solution maps similar patterns to the same decision.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/19/2015

Stochastic gradient method with accelerated stochastic dynamics

In this paper, we propose a novel technique to implement stochastic grad...
research
11/28/2021

The Galerkin analysis for the random periodic solution of semilinear stochastic evolution equations

In this paper, we study the numerical method for approximating the rando...
research
11/05/2018

Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundations

We develop the mathematical foundations of the stochastic modified equat...
research
01/12/2018

Generalization Error Bounds for Noisy, Iterative Algorithms

In statistical learning theory, generalization error is used to quantify...
research
11/19/2015

Stochastic modified equations and adaptive stochastic gradient algorithms

We develop the method of stochastic modified equations (SME), in which s...
research
04/26/2023

An efficient multiple harmonic balance method for computing quasi-periodic responses of nonlinear systems

Quasi-periodic responses composed of multiple base frequencies widely ex...
research
12/21/2021

Learning in Random Utility Models Via Online Decision Problems

This paper studies the Random Utility Model (RUM) in environments where ...

Please sign up or login with your details

Forgot password? Click here to reset