Generalisation in fully-connected neural networks for time series forecasting

02/14/2019
by   Anastasia Borovykh, et al.
0

In this paper we study the generalisation capabilities of fully-connected neural networks trained in the context of time series forecasting. Time series do not satisfy the typical assumption in statistical learning theory of the data being i.i.d. samples from some data-generating distribution. We use the input and weight Hessians, that is the smoothness of the learned function with respect to the input and the width of the minimum in weight space, to quantify a network's ability to generalise to unseen data. While such generalisation metrics have been studied extensively in the i.i.d. setting of for example image recognition, here we empirically validate their use in the task of time series forecasting. Furthermore we discuss how one can control the generalisation capability of the network by means of the training process using the learning rate, batch size and the number of training iterations as controls. Using these hyperparameters one can efficiently control the complexity of the output function without imposing explicit constraints.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/04/2021

Randomized Neural Networks for Forecasting Time Series with Multiple Seasonality

This work contributes to the development of neural forecasting models wi...
research
03/07/2022

Multivariate Time Series Forecasting with Latent Graph Inference

This paper introduces a new approach for Multivariate Time Series foreca...
research
02/02/2023

FV-MgNet: Fully Connected V-cycle MgNet for Interpretable Time Series Forecasting

By investigating iterative methods for a constrained linear model, we pr...
research
01/28/2021

Adjusting for Autocorrelated Errors in Neural Networks for Time Series Regression and Forecasting

In many cases, it is difficult to generate highly accurate models for ti...
research
10/13/2021

Bundle Networks: Fiber Bundles, Local Trivializations, and a Generative Approach to Exploring Many-to-one Maps

Many-to-one maps are ubiquitous in machine learning, from the image reco...
research
07/03/2021

Slope and generalization properties of neural networks

Neural networks are very successful tools in for example advanced classi...
research
09/19/2019

Shape and Time Distortion Loss for Training Deep Time Series Forecasting Models

This paper addresses the problem of time series forecasting for non-stat...

Please sign up or login with your details

Forgot password? Click here to reset