Large-Scale Learning with Fourier Features and Tensor Decompositions

09/03/2021
by   Frederiek Wesel, et al.
0

Random Fourier features provide a way to tackle large-scale machine learning problems with kernel methods. Their slow Monte Carlo convergence rate has motivated the research of deterministic Fourier features whose approximation error decreases exponentially with the number of frequencies. However, due to their tensor product structure these methods suffer heavily from the curse of dimensionality, limiting their applicability to two or three-dimensional scenarios. In our approach we overcome said curse of dimensionality by exploiting the tensor product structure of deterministic Fourier features, which enables us to represent the model parameters as a low-rank tensor decomposition. We derive a monotonically converging block coordinate descent algorithm with linear complexity in both the sample size and the dimensionality of the inputs for a regularized squared loss function, allowing to learn a parsimonious model in decomposed form using deterministic Fourier features. We demonstrate by means of numerical experiments how our low-rank tensor approach obtains the same performance of the corresponding nonparametric model, consistently outperforming random Fourier features.

READ FULL TEXT
research
09/15/2023

Low-rank Tensor Train Decomposition Using TensorSketch

Tensor train decomposition is one of the most powerful approaches for pr...
research
09/11/2023

Quantized Fourier and Polynomial Features for more Expressive Tensor Network Models

In the context of kernel machines, polynomial and Fourier features are c...
research
08/27/2020

Nonparametric Multivariate Density Estimation: A Low-Rank Characteristic Function Approach

Effective non-parametric density estimation is a key challenge in high-d...
research
03/07/2015

Higher order Matching Pursuit for Low Rank Tensor Learning

Low rank tensor learning, such as tensor completion and multilinear mult...
research
03/17/2020

Nonlinear system identification with regularized Tensor Network B-splines

This article introduces the Tensor Network B-spline model for the regula...
research
02/04/2021

Wind Field Reconstruction with Adaptive Random Fourier Features

We investigate the use of spatial interpolation methods for reconstructi...
research
06/24/2018

A Unified Analysis of Random Fourier Features

We provide the first unified theoretical analysis of supervised learning...

Please sign up or login with your details

Forgot password? Click here to reset