Tensor-variate Mixture of Experts

02/28/2019
by   Noémie Jaquier, et al.
0

When data are organized in matrices or arrays of higher dimensions (tensors), classical regression methods first transform these data into vectors, therefore ignoring the underlying structure of the data and increasing the dimensionality of the problem. This flattening operation typically leads to overfitting when only few training data is available. In this paper, we present a mixture of experts model that exploits tensorial representations for regression of tensor-valued data. The proposed formulation takes into account the underlying structure of the data and remains efficient when few training data are available. Evaluation on artificially generated data, as well as offline and real-time experiments recognizing hand movements from tactile myography prove the effectiveness of the proposed approach.

READ FULL TEXT

page 7

page 8

page 9

page 11

research
03/02/2022

Parameter-Efficient Mixture-of-Experts Architecture for Pre-trained Language Models

The state-of-the-art Mixture-of-Experts (short as MoE) architecture has ...
research
03/09/2022

Parsimonious Bayesian sparse tensor regression using the Tucker decomposition

Tensors, or multidimensional data arrays, require dimension reduction in...
research
02/12/2023

Bayesian Methods in Tensor Analysis

Tensors, also known as multidimensional arrays, are useful data structur...
research
01/31/2019

Multi-dimensional Tensor Sketch

Sketching refers to a class of randomized dimensionality reduction metho...
research
03/08/2021

Low-Rank Tensor Regression for X-Ray Tomography

Tomographic imaging is useful for revealing the internal structure of a ...
research
03/23/2020

Efficient Tensor Kernel methods for sparse regression

Recently, classical kernel methods have been extended by the introductio...

Please sign up or login with your details

Forgot password? Click here to reset