DeepAI AI Chat
Log In Sign Up

Distribution-Based Invariant Deep Networks for Learning Meta-Features

06/24/2020
by   Gwendoline de Bie, et al.
Cole Normale Suprieure
Laboratoire de Recherche en Informatique (LRI)
15

Recent advances in deep learning from probability distributions enable to achieve classification or regression from distribution samples, invariant under permutation of the samples. This paper extends the distribution-based deep neural architectures to achieve classification or regression from distribution samples, invariant under permutation of the descriptive features, too. The motivation for this extension is the Auto-ML problem, aimed to identify a priori the ML configuration best suited to a dataset. Formally, a distribution-based invariant deep learning architecture is presented, and leveraged to extract the meta-features characterizing a dataset. The contribution of the paper is twofold. On the theoretical side, the proposed architecture inherits the NN properties of universal approximation, and the robustness of the approach w.r.t. moderate perturbations is established. On the empirical side, a proof of concept of the approach is proposed, to identify the SVM hyper-parameters best suited to a large benchmark of diversified small size datasets.

READ FULL TEXT

page 1

page 2

page 3

page 4

03/05/2019

Universal approximations of permutation invariant/equivariant functions by deep neural networks

In this paper,we develop a theory of the relationship between permutatio...
10/15/2019

Improved Generalization Bound of Permutation Invariant Deep Neural Networks

We theoretically prove that a permutation invariant property of deep neu...
02/08/2021

Improved Brain Age Estimation with Slice-based Set Networks

Deep Learning for neuroimaging data is a promising but challenging direc...
11/29/2019

Deep Networks with Adaptive Nyström Approximation

Recent work has focused on combining kernel methods and deep learning to...
05/18/2022

A Classification of G-invariant Shallow Neural Networks

When trying to fit a deep neural network (DNN) to a G-invariant target f...
09/21/2019

Single Class Universum-SVM

This paper extends the idea of Universum learning [1, 2] to single-class...