Randall Balestriero

research

∙ 05/25/2023

Martian time-series unraveled: A multi-scale nested approach with factorial variational autoencoders

Unsupervised source separation involves unraveling an unknown set of sou...

0 Ali Siahkoohi, et al. ∙

research

∙ 04/24/2023

A Cookbook of Self-Supervised Learning

Self-supervised learning, dubbed the dark matter of intelligence, is a p...

0 Randall Balestriero, et al. ∙

research

∙ 04/11/2023

A surprisingly simple technique to control the pretraining bias for better transfer: Expand or Narrow your representation

Self-Supervised Learning (SSL) models rely on a pretext task to learn re...

0 Florian Bordes, et al. ∙

research

∙ 03/27/2023

Active Self-Supervised Learning: A Few Low-Cost Relationships Are All You Need

Self-Supervised Learning (SSL) has emerged as the solution of choice to ...

0 Vivien Cabannes, et al. ∙

research

∙ 03/03/2023

Towards Democratizing Joint-Embedding Self-Supervised Learning

Joint Embedding Self-Supervised Learning (JE-SSL) has seen rapid develop...

0 Florian Bordes, et al. ∙

research

∙ 03/01/2023

An Information-Theoretic Perspective on Variance-Invariance-Covariance Regularization

In this paper, we provide an information-theoretic perspective on Varian...

0 Ravid Shwartz-Ziv, et al. ∙

research

∙ 03/01/2023

FAIR-Ensemble: When Fairness Naturally Emerges From Deep Ensembling

Ensembling independent deep neural networks (DNNs) is a simple and effec...

4 Wei-Yin Ko, et al. ∙

research

∙ 02/20/2023

Unsupervised Learning on a DIET: Datum IndEx as Target Free of Self-Supervision, Reconstruction, Projector Head

Costly, noisy, and over-specialized, labels are to be set aside in favor...

0 Randall Balestriero, et al. ∙

research

∙ 02/06/2023

The SSL Interplay: Augmentations, Inductive Bias, and Generalization

Self-supervised learning (SSL) has emerged as a powerful framework to le...

0 Vivien Cabannes, et al. ∙

research

∙ 11/07/2022

On minimal variations for unsupervised representation learning

Unsupervised representation learning aims at describing raw data efficie...

0 Vivien Cabannes, et al. ∙

research

∙ 11/03/2022

ImageNet-X: Understanding Model Mistakes with Factor of Variation Annotations

Deep learning vision systems are widely deployed across applications whe...

2 Badr Youbi Idrissi, et al. ∙

research

∙ 11/02/2022

POLICE: Provably Optimal Linear Constraint Enforcement for Deep Neural Networks

Deep Neural Networks (DNNs) outshine alternative function approximators ...

0 Randall Balestriero, et al. ∙

research

∙ 10/13/2022

The Hidden Uniform Cluster Prior in Self-Supervised Learning

A successful paradigm in representation learning is to perform self-supe...

0 Mahmoud Assran, et al. ∙

research

∙ 10/05/2022

RankMe: Assessing the downstream performance of pretrained self-supervised representations by their rank

Joint-Embedding Self Supervised Learning (JE-SSL) has seen a rapid devel...

0 Quentin Garrido, et al. ∙

research

∙ 09/29/2022

Variance Covariance Regularization Enforces Pairwise Independence in Self-Supervised Representations

Self-Supervised Learning (SSL) methods such as VICReg, Barlow Twins or W...

13 Grégoire Mialon, et al. ∙

research

∙ 09/29/2022

Joint Embedding Self-Supervised Learning in the Kernel Regime

The fundamental goal of self-supervised learning (SSL) is to produce use...

3 Bobak T. Kiani, et al. ∙

research

∙ 09/29/2022

Batch Normalization Explained

A critically important, ubiquitous, and yet poorly understood ingredient...

7 Randall Balestriero, et al. ∙

research

∙ 07/20/2022

What Do We Maximize in Self-Supervised Learning?

In this paper, we examine self-supervised learning methods, particularly...

175 Ravid Shwartz-Ziv, et al. ∙

research

∙ 06/27/2022

Guillotine Regularization: Improving Deep Networks Generalization by Removing their Head

One unexpected technique that emerged in recent years consists in traini...

33 Florian Bordes, et al. ∙

research

∙ 05/23/2022

Contrastive and Non-Contrastive Self-Supervised Learning Recover Global and Local Spectral Embedding Methods

Self-Supervised Learning (SSL) surmises that inputs and pairwise positiv...

155 Randall Balestriero, et al. ∙

research

∙ 04/07/2022

The Effects of Regularization and Data Augmentation are Class Dependent

Regularization is a fundamental technique to prevent over-fitting and to...

8 Randall Balestriero, et al. ∙

research

∙ 04/07/2022

DeepTensor: Low-Rank Tensor Decomposition with Deep Network Priors

DeepTensor is a computationally efficient framework for low-rank decompo...

16 Vishwanath Saragadam, et al. ∙

research

∙ 03/10/2022

projUNN: efficient method for training deep networks with unitary matrices

In learning with recurrent or very deep feed-forward networks, employing...

6 Bobak Kiani, et al. ∙

research

∙ 03/07/2022

Singular Value Perturbation and Deep Network Optimization

We develop new theoretical results on matrix perturbation to shed light ...

3 Rudolf H. Riedi, et al. ∙

research

∙ 02/23/2022

NeuroView-RNN: It's About Time

Recurrent Neural Networks (RNNs) are important tools for processing sequ...

12 CJ Barberan, et al. ∙

research

∙ 02/16/2022

A Data-Augmentation Is Worth A Thousand Samples: Exact Quantification From Analytical Augmented Sample Moments

Data-Augmentation (DA) is known to improve performance across tasks and ...

8 Randall Balestriero, et al. ∙

research

∙ 02/16/2022

Spatial Transformer K-Means

K-means defines one of the most employed centroid-based clustering algor...

27 Romain Cosentino, et al. ∙

research

∙ 12/16/2021

High Fidelity Visualization of What Your Self-Supervised Representation Knows About

Discovering what is learned by neural networks remains a challenge. In s...

7 Florian Bordes, et al. ∙

research

∙ 10/18/2021

Learning in High Dimension Always Amounts to Extrapolation

The notion of interpolation and extrapolation is fundamental in various ...

26 Randall Balestriero, et al. ∙

research

∙ 10/15/2021

MaGNET: Uniform Sampling from Deep Generative Network Manifolds Without Retraining

Deep Generative Networks (DGNs) are extensively employed in Generative A...

9 Ahmed Imtiaz Humayun, et al. ∙

research

∙ 10/15/2021

NeuroView: Explainable Deep Network Decision Making

Deep neural networks (DNs) provide superhuman performance in numerous co...

52 CJ Barberan, et al. ∙

research

∙ 04/01/2021

Fast Jacobian-Vector Product for Deep Networks

Jacobian-vector products (JVPs) form the backbone of many recent develop...

0 Randall Balestriero, et al. ∙

research

∙ 01/07/2021

Max-Affine Spline Insights Into Deep Network Pruning

In this paper, we study the importance of pruning in Deep Networks (DNs)...

0 Randall Balestriero, et al. ∙

research

∙ 12/16/2020

Interpretable Image Clustering via Diffeomorphism-Aware K-Means

We design an interpretable clustering algorithm aware of the nonlinear s...

0 Romain Cosentino, et al. ∙

research

∙ 12/14/2020

Sparse Multi-Family Deep Scattering Network

In this work, we propose the Sparse Multi-Family Deep Scattering Network...

0 Romain Cosentino, et al. ∙

research

∙ 12/09/2020

Scalable Neural Tangent Kernel of Recurrent Architectures

Kernels derived from deep neural networks (DNNs) in the infinite-width p...

0 Sina Alemohammad, et al. ∙

research

∙ 10/27/2020

Wearing a MASK: Compressed Representations of Variable-Length Sequences Using Recurrent Neural Tangent Kernels

High dimensionality poses many challenges to the use of data, from visua...

0 Sina Alemohammad, et al. ∙

research

∙ 09/20/2020

Provable Finite Data Generalization with Group Autoencoder

Deep Autoencoders (AEs) provide a versatile framework to learn a compres...

0 Romain Cosentino, et al. ∙

research

∙ 06/25/2020

Ensembles of Generative Adversarial Networks for Disconnected Data

Most current computer vision datasets are composed of disconnected sets,...

0 Lorenzo Luzi, et al. ∙

research

∙ 06/18/2020

The Recurrent Neural Tangent Kernel

The study of deep networks (DNs) in the infinite-width limit, via the so...

0 Sina Alemohammad, et al. ∙

research

∙ 06/17/2020

Analytical Probability Distributions and EM-Learning for Deep Generative Networks

Deep Generative Networks (DGNs) with probabilistic modeling of their out...

0 Randall Balestriero, et al. ∙

research

∙ 06/13/2020

Interpretable Super-Resolution via a Learned Time-Series Representation

We develop an interpretable and learnable Wigner-Ville distribution that...

0 Randall Balestriero, et al. ∙

research

∙ 05/21/2020

SymJAX: symbolic CPU/GPU/TPU programming

SymJAX is a symbolic programming version of JAX simplifying graph input/...

0 Randall Balestriero, et al. ∙

research

∙ 02/26/2020

Max-Affine Spline Insights into Deep Generative Networks

We connect a large class of Generative Deep Networks (GDNs) with spline ...

9 Randall Balestriero, et al. ∙

research

∙ 05/28/2019

A Hessian Based Complexity Measure for Deep Networks

Deep (neural) networks have been applied productively in a wide range of...

0 Hamid Javadi, et al. ∙

research

∙ 05/21/2019

The Geometry of Deep Networks: Power Diagram Subdivision

We study the geometry of deep (neural) networks (DNs) with piecewise aff...

0 Randall Balestriero, et al. ∙

research

∙ 10/22/2018

From Hard to Soft: Understanding Deep Network Nonlinearities via Vector Quantization and Statistical Inference

Nonlinearity is crucial to the performance of a deep (neural) network (D...

16 Randall Balestriero, et al. ∙

research

∙ 05/17/2018

A Spline Theory of Deep Networks (Extended Version)

We build a rigorous bridge between deep networks (DNs) and approximation...

6 Randall Balestriero, et al. ∙

research

∙ 02/27/2018

Semi-Supervised Learning Enabled by Multiscale Deep Neural Network Inversion

Deep Neural Networks (DNNs) provide state-of-the-art solutions in severa...

0 Randall Balestriero, et al. ∙

research

∙ 12/25/2017

Overcomplete Frame Thresholding for Acoustic Scene Analysis

In this work, we derive a generic overcomplete frame thresholding scheme...

0 Romain Cosentino, et al. ∙

Randall Balestriero

Featured Co-authors