Nicolas Ballas

research

∙ 07/31/2023

Predicting masked tokens in stochastic locations improves masked image modeling

Self-supervised learning is a promising paradigm in deep learning that e...

0 Amir Bar, et al. ∙

research

∙ 04/14/2023

DINOv2: Learning Robust Visual Features without Supervision

The recent breakthroughs in natural language processing for model pretra...

1 Maxime Oquab, et al. ∙

research

∙ 04/11/2023

A surprisingly simple technique to control the pretraining bias for better transfer: Expand or Narrow your representation

Self-Supervised Learning (SSL) models rely on a pretext task to learn re...

0 Florian Bordes, et al. ∙

research

∙ 01/23/2023

A Simple Recipe for Competitive Low-compute Self supervised Vision Models

Self-supervised methods in vision have been mostly focused on large arch...

0 Quentin Duval, et al. ∙

research

∙ 01/19/2023

Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture

This paper demonstrates an approach for learning highly semantic image r...

2 Mahmoud Assran, et al. ∙

research

∙ 12/10/2022

Uniform Masking Prevails in Vision-Language Pretraining

Masked Language Modeling (MLM) has proven to be an essential component o...

0 Siddharth Verma, et al. ∙

research

∙ 11/03/2022

ImageNet-X: Understanding Model Mistakes with Factor of Variation Annotations

Deep learning vision systems are widely deployed across applications whe...

2 Badr Youbi Idrissi, et al. ∙

research

∙ 10/13/2022

The Hidden Uniform Cluster Prior in Self-Supervised Learning

A successful paradigm in representation learning is to perform self-supe...

0 Mahmoud Assran, et al. ∙

research

∙ 06/01/2022

Cascaded Video Generation for Videos In-the-Wild

Videos can be created by first outlining a global view of the scene and ...

0 Lluis Castrejon, et al. ∙

research

∙ 04/14/2022

Masked Siamese Networks for Label-Efficient Learning

We propose Masked Siamese Networks (MSN), a self-supervised learning fra...

25 Mahmoud Assran, et al. ∙

research

∙ 12/31/2021

BARACK: Partially Supervised Group Robustness With Guarantees

While neural networks have shown remarkable success on classification ta...

4 Nimit Sohoni, et al. ∙

research

∙ 10/15/2021

Trade-offs of Local SGD at Scale: An Empirical Study

As datasets and models become increasingly large, distributed training h...

0 Jose Javier Gonzalez Ortiz, et al. ∙

research

∙ 06/04/2021

Hierarchical Video Generation for Complex Data

Videos can often be created by first outlining a global description of t...

0 Lluis Castrejon, et al. ∙

research

∙ 04/28/2021

Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samples

This paper proposes a novel method of learning by predicting view assign...

0 Mahmoud Assran, et al. ∙

research

∙ 10/06/2020

A Closer Look at Codistillation for Distributed Training

Codistillation has been proposed as a mechanism to share knowledge among...

0 Shagun Sodhani, et al. ∙

research

∙ 06/22/2020

Revisiting Loss Modelling for Unstructured Pruning

By removing parameters from deep neural networks, unstructured pruning m...

0 César Laurent, et al. ∙

research

∙ 06/18/2020

Recovering Petaflops in Contrastive Semi-Supervised Learning of Visual Representations

We investigate a strategy for improving the computational efficiency of ...

0 Mahmoud Assran, et al. ∙

research

∙ 10/01/2019

SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentum

Distributed optimization is essential for training large models on large...

0 Jianyu Wang, et al. ∙

research

∙ 08/16/2019

Needles in Haystacks: On Classifying Tiny Objects in Large Images

In some computer vision domains, such as medical or hyperspectral imagin...

6 Nick Pawlowski, et al. ∙

research

∙ 06/09/2019

Gossip-based Actor-Learner Architectures for Deep Reinforcement Learning

Multi-simulator training has contributed to the recent success of Deep R...

0 Mahmoud Assran, et al. ∙

research

∙ 04/27/2019

Improved Conditional VRNNs for Video Prediction

Predicting future frames for a video sequence is a challenging generativ...

8 Lluis Castrejon, et al. ∙

research

∙ 11/27/2018

Stochastic Gradient Push for Distributed Deep Learning

Large mini-batch parallel SGD is commonly used for distributed training ...

0 Mahmoud Assran, et al. ∙

research

∙ 07/13/2018

DNN's Sharpest Directions Along the SGD Trajectory

Recent work has identified that using a high learning rate or a small ba...

4 Stanisław Jastrzębski, et al. ∙

research

∙ 06/20/2018

A Dissection of Overfitting and Generalization in Continuous Reinforcement Learning

The risks and perils of overfitting in machine learning are well known. ...

0 Amy Zhang, et al. ∙

research

∙ 06/11/2018

Fast Approximate Natural Gradient Descent in a Kronecker-factored Eigenbasis

Optimization algorithms that leverage gradient covariance information, s...

0 Thomas George, et al. ∙

research

∙ 11/13/2017

Three Factors Influencing Minima in SGD

We study the properties of the endpoint of stochastic gradient descent (...

0 Stanisław Jastrzębski, et al. ∙

research

∙ 10/13/2017

Residual Connections Encourage Iterative Inference

Residual networks (Resnets) have become a prominent architecture in deep...

0 Stanisław Jastrzębski, et al. ∙

research

∙ 06/16/2017

A Closer Look at Memorization in Deep Networks

We examine the role of memorization in deep learning, drawing connection...

0 Devansh Arpit, et al. ∙

research

∙ 11/23/2016

A dataset and exploration of models for understanding video data through fill-in-the-blank question-answering

While deep convolutional neural networks frequently approach or exceed h...

0 Tegan Maharaj, et al. ∙

research

∙ 06/03/2016

Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations

We propose zoneout, a novel method for regularizing RNNs. At each timest...

0 David Krueger, et al. ∙

research

∙ 05/09/2016

Theano: A Python framework for fast computation of mathematical expressions

Theano is a Python library that allows to define, optimize, and evaluate...

0 The Theano Development Team, et al. ∙

research

∙ 11/24/2015

Dynamic Capacity Networks

We introduce the Dynamic Capacity Network (DCN), a neural network that c...

0 Amjad Almahairi, et al. ∙

research

∙ 11/19/2015

Delving Deeper into Convolutional Networks for Learning Video Representations

We propose an approach to learn spatio-temporal features in videos from ...

0 Nicolas Ballas, et al. ∙

research

∙ 11/14/2015

Oracle performance for visual captioning

The task of associating images and videos with a natural language descri...

0 Li Yao, et al. ∙

research

∙ 02/27/2015

Describing Videos by Exploiting Temporal Structure

Recent progress in using recurrent neural networks (RNNs) for image desc...

0 Li Yao, et al. ∙

research

∙ 12/19/2014

FitNets: Hints for Thin Deep Nets

While depth tends to improve network performances, it also makes gradien...

0 Adriana Romero, et al. ∙

Nicolas Ballas

Featured Co-authors