Stéphane d'Ascoli | DeepAI

DeepAI AI Chat

Featured Co-authors

Joan Bruna
96 publications
Florent Krzakala
88 publications
Yuanzhi Li
85 publications
Samy Bengio
55 publications
Yuhuai Wu
40 publications
Emmanuel Abbe
37 publications
Guillaume Lample
27 publications
Marc Lelarge
26 publications
Levent Sagun
22 publications
Giulio Biroli
21 publications
Hugo Touvron
20 publications

∙ 09/21/2023

Boolformer: Symbolic Regression of Logic Functions with Transformers

In this work, we introduce Boolformer, the first Transformer architectur...

0 Stéphane d'Ascoli, et al. ∙

∙ 06/27/2023

Length Generalization in Arithmetic Transformers

We examine how transformers cope with two challenges: learning basic int...

0 Samy Jelassi, et al. ∙

∙ 04/22/2022

End-to-end symbolic regression with transformers

Symbolic regression, the task of predicting the mathematical expression ...

63 Pierre-Alexandre Kamienny, et al. ∙

∙ 02/09/2022

Optimal learning rate schedules in high-dimensional non-convex optimization problems

Learning rate schedules are ubiquitously used to speed up and improve op...

0 Stéphane d'Ascoli, et al. ∙

∙ 01/12/2022

Deep Symbolic Regression for Recurrent Sequences

Symbolic regression, i.e. predicting a function from the observation of ...

13 Stéphane d'Ascoli, et al. ∙

∙ 06/10/2021

Transformed CNNs: recasting pre-trained convolutional layers with self-attention

Vision Transformers (ViT) have recently emerged as a powerful alternativ...

0 Stéphane d'Ascoli, et al. ∙

∙ 03/19/2021

ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases

Convolutional architectures have proven extremely successful for vision ...

5 Stéphane d'Ascoli, et al. ∙

∙ 03/09/2021

More data or more parameters? Investigating the effect of data structure on generalization

One of the central features of deep learning is the generalization abili...

0 Stéphane d'Ascoli, et al. ∙

∙ 11/24/2020

The dynamics of learning with feedback alignment

Direct Feedback Alignment (DFA) is emerging as an efficient and biologic...

13 Maria Refinetti, et al. ∙

∙ 11/03/2020

Conditioned Text Generation with Transfer for Closed-Domain Dialogue Systems

Scarcity of training data for task-oriented dialogue systems is a well k...

0 Stéphane d'Ascoli, et al. ∙

∙ 06/05/2020

Triple descent and the two kinds of overfitting: Where why do they appear?

A recent line of research has highlighted the existence of a double desc...

19 Stéphane d'Ascoli, et al. ∙

∙ 03/02/2020

Double Trouble in Double Descent : Bias and Variance(s) in the Lazy Regime

Deep neural networks can achieve remarkable generalization performances ...

19 Stéphane d'Ascoli, et al. ∙

∙ 11/09/2019

Conditioned Query Generation for Task-Oriented Dialogue Systems

Scarcity of training data for task-oriented dialogue systems is a well k...

7 Stéphane d'Ascoli, et al. ∙

∙ 06/16/2019

Finding the Needle in the Haystack with Convolutions: on the benefits of architectural bias

Despite the phenomenal success of deep neural networks in a broad range ...

0 Stéphane d'Ascoli, et al. ∙

∙ 01/06/2019

Scaling description of generalization with number of parameters in deep learning

We provide a description for the evolution of the generalization perform...

0 Mario Geiger, et al. ∙

∙ 10/22/2018

A jamming transition from under- to over-parametrization affects loss landscape and generalization

We argue that in fully-connected networks a phase transition delimits th...

0 Stefano Spigler, et al. ∙

∙ 09/25/2018

The jamming transition as a paradigm to understand the loss landscape of deep neural networks

Deep learning has been immensely successful at a variety of tasks, rangi...

0 Mario Geiger, et al. ∙

Are you a researcher?
Expose your work
to one of the largest
A.I. communities in the world