Julien Launay | DeepAI

DeepAI AI Chat

Featured Co-authors

Florent Krzakala
88 publications
Colin Raffel
57 publications
Sheng Shen
33 publications
Jason Phang
33 publications
Adam Roberts
32 publications
Iz Beltagy
32 publications
Alain Rakotomamonjy
28 publications
Djamé Seddah
21 publications
Hyung Won Chung
20 publications
Victor Sanh
17 publications
Laurent Meunier
17 publications

∙ 06/01/2023

The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only

Large language models are commonly trained on a mixture of filtered web ...

0 Guilherme Penedo, et al. ∙

∙ 10/27/2022

What Language Model to Train if You Have One Million GPU Hours?

The crystallization of modeling methods around the Transformer architect...

4 Teven Le Scao, et al. ∙

∙ 10/26/2022

Scaling Laws Beyond Backpropagation

Alternatives to backpropagation have long been studied to better underst...

0 Alessandro Cappelli, et al. ∙

∙ 04/12/2022

What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?

Large pretrained Transformer language models have been shown to exhibit ...

11 Thomas Wang, et al. ∙

∙ 10/16/2021

PAGnol: An Extra-Large French Generative Model

Access to large pre-trained models of varied architectures, in many diff...

0 Julien Launay, et al. ∙

∙ 09/24/2021

Is the Number of Trainable Parameters All That Actually Matters?

Recent work has identified simple empirical scaling laws for language mo...

0 Amélie Chatelain, et al. ∙

∙ 07/25/2021

LightOn Optical Processing Unit: Scaling-up AI and HPC with a Non von Neumann co-processor

We introduce LightOn's Optical Processing Unit (OPU), the first photonic...

0 Charles Brossollet, et al. ∙

∙ 06/07/2021

Photonic Differential Privacy with Direct Feedback Alignment

Optical Processing Units (OPUs) – low-power photonic chips dedicated to ...

7 Ruben Ohana, et al. ∙

∙ 01/06/2021

Adversarial Robustness by Design through Analog Computing and Synthetic Gradients

We propose a new defense mechanism against adversarial attacks inspired ...

5 Alessandro Cappelli, et al. ∙

∙ 12/11/2020

Hardware Beyond Backpropagation: a Photonic Co-Processor for Direct Feedback Alignment

The scaling hypothesis motivates the expansion of models past trillions ...

16 Julien Launay, et al. ∙

∙ 06/23/2020

Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures

Despite being the workhorse of deep learning, the backpropagation algori...

5 Julien Launay, et al. ∙

∙ 06/02/2020

Light-in-the-loop: using a photonics co-processor for scalable training of neural networks

As neural networks grow larger and more complex and data-hungry, trainin...

11 Julien Launay, et al. ∙

∙ 06/11/2019

Principled Training of Neural Networks with Direct Feedback Alignment

The backpropagation algorithm has long been the canonical training metho...

13 Julien Launay, et al. ∙

Are you a researcher?
Expose your work
to one of the largest
A.I. communities in the world