Ofir Press | DeepAI

AI Chat

Featured Co-authors

Lior Wolf
168 publications
Noah A. Smith
127 publications
Jonathan Berant
84 publications
Mike Lewis
64 publications
Omer Levy
63 publications
Colin Raffel
57 publications
Ludwig Schmidt
48 publications
Sheng Shen
33 publications
Jason Phang
33 publications
Iz Beltagy
32 publications
Sewon Min
29 publications

∙ 10/27/2022

What Language Model to Train if You Have One Million GPU Hours?

The crystallization of modeling methods around the Transformer architect...

4 Teven Le Scao, et al. ∙

∙ 10/07/2022

Measuring and Narrowing the Compositionality Gap in Language Models

We investigate the ability of language models to perform compositional r...

0 Ofir Press, et al. ∙

∙ 03/30/2022

Transformer Language Models without Positional Encodings Still Learn Positional Information

Transformers typically require some form of positional encoding, such as...

0 Adi Haviv, et al. ∙

∙ 08/27/2021

Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation

Since the introduction of the transformer model by Vaswani et al. (2017)...

0 Ofir Press, et al. ∙

∙ 12/31/2020

Shortformer: Better Language Modeling using Shorter Inputs

We explore the benefits of decreasing the input length of transformers. ...

0 Ofir Press, et al. ∙

∙ 11/10/2019

Improving Transformer Models by Reordering their Sublayers

Multilayer transformer networks consist of interleaved self-attention an...

0 Ofir Press, et al. ∙

∙ 03/11/2019

Partially Shuffling the Training Data to Improve Language Models

Although SGD requires shuffling the training data between epochs, curren...

0 Ofir Press, et al. ∙

∙ 10/31/2018

You May Not Need Attention

In NMT, how far can we get without attention and without separate encodi...

0 Ofir Press, et al. ∙

∙ 06/05/2017

Language Generation with Recurrent Generative Adversarial Networks without Pre-training

Generative Adversarial Networks (GANs) have shown great promise recently...

0 Ofir Press, et al. ∙

∙ 08/20/2016

Using the Output Embedding to Improve Language Models

We study the topmost weight matrix of neural network language models. We...

0 Ofir Press, et al. ∙

Are you a researcher?
Expose your work
to one of the largest
A.I. communities in the world