b'Omer Levy'

Few-Shot Question Answering by Pretraining Span Selection

In a number of question answering (QA) benchmarks, pretrained models hav...
01/02/2021 ∙ by Ori Ram, et al. ∙ 0 ∙ share

read it
Coreference Resolution without Span Representations

Since the introduction of deep pretrained language models, most task-spe...
01/02/2021 ∙ by Yuval Kirstain, et al. ∙ 0 ∙ share

read it
Transformer Feed-Forward Layers Are Key-Value Memories

Feed-forward layers constitute two-thirds of a transformer model's param...
12/29/2020 ∙ by Mor Geva, et al. ∙ 29 ∙ share

read it
The Turking Test: Can Language Models Understand Instructions?

Supervised machine learning provides the learner with a set of input-out...
10/22/2020 ∙ by Avia Efrat, et al. ∙ 0 ∙ share

read it
Neural Machine Translation without Embeddings

Many NLP models follow the embed-contextualize-predict paradigm, in whic...
08/21/2020 ∙ by Uri Shaham, et al. ∙ 0 ∙ share

read it
Aligned Cross Entropy for Non-Autoregressive Machine Translation

Non-autoregressive machine translation models significantly speed up dec...
04/03/2020 ∙ by Marjan Ghazvininejad, et al. ∙ 9 ∙ share

read it
Semi-Autoregressive Training Improves Mask-Predict Decoding

The recently proposed mask-predict decoding algorithm has narrowed the p...
01/23/2020 ∙ by Marjan Ghazvininejad, et al. ∙ 3 ∙ share

read it
Improving Transformer Models by Reordering their Sublayers

Multilayer transformer networks consist of interleaved self-attention an...
11/10/2019 ∙ by Ofir Press, et al. ∙ 0 ∙ share

read it
Blockwise Self-Attention for Long Document Understanding

We present BlockBERT, a lightweight and efficient BERT model that is des...
11/07/2019 ∙ by Jiezhong Qiu, et al. ∙ 0 ∙ share

read it
Generalization through Memorization: Nearest Neighbor Language Models

We introduce kNN-LMs, which extend a pre-trained neural language model (...
11/01/2019 ∙ by Urvashi Khandelwal, et al. ∙ 0 ∙ share

read it
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

We present BART, a denoising autoencoder for pretraining sequence-to-seq...
10/29/2019 ∙ by Mike Lewis, et al. ∙ 35 ∙ share

read it
Structural Language Models of Code

We address the problem of any-code completion - generating a missing pie...
09/30/2019 ∙ by Uri Alon, et al. ∙ 0 ∙ share

read it
Structural Language Models for Any-Code Generation

We address the problem of Any-Code Generation (AnyGen) - generating code...
09/30/2019 ∙ by Uri Alon, et al. ∙ 0 ∙ share

read it
BERT for Coreference Resolution: Baselines and Analysis

We apply BERT to coreference resolution, achieving strong improvements o...
08/24/2019 ∙ by Mandar Joshi, et al. ∙ 0 ∙ share

read it
RoBERTa: A Robustly Optimized BERT Pretraining Approach

Language model pretraining has led to significant performance gains but ...
07/26/2019 ∙ by Yinhan Liu, et al. ∙ 0 ∙ share

read it
SpanBERT: Improving Pre-training by Representing and Predicting Spans

We present SpanBERT, a pre-training method that is designed to better re...
07/24/2019 ∙ by Mandar Joshi, et al. ∙ 0 ∙ share

read it
What Does BERT Look At? An Analysis of BERT's Attention

Large pre-trained neural networks such as BERT have had great recent suc...
06/11/2019 ∙ by Kevin Clark, et al. ∙ 0 ∙ share

read it
Are Sixteen Heads Really Better than One?

Attention is a powerful and ubiquitous mechanism for allowing neural mod...
05/25/2019 ∙ by Paul Michel, et al. ∙ 0 ∙ share

read it
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems

In the last year, new models and methods for pretraining and transfer le...
05/02/2019 ∙ by Alex Wang, et al. ∙ 4 ∙ share

read it
Constant-Time Machine Translation with Conditional Masked Language Models

Most machine translation systems generate text autoregressively, by sequ...
04/19/2019 ∙ by Marjan Ghazvininejad, et al. ∙ 6 ∙ share

read it
Training on Synthetic Noise Improves Robustness to Natural Noise in Machine Translation

We consider the problem of making machine translation more robust to cha...
02/05/2019 ∙ by Vladimir Karpukhin, et al. ∙ 0 ∙ share

read it
pair2vec: Compositional Word-Pair Embeddings for Cross-Sentence Inference

Reasoning about implied relationships (e.g. paraphrastic, common sense, ...
10/20/2018 ∙ by Mandar Joshi, et al. ∙ 0 ∙ share

read it
code2seq: Generating Sequences from Structured Representations of Code

The ability to generate natural language sequences from source code snip...
08/04/2018 ∙ by Uri Alon, et al. ∙ 0 ∙ share

read it
Ultra-Fine Entity Typing

We introduce a new entity typing task: given a sentence with an entity m...
07/13/2018 ∙ by Eunsol Choi, et al. ∙ 2 ∙ share

read it
LSTMs Exploit Linguistic Attributes of Data

While recurrent neural networks have found success in a variety of natur...
05/29/2018 ∙ by Nelson F. Liu, et al. ∙ 0 ∙ share

read it
Jointly Predicting Predicates and Arguments in Neural Semantic Role Labeling

Recent BIO-tagging-based neural semantic role labeling models are very h...
05/12/2018 ∙ by Luheng He, et al. ∙ 0 ∙ share

read it
Deep RNNs Encode Soft Hierarchical Syntax

We present a set of experiments to demonstrate that deep recurrent neura...
05/11/2018 ∙ by Terra Blevins, et al. ∙ 0 ∙ share

read it
Long Short-Term Memory as a Dynamically Computed Element-wise Weighted Sum

LSTMs were introduced to combat vanishing gradients in simple RNNs by au...
05/09/2018 ∙ by Omer Levy, et al. ∙ 4 ∙ share

read it
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding

For natural language understanding (NLU) technology to be maximally usef...
04/20/2018 ∙ by Alex Wang, et al. ∙ 0 ∙ share

read it
A General Path-Based Representation for Predicting Program Properties

Predicting program properties such as names or expression types has a wi...
03/26/2018 ∙ by Uri Alon, et al. ∙ 0 ∙ share

read it
code2vec: Learning Distributed Representations of Code

We present a neural model for representing snippets of code as continuou...
03/26/2018 ∙ by Uri Alon, et al. ∙ 0 ∙ share

read it
Annotation Artifacts in Natural Language Inference Data

Large-scale datasets for natural language inference are created by prese...
03/06/2018 ∙ by Suchin Gururangan, et al. ∙ 0 ∙ share

read it
Simulating Action Dynamics with Neural Process Networks

Understanding procedural language requires anticipating the causal effec...
11/14/2017 ∙ by Antoine Bosselut, et al. ∙ 0 ∙ share

read it
Named Entity Disambiguation for Noisy Text

We address the task of Named Entity Disambiguation (NED) for noisy text....
06/28/2017 ∙ by Yotam Eshel, et al. ∙ 0 ∙ share

read it
Zero-Shot Relation Extraction via Reading Comprehension

We show that relation extraction can be reduced to answering simple read...
06/13/2017 ∙ by Omer Levy, et al. ∙ 0 ∙ share

read it
Recurrent Additive Networks

We introduce recurrent additive networks (RANs), a new gated RNN which i...
05/21/2017 ∙ by Kenton Lee, et al. ∙ 0 ∙ share

read it
A Strong Baseline for Learning Cross-Lingual Word Embeddings from Sentence Alignments

While cross-lingual word embeddings have been studied extensively in rec...
08/18/2016 ∙ by Omer Levy, et al. ∙ 0 ∙ share

read it
word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method

The word2vec software of Tomas Mikolov and colleagues (https://code.goog...
02/15/2014 ∙ by Yoav Goldberg, et al. ∙ 0 ∙ share

read it

Omer Levy

Featured Co-authors