Luke Zettlemoyer

is this you? claim profile

∙ 0 followers
Facebook ∙ University of Washington ∙ Allen Institute for Artificial Intelligence

Associate Professor in the Allen School of Computer Science & Engineering at the University of Washington

BASE Layers: Simplifying Training of Large, Sparse Models

We introduce a new balanced assignment of experts (BASE) layer for large...
03/30/2021 ∙ by Mike Lewis, et al. ∙ 0 ∙ share

read it
Multilingual Autoregressive Entity Linking

We present mGENRE, a sequence-to-sequence system for the Multilingual En...
03/23/2021 ∙ by Nicola De Cao, et al. ∙ 0 ∙ share

read it
FEWS: Large-Scale, Low-Shot Word Sense Disambiguation with the Dictionary

Current models for Word Sense Disambiguation (WSD) struggle to disambigu...
02/16/2021 ∙ by Terra Blevins, et al. ∙ 0 ∙ share

read it
Muppet: Massive Multi-task Representations with Pre-Finetuning

We propose pre-finetuning, an additional large-scale learning stage betw...
01/26/2021 ∙ by Armen Aghajanyan, et al. ∙ 4 ∙ share

read it
Bilingual Lexicon Induction via Unsupervised Bitext Construction and Word Alignment

Bilingual lexicons map words in one language to their translations in an...
01/01/2021 ∙ by Haoyue Shi, et al. ∙ 0 ∙ share

read it
Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning

Although pretrained language models can be fine-tuned to produce state-o...
12/22/2020 ∙ by Armen Aghajanyan, et al. ∙ 0 ∙ share

read it
Learning to Model and Ignore Dataset Bias with Mixed Capacity Ensembles

Many datasets have been shown to contain incidental correlations created...
11/07/2020 ∙ by Christopher Clark, et al. ∙ 0 ∙ share

read it
Detecting Hallucinated Content in Conditional Neural Sequence Generation

Neural sequence models can generate highly fluent sentences but recent s...
11/05/2020 ∙ by Chunting Zhou, et al. ∙ 2 ∙ share

read it
Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic Parsing

Task-oriented semantic parsing is a critical component of virtual assist...
10/07/2020 ∙ by Xilun Chen, et al. ∙ 0 ∙ share

read it
Nearest Neighbor Machine Translation

We introduce k-nearest-neighbor machine translation (kNN-MT), which pred...
10/01/2020 ∙ by Urvashi Khandelwal, et al. ∙ 0 ∙ share

read it
Grounded Adaptation for Zero-shot Executable Semantic Parsing

We propose Grounded Adaptation for Zero-shot Executable Semantic Parsing...
09/16/2020 ∙ by Victor Zhong, et al. ∙ 5 ∙ share

read it
Better Fine-Tuning by Reducing Representational Collapse

Although widely adopted, existing approaches for fine-tuning pre-trained...
08/06/2020 ∙ by Armen Aghajanyan, et al. ∙ 2 ∙ share

read it
DeLighT: Very Deep and Light-weight Transformer

We introduce a very deep and light-weight transformer, DeLighT, that del...
08/03/2020 ∙ by Sachin Mehta, et al. ∙ 0 ∙ share

read it
Pre-training via Paraphrasing

We introduce MARGE, a pre-trained sequence-to-sequence model learned wit...
06/26/2020 ∙ by Mike Lewis, et al. ∙ 7 ∙ share

read it
Moving Down the Long Tail of Word Sense Disambiguation with Gloss-Informed Biencoders

A major obstacle in Word Sense Disambiguation (WSD) is that word senses ...
05/06/2020 ∙ by Terra Blevins, et al. ∙ 0 ∙ share

read it
An Information Bottleneck Approach for Controlling Conciseness in Rationale Extraction

Decisions of complex language understanding models can be rationalized b...
05/01/2020 ∙ by Bhargavi Paranjape, et al. ∙ 0 ∙ share

read it
Active Learning for Coreference Resolution using Discrete Annotation

We improve upon pairwise annotation for active learning in coreference r...
04/28/2020 ∙ by Belinda Li, et al. ∙ 0 ∙ share

read it
AmbigQA: Answering Ambiguous Open-domain Questions

Ambiguity is inherent to open-domain question answering; especially when...
04/22/2020 ∙ by Sewon Min, et al. ∙ 7 ∙ share

read it
Aligned Cross Entropy for Non-Autoregressive Machine Translation

Non-autoregressive machine translation models significantly speed up dec...
04/03/2020 ∙ by Marjan Ghazvininejad, et al. ∙ 9 ∙ share

read it
Semi-Autoregressive Training Improves Mask-Predict Decoding

The recently proposed mask-predict decoding algorithm has narrowed the p...
01/23/2020 ∙ by Marjan Ghazvininejad, et al. ∙ 3 ∙ share

read it
Multilingual Denoising Pre-training for Neural Machine Translation

This paper demonstrates that multilingual denoising pre-training produce...
01/22/2020 ∙ by Yinhan Liu, et al. ∙ 0 ∙ share

read it
ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks

We present ALFRED (Action Learning From Realistic Environments and Direc...
12/03/2019 ∙ by Mohit Shridhar, et al. ∙ 5 ∙ share

read it
Knowledge Guided Text Retrieval and Reading for Open Domain Question Answering

This paper presents a general approach for open-domain question answerin...
11/10/2019 ∙ by Sewon Min, et al. ∙ 30 ∙ share

read it
Zero-shot Entity Linking with Dense Entity Retrieval

We consider the zero-shot entity-linking challenge where each entity is ...
11/10/2019 ∙ by Ledell Wu, et al. ∙ 0 ∙ share

read it
Enforcing Encoder-Decoder Modularity in Sequence-to-Sequence Models

Inspired by modular software design principles of independence, intercha...
11/09/2019 ∙ by Siddharth Dalmia, et al. ∙ 0 ∙ share

read it
Crowdsourcing a High-Quality Gold Standard for QA-SRL

Question-answer driven Semantic Role Labeling (QA-SRL) has been proposed...
11/08/2019 ∙ by Paul Roit, et al. ∙ 0 ∙ share

read it
Unsupervised Cross-lingual Representation Learning at Scale

This paper shows that pretraining multilingual language models at scale ...
11/05/2019 ∙ by Alexis Conneau, et al. ∙ 0 ∙ share

read it
Emerging Cross-lingual Structure in Pretrained Language Models

We study the problem of multilingual masked language modeling, i.e. the ...
11/04/2019 ∙ by Shijie Wu, et al. ∙ 0 ∙ share

read it
Generalization through Memorization: Nearest Neighbor Language Models

We introduce kNN-LMs, which extend a pre-trained neural language model (...
11/01/2019 ∙ by Urvashi Khandelwal, et al. ∙ 0 ∙ share

read it
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

We present BART, a denoising autoencoder for pretraining sequence-to-seq...
10/29/2019 ∙ by Mike Lewis, et al. ∙ 35 ∙ share

read it
JuICe: A Large Scale Distantly Supervised Dataset for Open Domain Context-based Code Generation

Interactive programming with interleaved code snippet cells and natural ...
10/05/2019 ∙ by Rajas Agashe, et al. ∙ 0 ∙ share

read it
A Discrete Hard EM Approach for Weakly Supervised Question Answering

Many question answering (QA) tasks only provide weak supervision for how...
09/11/2019 ∙ by Sewon Min, et al. ∙ 0 ∙ share

read it
Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases

State-of-the-art models often make use of superficial patterns in the da...
09/09/2019 ∙ by Christopher Clark, et al. ∙ 10 ∙ share

read it
BERT for Coreference Resolution: Baselines and Analysis

We apply BERT to coreference resolution, achieving strong improvements o...
08/24/2019 ∙ by Mandar Joshi, et al. ∙ 0 ∙ share

read it
RoBERTa: A Robustly Optimized BERT Pretraining Approach

Language model pretraining has led to significant performance gains but ...
07/26/2019 ∙ by Yinhan Liu, et al. ∙ 0 ∙ share

read it
SpanBERT: Improving Pre-training by Representing and Predicting Spans

We present SpanBERT, a pre-training method that is designed to better re...
07/24/2019 ∙ by Mandar Joshi, et al. ∙ 0 ∙ share

read it
Vision-and-Dialog Navigation

Robots navigating in human environments should use language to ask for a...
07/10/2019 ∙ by Jesse Thomason, et al. ∙ 2 ∙ share

read it
Sparse Networks from Scratch: Faster Training without Losing Performance

We demonstrate the possibility of what we call sparse learning: accelera...
07/10/2019 ∙ by Tim Dettmers, et al. ∙ 2 ∙ share

read it
E3: Entailment-driven Extracting and Editing for Conversational Machine Reading

Conversational machine reading systems help users answer high-level ques...
06/12/2019 ∙ by Victor Zhong, et al. ∙ 1 ∙ share

read it
Multi-hop Reading Comprehension through Question Decomposition and Rescoring

Multi-hop Reading Comprehension (RC) requires reasoning and aggregation ...
06/07/2019 ∙ by Sewon Min, et al. ∙ 2 ∙ share

read it
Compositional Questions Do Not Necessitate Multi-hop Reasoning

Multi-hop reading comprehension (RC) questions are challenging because t...
06/07/2019 ∙ by Sewon Min, et al. ∙ 6 ∙ share

read it
Better Character Language Modeling Through Morphology

We incorporate morphological supervision into character language models ...
06/03/2019 ∙ by Terra Blevins, et al. ∙ 0 ∙ share

read it
Evaluating Gender Bias in Machine Translation

We present the first challenge set and evaluation protocol for the analy...
06/03/2019 ∙ by Gabriel Stanovsky, et al. ∙ 0 ∙ share

read it
Transformers with convolutional context for ASR

The recent success of transformer networks for neural machine translatio...
04/26/2019 ∙ by Abdelrahman Mohamed, et al. ∙ 0 ∙ share

read it
Constant-Time Machine Translation with Conditional Masked Language Models

Most machine translation systems generate text autoregressively, by sequ...
04/19/2019 ∙ by Marjan Ghazvininejad, et al. ∙ 6 ∙ share

read it
Learning Programmatic Idioms for Scalable Semantic Parsing

Programmers typically organize executable source code using high-level c...
04/19/2019 ∙ by Srinivasan Iyer, et al. ∙ 0 ∙ share

read it
Cloze-driven Pretraining of Self-attention Networks

We present a new approach for pretraining a bi-directional transformer m...
03/19/2019 ∙ by Alexei Baevski, et al. ∙ 0 ∙ share

read it
Improving Semantic Parsing for Task Oriented Dialog

Semantic parsing using hierarchical representations has recently been pr...
02/15/2019 ∙ by Arash Einolghozati, et al. ∙ 2 ∙ share

read it
The Referential Reader: A Recurrent Entity Network for Anaphora Resolution

We present a new architecture for storing and accessing entity mentions ...
02/05/2019 ∙ by Fei Liu, et al. ∙ 0 ∙ share

read it
pair2vec: Compositional Word-Pair Embeddings for Cross-Sentence Inference

Reasoning about implied relationships (e.g. paraphrastic, common sense, ...
10/20/2018 ∙ by Mandar Joshi, et al. ∙ 0 ∙ share

read it

Luke Zettlemoyer

Featured Co-authors