
-
When Do You Need Billions of Words of Pretraining Data?
NLP is currently dominated by general-purpose pretrained language models...
read it
-
Asking Crowdworkers to Write Entailment Examples: The Best of Bad Options
Large-scale natural language inference (NLI) datasets such as SNLI or MN...
read it
-
Learning Which Features Matter: RoBERTa Acquires a Preference for Linguistic Generalizations (Eventually)
One reason pretraining on self-supervised linguistic tasks is effective ...
read it
-
Counterfactually-Augmented SNLI Training Data Does Not Yield Better Generalization Than Unaugmented Data
A growing body of work shows that models exploit annotation artifacts to...
read it
-
Precise Task Formalization Matters in Winograd Schema Evaluations
Performance on the Winograd Schema Challenge (WSC), a respected English ...
read it
-
CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models
Pretrained language models, especially masked language models (MLMs) hav...
read it
-
Can neural networks acquire a structural bias from raw linguistic data?
We evaluate whether BERT, a widely used neural network for sentence proc...
read it
-
Self-Training for Unsupervised Parsing with PRPN
Neural unsupervised parsing (UP) models learn to parse without access to...
read it
-
English Intermediate-Task Training Improves Zero-Shot Cross-Lingual Transfer Too
Intermediate-task training has been shown to substantially improve pretr...
read it
-
Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?
While pretrained models such as BERT have shown large gains across natur...
read it
-
Learning to Learn Morphological Inflection for Resource-Poor Languages
We propose to cast the task of morphological inflection - mapping a lemm...
read it
-
Collecting Entailment Data for Pretraining: New Protocols and Negative Results
Textual entailment (or NLI) data has proven useful as pretraining data f...
read it
-
jiant: A Software Toolkit for Research on General-Purpose Text Understanding Models
We introduce jiant, an open source toolkit for conducting multitask and ...
read it
-
BLiMP: A Benchmark of Linguistic Minimal Pairs for English
We introduce The Benchmark of Linguistic Minimal Pairs (shortened to BLi...
read it
-
Do Attention Heads in BERT Track Syntactic Dependencies?
We investigate the extent to which individual attention heads in pretrai...
read it
-
Inducing Constituency Trees through Neural Machine Translation
Latent tree learning(LTL) methods learn to parse sentences using only in...
read it
-
Investigating BERT's Knowledge of Language: Five Analysis Methods with NPIs
Though state-of-the-art sentence representation models can perform tasks...
read it
-
Towards Realistic Practices In Low-Resource Natural Language Processing: The Development Set
Development sets are impractical to obtain for real low-resource languag...
read it
-
Human vs. Muppet: A Conservative Estimate of Human Performance on the GLUE Benchmark
The GLUE benchmark (Wang et al., 2019b) is a suite of language understan...
read it
-
Human vs. Muppet: A Conservative Estimate of HumanPerformance on the GLUE Benchmark
The GLUE benchmark (Wang et al., 2019b) is a suite of language understan...
read it
-
What do you learn from context? Probing for sentence structure in contextualized word representations
Contextualized representation models such as ELMo (Peters et al., 2018a)...
read it
-
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
In the last year, new models and methods for pretraining and transfer le...
read it
-
Probing What Different NLP Tasks Teach Machines about Function Word Comprehension
We introduce a set of nine challenge tasks that test for the understandi...
read it
-
Identifying and Reducing Gender Bias in Word-Level Language Models
Many text corpora exhibit socially problematic biases, which can be prop...
read it
-
On Measuring Social Biases in Sentence Encoders
The Word Embedding Association Test shows that GloVe and word2vec word e...
read it
-
Grammatical Analysis of Pretrained Sentence Encoders with Acceptability Judgments
Recent pretrained sentence encoders achieve state of the art results on ...
read it
-
Looking for ELMo's friends: Sentence-Level Pretraining Beyond Language Modeling
Work on the problem of contextualized word representation -- the develop...
read it
-
Verb Argument Structure Alternations in Word and Sentence Embeddings
Verbs occur in different syntactic environments, or frames. We investiga...
read it
-
Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Tasks
Pretraining with language modeling and related unsupervised tasks has re...
read it
-
Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task Analysis
Recent work using auxiliary prediction task classifiers to investigate t...
read it
-
XNLI: Evaluating Cross-lingual Sentence Representations
State-of-the-art natural language processing systems rely on supervision...
read it
-
Grammar Induction with Neural Language Models: An Unusual Replication
A substantial thread of recent work on latent tree learning has attempte...
read it
-
Neural Network Acceptability Judgments
In this work, we explore the ability of artificial neural networks to ju...
read it
-
A Stable and Effective Learning Strategy for Trainable Greedy Decoding
As a widely used approximate search strategy for neural network decoders...
read it
-
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
For natural language understanding (NLU) technology to be maximally usef...
read it
-
ListOps: A Diagnostic Dataset for Latent Tree Learning
Latent tree learning models learn to parse a sentence without syntactic ...
read it
-
Training a Ranking Function for Open-Domain Question Answering
In recent years, there have been amazing advances in deep learning metho...
read it
-
Annotation Artifacts in Natural Language Inference Data
Large-scale datasets for natural language inference are created by prese...
read it
-
The Lifted Matrix-Space Model for Semantic Composition
Recent advances in tree structured sentence encoding models have shown t...
read it
-
Learning to parse from a semantic objective: It works. Is it syntax?
Recent work on reinforcement learning and other gradient estimators for ...
read it
-
The RepEval 2017 Shared Task: Multi-Genre Natural Language Inference with Sentence Representations
This paper presents the results of the RepEval 2017 Shared Task, which e...
read it
-
Sequential Attention: A Context-Aware Alignment Function for Machine Reading
In this paper we propose a neural network model with a novel Sequential ...
read it
-
Ruminating Reader: Reasoning with Gated Multi-Hop Attention
To answer the question in machine comprehension (MC) task, the models ne...
read it
-
Discourse-Based Objectives for Fast Unsupervised Sentence Representation Learning
This work presents a novel objective function for the unsupervised train...
read it
-
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
This paper introduces the Multi-Genre Natural Language Inference (MultiN...
read it
-
A Fast Unified Model for Parsing and Sentence Understanding
Tree-structured neural networks exploit valuable syntactic parse informa...
read it
-
Generating Sentences from a Continuous Space
The standard recurrent neural network language model (RNNLM) generates s...
read it
-
A large annotated corpus for learning natural language inference
Understanding entailment and contradiction is fundamental to understandi...
read it
-
Recursive Neural Networks Can Learn Logical Semantics
Tree-structured recursive neural networks (TreeRNNs) for sentence meanin...
read it