Samuel R. Bowman

is this you? claim profile

∙ 0 followers
NYU college ∙ Stanford University

Assistant Professor at New York University

Comparing Test Sets with Item Response Theory

Recent years have seen numerous NLP datasets introduced to evaluate the ...
06/01/2021 ∙ by Clara Vania, et al. ∙ 7 ∙ share

read it
What Ingredients Make for an Effective Crowdsourcing Protocol for Difficult NLU Data Collection Tasks?

Crowdsourcing is widely used to create data for common natural language ...
06/01/2021 ∙ by Nikita Nangia, et al. ∙ 0 ∙ share

read it
Does Putting a Linguist in the Loop Improve NLU Data Collection?

Many crowdsourced NLP datasets contain systematic gaps and biases that a...
04/15/2021 ∙ by Alicia Parrish, et al. ∙ 0 ∙ share

read it
What Will it Take to Fix Benchmarking in Natural Language Understanding?

Evaluation for many natural language understanding (NLU) tasks is broken...
04/05/2021 ∙ by Samuel R. Bowman, et al. ∙ 0 ∙ share

read it
When Do You Need Billions of Words of Pretraining Data?

NLP is currently dominated by general-purpose pretrained language models...
11/10/2020 ∙ by Yian Zhang, et al. ∙ 0 ∙ share

read it
Asking Crowdworkers to Write Entailment Examples: The Best of Bad Options

Large-scale natural language inference (NLI) datasets such as SNLI or MN...
10/13/2020 ∙ by Clara Vania, et al. ∙ 0 ∙ share

read it
Learning Which Features Matter: RoBERTa Acquires a Preference for Linguistic Generalizations (Eventually)

One reason pretraining on self-supervised linguistic tasks is effective ...
10/11/2020 ∙ by Alex Warstadt, et al. ∙ 0 ∙ share

read it
Counterfactually-Augmented SNLI Training Data Does Not Yield Better Generalization Than Unaugmented Data

A growing body of work shows that models exploit annotation artifacts to...
10/09/2020 ∙ by William Huang, et al. ∙ 0 ∙ share

read it
Precise Task Formalization Matters in Winograd Schema Evaluations

Performance on the Winograd Schema Challenge (WSC), a respected English ...
10/08/2020 ∙ by Haokun Liu, et al. ∙ 0 ∙ share

read it
CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models

Pretrained language models, especially masked language models (MLMs) hav...
09/30/2020 ∙ by Nikita Nangia, et al. ∙ 8 ∙ share

read it
Can neural networks acquire a structural bias from raw linguistic data?

We evaluate whether BERT, a widely used neural network for sentence proc...
07/14/2020 ∙ by Alex Warstadt, et al. ∙ 0 ∙ share

read it
Self-Training for Unsupervised Parsing with PRPN

Neural unsupervised parsing (UP) models learn to parse without access to...
05/27/2020 ∙ by Anhad Mohananey, et al. ∙ 0 ∙ share

read it
English Intermediate-Task Training Improves Zero-Shot Cross-Lingual Transfer Too

Intermediate-task training has been shown to substantially improve pretr...
05/26/2020 ∙ by Jason Phang, et al. ∙ 0 ∙ share

read it
Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?

While pretrained models such as BERT have shown large gains across natur...
05/01/2020 ∙ by Yada Pruksachatkun, et al. ∙ 0 ∙ share

read it
Learning to Learn Morphological Inflection for Resource-Poor Languages

We propose to cast the task of morphological inflection - mapping a lemm...
04/28/2020 ∙ by Katharina Kann, et al. ∙ 0 ∙ share

read it
Collecting Entailment Data for Pretraining: New Protocols and Negative Results

Textual entailment (or NLI) data has proven useful as pretraining data f...
04/24/2020 ∙ by Samuel R. Bowman, et al. ∙ 0 ∙ share

read it
jiant: A Software Toolkit for Research on General-Purpose Text Understanding Models

We introduce jiant, an open source toolkit for conducting multitask and ...
03/04/2020 ∙ by Yada Pruksachatkun, et al. ∙ 0 ∙ share

read it
BLiMP: A Benchmark of Linguistic Minimal Pairs for English

We introduce The Benchmark of Linguistic Minimal Pairs (shortened to BLi...
12/02/2019 ∙ by Alex Warstadt, et al. ∙ 0 ∙ share

read it
Do Attention Heads in BERT Track Syntactic Dependencies?

We investigate the extent to which individual attention heads in pretrai...
11/27/2019 ∙ by Phu Mon Htut, et al. ∙ 0 ∙ share

read it
Inducing Constituency Trees through Neural Machine Translation

Latent tree learning(LTL) methods learn to parse sentences using only in...
09/22/2019 ∙ by Phu Mon Htut, et al. ∙ 0 ∙ share

read it
Investigating BERT's Knowledge of Language: Five Analysis Methods with NPIs

Though state-of-the-art sentence representation models can perform tasks...
09/05/2019 ∙ by Alex Warstadt, et al. ∙ 0 ∙ share

read it
Towards Realistic Practices In Low-Resource Natural Language Processing: The Development Set

Development sets are impractical to obtain for real low-resource languag...
09/04/2019 ∙ by Katharina Kann, et al. ∙ 0 ∙ share

read it
Human vs. Muppet: A Conservative Estimate of Human Performance on the GLUE Benchmark

The GLUE benchmark (Wang et al., 2019b) is a suite of language understan...
05/24/2019 ∙ by Nikita Nangia, et al. ∙ 5 ∙ share

read it
Human vs. Muppet: A Conservative Estimate of HumanPerformance on the GLUE Benchmark

The GLUE benchmark (Wang et al., 2019b) is a suite of language understan...
05/24/2019 ∙ by Nikita Nangia, et al. ∙ 3 ∙ share

read it
What do you learn from context? Probing for sentence structure in contextualized word representations

Contextualized representation models such as ELMo (Peters et al., 2018a)...
05/15/2019 ∙ by Ian Tenney, et al. ∙ 0 ∙ share

read it
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems

In the last year, new models and methods for pretraining and transfer le...
05/02/2019 ∙ by Alex Wang, et al. ∙ 4 ∙ share

read it
Probing What Different NLP Tasks Teach Machines about Function Word Comprehension

We introduce a set of nine challenge tasks that test for the understandi...
04/25/2019 ∙ by Najoung Kim, et al. ∙ 0 ∙ share

read it
Identifying and Reducing Gender Bias in Word-Level Language Models

Many text corpora exhibit socially problematic biases, which can be prop...
04/05/2019 ∙ by Shikha Bordia, et al. ∙ 0 ∙ share

read it
On Measuring Social Biases in Sentence Encoders

The Word Embedding Association Test shows that GloVe and word2vec word e...
03/25/2019 ∙ by Chandler May, et al. ∙ 0 ∙ share

read it
Grammatical Analysis of Pretrained Sentence Encoders with Acceptability Judgments

Recent pretrained sentence encoders achieve state of the art results on ...
01/11/2019 ∙ by Alex Warstadt, et al. ∙ 0 ∙ share

read it
Looking for ELMo's friends: Sentence-Level Pretraining Beyond Language Modeling

Work on the problem of contextualized word representation -- the develop...
12/28/2018 ∙ by Samuel R. Bowman, et al. ∙ 0 ∙ share

read it
Verb Argument Structure Alternations in Word and Sentence Embeddings

Verbs occur in different syntactic environments, or frames. We investiga...
11/27/2018 ∙ by Katharina Kann, et al. ∙ 0 ∙ share

read it
Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Tasks

Pretraining with language modeling and related unsupervised tasks has re...
11/02/2018 ∙ by Jason Phang, et al. ∙ 0 ∙ share

read it
Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task Analysis

Recent work using auxiliary prediction task classifiers to investigate t...
09/26/2018 ∙ by Kelly W. Zhang, et al. ∙ 0 ∙ share

read it
XNLI: Evaluating Cross-lingual Sentence Representations

State-of-the-art natural language processing systems rely on supervision...
09/13/2018 ∙ by Alexis Conneau, et al. ∙ 8 ∙ share

read it
Grammar Induction with Neural Language Models: An Unusual Replication

A substantial thread of recent work on latent tree learning has attempte...
08/29/2018 ∙ by Phu Mon Htut, et al. ∙ 0 ∙ share

read it
Neural Network Acceptability Judgments

In this work, we explore the ability of artificial neural networks to ju...
05/31/2018 ∙ by Alex Warstadt, et al. ∙ 0 ∙ share

read it
A Stable and Effective Learning Strategy for Trainable Greedy Decoding

As a widely used approximate search strategy for neural network decoders...
04/21/2018 ∙ by Yun Chen, et al. ∙ 0 ∙ share

read it
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding

For natural language understanding (NLU) technology to be maximally usef...
04/20/2018 ∙ by Alex Wang, et al. ∙ 0 ∙ share

read it
ListOps: A Diagnostic Dataset for Latent Tree Learning

Latent tree learning models learn to parse a sentence without syntactic ...
04/17/2018 ∙ by Nikita Nangia, et al. ∙ 0 ∙ share

read it
Training a Ranking Function for Open-Domain Question Answering

In recent years, there have been amazing advances in deep learning metho...
04/12/2018 ∙ by Phu Mon Htut, et al. ∙ 0 ∙ share

read it
Annotation Artifacts in Natural Language Inference Data

Large-scale datasets for natural language inference are created by prese...
03/06/2018 ∙ by Suchin Gururangan, et al. ∙ 0 ∙ share

read it
The Lifted Matrix-Space Model for Semantic Composition

Recent advances in tree structured sentence encoding models have shown t...
11/09/2017 ∙ by WooJin Chung, et al. ∙ 0 ∙ share

read it
Learning to parse from a semantic objective: It works. Is it syntax?

Recent work on reinforcement learning and other gradient estimators for ...
09/04/2017 ∙ by Adina Williams, et al. ∙ 0 ∙ share

read it
The RepEval 2017 Shared Task: Multi-Genre Natural Language Inference with Sentence Representations

This paper presents the results of the RepEval 2017 Shared Task, which e...
07/25/2017 ∙ by Nikita Nangia, et al. ∙ 0 ∙ share

read it
Sequential Attention: A Context-Aware Alignment Function for Machine Reading

In this paper we propose a neural network model with a novel Sequential ...
05/05/2017 ∙ by Sebastian Brarda, et al. ∙ 0 ∙ share

read it
Ruminating Reader: Reasoning with Gated Multi-Hop Attention

To answer the question in machine comprehension (MC) task, the models ne...
04/24/2017 ∙ by Yichen Gong, et al. ∙ 0 ∙ share

read it
Discourse-Based Objectives for Fast Unsupervised Sentence Representation Learning

This work presents a novel objective function for the unsupervised train...
04/23/2017 ∙ by Yacine Jernite, et al. ∙ 0 ∙ share

read it
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference

This paper introduces the Multi-Genre Natural Language Inference (MultiN...
04/18/2017 ∙ by Adina Williams, et al. ∙ 0 ∙ share

read it
A Fast Unified Model for Parsing and Sentence Understanding

Tree-structured neural networks exploit valuable syntactic parse informa...
03/19/2016 ∙ by Samuel R. Bowman, et al. ∙ 0 ∙ share

read it

Samuel R. Bowman

Featured Co-authors