b'Noah A. Smith'

research

∙ 08/08/2023

SILO Language Models: Isolating Legal Risk In a Nonparametric Datastore

The legality of training language models (LMs) on copyrighted or otherwi...

0 Sewon Min, et al. ∙

research

∙ 07/19/2023

Efficiency Pentathlon: A Standardized Arena for Efficiency Evaluation

Rising computational demands of modern natural language processing (NLP)...

0 Hao Peng, et al. ∙

research

∙ 07/13/2023

DIALGEN: Collaborative Human-LM Generated Dialogues for Improved Understanding of Human-Human Conversations

Applications that could benefit from automatic understanding of human-hu...

0 Bo-Ru Lu, et al. ∙

research

∙ 06/24/2023

Estimating the Causal Effect of Early ArXiving on Paper Acceptance

What is the effect of releasing a preprint of a paper before it is submi...

0 Yanai Elazar, et al. ∙

research

∙ 06/16/2023

Reproducibility in NLP: What Have We Learned from the Checklist?

Scientific progress in NLP rests on the reproducibility of researchers' ...

0 Ian Magnusson, et al. ∙

research

∙ 06/09/2023

Morphosyntactic probing of multilingual BERT models

We introduce an extensive dataset for multilingual probing of morphologi...

0 Judit Acs, et al. ∙

research

∙ 06/03/2023

Stubborn Lexical Bias in Data and Models

In NLP, recent work has seen increased focus on spurious correlations be...

0 Sofia Serrano, et al. ∙

research

∙ 06/02/2023

Fine-Grained Human Feedback Gives Better Rewards for Language Model Training

Language models (LMs) often exhibit undesirable text generation behavior...

5 Zeqiu Wu, et al. ∙

research

∙ 03/24/2023

Scaling Expert Language Models with Unsupervised Domain Discovery

Large language models are typically trained densely: all parameters are ...

0 Suchin Gururangan, et al. ∙

research

∙ 01/11/2023

NarrowBERT: Accelerating Masked Language Model Pretraining and Inference

Large-scale language model pretraining is a very successful form of self...

0 Haoxin Li, et al. ∙

research

∙ 12/08/2022

Demystifying Prompts in Language Models via Perplexity Estimation

Language models can be prompted to perform a wide variety of zero- and f...

0 Hila Gonen, et al. ∙

research

∙ 12/01/2022

Data-Efficient Finetuning Using Cross-Task Nearest Neighbors

Language models trained on massive prompted multitask datasets like T0 (...

0 Hamish Ivison, et al. ∙

research

∙ 11/30/2022

Domain Mismatch Doesn't Always Prevent Cross-Lingual Transfer Learning

Cross-lingual transfer learning without labeled target language data or ...

0 Daniel Edmiston, et al. ∙

research

∙ 11/15/2022

PromptCap: Prompt-Guided Task-Aware Image Captioning

Image captioning aims to describe an image with a natural language sente...

10 Yushi Hu, et al. ∙

research

∙ 11/07/2022

How Much Does Attention Actually Attend? Questioning the Importance of Attention in Pretrained Transformers

The attention mechanism is considered the backbone of the widely-used Tr...

0 Michael Hassid, et al. ∙

research

∙ 10/16/2022

Modeling Context With Linear Attention for Scalable Document-Level Translation

Document-level machine translation leverages inter-sentence dependencies...

0 Zhaofeng Wu, et al. ∙

research

∙ 10/14/2022

Transparency Helps Reveal When Language Models Learn Meaning

Many current NLP systems are built from language models trained to optim...

0 Zhaofeng Wu, et al. ∙

research

∙ 10/07/2022

Measuring and Narrowing the Compositionality Gap in Language Models

We investigate the ability of language models to perform compositional r...

0 Ofir Press, et al. ∙

research

∙ 10/06/2022

Binding Language Models in Symbolic Languages

Though end-to-end neural approaches have recently been dominating NLP ta...

2 Zhoujun Cheng, et al. ∙

research

∙ 09/02/2022

Elaboration-Generating Commonsense Question Answering at Scale

In question answering requiring common sense, language models (e.g., GPT...

0 Wenya Wang, et al. ∙

research

∙ 08/05/2022

Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models

We present Branch-Train-Merge (BTM), a communication-efficient algorithm...

0 Margaret Li, et al. ∙

research

∙ 06/10/2022

Measuring the Carbon Intensity of AI in Cloud Instances

By providing unprecedented access to computational resources, cloud comp...

0 Jesse Dodge, et al. ∙

research

∙ 05/24/2022

Unsupervised Learning of Hierarchical Conversation Structure

Human conversations can evolve in many different ways, creating challeng...

11 Bo-Ru Lu, et al. ∙

research

∙ 03/16/2022

In-Context Learning for Few-Shot Dialogue State Tracking

Collecting and annotating task-oriented dialogues is time-consuming and ...

0 Yushi Hu, et al. ∙

research

∙ 01/25/2022

Whose Language Counts as High Quality? Measuring Language Ideologies in Text Data Selection

Language models increasingly rely on massive web dumps for diverse text ...

0 Suchin Gururangan, et al. ∙

research

∙ 11/14/2021

Time Waits for No One! Analysis and Challenges of Temporal Misalignment

When an NLP model is trained on text data from one time period and teste...

0 Kelvin Luu, et al. ∙

research

∙ 10/06/2021

ABC: Attention with Bounded-memory Control

Transformer architectures have achieved state-of-the-art results on a va...

0 Hao Peng, et al. ∙

research

∙ 10/01/2021

Expected Validation Performance and Estimation of a Random Variable's Maximum

Research in NLP is often supported by experimental results, and improved...

0 Jesse Dodge, et al. ∙

research

∙ 08/31/2021

Sentence Bottleneck Autoencoders from Transformer Language Models

Representation learning for text via pretraining a language model on a l...

0 Ivan Montero, et al. ∙

research

∙ 08/27/2021

Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation

Since the introduction of the transformer model by Vaswani et al. (2017)...

0 Ofir Press, et al. ∙

research

∙ 08/11/2021

DEMix Layers: Disentangling Domains for Modular Language Modeling

We introduce a new domain expert mixture (DEMix) layer that enables cond...

0 Suchin Gururangan, et al. ∙

research

∙ 07/02/2021

Scarecrow: A Framework for Scrutinizing Machine Text

Modern neural text generation systems can produce remarkably fluent and ...

0 Yao Dou, et al. ∙

research

∙ 06/30/2021

All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text

Human evaluations are typically considered the gold standard in natural ...

0 Elizabeth Clark, et al. ∙

research

∙ 06/30/2021

On the Power of Saturated Transformers: A View from Circuit Complexity

Transformers have become a standard architecture for many NLP problems. ...

0 William Merrill, et al. ∙

research

∙ 06/17/2021

Scientific Language Models for Biomedical Knowledge Base Completion: An Empirical Study

Biomedical knowledge graphs (KGs) hold rich information on entities such...

0 Rahul Nadkarni, et al. ∙

research

∙ 06/16/2021

Specializing Multilingual Language Models: An Empirical Study

Contextualized word representations from pretrained multilingual languag...

0 Ethan C. Chau, et al. ∙

research

∙ 05/07/2021

On-the-Fly Controlled Text Generation with Experts and Anti-Experts

Despite recent advances in natural language generation, it remains chall...

0 Alisa Liu, et al. ∙

research

∙ 05/07/2021

A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers

Readers of academic research papers often read with the goal of answerin...

21 Pradeep Dasigi, et al. ∙

research

∙ 04/22/2021

Provable Limitations of Acquiring Meaning from Ungrounded Form: What will Future Language Models Understand?

Language models trained on billions of tokens have recently led to unpre...

0 William Merrill, et al. ∙

research

∙ 04/18/2021

Go Forth and Prosper: Language Modeling with Ancient Textual History

We introduce a technique for improving document-level language models (L...

0 Rik Koncel-Kedziorski, et al. ∙

research

∙ 03/24/2021

Finetuning Pretrained Transformers into RNNs

Transformers have outperformed recurrent neural networks (RNNs) in natur...

14 Jungo Kasai, et al. ∙

research

∙ 03/03/2021

Random Feature Attention

Transformers are state-of-the-art models for a variety of sequence model...

0 Hao Peng, et al. ∙

research

∙ 01/29/2021

Challenges in Automated Debiasing for Toxic Language Detection

Biased associations have been a challenge in the development of classifi...

0 Xuhui Zhou, et al. ∙

research

∙ 01/17/2021

GENIE: A Leaderboard for Human-in-the-Loop Evaluation of Text Generation

Leaderboards have eased model development for many NLP datasets by stand...

7 Daniel Khashabi, et al. ∙

research

∙ 12/31/2020

Shortformer: Better Language Modeling using Shorter Inputs

We explore the benefits of decreasing the input length of transformers. ...

0 Ofir Press, et al. ∙

research

∙ 12/10/2020

Infusing Finetuning with Semantic Dependencies

For natural language processing systems, two kinds of evidence support t...

0 Zhaofeng Wu, et al. ∙

research

∙ 10/24/2020

Measuring Association Between Labels and Free-Text Rationales

Interpretable NLP has taking increasing interest in ensuring that explan...

0 Sarah Wiegreffe, et al. ∙

research

∙ 10/15/2020

Unsupervised Bitext Mining and Translation via Self-trained Contextual Embeddings

We describe an unsupervised method to create pseudo-parallel corpora for...

0 Phillip Keung, et al. ∙

research

∙ 10/15/2020

Natural Language Rationales with Full-Stack Visual Reasoning: From Pixels to Semantic Frames to Commonsense Graphs

Natural language rationales could provide intuitive, higher-level explan...

10 Ana Marasović, et al. ∙

research

∙ 10/06/2020

Plug and Play Autoencoders for Conditional Text Generation

Text autoencoders are commonly used for conditional generation tasks suc...

20 Florian Mai, et al. ∙

Noah A. Smith

Featured Co-authors