Ryan Cotterell

research

∙ 08/27/2023

An Analysis of On-the-fly Determinization of Finite-state Automata

In this paper we establish an abstraction of on-the-fly determinization ...

0 Ivan Baburin, et al. ∙

research

∙ 07/27/2023

A Geometric Notion of Causal Probing

Large language models rely on real-valued representations of text to mak...

0 Clément Guerner, et al. ∙

research

∙ 07/07/2023

On the Efficacy of Sampling Adapters

Sampling is a common strategy for generating text from probabilistic mod...

0 Clara Meister, et al. ∙

research

∙ 07/07/2023

Testing the Predictions of Surprisal Theory in 11 Languages

A fundamental result in psycholinguistics is that less predictable words...

0 Ethan Gotlieb Wilcox, et al. ∙

research

∙ 07/06/2023

Generalizing Backpropagation for Gradient-Based Interpretability

Many popular feature-attribution methods for interpreting deep neural ne...

0 Kevin Du, et al. ∙

research

∙ 07/06/2023

Efficient Semiring-Weighted Earley Parsing

This paper provides a reference description, in the form of a deduction ...

0 Andreas Opedal, et al. ∙

research

∙ 06/29/2023

Tokenization and the Noiseless Channel

Subword tokenization is a key part of many NLP pipelines. However, littl...

0 Vilém Zouhar, et al. ∙

research

∙ 06/29/2023

A Formal Perspective on Byte-Pair Encoding

Byte-Pair Encoding (BPE) is a popular algorithm used for tokenizing data...

0 Vilém Zouhar, et al. ∙

research

∙ 06/08/2023

Hexatagging: Projective Dependency Parsing as Tagging

We introduce a novel dependency parser, the hexatagger, that constructs ...

0 Afra Amini, et al. ∙

research

∙ 06/06/2023

LEACE: Perfect linear concept erasure in closed form

Concept erasure aims to remove specified features from a representation....

2 Nora Belrose, et al. ∙

research

∙ 06/06/2023

A Cross-Linguistic Pressure for Uniform Information Density in Word Order

While natural languages differ widely in both canonical word order and w...

7 Thomas Hikaru Clark, et al. ∙

research

∙ 06/06/2023

Convergence and Diversity in the Control Hierarchy

Weir has defined a hierarchy of language classes whose second member (ℒ_...

0 Alexandra Butoi, et al. ∙

research

∙ 06/05/2023

Structured Voronoi Sampling

Recently, there has been a growing interest in the development of gradie...

0 Afra Amini, et al. ∙

research

∙ 06/04/2023

A Fast Algorithm for Computing Prefix Probabilities

Multiple algorithms are known for efficiently calculating the prefix pro...

0 Franz Nowak, et al. ∙

research

∙ 05/24/2023

Learning the String Partial Order

We show that most structured prediction problems can be solved in linear...

0 Tianyu Liu, et al. ∙

research

∙ 05/23/2023

All Roads Lead to Rome? Exploring the Invariance of Transformers' Representations

Transformer models bring propelling advances in various NLP tasks, thus ...

2 Yuxin Ren, et al. ∙

research

∙ 05/22/2023

RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text

The fixed-size context of Transformer makes GPT models incapable of gene...

0 Wangchunshu Zhou, et al. ∙

research

∙ 05/18/2023

Efficient Prompting via Dynamic In-Context Learning

The primary way of building AI applications is shifting from training sp...

0 Wangchunshu Zhou, et al. ∙

research

∙ 05/18/2023

Discourse Centric Evaluation of Machine Translation with a Densely Annotated Parallel Corpus

Several recent papers claim human parity at sentence-level Machine Trans...

0 Yuchen Eleanor Jiang, et al. ∙

research

∙ 04/27/2023

Controlled Text Generation with Natural Language Instructions

Large language models generate fluent texts and can follow natural langu...

0 Wangchunshu Zhou, et al. ∙

research

∙ 03/30/2023

Discriminative Class Tokens for Text-to-Image Diffusion Models

Recent advances in text-to-image diffusion models have enabled the gener...

9 Idan Schwartz, et al. ∙

research

∙ 01/17/2023

Algorithms for Acyclic Weighted Finite-State Automata with Failure Arcs

Weighted finite-state automata (WSFAs) are commonly used in NLP. Failure...

0 Anej Svete, et al. ∙

research

∙ 12/20/2022

A Measure-Theoretic Characterization of Tight Language Models

Language modeling, a central task in natural language processing, involv...

0 Li Du, et al. ∙

research

∙ 12/08/2022

The Ordered Matrix Dirichlet for Modeling Ordinal Dynamics

Many dynamical systems exhibit latent states with intrinsic orderings su...

0 Niklas Stoehr, et al. ∙

research

∙ 11/25/2022

On the Effect of Anticipation on Reading Times

Over the past two decades, numerous studies have demonstrated how less p...

0 Tiago Pimentel, et al. ∙

research

∙ 11/23/2022

Schrödinger's Bat: Diffusion Models Sometimes Generate Polysemous Words in Superposition

Recent work has shown that despite their impressive capabilities, text-t...

0 Jennifer C. White, et al. ∙

research

∙ 11/14/2022

On Parsing as Tagging

There have been many proposals to reduce constituency parsing to tagging...

0 Afra Amini, et al. ∙

research

∙ 11/11/2022

The Architectural Bottleneck Principle

In this paper, we seek to measure how much information a component in a ...

0 Tiago Pimentel, et al. ∙

research

∙ 10/26/2022

Autoregressive Structured Prediction with Language Models

Recent years have seen a paradigm shift in NLP towards using pretrained ...

0 Tianyu Liu, et al. ∙

research

∙ 10/26/2022

Investigating the Role of Centering Theory in the Context of Neural Coreference Resolution Systems

Centering theory (CT; Grosz et al., 1995) provides a linguistic analysis...

0 Yuchen Eleanor Jiang, et al. ∙

research

∙ 10/26/2022

A Bilingual Parallel Corpus with Discourse Annotations

Machine translation (MT) has almost achieved human parity at sentence-le...

0 Yuchen Eleanor Jiang, et al. ∙

research

∙ 10/24/2022

Mutual Information Alleviates Hallucinations in Abstractive Summarization

Despite significant progress in the quality of language generated from a...

0 Liam van der Poel, et al. ∙

research

∙ 10/18/2022

Linear Guardedness and its Implications

Previous work on concept identification in neural representations has fo...

2 Shauli Ravfogel, et al. ∙

research

∙ 10/13/2022

Algorithms for Weighted Pushdown Automata

Weighted pushdown automata (WPDAs) are at the core of many natural langu...

0 Alexandra Butoi, et al. ∙

research

∙ 10/08/2022

An Ordinal Latent Variable Model of Conflict Intensity

For the quantitative monitoring of international relations, political ev...

0 Niklas Stoehr, et al. ∙

research

∙ 09/22/2022

Equivariant Transduction through Invariant Alignment

The ability to generalize compositionally is key to understanding the po...

0 Jennifer C. White, et al. ∙

research

∙ 09/14/2022

On the Intersection of Context-Free and Regular Languages

The Bar-Hillel construction is a classic result in formal language theor...

0 Clemente Pasti, et al. ∙

research

∙ 08/17/2022

On the Role of Negative Precedent in Legal Outcome Prediction

Every legal case sets a precedent by developing the law in one of the fo...

0 Josef Valvoda, et al. ∙

research

∙ 08/17/2022

Learning Transductions to Test Systematic Compositionality

Recombining known primitive concepts into larger novel combinations is a...

0 Josef Valvoda, et al. ∙

research

∙ 08/17/2022

Visual Comparison of Language Model Adaptation

Neural language models are widely used; however, their model parameters ...

10 Rita Sevastjanova, et al. ∙

research

∙ 07/04/2022

Probing via Prompting

Probing is a popular method to discern what linguistic information is co...

0 Jiaoda Li, et al. ∙

research

∙ 06/15/2022

The SIGMORPHON 2022 Shared Task on Morpheme Segmentation

The SIGMORPHON 2022 shared task on morpheme segmentation challenged syst...

0 Khuyagbaatar Batsuren, et al. ∙

research

∙ 05/31/2022

Cluster-based Evaluation of Automatically Generated Text

While probabilistic language generators have improved dramatically over ...

0 Tiago Pimentel, et al. ∙

research

∙ 05/14/2022

Naturalistic Causal Probing for Morpho-Syntax

Probing has become a go-to methodology for interpreting and analyzing de...

5 Afra Amini, et al. ∙

research

∙ 05/08/2022

A Structured Span Selector

Many natural language processing tasks, e.g., coreference resolution and...

0 Tianyu Liu, et al. ∙

research

∙ 05/07/2022

UniMorph 4.0: Universal Morphology

The Universal Morphology (UniMorph) project is a collaborative effort pr...

2 Khuyagbaatar Batsuren, et al. ∙

research

∙ 05/04/2022

Same Neurons, Different Languages: Probing Morphosyntax in Multilingual Pre-trained Models

The success of multilingual pre-trained models is underpinned by their a...

0 Karolina Stańczak, et al. ∙

research

∙ 05/03/2022

Exact Paired-Permutation Testing for Structured Test Statistics

Significance testing – especially the paired-permutation test – has play...

0 Ran Zmigrod, et al. ∙

research

∙ 04/19/2022

Probing for the Usage of Grammatical Number

A central quest of probing is to uncover how pre-trained models encode a...

0 Karim Lasri, et al. ∙

research

∙ 04/04/2022

Estimating the Entropy of Linguistic Distributions

Shannon entropy is often a quantity of interest to linguists studying th...

0 Aryaman Arora, et al. ∙

Ryan Cotterell

Featured Co-authors