Anna Rumshisky

research

∙ 07/11/2023

Stack More Layers Differently: High-Rank Training Through Low-Rank Updates

Despite the dominance and effectiveness of scaling, resulting in large n...

0 Vladislav Lialin, et al. ∙

research

∙ 06/14/2023

Recipes for Sequential Pre-training of Multilingual Encoder and Seq2Seq Models

Pre-trained encoder-only and sequence-to-sequence (seq2seq) models each ...

0 Saleh Soltan, et al. ∙

research

∙ 05/26/2023

Honey, I Shrunk the Language: Language Model Behavior at Reduced Scale

In recent years, language models have drastically grown in size, and the...

0 Vijeta Deshpande, et al. ∙

research

∙ 04/04/2023

Scalable and Accurate Self-supervised Multimodal Representation Learning without Aligned Video and Text Data

Scaling up weakly-supervised datasets has shown to be highly effective i...

0 Vladislav Lialin, et al. ∙

research

∙ 03/29/2023

Larger Probes Tell a Different Story: Extending Psycholinguistic Datasets Via In-Context Learning

Language model probing is often used to test specific capabilities of th...

0 Namrata Shivagunde, et al. ∙

research

∙ 03/28/2023

Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning

This paper presents a systematic overview and comparison of parameter-ef...

0 Vladislav Lialin, et al. ∙

research

∙ 11/15/2022

Reasoning Circuits: Few-shot Multihop Question Generation with Structured Rationales

Multi-hop Question Generation is the task of generating questions which ...

0 Saurabh Kulshreshtha, et al. ∙

research

∙ 10/08/2022

On Task-Adaptive Pretraining for Dialogue Response Selection

Recent advancements in dialogue response selection (DRS) are based on th...

0 Tzu-Hsiang Lin, et al. ∙

research

∙ 08/02/2022

AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Model

In this work, we demonstrate that multilingual large-scale sequence-to-s...

1 Saleh Soltan, et al. ∙

research

∙ 06/06/2022

Learning to Ask Like a Physician

Existing question answering (QA) datasets derived from electronic health...

0 Eric Lehman, et al. ∙

research

∙ 05/21/2022

Life after BERT: What do Other Muppets Understand about Language?

Existing pre-trained transformer analysis works usually focus only on on...

0 Vladislav Lialin, et al. ∙

research

∙ 05/20/2022

Down and Across: Introducing Crossword-Solving as a New NLP Benchmark

Solving crossword puzzles requires diverse reasoning capabilities, acces...

0 Saurabh Kulshreshtha, et al. ∙

research

∙ 05/06/2022

Federated Learning with Noisy User Feedback

Machine Learning (ML) systems are getting increasingly popular, and driv...

39 Rahul Sharma, et al. ∙

research

∙ 07/21/2021

Multi-Stream Transformers

Transformer-based encoder-decoder models produce a fused token-wise repr...

0 Mikhail Burtsev, et al. ∙

research

∙ 07/14/2021

An Efficient DP-SGD Mechanism for Large Scale NLP Models

Recent advances in deep learning have drastically improved performance o...

0 Christophe Dupuy, et al. ∙

research

∙ 05/14/2021

BERT Busters: Outlier LayerNorm Dimensions that Disrupt BERT

Multiple studies have shown that BERT is remarkably robust to pruning, y...

35 Olga Kovaleva, et al. ∙

research

∙ 10/15/2020

Continual Learning for Neural Semantic Parsing

A semantic parsing model is crucial to natural language processing appli...

0 Vladislav Lialin, et al. ∙

research

∙ 05/01/2020

When BERT Plays the Lottery, All Tickets Are Winning

Much of the recent success in NLP is due to the large Transformer-based ...

0 Sai Prasanna, et al. ∙

research

∙ 02/27/2020

A Primer in BERTology: What we know about how BERT works

Transformer-based models are now widely used in NLP, but we still do not...

0 Anna Rogers, et al. ∙

research

∙ 10/16/2019

Memory-Augmented Recurrent Networks for Dialogue Coherence

Recent dialogue approaches operate by reading each word in a conversatio...

0 David Donahue, et al. ∙

research

∙ 10/16/2019

Injecting Hierarchy with U-Net Transformers

The Transformer architecture has become increasingly popular over the pa...

0 David Donahue, et al. ∙

research

∙ 08/29/2019

NarrativeTime: Dense High-Speed Temporal Annotation on a Timeline

We present NarrativeTime, a new timeline-based annotation scheme for tem...

0 Anna Rogers, et al. ∙

research

∙ 08/28/2019

Solving Math Word Problems with Double-Decoder Transformer

This paper proposes a Transformer-based model to generate equations for ...

0 Yuanliang Meng, et al. ∙

research

∙ 08/21/2019

Revealing the Dark Secrets of BERT

BERT-based architectures currently give state-of-the-art performance on ...

0 Olga Kovaleva, et al. ∙

research

∙ 04/10/2019

What's in a Name? Reducing Bias in Bios without Access to Protected Attributes

There is a growing body of work that proposes methods for mitigating bia...

12 Alexey Romanov, et al. ∙

research

∙ 10/11/2018

Adversarial Text Generation Without Reinforcement Learning

Generative Adversarial Networks (GANs) have experienced a recent surge i...

0 David Donahue, et al. ∙

research

∙ 09/18/2018

Triad-based Neural Network for Coreference Resolution

We propose a triad-based neural network system that generates affinity s...

0 Yuanliang Meng, et al. ∙

research

∙ 08/27/2018

Adversarial Decomposition of Text Representation

In this paper, we present a method for adversarial decomposition of text...

0 Alexey Romanov, et al. ∙

research

∙ 03/06/2018

CliNER 2.0: Accessible and Accurate Clinical Concept Extraction

Clinical notes often describe important aspects of a patient's stay and ...

0 Willie Boag, et al. ∙

research

∙ 05/01/2017

Forced to Learn: Discovering Disentangled Representations Without Exhaustive Labels

Learning a better representation with neural networks is a challenging p...

0 Alexey Romanov, et al. ∙

research

∙ 03/17/2017

Temporal Information Extraction for Question Answering Using Syntactic Dependencies in an LSTM-based Architecture

In this paper, we propose to use a set of simple, uniform in architectur...

0 Yuanliang Meng, et al. ∙

research

∙ 12/28/2016

Here's My Point: Joint Pointer Architecture for Argument Mining

One of the major goals in automated argumentation mining is to uncover t...

0 Peter Potash, et al. ∙

research

∙ 12/09/2016

#HashtagWars: Learning a Sense of Humor

In this work, we present a new dataset for computational humor, specific...

0 Peter Potash, et al. ∙

research

∙ 12/09/2016

Evaluating Creative Language Generation: The Case of Rap Lyric Ghostwriting

Language generation tasks that seek to mimic human ability to use langua...

0 Peter Potash, et al. ∙

research

∙ 10/16/2015

Normalization of Relative and Incomplete Temporal Expressions in Clinical Narratives

We analyze the RI-TIMEXes in temporally annotated corpora and propose tw...

0 Weiyi Sun, et al. ∙

Anna Rumshisky

Featured Co-authors