research
∙
06/23/2023
Long-range Language Modeling with Self-retrieval
Retrieval-augmented language models (LMs) have received much attention r...
share
research
∙
05/31/2023
From Pixels to UI Actions: Learning to Follow Instructions via Graphical User Interfaces
Much of the previous work towards digital agents for graphical user inte...
research
∙
05/23/2023
ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding
We introduce ZeroSCROLLS, a zero-shot benchmark for natural language und...
research
∙
04/25/2023
Answering Questions by Meta-Reasoning over Multiple Chains of Thought
Modern systems for multi-hop question answering (QA) typically break que...
research
∙
01/30/2023
Crawling the Internal Knowledge-Base of Language Models
Language models are trained on large volumes of text, and as a result th...
research
∙
12/20/2022
What Are You Token About? Dense Retrieval as Distributions Over the Vocabulary
Dual encoders are now the dominant architecture for dense retrieval. Yet...
research
∙
12/13/2022
Diverse Demonstrations Improve In-context Compositional Generalization
In-context learning has shown great success in i.i.d semantic parsing sp...
research
∙
12/01/2022
Simplifying and Understanding State Space Models with Diagonal Linear RNNs
Sequence models based on linear state spaces (SSMs) have recently emerge...
research
∙
11/01/2022
Training Vision-Language Models with Less Bimodal Supervision
Standard practice in pretraining multimodal models, such as vision-langu...
research
∙
09/06/2022
Analyzing Transformers in Embedding Space
Understanding Transformer-based models has attracted significant attenti...
research
∙
08/01/2022
Efficient Long-Text Understanding with Short-Text Models
Transformer-based pretrained language models (LMs) are ubiquitous across...
research
∙
05/25/2022
QAMPARI: : An Open-domain Question Answering Benchmark for Questions with Many Answers from Multiple Paragraphs
Existing benchmarks for open-domain question answering (ODQA) typically ...
research
∙
04/28/2022
Inferring Implicit Relations with Language Models
A prominent challenge for modern language understanding systems is the a...
research
∙
02/13/2022
Scaling Laws Under the Microscope: Predicting Transformer Performance from Small Scale Experiments
Neural scaling laws define a predictable relationship between a model's ...
research
∙
01/15/2022
Unobserved Local Structures Make Compositional Generalization Hard
While recent work has convincingly showed that sequence-to-sequence mode...
research
∙
01/10/2022
SCROLLS: Standardized CompaRison Over Long Language Sequences
NLP benchmarks have largely focused on short texts, such as sentences an...
research
∙
12/16/2021
Learning To Retrieve Prompts for In-Context Learning
In-context learning is a recent paradigm in natural language understandi...
research
∙
12/14/2021
Learning to Retrieve Passages without Supervision
Dense retrievers for open-domain question answering (ODQA) have been sho...
research
∙
12/12/2021
Weakly Supervised Mapping of Natural Language to SQL through Question Decomposition
Natural Language Interfaces to Databases (NLIDBs), where users pose quer...
research
∙
09/22/2021
COVR: A test-bed for Visually Grounded Compositional Generalization with real images
While interest in models that generalize at test time to new composition...
research
∙
09/06/2021
Finding needles in a haystack: Sampling Structurally-diverse Training Sets from Synthetic Data for Compositional Generalization
Modern semantic parsers suffer from two principal limitations. First, tr...
research
∙
07/29/2021
Break, Perturb, Build: Automatic Perturbation of Reasoning Paths through Question Decomposition
Recent efforts to create challenge benchmarks that test the abilities of...
research
∙
07/15/2021
Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skills
Models pre-trained with a language modeling objective possess ample worl...
research
∙
06/13/2021
Memory-efficient Transformers via Top-k Attention
Following the success of dot-product attention in Transformers, numerous...
research
∙
04/17/2021
Question Decomposition with Dependency Graphs
QDMR is a meaning representation for complex questions, which decomposes...
research
∙
04/13/2021
What's in your Head? Emergent Behaviour in Multi-Task Transformer Models
The primary paradigm for multi-task training in natural language process...
research
∙
04/11/2021
Achieving Model Robustness through Discrete Adversarial Training
Discrete adversarial attacks are symbolic perturbations to a language in...
research
∙
03/17/2021
Value-aware Approximate Attention
Following the success of dot-product attention in Transformers, numerous...
research
∙
03/09/2021
BERTese: Learning to Speak to BERT
Large pre-trained language models have been shown to encode large amount...
research
∙
01/06/2021
Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies
A key limitation in current datasets for multi-hop reasoning is that the...
research
∙
01/02/2021
Few-Shot Question Answering by Pretraining Span Selection
In a number of question answering (QA) benchmarks, pretrained models hav...
research
∙
12/29/2020
Transformer Feed-Forward Layers Are Key-Value Memories
Feed-forward layers constitute two-thirds of a transformer model's param...
research
∙
10/23/2020
SmBoP: Semi-autoregressive Bottom-up Semantic Parsing
The de-facto standard decoding method for semantic parsing in recent yea...
research
∙
10/12/2020
Improving Compositional Generalization in Semantic Parsing
Generalization of models to out-of-distribution (OOD) data has captured ...
research
∙
09/30/2020
Learning Object Detection from Captions via Textual Scene Attributes
Object detection is a fundamental task in computer vision, requiring lar...
research
∙
09/23/2020
Scene Graph to Image Generation with Contextualized Object Layout Refinement
Generating high-quality images from scene graphs, that is, graphs that d...
research
∙
09/13/2020
Span-based Semantic Parsing for Compositional Generalization
Despite the success of sequence-to-sequence (seq2seq) models in semantic...
research
∙
09/02/2020
A Simple Global Neural Discourse Parser
Discourse parsing is largely dominated by greedy parsers with manually-d...
research
∙
07/01/2020
Latent Compositional Representations Improve Systematic Generalization in Grounded Question Answering
Answering questions that involve multi-step reasoning requires decomposi...
research
∙
06/11/2020
Teaching Pre-Trained Models to Systematically Reason Over Implicit Knowledge
To what extent can a neural network systematically reason over symbolic ...
research
∙
06/05/2020
GMAT: Global Memory Augmentation for Transformers
Transformer-based models have become ubiquitous in natural language proc...
research
∙
05/02/2020
Obtaining Faithful Interpretations from Compositional Neural Networks
Neural module networks (NMNs) are a popular approach for modeling compos...
research
∙
04/12/2020
Explaining Question Answering Models through Text Generation
Large pre-trained language models (LMs) have been shown to perform surpr...
research
∙
04/09/2020
Injecting Numerical Reasoning Skills into Language Models
Large pre-trained language models (LMs) are known to encode substantial ...
research
∙
04/06/2020
Evaluating the Evaluation of Diversity in Natural Language Generation
Despite growing interest in natural language generation (NLG) models tha...
research
∙
04/06/2020
Evaluating NLP Models via Contrast Sets
Standard test sets for supervised learning evaluate in-distribution gene...
research
∙
01/31/2020
Break It Down: A Question Understanding Benchmark
Understanding natural language questions entails the ability to break do...
research
∙
12/31/2019
oLMpics – On what Language Model Pre-training Captures
Recent success of pre-trained language models (LMs) has spurred widespre...
research
∙
10/05/2019
On the Limits of Learning to Actively Learn Semantic Representations
One of the goals of natural language understanding is to develop models ...
research
∙
09/25/2019