research
∙
07/31/2023
Structural Transfer Learning in NL-to-Bash Semantic Parsers
Large-scale pre-training has made progress in many fields of natural lan...
share
research
∙
06/05/2023
On "Scientific Debt" in NLP: A Case for More Rigour in Language Model Pre-Training Research
This evidence-based position paper critiques current research practices ...
research
∙
05/30/2023
Intriguing Properties of Quantization at Scale
Emergent properties have been widely adopted as a term to describe behav...
research
∙
11/22/2022
Simplicity Bias in Transformers and their Ability to Learn Sparse Boolean Functions
Despite the widespread success of Transformers on NLP tasks, recent work...
research
∙
10/21/2022
Augmenting Multi-Turn Text-to-SQL Datasets with Self-Play
The task of context-dependent text-to-SQL aims to convert multi-turn use...
research
∙
05/24/2022
Rethinking Evaluation Practices in Visual Question Answering: A Case Study on Out-of-Distribution Generalization
Vision-and-language (V L) models pretrained on large-scale multimodal ...
research
∙
05/23/2022
StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Models
Knowledge and language understanding of models evaluated through questio...
research
∙
03/14/2022
Revisiting the Compositional Generalization Abilities of Neural Sequence Models
Compositional generalization is a fundamental trait in humans, allowing ...
research
∙
03/01/2022
Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scale
Transformer language models that are trained on vast amounts of data hav...
research
∙
01/24/2022
Relational Memory Augmented Language Models
We present a memory-augmented approach to condition an autoregressive la...
research
∙
10/31/2021
A Systematic Investigation of Commonsense Understanding in Large Language Models
Large language models have shown impressive performance on many natural ...
research
∙
03/18/2021
Pretraining the Noisy Channel Model for Task-Oriented Dialogue
Direct decoding for task-oriented dialogue is known to suffer from the e...
research
∙
02/03/2021
Pitfalls of Static Language Modelling
Our world is open-ended, non-stationary and constantly evolving; thus wh...
research
∙
12/01/2020
Mutual Information Constraints for Monte-Carlo Objectives
A common failure mode of density models trained as variational autoencod...
research
∙
09/23/2020
The Struggles of Feature-Based Explanations: Shapley Values vs. Minimal Sufficient Subsets
For neural models to garner widespread public trust and ensure fairness,...
research
∙
05/27/2020
Syntactic Structure Distillation Pretraining For Bidirectional Encoders
Textual representation learners trained on large amounts of data have ac...
research
∙
05/07/2020
Learning to Segment Actions from Observation and Narration
We apply a generative segmental model of task structure, guided by narra...
research
∙
03/16/2020
A Survey on Contextual Embeddings
Contextual embeddings, such as ELMo and BERT, move beyond global word re...
research
∙
03/11/2020
Visual Grounding in Video for Unsupervised Word Translation
There are thousands of actively spoken languages on Earth, but a single ...
research
∙
01/29/2020
Learning Robust and Multilingual Speech Representations
Unsupervised speech representation learning has shown remarkable success...
research
∙
10/01/2019
Putting Machine Translation in Context with the Noisy Channel Model
We show that Bayes' rule provides a compelling mechanism for controlling...
research
∙
09/20/2019
A Critical Analysis of Biased Parsers in Unsupervised Parsing
A series of recent papers has used a parsing algorithm due to Shen et al...
research
∙
09/04/2019
Mogrifier LSTM
Many advances in Natural Language Processing have been based upon more e...
research
∙
08/21/2019
WikiCREM: A Large Unsupervised Corpus for Coreference Resolution
Pronoun resolution is a major area of natural language understanding. Ho...
research
∙
06/14/2019
Scalable Syntax-Aware Language Models Using Knowledge Distillation
Prior work has shown that, on small amounts of training data, syntactic ...
research
∙
01/31/2019
Learning and Evaluating General Linguistic Intelligence
We define general linguistic intelligence as the ability to reuse previo...
research
∙
12/04/2018
e-SNLI: Natural Language Inference with Natural Language Explanations
In order for machine learning to garner widespread public adoption, mode...
research
∙
12/04/2018
e-SNLI: Natural Language Inference withNatural Language Explanations
In order for machine learning to garner widespread public adoption, mode...
research
∙
11/27/2018
Learning with Stochastic Guidance for Navigation
Due to the sparse rewards and high degree of environment variation, rein...
research
∙
11/26/2018
Sentence Encoding with Tree-constrained Relation Networks
The meaning of a sentence is a function of the relations that hold betwe...
research
∙
11/23/2018
Unsupervised Word Discovery with Segmental Neural Language Models
We propose a segmental neural language model that combines the represent...
research
∙
10/04/2018
Transferring Physical Motion Between Domains for Neural Inertial Tracking
Inertial information processing plays a pivotal role in ego-motion aware...
research
∙
08/01/2018
Neural Arithmetic Logic Units
Neural networks can learn to represent and manipulate numerical informat...
research
∙
07/04/2018
Encoding Spatial Relations from Natural Language
Natural language processing has made significant inroads into learning t...
research
∙
05/23/2018
Pushing the bounds of dropout
We show that dropout training is best understood as performing MAP estim...
research
∙
12/19/2017
The NarrativeQA Reading Comprehension Challenge
Reading comprehension (RC)---in contrast to information retrieval---requ...
research
∙
10/26/2017
Understanding Grounded Language Learning Agents
Neural network-based systems can now learn to locate the referents of wo...
research
∙
07/18/2017
On the State of the Art of Evaluation in Neural Language Models
Ongoing innovations in recurrent neural network architectures have provi...
research
∙
06/20/2017
Grounded Language Learning in a Simulated 3D World
We are increasingly surrounded by artificially intelligent technology th...
research
∙
06/01/2017
Discovering Discrete Latent Topics with Neural Variational Inference
Topic models have been widely explored as probabilistic generative model...
research
∙
05/29/2017
Latent Intention Dialogue Models
Developing a dialogue agent that is capable of making autonomous decisio...
research
∙
05/11/2017
Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problems
Solving algebraic word problems requires executing a series of arithmeti...
research
∙
04/24/2017
Robust Incremental Neural Semantic Graph Parsing
Parsing sentences to linguistically-expressive semantic representations ...
research
∙
04/23/2017
Learning to Create and Reuse Words in Open-Vocabulary Neural Language Modeling
Fixed-vocabulary language models fail to account for one of the most cha...
research
∙
03/06/2017
Generative and Discriminative Text Classification with Recurrent Neural Networks
We empirically characterize the performance of discriminative and genera...
research
∙
11/28/2016
Learning to Compose Words into Sentences with Reinforcement Learning
We use reinforcement learning to learn tree-structured neural networks f...
research
∙
11/08/2016
The Neural Noisy Channel
We formulate sequence to sequence transduction as a noisy channel decodi...
research
∙
11/05/2016
Reference-Aware Language Models
We propose a general class of language models that treat reference as an...
research
∙
09/29/2016
Semantic Parsing with Semi-Supervised Sequential Autoencoders
We present a novel semi-supervised approach for sequence transduction an...
research
∙
09/26/2016