research
∙
07/06/2023
Lost in the Middle: How Language Models Use Long Contexts
While recent language models have the ability to take long contexts as i...
share
research
∙
06/05/2023
Has the Machine Learning Review Process Become More Arbitrary as the Field Has Grown? The NeurIPS 2021 Consistency Experiment
We present the NeurIPS 2021 consistency experiment, a larger-scale varia...
research
∙
05/23/2023
Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training
Given the massive cost of language model pre-training, a non-trivial imp...
research
∙
04/19/2023
Evaluating Verifiability in Generative Search Engines
Generative search engines directly generate responses to user queries, a...
research
∙
03/28/2023
Foundation Models and Fair Use
Existing foundation models are trained on copyrighted material. Deployin...
research
∙
03/13/2023
FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU
The high computational and memory requirements of large language model (...
research
∙
12/19/2022
Evaluating Human-Language Model Interaction
Many real-world applications of language models (LMs), such as code auto...
research
∙
12/04/2022
Melody transcription via generative pre-training
Despite the central role that melody plays in music perception, it remai...
research
∙
11/22/2022
Retrieval-Augmented Multimodal Language Modeling
Recent multimodal models such as DALL-E and CM3 have achieved remarkable...
research
∙
11/16/2022
Holistic Evaluation of Language Models
Language models (LMs) are becoming the foundation for almost all major l...
research
∙
10/27/2022
Contrastive Decoding: Open-ended Text Generation as Optimization
Likelihood, although useful as a training loss, is a poor search objecti...
research
∙
08/01/2022
What Can Transformers Learn In-Context? A Case Study of Simple Function Classes
In-context learning refers to the ability of a model to condition on a p...
research
∙
05/27/2022
Diffusion-LM Improves Controllable Text Generation
Controlling the behavior of language models (LMs) without re-training is...
research
∙
11/03/2021
An Explanation of In-context Learning as Implicit Bayesian Inference
Large pretrained language models such as GPT-3 have the surprising abili...
research
∙
07/19/2021
Just Train Twice: Improving Group Robustness without Training Group Information
Standard training via empirical risk minimization (ERM) can produce mode...
research
∙
04/13/2021
QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering
The problem of answering questions using knowledge from pre-trained lang...
research
∙
01/01/2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation
Fine-tuning is the de facto way to leverage large pretrained language mo...
research
∙
12/14/2020
WILDS: A Benchmark of in-the-Wild Distribution Shifts
Distribution shifts can cause significant degradation in a broad range o...
research
∙
10/15/2020
RNNs can generate bounded hierarchical languages with optimal memory
Recurrent neural networks empirically generate natural language with hig...
research
∙
06/26/2019
Selection Via Proxy: Efficient Data Selection For Deep Learning
Data selection methods such as active learning and core-set selection ar...
research
∙
04/15/2019
Pun Generation with Surprise
We tackle the problem of generating a pun sentence given a pair of homop...
research
∙
11/02/2018
Stronger Data Poisoning Attacks Break Data Sanitization Defenses
Machine learning models trained on data from the outside world can be co...
research
∙
08/28/2018
Mapping Natural Language Commands to Web Elements
The web provides a rich, open-domain environment with textual, structura...
research
∙
04/17/2018
Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer
We consider the task of text attribute transfer: transforming a sentence...
research
∙
02/11/2018
Learning a SAT Solver from Single-Bit Supervision
We present NeuroSAT, a message passing neural network that learns to sol...
research
∙
11/07/2017
Learning Overcomplete HMMs
We study the problem of learning overcomplete HMMs---those that have man...
research
∙
06/16/2016
