b'Adam Roberts'

research

∙ 05/22/2023

A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, Toxicity

Pretraining is the preliminary and fundamental step in developing capabl...

0 Shayne Longpre, et al. ∙

research

∙ 04/18/2023

UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretraining

Pretrained multilingual large language models have typically used heuris...

0 Hyung Won Chung, et al. ∙

research

∙ 01/31/2023

The Flan Collection: Designing Data and Methods for Effective Instruction Tuning

We study the design decisions of publicly available instruction tuning m...

0 Shayne Longpre, et al. ∙

research

∙ 01/30/2023

SingSong: Generating musical accompaniments from singing

We present SingSong, a system that generates instrumental music to accom...

4 Chris Donahue, et al. ∙

research

∙ 01/26/2023

MusicLM: Generating Music From Text

We introduce MusicLM, a model generating high-fidelity music from text d...

0 Andrea Agostinelli, et al. ∙

research

∙ 12/20/2022

Character-Aware Models Improve Visual Text Rendering

Current image generation models struggle to reliably produce well-formed...

0 Rosanne Liu, et al. ∙

research

∙ 11/17/2022

VeLO: Training Versatile Learned Optimizers by Scaling Up

While deep learning models have replaced hand-designed features across m...

0 Luke Metz, et al. ∙

research

∙ 11/15/2022

Large Language Models Struggle to Learn Long-Tail Knowledge

The internet contains a wealth of knowledge – from the birthdays of hist...

0 Nikhil Kandpal, et al. ∙

research

∙ 11/03/2022

Crosslingual Generalization through Multitask Finetuning

Multitask prompted finetuning (MTF) has been shown to help large languag...

0 Niklas Muennighoff, et al. ∙

research

∙ 06/11/2022

Multi-instrument Music Synthesis with Spectrogram Diffusion

An ideal music synthesizer should be both interactive and expressive, ge...

6 Curtis Hawthorne, et al. ∙

research

∙ 04/12/2022

What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?

Large pretrained Transformer language models have been shown to exhibit ...

11 Thomas Wang, et al. ∙

research

∙ 04/05/2022

PaLM: Scaling Language Modeling with Pathways

Large language models have been shown to achieve remarkable performance ...

6 Aakanksha Chowdhery, et al. ∙

research

∙ 03/31/2022

Scaling Up Models and Data with and

Recent neural network-based language models have benefited greatly from ...

8 Adam Roberts, et al. ∙

research

∙ 05/28/2021

ByT5: Towards a token-free future with pre-trained byte-to-byte models

Most widely-used pre-trained language models operate on sequences of tok...

0 Linting Xue, et al. ∙

research

∙ 02/23/2021

Do Transformer Modifications Transfer Across Implementations and Applications?

The research community has proposed copious modifications to the Transfo...

10 Sharan Narang, et al. ∙

research

∙ 01/01/2021

NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned

We review the EfficientQA competition from NeurIPS 2020. The competition...

15 Sewon Min, et al. ∙

research

∙ 12/14/2020

Extracting Training Data from Large Language Models

It has become common to publish large (billion parameter) language model...

0 Nicholas Carlini, et al. ∙

research

∙ 10/22/2020

mT5: A massively multilingual pre-trained text-to-text transformer

The recent "Text-to-Text Transfer Transformer" (T5) leveraged a unified ...

0 Linting Xue, et al. ∙

research

∙ 04/30/2020

WT5?! Training Text-to-Text Models to Explain their Predictions

Neural networks have recently achieved human-level performance on variou...

0 Sharan Narang, et al. ∙

research

∙ 02/10/2020

How Much Knowledge Can You Pack Into the Parameters of a Language Model?

It has recently been observed that neural language models trained on uns...

0 Adam Roberts, et al. ∙

research

∙ 01/14/2020

DDSP: Differentiable Digital Signal Processing

Most generative models of audio directly generate samples in one of two ...

11 Jesse Engel, et al. ∙

research

∙ 10/23/2019

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Transfer learning, where a model is first pre-trained on a data-rich tas...

0 Colin Raffel, et al. ∙

research

∙ 07/14/2019

The Bach Doodle: Approachable music composition with machine learning at scale

To make music composition more approachable, we designed the first AI-po...

6 Cheng-Zhi Anna Huang, et al. ∙

research

∙ 05/14/2019

Learning to Groove with Inverse Sequence Transformations

We explore models for translating abstract musical ideas (scores, rhythm...

0 Jon Gillick, et al. ∙

research

∙ 03/18/2019

Counterpoint by Convolution

Machine learning models of music typically break up the task of composit...

0 Cheng-Zhi Anna Huang, et al. ∙

research

∙ 02/23/2019

GANSynth: Adversarial Neural Audio Synthesis

Efficient audio synthesis is an inherently difficult machine learning ta...

20 Jesse Engel, et al. ∙

research

∙ 10/29/2018

Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset

Generating musical audio directly with neural networks is notoriously di...

0 Curtis Hawthorne, et al. ∙

research

∙ 06/01/2018

Learning a Latent Space of Multitrack Measures

Discovering and exploring the underlying structure of multi-instrumental...

0 Ian Simon, et al. ∙

research

∙ 03/13/2018

A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music

The Variational Autoencoder (VAE) has proven to be an effective model fo...

0 Adam Roberts, et al. ∙

research

∙ 11/15/2017

Latent Constraints: Learning to Generate Conditionally from Unconditional Generative Models

Deep generative neural networks have proven effective at both conditiona...

0 Jesse Engel, et al. ∙

research

∙ 10/30/2017

Onsets and Frames: Dual-Objective Piano Transcription

We consider the problem of transcribing polyphonic piano music with an e...

0 Curtis Hawthorne, et al. ∙

research

∙ 04/05/2017

Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders

Generative models in vision have seen rapid progress due to algorithmic ...

0 Jesse Engel, et al. ∙

Adam Roberts

Featured Co-authors