Language Models are Few-Shot Butlers

04/16/2021
by   Vincent Micheli, et al.
0

Pretrained language models demonstrate strong performance in most NLP tasks when fine-tuned on small task-specific datasets. Hence, these autoregressive models constitute ideal agents to operate in text-based environments where language understanding and generative capabilities are essential. Nonetheless, collecting expert demonstrations in such environments is a time-consuming endeavour. We introduce a two-stage procedure to learn from a small set of demonstrations and further improve by interacting with an environment. We show that language models fine-tuned with only 1.2 a simple reinforcement learning algorithm achieve a 51 success rate over existing methods in the ALFWorld environment.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/18/2018

Fine-tuned Language Models for Text Classification

Transfer learning has revolutionized computer vision, but existing appro...
research
05/22/2023

Small Language Models Improve Giants by Rewriting Their Outputs

Large language models (LLMs) have demonstrated impressive few-shot learn...
research
08/21/2023

SeqGPT: An Out-of-the-box Large Language Model for Open Domain Sequence Understanding

Large language models (LLMs) have shown impressive ability for open-doma...
research
12/27/2021

What do Large Language Models Learn about Scripts?

Script Knowledge (Schank and Abelson, 1975) has long been recognized as ...
research
06/28/2020

Progressive Generation of Long Text

Large-scale language models pretrained on massive corpora of text, such ...
research
09/09/2023

EPA: Easy Prompt Augmentation on Large Language Models via Multiple Sources and Multiple Targets

Large language models (LLMs) have shown promising performance on various...
research
07/09/2023

Assessing the efficacy of large language models in generating accurate teacher responses

(Tack et al., 2023) organized the shared task hosted by the 18th Worksho...

Please sign up or login with your details

Forgot password? Click here to reset