DeepAI
Log In Sign Up

Context-Aware Abbreviation Expansion Using Large Language Models

05/08/2022
by   Shanqing Cai, et al.
10

Motivated by the need for accelerating text entry in augmentative and alternative communication (AAC) for people with severe motor impairments, we propose a paradigm in which phrases are abbreviated aggressively as primarily word-initial letters. Our approach is to expand the abbreviations into full-phrase options by leveraging conversation context with the power of pretrained large language models (LLMs). Through zero-shot, few-shot, and fine-tuning experiments on four public conversation datasets, we show that for replies to the initial turn of a dialog, an LLM with 64B parameters is able to exactly expand over 70 to an effective keystroke saving rate of up to about 77 expansions. Including a small amount of context in the form of a single conversation turn more than doubles abbreviation expansion accuracies compared to having no context, an effect that is more pronounced for longer phrases. Additionally, the robustness of models against typo noise can be enhanced through fine-tuning on noisy data.

READ FULL TEXT

page 1

page 2

page 3

page 4

03/09/2022

Model-Agnostic Multitask Fine-tuning for Few-shot Vision-Language Transfer Learning

Despite achieving state-of-the-art zero-shot performance, existing visio...
11/29/2022

Context-Aware Robust Fine-Tuning

Contrastive Language-Image Pre-trained (CLIP) models have zero-shot abil...
04/07/2022

Learning to Compose Soft Prompts for Compositional Zero-Shot Learning

We introduce compositional soft prompting (CSP), a parameter-efficient l...
08/01/2021

Improving Social Meaning Detection with Pragmatic Masking and Surrogate Fine-Tuning

Masked language models (MLMs) are pretrained with a denoising objective ...
09/22/2022

Prompting for a conversation: How to control a dialog model?

Dialog modelling faces a difficult trade-off. Models are trained on a la...
01/17/2023

BERT-ERC: Fine-tuning BERT is Enough for Emotion Recognition in Conversation

Previous works on emotion recognition in conversation (ERC) follow a two...