Towards Interactive Language Modeling

12/14/2021
by   Maartje ter Hoeve, et al.
0

Interaction between caregivers and children plays a critical role in human language acquisition and development. Given this observation, it is remarkable that explicit interaction plays little to no role in artificial language modeling – which also targets the acquisition of human language, yet by artificial models. Moreover, an interactive approach to language modeling has the potential to make language models substantially more versatile and to considerably impact downstream applications. Motivated by these considerations, we pioneer the space of interactive language modeling. As a first contribution we present a road map in which we detail the steps that need to be taken towards interactive language modeling. We then lead by example and take the first steps on this road map, showing the initial feasibility of our approach. As such, this work aims to be the start of a larger research agenda on interactive language modeling.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/26/2019

Think Again Networks, the Delta Loss, and an Application in Language Modeling

This short paper introduces an abstraction called Think Again Networks (...
research
01/26/2017

emLam -- a Hungarian Language Modeling baseline

This paper aims to make up for the lack of documented baselines for Hung...
research
06/15/2022

DIRECTOR: Generator-Classifiers For Supervised Language Modeling

Current language models achieve low perplexity but their resulting gener...
research
05/10/2022

Human Language Modeling

Natural language is generated by people, yet traditional language modeli...
research
10/03/2022

The Effectiveness of Masked Language Modeling and Adapters for Factual Knowledge Injection

This paper studies the problem of injecting factual knowledge into large...
research
10/25/2021

No News is Good News: A Critique of the One Billion Word Benchmark

The One Billion Word Benchmark is a dataset derived from the WMT 2011 Ne...
research
11/22/2022

Converting OpenStreetMap Data to Road Networks for Downstream Applications

We study how to convert OpenStreetMap data to road networks for downstre...

Please sign up or login with your details

Forgot password? Click here to reset