Towards a continuous modeling of natural language domains

10/28/2016
by   Sebastian Ruder, et al.
0

Humans continuously adapt their style and language to a variety of domains. However, a reliable definition of `domain' has eluded researchers thus far. Additionally, the notion of discrete domains stands in contrast to the multiplicity of heterogeneous domains that humans navigate, many of which overlap. In order to better understand the change and variation of human language, we draw on research in domain adaptation and extend the notion of discrete domains to the continuous spectrum. We propose representation learning-based models that can adapt to continuous domains and detail how these can be used to investigate variation in language. To this end, we propose to use dialogue modeling as a test bed due to its proximity to language modeling and its social component.

READ FULL TEXT

page 3

page 4

research
04/08/2020

CALM: Continuous Adaptive Learning for Language Modeling

Training large language representation models has become a standard in t...
research
07/03/2020

Continuously Indexed Domain Adaptation

Existing domain adaptation focuses on transferring knowledge between dom...
research
10/12/2016

A Paradigm for Situated and Goal-Driven Language Learning

A distinguishing property of human intelligence is the ability to flexib...
research
10/13/2022

M2D2: A Massively Multi-domain Language Modeling Dataset

We present M2D2, a fine-grained, massively multi-domain corpus for study...
research
03/17/2019

AdaGraph: Unifying Predictive and Continuous Domain Adaptation through Graphs

The ability to categorize is a cornerstone of visual intelligence, and a...
research
05/02/2019

Continuous Learning for Large-scale Personalized Domain Classification

Domain classification is the task of mapping spoken language utterances ...
research
03/30/2022

On the Road to Online Adaptation for Semantic Image Segmentation

We propose a new problem formulation and a corresponding evaluation fram...

Please sign up or login with your details

Forgot password? Click here to reset