Modeling Emergent Lexicon Formation with a Self-Reinforcing Stochastic Process

06/22/2022
by   Brendon Boldt, et al.
0

We introduce FiLex, a self-reinforcing stochastic process which models finite lexicons in emergent language experiments. The central property of FiLex is that it is a self-reinforcing process, parallel to the intuition that the more a word is used in a language, the more its use will continue. As a theoretical model, FiLex serves as a way to both explain and predict the behavior of the emergent language system. We empirically test FiLex's ability to capture the relationship between the emergent language's hyperparameters and the lexicon's Shannon entropy.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/28/2022

Mathematically Modeling the Lexicon Entropy of Emergent Language

We formulate a stochastic process, FiLex, as a mathematical model of lex...
research
06/04/2018

Self-Normalization Properties of Language Modeling

Self-normalizing discriminative models approximate the normalized probab...
research
02/25/2016

Automated Word Prediction in Bangla Language Using Stochastic Language Models

Word completion and word prediction are two important phenomena in typin...
research
02/16/2021

Revisiting Language Encoding in Learning Multilingual Representations

Transformer has demonstrated its great power to learn contextual word re...
research
09/29/2020

Modeling partitions of individuals

Despite the central role of self-assembled groups in animal and human so...
research
02/14/2023

The Stable Entropy Hypothesis and Entropy-Aware Decoding: An Analysis and Algorithm for Robust Natural Language Generation

State-of-the-art language generation models can degenerate when applied ...
research
09/13/2016

Self-Sustaining Iterated Learning

An important result from psycholinguistics (Griffiths & Kalish, 2005) st...

Please sign up or login with your details

Forgot password? Click here to reset