research
∙
08/23/2023
Curriculum Learning with Adam: The Devil Is in the Wrong Details
Curriculum learning (CL) posits that machine learning models – similar t...
share
research
∙
05/19/2023
Evaluating task understanding through multilingual consistency: A ChatGPT case study
At the staggering pace with which the capabilities of large language mod...
research
∙
10/23/2022
The Curious Case of Absolute Position Embeddings
Transformer language models encode the notion of word order using positi...
research
∙
10/04/2022
Text Characterization Toolkit
In NLP, models are usually evaluated by reporting single-number performa...
research
∙
12/14/2021
Towards Interactive Language Modeling
Interaction between caregivers and children plays a critical role in hum...
research
∙
12/13/2021
Sparse Interventions in Language Models with Differentiable Masking
There has been a lot of interest in understanding what information is ca...
research
∙
10/14/2021
Causal Transformers Perform Below Chance on Recursive Nested Constructions, Unlike Humans
Recursive processing is considered a hallmark of human linguistic abilit...
research
∙
10/06/2021
How BPE Affects Memorization in Transformers
Training data memorization in NLP can both be beneficial (e.g., closed-b...
research
∙
08/12/2021
The paradox of the compositionality of natural language: a neural machine translation case study
Moving towards human-like linguistic performance is often argued to requ...
research
∙
05/28/2021
Language Models Use Monotonicity to Assess NPI Licensing
We investigate the semantic knowledge of language models (LMs), focusing...
research
∙
04/26/2021
Attention vs non-attention for a Shapley-based explanation method
The field of explainable AI has recently seen an explosion in the number...
research
∙
04/14/2021
Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little
A possible explanation for the impressive performance of masked language...
research
∙
01/27/2021
Language Modelling as a Multi-Task Problem
In this paper, we propose to study language modelling as a multi-task pr...
research
∙
10/05/2020
The Grammar of Emergent Languages
In this paper, we consider the syntactic properties of languages emerged...
research
∙
06/19/2020
Exploring Processing of Nested Dependencies in Neural-Network Language Models and Humans
Recursive processing in sentence comprehension is considered a hallmark ...
research
∙
04/08/2020
Internal and external pressures on language emergence: least effort, object constancy and frequency
In previous work, artificial agents were shown to achieve almost perfect...
research
∙
01/10/2020
Co-evolution of language and agents in referential games
Referential games offer a grounded learning environment for neural agent...
research
∙
11/10/2019
Location Attention for Extrapolation to Longer Sequences
Neural networks are surprisingly good at interpolating and perform remar...
research
∙
09/19/2019
Analysing Neural Language Models: Contextual Decomposition Reveals Default Reasoning in Number and Gender Assignment
Extensive research has recently shown that recurrent neural language mod...
research
∙
08/22/2019
The compositionality of neural networks: integrating symbolism and connectionism
Despite a multitude of empirical studies, little consensus exists on whe...
research
∙
06/07/2019
Assessing incrementality in sequence-to-sequence models
Since their inception, encoder-decoder models have successfully been app...
research
∙
06/04/2019
On the Realization of Compositionality in Neural Networks
We present a detailed comparison of two types of sequence to sequence mo...
research
∙
06/04/2019
Transcoding compositionally: using attention to find more generalizable solutions
While sequence-to-sequence models have shown remarkable generalization p...
research
∙
03/18/2019
The emergence of number and syntax units in LSTM language models
Recent work has shown that LSTMs trained on a generic language modeling ...
research
∙
01/16/2019
Formal models of Structure Building in Music, Language and Animal Songs
Human language, music and a variety of animal vocalisations constitute w...
research
∙
09/17/2018
The Fast and the Flexible: training neural networks to learn to follow instructions from small data
Learning to follow human instructions is a challenging task because whil...
research
∙
08/31/2018
Do Language Models Understand Anything? On the Ability of LSTMs to Understand Negative Polarity Items
In this paper, we attempt to link the inner workings of a neural languag...
research
∙
08/28/2018
Analysing the potential of seq-to-seq models for incremental interpretation in task-oriented dialogue
We investigate how encoder-decoder models trained on a synthetic dataset...
research
∙
08/24/2018
Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Information
How do neural language models keep track of number agreement between sub...
research
∙
05/20/2018
Learning compositionally through attentive guidance
In this paper, we introduce Attentive Guidance (AG), a new mechanism to ...
research
∙
11/28/2017