Assessing the Linguistic Productivity of Unsupervised Deep Neural Networks

by   Lawrence Phillips, et al.

Increasingly, cognitive scientists have demonstrated interest in applying tools from deep learning. One use for deep learning is in language acquisition where it is useful to know if a linguistic phenomenon can be learned through domain-general means. To assess whether unsupervised deep learning is appropriate, we first pose a smaller question: Can unsupervised neural networks apply linguistic rules productively, using them in novel situations? We draw from the literature on determiner/noun productivity by training an unsupervised, autoencoder network measuring its ability to combine nouns with determiners. Our simple autoencoder creates combinations it has not previously encountered and produces a degree of overlap matching adults. While this preliminary work does not provide conclusive evidence for productivity, it warrants further investigation with more complex models. Further, this work helps lay the foundations for future collaboration between the deep learning and cognitive science communities.


page 1

page 2

page 3

page 4


Linguistic generalization and compositionality in modern artificial neural networks

In the last decade, deep artificial neural networks have achieved astoun...

Probabilistic Method of Measuring Linguistic Productivity

In this paper I propose a new way of measuring linguistic productivity t...

The relationship among research productivity, research collaboration, and their determinants

This work provides an in-depth analysis of the relation between the diff...

A Survey in Automatic Irony Processing: Linguistic, Cognitive, and Multi-X Perspectives

Irony is a ubiquitous figurative language in daily communication. Previo...

Genetic Algorithms for Evolving Deep Neural Networks

In recent years, deep learning methods applying unsupervised learning to...

Deep Learning and the Global Workspace Theory

Recent advances in deep learning have allowed Artificial Intelligence (A...

Throwing fuel on the embers: Probability or Dichotomy, Cognitive or Linguistic?

Prof. Robert Berwick's abstract for his forthcoming invited talk at the ...

Please sign up or login with your details

Forgot password? Click here to reset