How clever is the FiLM model, and how clever can it be?

09/09/2018
by   Alexander Kuhnle, et al.
0

The FiLM model achieves close-to-perfect performance on the diagnostic CLEVR dataset and is distinguished from other such models by having a comparatively simple and easily transferable architecture. In this paper, we investigate in more detail the ability of FiLM to learn various linguistic constructions. Our main results show that (a) FiLM is not able to learn relational statements straight away except for very simple instances, (b) training on a broader set of instances as well as pretraining on simpler instance types can help alleviate these learning difficulties, (c) mixing is less robust than pretraining and very sensitive to the compositional structure of the dataset. Overall, our results suggest that the approach of big all-encompassing datasets and the paradigm of "the effectiveness of data" may have fundamental limitations.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 2

page 7

02/15/2022

Impact of Pretraining Term Frequencies on Few-Shot Reasoning

Pretrained Language Models (LMs) have demonstrated ability to perform nu...
04/19/2020

Are we pretraining it right? Digging deeper into visio-linguistic pretraining

Numerous recent works have proposed pretraining generic visio-linguistic...
10/06/2021

Pretraining Reinforcement Learning: Sharpening the Axe Before Cutting the Tree

Pretraining is a common technique in deep learning for increasing perfor...
10/11/2020

Learning Which Features Matter: RoBERTa Acquires a Preference for Linguistic Generalizations (Eventually)

One reason pretraining on self-supervised linguistic tasks is effective ...
06/25/2021

On the Robustness of Pretraining and Self-Supervision for a Deep Learning-based Analysis of Diabetic Retinopathy

There is an increasing number of medical use-cases where classification ...
05/12/2021

Improving Code Autocompletion with Transfer Learning

Software language models have achieved promising results predicting code...
05/10/2022

SuMe: A Dataset Towards Summarizing Biomedical Mechanisms

Can language models read biomedical texts and explain the biomedical mec...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.