Visualizing the Obvious: A Concreteness-based Ensemble Model for Noun Property Prediction

10/24/2022
by   Yue Yang, et al.
0

Neural language models encode rich knowledge about entities and their relationships which can be extracted from their representations using probing. Common properties of nouns (e.g., red strawberries, small ant) are, however, more challenging to extract compared to other types of knowledge because they are rarely explicitly stated in texts. We hypothesize this to mainly be the case for perceptual properties which are obvious to the participants in the communication. We propose to extract these properties from images and use them in an ensemble model, in order to complement the information that is extracted from language models. We consider perceptual properties to be more concrete than abstract properties (e.g., interesting, flawless). We propose to use the adjectives' concreteness score as a lever to calibrate the contribution of each source (text vs. images). We evaluate our ensemble model in a ranking task where the actual properties of a noun need to be ranked higher than other non-relevant properties. Our results show that the proposed combination of text and images greatly improves noun property prediction compared to powerful text-based language models.

READ FULL TEXT
research
07/24/2020

Named entity recognition in chemical patents using ensemble of contextual language models

Chemical patent documents describe a broad range of applications holding...
research
05/13/2022

A Property Induction Framework for Neural Language Models

To what extent can experience from language contribute to our conceptual...
research
04/15/2021

Time-Stamped Language Model: Teaching Language Models to Understand the Flow of Events

Tracking entities throughout a procedure described in a text is challeng...
research
10/06/2022

Modelling Commonsense Properties using Pre-Trained Bi-Encoders

Grasping the commonsense properties of everyday concepts is an important...
research
01/24/2022

Polyphone disambiguation and accent prediction using pre-trained language models in Japanese TTS front-end

Although end-to-end text-to-speech (TTS) models can generate natural spe...
research
09/23/2021

Transferring Knowledge from Vision to Language: How to Achieve it and how to Measure it?

Large language models are known to suffer from the hallucination problem...
research
04/13/2022

CAMERO: Consistency Regularized Ensemble of Perturbed Language Models with Weight Sharing

Model ensemble is a popular approach to produce a low-variance and well-...

Please sign up or login with your details

Forgot password? Click here to reset