-
Why Do Masked Neural Language Models Still Need Common Sense Knowledge?
Currently, contextualized word representations are learned by intricate ...
11/08/2019 ∙ by Sunjae Kwon, et al. ∙
0
∙
share
read it
-
Improving Neural Story Generation by Targeted Common Sense Grounding
Stories generated with neural language models have shown promise in gram...
08/26/2019 ∙ by Huanru Henry Mao, et al. ∙
0
∙
share
read it
-
Multi-Sense Language Modelling
The effectiveness of a language model is influenced by its token represe...
12/10/2020 ∙ by Andrea Lekkas, et al. ∙
0
∙
share
read it
-
Linguistically-Informed Transformations (LIT): A Method forAutomatically Generating Contrast Sets
Although large-scale pretrained language models, such as BERT and RoBERT...
10/16/2020 ∙ by Chuanrong Li, et al. ∙
0
∙
share
read it
-
The Sensitivity of Language Models and Humans to Winograd Schema Perturbations
Large-scale pretrained language models are the major driving force behin...
05/04/2020 ∙ by Mostafa Abdou, et al. ∙
0
∙
share
read it
-
Linguistic dependencies and statistical dependence
What is the relationship between linguistic dependencies and statistical...
04/18/2021 ∙ by Jacob Louis Hoover, et al. ∙
0
∙
share
read it
-
Duluth at SemEval-2020 Task 7: Using Surprise as a Key to Unlock Humorous Headlines
We use pretrained transformer-based language models in SemEval-2020 Task...
09/06/2020 ∙ by Shuning Jin, et al. ∙
0
∙
share
read it
Do Language Embeddings Capture Scales?
10/11/2020 ∙ by Xikun Zhang, et al. ∙
0
∙
share
Pretrained Language Models (LMs) have been shown to possess significant linguistic, common sense, and factual knowledge. One form of knowledge that has not been studied yet in this context is information about the scalar magnitudes of objects. We show that pretrained language models capture a significant amount of this information but are short of the capability required for general common-sense reasoning. We identify contextual information in pre-training and numeracy as two key factors affecting their performance and show that a simple method of canonicalizing numbers can have a significant effect on the results.
READ FULL TEXT
Comments
There are no comments yet.