Modeling Semantic Compositionality with Sememe Knowledge

07/10/2019 ∙ by Fanchao Qi, et al. ∙ HUAWEI Technologies Co., Ltd. Beihang University Tsinghua University 0

Semantic compositionality (SC) refers to the phenomenon that the meaning of a complex linguistic unit can be composed of the meanings of its constituents. Most related works focus on using complicated compositionality functions to model SC while few works consider external knowledge in models. In this paper, we verify the effectiveness of sememes, the minimum semantic units of human languages, in modeling SC by a confirmatory experiment. Furthermore, we make the first attempt to incorporate sememe knowledge into SC models, and employ the sememeincorporated models in learning representations of multiword expressions, a typical task of SC. In experiments, we implement our models by incorporating knowledge from a famous sememe knowledge base HowNet and perform both intrinsic and extrinsic evaluations. Experimental results show that our models achieve significant performance boost as compared to the baseline methods without considering sememe knowledge. We further conduct quantitative analysis and case studies to demonstrate the effectiveness of applying sememe knowledge in modeling SC. All the code and data of this paper can be obtained on



There are no comments yet.


page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Semantic compositionality (SC) is defined as the linguistic phenomenon that the meaning of a syntactically complex unit is a function of meanings of the complex unit’s constituents and their combination rule (Pelletier1994). Some linguists regard SC as the fundamental truth of semantics (Pelletier2016). In the field of NLP, SC has proved effective in many tasks including language modeling (Mitchell2009)

, sentiment analysis

(maas2011learning; Socher2013), syntactic parsing (socher2013parsing), etc.

Most literature on SC pays attention to using vector-based distributional models of semantics to learn representations of multiword expressions (MWEs), i.e., embeddings of phrases or compounds.

Mitchell2008 conduct a pioneering work in which they introduce a general framework to formulate this task:


where is the compositionality function, p denotes the embedding of an MWE, and represent the embeddings of the MWE’s two constituents, stands for the combination rule and refers to the additional knowledge which is needed to construct the semantics of the MWE.

Among the proposed approaches for this task, most of them ignore and , centering on reforming compositionality function (baroni2010nouns; grefenstette2011experimental; Socher2012; Socher2013). Some try to integrate combination rule into SC models (Blacoe2012; Zhao2015; Weir2016; Kober2016). Few works consider external knowledge . Zhu2016 try to incorporate task-specific knowledge into an LSTM model for sentence-level SC. As far as we know, however, no previous work attempts to use general knowledge in modeling SC.

SCD Our Computation Formulae Examples
MWEs and Constituents Sememes
3 农民起义(peasant uprising) 事情fact,职位occupation,政politics,暴动uprise,人human,农agricultural
农民  (peasant) 职位occupation,human,agricultural
  起义(uprising) 暴动uprise,事情fact,politics
2 几何图形(geometric figure) 数学math,图像image
几何  (geometry; how much) 数学math,知识knowledge,疑问question,功能词funcword
  图形(figure) 图像image
1 (engage a test)考试exam,从事engage
 (deal with; echo; agree)处理handle,回应respond,同意agree,遵循obey,功能词funcword,姓surname
(quiz; check)考试exam,查check
  (draw)画draw,部件part,图像image, 文字character,表示express