Learning Contextualized Semantics from Co-occurring Terms via a Siamese Architecture

06/17/2015
by   Ubai Sandouk, et al.
0

One of the biggest challenges in Multimedia information retrieval and understanding is to bridge the semantic gap by properly modeling concept semantics in context. The presence of out of vocabulary (OOV) concepts exacerbates this difficulty. To address the semantic gap issues, we formulate a problem on learning contextualized semantics from descriptive terms and propose a novel Siamese architecture to model the contextualized semantics from descriptive terms. By means of pattern aggregation and probabilistic topic models, our Siamese architecture captures contextualized semantics from the co-occurring descriptive terms via unsupervised learning, which leads to a concept embedding space of the terms in context. Furthermore, the co-occurring OOV concepts can be easily represented in the learnt concept embedding space. The main properties of the concept embedding space are demonstrated via visualization. Using various settings in semantic priming, we have carried out a thorough evaluation by comparing our approach to a number of state-of-the-art methods on six annotation corpora in different domains, i.e., MagTag5K, CAL500 and Million Song Dataset in the music domain as well as Corel5K, LabelMe and SUNDatabase in the image domain. Experimental results on semantic priming suggest that our approach outperforms those state-of-the-art methods considerably in various aspects.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/10/2015

Measuring Semantic Relatedness using Mined Semantic Analysis

Mined Semantic Analysis (MSA) is a novel concept space model which emplo...
research
04/16/2020

Multiple Visual-Semantic Embedding for Video Retrieval from Query Sentence

Visual-semantic embedding aims to learn a joint embedding space where re...
research
10/12/2020

Evaluation of Siamese Networks for Semantic Code Search

With the increase in the number of open repositories and discussion foru...
research
12/11/2017

Learning Modality-Invariant Representations for Speech and Images

In this paper, we explore the unsupervised learning of a semantic embedd...
research
01/02/2019

Action2Vec: A Crossmodal Embedding Approach to Action Learning

We describe a novel cross-modal embedding space for actions, named Actio...
research
11/25/2019

Bridging the Gap between Semantics and Multimedia Processing

In this paper, we give an overview of the semantic gap problem in multim...
research
01/30/2021

Fundamentals of Semantic Numeration Systems. Can the Context be Calculated?

This work is the first to propose the concept of a semantic numeration s...

Please sign up or login with your details

Forgot password? Click here to reset