Image Annotation using Multi-Layer Sparse Coding

05/06/2017
by   Amara Tariq, et al.
0

Automatic annotation of images with descriptive words is a challenging problem with vast applications in the areas of image search and retrieval. This problem can be viewed as a label-assignment problem by a classifier dealing with a very large set of labels, i.e., the vocabulary set. We propose a novel annotation method that employs two layers of sparse coding and performs coarse-to-fine labeling. Themes extracted from the training data are treated as coarse labels. Each theme is a set of training images that share a common subject in their visual and textual contents. Our system extracts coarse labels for training and test images without requiring any prior knowledge. Vocabulary words are the fine labels to be associated with images. Most of the annotation methods achieve low recall due to the large number of available fine labels, i.e., vocabulary words. These systems also tend to achieve high precision for highly frequent words only while relatively rare words are more important for search and retrieval purposes. Our system not only outperforms various previously proposed annotation systems, but also achieves symmetric response in terms of precision and recall. Our system scores and maintains high precision for words with a wide range of frequencies. Such behavior is achieved by intelligently reducing the number of available fine labels or words for each image based on coarse labels assigned to it.

READ FULL TEXT

page 5

page 6

page 7

page 12

page 13

research
01/31/2016

WASSUP? LOL : Characterizing Out-of-Vocabulary Words in Twitter

Language in social media is mostly driven by new words and spellings tha...
research
05/15/2017

Learning Semantics for Image Annotation

Image search and retrieval engines rely heavily on textual annotation in...
research
12/24/2020

SubICap: Towards Subword-informed Image Captioning

Existing Image Captioning (IC) systems model words as atomic units in ca...
research
03/15/2023

Finding Similar Exercises in Retrieval Manner

When students make a mistake in an exercise, they can consolidate it by ...
research
07/14/2011

Label-Specific Training Set Construction from Web Resource for Image Annotation

Recently many research efforts have been devoted to image annotation by ...
research
05/09/2022

XSTEM: An exemplar-based stemming algorithm

Stemming is the process of reducing related words to a standard form by ...
research
06/02/2022

Distilling Knowledge from Object Classification to Aesthetics Assessment

In this work, we point out that the major dilemma of image aesthetics as...

Please sign up or login with your details

Forgot password? Click here to reset