Love Thy Neighbors: Image Annotation by Exploiting Image Metadata

08/30/2015
by   Justin Johnson, et al.
1

Some images that are difficult to recognize on their own may become more clear in the context of a neighborhood of related images with similar social-network metadata. We build on this intuition to improve multilabel image annotation. Our model uses image metadata nonparametrically to generate neighborhoods of related images using Jaccard similarities, then uses a deep neural network to blend visual information from the image and its neighbors. Prior work typically models image metadata parametrically, in contrast, our nonparametric treatment allows our model to perform well even when the vocabulary of metadata changes between training and testing. We perform comprehensive experiments on the NUS-WIDE dataset, where we show that our model outperforms state-of-the-art methods for multilabel image annotation even when our model is forced to generalize to new types of metadata.

READ FULL TEXT

page 1

page 4

page 5

research
10/13/2019

A CNN-RNN Framework for Image Annotation from Visual Cues and Social Network Metadata

Images represent a commonly used form of visual communication among peop...
research
07/16/2012

Image Labeling on a Network: Using Social-Network Metadata for Image Classification

Large-scale image retrieval benchmarks invariably consist of images from...
research
02/06/2020

Context Aware Image Annotation in Active Learning

Image annotation for active learning is labor-intensive. Various automat...
research
12/09/2021

How Universal is Genre in Universal Dependencies?

This work provides the first in-depth analysis of genre in Universal Dep...
research
04/01/2020

Transfer Learning of Photometric Phenotypes in Agriculture Using Metadata

Estimation of photometric plant phenotypes (e.g., hue, shine, chroma) in...
research
04/25/2011

Bayesian approach for near-duplicate image detection

In this paper we propose a bayesian approach for near-duplicate image de...

Please sign up or login with your details

Forgot password? Click here to reset