Tag Prediction at Flickr: a View from the Darkroom

12/06/2016
by   Kofi Boakye, et al.
0

Automated photo tagging has established itself as one of the most compelling applications of deep learning. While deep convolutional neural networks have repeatedly demonstrated top performance on standard datasets for classification, there are a number of often overlooked but important considerations when deploying this technology in a real-world scenario. In this paper, we present our efforts in developing a large-scale photo tagging system for Flickr photo search. We discuss topics including how to 1) select the tags that matter most to our users; 2) develop lightweight, high-performance models for tag prediction; and 3) leverage the power of large amounts of noisy data for training. Our results demonstrate that, for real-world datasets, training exclusively with this noisy data yields performance on par with the standard paradigm of first pre-training on clean data and then fine-tuning. In addition, we observe that the models trained with user-generated data can yield better fine-tuning results when a small amount of clean data is available. As such, we advocate for the approach of harnessing user-generated data in large-scale systems.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/02/2014

A Data-Driven Approach for Tag Refinement and Localization in Web Videos

Tagging of visual content is becoming more and more widespread as web-ba...
research
01/06/2017

Learning From Noisy Large-Scale Datasets With Minimal Supervision

We present an approach to effectively use millions of images with noisy ...
research
02/07/2020

Improving the Adversarial Robustness of Transfer Learning via Noisy Feature Distillation

Fine-tuning through knowledge transfer from a pre-trained model on a lar...
research
03/29/2023

RetClean: Retrieval-Based Data Cleaning Using Foundation Models and Data Lakes

Can foundation models (such as ChatGPT) clean your data? In this proposa...
research
12/11/2021

Auto-Tag: Tagging-Data-By-Example in Data Lakes

As data lakes become increasingly popular in large enterprises today, th...
research
04/01/2020

Adversarial Learning for Personalized Tag Recommendation

We have recently seen great progress in image classification due to the ...

Please sign up or login with your details

Forgot password? Click here to reset