On- Device Information Extraction from Screenshots in form of tags

01/11/2020
by   Sumit Kumar, et al.
0

We propose a method to make mobile screenshots easily searchable. In this paper, we present the workflow in which we: 1) preprocessed a collection of screenshots, 2) identified script presentin image, 3) extracted unstructured text from images, 4) identifiedlanguage of the extracted text, 5) extracted keywords from the text, 6) identified tags based on image features, 7) expanded tag set by identifying related keywords, 8) inserted image tags with relevant images after ranking and indexed them to make it searchable on device. We made the pipeline which supports multiple languages and executed it on-device, which addressed privacy concerns. We developed novel architectures for components in the pipeline, optimized performance and memory for on-device computation. We observed from experimentation that the solution developed can reduce overall user effort and improve end user experience while searching, whose results are published.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 7

12/05/2020

On-Device Tag Generation for Unstructured Text

With the overwhelming transition to smart phones, storing important info...
09/07/2016

Automatic Visual Theme Discovery from Joint Image and Text Corpora

A popular approach to semantic image understanding is to manually tag im...
03/08/2019

Image Privacy Prediction Using Deep Neural Networks

Images today are increasingly shared online on social networking sites s...
04/16/2021

ScreenSeg: On-Device Screenshot Layout Analysis

We propose a novel end-to-end solution that performs a Hierarchical Layo...
12/04/2020

On-Device Sentence Similarity for SMS Dataset

Determining the sentence similarity between Short Message Service (SMS) ...
03/30/2021

FONTNET: On-Device Font Understanding and Prediction Pipeline

Fonts are one of the most basic and core design concepts. Numerous use c...
06/20/2013

Analysing Word Importance for Image Annotation

Image annotation provides several keywords automatically for a given ima...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.