Log In Sign Up

On-Device Document Classification using multimodal features

by   Sugam Garg, et al.

From small screenshots to large videos, documents take up a bulk of space in a modern smartphone. Documents in a phone can accumulate from various sources, and with the high storage capacity of mobiles, hundreds of documents are accumulated in a short period. However, searching or managing documents remains an onerous task, since most search methods depend on meta-information or only text in a document. In this paper, we showcase that a single modality is insufficient for classification and present a novel pipeline to classify documents on-device, thus preventing any private user data transfer to server. For this task, we integrate an open-source library for Optical Character Recognition (OCR) and our novel model architecture in the pipeline. We optimise the model for size, a necessary metric for on-device inference. We benchmark our classification model with a standard multimodal dataset FOOD-101 and showcase competitive results with the previous State of the Art with 30 compression.


page 1

page 2

page 3

page 4


Deep Learning for Technical Document Classification

In large technology companies, the requirements for managing and organiz...

On-Device Sentence Similarity for SMS Dataset

Determining the sentence similarity between Short Message Service (SMS) ...

Lights, Camera, Action! A Framework to Improve NLP Accuracy over OCR documents

Document digitization is essential for the digital transformation of our...

VisualWordGrid: Information Extraction From Scanned Documents Using A Multimodal Approach

We introduce a novel approach for scanned document representation to per...

Can You Read Me Now? Content Aware Rectification using Angle Supervision

The ubiquity of smartphone cameras has led to more and more documents be...

Adaptive Beam Search to Enhance On-device Abstractive Summarization

We receive several essential updates on our smartphones in the form of S...

TIDF-DLPM: Term and Inverse Document Frequency based Data Leakage Prevention Model

Confidentiality of the data is being endangered as it has been categoriz...