On-Device Document Classification using multimodal features

01/06/2021
by   Sugam Garg, et al.
0

From small screenshots to large videos, documents take up a bulk of space in a modern smartphone. Documents in a phone can accumulate from various sources, and with the high storage capacity of mobiles, hundreds of documents are accumulated in a short period. However, searching or managing documents remains an onerous task, since most search methods depend on meta-information or only text in a document. In this paper, we showcase that a single modality is insufficient for classification and present a novel pipeline to classify documents on-device, thus preventing any private user data transfer to server. For this task, we integrate an open-source library for Optical Character Recognition (OCR) and our novel model architecture in the pipeline. We optimise the model for size, a necessary metric for on-device inference. We benchmark our classification model with a standard multimodal dataset FOOD-101 and showcase competitive results with the previous State of the Art with 30 compression.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/21/2023

bbOCR: An Open-source Multi-domain OCR Pipeline for Bengali Documents

Despite the existence of numerous Optical Character Recognition (OCR) to...
research
12/04/2020

On-Device Sentence Similarity for SMS Dataset

Determining the sentence similarity between Short Message Service (SMS) ...
research
06/27/2021

Deep Learning for Technical Document Classification

In large technology companies, the requirements for managing and organiz...
research
08/06/2021

Lights, Camera, Action! A Framework to Improve NLP Accuracy over OCR documents

Document digitization is essential for the digital transformation of our...
research
10/07/2020

VisualWordGrid: Information Extraction From Scanned Documents Using A Multimodal Approach

We introduce a novel approach for scanned document representation to per...
research
01/24/2023

Sherlock in OSS: A Novel Approach of Content-Based Searching in Object Storage System

Object Storage Systems (OSS) inside a cloud promise scalability, durabil...
research
12/22/2021

Adaptive Beam Search to Enhance On-device Abstractive Summarization

We receive several essential updates on our smartphones in the form of S...

Please sign up or login with your details

Forgot password? Click here to reset