HYCEDIS: HYbrid Confidence Engine for Deep Document Intelligence System

06/01/2022
by   Bao-Sinh Nguyen, et al.
0

Measuring the confidence of AI models is critical for safely deploying AI in real-world industrial systems. One important application of confidence measurement is information extraction from scanned documents. However, there exists no solution to provide reliable confidence score for current state-of-the-art deep-learning-based information extractors. In this paper, we propose a complete and novel architecture to measure confidence of current deep learning models in document information extraction task. Our architecture consists of a Multi-modal Conformal Predictor and a Variational Cluster-oriented Anomaly Detector, trained to faithfully estimate its confidence on its outputs without the need of host models modification. We evaluate our architecture on real-wold datasets, not only outperforming competing confidence estimators by a huge margin but also demonstrating generalization ability to out-of-distribution data.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/30/2020

Towards a Multi-modal, Multi-task Learning based Pre-training Framework for Document Representation Learning

In this paper, we propose a multi-task learning-based framework that uti...
research
05/25/2021

ViBERTgrid: A Jointly Trained Multi-Modal 2D Document Representation for Key Information Extraction from Documents

Recent grid-based document representations like BERTgrid allow the simul...
research
03/17/2022

Confidence Dimension for Deep Learning based on Hoeffding Inequality and Relative Evaluation

Research on the generalization ability of deep neural networks (DNNs) ha...
research
10/25/2022

Useful Confidence Measures: Beyond the Max Score

An important component in deploying machine learning (ML) in safety-crit...
research
07/11/2022

GMN: Generative Multi-modal Network for Practical Document Information Extraction

Document Information Extraction (DIE) has attracted increasing attention...
research
03/08/2023

Simple and Efficient Confidence Score for Grading Whole Slide Images

Grading precancerous lesions on whole slide images is a challenging task...
research
08/29/2022

Confidence Estimation for Object Detection in Document Images

Deep neural networks are becoming increasingly powerful and large and al...

Please sign up or login with your details

Forgot password? Click here to reset