Selective Probabilistic Classifier Based on Hypothesis Testing

05/09/2021
by   Saeed Bakhshi Germi, et al.
0

In this paper, we propose a simple yet effective method to deal with the violation of the Closed-World Assumption for a classifier. Previous works tend to apply a threshold either on the classification scores or the loss function to reject the inputs that violate the assumption. However, these methods cannot achieve the low False Positive Ratio (FPR) required in safety applications. The proposed method is a rejection option based on hypothesis testing with probabilistic networks. With probabilistic networks, it is possible to estimate the distribution of outcomes instead of a single output. By utilizing Z-test over the mean and standard deviation for each class, the proposed method can estimate the statistical significance of the network certainty and reject uncertain outputs. The proposed method was experimented on with different configurations of the COCO and CIFAR datasets. The performance of the proposed method is compared with the Softmax Response, which is a known top-performing method. It is shown that the proposed method can achieve a broader range of operation and cover a lower FPR than the alternative.

READ FULL TEXT
research
10/23/2022

Achievable Error Exponents for Almost Fixed-Length Hypothesis Testing and Classification

We revisit multiple hypothesis testing and propose a two-phase test, whe...
research
03/24/2022

Methods for Large-scale Single Mediator Hypothesis Testing: Possible Choices and Comparisons

Mediation hypothesis testing for a large number of mediators is challeng...
research
09/07/2018

Multi-level hypothesis testing for populations of heterogeneous networks

In this work, we consider hypothesis testing and anomaly detection on da...
research
12/30/2019

B-Value and Empirical Equivalence Bound: A New Procedure of Hypothesis Testing

In this study, we propose a two-stage procedure for hypothesis testing, ...
research
01/08/2022

A Baseline Statistical Method For Robust User-Assisted Multiple Segmentation

Recently, several image segmentation methods that welcome and leverage d...
research
01/26/2023

Permutation-based Hypothesis Testing for Neural Networks

Neural networks are powerful predictive models, but they provide little ...
research
07/02/2018

Power Imbalance Detection in Smart Grid via Grid Frequency Deviations: A Hidden Markov Model based Approach

We detect the deviation of the grid frequency from the nominal value (i....

Please sign up or login with your details

Forgot password? Click here to reset