DeepAI AI Chat
Log In Sign Up

Testing Optimality of Sequential Decision-Making

01/04/2018
by   Meik Dörpinghaus, et al.
Max Planck Society
RWTH Aachen University
Posteo
TU Dresden
0

This paper provides a statistical method to test whether a system that performs a binary sequential hypothesis test is optimal in the sense of minimizing the average decision times while taking decisions with given reliabilities. The proposed method requires samples of the decision times, the decision outcomes, and the true hypotheses, but does not require knowledge on the statistics of the observations or the properties of the decision-making system. The method is based on fluctuation relations for decision time distributions which are proved for sequential probability ratio tests. These relations follow from the martingale property of probability ratios and hold under fairly general conditions. We illustrate these tests with numerical experiments and discuss potential applications.

READ FULL TEXT

page 9

page 26

03/14/2023

A Characterization of Most(More) Powerful Test Statistics with Simple Nonparametric Applications

Data-driven most powerful tests are statistical hypothesis decision-maki...
01/18/2023

Sequential Processing of Observations in Human Decision-Making Systems

In this work, we consider a binary hypothesis testing problem involving ...
12/29/2022

Near-Optimal Non-Parametric Sequential Tests and Confidence Sequences with Possibly Dependent Observations

Sequential testing, always-valid p-values, and confidence sequences prom...
10/13/2015

Fast sequential forensic camera identification

Two sequential camera source identification methods are proposed. Sequen...
03/23/2020

Anticipatory Psychological Models for Quickest Change Detection: Human Sensor Interaction

We consider anticipatory psychological models for human decision makers ...
05/21/2019

Latency Analysis for Sequential Detection in Low-Complexity Binary Radio Systems

We consider the problem of making a quick decision in favor of one of tw...
11/10/2018

Minimax Optimal Sequential Hypothesis Tests for Markov Processes

Under mild Markov assumptions, sufficient conditions for strict minimax ...