The Power of Log-Sum-Exp: Sequential Density Ratio Matrix Estimation for Speed-Accuracy Optimization

05/28/2021
by   Taiki Miyagawa, et al.
0

We propose a model for multiclass classification of time series to make a prediction as early and as accurate as possible. The matrix sequential probability ratio test (MSPRT) is known to be asymptotically optimal for this setting, but contains a critical assumption that hinders broad real-world applications; the MSPRT requires the underlying probability density. To address this problem, we propose to solve density ratio matrix estimation (DRME), a novel type of density ratio estimation that consists of estimating matrices of multiple density ratios with constraints and thus is more challenging than the conventional density ratio estimation. We propose a log-sum-exp-type loss function (LSEL) for solving DRME and prove the following: (i) the LSEL provides the true density ratio matrix as the sample size of the training set increases (consistency); (ii) it assigns larger gradients to harder classes (hard class weighting effect); and (iii) it provides discriminative scores even on class-imbalanced datasets (guess-aversion). Our overall architecture for early classification, MSPRT-TANDEM, statistically significantly outperforms baseline models on four datasets including action recognition, especially in the early stage of sequential observations. Our code and datasets are publicly available at: https://github.com/TaikiMiyagawa/MSPRT-TANDEM.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/20/2023

Toward Asymptotic Optimality: Sequential Unsupervised Regression of Density Ratio for Early Classification

Theoretically-inspired sequential density ratio estimation (SDRE) algori...
research
06/10/2020

Deep Neural Networks for the Sequential Probability Ratio Test on Non-i.i.d. Data Series

Classifying sequential data as early as and as accurately as possible is...
research
09/14/2023

Some notes concerning a generalized KMM-type optimization method for density ratio estimation

In the present paper we introduce new optimization algorithms for the ta...
research
02/15/2013

Density Ratio Hidden Markov Models

Hidden Markov models and their variants are the predominant sequential c...
research
02/21/2023

Density Ratio Estimation and Neyman Pearson Classification with Missing Data

Density Ratio Estimation (DRE) is an important machine learning techniqu...
research
10/10/2021

A computational approach to the Kiefer-Weiss problem for sampling from a Bernoulli population

We present a computational approach to solution of the Kiefer-Weiss prob...
research
03/07/2023

Generative Modeling with Flow-Guided Density Ratio Learning

We present Flow-Guided Density Ratio Learning (FDRL), a simple and scala...

Please sign up or login with your details

Forgot password? Click here to reset