DeepAI AI Chat
Log In Sign Up

Non-Asymptotic Behavior of the Maximum Likelihood Estimate of a Discrete Distribution

by   Sina Molavipour, et al.
KTH Royal Institute of Technology

In this paper, we study the maximum likelihood estimate of the probability mass function (pmf) of n independent and identically distributed (i.i.d.) random variables, in the non-asymptotic regime. We are interested in characterizing the Neyman--Pearson criterion, i.e., the log-likelihood ratio for testing a true hypothesis within a larger hypothesis. Wilks' theorem states that this ratio behaves like a χ^2 random variable in the asymptotic case; however, less is known about the precise behavior of the ratio when the number of samples is finite. In this work, we find an explicit bound for the difference between the cumulative distribution function (cdf) of the log-likelihood ratio and the cdf of a χ^2 random variable. Furthermore, we show that this difference vanishes with a rate of order 1/√(n) in accordance with Wilks' theorem.


page 1

page 2

page 3

page 4


Maximum likelihood estimation for matrix normal models via quiver representations

In this paper, we study the log-likelihood function and Maximum Likeliho...

Asymptotic accuracy of the saddlepoint approximation for maximum likelihood estimation

The saddlepoint approximation gives an approximation to the density of a...

Bounds for the asymptotic distribution of the likelihood ratio

In this paper we give an explicit bound on the distance to chisquare for...

A Note on Taylor's Expansion and Mean Value Theorem With Respect to a Random Variable

We introduce a stochastic version of Taylor's expansion and Mean Value T...

A useful variant of Wilks' theorem for grouped data

This paper provides a generalization of a classical result obtained by W...

An analysis of the maximum likelihood estimates for the Lomax distribution

The Lomax distribution is a popularly used heavy-tailed distribution tha...

A note on the Lomax distribution

The Lomax distribution is a popularly used heavy-tailed distribution tha...