A unified perspective on modified Poisson likelihoods for limited Monte Carlo data
share
Counting experiments often rely on Monte Carlo simulations for predictions of Poisson expectations. The accompanying uncertainty from the finite Monte Carlo sample size can be incorporated into parameter estimation by modifying the Poisson likelihood. We first review previous Frequentist methods of this type by Barlow et al, Bohm et al, and Chirkin, as well as recently proposed probabilistic methods by the author and Argüelles et al. We show that all these approaches can be understood in a unified way: they all approximate the underlying probability distribution of the sum of weights in a given bin, the compound Poisson distribution (CPD). The Probabilistic methods marginalize the Poisson mean with a distribution that approximates the CPD, while the Frequentist counterparts optimize the same integrand treating the mean as a nuisance parameter. With this viewpoint we can motivate three new probabilistic likelihoods based on generalized gamma-Poisson mixture distributions which we derive in analytic form. Afterwards, we test old and new formulas in different parameter estimation settings consisting of a "background" and "signal" dataset. The new formulas outperform existing approaches in terms of likelihood-ratio bias and coverage in all tested scenarios. We further find a surprising outcome: usage of the exact CPD is actually bad for parameter estimation. A continuous approximation performs much better and in principle allows to perform bias-free inference at any level of simulated livetime if the first two moments of the CPD of each dataset are known exactly. Finally, we also discuss the situation where new Monte Carlo simulation is produced for a given likelihood evaluation which leads to fluctuations in the likelihood function. Two of the new formulas allow to include this Poisson uncertainty directly into the likelihood which substantially decreases these fluctuations.
READ FULL TEXT