Optimal statistical inference in the presence of systematic uncertainties using neural network optimization based on binned Poisson likelihoods with nuisance parameters

03/16/2020
by   Stefan Wunsch, et al.
0

Data analysis in science, e.g., high-energy particle physics, is often subject to an intractable likelihood if the observables and observations span a high-dimensional input space. Typically the problem is solved by reducing the dimensionality using feature engineering and histograms, whereby the latter technique allows to build the likelihood using Poisson statistics. However, in the presence of systematic uncertainties represented by nuisance parameters in the likelihood, the optimal dimensionality reduction with a minimal loss of information about the parameters of interest is not known. This work presents a novel strategy to construct the dimensionality reduction with neural networks for feature engineering and a differential formulation of histograms so that the full workflow can be optimized with the result of the statistical inference, e.g., the variance of a parameter of interest, as objective. We discuss how this approach results in an estimate of the parameters of interest that is close to optimal and the applicability of the technique is demonstrated with a simple example based on pseudo-experiments and a more complex example from high-energy particle physics.

READ FULL TEXT
research
03/10/2021

A Local Similarity-Preserving Framework for Nonlinear Dimensionality Reduction with Neural Networks

Real-world data usually have high dimensionality and it is important to ...
research
07/18/2019

Neural Networks for Full Phase-space Reweighting and Parameter Tuning

Precise scientific analysis in collider-based particle physics is possib...
research
06/06/2015

Approximating Likelihood Ratios with Calibrated Discriminative Classifiers

In many fields of science, generalized likelihood ratio tests are establ...
research
03/10/2022

neos: End-to-End-Optimised Summary Statistics for High Energy Physics

The advent of deep learning has yielded powerful tools to automatically ...
research
04/30/2018

A Guide to Constraining Effective Field Theories with Machine Learning

We develop, discuss, and compare several inference techniques to constra...
research
06/21/2023

Hierarchical Neural Simulation-Based Inference Over Event Ensembles

When analyzing real-world data it is common to work with event ensembles...
research
03/23/2017

Why do similarity matching objectives lead to Hebbian/anti-Hebbian networks?

Modeling self-organization of neural networks for unsupervised learning ...

Please sign up or login with your details

Forgot password? Click here to reset