DeepAI
Log In Sign Up

Hierarchical correlation reconstruction with missing data

04/17/2018
by   Jarek Duda, et al.
0

Machine learning often needs to estimate density from a multidimensional data sample, where we would also like to model correlations between coordinates. Additionally, we often have missing data case: that data points have only partial information - can miss information about some coordinates. This paper adapts rapid parametric density estimation technique for this purpose: modelling density as a linear combination, for which L^2 optimization says that estimated coefficient for a given function is just average over the sample of this function. Hierarchical correlation reconstruction first models probability density for each separate coordinate using all its appearances in data sample, then adds corrections from independently modelled pairwise correlations using all samples having both coordinates, and so on independently adding correlations for growing numbers of variables using decreasing evidence in our data sample. A basic application of such modelled multidimensional density can be imputation of missing coordinates: by inserting known coordinates to the density, and taking expected values for the missing coordinates, and maybe also variance to estimate their uncertainty.

READ FULL TEXT

page 1

page 2

page 3

04/17/2018

Hierarchical correlation reconstruction with missing data, for example for biology-inspired neuron

Machine learning often needs to estimate density from a multidimensional...
12/19/2018

Credibility evaluation of income data with hierarchical correlation reconstruction

In situations like tax declarations or analyzes of household budgets we ...
06/04/2020

Handling missing data in model-based clustering

Gaussian Mixture models (GMMs) are a powerful tool for clustering, class...
05/28/2022

Angle-Uniform Parallel Coordinates

We present angle-uniform parallel coordinates, a data-independent techni...
04/25/2022

The Galactic 3D large-scale dust distribution via Gaussian process regression on spherical coordinates

Knowing the Galactic 3D dust distribution is relevant for understanding ...
10/11/2018

A random model for multidimensional fitting method

Multidimensional fitting (MDF) method is a multivariate data analysis me...
08/09/2022

Dealing with missing data under stratified sampling designs where strata are study domains

A quick count seeks to estimate the voting trends of an election and com...