Deep PDF: Probabilistic Surface Optimization and Density Estimation

by   Dmitry Kopitkov, et al.

A probability density function (pdf) encodes the entire stochastic knowledge about data distribution, where data may represent stochastic observations in robotics, transition state pairs in reinforcement learning or any other empirically acquired modality. Inferring data pdf is of prime importance, allowing to analyze various model hypotheses and perform smart decision making. However, most density estimation techniques are limited in their representation expressiveness to specific kernel type or predetermined distribution family, and have other restrictions. For example, kernel density estimation (KDE) methods require meticulous parameter search and are extremely slow at querying new points. In this paper we present a novel non-parametric density estimation approach, DeepPDF, that uses a neural network to approximate a target pdf given samples from thereof. Such a representation provides high inference accuracy for a wide range of target pdfs using a relatively simple network structure, making our method highly statistically robust. This is done via a new stochastic optimization algorithm, Probabilistic Surface Optimization (PSO), that turns to advantage the stochastic nature of sample points in order to force network output to be identical to the output of a target pdf. Once trained, query point evaluation can be efficiently done in DeepPDF by a simple network forward pass, with linear complexity in the number of query points. Moreover, the PSO algorithm is capable of inferring the frequency of data samples and may also be used in other statistical tasks such as conditional estimation and distribution transformation. We compare the derived approach with KDE methods showing its superior performance and accuracy.


page 1

page 2

page 3

page 4


Deconvolutional Density Network: Free-Form Conditional Density Estimation

Conditional density estimation is the task of estimating the probability...

General Probabilistic Surface Optimization and Log Density Estimation

In this paper we contribute a novel algorithm family, which generalizes ...

Kernel density estimation-based sampling for neural network classification

Imbalanced data occurs in a wide range of scenarios. The skewed distribu...

Universal Approximation of Edge Density in Large Graphs

In this paper, we present a novel way to summarize the structure of larg...

Robust Stochastic Configuration Networks with Kernel Density Estimation

Neural networks have been widely used as predictive models to fit data d...

Design optimization of stochastic complex systems via iterative density estimation

Reliability-based design optimization (RBDO) provides a rational and sou...

A Two-Stage Stochastic Programming Model for Car-Sharing Problem using Kernel Density Estimation

Car-sharing problem is a popular research field in sharing economy. In t...

Please sign up or login with your details

Forgot password? Click here to reset