Lipschitz Parametrization of Probabilistic Graphical Models

02/14/2012
by   Jean Honorio, et al.
0

We show that the log-likelihood of several probabilistic graphical models is Lipschitz continuous with respect to the lp-norm of the parameters. We discuss several implications of Lipschitz parametrization. We present an upper bound of the Kullback-Leibler divergence that allows understanding methods that penalize the lp-norm of differences of parameters as the minimization of that upper bound. The expected log-likelihood is lower bounded by the negative lp-norm, which allows understanding the generalization ability of probabilistic models. The exponential of the negative lp-norm is involved in the lower bound of the Bayes error rate, which shows that it is reasonable to use parameters as features in algorithms that rely on metric spaces (e.g. classification, dimensionality reduction, clustering). Our results do not rely on specific algorithms for learning the structure or parameters. We show preliminary results for activity recognition and temporal segmentation.

READ FULL TEXT
research
06/19/2022

Bounding Evidence and Estimating Log-Likelihood in VAE

Many crucial problems in deep learning and statistics are caused by a va...
research
07/22/2020

Approximate Covering with Lower and Upper Bounds via LP Rounding

In this paper, we study the lower- and upper-bounded covering (LUC) prob...
research
04/12/2019

Information Theoretic Lower Bounds on Negative Log Likelihood

In this article we use rate-distortion theory, a branch of information t...
research
03/15/2012

Inference-less Density Estimation using Copula Bayesian Networks

We consider learning continuous probabilistic graphical models in the fa...
research
08/18/2016

Parameter Learning for Log-supermodular Distributions

We consider log-supermodular models on binary variables, which are proba...
research
03/03/2011

The Local Rademacher Complexity of Lp-Norm Multiple Kernel Learning

We derive an upper bound on the local Rademacher complexity of ℓ_p-norm ...
research
02/06/2013

Score and Information for Recursive Exponential Models with Incomplete Data

Recursive graphical models usually underlie the statistical modelling co...

Please sign up or login with your details

Forgot password? Click here to reset