Lipschitz Parametrization of Probabilistic Graphical Models

02/14/2012
by   Jean Honorio, et al.
0

We show that the log-likelihood of several probabilistic graphical models is Lipschitz continuous with respect to the lp-norm of the parameters. We discuss several implications of Lipschitz parametrization. We present an upper bound of the Kullback-Leibler divergence that allows understanding methods that penalize the lp-norm of differences of parameters as the minimization of that upper bound. The expected log-likelihood is lower bounded by the negative lp-norm, which allows understanding the generalization ability of probabilistic models. The exponential of the negative lp-norm is involved in the lower bound of the Bayes error rate, which shows that it is reasonable to use parameters as features in algorithms that rely on metric spaces (e.g. classification, dimensionality reduction, clustering). Our results do not rely on specific algorithms for learning the structure or parameters. We show preliminary results for activity recognition and temporal segmentation.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

07/22/2020

Approximate Covering with Lower and Upper Bounds via LP Rounding

In this paper, we study the lower- and upper-bounded covering (LUC) prob...
04/12/2019

Information Theoretic Lower Bounds on Negative Log Likelihood

In this article we use rate-distortion theory, a branch of information t...
08/18/2016

Parameter Learning for Log-supermodular Distributions

We consider log-supermodular models on binary variables, which are proba...
03/15/2012

Inference-less Density Estimation using Copula Bayesian Networks

We consider learning continuous probabilistic graphical models in the fa...
03/03/2011

The Local Rademacher Complexity of Lp-Norm Multiple Kernel Learning

We derive an upper bound on the local Rademacher complexity of ℓ_p-norm ...
02/09/2020

Projected Stein Variational Gradient Descent

The curse of dimensionality is a critical challenge in Bayesian inferenc...
01/23/2018

Generalized two-dimensional linear discriminant analysis with regularization

Recent advances show that two-dimensional linear discriminant analysis (...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.