DeepAI AI Chat
Log In Sign Up

On the Compressibility of Affinely Singular Random Vectors

by   Mohammad-Amin Charusaie, et al.

There are several ways to measure the compressibility of a random measure; they include the general rate-distortion curve, as well as more specific notions such as Renyi information dimension (RID), and dimensional-rate bias (DRB). The RID parameter indicates the concentration of the measure around lower-dimensional subsets of the space while the DRB parameter specifies the compressibility of the distribution over these lower-dimensional subsets. While the evaluation of such compressibility parameters is well-studied for continuous and discrete measures (e.g., the DRB is closely related to the entropy and differential entropy in discrete and continuous cases, respectively), the case of discrete-continuous measures is quite subtle. In this paper, we focus on a class of multi-dimensional random measures that have singularities on affine lower-dimensional subsets. These cases are of interest when working with linear transformation of component-wise independent discrete-continuous random variables. Here, we evaluate the RID and DRB for such probability measures. We further provide an upper-bound for the RID of multi-dimensional random measures that are obtained by Lipschitz functions of component-wise independent discrete-continuous random variables (x). The upper-bound is shown to be achievable when the Lipschitz function is A x, where A satisfies (A) = (A)+1 (e.g., Vandermonde matrices). The above results in the case of discrete-domain moving-average processes with non-Gaussian excitation noise allow us to evaluate the block-average RID and to find a relationship between this parameter and other existing compressibility measures.


page 1

page 5


Cramér-Rao-type Bound and Stam's Inequality for Discrete Random Variables

The variance and the entropy power of a continuous random variable are b...

Modeling Count Data via Copulas

Copula models have been widely used to model the dependence between cont...

On Optimal Uniform Concentration Inequalities for Discrete Entropies in the High-dimensional Setting

We prove an exponential decay concentration inequality to bound the tail...

Large deviation principles induced by the Stiefel manifold, and random multi-dimensional projections

Given an n-dimensional random vector X^(n) , for k < n, consider its k-d...

Rate-Distortion Theory for General Sets and Measures

This paper is concerned with a rate-distortion theory for sequences of i...

Typicality for stratified measures

Stratified measures on Euclidean space are defined here as convex combin...