Learning high-dimensional probability distributions using tree tensor networks

by   Erwan Grelier, et al.

We consider the problem of the estimation of a high-dimensional probability distribution using model classes of functions in tree-based tensor formats, a particular case of tensor networks associated with a dimension partition tree. The distribution is assumed to admit a density with respect to a product measure, possibly discrete for handling the case of discrete random variables. After discussing the representation of classical model classes in tree-based tensor formats, we present learning algorithms based on empirical risk minimization using a L^2 contrast. These algorithms exploit the multilinear parametrization of the formats to recast the nonlinear minimization problem into a sequence of empirical risk minimization problems with linear models. A suitable parametrization of the tensor in tree-based tensor format allows to obtain a linear model with orthogonal bases, so that each problem admits an explicit expression of the solution and cross-validation risk estimates. These estimations of the risk enable the model selection, for instance when exploiting sparsity in the coefficients of the representation. A strategy for the adaptation of the tensor format (dimension tree and tree-based ranks) is provided, which allows to discover and exploit some specific structures of high-dimensional probability distributions such as independence or conditional independence. We illustrate the performances of the proposed algorithms for the approximation of classical probabilistic models (such as Gaussian distribution, graphical models, Markov chain).


page 1

page 2

page 3

page 4


Learning with tree-based tensor formats

This paper is concerned with the approximation of high-dimensional funct...

Learning with tree tensor networks: complexity estimates and model selection

In this paper, we propose and analyze a model selection method for tree ...

Approximation by tree tensor networks in high dimensions: Sobolev and compositional functions

This paper is concerned with convergence estimates for fully discrete tr...

Active learning of tree tensor networks using optimal least-squares

In this paper, we propose new learning algorithms for approximating high...

Generative Modeling via Tree Tensor Network States

In this paper, we present a density estimation framework based on tree t...

Analysis of tensor methods for stochastic models of gene regulatory networks

The tensor-structured parametric analysis (TPA) has been recently develo...