DeepAI
Log In Sign Up

Estimating a regression function in exponential families by model selection

03/13/2022
by   Juntong Chen, et al.
0

Let X_1=(W_1,Y_1),…,X_n=(W_n,Y_n) be n pairs of independent random variables. We assume that, for each i∈{1,…,n}, the conditional distribution of Y_i given W_i belongs to a one-parameter exponential family with parameter γ^⋆(W_i)∈ℝ, or at least, is close enough to a distribution of this form. The objective of the present paper is to estimate these conditional distributions on the basis of the observation X=(X_1,…,X_n) and to do so, we propose a model selection procedure together with a non-asymptotic risk bound for the resulted estimator with respect to a Hellinger-type distance. When γ^⋆ does exist, the procedure allows to obtain an estimator γ of γ^⋆ adapted to a wide range of the anisotropic Besov spaces. When γ^⋆ has a general additive or multiple index structure, we construct suitable models and show the resulted estimators by our procedure based on such models can circumvent the curse of dimensionality. Moreover, we consider model selection problems for ReLU neural networks and provide an example where estimation based on neural networks enjoys a much faster converge rate than the classical models. Finally, we apply this procedure to solve variable selection problem in exponential families. The proofs in the paper rely on bounding the VC dimensions of several collections of functions, which can be of independent interest.

READ FULL TEXT

page 1

page 2

page 3

page 4

12/25/2022

Estimator selection for regression functions in exponential families with application to changepoint detection

We observe n independent pairs of random variables (W_i, Y_i) for which ...
11/03/2020

Robust estimation of a regression function in exponential families

We observe n pairs X_1=(W_1,Y_1),…,X_n=(W_n,Y_n) of independent random v...
02/26/2021

General dependence structures for some models based on exponential families with quadratic variance functions

We describe a procedure to introduce general dependence structures on a ...
11/11/2018

Adaptive model selection method for a conditionally Gaussian semimartingale regression in continuous time

This paper considers the problem of robust adaptive efficient estimating...
10/29/2020

Staged trees are curved exponential families

Staged tree models are a discrete generalization of Bayesian networks. W...
11/15/2017

Kernel Conditional Exponential Family

A nonparametric family of conditional distributions is introduced, which...