Regularized Modal Regression on Markov-dependent Observations: A Theoretical Assessment

by   Tielang Gong, et al.

Modal regression, a widely used regression protocol, has been extensively investigated in statistical and machine learning communities due to its robustness to outliers and heavy-tailed noises. Understanding modal regression's theoretical behavior can be fundamental in learning theory. Despite significant progress in characterizing its statistical property, the majority of the results are based on the assumption that samples are independent and identical distributed (i.i.d.), which is too restrictive for real-world applications. This paper concerns the statistical property of regularized modal regression (RMR) within an important dependence structure - Markov dependent. Specifically, we establish the upper bound for RMR estimator under moderate conditions and give an explicit learning rate. Our results show that the Markov dependence impacts on the generalization error in the way that sample size would be discounted by a multiplicative factor depending on the spectral gap of underlying Markov chain. This result shed a new light on characterizing the theoretical underpinning for robust regression.



There are no comments yet.


page 1

page 2

page 3

page 4


Adaptive Huber Regression on Markov-dependent Data

High-dimensional linear regression has been intensively studied in the c...

A Statistical Learning Assessment of Huber Regression

As one of the triumphs and milestones of robust statistics, Huber regres...

Hoeffding's lemma for Markov Chains and its applications to statistical learning

We establish the counterpart of Hoeffding's lemma for Markov dependent r...

A Statistical Learning Approach to Modal Regression

This paper studies the nonparametric modal regression problem systematic...

SIMEX Estimation in Parametric Modal Regression with Measurement Error

For a class of parametric modal regression models with measurement error...

Error Bounds of the Invariant Statistics in Machine Learning of Ergodic Itô Diffusions

This paper studies the theoretical underpinnings of machine learning of ...

Characterizing the Exact Behaviors of Temporal Difference Learning Algorithms Using Markov Jump Linear System Theory

In this paper, we provide a unified analysis of temporal difference lear...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.