Random Forests for dependent data

07/30/2020
by   Arkajyoti Saha, et al.
0

Random forest (RF) is one of the most popular methods for estimating regression functions. The local nature of the RF algorithm, based on intra-node means and variances, is ideal when errors are i.i.d. For dependent error processes like time series and spatial settings where data in all the nodes will be correlated, operating locally ignores this dependence. Also, RF will involve resampling of correlated data, violating the principles of bootstrap. Theoretically, consistency of RF has been established for i.i.d. errors, but little is known about the case of dependent errors. We propose RF-GLS, a novel extension of RF for dependent error processes in the same way Generalized Least Squares (GLS) fundamentally extends Ordinary Least Squares (OLS) for linear models under dependence. The key to this extension is the equivalent representation of the local decision-making in a regression tree as a global OLS optimization which is then replaced with a GLS loss to create a GLS-style regression tree. This also synergistically addresses the resampling issue, as the use of GLS loss amounts to resampling uncorrelated contrasts (pre-whitened data) instead of the correlated data. For spatial settings, RF-GLS can be used in conjunction with Gaussian Process correlated errors to generate kriging predictions at new locations. RF becomes a special case of RF-GLS with an identity working covariance matrix. We establish consistency of RF-GLS under beta- (absolutely regular) mixing error processes and show that this general result subsumes important cases like autoregressive time series and spatial Matern Gaussian Processes. As a byproduct, we also establish consistency of RF for beta-mixing processes, which to our knowledge, is the first such result for RF under dependence. We empirically demonstrate the improvement achieved by RF-GLS over RF for both estimation and prediction under dependence.

READ FULL TEXT

page 26

page 27

research
02/27/2023

Random forests for binary geospatial data

Binary geospatial data is commonly analyzed with generalized linear mixe...
research
11/23/2022

Consistency of The Oblique Decision Tree and Its Random Forest

The classification and regression tree (CART) and Random Forest (RF) are...
research
10/26/2022

Ensemble Projection Pursuit for General Nonparametric Regression

The projection pursuit regression (PPR) has played an important role in ...
research
04/18/2023

Neural networks for geospatial data

Analysis of geospatial data has traditionally been model-based, with a m...
research
03/08/2023

A path in regression Random Forest looking for spatial dependence: a taxonomy and a systematic review

Random Forest (RF) is a well-known data-driven algorithm applied in seve...
research
10/31/2017

Partial Least Squares Random Forest Ensemble Regression as a Soft Sensor

Six simple, dynamic soft sensor methodologies with two update conditions...
research
02/13/2023

Precise Asymptotic Analysis of Deep Random Feature Models

We provide exact asymptotic expressions for the performance of regressio...

Please sign up or login with your details

Forgot password? Click here to reset