
Histogram Transform Ensembles for Density Estimation
We investigate an algorithm named histogram transform ensembles (HTE) de...
read it

Gradient Boosted Binary Histogram Ensemble for Largescale Regression
In this paper, we propose a gradient boosting algorithm for largescale ...
read it

GBHT: Gradient Boosting Histogram Transform for Density Estimation
In this paper, we propose a density estimation algorithm called Gradient...
read it

Optimal learning rates for Kernel Conjugate Gradient regression
We prove rates of convergence in the statistical sense for kernelbased ...
read it

Convergence rates of efficient global optimization algorithms
Efficient global optimization is the problem of minimizing an unknown fu...
read it

Stochastic LBFGS: Improved Convergence Rates and Practical Acceleration Strategies
We revisit the stochastic limitedmemory BFGS (LBFGS) algorithm. By pro...
read it

Tectonic environments of South American porphyry copper magmatism through time revealed by spatiotemporal data mining
Porphyry ore deposits are known to be associated with arc magmatism on t...
read it
Histogram Transform Ensembles for Largescale Regression
We propose a novel algorithm for largescale regression problems named histogram transform ensembles (HTE), composed of random rotations, stretchings, and translations. First of all, we investigate the theoretical properties of HTE when the regression function lies in the Hölder space C^k,α, k ∈N_0, α∈ (0,1]. In the case that k=0, 1, we adopt the constant regressors and develop the naïve histogram transforms (NHT). Within the space C^0,α, although almost optimal convergence rates can be derived for both single and ensemble NHT, we fail to show the benefits of ensembles over single estimators theoretically. In contrast, in the subspace C^1,α, we prove that if d ≥ 2(1+α)/α, the lower bound of the convergence rates for single NHT turns out to be worse than the upper bound of the convergence rates for ensemble NHT. In the other case when k ≥ 2, the NHT may no longer be appropriate in predicting smoother regression functions. Instead, we apply kernel histogram transforms (KHT) equipped with smoother regressors such as support vector machines (SVMs), and it turns out that both single and ensemble KHT enjoy almost optimal convergence rates. Then we validate the above theoretical results by numerical experiments. On the one hand, simulations are conducted to elucidate that ensemble NHT outperform single NHT. On the other hand, the effects of bin sizes on accuracy of both NHT and KHT also accord with theoretical analysis. Last but not least, in the realdata experiments, comparisons between the ensemble KHT, equipped with adaptive histogram transforms, and other stateoftheart largescale regression estimators verify the effectiveness and accuracy of our algorithm.
READ FULL TEXT
Comments
There are no comments yet.