
Gradientbased Optimization for Regression in the Functional TensorTrain Format
We consider the task of lowmultilinearrank functional regression, i.e., learning a lowrank parametric representation of functions from scattered realvalued data. Our first contribution is the development and analysis of an efficient gradient computation that enables gradientbased optimization procedures, including stochastic gradient descent and quasiNewton methods, for learning the parameters of a functional tensortrain (FT). The functional tensortrain uses the tensortrain (TT) representation of lowrank arrays as an ansatz for a class of lowmultilinearrank functions. The FT is represented by a set of matrixvalued functions that contain a set of univariate functions, and the regression task is to learn the parameters of these univariate functions. Our second contribution demonstrates that using nonlinearly parameterized univariate functions, e.g., symmetric kernels with moving centers, within each core can outperform the standard approach of using a linear expansion of basis functions. Our final contributions are new rank adaptation and groupsparsity regularization procedures to minimize overfitting. We use several benchmark problems to demonstrate at least an order of magnitude lower accuracy with gradientbased optimization methods than standard alternating least squares procedures in the lowsample number regime. We also demonstrate an order of magnitude reduction in accuracy on a test problem resulting from using nonlinear parameterizations over linear parameterizations. Finally we compare regression performance with 22 other nonparametric and parametric regression methods on 10 realworld data sets. We achieve topfive accuracy for seven of the data sets and best accuracy for two of the data sets. These rankings are the best amongst parametric models and competetive with the best nonparametric methods.
01/03/2018 ∙ by Alex A. Gorodetsky, et al. ∙ 0 ∙ shareread it

A Generalized Framework for Approximate Control Variates
We describe and analyze a Monte Carlo (MC) sampling framework for accelerating the estimation of statistics of computationally expensive simulation models using an ensemble of models with lower cost. Our approach uses control variates, with unknown means that must be estimated from data, to reduce the variance in statistical estimators relative to MC. Our framework unifies existing multilevel, multiindex, and multifidelity MC algorithms and leads to new and more efficient sampling schemes. Our results indicate that the variance reduction achieved by existing algorithms that explicitly or implicitly estimate control means, such as multilevel MC and multifidelity MC, is limited to that of a single linear control variate with known mean regardless of the number of control variates. We show how to circumvent this limitation and derive a new family of schemes that make full use of all available information sources. In particular, we demonstrate that a significant gap can exist, of orders of magnitude in some cases, between the variance reduction achievable by current We also present initial sample allocation approaches for exploiting this gap, which yield the greatest benefit when augmenting the highfidelity model evaluations is impractical because, for instance, they arise from a legacy database. Several analytic examples and two PDE problems (viscous Burger's and steady state diffusion) are considered to demonstrate the methodology.
11/12/2018 ∙ by Alex A. Gorodetsky, et al. ∙ 0 ∙ shareread it
Alex A. Gorodetsky
is this you? claim profile