Stochastic Approximation for Online Tensorial Independent Component Analysis

12/28/2020 ∙ by Chris Junchi Li, et al. ∙ 0

Independent component analysis (ICA) has been a popular dimension reduction tool in statistical machine learning and signal processing. In this paper, we present a convergence analysis for an online tensorial ICA algorithm, by viewing the problem as a nonconvex stochastic approximation problem. For estimating one component, we provide a dynamics-based analysis to prove that our online tensorial ICA algorithm with a specific choice of stepsize achieves a sharp finite-sample error bound. In particular, under a mild assumption on the data-generating distribution and a scaling condition such that d^4 / T is sufficiently small up to a polylogarithmic factor of data dimension d and sample size T, a sharp finite-sample error bound of Õ(√(d / T)) can be obtained. As a by-product, we also design an online tensorial ICA algorithm that estimates multiple independent components in parallel, achieving desirable finite-sample error bound for each independent component estimator.



There are no comments yet.


page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.