A Dynamic Fast Gaussian Transform
share
The Fast Gaussian Transform (FGT) enables subquadratic-time multiplication of an n× n Gaussian kernel matrix 𝖪_i,j= exp ( - x_i - x_j _2^2 ) with an arbitrary vector h ∈ℝ^n, where x_1,…, x_n ∈ℝ^d are a set of fixed source points. This kernel plays a central role in machine learning and random feature maps. Nevertheless, in most modern ML and data analysis applications, datasets are dynamically changing, and recomputing the FGT from scratch in (kernel-based) algorithms, incurs a major computational overhead (≳ n time for a single source update ∈ℝ^d). These applications motivate the development of a dynamic FGT algorithm, which maintains a dynamic set of sources under kernel-density estimation (KDE) queries in sublinear time, while retaining Mat-Vec multiplication accuracy and speed. Our main result is an efficient dynamic FGT algorithm, supporting the following operations in log^O(d)(n/ϵ) time: (1) Adding or deleting a source point, and (2) Estimating the "kernel-density" of a query point with respect to sources with ϵ additive accuracy. The core of the algorithm is a dynamic data structure for maintaining the "interaction rank" between source and target boxes, which we decouple into finite truncation of Taylor series and Hermite expansions.
READ FULL TEXT