Adaptive Kernel Learning in Heterogeneous Networks

08/01/2019
by   Hrusikesha Pradhan, et al.
3

We consider the framework of learning over decentralized networks, where nodes observe unique, possibly correlated, observation streams. We focus on the case where agents learn a regression function that belongs to a reproducing kernel Hilbert space (RKHS). In this setting, a decentralized network aims to learn nonlinear statistical models that are optimal in terms of a global stochastic convex functional that aggregates data across the network, with only access to a local data stream. We incentivize coordination while respecting network heterogeneity through the introduction of nonlinear proximity constraints. To solve it, we propose applying a functional variant of stochastic primal-dual (Arrow-Hurwicz) method which yields a decentralized algorithm. To handle the fact that the RKHS parameterization has complexity proportionate with the iteration index, we project the primal iterates onto Hilbert subspaces that are greedily constructed from the observation sequence of each node. The resulting proximal stochastic variant of Arrow-Hurwicz, dubbed Heterogeneous Adaptive Learning with Kernels (HALK), is shown to converge in expectation, both in terms of primal sub-optimality and constraint violation to a neighborhood that depends on a given constant step-size selection. Simulations on a correlated spatio-temporal random field estimation problem validate our theoretical results, which are born out in practice for networked oceanic sensing buoys estimating temperature and salinity from depth measurements.

READ FULL TEXT

page 1

page 10

research
10/11/2017

Decentralized Online Learning with Kernels

We consider multi-agent stochastic optimization problems over reproducin...
research
05/23/2022

Theoretical Analysis of Primal-Dual Algorithm for Non-Convex Stochastic Decentralized Optimization

In recent years, decentralized learning has emerged as a powerful tool n...
research
12/08/2020

A Primal-Dual Framework for Decentralized Stochastic Optimization

We consider the decentralized convex optimization problem, where multipl...
research
04/13/2016

Algorithms for stochastic optimization with expectation constraints

This paper considers the problem of minimizing an expectation function o...
research
01/28/2020

COKE: Communication-Censored Kernel Learning for Decentralized Non-parametric Learning

This paper studies the decentralized optimization and learning problem w...
research
12/15/2022

Let's consider more general nonlinear approaches to study teleconnections of climate variables

The recent work by (Rieger et al 2021) is concerned with the problem of ...

Please sign up or login with your details

Forgot password? Click here to reset