Communication-Efficient Accurate Statistical Estimation

06/12/2019
by   Jianqing Fan, et al.
0

When the data are stored in a distributed manner, direct application of traditional statistical inference procedures is often prohibitive due to communication cost and privacy concerns. This paper develops and investigates two Communication-Efficient Accurate Statistical Estimators (CEASE), implemented through iterative algorithms for distributed optimization. In each iteration, node machines carry out computation in parallel and communicate with the central processor, which then broadcasts aggregated information to node machines for new updates. The algorithms adapt to the similarity among loss functions on node machines, and converge rapidly when each node machine has large enough sample size. Moreover, they do not require good initialization and enjoy linear converge guarantees under general conditions. The contraction rate of optimization errors is presented explicitly, with dependence on the local sample size unveiled. In addition, the improved statistical accuracy per iteration is derived. By regarding the proposed method as a multi-step statistical estimator, we show that statistical efficiency can be achieved in finite steps in typical statistical applications. In addition, we give the conditions under which the one-step CEASE estimator is statistically efficient. Extensive numerical experiments on both synthetic and real data validate the theoretical results and demonstrate the superior performance of our algorithms.

READ FULL TEXT
research
10/15/2016

Communication-efficient Distributed Sparse Linear Discriminant Analysis

We propose a communication-efficient distributed estimation method for s...
research
07/15/2023

Byzantine-robust distributed one-step estimation

This paper proposes a Robust One-Step Estimator(ROSE) to solve the Byzan...
research
01/17/2020

Communication-Efficient Distributed Estimator for Generalized Linear Models with a Diverging Number of Covariates

Distributed statistical inference has recently attracted immense attenti...
research
11/04/2015

A Distributed One-Step Estimator

Distributed statistical inference has recently attracted enormous attent...
research
07/13/2022

Communication-efficient Distributed Newton-like Optimization with Gradients and M-estimators

In modern data science, it is common that large-scale data are stored an...
research
06/15/2020

Distributed Newton Can Communicate Less and Resist Byzantine Workers

We develop a distributed second order optimization algorithm that is com...
research
07/30/2022

Efficient estimation and inference for the signed β-model in directed signed networks

This paper proposes a novel signed β-model for directed signed network, ...

Please sign up or login with your details

Forgot password? Click here to reset