Robust Generalized Method of Moments: A Finite Sample Viewpoint

by   Dhruv Rohatgi, et al.

For many inference problems in statistics and econometrics, the unknown parameter is identified by a set of moment conditions. A generic method of solving moment conditions is the Generalized Method of Moments (GMM). However, classical GMM estimation is potentially very sensitive to outliers. Robustified GMM estimators have been developed in the past, but suffer from several drawbacks: computational intractability, poor dimension-dependence, and no quantitative recovery guarantees in the presence of a constant fraction of outliers. In this work, we develop the first computationally efficient GMM estimator (under intuitive assumptions) that can tolerate a constant ϵ fraction of adversarially corrupted samples, and that has an ℓ_2 recovery guarantee of O(√(ϵ)). To achieve this, we draw upon and extend a recent line of work on algorithmic robust statistics for related but simpler problems such as mean estimation, linear regression and stochastic optimization. As two examples of the generality of our algorithm, we show how our estimation algorithm and assumptions apply to instrumental variables linear and logistic regression. Moreover, we experimentally validate that our estimator outperforms classical IV regression and two-stage Huber regression on synthetic and semi-synthetic datasets with corruption.


page 1

page 2

page 3

page 4


Robust and Sparse Regression in GLM by Stochastic Optimization

The generalized linear model (GLM) plays a key role in regression analys...

Generalized Resilience and Robust Statistics

Robust statistics traditionally focuses on outliers, or perturbations in...

Learning Entangled Single-Sample Distributions via Iterative Trimming

In the setting of entangled single-sample distributions, the goal is to ...

Universal Robust Regression via Maximum Mean Discrepancy

Many datasets are collected automatically, and are thus easily contamina...

A spectral algorithm for robust regression with subgaussian rates

We study a new linear up to quadratic time algorithm for linear regressi...

Private Robust Estimation by Stabilizing Convex Relaxations

We give the first polynomial time and sample (ϵ, δ)-differentially priva...

Dual IV: A Single Stage Instrumental Variable Regression

We present a novel single-stage procedure for instrumental variable (IV)...