Bounded-memory adjusted scores estimation in generalized linear models with large data sets

by   Patrick Zietkiewicz, et al.
University of Warwick

The widespread use of maximum Jeffreys'-prior penalized likelihood in binomial-response generalized linear models, and in logistic regression, in particular, are supported by the results of Kosmidis and Firth (2021, Biometrika), who show that the resulting estimates are also always finite-valued, even in cases where the maximum likelihood estimates are not, which is a practical issue regardless of the size of the data set. In logistic regression, the implied adjusted score equations are formally bias-reducing in asymptotic frameworks with a fixed number of parameters and appear to deliver a substantial reduction in the persistent bias of the maximum likelihood estimator in high-dimensional settings where the number of parameters grows asymptotically linearly and slower than the number of observations. In this work, we develop and present two new variants of iteratively reweighted least squares for estimating generalized linear models with adjusted score equations for mean bias reduction and maximization of the likelihood penalized by a positive power of the Jeffreys-prior penalty, which eliminate the requirement of storing O(n) quantities in memory, and can operate with data sets that exceed computer memory or even hard drive capacity. We achieve that through incremental QR decompositions, which enable IWLS iterations to have access only to data chunks of predetermined size. We assess the procedures through a real-data application with millions of observations, and in high-dimensional logistic regression, where a large-scale simulation experiment produces concrete evidence for the existence of a simple adjustment to the maximum Jeffreys'-penalized likelihood estimates that delivers high accuracy in terms of signal recovery even in cases where estimates from ML and other recently-proposed corrective methods do not exist.


Jeffreys' prior, finiteness and shrinkage in binomial-response generalized linear models

This paper studies the finiteness properties of a reduced-bias estimator...

Mean and median bias reduction: A concise review and application to adjacent-categories logit models

The estimation of categorical response models using bias-reducing adjust...

Conjugate priors and bias reduction for logistic regression models

Logistic regression models for binomial responses are routinely used in ...

Mean and median bias reduction in generalized linear models

This paper presents an integrated framework for estimation and inference...

Accurate inference in negative binomial regression

Negative binomial regression is commonly employed to analyze overdispers...

The empirical likelihood prior applied to bias reduction of general estimating equations

The practice of employing empirical likelihood (EL) components in place ...

Approximate co-sufficient sampling with regularization

In this work, we consider the problem of goodness-of-fit (GoF) testing f...

Please sign up or login with your details

Forgot password? Click here to reset